The forgotten 80-year-old machine that shaped the internet – and could help us survive AI

Many years ago, long before the internet or artificial intelligence, an American engineer called Vannevar Bush was trying to solve a problem. He could see how difficult it had become for professionals to research anything, and saw the potential for a better way.

This was in the 1940s, when anyone looking for articles, books or other scientific records had to go to a library and search through an index. This meant drawers upon drawers filled with index cards, typically sorted by author, title or subject.

When you had found what you were looking for, creating copies or excerpts was a tedious, manual task. You would have to be very organised in keeping your own records. And woe betide anyone who was working across more than one discipline. Since every book could physically only be in one place, they all had to be filed solely under a primary subject. So an article on cave art couldn’t be in both art and archaeology, and researchers would often waste extra time trying to find the right location.

Get your news from actual experts, straight to your inbox. Sign up to our daily newsletter to receive all The Conversation UK’s latest coverage of news and research, from politics and business to the arts and sciences.

This had always been a challenge, but an explosion in research publications in that era had made it far worse than before. As Bush wrote in an influential essay, As We May Think, in The Atlantic in July 1945:

There is a growing mountain of research. But there is increased evidence that we are being bogged down today as specialisation extends. The investigator is staggered by the findings and conclusions of thousands of other workers – conclusions which he cannot find time to grasp, much less to remember, as they appear.

Bush was dean of the school of engineering at MIT (the Massachusetts Institute of Technology) and president of the Carnegie Institute. During the second world war, he had been the director of the Office of Scientific Research and Development, coordinating the activities of some 6,000 scientists working relentlessly to give their country a technological advantage. He could see that science was being drastically slowed down by the research process, and proposed a solution that he called the “memex”.

The memex was to be a personal device built into a desk that required little physical space. It would rely heavily on microfilm for data storage, a new technology at the time. The memex would use this to store large numbers of documents in a greatly compressed format that could be projected onto translucent screens.

Most importantly, Bush’s memex was to include a form of associative indexing for tying two items together. The user would be able to use a keyboard to click on a code number alongside a document to jump to an associated document or view them simultaneously – without needing to sift through an index.

Bush acknowledged in his essay that this kind of keyboard click-through wasn’t yet technologically feasible. Yet he believed it would be soon, pointing to existing systems for handling data such as punched cards as potential forerunners.

He envisaged that a user would create the connections between items as they developed their personal research library, creating chains of microfilm frames in which the same document or extract could be part of multiple trails at the same time.

New additions could be inserted either by photographing them on to microfilm or by purchasing a microfilm of an existing document. Indeed, a user would be able to augment their memex with vast reference texts. “New forms of encyclopedias will appear,” said Bush, “ready-made with a mesh of associative trails running through them, ready to be dropped into the memex”. Fascinatingly, this isn’t far from today’s Wikipedia.

Where it led

Bush thought the memex would help researchers to think in a more natural, associative way that would be reflected in their records. He is thought to have inspired the American inventors Ted Nelson and Douglas Engelbart, who in the 1960s independently developed hypertext systems, in which documents contained hyperlinks that could directly access other documents. These became the foundation of the world wide web as we know it.

Beyond the practicalities of having easy access to so much information, Bush believed that the added value in the memex lay in making it easier for users to manipulate ideas and spark new ones. His essay drew a distinction between repetitive and creative thought, and foresaw that there would soon be new “powerful mechanical aids” to help with the repetitive variety.

He was perhaps mostly thinking about mathematics, but he left the door open to other thought processes. And 80 years later, with AI in our pockets, we’re automating far more thinking than was ever possible with a calculator.

If this sounds like a happy ending, Bush did not sound overly optimistic when he revisited his own vision in his 1970 book Pieces of the Action. In the intervening 25 years, he had witnessed technological advances in areas like computing that were bringing the memex closer to reality.

Yet Bush felt that the technology had largely missed the philosophical intent of his vision – to enhance human reasoning and creativity:

In 1945, I dreamed of machines that would think with us. Now, I see machines that think for us – or worse, control us.

Bush would die just four years later at the age of 84, but these concerns still feel strikingly relevant today. While it’s great that we do not need to search for a book by flipping through index cards in chests of drawers, we might feel more uneasy about machines doing most of the thinking for us.

Is this technology enhancing and sharpening our skills, or is it making us lazy? No doubt everyone is different, but the danger is that whatever skills we leave to the machines, we eventually lose, and younger generations may not even get the opportunity to learn them in the first place.

The lesson from As We May Think is that a purely technical solution like the memex is not enough. Technology still needs to be human-centred, underpinned by a philosophical vision. As we contemplate a great automation in human thinking in the years ahead, the challenge is to somehow protect our creativity and reasoning at the same time.

Martin Rudorfer does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.