The History of AI, Part 5: Perceptrons, ELIZA, SHRDLU, and Shakey

The History of AI, Part 5 of 24. In Part 4 the field got its name at Dartmouth and entered its "golden years." This chapter is a guided tour of the machines and programs that made those years feel golden: a learning machine that thrilled the press, a chatbot that unsettled its own creator, a program that could talk about blocks, and a robot that wobbled around a laboratory.

If you could visit an AI lab in 1966, what would you see?

You wouldn't see a robot butler. You wouldn't see anything that looked like a modern chatbot or a self-driving car. What you'd see are mainframes the size of refrigerators, teletype terminals clattering out text, and graduate students arguing in front of blackboards covered with logic notation.

But if you looked at what the programs did, you'd be astonished. They solved algebra word problems written in English. They held conversations. They manipulated imaginary blocks in response to typed commands. They steered a wheeled robot around a room. They learned to recognize simple patterns from examples.

This chapter looks at four landmark creations of the mid-1960s and early 1970s, each of which taught the field something important, usually something it didn't want to hear. We'll write a little code for two of them so you can feel how they worked.

The Perceptron: a machine that learns

Let's start with the connectionist side of the story, which we left in Part 3 with McCulloch, Pitts, and Hebb.

Frank Rosenblatt's Mark I Perceptron, a U.S. Navy-funded machine that learned to identify letters. Public domain, via Wikimedia Commons.

Frank Rosenblatt (1928-1971) was a psychologist and researcher at the Cornell Aeronautical Laboratory in Buffalo, New York. He was fascinated by how brains learn, and he wanted to build a machine that learned in a similar way, not by being programmed with rules, but by being shown examples and adjusting itself.

In 1957 he described his design in a technical report, and in 1958 he published a more complete account in the journal Psychological Review: the Perceptron.

How the perceptron works

The perceptron is a direct descendant of the McCulloch-Pitts neuron, with one crucial upgrade: the weights are learned.

It takes a set of inputs (say, the brightness values of pixels in an image), multiplies each by a weight, adds them up, and compares the total to a threshold. If the total is high enough, the output is 1 ("yes, it's a triangle"); otherwise it's 0.

The learning rule is beautifully simple:

1. Show the perceptron an example and compute its output.
2. If the output is correct, do nothing.
3. If the output is wrong, nudge the weights in the direction that would have given the right answer: increase weights for active inputs if the output should have been 1, decrease them if it should have been 0.

Repeat over all the examples, many times. Here it is in a few lines of Python:

import numpy as np def train_perceptron(X, y, epochs=20, lr=1.0): w = np.zeros(X.shape[1]) b = 0.0 for epoch in range(epochs): errors = 0 for xi, target in zip(X, y): pred = 1 if xi @ w + b > 0 else 0 update = lr * (target - pred) # 0 if correct, +/-1 if wrong  w += update * xi b += update errors += int(update != 0) if errors == 0: # every example classified correctly  return w, b, epoch + 1 return w, b, epochs X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]) # AND is learnable... w, b, n = train_perceptron(X, np.array([0, 0, 0, 1])) print("AND learned in", n, "epochs; weights =", w, "bias =", b) # ...but XOR never converges, no matter how long you train. w, b, n = train_perceptron(X, np.array([0, 1, 1, 0]), epochs=1000) print("XOR after 1000 epochs: still making mistakes") 

Run this and AND is learned within a few passes. XOR never settles. Keep that in mind, because it is about to become the most famous failure in the history of AI.

The Mark I Perceptron and the press conference

Rosenblatt didn't stop at theory. He built hardware. The Mark I Perceptron, completed around 1958 to 1960 and funded by the US Office of Naval Research, was a physical machine. It had a 20 by 20 grid of 400 photocells as its "retina," connected through a tangle of wires to banks of units whose weights were stored in motor-driven potentiometers, literally little knobs that turned to adjust a connection's strength during learning. You could show it a card with a letter on it, and over many trials it learned to tell the letters apart.

In July 1958 the Navy and Rosenblatt held a press demonstration (using an IBM 704 computer to simulate the perceptron). The next day The New York Times reported on what it called the embryo of an electronic computer that the Navy expected would eventually be able to walk, talk, see, write, reproduce itself, and be conscious of its existence. That was a staggering thing for a newspaper to say about a machine that could, at that moment, distinguish marks on the left of a card from marks on the right.

Rosenblatt himself was more careful in his scientific papers than the press coverage suggested, but he was also a confident and charismatic promoter, and he did make bold claims about where the research could go. He proved the perceptron convergence theorem: if the data can be separated by a straight line (more precisely, a hyperplane), the learning rule is guaranteed to find a separating line in a finite number of steps. That was a real mathematical result, and it made the perceptron respectable.

A cousin: ADALINE

Around the same time, Bernard Widrow and his student Marcian "Ted" Hoff at Stanford developed ADALINE (Adaptive Linear Neuron) in 1960, using a slightly different learning rule, the least mean squares (LMS) or "delta" rule. Rather than only reacting to hard right/wrong errors, it nudged weights in proportion to how far off its output was. That rule is a direct ancestor of gradient descent, the workhorse of modern deep learning. (Hoff later went on to co-invent the Intel 4004 microprocessor.)

The seeds of the problem

For a few years, the perceptron was the toast of the field. But there was a catch, and Rosenblatt's critics were circling. The perceptron could only learn problems where the classes could be separated by a straight line. XOR, in which the correct output for (0,1) and (1,0) is 1 but for (0,0) and (1,1) is 0, can't be separated by any single line.

Rosenblatt knew that multi-layer perceptrons could in principle overcome this, and he explored them. But he had no good way to train the extra layers. That missing piece, a learning rule for multi-layer networks, wasn't widely solved until the 1980s. We will pick up this story in Part 6, because it ended in a controversy that nearly killed the whole approach.

Rosenblatt himself did not live to see the vindication. He died in a boating accident on his 43rd birthday, 11 July 1971.

ELIZA: the chatbot that wasn't thinking

Now to the other end of the spectrum: a program that understood nothing and yet unsettled everyone who used it.

Joseph Weizenbaum (1923-2008) was a German-born computer scientist at MIT who had escaped Nazi Germany as a teenager. Between 1964 and 1966, working on MIT's Project MAC, he wrote a program called ELIZA, named after Eliza Doolittle in Pygmalion (the same Pygmalion myth we met in Part 1, now with a nice irony: the program "learns" to speak, but only superficially).

Joseph Weizenbaum, creator of ELIZA, in 1982. Public domain, via Wikimedia Commons.

ELIZA was intended as a demonstration of how superficial human-machine conversation could be. Its most famous script, DOCTOR, imitated a Rogerian psychotherapist, the type of therapist who mostly reflects the patient's words back as questions. This was a clever choice: it let the program say very little and still seem engaged.

How ELIZA worked (spoiler: it's simple)

ELIZA did pattern matching and substitution. It scanned your input for keywords, ranked by priority, matched a template, and filled a response template with fragments of what you wrote, swapping pronouns ("my" becomes "your", "I" becomes "you"). If it found nothing, it fell back on generic prompts like "Please go on" or "Tell me more."

Here is a miniature ELIZA in Python:

import random, re REFLECT = {"i": "you", "my": "your", "me": "you", "am": "are", "you": "I", "your": "my"} RULES = [ (r"i need (.*)", ["Why do you need {0}?", "Would it really help you to get {0}?"]), (r"i am (.*)", ["How long have you been {0}?", "Why do you think you are {0}?"]), (r"i feel (.*)", ["Tell me more about feeling {0}.", "Do you often feel {0}?"]), (r".*\bmother\b.*", ["Tell me more about your family."]), (r".*", ["Please go on.", "I see. Can you elaborate?"]), ] def reflect(fragment): return " ".join(REFLECT.get(w, w) for w in fragment.lower().split()) def respond(text): for pattern, answers in RULES: match = re.match(pattern, text.lower().strip(" .!?")) if match: groups = [reflect(g) for g in match.groups()] return random.choice(answers).format(*groups) print(respond("I am worried about my exams")) # e.g. "How long have you been worried about your exams?" print(respond("My mother never listens")) # "Tell me more about your family." 

That's the essence. No understanding, no memory, no model of the world: just clever string handling. And yet.

The ELIZA effect

Weizenbaum was startled by what happened next. People talked to ELIZA. They confided in it. His own secretary, who had watched him build the program and knew exactly what it was, reportedly asked him to leave the room so she could have a private conversation with it. Some psychiatrists suggested that ELIZA might be developed into an automated therapy tool. Others, hearing about it, concluded that it demonstrated that computers could understand natural language.

Weizenbaum found all of this disturbing. He had meant ELIZA to show how little it takes to create an illusion of understanding. Instead people were taking the illusion at face value. The phenomenon is now called the ELIZA effect: the human tendency to read understanding, empathy, and intention into the output of a computer program, simply because it uses language fluently.

The effect is arguably more relevant today than ever. When a modern chatbot says "I understand how you feel," many people respond as though something understands. Whether and in what sense the system "understands" is a serious and unresolved question, but the ELIZA effect reminds us that our feeling of being understood is a poor test.

Weizenbaum became one of the field's most outspoken internal critics. In his 1976 book Computer Power and Human Reason, he argued that there are tasks that should not be delegated to machines, even if machines could perform them, such as roles that require empathy, wisdom, and human responsibility. He was dismissed by many of his peers, but many of his concerns, about overreliance on automation and the erosion of human judgment, read as prescient today.

PARRY, the paranoid program

In 1972, the Stanford psychiatrist Kenneth Colby wrote a more elaborate program called PARRY, which simulated a person with paranoid schizophrenia. It had an internal model with variables like "fear" and "anger" that shifted during conversation. In an early version of a Turing-style test, psychiatrists were shown transcripts of conversations with PARRY and with real patients, and they couldn't reliably tell them apart. That same year, ELIZA and PARRY were connected over the ARPANET to talk to each other, which produced a strange, stilted, slightly comic exchange.

The lesson was not that these programs were intelligent. It was that conversation is a very forgiving test. If the user is willing to supply meaning, a program can get away with remarkably little.

Microworlds: the strategy of the age

Before we move on to the next two systems, it helps to understand the research strategy that produced them. Marvin Minsky and Seymour Papert at MIT promoted the idea of microworlds: simplified, artificial domains small enough to be fully described, yet rich enough to require real reasoning. The logic was reasonable: if you can't solve intelligence in the full chaos of the real world, solve it in a restricted world first, then scale up.

Some famous results came from this approach in the 1960s:

• STUDENT (Daniel Bobrow, 1964) solved high-school algebra word problems stated in English by translating them into equations.
• SAINT (James Slagle, 1961) performed symbolic integration at the level of a good first-year calculus student.
• ANALOGY (Thomas Evans, 1963) solved geometric analogy puzzles of the kind found on IQ tests ("A is to B as C is to which of these?").

Each worked within a narrow slice of reality and did impressively well inside it. The hope was that these slices could be gradually connected into a general intelligence. The hard truth, which took years to sink in, was that they didn't scale. Techniques that worked in a toy domain often broke when exposed to the unconstrained world.

Two systems in particular embodied the promise and the problem.

SHRDLU: understanding language in a world of blocks

In 1968 to 1970, a PhD student at MIT named Terry Winograd created one of the most impressive AI demonstrations ever made: SHRDLU (the name came from the old printers' etaoin shrdlu, the second column of letters on a Linotype machine).

SHRDLU lived in a simulated blocks world: a table with blocks, pyramids, and boxes of different colors and sizes, plus a robot arm that could pick things up and move them. You could talk to it in ordinary English, and it would reply and act. A famously cited exchange goes something like this:

Person: Pick up a big red block.
Computer: OK.
Person: Grasp the pyramid.
Computer: I don't understand which pyramid you mean.
Person: Find a block which is taller than the one you are holding and put it into the box.
Computer: By "it", I assume you mean the block which is taller than the one I am holding.

Let that sink in. In 1970, a program was resolving pronouns ("it"), asking for clarification when a reference was ambiguous, planning sequences of physical actions (it would have to put down the block it was holding first, and it could explain why it had done so), and remembering what had happened earlier in the conversation. It could answer questions like "Why did you do that?" by reading back its own reasoning.

How did it work? Rather than treating language understanding as a separate problem from reasoning, Winograd integrated everything. SHRDLU had a parser to analyze grammar, a semantic component to map sentences onto its world model, and a planner and reasoner to figure out what actions to take. Crucially, these components could consult each other: knowledge about blocks helped decide how to parse an ambiguous sentence. It was programmed in Lisp and a language called Micro-Planner.

Why SHRDLU didn't scale

SHRDLU was widely admired, and it seemed to suggest that machines were close to understanding language. But the limitations were severe, and Winograd himself was the first to say so.

• The world was tiny and perfect. The blocks world contained a few dozen objects with precisely known properties. There was no noise, no ambiguity in perception, no unknown objects.
• The knowledge was hand-coded. Every word, every rule of grammar, every fact about the domain had been written in by a human expert.
• It was brittle. Step outside the narrow vocabulary and domain, and it collapsed. Extending it to a kitchen, for instance, would not have been a matter of adding a few more words; it would have required encoding vast amounts of everyday knowledge about food, containers, and cooking.

Winograd later turned away from the approach. Influenced by philosophers like Martin Heidegger and by colleagues like Fernando Flores, he came to believe that human understanding is not a matter of manipulating symbolic representations in the head but is rooted in embodied, social activity. He moved into human-computer interaction, taught at Stanford, and (as a minor footnote) later served as the doctoral advisor of Google co-founder Larry Page. His name lives on in the Winograd Schema Challenge, a test of commonsense reasoning (in which pronoun resolution depends on real-world knowledge, as in "The trophy doesn't fit in the suitcase because it is too big"), which was proposed in 2011 by Hector Levesque, inspired by an example in Winograd's work.

SHRDLU is the poster child for a pattern that will repeat in Parts 7 and 8: impressive demos in narrow domains, followed by a hard collision with the open-ended messiness of reality.

Shakey: the first robot that reasoned

If SHRDLU lived in a simulated world, Shakey lived in a real one.

Shakey the robot, built at SRI International (Computer History Museum). Image: The wub, CC BY-SA 4.0, via Wikimedia Commons.

From 1966 to 1972, a team at the Stanford Research Institute (SRI), led by Charles Rosen and including Nils Nilsson, Peter Hart, Bertram Raphael, and Richard Fikes, built Shakey the Robot, funded by DARPA. It was a boxy wheeled platform about the size of a small refrigerator, topped with a TV camera, a range-finder, and bump sensors ("cat whiskers"). Its trademark was a wobbling motion when it moved, which earned it the name.

Shakey could be given a goal in plain language, such as "push the block off the platform," and it would work out a plan. It had to look at its environment through the camera, build a model of the rooms, decide which actions would achieve the goal, and carry them out, adapting if something went wrong, like a door being closed or a box being in the way.

It was, in a real sense, the first mobile robot that reasoned about its own actions. It was controlled by software running on a large computer (a PDP-10, later PDP-15) that communicated with the robot over a radio link, because the robot couldn't carry that much computing power. A single move could take minutes, and complex tasks could take hours.

Shakey's gifts to computer science

What makes Shakey so important is less the robot than the algorithms invented to make it work. Several of them are still in everyday use:

• A* search (1968, by Hart, Nilsson, and Raphael), a path-finding algorithm that finds the shortest route efficiently by combining the cost so far with an estimate of the remaining distance. A* is used in video games, GPS navigation, and robotics today. Here's the core idea in one line: choose the next node that minimizes f(n) = g(n) + h(n), the cost to reach n plus a heuristic estimate of the cost to get from n to the goal.
• STRIPS (1971, by Fikes and Nilsson), the Stanford Research Institute Problem Solver, a way of representing actions in terms of preconditions and effects (e.g., "to push a box, the robot must be next to it; afterwards, the box has moved"). STRIPS-style planning is the ancestor of modern AI planners.
• The Hough transform and related methods for detecting lines and shapes in images, which were refined in this period and used in computer vision.

A* is a lovely illustration of how the field's search ideas matured. Here's a compact version on a grid:

import heapq def astar(grid, start, goal): def h(p): # Manhattan distance heuristic  return abs(p[0] - goal[0]) + abs(p[1] - goal[1]) open_set = [(h(start), 0, start, [start])] seen = set() while open_set: f, g, pos, path = heapq.heappop(open_set) if pos == goal: return path if pos in seen: continue seen.add(pos) for dx, dy in ((1, 0), (-1, 0), (0, 1), (0, -1)): nx, ny = pos[0] + dx, pos[1] + dy if 0 <= nx < len(grid) and 0 <= ny < len(grid[0]) and grid[nx][ny] == 0: heapq.heappush(open_set, (g + 1 + h((nx, ny)), g + 1, (nx, ny), path + [(nx, ny)])) return None grid = [ [0, 0, 0, 0], [1, 1, 0, 1], [0, 0, 0, 0], [0, 1, 1, 0], ] print(astar(grid, (0, 0), (3, 3))) 

Compare it with the water jug search from Part 4. The only real difference is that the next state to explore is chosen using an estimate of how close it is to the goal. That one change is a textbook example of using knowledge to tame the combinatorial explosion.

The media sensation

In 1970, Life magazine ran a feature on Shakey, calling it the "first electronic person" and quoting researchers who suggested that machines with human-level intelligence might arrive within years. (The researchers later said that some of the quoted timelines were exaggerated by the writer, but the headline captured the mood.) The article is a perfect snapshot of the era's hype, and also of the gap between rhetoric and reality: the actual robot was slow and fragile, and operated only in carefully constructed rooms with plain painted walls and specially marked objects.

Shakey now sits in the Computer History Museum in California, a delightful relic. In terms of legacy, though, its true heirs are the autonomous vehicles and warehouse robots of today, whose planners descend from STRIPS and whose route-finders descend from A*.

What these four teach us

Let's lay the four side by side, because the contrast is instructive.

A few patterns emerge:

1. Narrow success, broad failure. Each program worked in a restricted domain. Scaling up required vastly more knowledge and flexibility than anyone had built.
2. Hand-coded knowledge is a bottleneck. The programs were smart because humans had carefully written in what they needed to know. Getting from "dozens of facts" to "everything a person knows" was a gap of many orders of magnitude.
3. Perception was harder than reasoning. Shakey could plan elegantly once it knew where things were, but figuring out where things were from camera images was painfully slow and error-prone.
4. Hype outruns results. The press coverage of the Perceptron and Shakey promised far more than the machines could deliver.

None of this means the work was a failure. Quite the opposite: it created tools (A*, STRIPS, parsing methods, pattern matching, knowledge representation) that are still used. But it set the stage for a collision between expectations and reality.

A glimpse of the cracks

By the end of the 1960s, there were growing signs of trouble.

• Machine translation, one of the most heavily funded early goals, was progressing slowly. A government committee would soon recommend cutting funding.
• Neural networks were under mathematical attack. A book was in preparation by two of AI's most prominent figures that would argue the perceptron approach was a dead end.
• Researchers working on problems like vision and robotics were finding that the real world was far messier than their microworlds.
• Funding agencies were beginning to ask: where are the results we were promised?

The golden years were about to meet the first great reckoning, and the connectionist camp was about to take the hardest hit.

Timeline: 1957 to 1972

• 1957: Rosenblatt describes the Perceptron; Newell and Simon begin work on the General Problem Solver.
• 1958: McCarthy creates Lisp; the Perceptron's press demonstration.
• 1960: Widrow and Hoff's ADALINE.
• 1964 to 1966: Weizenbaum builds ELIZA; Bobrow's STUDENT solves algebra word problems; SRI starts Shakey.
• 1968: A* search published.
• 1969: Perceptrons by Minsky and Papert (Part 6).
• 1970: Winograd's SHRDLU; Life magazine's Shakey feature.
• 1971: STRIPS planner; Rosenblatt dies.
• 1972: PARRY, the simulated paranoid patient; Prolog is created.

The ELIZA effect, today

Weizenbaum's observation has aged remarkably well. Modern chatbots are vastly more capable than ELIZA, but the human reaction is the same: when something talks to us fluently, we reach for the vocabulary of minds, saying it "wants," "thinks," or "feels." A few practical habits help keep your balance:

1. Separate fluency from reliability. A smooth answer tells you about the system's skill at producing language, not about whether the content is true.
2. Notice when you're supplying the meaning. Ask whether the system really said something specific, or whether you filled in the gaps charitably.
3. Remember who benefits. Products that feel like friends keep users engaged, and engagement is a business metric. That doesn't make them bad, but it's worth noticing.

Quick glossary

• Pattern matching: comparing input against templates to choose a response.
• Microworld: a simplified, self-contained domain used for AI research, such as SHRDLU's blocks.
• Parser: a program that analyzes the grammatical structure of a sentence.
• Precondition and effect: in planning, what must be true to take an action and what changes once it's taken.
• Heuristic search: search guided by an estimate of how close a state is to the goal (A*).

Key takeaways

• Rosenblatt's Perceptron (1958) was the first practical learning neural network. It learned by adjusting weights, and its convergence theorem was real, but it could solve only linearly separable problems.
• ELIZA (1966) used simple pattern matching to simulate a Rogerian therapist and revealed the ELIZA effect: people readily attribute understanding to fluent-sounding programs. Its creator, Weizenbaum, became a leading critic of overreliance on AI.
• SHRDLU (1970) integrated language understanding, reasoning, and action in a blocks world, a stunning demo that also exposed the brittleness of hand-coded, narrow-domain systems.
• Shakey (1966-72) was the first mobile robot to plan its own actions, and it gave computer science A* and STRIPS.
• Early AI's microworld strategy produced impressive results that did not scale to the open-ended real world.

Try it yourself

1. Fix the perceptron's blind spot. Add a hidden layer by hand: build XOR from two perceptrons (one computing OR, one computing NAND) feeding a third computing AND. You've just solved XOR with a two-layer network, exactly what Rosenblatt couldn't train.
2. Make ELIZA a little smarter. Add rules for "because," "always," and "dream," and give it a short memory so it can circle back to something you mentioned earlier ("Earlier you said your exams worried you..."). Notice how much more convincing it feels. What does that say about how we judge understanding?
3. Change the heuristic in A*. Set h to zero (this turns A* into plain uniform-cost search) and count how many cells each version explores on a larger grid. Then try a deliberately bad heuristic and see how the path quality changes.

Next up

In Part 6, we confront the first great downturn. A 1969 book called Perceptrons by Marvin Minsky and Seymour Papert delivers what many see as the knockout blow to neural networks. In 1966 the ALPAC report slashes funding for machine translation. And in 1973 the British mathematician Sir James Lighthill delivers a devastating review of AI that helps trigger the first AI winter. How does a field with so much promise freeze over?

If you've ever been fooled by a chatbot, or if you've built a toy AI project that fell apart on real data, share your story in the comments. It will fit right in with the next chapter.

See you in Part 6.

☕ If this series helped you, you can support my writing on Ko-fi.

Source：https://dev.to/suraj_nandan/the-history-of-ai-part-5-perceptrons-eliza-shrdlu-and-shakey-24oe