Pong Without Hands
In April 2021, a company called Neuralink posted a video of a monkey named Pager playing the video game Pong [1]. The joystick in front of Pager was unplugged [2]. The paddle still went wherever Pager wanted it to go.
About six weeks earlier, surgeons had placed two small implants in Pager's head, one on each side of his brain [1, 2]. Each implant listens to the brain through 1,024 tiny electrodes, carried on threads thinner than a hair [1].
Make your guess. How does the computer know which way Pager wants the paddle to go?
- It finds the one brain cell that means "up" and the one that means "down"
- Many brain cells each fire a little faster for their own favourite direction, and the computer adds up their votes
- It picks up tiny twitches in Pager's arm muscles
Show answer
B. Many brain cells each fire a little faster for their own favourite direction, and the computer adds up their votes
Many cells vote, and no single cell decides. Each brain cell fires a little faster when a move goes its favourite way. One vote alone is noisy, but the sum of many votes points where Pager is aiming. Muscles play no part: the implants sit in the brain, not in the arm.
Start where Neuralink started: with a computer that knows nothing about the brain it is listening to. Can a computer learn to move a cursor from brain signals alone?
Train the decoder
The program that turns brain signals into movement is called a decoder. The decoder below listens to 40 simulated brain cells and starts out knowing nothing about them.
Your goal: press Train until the cursor hits all 8 targets with the joystick unplugged.
Before training, the cursor scribbled. Each press let the decoder hear two joystick reaches, and the paths straightened out. Neuralink trained Pager's decoder the same way: Pager first played with a working joystick, and the computer matched his brain activity to the joystick's moves [1].
Then the joystick was unplugged, and the decoder took over. Here is Neuralink's own video of Pager at play:
So what did the decoder learn from those reaches? To find out, listen to a single brain cell.
Listen to one neuron
Your brain holds tens of billions of cells called neurons. A neuron signals other neurons with tiny electrical pulses called spikes, each about a thousandth of a second long.
Pager's implants sit in the motor cortex, the part of the brain that plans and controls movement [1]. Each electrode picks up the spikes of a few nearby neurons, the way a microphone on one table picks up one conversation in a busy café.
Your goal: press the arrows to move the arm, and find the direction that makes this neuron fire fastest. Then press that arrow a second time.
Each neuron in the motor cortex has a favourite direction, called its preferred direction. The neuron fires fastest for moves that way, less for moves to the side, and least for moves the opposite way.
In 1982, Apostolos Georgopoulos and his team found this pattern by recording neurons while monkeys moved an arm in 8 directions [3]. About 3 in 4 of the active neurons fired most for one direction and less and less for directions farther away. Different neurons preferred different directions.
The decoder in the first game learned just one thing: each neuron's preferred direction. But the neuron above showed a problem. The same arrow gave a different count each time. How can a computer trust a neuron that never gives the same answer twice?
Many noisy votes
A neuron's count jumps around from one try to the next. What would make the computer's guess steadier?
- Listening to one neuron for longer
- Listening to many neurons at once
- Both would help
Show answer
C. Both would help
Both help, because random ups and downs partly cancel when you add many counts. But a cursor can't wait a whole second for each answer: Neuralink's decoder gets fresh spike counts every 25 milliseconds [1]. So the practical fix is more neurons at once.
Here is how the decoder combines neurons. Each neuron casts a vote along its preferred direction. A neuron firing faster than usual votes toward its direction; a neuron firing slower than usual votes away from its direction. All the votes add up to one arrow, which researchers call a population vector.
Your goal: switch from 1 neuron to 1,000, and watch the blue arrow settle onto the dashed one.
One neuron can only say "more of my direction" or "less of my direction", so the blue arrow flips around. With 1,000 neurons, the random errors mostly cancel. Multiply the neurons by 100, and the error shrinks about 10 times.
That is why the number of electrodes matters. In 2006, a paralysed man moved a computer cursor through an implant with 96 electrodes [4]. Each of Neuralink's implants has 1,024 [1].
More neurons tame the noise. But Pager could train his decoder with a joystick. How do you train a decoder for someone who can't move a joystick at all?
No joystick needed
Paralysis often cuts the wiring between brain and body, not the brain itself. The motor cortex still fires when a person tries or imagines a move. In the 2006 study, a man known in the paper as MN, who could not move his arms or legs, opened simulated email and worked a TV with brain signals alone, three years after his spinal cord injury [4].
To train a decoder without a joystick, the computer moves the cursor toward targets by itself while the person imagines steering it. The decoder learns from the imagined moves [1].
In early 2024, Noland Arbaugh became the first person to receive Neuralink's implant, in a study called PRIME [5]. Before the implant, Arbaugh used a stylus held in his mouth, which a caregiver had to put in place. With the implant, Arbaugh plays chess and Civilization VI online, browses the web, and has played Mario Kart, all by moving a cursor with his mind [5].
The implant has had trouble too. In the weeks after the surgery, some of the threads pulled back out of Arbaugh's brain, fewer electrodes worked, and his score dropped. Neuralink changed the software to make better use of the signals left, and the scores climbed past where they had started [5].
Decades in the making
| When | What happened |
|---|---|
| 1960s | Experiments by Eberhard Fetz, where one expert says the idea began [2] |
| 1982 | Georgopoulos's team finds preferred directions in the motor cortex [3] |
| 2002 | The first published studies of monkeys moving a cursor with brain signals alone [2] |
| 2006 | MN moves a cursor and opens a prosthetic hand, through cables that pass through the skin [1, 4] |
| 2021 | Pager plays MindPong with a wireless implant [1] |
| 2024 | The first person receives a Neuralink implant [5] |
| January 2026 | Neuralink reports 21 participants in its trials worldwide [6] |
Scientists outside the company put the video in this context. Andrew Jackson, who studies neural interfaces at Newcastle University, told CNBC that monkeys moving cursors with their brains was not new. The new part, he said, was the wireless link: no cables through the skin, where cables can let in infection [2].
A device that reads the brain can give back a lot. A device that reads the brain also raises questions a mouse never did. What are those questions?
The hard questions
Guess first. More than 350 blind people got bionic eye implants from a company called Second Sight. Then the company ran short of money and laid off most of its staff. What happened to the people with the implants?
- The company removed every implant for free
- Another company took over the repairs
- Support faded, and the implants inside their eyes can't be repaired or replaced
Show answer
C. Support faded, and the implants inside their eyes can't be repaired or replaced
Support faded. Promised upgrades never came, the company offered only limited help with the outside parts, and the implants themselves can't be repaired or replaced [8]. A brain implant raises the same question: who keeps the device working if its maker shuts down?
The Second Sight story raises one hard question: who keeps a brain implant working for decades? People are arguing about three more right now:
- The animals. Monkeys like Pager make this research possible. Neuralink says its animals take part in tasks by choice and are trained with treats [1]. In 2022, Reuters reported a federal investigation into possible animal-welfare violations at Neuralink, after employees said tests were being rushed [7]. How much harm to animals is acceptable for a medical advance?
- The surgery. Every implant means brain surgery, and hardware can shift inside the brain, as the threads in Arbaugh's implant did [5]. How much risk should a person accept for a better cursor?
- The data. Brain signals may reveal more than where you want a cursor to go. In 2024, Colorado added brain data to the sensitive data protected by its privacy law [9]. Who should be allowed to keep recordings of your brain?
Those questions have no formula. The decoder does. How does a sum of noisy votes find the right direction?
How it really works
The whole decoder fits in two lines of math. First, how fast neuron \(i\) fires during a move in direction \(\theta\), where \(\theta_i\) is the neuron's preferred direction:
Georgopoulos's team found that this cosine curve fits most of the direction-tuned neurons they recorded [3]. Second, the decoder's answer: each neuron votes along its preferred direction, weighted by how far the neuron's spike count \(n_i\) is above or below its usual count \(\bar n_i\):
An animation of how the decoder adds the votes of seven neurons. Seven gray vote arrows point in different directions, and they slide one by one so that each arrow starts where the one before it ends. A blue arrow for the sum grows with each vote and ends close to a dashed arrow for where the monkey is aiming. The animation plays by itself and repeats with a new set of noisy votes each time. It pauses for a moment once the sum is complete.
Text version
A dot marks the start, and a dashed arrow from the dot shows where the monkey is aiming, up and to the right. Seven thin gray arrows fan out from the dot, one for each neuron's vote. The arrows point in many directions, and some point away from the target, because each vote is noisy. One at a time, each vote arrow slides without turning until its tail sits on the head of the previous arrow, so the arrows form a zigzag chain. After each arrow lands, a thick blue arrow from the dot stretches to the new end of the chain. When all seven votes have landed, the blue sum arrow points close to the dashed arrow, the gray votes fade back, and the animation holds still for a moment. Then everything fades out and the animation starts again with seven new noisy votes.
The full recipe, step by step (try step 3 yourself first)
- Counting spikes. Neuralink's implant spots spikes on each of its 1,024 electrodes, counts the spikes in slices of 25 ms, and sends the counts over Bluetooth to a computer [1]. Here \(n_i\) is the count on electrode \(i\) in one slice.
- One neuron is a straight-line function of the direction. Write the move's direction as \(x = \cos\theta\) and \(y = \sin\theta\), and expand the cosine:
\[ r_i = b_i + (m_i\cos\theta_i)\,x + (m_i\sin\theta_i)\,y \]The rate is a constant plus a weight times \(x\) plus a weight times \(y\). That is why weighted sums can decode it.
- Why the votes point the right way. Take \(N\) neurons with the same \(m\) and preferred directions spread evenly around the circle. On average \(n_i - \bar n_i\) is proportional to \(m\cos(\theta - \theta_i)\). The \(x\) part of the sum is
\[ \begin{aligned} &\sum_i \cos(\theta-\theta_i)\cos\theta_i \\ &= \cos\theta\sum_i \cos^2\theta_i + \sin\theta\sum_i \sin\theta_i\cos\theta_i \\ &= \tfrac N2\cos\theta \end{aligned} \]because evenly spread angles give \(\sum\cos^2\theta_i = N/2\) and \(\sum\sin\theta_i\cos\theta_i = 0\). The \(y\) part works the same way, so the average sum is \(\tfrac{N m}{2}\,\hat u(\theta)\): it points exactly at \(\theta\).
- Why more neurons beat the noise. A spike count with average \(\lambda\) is close to a Poisson count, which swings by about \(\sqrt{\lambda}\). In the sum, the signal grows like \(N\) and the random part grows like \(\sqrt N\), so the angle error shrinks like
\[ \text{error} \propto \frac{1}{\sqrt N} \]100 times more neurons, about 10 times less error.
- Learning the preferred directions. During training, the direction of each reach \(\varphi_t\) is known. The game estimates each neuron's preferred direction from its counts:
\[ \tan\hat\theta_i = \frac{\sum_t n_{i,t}\sin\varphi_t}{\sum_t n_{i,t}\cos\varphi_t} \]taking the angle on the side where the counts point. With few reaches, the estimates are rough; with more, the estimates close in on the truth.
- What real decoders add. Real decoders learn a weight for every electrode by fitting the training data (a method called least squares), which copes with neurons that fire together or crowd around one direction. Neuralink's software also mixes counts from the last 25 ms up to the last 250 ms, and outputs a velocity that moves the cursor [1]. Newer decoders use neural networks: the 2021 handwriting study used one [10].
- The loop closes through the person. The person watches the cursor and corrects course, so the decoder never has to be perfect. The monkey in the first game does the same: the monkey always aims from where the cursor is now.
Pager's paddle, explained
Now you can explain the video. Pager's implants listened to neurons in his motor cortex, and each neuron fired a little faster for its own preferred direction. While Pager used the joystick, the decoder learned those preferred directions. Then the decoder turned every 25 ms of spikes into a sum of votes, and the sum moved the paddle.
Each vote is noisy, so the decoder needs many votes, and the paddle still wobbles a little. The same recipe now lets people who can't move their hands use a computer. The recipe doesn't answer the hard questions about animals, surgery and brain data: people have to.
Where it shows up
- Typing by imagined handwriting. In 2021, a person whose hand was paralysed imagined writing letters, and a decoder turned his brain activity into text at 90 characters a minute [10].
- Speech. In 2023, a decoder turned the attempted speech of a person who could no longer speak clearly, because of the disease ALS, into text at 62 words a minute [11].
- Hearing. A cochlear implant runs the other way: the implant turns sound into electrical pulses for the hearing nerve. About 737,000 cochlear implants had been fitted worldwide by the end of 2019 [12].
- Machine learning everywhere. A decoder learns from examples. The same idea, fed with taps or sound instead of spikes, reads your handwriting and your voice on a phone.
Where to go next
One monkey's paddle opens a lot of doors. Here are the questions worth chasing next:
- Could a decoder read whole words instead of directions, and how would it tell the words apart?
- Why do implant threads move after surgery, and what would an implant need to last for decades?
- Can signals go the other way, from a computer into the brain, to give back a sense of touch?
- How does a computer learn from examples, and what is a neural network?
- Who decides when an experimental brain implant is safe enough to try in a person?
What to learn next: start with how a neuron fires (search "action potential"), then vectors and how to add vectors head to tail. Next, try "linear regression", the simplest way a computer learns from examples. Search "BrainGate" for the long-running academic trials, and "neuroethics" for the hard questions.
Ask Phy
- Explain how a neuron fires a spike, step by step, with a drawing.
- Show me why adding 100 times more noisy votes makes the error about 10 times smaller.
- What are the strongest arguments for and against testing brain implants on monkeys?
- Pager Plays MindPong (Neuralink, April 2021) — The company's own account: 1,024 electrodes per implant, motor cortex, spike counts every 25 ms, joystick calibration.
- Elon Musk's Neuralink explains how a monkey used its brain-computer tech to play Pong (CNBC, Sam Shead, April 2021) — Pager's age, the unplugged joystick, and comments from Andrew Jackson of Newcastle University.
- On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex (Georgopoulos, Kalaska, Caminiti and Massey, Journal of Neuroscience, 1982) — Preferred directions and cosine-shaped tuning curves.
- Neuronal ensemble control of prosthetic devices by a human with tetraplegia (Hochberg et al., Nature, 2006) — Participant MN: a 96-electrode array, a cursor, email and a TV.
- PRIME Study Progress Update: User Experience (Neuralink, May 2024) — Noland Arbaugh's first 100 days: games, bits per second, retracted threads.
- Elon Musk's Neuralink says it has 21 participants enrolled in trials (Reuters, via Yahoo News, January 2026)
- Exclusive: Musk's Neuralink faces federal probe, employee backlash over animal tests (Reuters, December 2022)
- Their Bionic Eyes Are Now Obsolete and Unsupported (IEEE Spectrum, Eliza Strickland and Mark Harris, February 2022)
- HB24-1058, Protect Privacy of Biological Data (Colorado General Assembly, 2024) — Adds neural data to the sensitive data in the Colorado Privacy Act.
- High-performance brain-to-text communication via handwriting (Willett et al., Nature, 2021)
- A high-performance speech neuroprosthesis (Willett et al., Nature, 2023)
- Cochlear Implants (National Institute on Deafness and Other Communication Disorders, NIH)