SuperCooperators. Roger Highfield
screamed: “Martin, this was excellent for your career!”
He and I are so different, the odd couple. He is a compact, frizzy haired wisecracker who has little sympathy for religion. I tower over him, a balding Catholic with a Schwarzenegger-like English accent that is a gift when it comes to recording the message on telephone answering machines (“I’m away at the moment but I’ll be back!”). Bob is endowed with a heady blend of traits: a passion for precision, an equal love of profanity, and a hilarious disdain for his peers (“A biologist is someone who wanted to be a scientist but was not good enough to be a physicist”). We were united by our love of games, from the mathematical to the physical, and we both wanted to win. He was bemused when I told him that, remarkably enough, my German lacks an everyday word for “competitive.”
Our rapport had an energizing effect on my work. For my first project, I followed up an idea that first came to me at a high-powered gathering organized by the German Nobel laureate Manfred Eigen in Klosters, Switzerland. During a talk there by Bill Haseltine on the human immunodeficiency virus, HIV, I realized that the body of an AIDS victim must harbor a swarm of closely related replicating viruses. This reminded me of my work with Peter Schuster in mathematical biology. One day, I thought to myself, I would like to develop a mathematical model of virus infections. But my respect at that time for the difficulty of solving problems was almost paralyzing.
I was fortunate that Bob had already studied the virus with a colleague, Roy Anderson. Together, they had charted how the virus spreads between people. But I wanted to take this approach in a new direction. I wanted to model what happens inside a person who is unfortunate enough to have been infected with the virus. That would require explaining how the virus spreads between cells in the face of attacks from the body’s immune system. To find out how HIV fares in the human body, I would have to use a brand of mathematics similar to that used in my tournaments with Karl.
I discovered that I could explain the puzzlingly long delay between HIV infection and AIDS and why this period can vary so greatly between patients—it could show up in less than two years in one person and yet lurk for more than a decade in another. What was remarkable was that I could draw my conclusions from existing data without the need for new experiments on animals or trials on patients. All I needed was a ready supply of computer number crunching power to explore the way that the virus breeds and mutates inside the body.
Bob was so thrilled by this result that he insisted that I show my findings to Roy Anderson, who was by then working at Imperial College London. He too was amazed. After I published the first results in the journal AIDS in 1990, an extended version of my theory and clinical data came out in the prestigious journal Science the year after. I worked, too, on hepatitis B virus with Barry Blumberg, master of Balliol College, who won the Nobel Prize for discovering the virus and making a vaccine. This kind of research helped to establish the field that is now known as virus dynamics, where mathematical models chart out the progress of virus infections within infected hosts.
SOARING EAGLES, DIVING STRATEGIES
Karl and I had so many games left to play, with so many variants and so many potential outcomes. In 1992, our work on Generous Tit for Tat was published in the British journal Nature, which shares with the American journal Science the distinction of being the journal that scientists want to appear in most of all. Karl and I had plenty of new ideas when it came to extending our work. My second summer at Oxford, I once again returned to Austria to resume our explorations of the Prisoner’s Dilemma.
Previously, Karl and I had calculated the strategies that emerge when the decision of a player only depended on the opponent’s last move. But, of course, this only gives a partial picture of what can happen. We now wanted to look at strategies that also take into account the player’s own moves. Let me give you an example to show exactly what I mean by this. Put yourself in the position of a contestant in one of our tournaments. You might be less annoyed with a fellow player who had defected if you had defected too. Equally, you might be more angry with him if you had cooperated.
To find out whether this influenced the winning strategies, I found myself with my new portable computer and Karl in a room of Rosenburg Castle, a fabulous medieval heap in lower Austria, complete with an arcaded yard that was once used for jousting. I was working in this fairy-tale setting because I had to be with Karl. And Karl was there because he had to be with his wife, who was staying in Rosenburg to carry out research on the historical building.
Although I did not know what would happen in the new computer experiments, I had a pretty good idea. Generous Tit for Tat would win the games again. Simple. As Karl and I went through the motions to show that this was indeed the case, there was only one distraction. The castle has a fine collection of birds of prey that, at preordained times, performed in the big courtyard. Handlers clad in Renaissance garb lured the raptors to make spectacular dives, when they skimmed the heads of spectators. They soared and swooped between various points on the façade as Karl and I looked on.
We ran our simulations again and again, taking a break every so often to watch the highlight of these displays, a thousand foot dive by a golden eagle. By then these magnificent birds were a welcome distraction because there was a problem. My favorite, the Generous Tit for Tat strategy, was being beaten again and again in the jousting tournaments on my laptop. This was frustrating, since I had confidently expected this strategy to rule the roost. I found myself wishing that there were more birds to take my mind off my work. There had to be a bug in my program. I checked. And I checked again. I couldn’t find it. I pride myself on being able to do this and had a fail-safe rule: “the bug is always where you are not looking.” Finally the simple truth dawned on me. There wasn’t a glitch this time.
The losing streak of Generous Tit for Tat was telling me something important but at that particular moment I wasn’t listening. I hunted for a way to make the problem go away. But I could not save Tit for Tat. After a few days, having reluctantly decided that the result might be real, I took a closer look and found that a new strategy consistently won. It consisted of the following instructions and they seemed bizarre at first glance:
If we have both cooperated in the last round, then I will cooperate once again.
If we have both defected, then I will cooperate (with a certain probability).
If you have cooperated and I have defected, then I will defect again.
If you have defected and I have cooperated, then I will defect.
Overall, this means whenever we have done the same, then I will cooperate; whenever we have done something different, then I will defect. In other words, this winning strategy does the following: If I am doing well, I will repeat my move. If I am doing badly, I will change what I am doing. I now became intrigued. My mood lifted.
Back in Oxford I told the distinguished biologist John Krebs about the winning strategy when I bumped into him in the corridor of the Zoology Department. He recognized it instantly. “This sounds like Win Stay, Lose Shift, a strategy which is often considered by animal behavioralists.” The strategy was much loved by pigeons, rats, mice, and monkeys. It was used to train horses too. And it had been studied for a century. John was amazed at how the strategy had evolved by itself in a simple and idealized computer simulation of cooperation. So was I.
Now I had to figure out why Win Stay, Lose Shift was better than either Tit for Tat or Generous Tit for Tat. The answer was revealed by studying the details of the cycles of cooperation and defection that turned in my laptop. In the earlier work, one can mark the end of one cycle and the start of the next by the emergence of a population of unconditional cooperators. With random mutations thrown in the mix, a defector always emerges to take over that docile population, marking the start of a new cycle. I discovered that the secret of Win Stay, Lose Shift lay at this stage, when cooperation is at a peak and nice strategies abound. It turns out that unconditional cooperators can undermine the strategies of Tit for Tat and Generous Tit for Tat. But they can’t beat Win Stay, Lose Shift.
In a game with some realistic randomness, Win Stay, Lose Shift discovers that mindless (or unconditional) cooperators can be exploited. The reason is easy to understand: any little