In 2018, I skipped my own college graduation. Two days before, at dawn, I instead flew to the Bay Area to spend that weekend attending a retreat for young people in the technology industry. It was a three day long extravaganza, half conference, half party, funded by charging venture capitalists for the right to come in and chat about artificial intelligence and life extension with 22 year olds. The overarching topic of the weekend was how new technological tools could make the world a better place, as reflected in the retreat's mission statement:

1. Technology isn’t an industry; it’s a tool that’s changing every industry.

2. People don’t make impact alone; great things are accomplished together.

3. We feel a deep sense of obligation to make the world better.

While the retreat had been running annually for half a decade, I suspect 2018 felt a little different. Amidst the growing unpopularity of the tech industry, we were meeting just a few months after the Cambridge Analytica scandal broke. Many of us had growing disillusions of one sort or another with the large tech companies. Increasingly, it was hard to say their contributions to the world outweighed their misdeeds.

The young founders I met still hoped their startups could challenge the power of the large tech companies, and thus solve the ethical problems those companies bring to our industry. As just one of many examples, at some point I had dinner with one of the retreat directors, who told me excitedly about a startup that was optimizing neural network meta parameters. She believed neural networks, a technology that requires vast data collection and in some cases millions of dollars in server resources, could be wielded by startups against the establishment. Large corporations supposedly were too slow to capture this new market.

The founders at the retreat were mostly in the earlier stages of their startups, and perhaps this obscured the absurdity of this David vs. Goliath mentality in an industry where the most common successful outcome is getting acquired by a large tech company. But for now, let's pretend our startup is able to overcome the monumental challenges against it, will refuse to sell, and will actually stay independent while growing exponentially. How would we dodge the shortcomings of the tech companies that exist already?

At the retreat, I often heard "incentive alignment" as a solution. This philosophy recognizes the profit models of many successful technology companies have negative consequences. It knows any business is incentivized to turn a profit by any means possible. So the solution is to find a business model where the more a company profits, the more good it does in the world. This makes the business incentive aligned with their morals. I admire one company that abandoned many months of work and an otherwise promising platform in the therapy world, purely because they realized their profit model would incentivize bad outcomes for their patients: if their patients rapidly improved, they would also rapidly cancel their monthly subscription.

Unfortunately, there are major shortcomings with this approach. After all, Google was an incentive aligned company. In their 2004 IPO letter:

Google is not a conventional company. We do not intend to become one. Throughout Google’s evolution as a privately held company, we have managed Google differently...our goal is to develop services that improve the lives of as many people as possible—to do things that matter...Advertising is our principal source of revenue, and the ads we provide are relevant and useful rather than intrusive and annoying.

Their belief was that, unlike existing companies that had perverse incentives, the ads they showed were relevant to people's search queries, making their drive for profit aligned with their goal of universal access to information. This is the same argument made by startups claiming incentive alignment today. If a startup is not acquired, they face the eternal hockystick graph: the demand from investors to grow exponentially, forever. Even if I say I can start with a business model that is incentive aligned, if my company is successful, this need for growth will eventually force me to turn to more dubious sources, like how Google now tries to build software for military drones.

In addition, when we "avoid perverse incentives", we are not challenging or changing businesses in our industry that already have perverse incentives. The incentive aligned world that we'd like to imagine has no way to ethically build an Uber, a company incentivized to pay their drivers as little as possible, without overtime, vacation, or health care. This theory is also incompatible with building an Airbnb, a company whose short-term rentals drive up long-term rents for locals. As a theory entirely built around choosing an ethical business model, it has no way to challenge unethical business models that already exist. Like many technological solutions, incentive alignment provides a thin veneer of egalitarianism without actually having to challenge power.

Google is a particularly great case study, since it shows what happens when a startup promises to early employees they are "doing good" through incentive alignment. This promise had enough power to make employees at one of the largest corporations in the world believe their work was countercultural. This kept expectations sky-high, as Ben Tarnoff in Logic points out:

there was another important reason that Google became a hotspot for organizing. Its employees tended to have a more utopian outlook, aptly summarized by its former slogan, “Don’t be evil.” Employees expected executives to put ethical considerations above profit-making. When this expectation was not fulfilled, it created a sense of betrayal that fed a process of radicalization

These disparities between expectations and reality grew and became more visible — donations to anti-LGBT candidates, close partnerships with large oil companies, work on military drones, and of course, the Andy Rubin payouts.

I was working at Google during the 2018 Walkout for Real Change, a protest over the $90 million payout to a known sexual harasser, attended by more than 20,000 Google employees worldwide. At the walkout, I heard frustration from old-timers and "nooglers" (god help us) alike that they wish they could return to the good old days, back before Google built software for drones, before they fired dissenters and gave massive payouts to sexual harassers. Back when Larry and Sergey were running the place. At a later protest after I quit, my coworkers held up signs: "Save Our Culture".

I wonder if what some of us really wanted was a return to the days when we could still believe Google's work was ethical, back when this dream of incentive alignment was all we needed to believe in the morality of our executives. I am skeptical of this nostalgia; incentive alignment was perhaps a more convincing myth in the early days, but even then it failed to reflect reality. We were protesting payouts from 2013, smack in the middle of this old "open culture" we were asking for a return to. We wanted a return to the old days when Larry and Sergey gave weekly talks, while protesting the Rubin payouts that those two approved. Of course, my sexually assaulted coworkers never had the option of ignorance. The company had always poorly handled sexual harassment and assault, these problems were just hidden to the largely white and male workforce. The myth of incentive alignment was always a myth, as Larry and Sergey themselves point out in their 1998 PageRank paper:

We expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers. Since it is very difficult even for experts to evaluate search engines, search engine bias is particularly insidious.

Larry and Sergey have since retreated from the public eye, but in an unusual arrangement for a public company, still own a majority voting stake. Even if we set aside the perverse incentives with advertising, they can at least use their dictatorial control over the company to ensure we don't build for military drones, right? Certainly we should find them at fault, but it's here that we, the employees, are complicit in these "perverse incentives" too. Our compensation in stock is intimately tied to this necessity for exponential growth. We expect our stocks to go up, too. Even in an imaginary world where Larry and Sergey wanted to limit Google's growth to work only on what they believed to be ethical, the compensation scheme provided to employees would require exponential growth to keep salaries competitive. Startup founders can't keep their promise of incentive alignment, even if they want to. Our industry has tried it before, and it doesn't work. Without a radically different approach, the argument in the face of this history boils down to saying existing big company CEOs are just evil, and that our startup has nicer, more ethical people running it. Perhaps our startup can also adopt a nice slogan to represent this, like "don't be evil".

Incentive alignment as an ideology is alive and well today. When Sundar Pichai testified to Congress last week, in response to allegations of anticompetitive behavior, he argued Google's interests were in fact aligned with their small business users. Under oath, he told Congress that whether Google was "building tools for small businesses or platforms like Android, Google succeeds when others succeed." While we continue to vent frustrations at the large tech companies, many of the folks I know in tech continue to believe in Sundar's message here. They might be angry about the Rubin payouts and drone software, but still see the core product of search ads as incentive aligned with both users and small businesses. Like Sundar, they highlight how search is an incredible tool for users around the world. If we want to understand the staying power of incentive alignment, we have to understand its connection to that concept — "an incredible tool".

It's not just Sundar saying this. The theme of "tools" appears again and again throughout all these examples of incentive alignment. At last week's hearing, in response to accusations that Apple used monopoly power to squeeze developers, Tim Cook claimed Apple in fact gave developers "cutting edge tools like compilers". Zuckerberg also claimed incentive alignment, saying Facebook gives "small businesses access to tools that only the largest players used to have". In a 2018 congressional hearing, he also described how "people everywhere have gotten a powerful new tool to stay connected". The ethical technology retreat's mission statement described tech as "a tool that’s changing every industry". Giving the world free access to tools like email and search formed the core of Google's early public goodwill.

"Technology is a tool" is the only theory of technology compatible with the myth of incentive alignment, which is why we often see the two claims appear together. For example, describing Google Search as a tool highlights the information-finding abilities it gives to its users. The story goes that in order to grow profits, Google needs to make Search an even better tool, and by making it a better tool, the abilities of users are even more greatly enhanced. Google Search is only incentive aligned with its users when we think of it as a tool. This enframing of technology also allows tech CEOs to claim that their users have choices. Sundar illustrated this connection last week when he argued "we have extensive tools for advertisers and above all for users, we give a choice". When we imagine Google as a tool like a hammer, we find it easier to see it as something you can only pick up when you want to use it. It's no surprise, then, that tech CEOs at antitrust hearings describe their technologies as tools. Doing so promotes this idea that users have choices, and that the companies' incentives are aligned with their users.

Google Search is not valuable because it is a fantastic tool in the same way a hammer company is valuable because they make fantastic hammers. Much of Google's value comes from its near-monopoly on global internet searches. Its 92% market share makes it the gatekeeper of much of the world's information, requiring advertisers to spend $31B annually to get their information in front of users. This monopoly is also essential for collecting behavior data — necessary to build neural networks that further improve search results. This data collection creates a power-centralizing feedback loop. Today, no matter how much money you spend, you can not make a search engine with the same results quality as Google.

Many, many software companies today draw their value from monopoly in this way. Grubhub was just acquired last month for $7.3 billion. Does Grubhub have unique technology that is worth $7.3 billion? Or is it that they can use all the users who have already registered on their platform as a bartering tool to take, in some cases, over 40% in platform fees from struggling restaurants during a pandemic? Amazon draws its value from its monopsony of purchasers, Facebook from how a social network is more valuable when all your friends are on it, and Uber and Lyft form a sort of oligopoly that aggregates both supply and demand. It's impossible for a monopoly like this to be incentive aligned. Their monopoly power makes them essential to many people's lives and livelihoods, giving them incredible power over the so-called "users" of their technology. For a perfect monopoly, the question of serving user interests becomes irrelevant to making money. While it would be unfair to describe Grubhub or many other tech companies as perfect monopolies, describing their technologies in the language of tools obscures the monopoly power they do wield. It's not incorrect to describe these technologies as mere tools, but this ideology's inability to see technology as monopoly means this way of speaking limits our thinking.

Beyond just being a failed theory of ethical technology, incentive alignment in many cases works against this stated goal, because tech companies wield it to obscure their monopoly power. If we want to truly challenge this power, we have to stop promoting the myth of incentive alignment, and we must stop describing technologies like Google Search using the vocabulary of tools.

← More Posts