I am an effective altruist, so I’m naturally skeptical of schemes to improve the world that don’t have a lot of evidence behind them. I try to do a lot to support efforts to alleviate global poverty (mostly donating money). I like giving that way because I think it’s very certain that it will do good. I’ve never chosen to donate to research, although I am a researcher by career.
A lot of effective altruists believe that AI safety is very important. They believe it should be a high priority, and it should be given a lot of resources. If that describes you, this essay isn’t for you. You probably already believe most of the important things I’m about to say. And you might find the unimportant things to be infuriating.
A lot of other people believe that AI safety is a huge waste of money. Some people think that’s superhuman AI is a science fiction story, and it’s not likely to come to pass for a long time, if ever. Some people think that superhuman AI is definitely going to happen, only there’s nothing we can do about it. Those are both reasons to think that AI safety is not a good use of money.
This is an essay for the critics and the skeptics.
I’m one of the skeptics. AI safety seems like it has a lot of potential to waste money, effort, and talent. A lot of people might worry about it, and that might not make any difference in the world at all.
AI safety is ultimately motivated by fear of the unknown. Artificial intelligence might overtake humanity in intelligence or creative capacity. And this might happen sooner, or later, at some unknown point in the future. This is an apocalyptic fear. The fear of death. Humans do not always make good decisions in the face of such fear. Salesman know this. Scams and poor investments are often sold on the back of fear. Social movements can make bad decisions because of fear. So I don’t have the assumption that it’s a good idea.
I’m an expert in artificial intelligence. I do what AI safety people call “capabilities” research. That means I try to make smarter and better AI. The implication is that smarter AI is bad. It means the AI apocalypse is only going to happen sooner. AI safety people often tell me my work is bad and I should stop doing it. I hate it when they say that! It’s annoying and it makes me feel bad.
But I’m an expert. It’s what I do for work. But more than that, it’s the subject that’s occupied my mind on long walks and late nights for many years. The AI safety research that I hear about doesn’t sound like it’s going to work. Some of it might, but I’m skeptical. I don’t think modern machine learning fully understands the breadth of human intelligence. And because of that, AI safety researchers don’t either. I don’t want to go into too much detail here, but I have technical concerns about the details of AI alignment plans.
Even if AI alignment succeeds on technical merits, it still might fail. Artificial intelligence exists in a competitive world. Even an AI which is aligned with the goals of its user or creator might still be very harmful. AI safety researchers are thinking about this. But as far as I know, there are no promising solutions to the thorny problems of geopolitics.
So AI safety research seems poorly motivated. It might not work on the technical merits. It might not work socially or politically. And AI safety researchers are real buzzkills about cool AI projects.
I believe all that. But I still think that AI safety is one of the most important issues to work on. I don’t think it gets enough effort.
Why?
I think the stakes are very high. And I think the timelines are short.
I firmly believe that strong artificial intelligence is possible. There’s nothing inherently special about the human mind. We humans are complex mechanisms. There’s a lot to say about how we think. But we’re not magical. AI will match and exceed us in time. The only question is how long will it take.
As an outsider to the field, you might have noticed that the pace of artificial intelligence has been increasing. AI generated art or essays are becoming commonplace now. There is no reason to believe that that trend will slow down.
What will the world be like when computer programs are smarter than people?
Who will be left with a job? People talk about truckers losing out to self-driving trucks. And with AI generated art, there’s a sense that creative work might have to change. I’m a programmer. AI generated code is getting better every day. I’m going to have to change.
Conspiracy theories are alluring because they promise that the world is in someone’s control. But the truth is, the world is run by individuals. Important decisions are made in the same way that unimportant decisions are. They might seem like they’re made by faceless corporations or governments, but decisions are made by people.
An AI accountant might do a better job at accounting than a human. An AI doctor or lawyer also. Business decisions are made the best way that people know how. And in the future, the best way will be to use AI.
In the future, decisions will increasingly not be made by people. They will be made by machines. This is a fundamental shift. Humankind is going to lose control of our destiny.
I don’t know if there’s a way to stop that.
I’ve always wanted to live in a Star Trek future. This fundamental shift might lead us to that good future. But I don’t know.
I do know that AI technology is being developed very quickly right now. I think that means that important decisions are being made right now. Maybe they won’t seem important until later.
But we have to try.
In Defense of AI safety Research
