Broad claim: There is not just one central problem of intelligence, not just one master algorithm. Instead, intelligence consists of solving distinct types of problems using distinct algorithms. By analyzing what types of problems are composite to intelligence, we can find algorithms that match those problems, and by composing those algorithms properly, we can construct a model which solves the full range of problems that humans solve. 

Increasing Functional Scope in AI

I propose a basic divide in thinking about intelligence. There are three types of AI:

1. Narrow AI. Can solve some very specific problem. For example, Deep Blue. 
2. Maximizers or Minimizers. Can solve a fairly broad class of problems, but they require all of those problems to be posed in the same way. For example, backpropagation for function approximation, or AlphaZero for board games with known rules. 
3. Composite AI. Can combine multiple narrow AI algorithms or broader maximizers to tackle complex or poorly posed problems. For example, self driving car software or Google search backend. 

Can All Problems be Posed the Same Way?

I find that when neuroscientists and ML practitioners talk about intelligence, they often claim that intelligence is about one type of problem, broadly solved. Consider these claims (all of which I have heard suggested by serious researchers):

1. Intelligence is about making accurate predictions
2. Intelligence is about maximizing reward
3. Intelligence is about compressing information
4. Intelligence is about constructing a probabilistic world model
5. Intelligence is about pathfinding in arbitrary spaces
6. Intelligence is about minimizing free energy
7. Intelligence is about applying narrow AIs in just the right way, like a conductor

Are any of these correct? Perhaps one of them subsumes all the others. Think about what you believe. Do you think that if we could perfectly solve one of these problems, that the others would be trivial? 

Oracles of AI

In computer science, the term “oracle” is sometimes used to describe a black box function which can do some unusual piece of computation, and which can be invoked by a larger algorithm. It allows you to ask questions like “if I had an oracle that could sort lists in constant time, could I use that to find a path through a graph faster?” It also allows you to ask questions about noncomputable functions, like the halting problem.

I want to extend this concept of an oracle to artificial intelligence by imagining a composite AI that has access to smaller functions which can answer any question to a high degree of correctness, so long as the question is posed in the right way. 

I use the term oracle to emphasize that we should view them as idealized black boxes whose internals are somewhat mysterious. For now it is not important to ask how they work, but instead ask what they do. 

Here are some examples:

1. Function Approximation: Given the inputs to a function, what output minimizes prediction error? A common function would be predicting the next state of the world. 
2. Reward Maximization: Given an agent state, select an action that maximizes a reward signal. 
3. Relevant Memory Selection: Given a ton of memories about the world, select just the information that will be most useful right now. 
4. Pathfinding. Given a graph, find the shortest path from point A to point B.

This is not a complete list. What other sorts of oracle might it be useful for a composite AI to contain?

Limitations

I want to emphasize that a black box like this might be useful even if it is not 100% accurate. A reward maximizer might still be very good at games even if its play is sometimes suboptimal, and so on.

It’s also helpful to note that an oracle AI might have limitations on its input and output. For example, it might only be able to work with images below a certain resolution. Or it might be able to say what next move is optimal, but not produce an entire decision tree (because that would be too big).

No One Oracle Can Do Everything

Question: If you had one of those oracles that you could question at will, could you solve all the others? 

Perhaps you think that some pairs exist such that you could use one oracle to easily solve another one. For example, if you can use error minimization to predict future reward given a state and action pair, then you can maximize reward. This is called Q learning, and it does a decent job with many RL style problems. 

Or perhaps you think the opposite, that knowing things is just a subset of reward maximizing action selection, so a reward-maximizer can tell you the right answers because the answer that minimizes error is also the answer that maximizes reward. Maybe you can think of a politician who has this attitude to the truth. 

Certainly some people have expressed to me that one of these oracles can do everything. I think that’s a seductive idea because it suggests that all of AI can be solved by finding the perfect answer to one well-defined set of questions. 

Answer: No, no one oracle can do everything that all of the others can do. 

This is because of practical concerns about our ability to invoke oracles at will. Think back to the way we defined these oracles in the first place; they answer questions to some degree of correctness given a well posed input. They are limited in their bandwidth both for input and output, and suffer increases in training time and training data requirements as we broaden their range and domain. They are also limited in the sense that while they can be invoked “at will”, they will take non-zero time to use, and thus suffer from exponential time costs when exploring a space (for example when using Q learning over a high dimensional action space). 

I think this is clear when you look at modern ML systems like GPT-3, which are primarily focused on error-minimization. It is true that GPT3 can play chess, and it does so because “predicting the move that an expert would make” is a fairly good proxy for the question “pick the best next move”. But it’s clearly not the same, and it plays differently as a result. 

Example Decompositions

Let’s take a look at some intelligent systems to examine how they could be broken up using this sort of functional decomposition.

In the transformer architecture, such as BERT, GPT3 or LaMDA, the algorithm essentially contains the functions of world prediction and of relevant data selection. The KQA attention mechanism surfaces relevant information, and then a feedforward network performs next token prediction. These two elements are repeated in series to form the full transformer.

Reinforcement learning can be implemented for simple environments with a relatively straightforward Deep Q Network, which uses a NN to approximate the Q function, which approximates a solution to the Bellman equation. The NN performs function approximation, which is doing one sort of intelligence, but it does so within the superstructure of the Bellman equation, which promotes reward seeking behavior.

An example of a more complex RL algorithm might be AlphaStar, the algorithm which learns to play Starcraft. In addition to the elements that exist with a DQN, it also maintains a probabilistic world model.

It may seem strange to split up these algorithms in this way. It might seem unnecessary or unhelpful or even like it makes understanding more confusing. I guess all I can say is that it is helpful for me to look at algorithms this way, as a way of asking what intelligent work needs to occur within the AI.

What are the Functional Components of Human Intelligence?

Humans implement a very general intelligence that contains many different oracles to perform different functions internally. It would obviously be of great interest to the development of AGI to be able to give a full list of functions that humans implement together to form their intelligence. But unfortunately that full list will need to wait for a future post. For now, let me point out some ways in which human intelligence appears to be performing quite different functions simultaneously.

A human looks into an old house and sees a long table with a door beyond. That’s perception. Now they turn off their flashlight and walk blindly into the room, successfully navigating the table. That’s maintenance of a world model. Maybe it’s also pathfinding. They think back to the last time they were here and recall the location of the lightswitch. That’s a relevancy process filtering over a large set of memories. They move their arm to flip the switch. That’s inverse kinematics.

A person becomes very depressed. They stop going to work, they can barely get out of bed. And yet, they remember facts about the world, they make accurate predictions about what will happen. Why? In functional terms, they are failing to effectively perform the function of reward maximization, even though other functions of intelligence are working fine.

By investigating situations like this, we can come up with a list of functional oracles that we might attribute to human cognition.

Next

In the next entry in this series, I’ll try to outline a way to do this sort of functional decomposition on human intelligence in a way that gets a list of components.

Later in the series, I’ll talk about how we can think about these functions in the context of their environment. Then I’ll talk about composition of functions and how they can work together, because it’s not as simple as having totally separate modules. Lastly, I’ll talk about the implications for AI alignment, and how we can “defeat AI in detail” by finding an alignment solution for each of these functions.