A theme of this blog is that intelligent behavior of humans, animals, and AI systems is made up of mechanisms that we can understand individually. It’s commonly understood that the mechanisms of artificial intelligence mirror the functionality that we find in biological behavioral systems like the brain, but I find that computer scientists underestimate the sheer diversity of mechanisms that occur in biological systems.

AI models like ChatGPT or AlphaGo are very large, but they often consist of a few patterns repeated at scale. For example, AlphaGo contains a convolutional NN plus lookahead tree search.

👁️ I learned something in a freshman biology class in college that has stuck with me ever since. The human eye has seven different mechanisms for adjusting to varying light levels in the environment. Seven! Why does such a simple behavioral response have such a complicated implementation?

Let’s look at something simple in the human eye, and we’ll see that what seems like a very simple response is actually driven by a large number of distinct mechanisms. As we go, we’ll ask what we can learn that may be useful in ML.

Human Eye in Light and Dark

The human eye is typically capable of perceiving the world in bright daylight and, to a lesser extent, on a dark moonless night. Our ability to handle brightness spans about nine orders of magnitude, although at any given time we can only perceive about three orders of magnitude of brightness. (source: wikipedia)

You may think that the workings of the eye are of interest only to biologists. But I think it contains lessons for engineers. As you read through this list, I encourage you to notice how different these mechanisms are from each other, and the implications of that for design.

I need to add a caveat: the ennumeration of these is a little bit arbitrary. You could split them up in other ways, and this is a bit of a simplification. Still, I hope you find it interesting. You don’t need to follow all the details.

The most visible response to the eye is the (1) dilation of the pupils, which physically allows more light to enter the eye. This adjustment allows for 10 times as much light to enter the eye, a small fraction of the overall dynamic range. From this we learn that the obvious solution is not always enough.

The retina, on the back of the eyeball, contains (2) both rods and cones, two different types of light-responsive cell. Although both types of receptor are always active, the rods are specialized for night vision, while the cones are specialized for vision in daylight. Only the cones are capable of seeing color, which is why the world looks greyscale at night. This mechanism is entirely passive, as it relies on two ever-present sets of sensors, which is markedly different from the other mechanisms on this list.

Within the light receptor, we get to a more complicated mechanism for adjusting sensitivity. Photons are detected by photoreceptor chemicals, like 11-cis-retinal, which changes shape when hit by a photon into all-trans-retinal. That change in shape is detected by a protein, causing a chemical cascade which results in neural activity in the eye. But, the chemical all-trans-retinal is left over, and another protein is needed to change it back.

This brings us to the next mechanism for adapting to differing brightness. In the dark, rods and cones want to keep up a (3) high concentration of photoreceptive chemicals and proteins, while in the light, they can get by with a lower concentration. Not only is a lower concentration cheaper for the cell, it prevents the cell from saturating its sensitivity.

The rod and cone cells are at the back of your eye, but they don’t connect directly to your optic nerve. There are several layers of other cells in between. Here’s a picture, and don’t sweat the details:

The intermediate cells, labeled bipolar and ganglion cells in the diagram, connect to each other with synapses which exhibit (4) synaptic plasticity, which can create a negative feedback loop over time, where a high rate of activity reduces the strength of the synapse, which will reduce the strength of future inputs to the cell, which reduces the activity. And conversely, if activity is low, the synapse will strengthen. The result is that the average firing rate stays within acceptable bounds due to this competition between adjacent neurons.

Individual neurons are able to track a rolling average of their own activity, and they use that internal average to do a (5) normalization of firing rate. This adaptation occurs over a timescale of milliseconds to tens of minutes, meaning that it’s relevant for a human adapting to the dark.

This normalization is ultimately the goal of all of these mechanisms. If the light sensor is never active, the animal (e.g. the person) is effectively blind. But if the light sensor is always saturated, the animal will only see white, also effectively blinding them. This entire mechanism is about staying somewhere in the middle.

There are also connections between these cells which carry information forward. In particular, these cells inhibit each other, meaning that when a cell is active, it deactivates its neighbors. This (6) lateral inhibition is responsible for the grid illusion and other brightness related illusions:

The white crosses between the squares appear darkened because of this lateral inhibition. They’re surrounded by white pixels, so the ganglion cells which represent them are inhibited by all of their highly active neighbors. This lateral inhibition helps prevent a bright day from overwhelming the eye, but it also increases the contrast at edges, which assists in processing the image.

Finally, the visual system exhibits significant (7) top down normalization. The visual cortex, which does most visual processing, sends dense feedback projections to earlier stops in the visual system. After leaving the eye, information flows through the lateral geniculate nucleus (of the thalamus), before reaching the primary visual cortex. But in fact, feedback connections from the visual cortex to the LGN actually outnumber feedforward connections, indicating that this top-down normalization is quite strong.

Although I’ve enumerated these mechanisms, the lines between them aren’t always clear. For example, number 3 encompasses many different biochemical processes within the cell that I’ve lumped together because I’m not a biochemist, but in reality there is a lot of complexity there. And number 6, lateral inhibition, is something that occurs all throughout the visual processing system, not just in ganglion cells.

Implications

There are so many different mechanisms for one small behavior! The human brain does so much more than respond to changes in visual light level. How can we make sense of all the mechanisms that must be doing those other things?

I find it most useful to mentally organize these mechanisms functionally, focusing on what they accomplish rather than how exactly how they function. From that perspective, each of these mechanisms chooses parameters for the visual system, seeking to maximize the amount of information that is passed along. We can express this behavior mathematically as $\underset{params(👀)}{\mathrm{argmax}}[H(👀)]$; they choose parameters for the visual system which maximize the information content of the visual stream.

By framing these mechanisms in functional terms, we’re able to abstract over their implementation, not just their biological details, but we can also abstract away their algorithmic details. We can ask: are there other mechanisms within the brain that adjust parameters in order to maximize information bandwidth? There are, and in fact the brain is constantly balancing between too much activity, which would result in a seizure, and too little activity, which could result in unconsciousness.

Of course, the algorithmic details are important too. For example, lateral inhibition is implementing a negative feedback loop as the proportional component of a PID controller. We can also note that each of these mechanisms is able to choose parameters independently within their part of the overall system, which tells us that not much information needs to be transmitted on top of the visual information.

Despite implementing the same function, these different mechanisms are mechanically quite different, which provides robustness to the organism. These mechanisms have different timescales, response characteristics, and failure modes. We may find that a similar diversity of mechanism brings robustness to AI safety.

Comparison to Cameras

We can compare this to a modern camera, which also has several mechanisms to adjust to changes in brightness! A camera can change its aperture size, its shutter speed, and digital cameras can change their ISO sensitivity, which controls the sensitivity of photosensitive diodes in the sensor array. These three attributes are referred to as the Exposure Triangle, which can be adjusted manually by the photographer, or automatically by the camera:

Some post-processing can also be done to adjust an image which turns out too dark or too light, giving us about four mechanisms for adjusting light sensitivity in photography.

Of these, aperture has a clear parallel in pupil dilation, while ISO sensitivity has a less clear parallel to photoreceptor density in the retina.

Takeaway

Intelligent systems are behaviorally complex, but behind each individual behavior, there are often many different mechanisms that work together to form an intelligent response.

This post looks at something very simple; the response of a visual system to brightness and darkness. Before reading this, you may have thought that pupil dilation was doing most of the work, although it’s only response for about 1 out of 9 orders of magnitude of sensitivity change. Intelligence, in a human or an ant or a camera, is essentially about adapting to our environment.

It may seem that only simple behaviors can be built up by a collection of simple mechanisms, and that intelligent behaviors are different. But even within the eye, we see learning occurring, such as the negative feedback loop in the synaptic plasticity on ganglion cells. When a mouse learns to avoid the taste of rat poison, that is a behavior which is built up out of smaller mechanisms, which are each easier to understand in isolation.

It is often tempting, when you are engineering a system, to build just one solid mechanism that will do a good job. But take inspiration from biology, and build many mechanisms that work together. In neuroscience, a good explanation is almost always only part of the story.