IHTAI: Integrated Hierarchical-Temporal Artificial Intelligence

Create Virtual Minds in Your Browser

What Does it Do?

IHTAI is an unsupervised online machine learning library which facilitates the development of virtual agents that act based on a combination of preprogrammed reflex behaviors, ideal drive states which the agent constantly strives to better reach, and subjective stimuli experience. Learning is a continuous process, allowing for quick adaptation to changes in environment. Instead of supplying an agent with prior data like in most machine learning approaches, an IHTAI agent continually collects its own prior data in a manner similar to a biological organism.

One can think of IHTAI as body-agnostic. For a developer, this means that you program an agent's drives, set ideal values for those drives, and IHTAI will develop emergent behaviors on its own to best reach those drive goals.

To see an example of IHTAI in action, visit http://chrisnatale.info/projects/ihtai/physicsdemo.html (it uses PhysicsJS for the physics simulation). The circle's ability to move is controlled by an IHTAI instance, and it 'feeds' by getting within a certain range of the squares that are generated. When the square is out of range, the agent's hunger drive increases. However, by moving, the agent's tiredness drive increases. Our IHTAI agent is instantiated with a goal of achieving both a hunger and tiredness drive value of 0 as often as possible. However, its default reflex behavior only allows for it to constantly move towards the square, stopping when its tiredness drive reaches 100 and it falls asleep. This will not allow it to achieve anything close to an ideal drive state.

IHTAI's consciousness module steps in and allows the agent to develop new behaviors on top of its existing reflexes. These new behaviors allow the agent to much more effectively approach its ideal drive states. Learned behaviors for specific situations replace reflexes, improve with time, and dynamically adapt to changing circumstances.

Although the IHTAI agent starts out behaving in an ungainly manner (think of it as a newborn baby), after a few minutes the behaviors become more controlled and quickly overtake the effectiveness of the agent's reflexes. Within 15 minutes, the drive satiating behaviors far exceed reflex performance.

Update(7/20/15): I've also started a YouTube channel showcasing experiments performed with IHTAI, available at https://youtu.be/6cpIdHlsQEY?list=PLvKPARrybGuJtQ7nXthuTvhda1FTn_fQF.

Take a look at the Getting Started section for code samples. Also visit the library documentation. What follows is an explanation of the main theoretical components of IHTAI.

Clusters

Upon instantiation, an IHTAI agent is given a set number of experience clusters that it can hold. An experience cluster contains a vector that represents a specific combination of Drive and Input/Output Stimuli (DIOS) for a moment in time. As the agent encounters unique combinations of DIOS, clusters are added to its memory. These clusters are accessed quickly through hashing.

Once the agent reaches its maximum allotment of clusters, a k-d tree is built to allow for quick nearest-neighbor searches of available cluster DIOS vectors. From this point on, any DIOS that doesn't directly match an existing cluster instead activates the nearest neighbor cluster. In this way, DIOS combinations can be stored efficiently and accessed quickly.

The Memorizer

IHTAI's memorizer uses clusters to build a dynamic temporal model that answers the question: "what is the most useful action that an agent should take when presented with a specific DIOS?"

For a unique experience (as represented by a cluster), the memorizer builds a hierarchical temporal list of possible actions, and how close these actions have gotten the agent to its ideal drive state. For example, the agent will know that three cycles from now, its best known behavior in this situation resulted in its drive values being a certain distance from its ideal drive values. Four cycles from now, its best possible action will result in another distance from the ideal drive state, which will also be stored. This continues until the maximum specified memory distance is reached.

The end state with the lowest score (closest to the ideal state) for a particular stimuli combination is selected. The action an agent carried out after its current state on the path to this lowest score DIOS is returned. Thus, an IHTAI agent is able to act in a way that, based on its experience, will most likely lead to the lowest possible drive state. This information is stored in a min-heap keyed on scores, allowing for quick access.

Imagination

Without something to cause an agent to attempt new output drive states beyond what it learns from reflexes, behaviors would be quite limited. An imagination algorithm (currently named 'daydream') accomplishes this. It works by selecting a random cluster and passing this into the memorizer, instead of the actual cluster that would be selected based on the agent's sensory experience. If the random cluster hasn't been acted on in this situation before, or if its memorizer.query() square distance response is less than the actual sensory experience counterpart, the agent tries the new behavior.

IHTAI is an attempt to model artificial intelligence as a continuous process of becoming. This is a very different way of conceptualizing intelligent agents than found in mainstream thought. Typically, agents are conceived as atomic actors with static traits. In contrast, IHTAI agents are created with drives that, when combined with sensory input, continually shape and reshape their behaviors over time. An IHTAI agent's nature is inherently fluid and adaptive.

The theory behind IHTAI comes from two main sources of inspiration. One is the Critical Realist school of philosophy as described by Roy Bhaskar and particularly as interpreted by David Graeber in his writings on anthropological value theory. The other is a hierarchical conception of human intelligence presented by Ray Kurzweil in How to Create a Mind, and Jeff Hawkins' somewhat similar views from On Intelligence.

Critical Realism posits that there is a real world which exists outside the minds of individuals, regardless of whether anyone observes it. However, the observable phenomena of the real is actually produced through underlying hierarchically ordered forces, which exist across observable and articulable facts describing surface phenomena.

For example, chemistry relies upon, but is not reducible to, Newtonian physics. In the same manner, psychology relies upon both chemistry and physics, but cannot be reduced to either. A higher layer is emergent from a lower. This does not prevent forces operating at a higher level from affecting a lower one, though. For example, a human being can use its intelligence to design an earth mover which changes the environment in a way described by physics. However, these changes(dependent as they are on human psychology) could never be predicted by physics.

In this way, Critical Realism opens the door to imagining how complex beings can both influence and be influenced by the world around them.

The hierarchical theory of intelligence assumes that concepts in a human mind are referentially built out of simpler concepts. For example, a visual representation of the letter 'a' is actually stored as a combination of lines and curves, parts of which are re-used to create many combinations of similar shapes. The curves in 'b' and 'p' would both be represented by the same composite concept in human memory.

As an agent experiences new stimuli, these stimuli become part of more abstract concepts. The letter 'a' may come to be a component of the word 'apple', which may become a component of the even more abstract notion of 'food'.

IHTAI models this type of hierarchical referential intelligence in one dimension: time. The theory is that taxonomies can be represented in the mind through stimuli playback in a temporal manner. For example, an agent could describe the stimuli components that make up a fruit bowl and what they would expect to appear on a supermarket produce shelf by replaying the same series of stimuli. These longer temporal chains could then become part of even longer temporal chains, all while referencing a fixed number of stimuli combinations. Only focusing on temporal referentiality also greatly simplifies the hierarchical pattern matching algorithms, and makes it possible for them to run quickly even in a relatively slow interpreted language like JavaScript.

I should note that the actual implementation of hierarchical intelligence in IHTAI is quite different from the one argued for by Ray Kurzweil. IHTAI uses a kd-tree and minheap as the core structures to organize hierarchical temporal knowledge out of candidate stimuli, due to the fact that they are fast enough to integrate new data into new behavior in real-time. Kurzweil conceives of an approach using Hidden Markov Models (HMMs). While HMMs have many benefits, they do not allow for the real-time incorporation of new stimuli into an agent's behaviors(save for a min-batch approach, which ignores the vast majority of data in memory). I view this ability as vital to creating agents that are able to more effectively navigate the infinite vagaries of reality.