Intelligent agent

For the term in intelligent design, see intelligent designer.

In artificial intelligence, an intelligent agent (IA) is an autonomous entity which observes through sensors and acts upon an environment using actuators (i.e. it is an agent) and directs its activity towards achieving goals (i.e. it is "rational", as defined in economics). Intelligent agents may also learn or use knowledge to achieve their goals. They may be very simple or very complex: a reflex machine such as a thermostat is an intelligent agent, as is a human being, as is a community of human beings working together towards a goal.

Intelligent agents are often described schematically as an abstract functional system similar to a computer program. For this reason, intelligent agents are sometimes called abstract intelligent agents (AIA) to distinguish them from their real world implementations as computer systems, biological systems, or organizations. Some definitions of intelligent agents emphasize their autonomy, and so prefer the term autonomous intelligent agents. Still others (notably Russell & Norvig (2003)) considered goal-directed behavior as the essence of intelligence and so prefer a term borrowed from economics, "rational agent".

Intelligent agents in artificial intelligence are closely related to agents in economics, and versions of the intelligent agent paradigm are studied in cognitive science, ethics, the philosophy of practical reason, as well as in many interdisciplinary socio-cognitive modeling and computer social simulations. Intelligent agents are also closely related to software agents (an autonomous computer program that carries out tasks on behalf of users). In computer science, the term intelligent agent may be used to refer to a software agent that has some intelligence, regardless if it is not a rational agent by Russell and Norvig's definition. For example, autonomous programs used for operator assistance or data mining (sometimes referred to as bots) are also called "intelligent agents".

1 A variety of definitions

Intelligent agents have been defined many different ways. According to Nikola Kasabov IA systems should exhibit the following characteristics:

• Accommodate new problem solving rules incrementally.
• Adapt online and in real time.
• Are able to analyze itself in terms of behavior, error and success.
• Learn and improve through interaction with the environment (embodiment).
• Learn quickly from large amounts of data.
• Have memory-based exemplar storage and retrieval capacities.
• Have parameters to represent short and long term memory, age, forgetting, etc.

2 Structure of agents

A simple agent program can be defined mathematically as an agent function which maps every possible percepts sequence to a possible action the agent can perform or to a coefficient, feedback element, function or constant that affects eventual actions: f: P → A.

Agent function is an abstract concept as it could incorporate various principles of decision making like calculation of utility of individual options, deduction over logic rules, fuzzy logic, etc.

The program agent, instead, maps every possible percept to an action. We use the term percept to refer to the agent's perceptional inputs at any given instant. In the following figures an agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.

3 Classes of intelligent agents

Russell & Norvig (2003) group agents into five classes based on their degree of perceived intelligence and capability:

1. simple reflex agents
2. model-based reflex agents
3. goal-based agents
4. utility-based agents
5. learning agents

3.1 Simple reflex agents

graph LR
    subgraph Agent
        direction TB
        Sensors[Sensors]
        State[What the world
is like now]
        Rules([Condition-action rules])
        Action[What action I
should do now]
        Actuators[Actuators]
        Sensors --> State
        State --> Action
        Rules --> Action
        Action --> Actuators
    end
    Environment([Environment])
    Environment -- Precepts --> Sensors
    Actuators -- Actions --> Environment
    style Agent fill:#ffffff,stroke:#000,stroke-width:3px,color:#000
    style Environment fill:#ffffff,stroke:#000,stroke-width:3px,color:#000
    style State fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Action fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Rules fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Sensors fill:none,stroke:none,color:#000
    style Actuators fill:none,stroke:none,color:#000
        

Simple reflex agent

Simple reflex agents act only on the basis of the current percept, ignoring the rest of the percept history. The agent function is based on the condition-action rule: if condition then action.

This agent function only succeeds when the environment is fully observable. Some reflex agents can also contain information on their current state which allows them to disregard conditions whose actuators are already triggered.

Infinite loops are often unavoidable for simple reflex agents operating in partially observable environments. Note: If the agent can randomize its actions, it may be possible to escape from infinite loops.

3.2 Model-based reflex agents

graph LR
    subgraph Agent
        direction TB
        Sensors[Sensors]
        State([State])
        Evolves([How the world evolves])
        ActionsDo([What my actions do])
        WorldNow[What the world
is like now]
        Rules([Condition-action rules])
        Action[What action I
should do now]
        Actuators[Actuators]
        Sensors --> WorldNow
        Sensors -.-> State
        State --> WorldNow
        Evolves --> WorldNow
        ActionsDo --> WorldNow
        WorldNow --> Action
        Rules --> Action
        Action --> Actuators
    end
    Environment([Environment])
    Environment -- Precepts --> Sensors
    Actuators -- Actions --> Environment
    style Agent fill:#ffffff,stroke:#000,stroke-width:3px,color:#000
    style Environment fill:#ffffff,stroke:#000,stroke-width:3px,color:#000
    style State fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Evolves fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style ActionsDo fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Rules fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style WorldNow fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Action fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Sensors fill:none,stroke:none,color:#000
    style Actuators fill:none,stroke:none,color:#000
        

Model-based reflex agent

A model-based agent can handle a partially observable environment. Its current state is stored inside the agent maintaining some kind of structure which describes the part of the world which cannot be seen. This knowledge about "how the world works" is called a model of the world, hence the name "model-based agent".

A model-based reflex agent should maintain some sort of internal model that depends on the percept history and thereby reflects at least some of the unobserved aspects of the current state. It then chooses an action in the same way as reflex agent.

3.3 Goal-based agents

graph LR
    subgraph Agent
        direction TB
        Sensors[Sensors]
        State([State])
        Evolves([How the world evolves])
        ActionsDo([What my actions do])
        WorldNow[What the world
is like now]
        ActionA[What it will be like if
I do action A]
        Goals([Goals])
        ActionNow[What action I
should do now]
        Actuators[Actuators]
        Sensors --> WorldNow
        Sensors -.-> State
        State --> WorldNow
        Evolves --> WorldNow
        ActionsDo --> WorldNow
        Evolves --> ActionA
        ActionsDo --> ActionA
        WorldNow --> ActionA
        Goals --> ActionNow
        ActionA --> ActionNow
        ActionNow --> Actuators
    end
    Environment([Environment])
    Environment -- Precepts --> Sensors
    Actuators -- Actions --> Environment
    style Agent fill:#ffffff,stroke:#000,stroke-width:3px,color:#000
    style Environment fill:#ffffff,stroke:#000,stroke-width:3px,color:#000
    style State fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Evolves fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style ActionsDo fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Goals fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style WorldNow fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style ActionA fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style ActionNow fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Sensors fill:none,stroke:none,color:#000
    style Actuators fill:none,stroke:none,color:#000
        

Model-based, goal-based agent

Goal-based agents further expand on the capabilities of the model-based agents, by using "goal" information. Goal information describes situations that are desirable. This allows the agent a way to choose among multiple possibilities, selecting the one which reaches a goal state.

Search and planning are the subfields of artificial intelligence devoted to finding action sequences that achieve the agent's goals.

In some instances the goal-based agent appears to be less efficient; it is more flexible because the knowledge that supports its decisions is represented explicitly and can be modified.

3.4 Utility-based agents

graph LR
    subgraph Agent
        direction TB
        Sensors[Sensors]
        State([State])
        Evolves([How the world evolves])
        ActionsDo([What my actions do])
        WorldNow[What the world
is like now]
        ActionA[What it will be like if
I do action A]
        Utility([Utility])
        Happy[How happy I will be
in such a state]
        ActionNow[What action I
should do now]
        Actuators[Actuators]
        Sensors --> WorldNow
        Sensors -.-> State
        State --> WorldNow
        Evolves --> WorldNow
        ActionsDo --> WorldNow
        Evolves --> ActionA
        ActionsDo --> ActionA
        WorldNow --> ActionA
        Utility --> Happy
        ActionA --> Happy
        Happy --> ActionNow
        ActionNow --> Actuators
    end
    Environment([Environment])
    Environment -- Precepts --> Sensors
    Actuators -- Actions --> Environment
    style Agent fill:#ffffff,stroke:#000,stroke-width:3px,color:#000
    style Environment fill:#ffffff,stroke:#000,stroke-width:3px,color:#000
    style State fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Evolves fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style ActionsDo fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Utility fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style WorldNow fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style ActionA fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Happy fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style ActionNow fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style Sensors fill:none,stroke:none,color:#000
    style Actuators fill:none,stroke:none,color:#000
        

Model-based, utility-based agent

Goal-based agents only distinguish between goal states and non-goal states. It is possible to define a measure of how desirable a particular state is. This measure can be obtained through the use of a utility function which maps a state to a measure of the utility of the state. A more general performance measure should allow a comparison of different world states according to exactly how happy they would make the agent. The term utility can be used to describe how "happy" the agent is.

A rational utility-based agent chooses the action that maximizes the expected utility of the action outcomes—that is, what the agent expects to derive, on average, given the probabilities and utilities of each outcome. A utility-based agent has to model and keep track of its environment, tasks that have involved a great deal of research on perception, representation, reasoning, and learning.

3.5 Learning agents

graph LR
    subgraph AGENT
        direction TB
        Sensors[Sensors]
        PerfStd[Performance
Standard]
        Critic[Critic]
        LearnElem[Learning
element]
        PerfElem[Performance
element]
        ProbGen[Problem
Generator]
        Effectors[Effectors]
        PerfStd --> Critic
        Sensors --> Critic
        Sensors --> PerfElem
        Critic -- feedback --> LearnElem
        LearnElem -- changes --> PerfElem
        PerfElem -- knowledge --> LearnElem
        LearnElem -- learning goals --> ProbGen
        ProbGen -- experiments --> PerfElem
        PerfElem --> Effectors
    end
    Environment([ENVIRONMENT])
    Environment -- percepts --> Sensors
    Effectors -- actions --> Environment
    style AGENT fill:#ffffff,stroke:#000,stroke-width:3px,color:#000
    style Environment fill:#ffffff,stroke:#000,stroke-width:3px,color:#000
    style Critic fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style LearnElem fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style PerfElem fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style ProbGen fill:#ffffff,stroke:#000,stroke-width:2px,color:#000
    style PerfStd fill:none,stroke:none,color:#000
    style Sensors fill:none,stroke:none,color:#000
    style Effectors fill:none,stroke:none,color:#000
        

A general learning agent

Learning has an advantage that it allows the agents to initially operate in unknown environments and to become more competent than its initial knowledge alone might allow. The most important distinction is between the "learning element", which is responsible for making improvements, and the "performance element", which is responsible for selecting external actions.

The learning element uses feedback from the "critic" on how the agent is doing and determines how the performance element should be modified to do better in the future. The performance element is what we have previously considered to be the entire agent: it takes in percepts and decides on actions.

The last component of the learning agent is the "problem generator". It is responsible for suggesting actions that will lead to new and informative experiences.