What is AI?

This post is an excerpt of an article written by Suresh Kumar, who is the Head of AI Solutions at PARC.  

When John McCarthy and team coined the term Artificial Intelligence (AI) in 1955, they itemized some aspects of the term, but did not really put forth a clean definition. So, the term AI has been vague from the start. This article is about removing that vagueness from the definition.

Definition of AI

For a formal definition of AI, I am going to use the one recently put forth by the EU, more specifically, the European Commission’s High-Level Expert Group on AI. The full definition is verbose. While I will close this post with it, let’s start with a pictorial representation included in the definition document and this abbreviated definition:

 Artificial Intelligence (AI) refers to systems that display intelligent behavior by analyzing their environment and taking actions  – with some degree of autonomy – to achieve specific goals.

Pictorial representation of the EU’s AI definition

Back to Intelligence

While we now have an AI definition we can use, we still need to agree on what is intelligence and what is not, so that we can look at a computer system or a machine (e.g., robot) and tell if it’s using Artificial Intelligence or not.

Defining ‘Intelligence’ is a monumental task. Despite these complications with defining intelligence, the EU discusses the notion of rationality. The ability to be rational is a significant component of intelligence. We could therefore use rational behavior capability as a measure to determine if a system exhibits intelligence.

Rational behavior is about (a) obtaining a goal to achieve and information about the current environment, (b) using the knowledge we have at our disposal, and (c) having the ability to choose the best course of action to achieve that goal. Note that best course of action implicitly means that there could be multiple courses of action, and we are using our knowledge to pick the best one.

Rationality: Goal → Knowledge → Best course of action 

On Knowledge

AI is about imparting machines with knowledge so that, given a goal, the machine can use it to make a rational decision by itself. A ‘rational decision’ is the best course of action given the current state of the surroundings or the problem. The key here is the knowledge. How does one give a machine knowledge? There are two predominant techniques to doing that. Each technique is a box inside the bigger AI box in the EU’s definition.

Two ways to impart knowledge to a machine:
The two techniques are called Machine Learning and Symbolic AI (this is labeled ‘Reasoning’ in the image above). Machine Learning is also called Subsymbolic AI, Connectionist AI, or Statistical Learning. Machine Learning is exceptionally popular now, and rightfully so , as it has been stunningly effective in building solutions this decade.

Symbolic AI (Reasoning)

As the name indicates, in the Symbolic AI approach, we give machines knowledge by hand-crafting it in some symbolic form, like text. There are a few ways to do this, as demonstrated in the illustrations below.

Knowledge Representation & Reasoning Form of AI

In the example below, we have hand-crafted some knowledge.

Now let’s say, we have some information about David as shown below.

Now, given this information, our goal is to answer some questions about David.

Is David over 18 years old?
Can I serve hamburgers to David?

In a Symbolic AI system, the machine will take the input about David, and the goal of answering the questions. Then, it will use the Symbolic hand-crafted knowledge and ‘reason’. The machine can start with the input that David’s a good driver, and then go through the knowledge map. From the hand-crafted knowledge, the machine can see that for him to be a driver, he needs a license, and to have a license he needs to be at least 21 years old. So David is over 18 years old. Similarly, it can reason the answer for the other question as well that I shouldn’t serve hamburgers to David.

Now imagine if we expand the knowledge base to include all the knowledge about the world. Maybe an AI machine (i.e., Watson) can participate in a quiz competition like Jeopardy and beat the all-time human champions? That’s exactly what happened.

Search & Optimize Form of AI

Another way to impart knowledge to a machine is to give it information about a particular environment  – what is allowed, what is not allowed, what is the best outcome towards a goal, etc. This way, the machine is searching through a large number of solutions and picking the optimal one. This is similar to how we humans would play chess. We look ahead to the extent our mind allows. Since the machine is searching through a large number of solutions, this technique is often called Search and Optimize. Optimize comes from the fact that, in general, a search technique will require a way to optimize. In summary, we give the machine a model of ‘chess’ and then we use that model to have the machine make decisions when it is playing chess.

Planning & Scheduling Form of AI

Planning and Scheduling is an extension of Search and Optimize. In Planning and Scheduling, we impart the machine with similar knowledge like we did in Search and Optimize. Except, here the problem scenario requires planning not just one move, but a few steps and scheduling those steps. Scheduling simply means knowing when to execute each step.

Limitations of Symbolic AI

In our example of Knowledge Representation AI, it is very difficult to represent all the knowledge in the world. Even if we are trying to represent a very narrow domain (let’s say car insurance), there is just too much to represent, and there is additional work every time the information changes or new information is added.

In our example of Search and Optimize or Planning and Scheduling, not all problem scenarios will have clear indications on the possible solution space or a possible evaluation approach of potential solutions. These techniques therefore become very challenging to use, if not outright impossible when considering problems such as: Can we build an AI solution to predict the price of houses? There’s no clear way to model a house price like we modeled chess.

Machine Learning (Subsymbolic AI)

Enter Machine Learning, also called Subsymbolic AI, or Statistical Learning, or Connectionist AI. With the availability of large amounts of data and virtually unlimited computing, we are seeing the incredible power of this approach to AI.

Human learning modes

There are two means by which we exhibit our intelligence. One is through accumulated knowledge. We have knowledge that red apples are sweet and green apples are tart. The second is what we have learned from examples or experience. We learned by identifying patterns. We know that apples with brown spots are most likely rotten. We have learned from experience to detect the nature of the spots that indicate rotten apple, versus spots that are just a discoloration of the apple’s skin.

Symbolic AI that we have been talking about so far, is like the accumulated knowledge part. There’s some knowledge and it is used to reason. Machine Learning is like the learning from experience part. Once we have seen enough examples of apples with spots, we know which spots were an indication of rotten apples and which ones were just discolorations. We are then able to generalize in our mind about what spots are most likely signs of rotten apples.

Deep Learning (or Neural Networks)

One of the key reasons for explosion of the Machine Learning approach has been due to the power of Deep Learning. It is loosely modeled after how neurons are connected in a human brain.

A Deep Learning network contains layers and layers of such artificial neurons, much like how a human brain contains interconnected neurons.

Deep neural network with weights in every layer

Deep Learning and Neural Networks are used interchangeably. The accurate distinction is that Deep Learning is a Neural Network with more than one hidden layer. Because of the way a Neural Network is structured, each feature is associated with a very large number of weights, combined with a very large number of other features. This enables the Neural Network to retain more information about the data that it is trained on.

Limitations of Machine Learning Approach: Bias & Explainability

Bias:
Since the Machine Learning approach is looking for patterns in the data, the results are only as good as the data that is used to learn the knowledge. If the data has bias in it, the knowledge will carry that bias. The model’s prediction will hence be biased. There are many examples of bias in Machine Learning models.

Explainability:
The other critical challenge with Machine Learning, specifically Deep Learning is explainability. The mathematical formula gets so incredibly large that it is impossible to tell why the formula is predicting what it’s predicting. There are some other challenges, which are discussed in the article: Deep Learning: A Critical Appraisal. While there are challenges, the results of Deep Learning cannot be overstated. Deep Learning has enabled us to do things we could not have imagined just a few years back.

About Robotics

Robotics is a field that uses both approaches to AI: Symbolic and increasingly, Machine Learning. For example, a robot’s vision sensors may use Machine Learning, and its motion actuators may use Symbolic AI (e.g., Planning and Scheduling). A robot will also include various other techniques outside of the field of AI (e.g. PID controller for physically moving parts of a robot) to integrate it into the physical world. Because of the use of non-AI techniques as well, the robotics box is sticking out of the overall AI box in the definition picture.

Here is the verbal definition of AI from EU’s definition document. While the language below is verbose, it should now make sense.

Formal EU Definition:

“Artificial Intelligence (AI) systems are software (and possibly also hardware) systems designed by humans that, given a complex goal, act in the physical or digital dimension by perceiving their environment through data acquisition, interpreting the collected structured or unstructured data, reasoning on the knowledge, or processing the information, derived from this data and deciding the best action(s) to take to achieve the given goal. AI systems can either use symbolic rules or learn a numeric model, and they can also adapt their behavior by analyzing how the environment is affected by their previous actions.

As a scientific discipline, AI includes several approaches and techniques, such as Machine Learning (of which deep learning and reinforcement learning are specific examples), machine reasoning (which includes planning, scheduling, knowledge representation and reasoning, search, and optimization), and robotics (which includes control, perception, sensors and actuators, as well as the integration of all other techniques into cyber -physical systems).”

Takeaway

The key takeaway is about what the DARPA Perspective on AI defines as the “third wave of AI,” which combines the Symbolic AI and Machine Learning approaches. If you are serious about cutting-edge AI, building hybrid/combination models that combine these two approaches is the future.

Read the entire article