Tutorials References Exercises Videos NEW Menu
Paid Courses Website NEW

Perceptrons

A Perceptron is an Artificial Neuron

It is the simplest possible Neural Network

Neural Networks are the building blocks of Artificial Intelligence.

Frank Rosenblatt

Frank Rosenblatt (1928 – 1971) was an American psychologist notable in the field of Artificial Intelligence.

In 1957 he started something really big.

Scientists had discovered that brain cells (Neurons) receive input from our senses by electrical signals.

The Neurons, then again use electrical signals to store information, and to make decisions based on previous input.

Frank had the idea that Artificial Neurons could simulate brain principles, with the ability to learn and make decisions.

From these thoughts, he "invented" the Perceptron.

The Perceptron was tested on an IBM 704 computer at Cornell Aeronautical Laboratory in 1957.


The Perceptron

The original Perceptron was designed to take a number of binary inputs, and produce one binary output (0 or 1).

The idea was to use different weights to represent the importance of each input, and that the sum of the values should be greater than a threshold value before making a decision like true or false (0 or 1).

Perceptron


Perceptron Example

Imagine a perceptron (in your brain).

The perceptron tries to decide if you should go to a concert.

Is the artist good? Is the weather good?

What weights should these facts have?

CriteriaInputWeight
Artists is Goodx1 = 0 or 1w1 = 0.7
Weather is Goodx2 = 0 or 1w2 = 0.6
Friend Will Comex3 = 0 or 1w3 = 0.5
Food is Servedx4 = 0 or 1w4 = 0.3
Alcohol is Servedx5 = 0 or 1w5 = 0.4

The Perceptron Algorithm

Frank Rosenblatt suggested this algorithm:

  1. Set a threshold value
  2. Multiply all inputs with its weights
  3. Sum all the results
  4. Activate the output

1. Set a threshold value:

  • Threshold = 1.5

2. Multiply all inputs with its weights:

  • x1 * w1 = 1 * 0.7 = 0.7
  • x2 * w2 = 0 * 0.6 = 0
  • x3 * w3 = 1 * 0.5 = 0.5
  • x4 * w4 = 0 * 0.3 = 0
  • x5 * w5 = 1 * 0.4 = 0.4

3. Sum all the results:

  • 0.7 + 0 + 0.5 + 0 + 0.4 = 1.6 (The Weighted Sum)

4. Activate the Output:

  • Return true if the sum > 1.5 ("Yes I will go to the Concert")

If the treshold value is 1.5 for you, it might be different for someone else.

Example

const treshold = 1.5;
const inputs = [1, 0, 1, 0, 1];
const weights = [0.7, 0.6, 0.5, 0.3, 0.4];

let sum = 0;
for (let i = 0; i < inputs.length; i++) {
  sum += inputs[i] * weights[i];
}

const activate = (sum > 1.5);

Try it Yourself »


Perceptron Terminology

  • Perceptron Inputs
  • Node values
  • Node Weights
  • Activation Function

Perceptron Inputs

Perceptron inputs are called nodes.

The nodes have both a value and a weight.


Node Values

In the example above the node values are: 1, 0, 1, 0, 1


Node Weights

Weights shows the strength of each node.

In the example above the node weights are: 0.7, 0.6, 0.5, 0.3, 0.4


The Activation Function

The activation functions maps the result (the weighted sum) into a required value like 0 or 1.

The binary output (0 or 1) can be interpreted as (no or yes) or (false or true).

In the example above, the activation function is simple: (sum > 1.5)

In Neuroscience, there is a debate if single-neuron encoding or distributed encoding is most relevant for understanding how the brain functions.

It is obvious that a decision like the one above, is not made by one neuron alone.

At least there must be other neurons deciding if the artist is good, if the weather is good...

Neural Networks

The Perceptron defines the first step into Neural Networks.

The perceptron is a Single-Layer Neural Network.

The Neural Network is a Multi-Layer Perceptron.