# Activation function

An activation function $a: \mathbb{R} \rightarrow \mathbb{R}$ gets applied in a perceptron after the weighted sum of the inputs. It is the non-linear term. Classic activation functions are

• Identity function,
• Binary step ,
• Sigmoid function ,
• Rectified linear unit (ReLU) , or
• Hyperbolic tangent (tanh) .