A hyperbolic tangent function is another type of activation function used in Deep Learning, which is a smoother, zero-centred function.
x: an input data point
In terms of sigmoid function:
Range: (-1, 1)
# Import Numpy Library import numpy as np #Hyperbolic Tangent Function def tanh(x): return 2*(sigmoid(2*x)) - 1
Used in multi-layered neural networks.
Mostly appears in recurrent neural networks for natural language processing and speech recognition.
Tanh function gives better results than sigmoid function when dealing with multi-layered neural networks.
Since the range of Tanh function is (-1, 1), the outputs are zero centered and having a higher convergence, the network works on normalized data resulting in faster learning.
Though Tanh is a zero centered function, it still suffers from vanishing gradient during backpropagation.
Produces some dead neurons when the input value is 0.
Hard Hyperbolic function is a variant of Tanh function, which reduces the computational speed thus aids in NLP applications.