tanh

Pros: same as sigmoid but the result ranges between (-1, 1).

Cons: vanishing gradient: the inputs that are far from the origin are near zero.

The hyperbolic tangent function, commonly abbreviated as tanh, is a mathematical function that maps input values to a range between -1 and 1. It is often used as an activation function in neural networks, particularly in the hidden layers, due to its symmetric nature and ability to model non-linear relationships. The tanh function is defined as:

$\text{tanh}(x) = \frac{e^{x} - e^{-x}}{e^{x} + e^{-x}}$

Here's a Python implementation of the tanh function:

import numpy as np

def tanh(x):
    """
    Compute the hyperbolic tangent function for an input array x.

    Parameters:
    x : array_like
        Input array.

    Returns:
    ndarray
        Hyperbolic tangent function applied to each element of x.
    """
    return np.tanh(x)

In this implementation, np.tanh(x) computes the hyperbolic tangent of each element in the input array x, producing output values in the range [-1, 1].

You can use this tanh() function to compute the tanh transformation of scalar values, arrays, or matrices in Python. It's commonly used in neural networks as an activation function, particularly in situations where the output of a neuron needs to be centered around zero and exhibit saturation behavior.