Sigmoid function

Sigmoid function maps the prediction to probabilities - maps any real value into another value between 0 and 1.

Pros:

1. It is nonlinear so it can be used to activate hidden layers in a neural network.

2. It is differentiable everywhere so gradient-based back-propagation can be used with it.

3. Its output ranges from 0 to 1 so it can generate probabilities.

Cons:

1. The gradient for inputs that are far from the origin is near zero, so gradient-based learning is slow for saturated neurons using sigmoid.

2. When used as the final activation in a classifier, the sum of all classes is not necessarily 1.

def sigmoid(z):
	return 1.0 / (1 + np.exp(-z))


# derivative:
# s'(z) = s(z)*(1-s(z))
def sigmoid_direv(z):
	return 1.0 / (1 + np.exp(-z)) * (1 - 1.0 / (1 + np.exp(-z)))

The sigmoid function, also known as the logistic function, is a mathematical function that maps input values to a range between 0 and 1. It is commonly used in binary classification tasks in machine learning and neural networks to produce probabilities of class membership. The sigmoid function is defined as:

$\text{sigmoid}(x) = \frac{1}{1 + e^{-x}}$

Here's a Python implementation of the sigmoid function:

import numpy as np

def sigmoid(x):
    """
    Compute the sigmoid function for an input array x.

    Parameters:
    x : array_like
        Input array.

    Returns:
    ndarray
        Sigmoid function applied to each element of x.
    """
    return 1 / (1 + np.exp(-x))

In this implementation:

• np.exp(-x) computes the exponential of each element in the input array x.

• 1 / (1 + np.exp(-x)) applies the sigmoid transformation to each element of the input array, producing output values in the range [0, 1].

You can use this sigmoid() function to compute the sigmoid transformation of scalar values, arrays, or matrices in Python. It's commonly used as an activation function in neural networks, where it introduces non-linearity and maps the network's output to a probability distribution over classes.