How to calculate a logistic sigmoid function in Python?

Precision matters

When dealing with the sigmoid function, numerical precision plays a key role, particularly at the extremes where the function tends towards 0 or 1. Standard implementation using math.exp might stumble upon large negative values. Here's where SciPy's expit serves as a more robust and stable solution:

from scipy.special import expit
# Exotic name, huh? It's actually the hip cousin of Sigmoid, robust and stable.
print(expit(-1000))  # Output: 0.0, stable for large negative inputs


Speed up with vectors

Encountering large data arrays challenges? Use NumPy's vectorized operations instead of Python's math module. Performance gains are significant, especially for larger datasets:

def sigmoid_array(x):
    # Big arrays are no big deal for mighty NumPy!
    return 1 / (1 + np.exp(-np.array(x)))

# Test the function on some largish data:
large_data = np.array([-3, -1, 0, 1, 3])
print(sigmoid_array(large_data))


Beloved alternatives

For a mathematically polished alternative, employ the tanh method within a lambda function:

import math

sigmoid_alt = lambda x: .5 * (math.tanh(.5 * x) + 1)
# Notice there's tanh in the mix? Say hello, hyperbolic tangent!
print(sigmoid_alt(0.5))


For performance-critical applications, consider using precomputed tables for sigmoid values.

Normalization

Ensure data is normalized (i.e., centered around zero) before applying the sigmoid function for better efficiency and accuracy:

def normalize_and_apply_sigmoid(x, mean, std):
    normalized_x = (x - mean) / std
    return sigmoid(normalized_x)  # A normalized, sigmoided datapoint


Crafting the curve

Depending on the situation, you might want to adjust the steepness of the sigmoid curve. Introducing a gain factor (k) lets you control the curve shape:

def custom_sigmoid(x, k=1):
    # ‍⚕️💉 Injecting a bit of customization!
    return 1 / (1 + np.exp(-k*x)) 


Visual realities

Plot the sigmoid curve using matplotlib to visualize its shape and behavior under various circumstances:

import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(-10, 10, 100)
# Art class, folks! 🎨
plt.plot(x, sigmoid(x), label='Sigmoid Function')
plt.xlabel('x')
plt.ylabel('sigmoid(x)')
plt.title('Visual Representation of the Sigmoid Function')
plt.legend()
plt.grid(True)
plt.show()


Adjust the parameters of the sigmoid function in your plots to visualize their effects.

Optimize and compare

Optimal computational paths such as logaddexp.reduce significantly reduce computational overhead for large inputs. A 2x2 grid of plots for comparing sigmoid functions helps visualize behavior for different curve parameters:

# A grid of plots? Yes, please!
fig, ax = plt.subplots(2, 2)
k_values = [0.5, 1, 2, 5]
for i, ax in enumerate(ax.flatten()):
    ax.plot(x, custom_sigmoid(x, k=k_values[i]), label=f'k={k_values[i]}')
    ax.set_title(f'Sigmoid with k={k_values[i]}')
    ax.label_outer()
    ax.legend()
fig.tight_layout()
plt.show()


Mirror, mirror on the x-axis

Understand how positive and negative x values affect the sigmoid. It approaches 1 for positive infinity and 0 for negative infinity. Certain applications may require mirrored sigmoid functions or similar adaptations.