How do I remove NaN values from a NumPy array?

python

numpy

dataframe

performance

byNikita Barsukov·Feb 13, 2025

To banish NaNs efficiently from a NumPy array, boolean indexing coupled with np.isnan() is your knight in shining armor. Here's a demonstration:

import numpy as np

# A wild array appears, with NaNs lurking...
arr = np.array([1, np.nan, 3])

# Our knight enters the fray, banishing the NaN beasts!
clean_arr = arr[~np.isnan(arr)]

print(clean_arr)  # Output: [1. 3.] // And there was peace in the kingdom

Effective approaches to NaN-removal

Performance is key when dealing with large arrays. You want to remove NaNs swiftly while preserving the array's structure. The above boolean indexing does exactly that while remaining efficient. However, there are alternative paths to follow.

Purging NaNs using the filter function

If you prefer a more functional programming style, you might find the filter function useful:

# Filter and lambda, the dynamic duo
filtered_arr = np.array(list(filter(lambda v: v==v, arr))) // Talk about a self-esteem exercise for 'v'

This approach, while elegant, transforms NumPy arrays into lists during processing which, for large datasets, could slow things down.

Engaging list comprehensions for NaN combat

List comprehensions offer the option to tidy up those pesky NaNs in a Pythonic way:

# Leave the NaNs behind on our epic list comprehension quest
comprehended_arr = np.array([v for v in arr if not np.isnan(v)]) // Short, sweet, and NaN-free

This method beautifully synthesizes the brevity of list comprehensions with the efficient reconstruction of a NumPy array.

The matter of array types

None of these methods discriminate on the basis of array data types. Whether integers, floats, or a fancy cocktail of complex numbers, these techniques discard NaN values without prejudice.

Parameters for multidimensional arrays

Got NaNs in high dimensions? Fear not! Our removal techniques still apply.

Multidimensional mission: 2D arrays

Let's put a two-dimensional array on the chopping block:

two_d_arr = np.array([[1, 2, np.nan],[4, np.nan, 6]])
mask = ~np.isnan(two_d_arr)
filtered_2d_arr = np.array([row[mask[row]] for row in range(len(mask))]) // Mask on, NaNs gone!

Our trusty boolean indexing gracefully scales, making NaNs in higher dimensions a trivial problem.

Deploying np.ma or 'Masked Array' module

The NumPy's Masked Array module np.ma (covered below) provides dedicated support for handling missing data in arrays.

Be mindful of performance

For gigantic multidimensional datasets, resource performance may vary. Make sure to benchmark methods to find what's optimized for your specific case.