Create numpy matrix filled with NaNs

Performance aspects: np.full() vs np.empty().fill()

Using np.full()

The np.full() function works out of the box and rightly so:

# "I want a full house of NaNs, because I'm cool like that."
nan_matrix = np.full((1000, 1000), np.nan)


More performant way with np.empty().fill()

However, if you're dealing with large datasets and your performance need to be as slick as a Tesla Model S on a German autobahn, consider using the np.empty().fill() combo. Here's how:

# "Show me the void before I fill it... with NaNs."
nan_matrix = np.empty((1000, 1000))
nan_matrix.fill(np.nan)


This operation is in-place — it alters the matrix itself, and hence returns None.

Performance superiority: Benchmarking

Running a marathon between np.full() and np.empty().fill(). Who's gonna win?

import numpy as np 
import timeit

# "Let's see who's the Usain Bolt of creating NaN matrices."
setup = "import numpy as np"
stmt1 = "nan_mat = np.full((500,500), np.nan)"
stmt2 = """
nan_matrix = np.empty((500,500))
nan_matrix.fill(np.nan)
"""
print(timeit.timeit(setup=setup, stmt=stmt1)) # Time for np.full()
print(timeit.timeit(setup=setup, stmt=stmt2)) # Time for np.empty().fill()


For large sizes, expect np.empty().fill() to have the edge. But hey, don't take my word for it... run the code and see for yourself!

Creating NaN matrices: Beyond the basics

Custom NaN matrix function

Savor the luxury of having your own function for creating NaN matrices to fit your wildest desires. Shape describes the matrix dimensions and dtype=float defines the type of the data.

def create_nan_matrix(shape, dtype=float):
    nan_matrix = np.empty(shape, dtype)
    nan_matrix.fill(np.nan)
    return nan_matrix


Comparing nan_matrix[:] = np.nan and nan_matrix.fill(np.nan)

If you're left wondering "What if I just set all the elements to np.nan?", consider this fact: nan_matrix.fill(np.nan) is typically faster because it performs a very low-level in-place operation.

Kneading NumPy: Advanced insights

np.full(): More like np.fun()!

np.full() has been around since NumPy version 1.8. So, assuming you're not coding on a dinosaur, you should have access to this nifty function.

Multiplying ones with np.nan, because why not

One seldom used alternative method is to create an array of ones and multiply it by np.nan.

# "NaNs! NaNs everywhere."
nan_matrix = np.ones((3,3)) * np.nan


This isn't the most efficient solution, but it's still an option if readability is your top concern.

Data type dance

If you're trying to cram your NaNs into a non-float64 type, you're gonna have a bad time. By default, NaNs in NumPy are float64.

What's under the hood

There's a lot to learn from NumPy's documentation and the community. They can help you understand the implementation of its functions better, making you a more capable NumPy Matrix "A NaNtist".