Converting between datetime, Timestamp and datetime64

python

pandas

datetime

timestamp

byNikita Barsukov·Aug 30, 2024

Welcome to the world of datetime, Timestamp, and datetime64. Here's your quick guide on how to convert between these three:

To convert datetime to Timestamp: Use the following line of code pd.Timestamp(your_datetime).
To convert Timestamp to datetime: Here’s the trick up your sleeve: your_timestamp.to_pydatetime().
To convert datetime to datetime64: Easily done with np.datetime64(your_datetime).
To convert datetime64 to datetime: Just a slight twist with astype - your_datetime64.astype('M8[ms]').astype(datetime).

Here’s the whole shebang:

import pandas as pd
import numpy as np
from datetime import datetime

your_datetime = datetime.now() # grabs the current datetime
your_timestamp = pd.Timestamp(your_datetime) # it's timestamping time!
your_datetime64 = np.datetime64(your_datetime) # let's go 64-bits deep
back_to_datetime = your_datetime64.astype('M8[ms]').astype(datetime) # Back to the future, or rather, datetime

Always select the right datetime type that fits your application. Proper conversions maintain data integrity and keep your data ready for action!

Quick-and-dirty conversion tips

The Pandas Way

Pandas showcases its flexibility with pd.to_datetime. This function can handle a range of formats and smoothly convert them to a pandas Timestamp or DatetimeIndex:

pd.to_datetime('2023-04-12') # Y-M-D, Pandas likes it this way
pd.to_datetime('12/04/2023', dayfirst=True) # D-M-Y, for our friends across the pond and others

pd.to_datetime(1670407200, unit='s') # UNIX timestamp. No, UNIX is not about unicorns 🦄

Just remember to stay aware of the time zones while converting. Timezone-naive and timezone-aware types are not the same beast! Lookout for the tz parameter.

Navigating the Epoch

datetime.datetime.utcfromtimestamp might be your go-to for the epoch to datetime conversion, but be careful of overflows! Remember, the UNIX timestamps start from a zero that’s pinned to January 1, 1970 - anything before that is a no-go with this method. The solution? Use pd.to_datetime with unit='s' or unit='ms'.

NumPy Know-How

When dealing with numpy's date and time types, np.timedelta64(1, 's') can come in really handy:

np_datetime64 = np.datetime64('2023-04-12T12:34:56')
unix_timestamp = (np_datetime64 - np.datetime64('1970-01-01T00:00:00Z')) / np.timedelta64(1, 's')
print(unix_timestamp) # Now will you look at that!

Working with arrays of dates and times? The tolist() method can help convert a numpy.datetime64 array to a list of Python datetime objects in a jiffy.

Don’t forget to double-check compatibility between pandas and numpy versions for effortless conversions. Upgrade your libraries if necessary!

Dealing with anomalies and performance pitfalls

Outliers and Oddities

Date and time conversions sometimes throw curveballs - leap seconds, Daylight Saving Time shifts, and historical time changes can be sources of confusion. Always validate critical time data manually to avoid temporal heartache.

Speed Matters

For high-performance scenarios, prefer NumPy operations as they beat Pandas hands down with their speed. The secret sauce is in their lower-level implementation and reduced overhead.

Going beyond standard Python

Like to think outside the box? Consider using packages like Arrow or Maya. They offer a friendlier interface for handling time zones and date-time formatting.

explain-codes / Python / Converting between datetime, Timestamp and datetime64

Linked

How can I convert a datetime object to milliseconds since epoch (unix time) in Python?



Convert string date to timestamp in Python



How to convert local time string to UTC?



Getting today's date in YYYY-MM-DD in Python?



Converting Unix timestamp string to readable date

