Converting between datetime, Timestamp and datetime64
datetime, numpy, pandas, python
Solution
To convert `numpy.datetime64` to `datetime` object that represents time in UTC on `numpy-1.8`:
>>> from datetime import datetime
>>> import numpy as np
>>> dt = datetime.utcnow()
>>> dt
datetime.datetime(2012, 12, 4, 19, 51, 25, 362455)
>>> dt64 = np.datetime64(dt)
>>> ts = (dt64 - np.datetime64('1970-01-01T00:00:00Z')) / np.timedelta64(1, 's')
>>> ts
1354650685.3624549
>>> datetime.utcfromtimestamp(ts)
datetime.datetime(2012, 12, 4, 19, 51, 25, 362455)
>>> np.__version__
'1.8.0.dev-7b75899'
The above example assumes that a naive `datetime` object is interpreted by `np.datetime64` as time in UTC.
To convert `datetime` to `np.datetime64` and back (`numpy-1.6`):
>>> np.datetime64(datetime.utcnow()).astype(datetime)
datetime.datetime(2012, 12, 4, 13, 34, 52, 827542)
It works both on a single `np.datetime64` object and a numpy array of `np.datetime64`.
Think of `np.datetime64` the same way you would about `np.int8`, `np.int16`, etc and apply the same methods to convert between Python objects such as `int`, `datetime` and corresponding numpy objects.
Your "nasty example" works correctly:
>>> from datetime import datetime
>>> import numpy
>>> numpy.datetime64('2002-06-28T01:00:00.000000000+0100').astype(datetime)
datetime.datetime(2002, 6, 28, 0, 0)
>>> numpy.__version__
'1.6.2' # current version available via pip install numpy
I can reproduce the `long` value on `numpy-1.8.0` installed as:
pip install git+https://github.com/numpy/numpy.git#egg=numpy-dev
The same example:
>>> from datetime import datetime
>>> import numpy
>>> numpy.datetime64('2002-06-28T01:00:00.000000000+0100').astype(datetime)
1025222400000000000L
>>> numpy.__version__
'1.8.0.dev-7b75899'
It returns `long` because for `numpy.datetime64` type `.astype(datetime)` is equivalent to `.astype(object)` that returns Python integer (`long`) on `numpy-1.8`.
To get `datetime` object you could:
>>> dt64.dtype
dtype('<M8[ns]')
>>> ns = 1e-9 # number of seconds in a nanosecond
>>> datetime.utcfromtimestamp(dt64.astype(int) * ns)
datetime.datetime(2002, 6, 28, 0, 0)
To get `datetime64` that uses seconds directly:
>>> dt64 = numpy.datetime64('2002-06-28T01:00:00.000000000+0100', 's')
>>> dt64.dtype
dtype('<M8[s]')
>>> datetime.utcfromtimestamp(dt64.astype(int))
datetime.datetime(2002, 6, 28, 0, 0)
The numpy docs say that the datetime API is experimental and may change in future numpy versions.
Problem
How do I convert a `numpy.datetime64` object to a `datetime.datetime` (or `Timestamp`)? In the following code, I create a datetime, timestamp and datetime64 objects. ``` import datetime import numpy as np import pandas as pd dt = datetime.datetime(2012, 5, 1) # A strange way to extract a Timestamp object, there's surely a better way? ts = pd.DatetimeIndex([dt])[0] dt64 = np.datetime64(dt) In [7]: dt Out[7]: datetime.datetime(2012, 5, 1, 0, 0) In [8]: ts Out[8]: <Timestamp: 2012-05-01 00:00:00> In [9]: dt64 Out[9]: numpy.datetime64('2012-05-01T01:00:00.000000+0100') ``` Note: it's easy to get the datetime from the Timestamp: ``` In [10]: ts.to_datetime() Out[10]: datetime.datetime(2012, 5, 1, 0, 0) ``` But how do we extract the `datetime` or `Timestamp` from a `numpy.datetime64` (`dt64`)? . Update: a somewhat nasty example in my dataset (perhaps the motivating example) seems to be: ``` dt64 = numpy.datetime64('2002-06-28T01:00:00.000000000+0100') ``` which should be `datetime.datetime(2002, 6, 28, 1, 0)`, and not a long (!) (`1025222400000000000L`)...