I have a time series that I have pulled from a netCDF file and I'm trying to convert them to a datetime format. The format of the time series is in 'days since 1990-01-01 00:00:00 +10' (+10 being GMT: +10)
time = nc_data.variables['time'][:]
time_idx = 0 # first timestamp
print time[time_idx]
9465.0
My desired output is a datetime object like so (also GMT +10):
"2015-12-01 00:00:00"
I have tried converting this using the time module without much success although I believe I may be using wrong (I'm still a novice in python and programming).
import time
time_datetime = time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(time[time_idx]*24*60*60))
Any advice appreciated,
Cheers!
The datetime module's timedelta is probably what you're looking for.
For example:
from datetime import date, timedelta
days = 9465 # This may work for floats in general, but using integers
# is more precise (e.g. days = int(9465.0))
start = date(1990,1,1) # This is the "days since" part
delta = timedelta(days) # Create a time delta object from the number of days
offset = start + delta # Add the specified number of days to 1990
print(offset) # >>> 2015-12-01
print(type(offset)) # >>> <class 'datetime.date'>
You can then use and/or manipulate the offset object, or convert it to a string representation however you see fit.
You can use the same format as for this date object as you do for your time_datetime:
print(offset.strftime('%Y-%m-%d %H:%M:%S'))
Output:
2015-12-01 00:00:00
Instead of using a date object, you could use a datetime object instead if, for example, you were later going to add hours/minutes/seconds/timezone offsets to it.
The code would stay the same as above with the exception of two lines:
# Here, you're importing datetime instead of date
from datetime import datetime, timedelta
# Here, you're creating a datetime object instead of a date object
start = datetime(1990,1,1) # This is the "days since" part
Note: Although you don't state it, but the other answer suggests you might be looking for timezone aware datetimes. If that's the case, dateutil is the way to go in Python 2 as the other answer suggests. In Python 3, you'd want to use the datetime module's tzinfo.
netCDF num2date is the correct function to use here:
import netCDF4
ncfile = netCDF4.Dataset('./foo.nc', 'r')
time = ncfile.variables['time'] # do not cast to numpy array yet
time_convert = netCDF4.num2date(time[:], time.units, time.calendar)
This will convert number of days since 1900-01-01 (i.e. the units of time) to python datetime objects. If time does not have a calendar attribute, you'll need to specify the calendar, or use the default of standard.
We can do this in a couple steps. First, we are going to use the dateutil library to handle our work. It will make some of this easier.
The first step is to get a datetime object from your string (1990-01-01 00:00:00 +10). We'll do that with the following code:
from datetime import datetime
from dateutil.relativedelta import relativedelta
import dateutil.parser
days_since = '1990-01-01 00:00:00 +10'
days_since_dt = dateutil.parser.parse(days_since)
Now, our days_since_dt will look like this:
datetime.datetime(1990, 1, 1, 0, 0, tzinfo=tzoffset(None, 36000))
We'll use that in our next step, of determining the new date. We'll use relativedelta in dateutils to handle this math.
new_date = days_since_dt + relativedelta(days=9465.0)
This will result in your value in new_date having a value of:
datetime.datetime(2015, 12, 1, 0, 0, tzinfo=tzoffset(None, 36000))
This method ensures that the answer you receive continues to be in GMT+10.
Related
I have the following problem I need to convert some datetime object to time_ns (present in time)
all I can find is to convert the now datetime to that fixed nanoseconds number (i understand that it is calculated from a fixed date in 1970)
import time
now = time.time_ns()
all I want is to convert some normal datetime object to that fixed nanoseconds number
from datetime import datetime
x = datetime(2022, 2, 22, 15, 41, 50)
i don't want to be restricted to only now dates. is there some function in the library that does that? for the moment i cannot find anything
thank you very much
Since python 3.3 datetime has a timestamp function. Make sure to replace the timezone, otherwise local timezone will being taken and if you want to have nanosecond number you can multiply the seconds number.
from datetime import datetime;
print(datetime(2022,2,22,15,41,50).replace(tzinfo=timezone.utc).timestamp()*10**9)
A client has specified that they use DateTime to store their dates using the format 2021-06-22T11:17:09.465Z, and so far I've been able only to obtain it in string dates, because If I want to maintain the milliseconds it saves them like 2021-06-22T11:17:09.465000.
Is there any possible way to force DateTime to use milliseconds instead of microseconds? I'm aware of the %f for microseconds in the format, but I've tried everything I can think of to reduce those 3 decimals while keeping it DateTime with no results however.
I suggest to use the timespec parameter, as described in python docs https://docs.python.org/3/library/datetime.html#datetime.datetime.isoformat:
>>> from datetime import datetime
>>> datetime.now().isoformat(timespec='minutes')
'2002-12-25T00:00'
>>> dt = datetime(2015, 1, 1, 12, 30, 59, 0)
>>> datetime.now().isoformat(timespec='milliseconds')
'2021-12-02T14:03:57.937'
Something like this works:
from datetime import datetime
dt = datetime.now()
print(f"{dt:%Y/%m/%dT%H:%M:%S}.{f'{dt:%f}'[:3]}")
Hope I help.
I assume you're looking for this? See also my general comment at question.
The variable 3 in [:3] can be adjusted to your liking for amount of zeros in ms to ns range. Use the type() to show you its a DateTime object.
import time
from datetime import datetime
tm = time.time()
print(tm)
dt = str(tm).split('.')
print(dt)
timestamp = float(dt[0] + '.' + dt[1][:3])
dt_object = datetime.fromtimestamp(timestamp)
print(dt_object)
This prints for example:
tm : 1638463260.919723
dt : ['1638463260', '919723']
and
dd_object : 2021-12-02 17:41:00.919000
You can divide nanoseconds by 1000000000 to get seconds and by 1000000 to get milliseconds.
Here is some code that will get nanoseconds:
tim = time.time_ns()
You can then combine the output of this with the rest of the format. Probably not the cleanest solution but it should work.
from datetime import datetime
import pandas as pd
date="2020-02-07T16:05:16.000000000"
#Convert using datetime
t1=datetime.strptime(date[:-3],'%Y-%m-%dT%H:%M:%S.%f')
#Convert using Pandas
t2=pd.to_datetime(date)
#Subtract the dates
print(t1-t2)
#subtract the date timestamps
print(t1.timestamp()-t2.timestamp())
In this example, my understanding is that both datetime and pandas should use timezone naive dates. Can anyone explain why the difference between the dates is zero, but the difference between the timestamps is not zero? It's off by 5 hours for me, which is my time zone offset from GMT.
Naive datetime objects of Python's datetime.datetime class represent local time. This is kind of obvious from the docs but can be a brain-teaser to work with nevertheless. If you call the timestamp method on it, the returned POSIX timestamp refers to UTC (seconds since the epoch) as it should.
Coming from the Python datetime object, the behavior of a naive pandas.Timestamp can be counter-intuitive (and I think it's not so obvious). Derived the same way from a tz-naive string, it doesn't represent local time but UTC. You can verify that by localizing the datetime object to UTC:
from datetime import datetime, timezone
import pandas as pd
date = "2020-02-07T16:05:16.000000000"
t1 = datetime.strptime(date[:-3], '%Y-%m-%dT%H:%M:%S.%f')
t2 = pd.to_datetime(date)
print(t1.replace(tzinfo=timezone.utc).timestamp() - t2.timestamp())
# 0.0
The other way around you can make the pandas.Timestamp timezone-aware, e.g.
t3 = pd.to_datetime(t1.astimezone())
# e.g. Timestamp('2020-02-07 16:05:16+0100', tz='Mitteleuropäische Zeit')
# now both t1 and t3 represent my local time:
print(t1.timestamp() - t3.timestamp())
# 0.0
My bottom line is that if you know that the timestamps you have represent a certain timezone, work with timezone-aware datetime, e.g. for UTC
import pytz # need to use pytz here since pandas uses that internally
t1 = datetime.strptime(date[:-3], '%Y-%m-%dT%H:%M:%S.%f').replace(tzinfo=pytz.UTC)
t2 = pd.to_datetime(date, utc=True)
print(t1 == t2)
# True
print(t1-t2)
# 0 days 00:00:00
print(t1.timestamp()-t2.timestamp())
# 0.0
I need help with converting time to a readable format. My time array (has 580 elements) is # of days since January 1st, 1900. How do I convert that to a normal time format (ie mm-dd-yyyy)?
For example, input time is 43,887 and output should read 02-27-2020 after adding it to Jan 1, 1900.
Thank you!
datetime.datetime and timedelta class can be helpful here.
from datetime import datetime, timedelta
def convert(inp, date1):
new_date = date1 + timedelta(inp)
return new_date.strftime("%m-%d-%Y")
date1 = datetime(1900, 1, 1)
print(convert(43887, date1))
You can use the datetime.datetime class can help you here. The following works, if those values are treated as integer days (you don't specify what they are).
from datetime import datetime
dt = datetime.fromordinal(43887)
dt.strftime('%d-%m-%Y')
How can I calculate the next day from a string like 20110531 in the same YYYYMMDD format? In this particular case, I like to have 20110601 as the result. Calculating "tomorrow" or next day in static way is not that tough, like this:
>>> from datetime import date, timedelta
>>> (date.today() + timedelta(1)).strftime('%Y%m%d')
'20110512'
>>>
>>> (date(2011,05,31) + timedelta(1)).strftime('%Y%m%d')
'20110601'
But how can I use a string like dt = "20110531" to get the same result as above?
Here is an example of how to do it:
import time
from datetime import date, timedelta
t=time.strptime('20110531','%Y%m%d')
newdate=date(t.tm_year,t.tm_mon,t.tm_mday)+timedelta(1)
print newdate.strftime('%Y%m%d')
>>> from datetime import datetime
>>> print datetime.strptime('20110531', '%Y%m%d')
2011-05-31 00:00:00
And then do math on that date object as you show in your question.
The datetime library docs.
You are most of the way there! along with the strftime function which converts a date to a formatted string, there is also a strptime function which converts back the other way.
To solve your problem you can just replace date.today() with strptime(yourDateString, '%Y%m%d').
ED: and of course you will also have to add strptime to the end of your from datetime import line.