Related
I need to add +3 hours to a date in iso 8601 format in python, for example "2022-09-21T22:31:59Z" due to time difference. In this time information that is returned to me from the API, I only need Y/M/D information, but due to the +3 hour difference, the day information needs to go one step further in the date information, as will be experienced in the date I conveyed in the example. How can I overcome this problem? I think the format of the date format is ISO 8601 but can you correct me if I am wrong?
ex. api response;
"createdDateTime": "2022-09-21T22:31:59Z"
what i need;
"createdDateTime": "2022-09-21T22:31:59Z" to "2022-09-22T01:31:59Z"
Try this code it will definitely work:
from datetime import datetime,timedelta
parsed_date=datetime.strptime("2022-09-21T22:31:59Z", "%Y-%m-%dT%H:%M:%SZ")
Updated_date = parsed_date+ timedelta(hours=3)
print(Updated_date)
If you have a proper JSON string you can parse it with json, extract the string value, parse that with datetime.fromisoformat into a datetime value and then get the date from it :
import json
from datetime import datetime
data=json.loads('{"createdDateTime": "2022-09-21T22:31:59+00:00"}')
isostr=data['createdDateTime'].replace('Z','+00:00')
fulldate=datetime.fromisoformat(isostr)
fulldate.date()
-----
datetime.date(2022, 9, 21)
The replacement is necessary because fromisoformat doesn't understand Z
Adding 3 hours to fulldate will return 1 AM in the next day:
fulldate + timedelta(hours=3)
------
datetime.datetime(2022, 9, 22, 1, 31, 59, tzinfo=datetime.timezone.utc)
fulldate is in UTC. It can be converted to another timezone offset using astimezone
fulldate.astimezone(tz=timezone(timedelta(hours=3)))
---
datetime.datetime(2022, 9, 22, 1, 31, 59, tzinfo=datetime.timezone(datetime.timedelta(seconds=10800)))
Or in a more readable form:
fulldate.astimezone(tz=timezone(timedelta(hours=3))).isoformat()
---------------------------
'2022-09-22T01:31:59+03:00'
This is 1AM in the next day but with a +3:00 offset. This is still the same time as 22PM at UTC.
It's also possible to just replace the UTC offset with another one, without changing the time values, using replace:
fulldate.replace(tzinfo=timezone(timedelta(hours=3))).isoformat()
----------------------------
'2022-09-21T22:31:59+03:00'
That's the original time with a different offset. That's no longer the same time as 2022-09-21T22:31:59Z
I have a time series that I have pulled from a netCDF file and I'm trying to convert them to a datetime format. The format of the time series is in 'days since 1990-01-01 00:00:00 +10' (+10 being GMT: +10)
time = nc_data.variables['time'][:]
time_idx = 0 # first timestamp
print time[time_idx]
9465.0
My desired output is a datetime object like so (also GMT +10):
"2015-12-01 00:00:00"
I have tried converting this using the time module without much success although I believe I may be using wrong (I'm still a novice in python and programming).
import time
time_datetime = time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(time[time_idx]*24*60*60))
Any advice appreciated,
Cheers!
The datetime module's timedelta is probably what you're looking for.
For example:
from datetime import date, timedelta
days = 9465 # This may work for floats in general, but using integers
# is more precise (e.g. days = int(9465.0))
start = date(1990,1,1) # This is the "days since" part
delta = timedelta(days) # Create a time delta object from the number of days
offset = start + delta # Add the specified number of days to 1990
print(offset) # >>> 2015-12-01
print(type(offset)) # >>> <class 'datetime.date'>
You can then use and/or manipulate the offset object, or convert it to a string representation however you see fit.
You can use the same format as for this date object as you do for your time_datetime:
print(offset.strftime('%Y-%m-%d %H:%M:%S'))
Output:
2015-12-01 00:00:00
Instead of using a date object, you could use a datetime object instead if, for example, you were later going to add hours/minutes/seconds/timezone offsets to it.
The code would stay the same as above with the exception of two lines:
# Here, you're importing datetime instead of date
from datetime import datetime, timedelta
# Here, you're creating a datetime object instead of a date object
start = datetime(1990,1,1) # This is the "days since" part
Note: Although you don't state it, but the other answer suggests you might be looking for timezone aware datetimes. If that's the case, dateutil is the way to go in Python 2 as the other answer suggests. In Python 3, you'd want to use the datetime module's tzinfo.
netCDF num2date is the correct function to use here:
import netCDF4
ncfile = netCDF4.Dataset('./foo.nc', 'r')
time = ncfile.variables['time'] # do not cast to numpy array yet
time_convert = netCDF4.num2date(time[:], time.units, time.calendar)
This will convert number of days since 1900-01-01 (i.e. the units of time) to python datetime objects. If time does not have a calendar attribute, you'll need to specify the calendar, or use the default of standard.
We can do this in a couple steps. First, we are going to use the dateutil library to handle our work. It will make some of this easier.
The first step is to get a datetime object from your string (1990-01-01 00:00:00 +10). We'll do that with the following code:
from datetime import datetime
from dateutil.relativedelta import relativedelta
import dateutil.parser
days_since = '1990-01-01 00:00:00 +10'
days_since_dt = dateutil.parser.parse(days_since)
Now, our days_since_dt will look like this:
datetime.datetime(1990, 1, 1, 0, 0, tzinfo=tzoffset(None, 36000))
We'll use that in our next step, of determining the new date. We'll use relativedelta in dateutils to handle this math.
new_date = days_since_dt + relativedelta(days=9465.0)
This will result in your value in new_date having a value of:
datetime.datetime(2015, 12, 1, 0, 0, tzinfo=tzoffset(None, 36000))
This method ensures that the answer you receive continues to be in GMT+10.
I have dates in the form 26/11/2015. How can I convert them into the format 26-Nov-2015 and still keep them as dates and not strings?
Your question does not make much sense. If you keep them as dates, they have no format. The format is only manifested when you convert them to strings.
So the answer is: Store the dates as date (or datetime) objects, and use datetime.strftime with some specific format whenever you need them as a string:
>>> from datetime import date
>>> d = date(2016, 11, 26)
>>> d.strftime("%Y/%m/%d")
'2016/11/26'
>>> d.strftime("%d-%b-%Y")
'26-Nov-2016'
Conversely, use strptime to parse strings in different formats to dates:
>>> datetime.datetime.strptime("26-Nov-2015", "%d-%b-%Y")
datetime.datetime(2015, 11, 26, 0, 0)
from datetime import datetime
date = datetime.strptime('26/11/2015', '%d/%m/%Y')
print date.strftime("%d-%B-%Y")
In the above example, we are taking your input string 'dd/mm/yyyy' and turning it into a python datetime saving it to a variable called date (for future usage as per your request), and then printing it out in the format requested.
You want to use the datetime module I think. For example:
from datetime import date
a = date(2015, 11, 26)
a.strftime("%A %d of %B, %Y")
should give you 'Thursday 26 of November, 2015'
Or for your specific formatting request:
a.strftime("%d-%b-%Y") #'26-Nov-2015'
Hope this helps, good luck!
I have dt = datetime(2013,9,1,11), and I would like to get a Unix timestamp of this datetime object.
When I do (dt - datetime(1970,1,1)).total_seconds() I got the timestamp 1378033200.
When converting it back using datetime.fromtimestamp I got datetime.datetime(2013, 9, 1, 6, 0).
The hour doesn't match. What did I miss here?
solution is
import time
import datetime
d = datetime.date(2015,1,5)
unixtime = time.mktime(d.timetuple())
If you want to convert a python datetime to seconds since epoch you should do it explicitly:
>>> import datetime
>>> datetime.datetime(2012, 04, 01, 0, 0).strftime('%s')
'1333234800'
>>> (datetime.datetime(2012, 04, 01, 0, 0) - datetime.datetime(1970, 1, 1)).total_seconds()
1333238400.0
In Python 3.3+ you can use timestamp() instead:
>>> import datetime
>>> datetime.datetime(2012, 4, 1, 0, 0).timestamp()
1333234800.0
What you missed here is timezones.
Presumably you've five hours off UTC, so 2013-09-01T11:00:00 local and 2013-09-01T06:00:00Z are the same time.
You need to read the top of the datetime docs, which explain about timezones and "naive" and "aware" objects.
If your original naive datetime was UTC, the way to recover it is to use utcfromtimestamp instead of fromtimestamp.
On the other hand, if your original naive datetime was local, you shouldn't have subtracted a UTC timestamp from it in the first place; use datetime.fromtimestamp(0) instead.
Or, if you had an aware datetime object, you need to either use a local (aware) epoch on both sides, or explicitly convert to and from UTC.
If you have, or can upgrade to, Python 3.3 or later, you can avoid all of these problems by just using the timestamp method instead of trying to figure out how to do it yourself. And even if you don't, you may want to consider borrowing its source code.
(And if you can wait for Python 3.4, it looks like PEP 341 is likely to make it into the final release, which means all of the stuff J.F. Sebastian and I were talking about in the comments should be doable with just the stdlib, and working the same way on both Unix and Windows.)
Rather than this expression to create a POSIX timestamp from dt,
(dt - datetime(1970,1,1)).total_seconds()
Use this:
int(dt.strftime("%s"))
I get the right answer in your example using the second method.
EDIT: Some followup... After some comments (see below), I was curious about the lack of support or documentation for %s in strftime. Here's what I found:
In the Python source for datetime and time, the string STRFTIME_FORMAT_CODES tells us:
"Other codes may be available on your platform.
See documentation for the C library strftime function."
So now if we man strftime (on BSD systems such as Mac OS X), you'll find support for %s:
"%s is replaced by the number of seconds since the Epoch, UTC (see mktime(3))."
Anyways, that's why %s works on the systems it does. But there are better solutions to OP's problem (that take timezones into account). See #abarnert's accepted answer here.
For working with UTC timezones:
time_stamp = calendar.timegm(dt.timetuple())
datetime.utcfromtimestamp(time_stamp)
You've missed the time zone info (already answered, agreed)
arrow package allows to avoid this torture with datetimes; It is already written, tested, pypi-published, cross-python (2.6 — 3.xx).
All you need: pip install arrow (or add to dependencies)
Solution for your case
dt = datetime(2013,9,1,11)
arrow.get(dt).timestamp
# >>> 1378033200
bc = arrow.get(1378033200).datetime
print(bc)
# >>> datetime.datetime(2013, 9, 1, 11, 0, tzinfo=tzutc())
print(bc.isoformat())
# >>> '2013-09-01T11:00:00+00:00'
If your datetime object represents UTC time, don't use time.mktime, as it assumes the tuple is in your local timezone. Instead, use calendar.timegm:
>>> import datetime, calendar
>>> d = datetime.datetime(1970, 1, 1, 0, 1, 0)
>>> calendar.timegm(d.timetuple())
60
def dt2ts(dt, utc=False):
if utc:
return calendar.timegm(dt.timetuple())
if dt.tzinfo is None:
return int(time.mktime(dt.timetuple()))
utc_dt = dt.astimezone(tz.tzutc()).timetuple()
return calendar.timegm(utc_dt)
If you want UTC timestamp :time.mktime just for local dt .Use calendar.timegm is safe but dt must the utc zone so change the zone to utc. If dt in UTC just use calendar.timegm.
def datetime_to_epoch(d1):
"""
January 1st, 1970 at 00:00:00 UTC is referred to as the Unix epoch
:param d1: input date
:return: seconds since unix epoch
"""
if not d1.tzinfo:
raise ValueError("date is missing timezone information")
d2 = datetime(1970, 1, 1, tzinfo=timezone.utc)
time_delta = d1 - d2
ts = int(time_delta.total_seconds())
return ts
def epoch_to_datetime_string(timestamp, tz_name="UTC", **kwargs):
"""
method to convert unix timestamp to date time string
:param ts: 10 digit unix timestamp in seconds
:param tz_name: timezone name
:param kwargs: formatter=<formatter-string>
:return: date time string in timezone
"""
naive_date = datetime.fromtimestamp(timestamp)
aware_date = naive_date.astimezone(pytz.timezone(tz_name))
formatter = kwargs.pop("formatter", "%d %b %Y %H:%M:%S")
return aware_date.strftime(formatter)
Well, when converting TO unix timestamp, python is basically assuming UTC, but while converting back it will give you a date converted to your local timezone.
See this question/answer;
Get timezone used by datetime.datetime.fromtimestamp()
This class will cover your needs, you can pass the variable into ConvertUnixToDatetime & call which function you want it to operate based off.
from datetime import datetime
import time
class ConvertUnixToDatetime:
def __init__(self, date):
self.date = date
# Convert unix to date object
def convert_unix(self):
unix = self.date
# Check if unix is a string or int & proceeds with correct conversion
if type(unix).__name__ == 'str':
unix = int(unix[0:10])
else:
unix = int(str(unix)[0:10])
date = datetime.utcfromtimestamp(unix).strftime('%Y-%m-%d %H:%M:%S')
return date
# Convert date to unix object
def convert_date(self):
date = self.date
# Check if datetime object or raise ValueError
if type(date).__name__ == 'datetime':
unixtime = int(time.mktime(date.timetuple()))
else:
raise ValueError('You are trying to pass a None Datetime object')
return type(unixtime).__name__, unixtime
if __name__ == '__main__':
# Test Date
date_test = ConvertUnixToDatetime(datetime.today())
date_test = date_test.convert_date()
print(date_test)
# Test Unix
unix_test = ConvertUnixToDatetime(date_test[1])
print(unix_test.convert_unix())
import time
from datetime import datetime
time.mktime(datetime.now().timetuple())
Is there a way to get the UTC timestamp by specifying the date? What I would expect:
datetime(2008, 1, 1, 0, 0, 0, 0)
should result in
1199145600
Creating a naive datetime object means that there is no time zone information. If I look at the documentation for datetime.utcfromtimestamp, creating a UTC timestamp means leaving out the time zone information. So I would guess, that creating a naive datetime object (like I did) would result in a UTC timestamp. However:
then = datetime(2008, 1, 1, 0, 0, 0, 0)
datetime.utcfromtimestamp(float(then.strftime('%s')))
results in
2007-12-31 23:00:00
Is there still any hidden time zone information in the datetime object? What am I doing wrong?
Naïve datetime versus aware datetime
Default datetime objects are said to be "naïve": they keep time information without the time zone information. Think about naïve datetime as a relative number (ie: +4) without a clear origin (in fact your origin will be common throughout your system boundary).
In contrast, think about aware datetime as absolute numbers (ie: 8) with a common origin for the whole world.
Without timezone information you cannot convert the "naive" datetime towards any non-naive time representation (where does +4 targets if we don't know from where to start ?). This is why you can't have a datetime.datetime.toutctimestamp() method. (cf: http://bugs.python.org/issue1457227)
To check if your datetime dt is naïve, check dt.tzinfo, if None, then it's naïve:
datetime.now() ## DANGER: returns naïve datetime pointing on local time
datetime(1970, 1, 1) ## returns naïve datetime pointing on user given time
I have naïve datetimes, what can I do ?
You must make an assumption depending on your particular context:
The question you must ask yourself is: was your datetime on UTC ? or was it local time ?
If you were using UTC (you are out of trouble):
import calendar
def dt2ts(dt):
"""Converts a datetime object to UTC timestamp
naive datetime will be considered UTC.
"""
return calendar.timegm(dt.utctimetuple())
If you were NOT using UTC, welcome to hell.
You have to make your datetime non-naïve prior to using the former
function, by giving them back their intended timezone.
You'll need the name of the timezone and the information about
if DST was in effect when producing the target naïve datetime (the
last info about DST is required for cornercases):
import pytz ## pip install pytz
mytz = pytz.timezone('Europe/Amsterdam') ## Set your timezone
dt = mytz.normalize(mytz.localize(dt, is_dst=True)) ## Set is_dst accordingly
Consequences of not providing is_dst:
Not using is_dst will generate incorrect time (and UTC timestamp)
if target datetime was produced while a backward DST was put in place
(for instance changing DST time by removing one hour).
Providing incorrect is_dst will of course generate incorrect
time (and UTC timestamp) only on DST overlap or holes. And, when
providing
also incorrect time, occuring in "holes" (time that never existed due
to forward shifting DST), is_dst will give an interpretation of
how to consider this bogus time, and this is the only case where
.normalize(..) will actually do something here, as it'll then
translate it as an actual valid time (changing the datetime AND the
DST object if required). Note that .normalize() is not required
for having a correct UTC timestamp at the end, but is probably
recommended if you dislike the idea of having bogus times in your
variables, especially if you re-use this variable elsewhere.
and AVOID USING THE FOLLOWING: (cf: Datetime Timezone conversion using pytz)
dt = dt.replace(tzinfo=timezone('Europe/Amsterdam')) ## BAD !!
Why? because .replace() replaces blindly the tzinfo without
taking into account the target time and will choose a bad DST object.
Whereas .localize() uses the target time and your is_dst hint
to select the right DST object.
OLD incorrect answer (thanks #J.F.Sebastien for bringing this up):
Hopefully, it is quite easy to guess the timezone (your local origin) when you create your naive datetime object as it is related to the system configuration that you would hopefully NOT change between the naive datetime object creation and the moment when you want to get the UTC timestamp. This trick can be used to give an imperfect question.
By using time.mktime we can create an utc_mktime:
def utc_mktime(utc_tuple):
"""Returns number of seconds elapsed since epoch
Note that no timezone are taken into consideration.
utc tuple must be: (year, month, day, hour, minute, second)
"""
if len(utc_tuple) == 6:
utc_tuple += (0, 0, 0)
return time.mktime(utc_tuple) - time.mktime((1970, 1, 1, 0, 0, 0, 0, 0, 0))
def datetime_to_timestamp(dt):
"""Converts a datetime object to UTC timestamp"""
return int(utc_mktime(dt.timetuple()))
You must make sure that your datetime object is created on the same timezone than the one that has created your datetime.
This last solution is incorrect because it makes the assumption that the UTC offset from now is the same than the UTC offset from EPOCH. Which is not the case for a lot of timezones (in specific moment of the year for the Daylight Saving Time (DST) offsets).
Another possibility is:
d = datetime.datetime.utcnow()
epoch = datetime.datetime(1970,1,1)
t = (d - epoch).total_seconds()
This works as both "d" and "epoch" are naive datetimes, making the "-" operator valid, and returning an interval. total_seconds() turns the interval into seconds. Note that total_seconds() returns a float, even d.microsecond == 0
Also note the calendar.timegm() function as described by this blog entry:
import calendar
calendar.timegm(utc_timetuple)
The output should agree with the solution of vaab.
A simple solution without using external modules:
from datetime import datetime, timezone
dt = datetime(2008, 1, 1, 0, 0, 0, 0)
int(dt.replace(tzinfo=timezone.utc).timestamp())
If input datetime object is in UTC:
>>> dt = datetime(2008, 1, 1, 0, 0, 0, 0)
>>> timestamp = (dt - datetime(1970, 1, 1)).total_seconds()
1199145600.0
Note: it returns float i.e., microseconds are represented as fractions of a second.
If input date object is in UTC:
>>> from datetime import date
>>> utc_date = date(2008, 1, 1)
>>> timestamp = (utc_date.toordinal() - date(1970, 1, 1).toordinal()) * 24*60*60
1199145600
See more details at Converting datetime.date to UTC timestamp in Python.
I feel like the main answer is still not so clear, and it's worth taking the time to understand time and timezones.
The most important thing to understand when dealing with time is that time is relative!
2017-08-30 13:23:00: (a naive datetime), represents a local time somewhere in the world, but note that 2017-08-30 13:23:00 in London is NOT THE SAME TIME as 2017-08-30 13:23:00 in San Francisco.
Because the same time string can be interpreted as different points-in-time depending on where you are in the world, there is a need for an absolute notion of time.
A UTC timestamp is a number in seconds (or milliseconds) from Epoch (defined as 1 January 1970 00:00:00 at GMT timezone +00:00 offset).
Epoch is anchored on the GMT timezone and therefore is an absolute point in time. A UTC timestamp being an offset from an absolute time therefore defines an absolute point in time.
This makes it possible to order events in time.
Without timezone information, time is relative, and cannot be converted to an absolute notion of time without providing some indication of what timezone the naive datetime should be anchored to.
What are the types of time used in computer system?
naive datetime: usually for display, in local time (i.e. in the browser) where the OS can provide timezone information to the program.
UTC timestamps: A UTC timestamp is an absolute point in time, as mentioned above, but it is anchored in a given timezone, so a UTC timestamp can be converted to a datetime in any timezone, however it does not contain timezone information. What does that mean? That means that 1504119325 corresponds to 2017-08-30T18:55:24Z, or 2017-08-30T17:55:24-0100 or also 2017-08-30T10:55:24-0800. It doesn't tell you where the datetime recorded is from. It's usually used on the server side to record events (logs, etc...) or used to convert a timezone aware datetime to an absolute point in time and compute time differences.
ISO-8601 datetime string: The ISO-8601 is a standardized format to record datetime with timezone. (It's in fact several formats, read on here: https://en.wikipedia.org/wiki/ISO_8601) It is used to communicate timezone aware datetime information in a serializable manner between systems.
When to use which? or rather when do you need to care about timezones?
If you need in any way to care about time-of-day, you need timezone information. A calendar or alarm needs time-of-day to set a meeting at the correct time of the day for any user in the world. If this data is saved on a server, the server needs to know what timezone the datetime corresponds to.
To compute time differences between events coming from different places in the world, UTC timestamp is enough, but you lose the ability to analyze at what time of day events occured (ie. for web analytics, you may want to know when users come to your site in their local time: do you see more users in the morning or the evening? You can't figure that out without time of day information.
Timezone offset in a date string:
Another point that is important, is that timezone offset in a date string is not fixed. That means that because 2017-08-30T10:55:24-0800 says the offset -0800 or 8 hours back, doesn't mean that it will always be!
In the summer it may well be in daylight saving time, and it would be -0700
What that means is that timezone offset (+0100) is not the same as timezone name (Europe/France) or even timezone designation (CET)
America/Los_Angeles timezone is a place in the world, but it turns into PST (Pacific Standard Time) timezone offset notation in the winter, and PDT (Pacific Daylight Time) in the summer.
So, on top of getting the timezone offset from the datestring, you should also get the timezone name to be accurate.
Most packages will be able to convert numeric offsets from daylight saving time to standard time on their own, but that is not necessarily trivial with just offset. For example WAT timezone designation in West Africa, is UTC+0100 just like CET timezone in France, but France observes daylight saving time, while West Africa does not (because they're close to the equator)
So, in short, it's complicated. VERY complicated, and that's why you should not do this yourself, but trust a package that does it for you, and KEEP IT UP TO DATE!
There is indeed a problem with using utcfromtimestamp and specifying time zones. A nice example/explanation is available on the following question:
How to specify time zone (UTC) when converting to Unix time? (Python)
The accepted answer seems not work for me. My solution:
import time
utc_0 = int(time.mktime(datetime(1970, 01, 01).timetuple()))
def datetime2ts(dt):
"""Converts a datetime object to UTC timestamp"""
return int(time.mktime(dt.utctimetuple())) - utc_0
Simplest way:
>>> from datetime import datetime
>>> dt = datetime(2008, 1, 1, 0, 0, 0, 0)
>>> dt.strftime("%s")
'1199163600'
Edit: #Daniel is correct, this would convert it to the machine's timezone. Here is a revised answer:
>>> from datetime import datetime, timezone
>>> epoch = datetime(1970, 1, 1, 0, 0, 0, 0, timezone.utc)
>>> dt = datetime(2008, 1, 1, 0, 0, 0, 0, timezone.utc)
>>> int((dt-epoch).total_seconds())
'1199145600'
In fact, its not even necessary to specify timezone.utc, because the time difference is the same so long as both datetime have the same timezone (or no timezone).
>>> from datetime import datetime
>>> epoch = datetime(1970, 1, 1, 0, 0, 0, 0)
>>> dt = datetime(2008, 1, 1, 0, 0, 0, 0)
>>> int((dt-epoch).total_seconds())
1199145600
I think the correct way to phrase your question is
Is there a way to get the timestamp by specifying the date in UTC?, because timestamp is just a number which is absolute, not relative. The relative (or timezone aware) piece is the date.
I find pandas very convenient for timestamps, so:
import pandas as pd
dt1 = datetime(2008, 1, 1, 0, 0, 0, 0)
ts1 = pd.Timestamp(dt1, tz='utc').timestamp()
# make sure you get back dt1
datetime.utcfromtimestamp(ts1)