I have an important test that says "Calculate users that logged in during the month of April normalized to the UTC timezone."
Items look as such:
[ {u'email': u' ybartoletti#littel.biz',
u'login_date': u'2014-05-08T22:30:57-04:00'},
{u'email': u'woodie.crooks#kozey.com',
u'login_date': u'2014-04-25T13:27:48-08:00'},
]
It seems to me that an item like 2014-04-13T17:12:20-04:00 means "April 13th, 2014, at 5:12:20 pm, 4 hours behind UTC". Then I just use strptime to convert to datetime (Converting JSON date string to python datetime), and subtract a timedelta of however many hours I get from a regex that grabs the end of string? I feel this way because some have a + at the end instead of -, like 2014-05-07T00:30:06+07:00
Thank you
It is probably best to use the dateutil.parser.parse and pytz packages for this purpose. This will allow you to parse a string and convert it to a datetime object with UTC timezone:
>>> s = '2014-05-08T22:30:57-04:00'
>>> import dateutil.parser
>>> import pytz
>>> pytz.UTC.normalize(dateutil.parser.parse(s))
datetime.datetime(2014, 5, 9, 2, 30, 57, tzinfo=<UTC>)
You can use arrow to easily parse date with time zone.
>>>import arrow
>>> a = arrow.get('2014-05-08T22:30:57-04:00').to('utc')
>>> a
<Arrow [2014-05-09T02:30:57+00:00]>
Get a datetime object or timestamp:
>>> a.datetime
datetime.datetime(2014, 5, 9, 2, 30, 57, tzinfo=tzutc())
>>> a.naive
datetime.datetime(2014, 5, 9, 2, 30, 57)
>>> a.timestamp
1399602657
The following solution should be faster and avoids importing external libraries. The downside is that it will only work if the date strings are all guaranteed to have the specified format. If that's not the case, then I would prefer Simeon's solution, which lets dateutil.parser.parse() take care of any inconsistencies.
import datetime as dt
def parse_date(datestr):
diff = dt.timedelta(hours=int(datestr[20:22]), minutes=int(datestr[23:]))
if datestr[19] == '-':
return dt.datetime.strptime(datestr[:19], '%Y-%m-%dT%H:%M:%S') - diff
return dt.datetime.strptime(datestr[:19], '%Y-%m-%dT%H:%M:%S') + diff
Related
I have a date string and want to convert it to the date type:
I have tried to use datetime.datetime.strptime with the format that I want but it is returning the time with the conversion.
when = alldates[int(daypos[0])]
print when, type(when)
then = datetime.datetime.strptime(when, '%Y-%m-%d')
print then, type(then)
This is what the output returns:
2013-05-07 <type 'str'>
2013-05-07 00:00:00 <type 'datetime.datetime'>
I need to remove the time: 00:00:00.
print then.date()
What you want is a datetime.date object. What you have is a datetime.datetime object. You can either change the object when you print as per above, or do the following when creating the object:
then = datetime.datetime.strptime(when, '%Y-%m-%d').date()
If you need the result to be timezone-aware, you can use the replace() method of datetime objects. This preserves timezone, so you can do
>>> from django.utils import timezone
>>> now = timezone.now()
>>> now
datetime.datetime(2018, 8, 30, 14, 15, 43, 726252, tzinfo=<UTC>)
>>> now.replace(hour=0, minute=0, second=0, microsecond=0)
datetime.datetime(2018, 8, 30, 0, 0, tzinfo=<UTC>)
Note that this returns a new datetime object -- now remains unchanged.
>>> print then.date(), type(then.date())
2013-05-07 <type 'datetime.date'>
To convert a string into a date, the easiest way AFAIK is the dateutil module:
import dateutil.parser
datetime_object = dateutil.parser.parse("2013-05-07")
It can also handle time zones:
print(dateutil.parser.parse("2013-05-07"))
>>> datetime.datetime(2013, 5, 7, 1, 12, 12, tzinfo=tzutc())
If you have a datetime object, say:
import pytz
import datetime
now = datetime.datetime.now(pytz.UTC)
and you want chop off the time part, then I think it is easier to construct a new object instead of "substracting the time part". It is shorter and more bullet proof:
date_part datetime.datetime(now.year, now.month, now.day, tzinfo=now.tzinfo)
It also keeps the time zone information, it is easier to read and understand than a timedelta substraction, and you also have the option to give a different time zone in the same step (which makes sense, since you will have zero time part anyway).
For me, I needed to KEEP a timetime object because I was using UTC and it's a bit of a pain. So, this is what I ended up doing:
date = datetime.datetime.utcnow()
start_of_day = date - datetime.timedelta(
hours=date.hour,
minutes=date.minute,
seconds=date.second,
microseconds=date.microsecond
)
end_of_day = start_of_day + datetime.timedelta(
hours=23,
minutes=59,
seconds=59
)
Example output:
>>> date
datetime.datetime(2016, 10, 14, 17, 21, 5, 511600)
>>> start_of_day
datetime.datetime(2016, 10, 14, 0, 0)
>>> end_of_day
datetime.datetime(2016, 10, 14, 23, 59, 59)
If you specifically want a datetime and not a date but want the time zero'd out you could combine date with datetime.min.time()
Example:
datetime.datetime.combine(datetime.datetime.today().date(),
datetime.datetime.min.time())
You can use simply pd.to_datetime(then) and pandas will convert the date elements into ISO date format- [YYYY-MM-DD].
You can pass this as map/apply to use it in a dataframe/series too.
You can usee the following code:
week_start = str(datetime.today() - timedelta(days=datetime.today().weekday() % 7)).split(' ')[0]
I have a date string and want to convert it to the date type:
I have tried to use datetime.datetime.strptime with the format that I want but it is returning the time with the conversion.
when = alldates[int(daypos[0])]
print when, type(when)
then = datetime.datetime.strptime(when, '%Y-%m-%d')
print then, type(then)
This is what the output returns:
2013-05-07 <type 'str'>
2013-05-07 00:00:00 <type 'datetime.datetime'>
I need to remove the time: 00:00:00.
print then.date()
What you want is a datetime.date object. What you have is a datetime.datetime object. You can either change the object when you print as per above, or do the following when creating the object:
then = datetime.datetime.strptime(when, '%Y-%m-%d').date()
If you need the result to be timezone-aware, you can use the replace() method of datetime objects. This preserves timezone, so you can do
>>> from django.utils import timezone
>>> now = timezone.now()
>>> now
datetime.datetime(2018, 8, 30, 14, 15, 43, 726252, tzinfo=<UTC>)
>>> now.replace(hour=0, minute=0, second=0, microsecond=0)
datetime.datetime(2018, 8, 30, 0, 0, tzinfo=<UTC>)
Note that this returns a new datetime object -- now remains unchanged.
>>> print then.date(), type(then.date())
2013-05-07 <type 'datetime.date'>
To convert a string into a date, the easiest way AFAIK is the dateutil module:
import dateutil.parser
datetime_object = dateutil.parser.parse("2013-05-07")
It can also handle time zones:
print(dateutil.parser.parse("2013-05-07"))
>>> datetime.datetime(2013, 5, 7, 1, 12, 12, tzinfo=tzutc())
If you have a datetime object, say:
import pytz
import datetime
now = datetime.datetime.now(pytz.UTC)
and you want chop off the time part, then I think it is easier to construct a new object instead of "substracting the time part". It is shorter and more bullet proof:
date_part datetime.datetime(now.year, now.month, now.day, tzinfo=now.tzinfo)
It also keeps the time zone information, it is easier to read and understand than a timedelta substraction, and you also have the option to give a different time zone in the same step (which makes sense, since you will have zero time part anyway).
For me, I needed to KEEP a timetime object because I was using UTC and it's a bit of a pain. So, this is what I ended up doing:
date = datetime.datetime.utcnow()
start_of_day = date - datetime.timedelta(
hours=date.hour,
minutes=date.minute,
seconds=date.second,
microseconds=date.microsecond
)
end_of_day = start_of_day + datetime.timedelta(
hours=23,
minutes=59,
seconds=59
)
Example output:
>>> date
datetime.datetime(2016, 10, 14, 17, 21, 5, 511600)
>>> start_of_day
datetime.datetime(2016, 10, 14, 0, 0)
>>> end_of_day
datetime.datetime(2016, 10, 14, 23, 59, 59)
If you specifically want a datetime and not a date but want the time zero'd out you could combine date with datetime.min.time()
Example:
datetime.datetime.combine(datetime.datetime.today().date(),
datetime.datetime.min.time())
You can use simply pd.to_datetime(then) and pandas will convert the date elements into ISO date format- [YYYY-MM-DD].
You can pass this as map/apply to use it in a dataframe/series too.
You can usee the following code:
week_start = str(datetime.today() - timedelta(days=datetime.today().weekday() % 7)).split(' ')[0]
I want to know how to convert this date format
"Thu 21st Aug '14, 4:58am"
to a timestamp with Python?
Another format that I need to convert:
"Yesterday, 7:22am"
I tried parse util without success...
If you haven't done so already, have a look at the parse function in dateutils.parser for parsing strings representing dates...
>>> from dateutil.parser import parse
>>> dt = parse("Thu 21st Aug '14, 4:58am")
>>> dt
datetime.datetime(2014, 8, 21, 4, 58)
...and then to convert a datetime object to a timestamp, you can do the following:
>>> import time
>>> import datetime
>>> time.mktime(dt.timetuple())
1408593480.0
As side remark, parse is a useful function which can recognise a huge range of different date formats. However it's sometimes too helpful and sees dates where perhaps a date is not intended:
>>> parse("14, m 23")
datetime.datetime(2014, 8, 23, 0, 14)
If you also want to parse expressions such as "Yesterday, 7:22am", you could do one of two things:
Replace "yesterday", "yester-day", "yday" and other variations with "25/08/2014" (or another appropriate date) and then use parse on the new string.
Use another library to parse the string. parsedatetime is one option...
Here's parsedatetime in action on your example:
>>> import parsedatetime.parsedatetime as pdt
>>> p = pdt.Calendar()
>>> d = p.parse("Yesterday, 7:22am")
>>> d
((2014, 8, 25, 7, 22, 0, 0, 237, -1), 3)
To turn this date representation d into a datetime object, you can unpack the tuple like so:
>>> dt = datetime.datetime(*d[0][:7])
>>> dt
datetime.datetime(2014, 8, 25, 7, 22)
Now dt can be easily converted to a timestamp in the way described above.
You can use this:
a = "Thu 21st Aug '14, 4:58am"
datetime.datetime.strptime(a, '%a %dst %b \'%y, %H:%M%p')
Why python 2.7 doesn't include Z character (Zulu or zero offset) at the end of UTC datetime object's isoformat string unlike JavaScript?
>>> datetime.datetime.utcnow().isoformat()
'2013-10-29T09:14:03.895210'
Whereas in javascript
>>> console.log(new Date().toISOString());
2013-10-29T09:38:41.341Z
Option: isoformat()
Python's datetime does not support the military timezone suffixes like 'Z' suffix for UTC. The following simple string replacement does the trick:
In [1]: import datetime
In [2]: d = datetime.datetime(2014, 12, 10, 12, 0, 0)
In [3]: str(d).replace('+00:00', 'Z')
Out[3]: '2014-12-10 12:00:00Z'
str(d) is essentially the same as d.isoformat(sep=' ')
See: Datetime, Python Standard Library
Option: strftime()
Or you could use strftime to achieve the same effect:
In [4]: d.strftime('%Y-%m-%dT%H:%M:%SZ')
Out[4]: '2014-12-10T12:00:00Z'
Note: This option works only when you know the date specified is in UTC.
See: datetime.strftime()
Additional: Human Readable Timezone
Going further, you may be interested in displaying human readable timezone information, pytz with strftime %Z timezone flag:
In [5]: import pytz
In [6]: d = datetime.datetime(2014, 12, 10, 12, 0, 0, tzinfo=pytz.utc)
In [7]: d
Out[7]: datetime.datetime(2014, 12, 10, 12, 0, tzinfo=<UTC>)
In [8]: d.strftime('%Y-%m-%d %H:%M:%S %Z')
Out[8]: '2014-12-10 12:00:00 UTC'
Python datetime objects don't have time zone info by default, and without it, Python actually violates the ISO 8601 specification (if no time zone info is given, assumed to be local time). You can use the pytz package to get some default time zones, or directly subclass tzinfo yourself:
from datetime import datetime, tzinfo, timedelta
class simple_utc(tzinfo):
def tzname(self,**kwargs):
return "UTC"
def utcoffset(self, dt):
return timedelta(0)
Then you can manually add the time zone info to utcnow():
>>> datetime.utcnow().replace(tzinfo=simple_utc()).isoformat()
'2014-05-16T22:51:53.015001+00:00'
Note that this DOES conform to the ISO 8601 format, which allows for either Z or +00:00 as the suffix for UTC. Note that the latter actually conforms to the standard better, with how time zones are represented in general (UTC is a special case.)
Short answer
datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
Long answer
The reason that the "Z" is not included is because datetime.now() and even datetime.utcnow() return timezone naive datetimes, that is to say datetimes with no timezone information associated. To get a timezone aware datetime, you need to pass a timezone as an argument to datetime.now. For example:
from datetime import datetime, timezone
datetime.utcnow()
#> datetime.datetime(2020, 9, 3, 20, 58, 49, 22253)
# This is timezone naive
datetime.now(timezone.utc)
#> datetime.datetime(2020, 9, 3, 20, 58, 49, 22253, tzinfo=datetime.timezone.utc)
# This is timezone aware
Once you have a timezone aware timestamp, isoformat will include a timezone designation. Thus, you can then get an ISO 8601 timestamp via:
datetime.now(timezone.utc).isoformat()
#> '2020-09-03T20:53:07.337670+00:00'
"+00:00" is a valid ISO 8601 timezone designation for UTC. If you want to have "Z" instead of "+00:00", you have to do the replacement yourself:
datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
#> '2020-09-03T20:53:07.337670Z'
The following javascript and python scripts give identical outputs. I think it's what you are looking for.
JavaScript
new Date().toISOString()
Python
from datetime import datetime
datetime.utcnow().isoformat()[:-3]+'Z'
The output they give is the UTC (zulu) time formatted as an ISO string with a 3 millisecond significant digit and appended with a Z.
2019-01-19T23:20:25.459Z
Your goal shouldn't be to add a Z character, it should be to generate a UTC "aware" datetime string in ISO 8601 format. The solution is to pass a UTC timezone object to datetime.now() instead of using datetime.utcnow():
from datetime import datetime, timezone
datetime.now(timezone.utc)
>>> datetime.datetime(2020, 1, 8, 6, 6, 24, 260810, tzinfo=datetime.timezone.utc)
datetime.now(timezone.utc).isoformat()
>>> '2020-01-08T06:07:04.492045+00:00'
That looks good, so let's see what Django and dateutil think:
from django.utils.timezone import is_aware
is_aware(datetime.now(timezone.utc))
>>> True
from dateutil.parser import isoparse
is_aware(isoparse(datetime.now(timezone.utc).isoformat()))
>>> True
Note that you need to use isoparse() from dateutil.parser because the Python documentation for datetime.fromisoformat() says it "does not support parsing arbitrary ISO 8601 strings".
Okay, the Python datetime object and the ISO 8601 string are both UTC "aware". Now let's look at what JavaScript thinks of the datetime string. Borrowing from this answer we get:
let date = '2020-01-08T06:07:04.492045+00:00';
const dateParsed = new Date(Date.parse(date))
document.write(dateParsed);
document.write("\n");
// Tue Jan 07 2020 22:07:04 GMT-0800 (Pacific Standard Time)
document.write(dateParsed.toISOString());
document.write("\n");
// 2020-01-08T06:07:04.492Z
document.write(dateParsed.toUTCString());
document.write("\n");
// Wed, 08 Jan 2020 06:07:04 GMT
Notes:
I approached this problem with a few goals:
generate a UTC "aware" datetime string in ISO 8601 format
use only Python Standard Library functions for datetime object and string creation
validate the datetime object and string with the Django timezone utility function, the dateutil parser and JavaScript functions
Note that this approach does not include a Z suffix and does not use utcnow(). But it's based on the recommendation in the Python documentation and it passes muster with both Django and JavaScript.
See also:
Stop using utcnow and utcfromtimestamp
What is the “right” JSON date format?
In Python >= 3.2 you can simply use this:
>>> from datetime import datetime, timezone
>>> datetime.now(timezone.utc).isoformat()
'2019-03-14T07:55:36.979511+00:00'
Python datetimes are a little clunky. Use arrow.
> str(arrow.utcnow())
'2014-05-17T01:18:47.944126+00:00'
Arrow has essentially the same api as datetime, but with timezones and some extra niceties that should be in the main library.
A format compatible with Javascript can be achieved by:
arrow.utcnow().isoformat().replace("+00:00", "Z")
'2018-11-30T02:46:40.714281Z'
Javascript Date.parse will quietly drop microseconds from the timestamp.
I use pendulum:
import pendulum
d = pendulum.now("UTC").to_iso8601_string()
print(d)
>>> 2019-10-30T00:11:21.818265Z
There are a lot of good answers on the post, but I wanted the format to come out exactly as it does with JavaScript. This is what I'm using and it works well.
In [1]: import datetime
In [1]: now = datetime.datetime.utcnow()
In [1]: now.strftime('%Y-%m-%dT%H:%M:%S') + now.strftime('.%f')[:4] + 'Z'
Out[3]: '2018-10-16T13:18:34.856Z'
Using only standard libraries, making no assumption that the timezone is already UTC, and returning the exact format requested in the question:
dt.astimezone(timezone.utc).replace(tzinfo=None).isoformat(timespec='milliseconds') + 'Z'
This does require Python 3.6 or later though.
>>> import arrow
>>> now = arrow.utcnow().format('YYYY-MM-DDTHH:mm:ss.SSS')
>>> now
'2018-11-28T21:34:59.235'
>>> zulu = "{}Z".format(now)
>>> zulu
'2018-11-28T21:34:59.235Z'
Or, to get it in one fell swoop:
>>> zulu = "{}Z".format(arrow.utcnow().format('YYYY-MM-DDTHH:mm:ss.SSS'))
>>> zulu
'2018-11-28T21:54:49.639Z'
By combining all answers above I came with following function :
from datetime import datetime, tzinfo, timedelta
class simple_utc(tzinfo):
def tzname(self,**kwargs):
return "UTC"
def utcoffset(self, dt):
return timedelta(0)
def getdata(yy, mm, dd, h, m, s) :
d = datetime(yy, mm, dd, h, m, s)
d = d.replace(tzinfo=simple_utc()).isoformat()
d = str(d).replace('+00:00', 'Z')
return d
print getdata(2018, 02, 03, 15, 0, 14)
pip install python-dateutil
>>> a = "2019-06-27T02:14:49.443814497Z"
>>> dateutil.parser.parse(a)
datetime.datetime(2019, 6, 27, 2, 14, 49, 443814, tzinfo=tzutc())
I have numerous UTC time stamps in the following format:
2012-04-30T23:08:56+00:00
I want to convert them to python datetime objects but am having trouble.
My code:
for time in data:
pythondata[i]=datetime.strptime(time,"%y-%m-%dT%H:%M:%S+00:00")
I get the following error:
ValueError: time data '2012-03-01T00:05:55+00:00' does not match format '%y-%m-%dT%H:%M:%S+00:00'
It looks like I have the proper format, so why doesn't this work?
Change the year marker in your time format string to %Y:
time = '2012-03-01T00:05:55+00:00'
datetime.strptime(time, "%Y-%m-%dT%H:%M:%S+00:00")
# => datetime.datetime(2012, 3, 1, 0, 5, 55)
See strftime() and strptime() behavior.
I highly recommend python-dateutil library, it allows conversion of multiple datetime formats from raw strings into datetime objects with/without timezone set
>>> from dateutil.parser import parse
>>> parse('2012-04-30T23:08:56+00:00')
datetime.datetime(2012, 4, 30, 23, 8, 56, tzinfo=tzutc())