Does Python str(datetime Object) return a string that cannot be readily parsed with strptime? Or do I miss something:
from datetime import datetime
import pytz
tz = pytz.timezone('America/Los_Angeles')
now = tz.localize(datetime.now())
print(str(now))
The above will return a time formatted like this 2020-02-11 12:38:35.110566-08:00. There is a colon in the offset part of the time. But looking at the string formats here https://strftime.org/ there is no format to read that in because 2020-02-11 12:38:35.110566-0800 would be required.
I am aware that str() is not the correct way to convert datetime to str (even though that would be convenient). However, I am dealing with an upstream API that provides the time format that way. Is there a way to convert that time without additional string manipulation. Like:
tm = datetime.strptime('2020-02-11 12:38:35.110566-08:00', '%Y-%m-%d %H:%M:%S.%f%z')
Which throws a ValueError. Python version is 3.6.7. Would you consider that a bug in pytz?
Related
According to the documentation, this is how the datetime type is returned:
A Python datetime.datetime. In requests and responses will be
represented as a str in ISO 8601 format, like:
2008-09-15T15:53:00+05:00
I'm having difficulties dealing with that date format on my front end. Is it possible to force FastAPI to return a DateTime so it only shows date + hours/minutes/seconds?
It's somewhat of a weird use case, but when converting a time-like value with pd.to_datetime pandas uses different values for the year, month, and day depending upon whether the format is specified.
pd.to_datetime('02:12:11', format='%H:%M:%S')
#Timestamp('1900-01-01 02:12:11')
pd.to_datetime('02:12:11')
#Timestamp('2021-03-17 02:12:11')
I had assumed that 1900-01-01T00:00:00.000 would be used to fill missing components in all cases, consistent with the datetime Technical Detail, (what the pandas docs link to) but somehow it decides to use 'today' to get the other components without a format.
Is this documented somewhere or expected behavior?
If I understand the source code correctly, if no datetime.datetime object is passed to objects_to_datetime64ns then it defaults to datetime.now unless you specifiy the format.
I have to say going through the source code gives me a greater respect to the core maintainers of the project.
python3.8 > site-packages > dateutil > parser._parser.py
using pd.to_datetime('01:04:06') as an example.
if parserinfo:
return parser(parserinfo).parse(timestr, **kwargs)
else:
return DEFAULTPARSER.parse(timestr, **kwargs)
# the line of code in question.
if default is None:
default = datetime.datetime.now().replace(hour=0, minute=0,
second=0, microsecond=0)
This, from what i understand returns datetime.datetime(2021, 3, 17, 1, 4, 6) to
result, tz_parsed = tslib.array_to_datetime(
data,
errors=errors,
utc=utc,
dayfirst=dayfirst,
yearfirst=yearfirst,
require_iso8601=require_iso8601,
)
Which returns your datetime object.
As if this is intended behavior I would urge on the side of yes as the code has been written in such a manner, however until it's made clear in the documents I would not give absolute certainty as pandas relies on the standard python lib for this conversion.
call stack for more information :
pandas depends on other libraries to do parsing, including python's datetime library and the dateutil package.
import pandas as pd
from datetime import datetime as dt
from dateutil import parser
print("pandas datetime with format ",pd.to_datetime('02:12:11', format='%H:%M:%S'))
print("datetime with format ", dt.strptime('02:12:11', '%H:%M:%S'))
print("pandas datetime without format ", pd.to_datetime('02:12:11'))
print("dateutil parser without format ", parser.parse('02:12:11'))
produces
pandas datetime with format 1900-01-01 02:12:11
datetime with format 1900-01-01 02:12:11
pandas datetime without format 2021-03-17 02:12:11
dateutil parser without format 2021-03-17 02:12:11
So the issue here is that the dateutil package uses the current date as the default, while strptime() uses the Unix origin.
Here is how the timestamp looks -
2015-07-17 06:01:51.066141+00:00
I'm looking around to convert this to unix date time.
datetime.strptime("2015-07-17 06:01:51.066141+00:00", "%Y-%m-%d %H:%M:%S.%f%z").strftime("%s")
ValueError: 'z' is a bad directive in format '%Y-%m-%d %H:%M:%S.%f%z'
throws error for me, probably because of wrong format being used.
PS: my virtualenv is on python 2.7
ideas please ?
python 2.7 strptime() does not support z directive, either you can use python 3.2+ or some other 3rd party library like dateutil.
For Python 2.7 use arrow:
import arrow
date_str = "2015-07-17 06:01:51.066141+00:00"
unix_time = arrow.get(date_str).timestamp
On PY3 (verified on 3.4), using only standard libs
The date string you show will not be parsed by the standard python datetime library since it has a colon in the timezone (see here). The colon can be easily removed since it's always in the same position (or use rfind to find its index starting from the right). Your simplest solution is:
import datetime
date_str = "2015-07-17 06:01:51.066141+00:00"
date_str_no_colon = date_str[:-3]+date_str[-2:] # remove last colon
dt_obj = datetime.datetime.strptime(date_str_no_colon, "%Y-%m-%d %H:%M:%S.%f%z")
unix_time = dt_obj.timestamp()
Note that arrow should still work with PY3, and is a better solution in general- you don't want to get into datetime parsing wars with python. It will win.
The way to parse the date is not right. You'll either need to parse it by hand, find another library (for example the dateutil.parser.parse method that will parse your string directly without supplying format string) that supports that format or make the timestamp in another format. Even with newer versions of python the %z directive does not accept TZ offsets in the +/-HH:MM format (with colon).
As the source of the timestamp is django.DateTimeField maybe this question can help?
For converting to unix timestamp you seem to have to do some work since there does not seem to be a direct method for that:
(t - datetime.utcfromtimestamp(0)).total_seconds()
where t is the datetime (assuming it's in UTC and there is no tzinfo) you want to convert to POSIX timestamp. If the assumption is not correct you need to put tzinfo in the zero timestamp you subtract as shown below where the assumption does not hold.
If you want to use dateutil.parser the complete solution would be:
(dateutil.parser.parse(timestamp) - datetime.utcfromtimestamp(0).replace(tzinfo=utc()).total_seconds()
strptime() has no support for timezones.
So, you can make the conversion ignoring the timezone in the following way:
datetime.strptime("2015-07-17 06:01:51.066141", "%Y-%m-%d %I:%M:%S.%f").strftime("%s")
'1437102111'
Or in order to avoid using %s, as suggested below in the commments :
from datetime import datetime
(datetime.strptime("2015-07-17 06:01:51.066141", "%Y-%m-%d %I:%M:%S.%f") - datetime(1970, 1, 1)).total_seconds()
1437112911.066141
Notice this is a working version for Python 2, you can also check solutions for other versions here
Otherwise, you will have to use other libraries (django.utils or email.utils) that support timezones, or implement the timezone parsing on your own.
P.S. :
strptime docs appear to have support for timezone, but in fact it has not been implemented. Try :
datetime.strptime("2015-07-17 06:01:51.066141+00:00", "%Y-%m-%d %I:%M:%S.%f%z").strftime("%s")
and you will see that it is not supported. You can also verify it by searching more about strptime()
There are two parts:
to convert "2015-07-17 06:01:51.066141+00:00" into a datetime object that represents UTC time, see Convert timestamps with offset to datetime obj using strptime. Or If you know that the utc offset is always +0000:
from datetime import datetime
utc_time = datetime.strptime(time_string, "%Y-%m-%d %H:%M:%S.%f+00:00")
to convert the UTC time to POSIX timestamp (unix time), see Converting datetime.date to UTC timestamp in Python:
from datetime import datetime
timestamp = (utc_time - datetime(1970, 1, 1)).total_seconds()
I am attempting to convert the following date (2012-12-25T08:52:00-05:00) to a datetime object in python. However, I cannot figure out what the -05:00 part of the date is referencing. I am simply trying to perform the following:
datetime.datetime.strptime('2012-12-25T08:52:00-05:00','%Y-%m-%dT%H:%M:%S')
But this comes up with an expected 'ValueError: unconverted data remains'. I'm just trying to figure out what the last part of the date is used for so that I can convert that string to a proper datetime object in python.
Happy Holidays!
Your date seems to be in the ISO 8601 format, I don't think datetime handles the timezone information at the end of the string format.
You can use pip install python-dateutil, its parser can return a datetime object :
import dateutil.parser
datestr = '2012-12-25T08:52:00-05:00'
dateutil.parser.parse(datestr)
>>> datetime.datetime(2012, 12, 25, 8, 52, tzinfo=tzoffset(None, -18000))
The -05:00 indicates the timezone offset from UTC, i.e. %z would be the correct strptime argument to parse it.
If the time is UTC the offset might be indicated using Z, e.g. 2012-12-25T08:52:00Z. Not sure if %z would actually accept this...
I need to convert a date from a string (entered into a url) in the form of 12/09/2008-12:40:49. Obviously, I'll need a UNIX Timestamp at the end of it, but before I get that I need the Date object first.
How do I do this? I can't find any resources that show the date in that format? Thank you.
You need the strptime method. If you're on Python 2.5 or higher, this is a method on datetime, otherwise you have to use a combination of the time and datetime modules to achieve this.
Python 2.5 up:
from datetime import datetime
dt = datetime.strptime(s, "%d/%m/%Y-%H:%M:%S")
below 2.5:
from datetime import datetime
from time import strptime
dt = datetime(*strptime(s, "%d/%m/%Y-%H:%M:%S")[0:6])
You can use the time.strptime() method to parse a date string. This will return a time_struct that you can pass to time.mktime() (when the string represents a local time) or calendar.timegm() (when the string is a UTC time) to get the number of seconds since the epoch.