DateTime format error in python while using aiofiles - python

These are my timestamp logs:
2023-02-16 21:38:59,873 : Device was cradled
2023-02-16 21:38:59,873 : Device connected to network, IP
I was trying to get them sorted with this:
from datetime import datetime as dt
sorted(files, key=lambda file: dt.strptime(file.split()[0],
"%Y-%m-%d %H:%M:%S"))
and getting an error:
ValueError: time data '2023-02-16' does not match format '%Y-%m-%d %H:%M:%S'

You might be better off installing one of the many log-parsing modules available instead of rolling your own, but in this case you can sort properly just by changing your split call.
For example, if your sample data is accurate, then there's a colon with space on either side of it between the timestamp and the log message, so you can split on that instead of space:
sorted(files, key=lambda line: dt.strptime(line.split(' : ')[0],
'%Y-%m-%d %H:%M:%S,%f'))

Related

Python convert string to datetime in variable formats

I am generating date and time information which is string from an API. The generated string is in your system's date/time format by default (Win 10 in my case). For example if you are using MM/DD/YYYY HH:MM:SS tt in your computer, the generated string would be something like "05/07/2019 06:00:00 AM".
For comparison purpose, I would then convert the string to datetime format by using datetime.datetime.strptime(i,"%m/%d/%Y %I:%M:%S %p"). This works prefectly fine, however if someone else whose system date/time format is different from' MM/DD/YYYY HH:MM:SS tt' runs my script, he would get a mismatch error as the string can no longer be converted by %m/%d/%Y %I:%M:%S %p.
So would it be possible to make the desired datetime format become a variable argument in the strptime function? Or even simpler, just make the format to be the same as the system's date/time format.
You can use try and except to fix this (although there might be another way)
try:
datetime.datetime.strptime(i,"%m/%d/%Y %I:%M:%S %p")
except TypeError:
try:
datetime.datetime.strptime(i,"%d/%m/%Y %I:%M:%S %p")
except:
try:
datetime.datetime.strptime(i,"%d/%m/%y %I:%M:%S %p")
# and so on....

Timestamp format iso with timezone and special format

I was trying some API e-commerce and i followed their example, but I keep getting Timestamp error / mismatch.
The API requires a timestamp in this format:
2017-08-29T09:54:00+07:00
I tried:
datetime.now().isoformat()
But it returns without the timezone:
'2017-08-29T09:54:57.702000'
then I tried:
datetime.now(pytz.timezone( 'Asia/Jakarta' )).isoformat()
but it returns:
2017-08-29T09:54:00.547000+07:00
Basically the python version doesn't use : for the minutes and seconds and it seems they also gives the microsecond which the API does not need. How do i get this format in python 2?
year-month-day then T hour-minute-second+Timezone
2017-08-29T09:54:00+07:00
I got it
datetime.now(pytz.timezone( 'Asia/Jakarta')).replace(microsecond=0).isoformat()

Converting timestamp to unix date python

Here is how the timestamp looks -
2015-07-17 06:01:51.066141+00:00
I'm looking around to convert this to unix date time.
datetime.strptime("2015-07-17 06:01:51.066141+00:00", "%Y-%m-%d %H:%M:%S.%f%z").strftime("%s")
ValueError: 'z' is a bad directive in format '%Y-%m-%d %H:%M:%S.%f%z'
throws error for me, probably because of wrong format being used.
PS: my virtualenv is on python 2.7
ideas please ?
python 2.7 strptime() does not support z directive, either you can use python 3.2+ or some other 3rd party library like dateutil.
For Python 2.7 use arrow:
import arrow
date_str = "2015-07-17 06:01:51.066141+00:00"
unix_time = arrow.get(date_str).timestamp
On PY3 (verified on 3.4), using only standard libs
The date string you show will not be parsed by the standard python datetime library since it has a colon in the timezone (see here). The colon can be easily removed since it's always in the same position (or use rfind to find its index starting from the right). Your simplest solution is:
import datetime
date_str = "2015-07-17 06:01:51.066141+00:00"
date_str_no_colon = date_str[:-3]+date_str[-2:] # remove last colon
dt_obj = datetime.datetime.strptime(date_str_no_colon, "%Y-%m-%d %H:%M:%S.%f%z")
unix_time = dt_obj.timestamp()
Note that arrow should still work with PY3, and is a better solution in general- you don't want to get into datetime parsing wars with python. It will win.
The way to parse the date is not right. You'll either need to parse it by hand, find another library (for example the dateutil.parser.parse method that will parse your string directly without supplying format string) that supports that format or make the timestamp in another format. Even with newer versions of python the %z directive does not accept TZ offsets in the +/-HH:MM format (with colon).
As the source of the timestamp is django.DateTimeField maybe this question can help?
For converting to unix timestamp you seem to have to do some work since there does not seem to be a direct method for that:
(t - datetime.utcfromtimestamp(0)).total_seconds()
where t is the datetime (assuming it's in UTC and there is no tzinfo) you want to convert to POSIX timestamp. If the assumption is not correct you need to put tzinfo in the zero timestamp you subtract as shown below where the assumption does not hold.
If you want to use dateutil.parser the complete solution would be:
(dateutil.parser.parse(timestamp) - datetime.utcfromtimestamp(0).replace(tzinfo=utc()).total_seconds()
strptime() has no support for timezones.
So, you can make the conversion ignoring the timezone in the following way:
datetime.strptime("2015-07-17 06:01:51.066141", "%Y-%m-%d %I:%M:%S.%f").strftime("%s")
'1437102111'
Or in order to avoid using %s, as suggested below in the commments :
from datetime import datetime
(datetime.strptime("2015-07-17 06:01:51.066141", "%Y-%m-%d %I:%M:%S.%f") - datetime(1970, 1, 1)).total_seconds()
1437112911.066141
Notice this is a working version for Python 2, you can also check solutions for other versions here
Otherwise, you will have to use other libraries (django.utils or email.utils) that support timezones, or implement the timezone parsing on your own.
P.S. :
strptime docs appear to have support for timezone, but in fact it has not been implemented. Try :
datetime.strptime("2015-07-17 06:01:51.066141+00:00", "%Y-%m-%d %I:%M:%S.%f%z").strftime("%s")
and you will see that it is not supported. You can also verify it by searching more about strptime()
There are two parts:
to convert "2015-07-17 06:01:51.066141+00:00" into a datetime object that represents UTC time, see Convert timestamps with offset to datetime obj using strptime. Or If you know that the utc offset is always +0000:
from datetime import datetime
utc_time = datetime.strptime(time_string, "%Y-%m-%d %H:%M:%S.%f+00:00")
to convert the UTC time to POSIX timestamp (unix time), see Converting datetime.date to UTC timestamp in Python:
from datetime import datetime
timestamp = (utc_time - datetime(1970, 1, 1)).total_seconds()

Django date format issue - does not match format

I need to change the date into the format MM/YY. The issue is I don't always don't what the date format is to begin with. It's my understanding that the code below would need to know what format the date was in first before I can change it is that correct?
newdate = datetime.datetime.strptime(str(mydate), '%Y/%m/%d').strftime('%m-%Y')
as a result I get the error:
time data '2020-09-30' does not match format '%Y/%m/%d' (
How to I convert my date to MM/YY?
I'm using Django 1.4.6
newdate = datetime.datetime.strptime(str(mydate), '%Y-%m-%d').strftime('%m/%Y')

how to convert and subtract dates, times in python

I have the following date/time:
2011-09-27 13:42:16
I need to convert it to:
9/27/2011 13:42:16
I also need to be able to subtract one date from another and get the result in HH:MM:SS format. I have tried to use the dateutil.parser.parse function, and it parses the date fine but sadly it doesn't seem to get the time correctly. I also tried to use another method I found on stackoverflow that uses "time", but I get an error that time is not defined.
You can use datetime's strptime function:
from datetime import datetime
date = '2011-09-27 13:42:16'
result = datetime.strptime(date, '%Y-%m-%d %H:%M:%S')
You were lucky, as I had that above line written for a project of mine.
To print it back out, try strftime:
print result.strftime('%m/%d/%Y %H:%M:%S')
Use python-dateutil:
import dateutil.parser as dateparser
mydate = dateparser.parse("2011-09-27 13:42:16",fuzzy=True)
print(mydate.strftime('%m/%d/%Y T%H:%M:%S'))
http://docs.python.org/library/datetime.html#datetime.datetime.strptime
and
http://docs.python.org/library/datetime.html#datetime.datetime.strftime
(And the rest of the datetime module.)

Categories