I have a date formated like this: "Thu Jan 05 17:42:26 MST 2023" and need to change it to be formated like this: "2023-01-06T04:58:00Z".
I thought this would be really easy. Parse the date with datetime.datetime.strptime(), adjust the timezone with datetime.datetime.astimezone(pytz.utc) and output it with datetime.datetime.strftime().
On my machine running Python 3.10 it is that simple. Even though strptime discards timezone, astimezone(pytz.utc) still works for some reason and outputs the correct time format.
On the server, running Python 2.7 it throws "ValueError: astimezone() requires an aware datetime". So I slice the timezone out of the initial string and localize the datetime. Works great except pytz.timezone() cannot parse daylight savings timezones like MDT. I know there is a parameter for DST but then I have to manually parse the timezone string to figure out DST.
The only way I have gotten this work work properly is by making a dictionary of every timezone and their offsets, parsing the intial string into a epoch timestamp, applying the offset, then formatting back into my desired string.
What am I doing wrong? Why is this so insanely difficult?
# Works fine on Python 3.10 but returns error on Python 2.7
def _formatTimestamp1(timestamp):
local_dt = datetime.strptime(str(timestamp), '%a %b %d %H:%M:%S %Z %Y')
utc_dt = local_dt.astimezone(pytz.utc)
fmt_dt = utc_dt.strftime("%Y-%m-%dT%H:%M:%SZ")
return str(fmt_dt)
# Works good but pytz.timezone() cannot handle paring DST times like 'MDT'
def _formatTimestamp2(timestamp):
local_dt = datetime.strptime(str(timestamp), '%a %b %d %H:%M:%S %Z %Y')
timezone = pytz.timezone(timestamp[20:-5])
aware = timezone.localize(local_dt)
utc_dt = aware.astimezone(pytz.utc)
fmt_dt = utc_dt.strftime("%Y-%m-%dT%H:%M:%SZ")
return str(fmt_dt)
# So far this is the best method, but requires building a timezone database
def _formatTimestamp3(timestamp):
tz_dict = {"MST":-7,"MDT":-6}
local_dt = datetime.strptime(str(timestamp), '%a %b %d %H:%M:%S %Z %Y')
utc_dt = datetime.fromtimestamp(local_dt.timestamp() - tz_dict[timestamp[20:-5]] * 60 * 60)
fmt_dt = utc_dt.strftime("%Y-%m-%dT%H:%M:%SZ")
return str(fmt_dt)
Related
I am trying to convert using strptime and for the life of me cannot figure this out
'07-17-2019 23:39 PM GMT-4' does not match format '%m-%d-%y %H:%M %p %z'
Thank you for any help
So far I've found that %y should be %Y. The best way to approach this is to test it a bit at a time. Like starting with datetime.strptime('07-17-2019', '%m-%d-%Y'). There's also something wrong with the GMT-4. %z will match -0400 fine, and %Z will match UTC, EST, and others, which might be better than an offset if you want to include daylight savings time, but it looks like that's all you get with strptime.
dateutil.parser.parse might provide you with more options and flexibility.
I found a way to do this, which is not super flexible, but should have a bit of give.
Firstly, a few points:
%y should be %Y, to match a 4-digit year.
Using a 24-hour time with AM/PM is confusing, so let's just ignore the AM/PM.
GMT-4 is not a standard timezone name, so we need to handle it manually.
#!/usr/bin/env python3
import datetime
s = '07-17-2019 23:39 PM GMT-4'
fmt = '%m-%d-%Y %H:%M' # Excludes AM/PM and timezone on purpose
words = s.split()
# Get just date and time as a string.
s_dt = ' '.join(words[:2])
# Convert date and time to datetime, tz-unaware.
unaware = datetime.datetime.strptime(s_dt, fmt)
# Get GMT offset as int - assumes whole hour.
gmt_offset = int(words[3].replace('GMT', ''))
# Convert GMT offset to timezone.
tz = datetime.timezone(datetime.timedelta(hours=gmt_offset))
# Add timezone to datetime
aware = unaware.replace(tzinfo=tz)
print(aware) # -> 2019-07-17 23:39:00-04:00
P.s. I used these to reverse-engineer a bit of it
tz = datetime.timezone(datetime.timedelta(hours=-4))
dt = datetime.datetime(2019, 7, 17, 23, 39, tzinfo=tz)
I'm currently trying to convert a file format into a slightly different style to allow easier importing into a program however I can't quite get my head around how to convert datetime strings between formats. The original I have is the following:
2016-12-15 17:26:45
However the required format for the date time is:
Thu Dec 15 17:19:03 2016
Does anyone know if there is an easy way to convert between these? These values are always in the same place and format so it doesn't need to be too dynamic so to speak outside of recognising what a certain day of the month is (if that can be done at all?)
Update - The conversion has worked for 1 date but not the other weirdly :/ The code to grab the two dates is the following:
startDate=startDate.replace("Started : ","")
startDate=startDate.replace(" (ISO format YYYY-MM-DD HH:MM:SS)","")
startDate=startDate.strip()
startDt = datetime.strptime(startDate, '%Y-%m-%d %H:%M:%S')
startDt=startDt.strftime('%a %b %d %H:%M:%S %Y ')
print (startDt)
This part works as inteded and outputs the required format:
"2016-12-15 17:26:45
Thu Dec 15 17:26:45 2016"
The end date part is a bit "ham fisted" so to speak and I'm sure there are better ways to do the re.sub search just to do anything in brackets but I'll edit that later.
endDate=endDate.replace("Ended : ","")
endDate=endDate.strip()
endDate = re.sub("\(.*?\)", "", endDate)
endDate.strip()
endDt = datetime.strptime(endDate, '%Y-%m-%d %H:%M:%S')
endDt=endDt.strftime('%a %b %d %H:%M:%S %Y ')
print (endDt)
This part however despite the outputs being an identical format
"2016-12-15 17:26:45
2016-12-15 21:22:11"
produces the following error:
endDt = datetime.strptime(endDate, '%Y-%m-%d %H:%M:%S')
File "C:\Python27\lib\_strptime.py", line 335, in _strptime
data_string[found.end():])
ValueError: unconverted data remains:
from datetime import datetime
dt = datetime.strptime('2016-06-01 1:33:45', '%Y-%m-%d %H:%M:%S')
dt.strftime('%a %b %d %H:%M:%S %Y ')
>>> 'Wed Jun 01 01:33:45 2016'
It's a pretty easy task with the Datetime module.
As it's been pointed out, checking the docs will get you a lot of useful info, starting from the directives to feed to the strptime and strftime (respectively, parse and format time) functions which you'll need here.
A working example for you case would be:
from datetime import datetime
myDateString = '2016-12-15 17:26:45'
myDateObj = datetime.strptime(myDateString, '%Y-%m-%d %H:%M:%S')
myDateFormat = myDateObj.strftime('%a %b %d %H:%M:%S %Y')
Check out this section of the docs to have a better understanding of the formatting placeholders.
You can use the datetime module:
from datetime import datetime
string = '2016-12-15 17:26:45'
date = datetime.strptime(string, '%Y-%m-%d %H:%M:%S')
date2 = date.strftime("%a %b %d %H:%M:%S %Z %Y")
print(date2)
Output:
Thu Dec 15 17:26:45 2016
I am trying to write a function that will convert a string date/time from local time to UTC in Python.
According to this question, you can use time.tzname to get some forms of the local timezone, but I have not found a way to use this in any of the datetime conversion methods. For example, this article shows there are a couple of things you can do with pytz and datetime to convert times, but all of them have timezones that are hardcoded in and are of different formats than what time.tznamereturns.
Currently I have the following code to translate a string-formatted time into milliseconds (Unix epoch):
local_time = time.strptime(datetime_str, "%m/%d/%Y %H:%M:%S") # expects UTC, but I want this to be local
dt = datetime.datetime(*local_time[:6])
ms = int((dt - datetime.datetime.utcfromtimestamp(0)).total_seconds() * 1000)
However, this is expecting the time to be input as UTC. Is there a way to convert the string formatted time as if it were in the local timezone? Thanks.
Essentially, I want to be able to do what this answer does, but instead of hard-coding in "America/Los_Angeles", I want to be able to dynamically specify the local timezone.
If I understand your question correctly you want this :
from time import strftime,gmtime,mktime,strptime
# you can pass any time you want
strftime("%Y-%m-%d %H:%M:%S", gmtime(mktime(strptime("Thu, 30 Jun 2016 03:12:40", "%a, %d %b %Y %H:%M:%S"))))
# and here for real time
strftime("%Y-%m-%d %H:%M:%S", gmtime(mktime(strptime(strftime("%a, %d %b %Y %H:%M:%S"), "%a, %d %b %Y %H:%M:%S"))))
make a time structure from a timetuple then use the structure to create a utc time
from datetime import datetime
def local_to_utc(local_st):
time_struct = time.mktime(local_st)
utc_st = datetime.utcfromtimestamp(time_struct)
return utc_st
d=datetime(2016,6,30,3,12,40,0)
timeTuple = d.timetuple()
print(local_to_utc(timeTuple))
output:
2016-06-30 09:12:40
I have a string time coming from a third party (external to my python program), and I need to compare that time to right now. How long ago was that time?
How can I do this?
I've looked at the datetime and time libraries, as well as pytz, and can't find an obvious way to do this. It should automatically incorporate DST because the third party doesn't explicitly state its offset, only the timezone (US/Eastern).
I've tried this, and it fails:
dt = datetime.datetime.strptime('June 10, 2016 12:00PM', '%B %d, %Y %I:%M%p')
dtEt = dt.replace(tzinfo=pytz.timezone('US/Eastern'))
now = datetime.datetime.now()
now - dtEt
TypeError: can't subtract offset-naive and offset-aware datetimes
Good question Zack! I've had this problem myself.
Here's some code to do so:
from datetime import datetime
import time
import calendar
import pytz
def howLongAgo(thirdPartyString, timeFmt):
# seconds since epoch
thirdPartySeconds = calendar.timegm(time.strptime(thirdPartyString, timeFmt))
nowSecondsUTC = time.time()
# hour difference with DST
nowEastern = datetime.now(pytz.timezone('US/Eastern'))
nowUTC = datetime.now(pytz.timezone('UTC'))
timezoneOffset = (nowEastern.day - nowUTC.day)*24 + (nowEastern.hour - nowUTC.hour) + (nowEastern.minute - nowUTC.minute)/60.0
thirdPartySecondsUTC = thirdPartySeconds - (timezoneOffset * 60 * 60)
return nowSecondsUTC - thirdPartySecondsUTC
howLongAgo('June 09, 2016 at 06:22PM', '%B %d, %Y at %I:%M%p')
# first argument always provided in ET, either EDT or EST
TypeError: can't subtract offset-naive and offset-aware datetimes
To fix the TypeError, use timezone aware datetime objects:
#!/usr/bin/env python
from datetime import datetime
import pytz # $ pip install pytz
tz = pytz.timezone('US/Eastern')
now = datetime.now(tz) # the current time (it works even during DST transitions)
then_naive = datetime.strptime('June 10, 2016 12:00PM', '%B %d, %Y %I:%M%p')
then = tz.localize(then_naive, is_dst=None)
time_difference_in_seconds = (now - then).total_seconds()
is_dst=None causes an exception for ambiguous/non-existing times. You could also use is_dst=False (default) or is_dst=True, see python converting string in localtime to UTC epoch timestamp.
I need to store a timestamp in a readable format, and then later on I need to convert it to epoch for comparison purposes.
I tried doing this:
import time
format = '%Y %m %d %H:%M:%S +0000'
timestamp1 = time.strftime(format,time.gmtime()) # '2016 03 25 04:06:22 +0000'
t1 = time.strptime(timestamp1, format) # time.struct_time(..., tm_isdst=-1)
time.sleep(1)
epoch_now = time.mktime(time.gmtime())
epoch_t1 = time.mktime(t1)
print "Delta: %s" % (epoch_now - epoch_t1)
Running this, instead of getting Delta of 1 sec, I get 3601 (1 hr 1 sec), CONSISTENTLY.
Investigating further, it seems that when I just do time.gmtime(), the struct has tm_isdst=0, whereas the converted struct t1 from timestamp1 string has tm_isdst=-1.
How can I ensure the isdst is preserved to 0. I think that's probably the issue here.
Or is there a better way to record time in human readable format (UTC), and yet be able to convert back to epoch properly for time diff calculation?
UPDATES:
After doing more research last night, I switched to using datetime because it preserves more information in the datetime object, and this is confirmed by albertoql answer below.
Here's what I have now:
from datetime import datetime
format = '%Y-%m-%d %H:%M:%S.%f +0000' # +0000 is optional; only for user to see it's UTC
d1 = datetime.utcnow()
timestamp1 = d1.strftime(format)
d1a = datetime.strptime(timestamp1, format)
time.sleep(1)
d2 = datetime.utcnow()
print "Delta: %s" % (d2 - d1a).seconds
I chose not to add tz to keep it simple/shorter; I can still strptime that way.
Below, first an explanation about the problem, then two possible solutions, one using time, another using datetime.
Problem explanation
The problem is on the observation that the OP made in the question: tm_isdst=-1. tm_isdst is a flag that determines whether daylight savings time is in effect or not (see for more details https://docs.python.org/2/library/time.html#time.struct_time).
Specifically, given the format of the string for the time from the OP (that complies with RFC 2822 Internet email standard), [time.strptime]4 does not store the information about the timezone, namely +0000. Thus, when the struct_time is created again according to the information in the string, tm_isdst=-1, namely unknown. The guess on how to fill in that information when making the calculation is based on the local system. For example, as if the system refers to North America, where daylight savings time is in effect, tm_isdst is set.
Solution with time
If you want to use only time package, then, the easiest way to parse directly the information is to specify that the time is in UTC, and thus adding %Z to the format. Note that time does not provide a way to store the information about the timezone in struct_time. As a result, it does not print the actual time zone associated with the time saved in the variable. The time zone is retrieved from the system. Therefore, it is not possible to directly use the same format for time.strftime. The part of the code for writing and reading the string would look like:
format = '%Y %m %d %H:%M:%S UTC'
format2 = '%Y %m %d %H:%M:%S %Z'
timestamp1 = time.strftime(format, time.gmtime())
t1 = time.strptime(timestamp1, format2)
Solution with datetime
Another solution involves the use datetime and dateutil packages, which directly support timezone, and the code could be (assuming that preserving the timezone information is a requirement):
from datetime import datetime
from dateutil import tz, parser
import time
time_format = '%Y %m %d %H:%M:%S %z'
utc_zone = tz.gettz('UTC')
utc_time1 = datetime.utcnow()
utc_time1 = utc_time1.replace(tzinfo=utc_zone)
utc_time1_string = utc_time1.strftime(time_format)
utc_time1 = parser.parse(utc_time1_string)
time.sleep(1)
utc_time2 = datetime.utcnow()
utc_time2 = utc_time2.replace(tzinfo=utc_zone)
print "Delta: %s" % (utc_time2 - utc_time1).total_seconds()
There are some aspects to pay attention to:
After the call of utcnow, the timezone is not set, as it is a naive UTC datetime. If the information about UTC is not needed, it is possible to delete both lines where the timezone is set for the two times, and the result would be the same, as there is no guess about DST.
It is not possible to use datetime.strptime because of %z, which is not correctly parsed. If the string contains the information about the timezone, then parser should be used.
It is possible to directly perform the difference from two instances of datetime and transform the resulting delta into seconds.
If it is necessary to get the time in seconds since the epoch, an explicit computation should be made, as there is no direct function that does that automatically in datetime (at the time of the answer). Below the code, for example for utc_time2:
epoch_time = datetime(1970,1,1)
epoch2 = (utc_time2 - epoch_time).total_seconds()
datetime.resolution, namely the smallest possible difference between two non-equal datetime objects. This results in a difference that is up to the resolution.