Usually what I do if I want to convert time zone is the following
# local interpreted as pst to utc
utc = pytz.utc
pst = pytz.timezone('America/Los_Angeles')
start_time_str = time.strftime('%Y-%m-%d %H:%M:%S', \
time.localtime(start_time))
start_time_datetime = \
datetime.datetime.strptime(start_time_str, "%Y-%m-%d %H:%M:%S")
start_time_datetime = \
start_time_datetime.replace(tzinfo=pst).astimezone(utc)
Now I want to do similar stuff such that I want localtime convert to pst
localtime = datetime.datetime.fromtimestamp(time.mktime(
time.localtime()))
I am not exactly sure how would you achieve this
Any help would be appreciated
Two things:
The "America/Los_Angeles" identifier represents the entire Pacific time zone, including both PST and PDT. It will use the either -8 or -7 as the time zone offset depending on the specific date and time it is used on.
Calling replace(tzinfo=...) is a mistake. Use localize instead. See the pytz documentation. This is discussed in the introduction and in the very first example.
Your code contains unnecessary (calling strptime after strftime) or just wrong (.replace()) conversions.
To create an aware datetime object given "seconds since epoch" start_time (a value returned by time.time()):
#!/usr/bin/env python
from datetime import datetime
import pytz # $ pip install pytz
tz = pytz.timezone('America/Los_Angeles')
start_time_datetime = datetime.fromtimestamp(start_time, tz)
As #Matt Johnson mentions, 'America/Los_Angeles' timezone id produces time with PST or PDT tzname depending on the date. No conversion is necessary.
The last code example in your question has both unnecessary conversions and may fail in some cases. If the intent is to get the current time in the local timezone as an aware datetime object then you could use tzlocal module:
#!/usr/bin/env python
from datetime import datetime
from tzlocal import get_localzone # $ pip install tzlocal
local_timezone = get_localzone() # pytz timezone corresponding to the local time
current_localtime = datetime.now(local_timezone)
First, as a note, local time is either Standard Time or Daylight Savings Time -- it cannot be both simultaneously. I believe what you are really getting at is asking whether there is a way to tell if a time is either a Daylight Savings Time or not. In other words, given a specific day, is a particular time in Daylight Savings Time? Another common use case is converting tabular data pegged at a specific time offset to the current correct time.
Okay, I can think of three ways to do this.
(1) You can convert the datetime into a time object and look at the .tm_isdst property, similar to this answer. You can do a test on it to see if that property equals one and then subtract a timedelta of one hour.
>>> time.localtime()
time.struct_time(tm_year=2019, tm_mon=11, tm_mday=16, tm_hour=12,
tm_min=36, tm_sec=32, tm_wday=5, tm_yday=320, tm_isdst=0)
>>> time.localtime().tm_isdst
0
(2) Second way is to crawl month by month over noon on the fifteenth of each month and check if the offset varies. (Note that some areas do not implement DST.) The greater value indicates a skip ahead for Daylight Savings Time. (For instance, PDT is UTC -7 while PST is UTC -8.) From those values you can calculate the time difference. Then you can test your time by checking the offset to see if it is in Standard Time or Daylight Savings Time. Offest is checked this way:
>>> my_special_pdt_datetime.strftime('%z')
'-0700'
(3) And, maybe the easiest, is to calculate the existing offset by getting offset = datetime.datetime.utcnow() - datetime.datetime.now() and then comparing that to the known offset from pytz:
>>> pytz.timezone('US/Pacific').localize(datetime.datetime(2019,1,1)).strftime('%z')
'-0800'
Related
Pretty sure it's an easy one but I don't get it.
My local TZ is currently GMT+3, and when I take timestamp from datetime.utcnow().timestamp() it is indeed giving me 3 hours less than datetime.now().timestamp()
During another process in my flow, I take that utc timestamp and need to turn it into datetime.
When I'm doing fromtimestamp I get the right utc hour, but when I'm using utcfromtimestamp I get another 3 hours offset.
The documentation, though, asks me to use fromtimestamp for local timezone, and utcfromtimestamp for utc usages.
What am I missing ? is the initial assumption for both funcs is that the timestamp is given in local timezone ?
Thank you :)
The key thing to notice when working with datetime objects and their POSIX timestamps (Unix time) at the same time is that naive datetime objects (the ones without time zone information) are assumed by Python to refer to local time (OS setting). In contrast, a POSIX timestamp (should) always refer to seconds since the epoch UTC. You can unambiguously obtain that e.g. from time.time(). In your example, not-so-obvious things happen:
datetime.now().timestamp() - now() gives you a naive datetime object that resembles local time. If you call for the timestamp(), Python converts the datetime to UTC and calculates the timestamp for that.
datetime.utcnow().timestamp() - utcnow() gives you a naive datetime object that resembles UTC. However, if you call timestamp(), Python assumes (since naive) that the datetime is local time - and converts to UTC again before calculating the timestamp! The resulting timestamp is therefore off from UTC by twice your local time's UTC offset.
A code example. Let's make some timestamps. Note that I'm on UTC+2 (CEST), so offset is -7200 s.
import time
from datetime import datetime, timezone
ts_ref = time.time() # reference POSIX timestamp
ts_utcnow = datetime.utcnow().timestamp() # dt obj UTC but naive - so also assumed local
ts_now = datetime.now().timestamp() # dt obj naive, assumed local
ts_loc_utc = datetime.now(tz=timezone.utc).timestamp() # dt obj localized to UTC
print(int(ts_utcnow - ts_ref))
# -7200 # -> ts_utcnow doesn't refer to UTC!
print(int(ts_now - ts_ref))
# 0 # -> correct
print(int(ts_loc_utc - ts_ref))
# 0 # -> correct
I hope this clarifies that if you call datetime.utcfromtimestamp(ts_utcnow), you get double the local time's UTC offset. Python assumes (which I think is pretty sane) that the timestamp refers to UTC - which in fact, it does not.
My suggestion would be to use timezone-aware datetime objects; like datetime.now(tz=timezone.utc). If you're working with time zones, the dateutil library or Python 3.9's zoneinfo module are very helpful. And if you want to dig deep, have a look at the datetime src code.
Given a pair of str objects representing an ISO 8601 time and time zone:
time_str = '09:30'
time_zone_str = 'America/New_York'
How can these 2 strings be parsed into a time (not datetime) object?
Note: It's obviously possible to split the time_str by ':' and use the time constructor but then the parsing would be a little tricky to count the number of elements in the resulting list to know the resolution (minute, second, microsecond) of the str. This is because ISO 8601 allows for different representations:
time_str_short = '09:30'
time_str_long = '09:30:00'
Thank you in advance for your consideration and response.
The answer to "can I do this?" (with a timezone) is both yes and no. Firstly let's convert the string to a time object. As one commenter mentioned, you can do this in python 3.7 with the fromisoformat method:
from datetime import time
time.fromisoformat("09:30")
If you are not using 3.7, you can do this by creating a datetime and then converting to a time object:
from datetime import datetime, time
as_time = datetime.datetime.strptime("09:00", "%H:%M").time()
Now to deal with the timezone. As the timezone is a name, we can use the very convenient pytz module to convert it to a tzinfo object:
pytz.timezone('America/New_York')
At this point you're probably tempted to just pass it to the time constructor as the tzinfo argument, but unfortunately that does not work with pytz:
Unfortunately using the tzinfo argument of the standard datetime constructors ‘’does not work’’ with pytz for many timezones.
~ http://pytz.sourceforge.net/
So we will have to use the localize method of the newly created tzinfo object. But unfortunately we will still not be able to successfully localize the time object with this timezone. The reason for this is that pytz needs to know the date in order to determine if this timezone is in daylight savings time or not. As we have not provided the date, achieving this is quite impossible and you will get odd results like:
>>> pytz.timezone('America/New_York').localize(as_dt).isoformat()
'1900-01-01T09:00:00-04:56'
Note the -04:56 offset, which is gibberish. There are a few options for getting to what you ultimately want.
One option is to assume the time is a time today:
as_time = datetime.datetime.strptime("09:00", "%H:%M").time()
tz = pytz.timezone('America/New_York')
local_time = tz.localize(datetime.datetime.now().replace(hour=as_time.hour, minute=as_time.minute))
The other option is to use naive timezone offsets rather than timezone names:
from datetime import timezone, timedelta
naive_tz = timezone(timedelta(hours=5))
datetime.time(9, 30).replace(tz_info=naive_tz)
But I would not recommend this method as it's quite brittle and would require some intermediate steps to derive from the TZ location name that are non-trivial.
A timezone without a date is meaningless, so no, you can't use both to produce a time object. While the standard library time object does support having a tzinfo attribute, the 'timezone' object is not really a timezone, but merely a time offset.
A timezone is more than just an offset from UTC. Timezone offsets are date-dependent, and because such details as the Daylight Savings winter / summer time distinction is partly the result of political decisions, what dates the timezone offset changes is also dependent on the year.
To be explicit, America/New_York is a timezone, not a time offset. The exact offset from UTC depends on the date; it'll be minus 4 hours in summer, 5 hours in winter!
So for a timezone such as America/New_York, you need to pick a date too. If you don't care about the date, pick a fixed date so your offset is at least consistent. If you are converting a lot of time stamps, store the timezone offset once as a timedelta(), then use that timedelta to shift time() objects to the right offset.
To parse just a timestring, pretend there is a date attached by using the datetime.strptime() method, then extract the time object:
from datetime import datetime
try:
timeobject = datetime.strptime(time_str, '%H:%M').time()
except ValueError:
# input includes seconds, perhaps
timeobject = datetime.strptime(time_str, '%H:%M:%S').time()
To update the time given a timezone, get a timezone database that supports your timezone string first; the pytz library is regularly updated.
from pytz import timezone
timezone = pytz.timezone(time_zone_str)
How you use it depends on what you are trying to do. If the input time is not in UTC, you can simply attach the timezone to a datetime() object with the datetime.combine()method, after which you can move it to the UTC timezone:
dt_in_timezone = datetime.combine(datetime.now(), timeobject, timezone)
utc_timeobject = dt_in_timezone.astimezone(pytz.UTC).time()
This assumes that 'today' is good enough to determine the correct offset.
If your time is a UTC timestamp, combine it with the UTC timezone, then use the pytz timezone; effectively the reverse:
dt_in_utc = datetime.combine(datetime.now(), timeobject, pytz.UTC)
timeobject_in_timezone = dt_in_timezone.astimezone(timezone).time()
To store just the offset for bulk application, pass in a reference date to the timezone.utcoffset() method:
utc_offset = timezone.utcoffset(datetime.now())
after which you can add this to any datetime object as needed to move from UTC to local time, or subtract it to go from local to UTC. Note that I said datetime, as time objects also don't support timedelta arithmetic; a timedelta can be larger than the number of seconds left in the day or the number of seconds since midnight, after all, so adding or subtracting could shift days as well as the time:
# new time after shifting
(datetime.combine(datetime.now(), timeobject) + utc_offset).time()
For completion sake, you can't pass in a pytz timezone to a time object; it just doesn't have any effect on the time. The timezone object returns None for the UTC offset in that case, because it can't give any meaningful answer without a date:
>>> from datetime import time
>>> from pytz import timezone
>>> tz = timezone('America/New_York')
>>> time_with_zone = time(12, 34, tzinfo=tz)
>>> time_with_zone
datetime.time(12, 34, tzinfo=<DstTzInfo 'America/New_York' LMT-1 day, 19:04:00 STD>)
>>> time_with_zone.utcoffset()
>>> time_with_zone.utcoffset() is None
True
>>> tz.utcoffset(None) is None # what time_with_zone.utcoffset() does under the hood
None
So for all intents an purposes, time_with_zone is just another naive time object as the tzinfo object attached doesn't actually have any effect.
Moreover, because there is no date to determine the correct timezone information, pytz selects the earliest known 'New York' timezone, and that's not exactly a recent one; look closely at that tzinfo representation:
tzinfo=<DstTzInfo 'America/New_York' LMT-1 day, 19:04:00 STD>
^^^^^^^^^^^^^^^^^^^^^^^
That's the timezone introduced in 1883 by the railroads; the 'modern' EST timezone was not introduced until the 20th century. This is why timezone objects are usually passed in a date when determining the offset:
>>> tz.utcoffset(datetime(1883, 6, 28))
datetime.timedelta(-1, 68640)
>>> tz.utcoffset(datetime(1918, 6, 28))
datetime.timedelta(-1, 72000)
Hope it works for you,
import datetime
# Hello World program in Python
print "Hello World!\n"
time_str = '09:30'
time_zone_str = 'America/New_York'
s = "I am looking for a course in Paris!"
print(s)
print(datetime.datetime.strptime(time_str, '%H:%M').time())
print(datetime.time(3, 55))
Thanks
I came across this exact issue, and I can't figure out how to achieve the solution in my case.
Guido says
The solution is to remove the tzinfo completely from the time after
converting to UTC.
This is what I have tried:
date_time = parser.parse(i.pubDate.text)
news.publication_date = date_time.replace(tzinfo=None).date()
And I get the same error:
NotImplementedError: DatetimeProperty publication_date_time can only support UTC. Please derive a new Property to support alternative timezones.
So it seems I have to convert the date to UTC first. And here my research has failed me.
I came across this solution:
The solution suggested is this:
def date_time_to_utc(date_time):
tz = pytz.timezone('???')
return tz.normalize(tz.localize(date_time)).astimezone(pytz.utc)
But I don't have the timezone. I am scraping the date from a html source. So the timezone could really be from anywhere in the world. Is there no easy and reliable way to convert a date time to UTC?
I could use both dateutil and pytz to achieve this. Many Thanks.
UPDATE
It has been a really long day. I have misread the stack trace. However the question remains valid.
date_time = (datetime}2015-01-13 18:13:26+00:00
news.publication_date_time = date_time
This caused the crash. And it seems by doing this, I pass the unit test:
news.publication_date_time = date_time.replace(tzinfo=None)
Is this the correct way converting a GMT 0 datetime to UTC datetime? Or in fact any timezone to UTC?
Is this the correct way converting a GMT 0 datetime to UTC datetime? Or in fact any timezone to UTC?
If aware datetime object is already in UTC (+0000) then your formula works:
naive_utc = aware_utc.replace(tzinfo=None)
where aware_utc is a timezone-aware datetime object that represents time in UTC.
But if aware datetime object is not in UTC; it fails. You should take into account a (possibly) non-zero UTC offset in the general case:
assert aware.tzinfo is not None and aware.utcoffset() is not None
# local time = utc time + utc offset (by definition)
# -> utc = local - offset
naive_utc = aware.replace(tzinfo=None) - aware.utcoffset()
where aware is a timezone-aware datetime object in an arbitrary timezone.
But I don't have the timezone. I am scraping the date from a html
source. So the timezone could really be from anywhere in the world. Is
there no easy and reliable way to convert a date time to UTC? I could
use both dateutil and pytz to achieve this. Many Thanks.
No. dateutil, pytz won't help you unless the date string itself contains the timezone (or at least its utc offset).
Remember: It is always noon somewhere on Earth i.e., if you collect date/time strings from different places on Earth then you can't compare them unless you attach the corresponding timezones. You can't convert it to UTC, you can't get a valid POSIX timestamp if you don't know the source timezone for the date.
I'm an idiot and it's late here, this time I read the question.
tstmp= date_time.replace(tzinfo=utc).total_seconds()
naive_date = datetime.utcfromtimestamp(tstmp)
First answer will just give you the current naive time
Try this:
dateTime = dateTime.replace(tzinfo=None)
dtUtcAware = pytz.UTC.localize(dateTime)
Ok let me first start by saying my timezone is CET/CEST. The exact moment it changes from CEST to CET (back from DST, which is GMT+2, to normal, which GMT+1, thus) is always the last Sunday of October at 3AM. In 2010 this was 31 October 3AM.
Now note the following:
>>> import datetime
>>> import pytz.reference
>>> local_tnz = pytz.reference.LocalTimezone()
>>> local_tnz.utcoffset(datetime.datetime(2010, 10, 31, 2, 12, 30))
datetime.timedelta(0, 3600)
This is wrong as explained above.
>>> local_tnz.utcoffset(datetime.datetime(2010, 10, 30, 2, 12, 30))
datetime.timedelta(0, 7200)
>>> local_tnz.utcoffset(datetime.datetime(2010, 10, 31, 2, 12, 30))
datetime.timedelta(0, 7200)
Now it is suddenly correct :/
I know there are several questions about this already, but the solution given is always "use localize", but my problem here is that the LocalTimezone does not provide that method.
In fact, I have several timestamps in milliseconds of which I need the utcoffset of the local timezone (not just mine, but of anyone using the program). One of these is 1288483950000 or Sun Oct 31 2010 02:12:30 GMT+0200 (CEST) in my timezone.
Currently I do the following to get the datetime object:
datetime.datetime.fromtimestamp(int(int(millis)/1E3))
and this to get the utcoffset in minutes:
-int(local_tnz.utcoffset(date).total_seconds()/60)
which, unfortunately, is wrong in many occasions :(.
Any ideas?
Note: I'm using python3.2.4, not that it should matter in this case.
EDIT:
Found the solution thanks to #JamesHolderness:
def datetimeFromMillis(millis):
return pytz.utc.localize(datetime.datetime.utcfromtimestamp(int(int(millis)/1E3)))
def getTimezoneOffset(date):
return -int(date.astimezone(local_tz).utcoffset().total_seconds()/60)
With local_tz equal to tzlocal.get_localzone() from the tzlocal module.
According to Wikipedia, the transition to and from Summer Time occurs at 01:00 UTC.
At 00:12 UTC you are still in Central European Summer Time (i.e. UTC+02:00), so the local time is 02:12.
At 01:12 UTC you are back in the standard Central European Time (i.e. UTC+01:00), so the local time is again 02:12.
When changing from Summer Time back to standard time, the local time goes from 02:59 back to 02:00 and the hour repeats itself. So when asking for the UTC offset of 02:12 (local time), the answer could truthfully be either +01:00 or +02:00 - it depends which version of 02:12 you are talking about.
On further investigation of the pytz library, I think your problem may be that you shouldn't be using the pytz.reference implementation, which may not deal with these ambiguities very well. Quoting from the comments in the source code:
Reference tzinfo implementations from the Python docs.
Used for testing against as they are only correct for the years
1987 to 2006. Do not use these for real code.
Working with ambiguous times in pytz
What you should be doing is constructing a timezone object for the appropriate timezone:
import pytz
cet = pytz.timezone('CET')
Then you can use the utcoffset method to calculate the UTC offset of a date/time in that timezone.
dt = datetime.datetime(2010, 10, 31, 2, 12, 30)
offset = cet.utcoffset(dt)
Note, that the above example will throw an AmbiguousTimeError exception, because it can't tell which of the two versions of 02:12:30 you meant. Fortunately pytz will let you specify whether you want the dst version or the standard version by setting the is_dst parameter. For example:
offset = cet.utcoffset(dt, is_dst = True)
Note that it doesn't harm to set this parameter on all calls to utcoffset, even if the time wouldn't be ambiguous. According to the documentation, it is only used during DST transition ambiguous periods to resolve that ambiguity.
How to deal with timestamps
As for dealing with timestamps, it's best you store them as UTC values for as long as possible, otherwise you potentially end up throwing away valuable information. So first convert to a UTC datetime with the datetime.utcfromtimestamp method.
dt = datetime.datetime.utcfromtimestamp(1288483950)
Then use pytz to localize the time as UTC, so the timezone is attached to the datetime object.
dt = pytz.utc.localize(dt)
Finally you can convert that UTC datetime into your local timezone, and obtain the timezone offset like this:
offset = dt.astimezone(cet).utcoffset()
Note that this set of calculations will produce the correct offsets for both 1288483950 and 1288487550, even though both timestamps are represented by 02:12:30 in the CET timezone.
Determining the local timezone
If you need to use the local timezone of your computer rather than a fixed timezone, you can't do that from pytz directly. You also can't just construct a pytz.timezone object using the timezone name from time.tzname, because the names won't always be recognised by pytz.
The solution is to use the tzlocal module - its sole purpose is to provide this missing functionality in pytz. You use it like this:
import tzlocal
local_tz = tzlocal.get_localzone()
The get_localzone() function returns a pytz.timezone object, so you should be able to use that value in all the places I've used the cet variable in the examples above.
Given a timestamp in milliseconds you can get the utc offset for the local timezone using only stdlib:
#!/usr/bin/env python
from datetime import datetime
millis = 1288483950000
ts = millis * 1e-3
# local time == (utc time + utc offset)
utc_offset = datetime.fromtimestamp(ts) - datetime.utcfromtimestamp(ts)
If we ignore time around leap seconds then there is no ambiguity or non-existent times.
It supports DST and changes of the utc offset for other reasons if OS maintains a historical timezone db e.g., it should work on Ubuntu for any past/present date but might break on Windows for past dates that used different utc offset.
Here's the same using tzlocal module that should work on *nix and Win32 systems:
#!/usr/bin/env python
from datetime import datetime
from tzlocal import get_localzone # pip install tzlocal
millis = 1288483950000
ts = millis * 1e-3
local_dt = datetime.fromtimestamp(ts, get_localzone())
utc_offset = local_dt.utcoffset()
See How to convert a python utc datetime to a local datetime using only python standard library?
To get the utc offset in minutes (Python 3.2+):
from datetime import timedelta
minutes = utc_offset / timedelta(minutes=1)
Don't use pytz.reference.LocalTimezone(), it is only for tests.
import pytz, datetime
tz = timezone('CET')
tz.utcoffset(datetime.datetime.now()).total_seconds()
7200.0
I am making a news blog for my project in which I have to display time under the news title as x minutes ago etc.
From the RSS feed of the news, I have the time stamp in the form of a string. For example:
timestamp = 'Tue, 12 Feb 2013 07:43:09 GMT'
I am trying to find the difference between this timestamp and the present time using datetime module in python. But for some reason it gives error:
ValueError: astimezone() cannot be applied to a naive datetime
I'll appreciate if somebody can point me in the right direction. Below is my attempt in Python:
from datetime import datetime
from pytz import timezone
timestamp = 'Tue, 12 Feb 2013 07:43:09 GMT'
t = datetime.strptime(timestamp, '%a, %d %b %Y %I:%M:%S %Z')
now_time = datetime.now(timezone('US/Pacific'))
# converting the timestamp to Pacific time
t_pacific = t.astimezone(timezone('US/Pacific')) # get error here
diff = t_pacific - t
Thanks!
Prakhar
Your example had a few problems (I see you've fixed them now):
First line should be:
from datetime import datetime
It looks as though you're missing a closing parenthesis on the 4th line:
now_time = datetime.now(timezone('US/Pacific')
What is timezone()? Where does that come from?
You don't really need to mess with timezones, I don't think — just use GMT (UTC). How about something more like this:
from datetime import datetime
timestamp = 'Tue, 12 Feb 2013 07:43:09 GMT'
t = datetime.strptime(timestamp, '%a, %d %b %Y %I:%M:%S %Z')
t_now = datetime.utcnow()
diff = t_now - t
The problem here is that t does not have a timezone—that's what the error message means by "naive datetime". From the docs:
There are two kinds of date and time objects: “naive” and “aware”… An aware object has sufficient knowledge of… time zone and daylight savings time information… A naive object does not…
You can verify that it's naive by doing this:
print(t.tzinfo)
The answer will be None.
As the astimezone docs say:
self must be aware (self.tzinfo must not be None, and self.utcoffset() must not return None).
The strptime function always generates a naive datetime.
You can fix this in various ways:
First convert t to a GMT datetime instead of a naive one, and then your conversion to 'US/Pacific' will work.
As the docs say, "If you merely want to attach a time zone object tz to a datetime dt without adjustment of date and time data, use dt.replace(tzinfo=tz)." Since you know the time is in UTC, just replace the empty tz with UTC, and you've got an aware time.
Convert to PST with a different mechanism than astimezone, on which will assume UTC or which allows you to specify the source.
There are various alternatives out there, but you're already using pytz, so see its documentation.
Convert now_time to UTC instead of converting t to PST.
The last one is probably the simplest and best for most use cases. Since you've only got now_time in PST because you explicitly asked for it that way, all you have to do is not do that (or explicitly ask for 'GMT' instead of 'US/Pacific'). Then you can just do your date arithmetic on UTC times.
If you need to display final results in PST, it's still often better to do the arithmetic in UTC, and convert at the end. (For example, you can have two times that are an hour apart, but have the same value in Pacific, because of 1am on DST day being repeated twice; that won't be an issue if you stay in UTC all the time.)