I have datetime values in local time zone, and I need to convert them into UTC. How can I do this conversion for historical records, considering daylight saving time in the past?
Local UTC
2018/07/20 09:00 ???
2018/12/31 11:00 ???
2019/01/17 13:00 ???
2020/08/15 18:00 ???
This is what I have so far:
import pytz
without_timezone = datetime(2018, 7, 20, 9, 0, 0, 0)
timezone = pytz.timezone("Europe/Vienna")
with_timezone = timezone.localize(without_timezone)
with_timezone
So, I assigned Europe/Vienna to all records (I assume that this considers daylight saving time, right?)
Now I need to convert it into UTC...
Assuming Local contains date/time as observed locally, i.e. including DST active/inactive, you would convert to datetime object, set time zone, and convert to UTC.
Ex:
from datetime import datetime, timezone
from zoneinfo import ZoneInfo # Python 3.9
Local = ["2018/07/20 09:00", "2018/12/31 11:00", "2019/01/17 13:00", "2020/08/15 18:00"]
# to datetime object and set time zone
LocalZone = ZoneInfo("Europe/Vienna")
Local = [datetime.strptime(s, "%Y/%m/%d %H:%M").replace(tzinfo=LocalZone) for s in Local]
for dt in Local:
print(dt.isoformat(" "))
# 2018-07-20 09:00:00+02:00
# 2018-12-31 11:00:00+01:00
# 2019-01-17 13:00:00+01:00
# 2020-08-15 18:00:00+02:00
# to UTC
UTC = [dt.astimezone(timezone.utc) for dt in Local]
for dt in UTC:
print(dt.isoformat(" "))
# 2018-07-20 07:00:00+00:00
# 2018-12-31 10:00:00+00:00
# 2019-01-17 12:00:00+00:00
# 2020-08-15 16:00:00+00:00
Note: with Python 3.9, you don't need third party libraries for time zone handling in Python anymore. There is a deprecation shim for pytz.
First, check your conversion value, here, in PDT, Universal is 5 hours behind, so convert accordingly, as for checking if its daylight savings time, write an if statement checking the date, and month and convert accordingly. Does this help?
Related
I am trying to parse a Unix timestamp using pd.to_datetime() vs. dt.datetime.fromtimestamp(), but their outputs are different. Which one is correct?
import datetime as dt
import pandas as pd
ts = 1674853200000
print(pd.to_datetime(ts, unit='ms'))
print(dt.datetime.fromtimestamp(ts / 1e3))
>> 2023-01-27 21:00:00
>> 2023-01-27 13:00:00
In contrast to pandas (numpy) datetime, vanilla Python datetime defaults to local time if you to not specify a time zone or UTC (= use naive datetime). Here's an illustration. If I reproduce your example in my Python environment, I get
from datetime import datetime, timezone
import pandas as pd
# ms since the Unix epoch, 1970-01-01 00:00 UTC
unix = 1674853200000
dt_py = datetime.fromtimestamp(unix/1e3)
dt_pd = pd.to_datetime(unix, unit="ms")
print(dt_py, dt_pd)
# 2023-01-27 22:00:00 # from fromtimestamp
# 2023-01-27 21:00:00 # from pd.to_datetime
Comparing the datetime objects with my local time UTC offset, there's the difference:
# my UTC offset at that point in time:
print(dt_py.astimezone().utcoffset())
# 1:00:00
# difference between dt_py and dt_pd:
print(dt_py-dt_pd)
# 0 days 01:00:00
To get consistent results between pandas and vanilla Python, i.e. avoid the ambiguity, you can use aware datetime:
dt_py = datetime.fromtimestamp(unix/1e3, tz=timezone.utc)
dt_pd = pd.to_datetime(unix, unit="ms", utc=True)
print(dt_py, dt_pd)
# 2023-01-27 21:00:00+00:00
# 2023-01-27 21:00:00+00:00
print(dt_py-dt_pd)
# 0 days 00:00:00
Both are correct. The main difference between them is that pd.to_datetime() is more flexible and can handle missing input data, while dt.datetime.fromtimestamp() assumes the input timestamp is in the local time zone. Generally, the choice of which one to use depends on the requirements of your use-case.
I would like to convert local times to UTC tine for different countries. I'm trying to do that with this way:
tz = pytz.timezone('Europe/Berlin')
x=tz.normalize(tz.localize(datetime.now())).astimezone(pytz.utc)
It gives me right result. But when I try to do that for Europe/Lisbon, I get wrong result. Can it be a problem or am I doing something wrong? There is no difference between two time zones but it gives me 3 hours difference as below.
Thanks in advance.
I am not sure why you get the wrong times. I tried this way and I get the right ones. It's just that I have frozen the current time to a variable and used it to check as a debug.
Lisbon has the time as UTC - no difference
P.S. I am in local time zone though, and hence you may see my time as different from yours but the difference seems to be right. Berlin is 1 hour ahead of UTC while Lisbon is same as UTC
import pytz
from datetime import datetime
tz = pytz.timezone('Europe/Berlin')
tb = tz.localize(datetime.now())
print(f"Berlin Time: {tb}")
x=tz.normalize(tb).astimezone(pytz.utc)
print(f"Berin to UTC: {x}")
tz2 = pytz.timezone('Europe/Lisbon')
tl = tz2.localize(datetime.now())
print(f"Lisbon Time: {tl}")
y=tz2.normalize(tl).astimezone(pytz.utc)
print(f"Lisbon to UTC: {tl}")
Here is the result:
Berlin Time: 2022-01-05 20:19:28.576504+01:00
Berin to UTC: 2022-01-05 19:19:28.576504+00:00
Lisbon Time: 2022-01-05 20:19:28.578506+00:00
Lisbon to UTC: 2022-01-05 20:19:28.578506+00:00
Process finished with exit code 0
This should work well for converting your local time to a timezone aware UTC datetime object
I'm guessing you might have issues with DST. The call to localize() requires you to specify if the timezone is serving DST or not. It defaults to False.
Other possibility is simply the fact that you're using your local times (considering you're not yourself in lisbon) and since you localize that time to lisbon timezone, of course it would result in incorrect time.
import pytz
import datetime
timezone = pytz.timezone('Europe/Lisbon')
original_time = datetime.datetime(2021, 10, 15, 13, 15) # change this to sample datetime to test different values
local_timezone_datetime = timezone.localize(original_time, False) # change False to True if DST is enabled on the timezone
converted_datetime = local_timezone_datetime.astimezone(pytz.utc)
print(converted_datetime)
Lemme know if you need a function to help you determine whether a timezone is serving DST.
From time and date website:
Generalized Time Zone in New York
ET Eastern Time UTC -5:00 / -4:00 Wed, 11:42:33
Time Zone Currently Being Used in New York
UTC -5 EST Eastern Standard Time New York Wed, 11:42:33
Time Zone Not Currently Being Observed in New York
UTC -4 EDT Eastern Daylight Time 10 Mar 2019
So basically New York time is EST or EDT depending on which period of the year we are currently on due to the DayLight timesaving.
When using US/Eastern in pytz, does it refer to ET, EST or EDT?. Does it take into consideration the EST/EDT time change?
I am using this code to get timestamps for the beginning and end of the day of a given date in Eastern Time:
import datetime
import pytz
session = datetime.datetime.strptime('26/02/2019', '%d/%m/%Y')
exchange_tz = pytz.timezone('US/Eastern')
session_localized = exchange_tz.localize(session)
_period1 = int((session_localized - datetime.datetime(1970, 1, 1, tzinfo=exchange_tz)).total_seconds())
_period2 = int((session_localized - datetime.datetime(1970, 1, 1, tzinfo=exchange_tz)).total_seconds()) + 24*60*60
>>> _period1
1551139200
>>> _period2
1551225600
I am not 100% sure that those timestamps have been properly generated. Because when I try to convert them back to localized datetimes I get 1 hour of difference:
_exchange_tz = pytz.timezone('US/Eastern')
_exchange_ts = _exchange_tz.localize( datetime.datetime.fromtimestamp(1551139200) )
>>> str(_exchange_ts)
'2019-02-26 01:00:00-05:00'
Which as far as I understand, it shall be 2019-02-26 00:00:00. What am I doing wrong?
I have also read in pytz documentation that normalize() function shall be used when doing arithmetics covering, but I am not sure if this apply to this case or not.
When using US/Eastern in pytz, does it refer to ET, EST or EDT?. Does it take into consideration the EST/EDT time change?
These are IANA (aka TZDB, Olson, etc.) time zone identifiers. US/Eastern is a link/alias to America/New_York. Both correctly model the history of the US Eastern time zone, including EST or EDT depending on when they are applicable.
Regarding your example, the timestamp 1551139200 equates to 2019-02-26T00:00:00Z (UTC). In US Eastern Time, that's 2019-02-25T19:00:00-05:00. That should be the result you get. You would first need to localize the input value to UTC (or in the case of UTC, you can assign tzinfo=pytz.utc safely). Then use the astimezone function to convert to US/Eastern.
So basically I have learned a bit with ISO 8601 where the format is
"2018-07-06T07:00:00.000"
and basically what I have achieved is that I starting of to change the ISO to a more formal timestamp which is:
etatime = str(datetime.datetime.strptime("2018-07-06T07:00:00.000", "%Y-%m-%dT%H:%M:%S.%f"))
which will give an output of:
2018-07-06 07:00:00
However I noticed the time is 1 hour behind the BST (British time) which should be added one hour.
My question is, is there possible to go from (2018-07-06T07:00:00.000) to (2018-07-06 08:00:00 BST)?
Assumptions: the input represents a UTC timestamp, and you want to localise that to London time. You probably do not want to localise it to BST time, since BST is the DST variation of GMT, and an actual location like London will switch between BST and GMT depending on the time of year. You'll want to install the pytz module.
from datetime import datetime, timezone
import pytz
date = '2018-07-06T07:00:00.000'
utc_date = datetime.strptime(date, '%Y-%m-%dT%H:%M:%S.%f').replace(tzinfo=timezone.utc)
london_date = utc_date.astimezone(pytz.timezone('Europe/London'))
datetime.datetime(2018, 7, 6, 8, 0, tzinfo=<DstTzInfo 'Europe/London' BST+1:00:00 DST>)
strptime gives you a naïve datetime object (without timezone information), .replace gives you an aware datetime object (with timezone information), which then enables you to simply convert that to a different timezone.
One suggestion is that you can use the timedelta function from datetime module:
from datetime import datetime, timedelta
etatime = datetime.strptime("2018-07-06T07:00:00.000", "%Y-%m-%dT%H:%M:%S.%f")
# Before adding one hour
print(etatime)
etatime = etatime + timedelta(hours=1)
# After adding one hour
print(etatime)
Output:
2018-07-06 07:00:00
2018-07-06 08:00:00
I want to find the last mid-night time stamp (only input is current timestamp). What is the best way?
I am writing python script for a global mobile application. The user request with the current timestamp, and in server side I want to find the last mid-night timestamp of the user with out affect time zone parameters.
I searched for it, I got a solution
import time
etime = int(time.time())
midnight = (etime - (etime % 86400)) + time.altzon
Its worked for me. But I am confused with time.altzon function, Is it create any problem for the users in different timezones.
To get the midnight timestamp of the client(mobile) you need to know the client's timezone.
from datetime import datetime
import pytz # pip install pytz
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
tz = pytz.timezone("America/New_York") # supply client's timezone here
# Get correct date for the midnight using given timezone.
# due to we are interested only in midnight we can:
# 1. ignore ambiguity when local time repeats itself during DST change e.g.,
# 2012-04-01 02:30:00 EST+1100 and
# 2012-04-01 02:30:00 EST+1000
# otherwise we should have started with UTC time
# 2. rely on .now(tz) to choose timezone correctly (dst/no dst)
now = datetime.now(tz)
print(now.strftime(fmt))
# Get midnight in the correct timezone (taking into account DST)
midnight = tz.localize(now.replace(hour=0, minute=0, second=0, microsecond=0, tzinfo=None),
is_dst=None)
print(midnight.strftime(fmt))
# Convert to UTC (no need to call `tz.normalize()` due to UTC has no DST transitions)
dt = midnight.astimezone(pytz.utc)
print(dt.strftime(fmt))
# Get POSIX timestamp
print((dt - datetime(1970,1,1, tzinfo=pytz.utc)).total_seconds())
Output
2012-08-09 08:46:29 EDT-0400
2012-08-09 00:00:00 EDT-0400
2012-08-09 04:00:00 UTC+0000
1344484800.0
Note: on my machine #phihag's answer produces 1344470400.0 that is different from the above (my machine is not in New York).