I'm creating a simple Django API where access tokens are used. The idea is that, when a user requests a token, said token is valid for the next 7 hours.
However when generating the token I obtain a Python NonExistentTimeError. Code is:
#Django view
...
expires = datetime.datetime.now() + datetime.timedelta(hours=7)
token = APIMasterToken.objects.create(
token=token,
expiration_date=expires
)
However I obtain an exception generating a token with said date:
NonExistentTimeError at /api/obtain_master_token/
2013-03-10 02:05:12.179508
What does this error mean and how can I solve it?
EDIT: I just read the documentation and it would appear this error is raised when a day occurs that has a clock change following DST. However, I don't know how to solve the problem. Shouldn't the timedelta function take care of this?
Django expects to work with timezone aware datetime objects.
From the Django documentation, the now() call would become:
import datetime
from django.utils.timezone import utc
now = datetime.datetime.utcnow().replace(tzinfo=utc)
expires = now + datetime.timedelta(hours=7)
Better still, use the now() function from django.utils.timezone.
Both datetime.now() and datetime.utcnow() return naive datetime objects which is not what Django requires. With a time zone applied to expires, Django is then able to convert back to UTC for storage as UTC in the database backend.
The NonExistentTimeError is actually thrown by code from the pytz module. The pytz documentation is definitely worth reading as there's many gotchas when dealing with time zones.
Related
I have a script on Heroku, hence I do not know where it resides every time it reboots (-> I cannot manually set the datetime.timedelta), and I need to run a routine at 5pm Rome time. This script has always worked, at least this winter, but now it's starting one hour later, so at 6pm.
import datetime
import pytz
tz = pytz.timezone("Europe/Rome")
now = datetime.datetime.now(tz=tz)
start = datetime.datetime(now.year, now.month, now.day, hour=17, tzinfo=tz)
This answer shows that
using the tzinfo argument of the standard datetime constructors 'does not work' with pytz for many timezones
so I tried refactoring using tz.localize(start), but on the first reboot the system scheduled the routine at 3pm, two hours before. I'm getting very confused: how can I set a specific time in a specific timezone?
Since I needed to solve this problem quickly, I just accepted the tip #MrFuppes gave me to upgrade the version of Python to 3.9 and since I was using 3.7 this wasn't a big deal, at least for my dependencies.
Using ZoneInfo:
from zoneinfo import ZoneInfo
import datetime
tz = ZoneInfo("Europe/Rome")
now = datetime.datetime.now(tz=tz)
start = datetime.datetime(now.year, now.month, now.day, hour=17, tzinfo=tz)
Does anyone have any tips to get the current UTC time, from online somewhere, and write some decent python code assuming my computer clock is always wrong?
current_datetime = datetime.datetime.utcnow() #---> assume always wrong
current_datetime = datetime.datetime.now() #---> assume always wrong
Using '.utcnow()' or '.now()' both depend upon the accuracy of my computer clock.
I want to write the code assuming that if it runs from a computer with a bad clock, it still gets the correct time.
BACKGROUND:
I am trying to retool my code to entirely live in UTC time.
My use case is to do some time series analysis.
I keep finding myself accidentally being off 5 hours from EST, or off 1 hour from daylight savings when doing calculations.
The tools within the datetime.datetime objects are great, however it would be nice be able to flag some setting when importing the datetime library and prevent reading my computer clock entirely, to avoid any accidental clock badness issue.
EXAMPLE OF CODE I AM LOOKING FOR:
import datetime
import requests
#force datetime libaries to never read my computer clock:
datetime.some_settings_function( readcomputerclock = False/'Never' )
#get the current time with some API:
current_utc_date_and_time_from_online = requests.get(...) #some api get request
current_utc_datetime = transform( current_utc_date_and_time_from_oneline )
#Transform back and forth to UTC Epoch time:
current_utc_epoch = current_utc_datetime.timestamp()
current_utc_datetime_again = datetime.datetime.fromtimestamp(current_utc_epoch)
#current_utc_datetime == current_utc_datetime_again
#Trigger exception with new settings, when i accidentally write code
# that would ask datetime library to attempt to read computer clock:
fail_code_line = datetime.datetime.now()
# >>> trigger some exception here
TLDR; I am looking for a reliable UTC api for python, and a way to prevent datetime from ever reading my computer clock again.
UPDATE: After accepting the provided answer it has become clear to me for my purposes, trusting my computer clock for a few seconds after updating my computer clock from a trusted source, then asking my computer clock for UTC time within those few seconds is good enough. It is a feasible coding practice to write a "get UTC time now" code using all the information within the accepted answer, that is accurate to within a second or two. (No I have not done the statistical confidence interval posterior on the accuracy) It is then further feasible to write all the rest of my code such that all logic will assume UTC time.
Getting correct, timezone aware datetimes and unix timestamps
Turns out this question was rather about how to convert to / from unix timestamps and datetimes.
The correct solution in python 3 should be:
from datetime import datetime, timezone
# get the current utc time
t = datetime.now(timezone.utc)
# convert to unix, this will keep the utc timezone
unix = t.timestamp()
# convert back to datetime, specifying that the timestamp is in UTC
t2 = datetime.fromtimestamp(unix, tz=timezone.utc)
Other timezones
Since python 3.9, the stdlib has the zoneinfo library, you can use this to convert between timezones.
For python < 3.9, you have to use a thirdparty library like dateutil.
from datetime import datetime
from zoneinfo import ZoneInfo
now_berlin = datetime.now(ZoneInfo('Europe/Berlin'))
now_ny = now_berlin.astimezone(ZoneInfo('America/New_York'))
print('Time in Berlin:', now_berlin)
print('Time in New York', now_ny)
Actually using ntp instead of the computer clock
You can use ntplib:
from ntplib import NTPClient
from datetime import datetime, timezone
client = NTPClient()
response = client.request('europe.pool.ntp.org', version=3)
time = datetime.fromtimestamp(resp.tx_time, tz=timezone.utc)
Edit: I however don't see a real reason, why just from traveling this should go wrong:
from datetime import datetime, timezone
dt = datetime.now(timezone.utc)
for more information see: https://blog.ganssle.io/articles/2019/11/utcnow.html
I am trying to compare the time of an AWS EC2 instance object that is of type datetime with another datetime being represented as datetime.datetime.now. The line of code in question looks like,
if launchTime < datetime.datetime.now()-datetime.timedelta(seconds=20):
Where launchTime is of type datetime. However when I run it I get the error
can't compare offset-naive and offset-aware datetimes: TypeError
And I'm unsure of how to convert launchTime in such a way where I can successfully compare it.
Edited fixed code below -----------------------------------------
if launchTime.replace(tzinfo=None) < datetime.datetime.now()-datetime.timedelta(minutes=4):
Full code as well in case any future people find it of value. It's Python 3 to stop EC2 instances that have been running for an "x" amount of time. In this case if an instance is running for five minutes. Terminate it. The lambda itself is set up with Cloudwatch to run every 4 minutes as well.
import boto3
import time
import datetime
#for returning data about our newly created instance later on in fuction
client = boto3.client('ec2')
def lambda_handler(event, context):
response = client.describe_instances()
#for each instance currently running/terminated/stopped
for r in response['Reservations']:
for i in r['Instances']:
#if its running then we want to see if its been running for more then 3 hours. If it has then we stop it.
if i["State"]["Name"] == "running":
launchTime = i["LaunchTime"]
#can change minutes=4 to anything
if launchTime.replace(tzinfo=None) < datetime.datetime.now()-datetime.timedelta(minutes=4):
response = client.stop_instances(
InstanceIds=[
i["InstanceId"]
]
)
The main problem is that I'm assuming launchTime is timezone aware, whereas datetime.now() is not (datetime.now().tzinfo == None).
There are a couple ways to solve this, but the easiest would be to remove the tzinfo from launchTime: if launchTime.replace(tzinfo=None) < datetime.datetime.now()-datetime.timedelta(seconds=20) should do the trick.
Alternatively, you can convert your datetime objects to Unix timestamps and then you don't have to deal with timezone silliness.
Try like this, you have to make sure pytz installed :
import pytz
utc=pytz.UTC
launchTime = utc.localize(launchTime)
Seems strange, but I cannot find an easy way to find the local timezone using pandas/pytz in Python.
I can do:
>>> pd.Timestamp('now', tz='utc').isoformat()
Out[47]: '2016-01-28T09:36:35.604000+00:00'
>>> pd.Timestamp('now').isoformat()
Out[48]: '2016-01-28T10:36:41.830000'
>>> pd.Timestamp('now').tz_localize('utc') - pd.Timestamp('now', tz='utc')
Out[49]: Timedelta('0 days 01:00:00')
Which will give me the timezone, but this is probably not the best way to do it...
Is there a command in pytz or pandas to get the system time zone? (preferably in python 2.7 )
I don't think this is possible using pytz or pandas, but you can always install python-dateutil or tzlocal:
from dateutil.tz import tzlocal
datetime.now(tzlocal())
or
from tzlocal import get_localzone
local_tz = get_localzone()
time.timezone should work.
The offset of the local (non-DST) timezone, in seconds west of UTC
(negative in most of Western Europe, positive in the US, zero in the
UK).
Dividing by 3600 will give you the offset in hours:
import time
print(time.timezone / 3600.0)
This does not require any additional Python libraries.
I have found that in many cases this works: (Since Python 3.6)
from datetime import datetime
# use this extension and it adds the timezone
tznow = datetime.now().astimezone()
print(tznow.isoformat())
2020-11-05T06:56:38.514560-08:00
# It shows that it does have a valid timezone
type(tznow.tzinfo)
<class 'datetime.timezone'>
I find this handy as it does not depend on external packages. It appears to work only in Python3 (but not in Python2)
Quite a few locale time related settings from OS level is covered by time module
import time
# Since Python 3.3
local_time = time.localtime() # returns a `time.struct_time`
tzname_local = local_time.tm_zone # 'EST'
dst = local_time.tm_isdst # _from docs_: may be set to 1 when daylight savings time is in effect,
# and 0 when it is not. A value of -1 indicates that this is not known,
# and will usually result in the correct state being filled in.
tm_gmtoff and tm_zone attributes are available on platforms with C library supporting the corresponding fields in struct tm.
see: https://docs.python.org/3/library/time.html#time.struct_time
# At least from Python 2.7.18
local_tzname = time.tzname # 'EST'
A tuple of two strings: the first is the name of the local non-DST timezone, the second is the name of the local DST timezone. If no DST timezone is defined, the second string should not be used.
see: https://docs.python.org/2.7/library/time.html#time.tzname)
Another trick is to use datetime.now().astimezone() as found here and the reason why it fails on python 2.x
from datetime import datetime
# Python 3 will return a datetime with local timezone,
local_now = datetime.now().astimezone()
# Doesn't work on python 2.x
# datetime.now().astimezone() -> TypeError: Required argument 'tz' (pos 1) not found
# datetime.now().astimezone(dateutil.tz.UTC) -> ValueError: astimezone() cannot be applied to a naive datetime
local_tz = local_now.tzinfo # datetime.timezone
local_tzname = local_tz.tzname(local_now)
print(local_tzname)
While it doesn't use pytz/Pandas, the other answers don't either, so I figured I should post what I'm using on mac/linux:
import subprocess
timezone = subprocess.check_output("date +%Z")
Benefits over the other answers: respects daylight savings time, doesn't require additional libraries to be installed.
I am developing an iOS application that needs to sync with a Python based REST service on GAE.
In the python backend I create my timestamps like this:
def create_timestamp(date):
midnight = datetime.time(0)
date_midnight_time = datetime.datetime.combine(date.date(), midnight)
return calendar.timegm(date_midnight_time.utctimetuple())
I pass in the function above datetime.datetime.today(). This would return for 27 Oct 2013 00:00:00 the value 1382832000.
On iOS there is a buildin function for that:
nextDate is set to today's date a bit complicated due an algorithm:
NSDate *date = [NSDate date];
NSCalendar *calendar = [NSCalendar currentCalendar];
NSDateComponents *components = [calendar components:(NSEraCalendarUnit | NSYearCalendarUnit | NSMonthCalendarUnit) fromDate:date];
[components setDay:27];
NSDate *nextDate = [calendar dateFromComponents:components];
[nextDate timeIntervalSince1970] which returns for 2013-10-27 00:00:00 BST the value 1382828400.000000
There is some discrepancy though.
Maybe its because that Python side is UTC and iOS shows the time in BST by default and I need to address that. As of last night the British Summer time is no more, but iOS still reports BST. Thats confusing though as a NSDate object is always in UTC from my understanding....
Once its working, is it safe to cast the iOS double
value to int, in order to get a round integer number similar to the Python side?
I know this is old, but I thought I'd respond, since this particular issue is something I've been looking for an answer for for a while:
The IOS timestamp you give refers to the correct midnight for GMT of that date.
The python timestamp you give refers to one hour earlier (11:00 pm on the prior day).
This took me forever to find, and it's a pretty smart way of doing it (particularly when you consider the many more circuitous options I've seen):
I tried this, and it works nicely when trying to get the timestamp for a datetime in your local zone (but..):
from datetime import datetime
def dt_to_timestamp(dt_object):
"""Not really a good, universal solution"""
return eval(dt_object.strftime('%s.%f'))
dt_to_timestamp(datetime.now()) - time.time()
# -0.0002155303955078125
..but it fails pretty badly once it comes to looking at objects outside of your local zone:
from tzlocal import get_localzone
from pytz import utc
utc_now = datetime.now(tz=localzone).astimezone(utc)
dt_to_timestamp(utc_now) - time.time()
# 21599.98956131935, more commonly known as 21600 -- my offset from UTC
This is what I finally ended up with:
from datetime import datetime
from pytz import utc
def dt_to_timestamp(dt_object):
"""Provides timestamp for any zone-aware datetime object.
Works with Python 2.7 and Python 3.3, possibly others.
Raises TypeError when naive datetime objects are given.
"""
epoch = datetime(1970, 1, 1, tzinfo=utc)
delta = dt_object - epoch
return delta.total_seconds()
# example usage:
from tzlocal import get_localzone
ts = 1382832000
utc_dt = utc.localize(datetime.utcfromtimestamp(ts))
local_dt = utc_dt.astimezone(get_localzone())
ts == dt_to_timestamp(utc_dt) == dt_to_timestamp(local_dt)
# True
It handles aware datetime objects accurately, whatever their timezone. If the caller doesn't know the timezone in order to turn it into an aware timezone, then there are other problems. :-) I'm of the opinion that one should always use an aware datetime objects if possible, and when not using aware datetime objects, use UTC datetimes.
I found the info for this answer (amongst a lot of other detail) here.