I am writing an app which depends heavily on dates and times. I want to be able to have an injectable concept of now() and today(). I was thinking that I could write my own versions of these two functions which would check some central setting, to which I will refer to as INJECTED_NOW. If INJECTED_NOW is None, the above functions would just return the values of datetime.datetime.now() and datetime.date.today(). However, if INJECTED_NOW has a datetime value, the above functions would use it to get now() and today().
I am wondering how I could store INJECTED_NOW so that it is mutable. I would like to be able to set it at the beginning of a test case and modify it before another test case. Similarly, I would like to be able to set it from the request, perhaps using middleware.
Does this approach make sense, and if so, how should I store INJECTED_NOW? I would like to avoid a DB access. Is there an alternate way of addressing this problem?
There's a recently released library called FreezeGun that lets specify datetimes like you describe:
http://stevepulec.com/freezegun/
Here is a way to do it using mock, for more information about mock see the docs
# this should be the code your are testing
import datetime
def one_minute_ago():
return (datetime.datetime.now() - datetime.timedelta(seconds=60)).time()
# this would be in your tests file
import mock
import sys
import unittest
class SomeTestcase(unittest.TestCase):
def test_one_minute_ago(self):
real_datetime = datetime.datetime
fake_now = datetime.datetime(2012, 12, 21, 11, 13, 13)
with mock.patch('datetime.datetime', spec=datetime.datetime) as datetime_mock:
datetime_mock.now.return_value = fake_now
self.assertEqual(one_minute_ago(), datetime.time(11, 12, 13))
if __name__ == '__main__':
sys.exit(unittest.main())
To test it just copy the code to a file and run it with Python.
Related
Imagine this situation :
I have a file named settings.py that contains
from datetime import datetime
FOO = datetime.now()
Now, in another file, I do from django.conf import settings (the fact that I'm using Django in this specific situation is irrelevant)
In the file I did the import when I use settings.FOO, no matter how much time has passed I will always get the same datetime value, which corresponds to the time when the import happened.
I understand how and why it works that way. My question is : Is there a way to always have an updated value "stored" in settings.FOO without changing the way I access that value ? (as settings.FOO is used in multiple file across multiple subprojects, I aim to keep that syntax)
In other words, I there a way to make my imported variable transparently call a function ?
As mentioned in the comments, there's almost never a good reason for doing it like this, and you would almost always rely on explicit callables for doing what you do, but by explicitly manipulating sys.modules, if all you care about is syntax, it is possible to do what you're trying to do.
settings.py:
from datetime import datetime
import sys
class _Bar:
#property
def FOO(self):
return datetime.now()
sys.modules[__name__] = _Bar()
Usage example:
>>> import settings
>>> settings.FOO
datetime.datetime(2020, 12, 4, 17, 0, 5, 987423)
>>> settings.FOO
datetime.datetime(2020, 12, 4, 17, 0, 7, 285826)
>>> import dateutil.parser, dateutil.tz as tz
>>> dateutil.parser.parse('2017-08-09 10:45 am').replace(tzinfo=tz.gettz('America/New_York'))
datetime.datetime(2017, 8, 9, 10, 45, tzinfo=tzfile('/usr/share/zoneinfo/America/New_York'))
Is that really the way that we're supposed to set a default timezone for parsing? I've read the documentation for the parser and examples but I cannot seem to find anything that says, "This is how to set the default timezone for dateutil.parser.parse", or even anything like it.
Because while this works, there are cases where it would do the wrong thing, if the zone were provided. Does that mean we should do this?
>>> d = dateutil.parser.parse('2017-08-09 10:45 am +06:00')
>>> d = d.replace(tzinfo=d.tzinfo or tz.gettz('America/Chicago'))
Because that's clunky, too.
What's the recommended way to set a default timezone when parsing?
There are basically two "correct" ways to do this. You can see that this was brought up as Issue #94 on dateutil's issue tracker, and "set a default time zone" is determined to be out of scope, since this is something that can be easily done with the information returned by the parser anyway (and thus no need to build it in to the parser itself). The two ways are:
Provide a default date that has a time zone. If you don't care what the default date is, you can just specify some date literal and be done with it. If you want the behavior to be basically the same as dateutil's default behavior (replacing missing elements from "today's date at midnight"), you have to have a bit of boilerplate:
from datetime import datetime, time
from dateutil import tz, parser
default_date = datetime.combine(datetime.now(),
time(0, tzinfo=tz.gettz("America/New_York")))
dt = parser.parse(some_dt_str, default=default_date)
Use your second method with .replace:
from dateutil import parser
def my_parser(*args, default_tzinfo=tz.gettz("America/New_York"), **kwargs):
dt = parser.parse(*args, **kwargs)
return dt.replace(tzinfo=dt.tzinfo or default_tzinfo)
This last one is probably slightly cleaner than the first, but has a slight performance decrease if run in a tight loop (since the first one only needs the default date created once), but dateutil's parser is actually quite slow, so an extra date construction is likely the least of your problems if you're running it in a tight loop.
Fleshing out Paul's comment - because a datetime has to be at least a year, month, and day, dateutil already has a default that it uses:
>>> from datetime import datetime
>>> datetime.now()
datetime.datetime(2017, 10, 13, 15, 16, 13, 548750)
>>> dateutil.parser.parse('2017')
datetime.datetime(2017, 10, 13, 0, 0)
Given this, the appropriate choice would be to create a default that contains the timezone and is either just the current date, or whatever date makes sense:
>>> dateutil.parser.parse('2017', default=datetime(2017, 10, 13, tzinfo=tz.gettz('America/New_York')))
Naturally you can store the default as something sensible, like default_datetime or something, then it becomes:
>>> dateutil.parser.parse('2017', default=default_datetime)
In my app I find myself using stftime a lot, and mostly with 2 strings formats - ("%d/%m/%Y") and ("%H:%M")
Instead of writing the string each time, I want to store those strings in some global var or something, so I can define the format strings in just one place in my app.
What is the pythonic way of doing that? Should I use a global dict, a class, a function, or maybe something else?
Maybe like this?
class TimeFormats():
def __init__(self):
self.date = "%d/%m/%Y"
self.time = "%H:%M"
Or like this?
def hourFormat(item):
return item.strftime("%H:%M")
Thanks for the help
you could use functools.partial to generate a function holding the format:
import time,functools
time_dhm = functools.partial(time.strftime,"%d/%m/%Y")
time_hm = functools.partial(time.strftime,"%H:%M")
print(time_dhm(time.localtime()))
print(time_hm(time.localtime()))
result:
18/01/2017
10:38
you only have to pass the time structure to the new function. The function holds the format.
Note: you can do the same with lambda:
time_dhm = lambda t : time.strftime("%d/%m/%Y",t)
I think it is better to create a custom function to achieve this. For example:
def datetime_to_str(datetime_obj):
return datetime_obj.strftime("%d/%m/%Y")
Sample run:
>>> from datetime import datetime
>>> datetime_to_str(datetime(1990, 3, 12))
'12/03/1990'
It will be much more friendly for the fellow developers as the function name is self explanatory. Each time conversion of datetime to str is needed, they will know which function is needed to be called. And in case you want to change the format through out the application; there will be single point of change.
You could create your own settings module, like django does.
settings.py:
# locally customisable values go in here
DATE_FORMAT = "%d/%m/%Y"
TIME_FORMAT = "%H:%M"
# etc.
# note this is Python code, so it's possible to derive default values by
# interrogating the external system, rather than just assigning names to constants.
# you can also define short helper functions in here, though some would
# insist that they should go in a separate my_utilities.py module.
# from moinuddin's answer
def datetime_to_str(datetime_obj):
return datetime_obj.strftime(DATE_FORMAT)
elsewhere
from settings import DATE_FORMAT
...
time.strftime( DATE_FORMAT, ...)
or
import settings
...
time.strftime( settings.DATE_FORMAT, ...)
https://pypi.python.org/pypi/DateTime mentions that it is better to use python datetime rather than Zope DateTime. I believe this is done by invoking External Methods. Is there any other way of doing this as I don't think one would like to create an external method for all the kind of stuff that needs to be imported ?
The class DateTime from module DateTime can be imported in the Zope restricted mode simply because it's declared as public in the package AccessControl. In its top level __init__.py
ModuleSecurityInfo('DateTime').declarePublic('DateTime')
It also contain this to allow import from four other utility builtin imports
for name in ('string', 'math', 'random', 'sets'):
ModuleSecurityInfo(name).setDefaultAccess('allow')
So to do so, you need to add similar lines to the __init__.py of your Zope package (or create one if you haven't already, see this tutorial), for instance:
from AccessControl import ModuleSecurityInfo
# just a specific class
ModuleSecurityInfo('datetime').declarePublic('datetime')
# or at a broader level for the entire module
ModuleSecurityInfo('datetime').setDefaultAccess('allow')
This of course assumes the package datetime is trusted enough to not be able to result in an attack against your Zope instance.
just for the record, I had this error when using strftime in a TAL expression (RestrictedPython) :
The container has no security assertions.
Access to 'strftime' of datetime.datetime(2021, 1, 31, 0, 0) denied.
I solved this by using allow_type from AccessControl :
from AccessControl import allow_type
from datetime import datetime
allow_type(datetime)
Gauthier
When I write with business logic, my code often depends on the current time. For example the algorithm which looks at each unfinished order and checks if an invoice should be sent (which depends on the no of days since the job was ended). In these cases creating an invoice is not triggered by an explicit user action but by a background job.
Now this creates a problem for me when it comes to testing:
I can test invoice creation itself easily
However it is hard to create an order in a test and check that the background job identifies the correct orders at the correct time.
So far I found two solutions:
In the test setup, calculate the job dates relative to the current date. Downside: The code becomes quite complicated as there are no explicit dates written anymore. Sometimes the business logic is pretty complex for edge cases so it becomes hard to debug due to all these relative dates.
I have my own date/time accessor functions which I use throughout my code. In the test I just set a current date and all modules get this date. So I can simulate an order creation in February and check that the invoice is created in April easily. Downside: 3rd party modules do not use this mechanism so it's really hard to integrate+test these.
The second approach was way more successful to me after all. Therefore I'm looking for a way to set the time Python's datetime+time modules return. Setting the date is usually enough, I don't need to set the current hour or second (even though this would be nice).
Is there such a utility? Is there an (internal) Python API that I can use?
Monkey-patching time.time is probably sufficient, actually, as it provides the basis for almost all the other time-based routines in Python. This appears to handle your use case pretty well, without resorting to more complex tricks, and it doesn't matter when you do it (aside from the few stdlib packages like Queue.py and threading.py that do from time import time in which case you must patch before they get imported):
>>> import datetime
>>> datetime.datetime.now()
datetime.datetime(2010, 4, 17, 14, 5, 35, 642000)
>>> import time
>>> def mytime(): return 120000000.0
...
>>> time.time = mytime
>>> datetime.datetime.now()
datetime.datetime(1973, 10, 20, 17, 20)
That said, in years of mocking objects for various types of automated testing, I've needed this approach only very rarely, as most of the time it's my own application code that needs the mocking, and not the stdlib routines. After all, you know they work already. If you are encountering situations where your own code has to handle values returned by library routines, you may want to mock the library routines themselves, at least when checking how your own app will handle the timestamps.
The best approach by far is to build your own date/time service routine(s) which you use exclusively in your application code, and build into that the ability for tests to supply fake results as required. For example, I do a more complex equivalent of this sometimes:
# in file apptime.py (for example)
import time as _time
class MyTimeService(object):
def __init__(self, get_time=None):
self.get_time = get_time or _time.time
def __call__(self):
return self.get_time()
time = MyTimeService()
Now in my app code I just do import apptime as time; time.time() to get the current time value, whereas in test code I can first do apptime.time = MyTimeService(mock_time_func) in my setUp() code to supply fake time results.
Update: Years later there's an alternative, as noted in Dave Forgac's answer.
The freezegun package was made specifically for this purpose. It allows you to change the date for code under test. It can be used directly or via a decorator or context manager. One example:
from freezegun import freeze_time
import datetime
#freeze_time("2012-01-14")
def test():
assert datetime.datetime.now() == datetime.datetime(2012, 1, 14)
For more examples see the project: https://github.com/spulec/freezegun
You can patch the system, by creating a custom datetime module (even a fake one - see example below) acting as a proxy and then insert it in sys.modules dictionary. From there on, each import to the datetime module will return your proxy.
There is still the caveat of datetime class, especially when someone does from datetime import datetime; for that, you can simply add another proxy only for that class.
Here is an example of what I am saying - of course it is just something I've thrown in 5 minutes, and may have several issues (for instance, the type of datetime class is not correct); but hopefully it may already be of use.
import sys
import datetime as datetime_orig
class DummyDateTimeModule(sys.__class__):
""" Dummy class, for faking datetime module """
def __init__(self):
sys.modules["datetime"] = self
def __getattr__(self, attr):
if attr=="datetime":
return DummyDateTimeClass()
else:
return getattr(datetime_orig, attr)
class DummyDateTimeClass(object):
def __getattr__(self, attr):
return getattr(datetime_orig.datetime, attr)
dt_fake = DummyDateTimeModule()
Finally - is it worth?
Frankly speaking, I like our second solution much more than this one :-).
Yes, python is a very dynamic language, where you can do quite a lot of interesting things, but patching code in this way has always a certain degree of risk, even if we are talking here of test code.
But mostly, I think the accessory function would make test patching more explicit, and also your code would be more explicit in terms of what it is going to be tested, thus increasing readability.
Therefore, if the change is not too expensive, I would go for your second approach.
I would use the helpers from the 'testfixtures' package to mock out the date, datetime or time calls you're making:
http://packages.python.org/testfixtures/datetime.html
Well one way to do it is to dynamic patch the time /datetime module
something like
import time
import datetime
class MyDatetime:
def now(self):
return time.time()
datetime.datetime = MyDatetime
print datetime.datetime().now()
there might be few ways of doing this, like creating the orders (with the current timestamp) and then changing that value in the DB directly by some external process (assuming data is in the DB).
I'll suggest something else. Have you though about running your application in a virtual machine, setting the time to say Feb, creating orders, and then just changing the VMs time? This approach is the closest as you can get to the real-life situation.