I'm looking for a way to store a timezone-aware datetime object in mongoDB.
I read here: Create an ISODate with pyMongo that storing a datetime object in MongoDB with pymongo will result in a ISODate object being stored (or an object that looks like this when querying from the shell)
Using the datetime.replace(tzinfo=[...]) method, there is a way to make the datetime object aware of its timezone. Unfortunately, when storing such object in mongoDB, the timezone information seems to be lost.
My question is: is there a way to store the timezone in the ISODate object itself, so that it looks like something like this: ISODate("2012-07-14T01:00:00+01:00") when querying the shell and so that the datetime object is still timezone aware when reloaded with pymongo later on?
Thanks a lot for your help!
No, there's not a way to do that. MongoDB's ISODate is just a wrapper around a Date object, which is just a moment in time represented by an integer corresponding to the number of milliseconds elapsed since Jan 1, 1970 UTC. There is no way for it to retain an offset.
You might consider storing it just as a string, perhaps in a secondary field so you can still query for that moment in time while still knowing the local time and offset.
Also, assuming you're using tzdb time zones from either pytz or dateutil, such as "Europe/London", you should recognize that there's no way to determine a time zone from just an offset. So if you really need to reconstruct a full tz aware datetime, then you'll also have to store the time zone identifier in a separate field.
from datetime import datetime
from dateutil.tz import *
def to_utc(date_time):
return date_time.replace(tzinfo=tzlocal()).astimezone(tzoffset(None, 0))
Related
class Test(Model):
time = DateTimeField()
# ...
row = Test.select()[0]
test.time
This returns a string that looks like this: 2017-01-23 01:01:39+01:00. How can I get it as a datetime object instead? Do I have to parse it manually?
Also I would be interested if there is any documentation on how to use the DateTimeField. The official documentation doesn't have anything on it.
Are you using SQLite? If so, SQLite doesn't have a dedicated datetime type, so datetimes are stored as strings in the DB. What peewee will do is recognize certain datetime formats coming out of the DB and convert them to datetime objects. What you need to do is ensure that either:
When you create/save your object, that you assign a datetime object to the field.
When reading back pre-existing data, that the data is in a recognized format.
The formats peewee supports out-of-the-box for datetime field are:
YYYY-mm-dd HH:MM:SS.ffffff
YYYY-mm-dd HH:MM:SS
YYYY-mm-dd
It looks like your has zone info. I'd suggest converting to UTC and dropping the zone info. That should fix it.
Have you tried adding a default like this?
time = DateTimeField(default=datetime.datetime.now())
Or when adding an entry add it as a datetime.datetime object directly:
test = Test(....., time=datetime.datetime.strptime("2018-3-15", '%Y-%m-%d'))
In the second case you don't need to specify anything in the class definition...
I have noticed that when I return records from my SQL database using the following: the_records = records.objects.filter(datetime__contains="2015-01-15"), I get back the wrong records because the timezone is affecting the function call somehow - I know this because if I temporarily disable the timezone, the right records are returned. Can anyone offer assistance on what I should do to fix this problem (I still need to use the timezone).
Regards, Mark
I'm assuming that datetime is a Django DateTime field, and you're trying to get the results that have a value that matches the date '2015-01-15', ignoring the actual time.
In that case, you probably want to do a date query, like: Records.objects.filter(datetime__date=datetime.date(2015, 1, 15))
If you need to query your db with a timezone specific date, you can just create a datetime object that is aware of it's timezone.
Example:
# Get your timezone
from django.utils import timezone
my_timezone = timezone.get_current_timezone()
# Get create your timezone aware datetime object
from datetime import datetime
query_date = datetime(2015,01,15).replace(tzinfo=my_timezone)
# now you can run your query with a timezone specific datetime object
the_records = records.objects.filter(datetime=query_date)
This should solve your issue and get you the accurate results you need.
Please let me know if you still have any questions.
This link has more info related to django timezones if you are interested in learning more.
I have been reading python's datetime documentation and it seems like there are few things which are not clearly mentioned there.
date = datetime.datetime(2014,10,1,11,45,30)
Which timezone will the above date be in? UTC?
If I have to make sure that the above date remains in EST what could be done. I am not clear on tzinfo object here?
If I have to convert these datetimes to some other time zones based on latitude and longitude what should I do?
Your code would create a "timezone naive" datetime object. That means - no timezone. It'll be interpreted as local time based on where it is used.
If you want to set a timezone, try using the pytz library.
import pytz # 3rd party: $ pip install pytz
u = datetime.utcnow()
u = u.replace(tzinfo=pytz.utc) # NOTE: it works only with a fixed utc offset
# Then you can change timezones, e.g. http://www.timezoneconverter.com/cgi-bin/zoneinfo?tz=America/New_York
print u.astimezone(pytz.timezone("America/New_York"))
As for the lat/lon conversion to a timezone, this isn't a simple task. Here's a question that discusses possible solutions.
This question already has answers here:
pytz localize vs datetime replace
(4 answers)
Closed 8 years ago.
Assume that I have a timezone-less datetime object:
import datetime
import pytz
fmt = "%Y-%m-%d %H:%M:%S %Z%z"
dtUnaware = datetime.datetime(1979,2,20,6)
print(dtUnaware.strftime(fmt))
This yields:
1979-02-20 06:00:00
So far, so good. Now, I want to assign a timezone to this object. It seems like I could use either datetime.replace or pytz.localize.
First:
dtAware1 = dtUnaware.replace(tzinfo=pytz.timezone('Asia/Jerusalem'))
print(dtAware1.strftime(fmt))
returns: 1979-02-20 06:00:00 LMT+0221. Secondly:
dtAware2 = pytz.timezone('Asia/Jerusalem').localize(dtUnaware, is_dst=None)
print(dtAware2.strftime(fmt))
returns 1979-02-20 06:00:00 IST+0200.
What's wrong with the first method? It seems to assign a wrong timezone. Am I doing something wrong?
From Pytz documentation : This library differs from the documented Python API for
tzinfo implementations; if you want to create local wallclock
times you need to use the localize() method documented in this
document... Unfortunately these
issues cannot be resolved without modifying the Python datetime
implementation (see PEP-431)
My reading of that is that a pytz timezone is not exactly the same thing as a standard timezone. If you had a genuine timezone, first method should be good, but you have not.
There is a flaw in the datetime API: when you assign a timezone to it, the timezone is not given the opportunity to know the date and time it should be configured for. The historical database that pytz uses goes back a long way, often to periods when the timezone name or offset were different than the ones in use today. Without being given the chance to adjust to a specific date period, the details of the timezone may be wrong.
Using localize is the way around this problem, since both the date and timezone are available within the function at the same time.
So, we have 'Europe/Moscow' TZ in our settings.
Currently this means daylight saving (this is going to change in the future, but at the moment it's UTC+03/04).
I understand that this TZ is used when saving dates to the DB, and when extracting them.
Now, I have to serialize the datetime object to ISO string, including the UTC offset. What is the correct way of doing this?
The dates don't contain the TZ info (i.e. d.strftime('%z') is empty)
I think I could convert them to UTC and serialize with +00:00, but how do I convert them to UTC if I don't know if the specific date is +03 (Moscow winter) or +04 (Moscow summer)
how do I convert them to UTC if I don't know if the specific date is +03 (Moscow winter) or +04 (Moscow summer)
There is no need for UTC conversion, pytz handles such thing for you.
Here's the code to convert from timezone-naive datetime to ISO with timezone offset:
from datetime import datetime
from pytz import timezone
server_timezone = timezone('Europe/Moscow')
server_timezone.localize(datetime(2011, 1, 1)).isoformat()
>>> '2011-01-01T00:00:00+03:00'
server_timezone.localize(datetime(2011, 7, 1)).isoformat()
>>> '2011-07-01T00:00:00+04:00'
First run new_dt = datetime.replace(tzinfo=tz) to create a new timezone-aware datetime. Then run your datetime.strftime() with %z.
Note that you can't then convert the date string back to a timezone-aware datetime directly -- datetime.strptime() doesn't support %z. So you need to instead create a naive datetime and a tzinfo then do datetime.replace(tzinfo=tz) as before.
Some useful external libraries:
http://pytz.sourceforge.net/
http://code.google.com/p/parsedatetime/
http://labix.org/python-dateutil
Also try searching right here on Stack Overflow for more questions on (Python OR django OR appengine) AND (datetime OR timezone OR date OR time OR tzinfo).
ISO-8601 has no notion of "Timezones", only dates and times, and as a convenience, times may be presented with an "offset".
To make matters even more annoying, datetime has only a half-hearted nod in acknowledgement of timezones; the tzinfo attribute on datetime objects is optional, and no implementation of that interface is provided by the main-line python.
The standard answer to this is to just always do everything in UTC; including having the server in UTC, not in local time. This is actually a pretty good idea; it's not the times that are different, only the way individual users prefer to read those times (which is similar to preferring '42' over '0b101010').
You can satisfy your users' (reasonable!) desire to view times in their local timezone by storing the preferred timezone (perhaps just a sitewide preference if everyone is in Moscow) separately from the actual times, and then you can use a robust timezone library (like pytz) to do the formatting to and from local time.