Background
I've been unable to find a deeper explanation of the real source of the time value used by Python. Most of the documentation states that it 'gets the value from CPython' but does not go into detail about where it comes from beyond that point.
Some countries have recently changed or are looking to change their DST policies. This means that devices in those countries may auto-adjust the time zone and produce an incorrect GMT-based time until they are updated with the current policies.
That's not terrible on it's own, because operating systems usually have a great mechanism for updates. However, updating Python on those same systems is a very murky area sometimes. The OS may handle it, or perhaps Python is bundled with the application. Perhaps the OS is still getting critical updates but not Python updates.
The Problem / Question
We do not control how Python is being updated with our code. The concern is that we may get an incorrect unix-time-stamp from Python due to the GMT time calculation being out-of-date.
If Python delegates the GMT time calculation to the OS, then we can rest easy. If Python does not delegate it, then we may have to force the value to come from the OS (e.g. time+%s).
So the question is: Does Python3 get it's GMT time from the OS or from it's own calculation based on local time?
Python does not keep its own database of timezone information; it delegates to the OS and/or the pytz third-party package for all timezone-related calculations. UTC or local timestamps are fetched from the OS via the C gmtime_*() and localtime_*() functions from the platform's time.h.
For the full story in source, have a look at the Python version of the datetime module, the C version of the same, the time module, and the pytime.c C module.
Related
There are several Python packages that implement the datetime.tzinfo interface, including pytz and dateutil. If someone hands me a timezone object and wants me to apply it to a datetime, the procedure is different depending on what kind of timezone object it is:
def apply_tz_to_datetime(dt: datetime.datetime, tz: datetime.tzinfo, ambiguous, nonexistent):
if isinstance(tz, dateutil.tz._common._tzinfo):
# do dt.replace(tz, fold=...)
elif isinstance(tz, pytz.tzinfo.BaseTzInfo):
# do tz.localize(dt, is_dst=...)
# other cases here
(The dateutil.tz case is a lot more complicated than I've shown, because there are a lot of cases to consider for non-existent or ambiguous datetimes, but the gist is always to either call dt.replace(tz, fold=...) or raise an exception.)
Checking dateutil.tz._common._tzinfo seems like a no-no, though, is there a better way?
I apologize for having to be the guy to say, "You shouldn't be doing that in the first place", but you indeed should not be trying to detect whether a time zone is a dateutil zone, you should instead just use it as a tzinfo object.
From your comments, it seems like the only reason you want to detect this is because pytz has a separate localization / normalization stage, but since pytz is the only library with this unusual interface, it should suffice to detect pytz zones.
As I mentioned in my comment on the dateutil issue, my recommendations are to either:
Not support pytz at all, if that is possible. It is effectively legacy software at this point, and if you have a new library you at least don't have any users who are already expecting to use it with pytz.
If that is not feasible, something like pytz-deprecation-shim might be a useful abstraction here. For a new library, I wouldn't recommend introducing time zones like those provided that also expose a pytz-like interface, but the helper functions (which don't require a dependency on pytz!) can be profitably used or re-implemented to either detect pytz zones or seamlessly upgrade them to their modern equivalents. You could also use this in combination with #1 by detecting if a zone is a pytz zone and throwing an error.
In any case, there is no particular reason to enumerate all the different time zone providers, since all except pytz use the standard interface.
It appears from the ratio of comments to answers (currently 9/0 = ∞), there is no available answer to the surface-level question (how to determine whether something is a dateutil.tz-style timezone object). I'll open a feature request ticket with the maintainers of the library.
I am developing an application to manage shifts of operation of Internet of Things devices which can be anywhere on the planet using different time zones, I am using python, mongodb and pymongo.
As we all know, when we store a date in mongodb it automatically converts the date of our time zone to UTC by changing the time but keeping the same absolute value below.
The problem I have is that when I query the database to get the date using pymongo I get exactly the same date and time that is stored in mongodb in UTC but in my time zone without having made the conversion change;
for example, Let's suppose I create a datetime in python using the "datetime" library:
MY_TIMEZONE = pytz.timezone('America/Bogota')
datetime.now().astimezone(MY_TIMEZONE)
and I get this date in my time zone:
'2021-02-25 00:00:00-05:00'
look at -05:00 at the end
when I save that date in my mongodb collection in atlas, it transforms the date to UTC which looks like this:
'2021-02-25 05:00:00.000+00:00'
look at +00:00 at the end
So far so good, the problem comes when I query the database to obtain that same date using pymongo method "find_one"
what i get is this:
'2021-02-25 05:00:00-05:00'
look at -05:00 at the end
the same time and date as UTC but in my time zone, which in theory should not happen, I should get the date in UTC or in the local time zone but with the change. That bug can be critical in the system I'm developing.
Does anyone know what could be happening, if it is some kind of bug in the current versions?
I am using python 3.8.5, pymongo 3.11.3 and mongodb 4.4.4
I have looked everywhere about this problem, but it seems that I am the only one that has happened, adding or subtracting the time difference in local is not an option, I guess I will have to choose to use timestamps to avoid this problem or another headache.
If your application is working with multiple time zones, you should configure your system (generally preferable) or your application to use UTC as the time zone.
If you do this, you have two time zones you generally need to keep track of: UTC, which is what every timestamp is stored in, and a local time zone, which is used for output and presentation.
If you don't do this (i.e. your system and/or application time zone is anything other than UTC), you need to keep track of 3 time zones and more than 3 possible combinations of how things can go wrong:
The time zone that your input is in (is it system time zone or the non-system local time?)
When you store a timestamp, is it provided in UTC, system local time or non-system local time?
etc.
If your application is working with a single time zone, AND all of your software is built for this correctly, you can generally assume all times are in local time and ignore time zones.
This last bit isn't always explicitly documented. For example in Ruby, the base driver timezone behavior is described here and it doesn't say that all times are returned in local time zone. Mongoid documentation has a more extensive treatise of time zones, because they are explicitly configurable.
For pymongo I suggest reviewing the entirety of documentation carefully looking for equivalent statements (and make sure you review the bson (de)serialization docs as well), and failing that you might need to read the source code to ascertain behavior.
In any event, working with multiple time zones when your system/application time zone is not UTC is really more difficult than it should be and I don't recommend it (if not for yourself, then whoever will have to work with your code after you).
I am a beginner in Python and trying to understand about working with dates and times. Could you help me understanding the following concept?
In time module, we have the ability to get the local time zone by doing the following,
import time
if time.daylight:
local_timezone_name = time.tzname[0]
utc_offset = time.timezone
else:
local_timezone_name = time.tzname[1]
utc_offset = time.altzone
So what I want to clarify are,
Isn't it suffice to just use the time module as above when dealing with local time zones instead of using pytz module? If not why pytz module is necessary?
What advantages do one gain by using pytz module when dealing with dates and times in Python? (I don't understand how it differs from what Python already has built into it and how it handles dates and times efficently?)
Thank you.
A long story.
Note: pytz is now obsolete (since two weeks), so with Python 3.9 you can uses directly standard libraries.
The problem: pytz is updated independently from Python releases, so it could change quickly the timezone changes. It is not so seldom that within a week or so a country decide to change daylight time implementation. This would cause a lot of problem using standard Python, were you do not update so often (and it will requires a lot of burdens, just to update one line in timezone file). So Python didn't include timezone information, but for UTC (but it could handle time offsets and daylight flag), and some information about your local timezone (and ev. daylight).
Now it seems they found a solution, but Python 3.9 is very new (it was released two weeks ago).
So, if you do not have python 3.9, pytz is the way to go. You have updated timezones and daylight periods.
I've read a bunch of posts on how flaky parsing time can be. I believe I have come up with a reliable way of converting an ISO8601-formatted timestamp here:
https://gist.github.com/3702066
The most important part being the astimezone(LOCALZONE) call when the date is parsed. This allowed time.mktime() to do the right thing and appears to handle daylight savings properly.
Are there obvious gotchas I've missed?
Your code does seem to work even for times that fall just before or just after daylight savings time transitions, but I am afraid it might still fail on those rare occasions when a location's timezone offset actually changes. I don't have an example to test with though.
So even if if does work (or almost always work), I think it's crazy to convert a UTC time string to a UTC timestamp in a manner which involves or passed through local time in any way. The local time zone should be irrelevant. It's an unwanted dependency. I'm not saying that you're crazy. You're just trying to work with the APIs you are given, and the C library's time APIs are badly designed.
Luckily, Python provides an alternative to mktime() that is what the C library should have provided: calendar.timegm(). With this function, I can rewrite your function like this:
parsed = parse_date(timestamp)
timetuple = parsed.timetuple()
return calendar.timegm(timetuple)
Because local time is not involved, this also removes the dependency on pytz and the nagging doubt that an obscure artifact of somebody's local timezone will cause an unwanted effect.
I'd like to be able to raise another application's Window using Python.
I did see this, which I suppose I could try:
X11: raise an existing window via command line?
However, I'd prefer to do it in Python if at all possible.
To activate another window, the right thing to do on the Xlib protocol layer is to send a _NET_ACTIVE_WINDOW message as described in the EWMH spec
http://standards.freedesktop.org/wm-spec/wm-spec-1.3.html
This could be done with python-xlib (presumably) or with gdk_window_focus() on a foreign GdkWindow using GDK through pygtk
_NET_ACTIVE_WINDOW is superior to XRaiseWindow() and has been in all the important WMs for many many years.
You should avoid XSetInputFocus() which will cause problems (especially if you get the timestamp wrong). The issue is that the WM can't intercept the SetInputFocus() so it causes weird race conditions and UI inconsistencies.
Really only _NET_ACTIVE_WINDOW works properly, which is why it was invented, because the previous hacks were bad.
There is a library called libwnck that will let you activate windows (among other things) but unfortunately it adds quite a lot of overhead because it always tracks all open windows from any app, even if you don't need to do that. However if you want to track windows from other apps anyway, then libwnck has a function to activate those windows that does the right thing and would be a good choice.
The strictly correct approach is to check for EWMH _NET_ACTIVE_WINDOW support (EWMH documents how to do this) and fall back to XRaiseWindow if the WM doesn't have _NET_ACTIVE_WINDOW. However, since any WM that's been actively worked on in the last many years has EWMH, lots of people are lazy about the fallback for legacy WMs.
You need to use python-xlib and call .circulate(Xlib.X.RaiseLowest) on the window object (which can be identified in many, many different ways -- can't guess which one is appropriate for you from the zero amount of info about it in your Q;-). For a great example of using python-xlib, check out the tinywm window manager -- after the C version, the author gives a Python version that takes about 30 non-blank, non-comment lines (for a usable, if tiny, window manager...!-).
You can have a look at the python ewmh package. Documentation contains examples, but here is how you can achieve what you want:
from ewmh import EWMH
import random
ewmh = EWMH()
# get every displayed windows
wins = ewmh.getClientList()
# let's active one window randomly
ewmh.setActiveWindow(random.choice(wins))
# flush requests - that's actually do the real job
ewmh.display.flush()