We Keep Coding

Simply save time as UTC with Django

I get a timestamp (via an extern API). When I try to save that, it always is two hours earlier.
I read some docs like Django Time zones, pytz and dateutil but I still don't understand how to tell Django that this specific time should not be converted to UTC because it already is UTC.
It looks like this:
t = '2020-05-29 08:47:39' # this is UTC
MyModel(
timestamp=t,
…
).save()
In the database it is stored as 2020-05-29 06:47:39. So it is shown on a template e. g. as 2020-05-29 08:47:39 whereas 2020-05-29 10:47:39 would be correct.
settings.py
TIME_ZONE = 'Europe/Berlin'
USE_TZ = True

I think I found the solution by myself: the string t must be converted to a datetime-object including the UTC timezone like this:
>>> t = '2020-05-29 08:47:39'
>>> import datetime
>>> dt = datetime.datetime.strptime(t, '%Y-%m-%d %H:%M:%S')
>>> dt
datetime.datetime(2020, 5, 29, 8, 47, 39)
>>> import pytz
>>> tz = pytz.utc.localize(dt)
>>> tz
datetime.datetime(2020, 5, 29, 8, 47, 39, tzinfo=<UTC>)
Saving this with Django, everything is perfect.

Confused by django timezone support

Django beginner here.
In official docs:
# Support for time zones is enabled in the default settings file, so
# Django expects a datetime with tzinfo for pub_date. Use timezone.now()
# instead of datetime.datetime.now() and it will do the right thing.
>>> from django.utils import timezone
>>> q = Question(question_text="What's new?", pub_date=timezone.now())
Trying do reproduce it in ./manage.py shell:
In [35]: from django.conf import settings
In [36]: settings.USE_TZ
Out[36]: True
In [37]: settings.TIME_ZONE
Out[37]: 'Europe/Moscow'
In [38]: from django.utils import timezone
# UTC
In [39]: timezone.now()
Out[39]: datetime.datetime(2015, 10, 16, 9, 47, 50, 755418, tzinfo=<UTC>)
# Actual time
In [40]: timezone.datetime.now()
Out[40]: datetime.datetime(2015, 10, 16, 12, 47, 54, 554197)
# UTC
In [41]: timezone.activate("Europe/Moscow"); timezone.now()
Out[41]: datetime.datetime(2015, 10, 16, 9, 47, 59, 405269, tzinfo=<UTC>)
# Actual time
In [42]: timezone.activate("Europe/Moscow"); timezone.datetime.now()
Out[42]: datetime.datetime(2015, 10, 16, 12, 48, 3, 179085)
When I'm running timezone.now() as specified in documentation, i'm getting UTC which is wrong. When i'm running timezone.datetime.now() (what i think is just call to datetime.datetime.now(), which is using system-wide timezone) i'm getting the right thing.
Tried with different timezones, still getting plain UTC.
What am I doing wrong?

timezone.now() behaves as it should: if USE_TZ=True; it returns the current time as an aware datetime object (in UTC).
2015-10-16 09:47:50+00:00 (UTC) is the same time moment as 2015-10-16 12:47:50+03:00 (MSK). The UTC time is rendered in the templates using your current time zone (defaults to TIME_ZONE setting and therefore it is unnecessary to set it explicitly using timezone.activate() here).
You can get the value explicitly using timezone.localtime(timezone.now()) (you don't need to).

Django: activate() not showing effect

I have the following line in python manager.py shell:
>>> import pytz
>>> from django.utils import timezone
>>> zone = "Asia/Kolkata"
>>> timezone.activate(pytz.timezone(zone))
>>> timezone.now()
datetime.datetime(2014, 12, 17, 1, 52, 0, 411937, tzinfo=<UTC>)
But the output which I get is still using UTC. Should not it be converted into "Asia/Kolkata"?
UPDATE
If i use commands suggested by dazedconfused below:
zone = "Asia/Kolkata"
if zone:
timezone.activate(pytz.timezone(zone))
else:
timezone.deactivate()
utc_date = datetime.utcnow()
aware_date = timezone.make_aware(utc_date, timezone.utc)
l_time = timezone.localtime(aware_date, timezone.get_current_timezone())
And now when i try to save it on my database(Mongodb on Mongolab) it gets saved as UTC
I have a DateTimeField in my database.
Although when i save it as a simple string it gets saved in current timezone that is "Asia/kolkata" Output as string: 2014-12-17 11:01:53.028852+05:30

It actually successfully sets the current time zone to "Asia/Kolkata"
You can verify by:
...
>>> timezone.get_current_timezone_name()
'Asia/Kolkata'
From the django documentation:
now():
Returns a datetime that represents the current point in time. Exactly
what’s returned depends on the value of USE_TZ:
If USE_TZ is False, this will be a naive datetime (i.e. a datetime
without an associated timezone) that represents the current time in
the system’s local timezone.
If USE_TZ is True, this will be an aware
datetime representing the current time in UTC. Note that now() will
always return times in UTC regardless of the value of TIME_ZONE; you
can use localtime() to convert to a time in the current time zone.
So, if your system's local timezone is 'Asia/Kolkata', you can set USE_TZ to False in your settings.py and the timezone.now() will return what you want.
Or, you'll have to use localtime() to convert the timezone to yours (continue from your shell results):
...
>>> import datetime
>>> utc_date = datetime.datetime.utcnow()
>>> aware_date = timezone.make_aware(utc_date, timezone.utc)
>>> timezone.localtime(aware_date, timezone.get_current_timezone())
datetime.datetime(2014, 12, 17, 8, 0, 36, 598113, tzinfo=<DstTzInfo 'Asia/Kolkata' IST+5:30:00 STD>)
Lastly, here's the documentation of the make_aware() function

Python Django: tzinfo doesn't work for DB insertion. But why does .now(local_tz) work?

I am using Django.
in settings:
TIME_ZONE = 'Europe/Copenhagen'
USE_TZ = True
Due to DST, clock skips an hour on 2013-3-31. 01:59 goes to 03:00
I views:
The date and time are given in local time. I want these to be inserted as utc.
Code below correctly saves as UTC, but gives RuntimeWarning: DateTimeField received a naive datetime
the_date = datetime.datetime(2013, 3, 31, 1, 59)
hit = hits(date= the_date); hit.save(); # Correctly saved as 00:59:00
the_date = datetime.datetime(2013, 3, 31, 3, 1)
hit = hits(date= the_date); hit.save(); # Correctly saved as 01:01:00
I thought I could avoid the warning by making the datetime aware. It does avoid the warning, but the conversion is now wrong.
tz = timezone(settings.TIME_ZONE)
the_date = datetime.datetime(2013, 3, 31, 3, 1, tzinfo = tz)
hit = hits(date= the_date); hit.save(); # Incorrectly saved as 02:01:00
The following works, with no runtime error:
I have installed pytz.
the_date = local_tz.localize(datetime.datetime(2013, 3, 31, 3, 1))
Getting to my question:
I get that tzinfo doesn't work because it doesn't account for daylight savings time. Fine, I won't use it. But then I was confused when the following seemed to work:
the_date = datetime.datetime.now(local_tz)
This correctly inserted as utc both in wintertime (where it subtracted 1 hour to get utc) and when I changed my computer systime to a date in the summer (where it subtracted 2 hours to get utc).
My question:
Does .now(local_tz) work or did I test it wrong? Why is this different than tzinfo = tz? Or am I using tzinfo wrong?

I recommend converting to UTC as soon as possible and only use UTC internally. Save for cases where you move across timezones (for example a marine vessel) and really need to save timezone information, for the constant timezone case, it's much simpler to just use local time for input/output and convert it to UTC at the user interface.
To convert from localtime to UTC, you need to use the pytz.timezone.normalize method which handles daylight savings time and other timezone transition. See this section of the pytz documentation. In your case, to convert the local datetime to UTC you need the following:
from pytz import timezone, utc
local_tz = timezone(settings.TIME_ZONE)
local_dt = datetime.datetime(2013, 3, 31, 3, 1, tzinfo = local_tz)
utc_dt = utc.normalize(local_dt.astimezone(utc))

Daylight savings time in Python

I am writing a program which deals a lot with timezones and crossing them. The two things I deal with most are creating a datetime object from "now" and then localizing a naive datetime object.
To create a datetime object from now in the pacific timezone, I am currently doing this (python 2.7.2+)
from datetime import datetime
import pytz
la = pytz.timezone("America/Los_Angeles")
now = datetime.now(la)
Is this correct with regards to DST? If not, I suppose I should be doing:
now2 = la.localize(datetime.now())
My question is why? Can anyone show me a case where the first is wrong and the seconds is right?
As for my seconds question, suppose I had a naive date and time from some user input for 9/1/2012 at 8:00am in Los Angeles, CA. Is the right way to make the datetime like this:
la.localize(datetime(2012, 9, 1, 8, 0))
If not, how should I be building these datetimes?

From the pytz documentation:
The preferred way of dealing with times is to always work in UTC, converting to localtime only when generating output to be read by humans.
So ideally you should be using utcnow instead of now.
Assuming for some reason that your hands are tied and you need to work with local times, you can still run into a problem with trying to localize the current time if you're doing it during the daylight saving transition window. The same datetime might occur twice, once during daylight time and again during standard time, and the localize method doesn't know how to settle the conflict unless you tell it explicitly with the is_dst parameter.
So to get the current UTC time:
utc = pytz.timezone('UTC')
now = utc.localize(datetime.datetime.utcnow())
And to convert it to your local time (but only when you must):
la = pytz.timezone('America/Los_Angeles')
local_time = now.astimezone(la)
Edit: as pointed out in the comments by #J.F. Sebastian, your first example using datetime.now(tz) will work in all cases. Your second example fails during the fall transition as I outlined above. I still advocate using UTC instead of local time for everything except display.

The first solution is correct with regards to DST, and the second solution is bad.
I'll give an example. Here in Europe, when running this code:
from datetime import datetime
import pytz # $ pip install pytz
la = pytz.timezone("America/Los_Angeles")
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
now = datetime.now(la)
now2 = la.localize(datetime.now())
now3 = datetime.now()
print(now.strftime(fmt))
print(now2.strftime(fmt))
print(now3.strftime(fmt))
I get the following:
2012-08-30 12:34:06 PDT-0700
2012-08-30 21:34:06 PDT-0700
2012-08-30 21:34:06
datetime.now(la) creates a datetime with the current time in LA, plus the timezone information for LA.
la.localize(datetime.now()) adds timezone information to the naive datetime, but does no timezone conversion; it just assumes the time was already in this timezone.
datetime.now() creates a naive datetime (without timezone information) with the local time.
As long as you are in LA, you will not see the difference, but if your code ever runs somewhere else, it will probably not do what you wanted.
Apart from that, if you ever need to deal seriously with timezones, it is better to have all your times in UTC, saving yourself a lot of trouble with DST.

This works:
# naive datetime
d = datetime.datetime(2016, 11, 5, 16, 43, 45)
utc = pytz.UTC # UTC timezone
pst = pytz.timezone('America/Los_Angeles') # LA timezone
# Convert to UTC timezone aware datetime
d = utc.localize(d)
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)
# show as in LA time zone (not converting here)
d.astimezone(pst)
>>> datetime.datetime(2016, 11, 5, 9, 43, 45,
tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
# we get Pacific Daylight Time: PDT
# add 1 day to UTC date
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 16, 43, 45, tzinfo=<UTC>)
d.astimezone(pst) # now cast to LA time zone
>>> datetime.datetime(2016, 11, 6, 8, 43, 45,
tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
# Daylight saving is applied -> we get Pacific Standard Time PST
This DOES NOT work:
# naive datetime
d = datetime.datetime(2016, 11, 5, 16, 43, 45)
utc = pytz.UTC # UTC timezone
pst = pytz.timezone('America/Los_Angeles') # LA timezone
# convert to UTC timezone aware datetime
d = utc.localize(d)
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)
# convert to 'America/Los_Angeles' timezone: DON'T DO THIS
d = d.astimezone(pst)
>>> datetime.datetime(2016, 11, 5, 9, 43, 45,
tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
# we are in Pacific Daylight Time PDT
# add 1 day to LA local date: DON'T DO THAT
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 9, 43, 45,
tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
# Daylight Saving is NOT respected, we are still in PDT time, not PST
Conclusion:
datetime.timedelta() DOES NOT account for daylight saving.
Do your time add/subtract in UTC timezone ALWAYS.
Cast to local time only for output / display.

The pytz website says:
Unfortunately using the tzinfo argument of the standard datetime
constructors ‘’does not work’’ with pytz for many timezones.
So you should not use datetime.now(la). I don't know the specifics, but some timezones operate on more exotic rules then we are used to, and python's datetime code can't handle them. By using pytz's code, they should be handled correctly since that's pytz's purpose. It may also have issues for the times that occur twice thanks to jumping times in daylight savings.
As for the second question, that's exactly what the documentation shows, so you should be good.