We Keep Coding

SqlAlchemy Time/Date issue with data stored as UTC in Postgresql DB

I know there are quite a few questions about Date/Time with Flask/SQLAlchemy and Postgres, but I have a problem that I just can't quite figure out. So, with my project I am downloading statistics from ESPN. All their game times, rightly so, are in UTC format. Example: 2020-11-26T02:56Z
So, I add this to me DB in a field of "timestamp with time zone" which I've learn is almost the standard. So, my issue I have is I'm trying to display a list of what games are "today" and I'm in the central time zone of America. I don't know how to offset my filter request for this. So, two questions:
should I keep the db format of timestamp with time zone and then figure out a way to filter those based on current day, central time zone, of those games
or
just convert the game times to Central Time before inserting them into my db. I don't foresee using this app outside of my current time zone, but I'm not 100% sure of that.
Am I just missing something obviously easy? When I view the db in PG Admin 4, the date/times are my timezone (db is on my laptop), but the api I'm writing returns json with it being UTC time.
This is what I originally tried, but obviously didn't work:
events = Event.query.filter(Event.gamedate > pytz.utc.localize(datetime.today()- timedelta(days=1)),Event.gamedate < pytz.utc.localize(datetime.today() + timedelta(days=1))).all()

When you load an UTC timestamp into timestamp with time zone, you have to set the timezone parameter correctly in that session:
SET timezone = 'UTC';

How should I handle datetimes with Django and PosgeSQL?

I have a Django app which needs to support different time zones. I'm saving events to my database which have timestamps and based on those timestamps I want to send notifications. Because the user can move into different time zone, I want to store timestamps as UTC time and convert them to user current time zone later. If I'm correct, PostgreSQL converts datetimes to UTC automatically? At the moment I get an error RuntimeWarning: DateTimeField event.timestamp received a naive datetime (2022-02-15 15:00:00) while time zone support is active.. I used to use Django's make_aware when saving the timestamp and didn't get this warning then. The questions I'd like to get an answer are:
Should I save aware datetimes to the database if I want to support multiple time zones?
What should I use in variables TIME_ZONE and USE_TZ in settings.py? UTC and True?
I store the time zone info to the database, what is the correct method to activate it when I want to get the local time for the user? django.utils.timezone.activate + get_current_timezone or is there some function like astimezone(string_from_db)?

Should I save aware datetimes to the database if I want to support multiple time zones?
The thing that helped me understand timezones is the fact, that time when presented as unix timestamp is a constant and does not depend on where the user is on the globe. Timestamp is saved into Postgres as such. Timezone is just an optional extra information, nothing more. One can take unix timestamp number and transform it into Timezone A, B or whatever he wishes. This depends totally on the request.
Storing timezone information to the database only says that you have to provide timezone when persisting model attribute, otherwise you will get that warning. Warning means that database doesn't know how to treat it and will consider it to be in default timezone, which is usually wrong.
I find it very convenient to store timezone information into the database, but not because it "supports multiple time zones", but because it is less confusing and cleaner. Your users are already in multiple time zones and they are all expecting to see time in their own timezone. If you use timezones, then you don't need to convert to and from UTC and ponder if this number is already converted or not.
I faced issues such as: when a user creates an item, after a single second he sees that an item was created 2 hours ago because you did not include timezone in creation time.
What should I use in variables?
USE_TZ = True
TIME_ZONE = "UTC"
In Django 5.0 the USE_TZ will become True by default.
what is the correct method to get the local time for the user
If you return time values with timezone, the JavaScript easily transforms them to user's timezone by default. If you are using Django templates and all times should appear in different timezone than the user is in, then it is a bit more involved: need to save user's preferred timezone in his profile and activate it with timezone.activate as you correctly pointed out.

PostgreSQL SQLalchemy and timestamp with time zone, why is incognito different from normal?

I am using flask sqlalchemy and postgreSQL and I have issues with the displayed datetimes, while investigating this issue I found an other weird thing:
Creating a DB entry in incognito mode (chrome browser tab) gives a different/wrong time.
EDIT: It has nothing to do with incognito mode, both cases happen in normal mode aswell. I have not figured out yet why.
This is the code:
I changed the default time zone of my database:
ALTER DATABASE postgres SET timezone TO 'Europe/Berlin';
the model:
class User(UserMixin, Base):
__tablename__ = 'users'
date_added = Column(DateTime(timezone=True), nullable=False)
The method I use to add datetime to DB:
date_added=datetime.today()
How it looks in the DB (my localtime at this time was 13:53:46):
Creating entry not in incognito
timestamp with time zone
2019-02-01 13:53:46.73817+01
Creating entry in incognito
timestamp with time zone
2019-02-01 12:53:46.73817+01
This really makes me worry. It is simply wrong. Even if I would convert the datetime objects to localtime. Both entries have been done at the same time, but show different results, how is this possible?
Also when viewing these dates in HTML, postgreSQL does not apply the offset, so the first date looks correct, but the second is wrong.
Initially I just wanted to find a way to store ALL datetime objects in Europe/Berlin and return them in Europe/Berlin time, so I dont have to convert UTC to Europe/Berlin, but now I think something went horribly wrong.
I also have double checked my code everywhere, I am not using other methods to manipulate datetime objects.
EDIT
I am saving a datetime everytime a user logs in. Currently I tried this on not incognito. My localtime is 14:13:33 BUT it saved into the DB:
2019-02-01 13:13:33.804339+01. How is this even possible? I know it cant be random, but right now it looks like its saving times randomly sometimes UTC with offset and sometimes Europe/Berlin with offset.
EDIT
I double checked all tables in question with SHOW timezone; and they all correctly return Europe/Berlin

datetime.today() returns a timestamp without timezone info of the current local time (the returned value is timezone naive). The root of your problem is that somewhere between SQL Alchemy's postgres adapter and postgres itself it has to guess at a timezone. As you might imagine, computer systems tend toward assuming UTC if no timezone is explicitly provided, but the precise logic for a toolsuite set can be complex and hard to debug (and my depend on your local time zone settings on your computer, system-level settings in the db, session level settings, and the preference of the toolmaker). You can sidestep this entire can of worms by either:
Storing all of your timestamps without a time zone in UTC, then converting to the desired timestamp as appropriate
Always using timestamps with time zone (i.e. replace datetime.today() with datetime.now() and pass in the desired time zone so that you are always dealing with a timezone aware value ), so there is no need for the computer to assume a timezone.
Note that in postgres, the timestamp with time zone type is still always stored as UTC with no extra information, the database simply uses the session-level configuration to decide which time zone to display it as when outputting it.

Why python datetime replace timezone is returning different timezone?

I am working on Python/Django project. I am trying to let user select date and time using jQuery plugin datetimepicker add-on. So when I select now option and post data django is saving the time in UTC offset. This is what is saved in database, 2017-03-30 13:38:00+00:00. I need to convert this time from user's timezone and save it in system as utc. Because later I will have script running which will look for data in database which is less than the utc time.
Actually the script is to let user post information on website and let them chose the publish date and time. So for example, If use posted an article which will be published on April 2nd 1pm Chicago time, I don't want other users to read the article before this time. So people around the world can read article as soon as it is April 2nd and 1PM in Chicago. So how can I make this functionality work?
My solution was to get the time and remove it's timezone information using replace(tzinfo=pytz.timezone('America/Chicago')) and when I print the time, I am getting 2017-03-30 13:38:00-05:51. The actual offset right now is -05:00. Can anyone help me to and tell me what I am doing wrong?
What I am doing for form is that I have publish_date object in my model and I am using django forms to create form. I have added class as an attribute in it and using jquery plugin,
$('.datepicker').datetimepicker({
timeFormat: 'HH:mm',
stepHour: 1,
stepMinute: 1,
});
So when User submits the form, on post method this my code,
form = PublishForm(request.POST)
if form.is_valid():
f = form.save(commit=False)
f.created_by_user_id = request.user.id
f.save()
and than to get the date all I am doing is f.publish_date and the other options I have used lice replace and localize are pretty standard.
Thanks

As noted in the comments, you appear to have two problems. First is that the datetimepicker is saving a date and time with a UTC timezone offset, when it should be applying a different timezone or leaving off the timezone offset entirely. Second is that pytz is using the wrong offset.
I don't know how to fix the first problem, but I can help you with the second. You need to use pytz's localize function. This only works when the datetime doesn't have a timezone attached, but since you know the timezone is incorrect anyway you can delete it first.
tz = pytz.timezone('America/Chicago')
dt = tz.localize(dt.replace(tzinfo=None))

The naming of the datetime replace(tzinfo = ...) function is unfortunate. In fact, its behaviour is random. Do not use this!
Mark's answer is the way to go. Use localize.

timestamp wth timezone vs django

My Oracle DB has a table with a column of type TIMESTAMP WITH TIMEZONE. The Django model converts it to datetime but loses the original TZ info and assumes UTC.
DB: 01-JAN-17 12.10.01.809000 AM -05:00
Django:2017-01-01 00:10:01.809000+00:00
USE_TZ = True
The problem is that the table contains timestamps from different timezones. So if I change the standard timezone with activate() it won't help.
Is there a way for Django to explicitly read the timezone from the database without assuming it?
PS.: I'm aware that it would be better/easiest if my database would only contain UTC values but this is what I got.

I faced the similar problem earlier, what i did was, I saved the time and timezone separately and used pytz and tz_local, which provides a timezone database and local timezone time respectively. While saving the time use one of these library to get the timezone and save time and timezone separately.