We Keep Coding

Converting localized string date representation to UTC in python

How do I convert the following localized datetime string to UTC datetime. The string has date, time and timezone mentioned in it e.g. May 15,2015, 04.24AM IST, here IST is Indian Standard Time. It can be any time zone.
I tried using pytz but couldn't make it work.

The thing is that it's quite difficult to parse a string with an abbreviated timezone information. But, if you know your timezone, then you can look up it's name recognized by pytz. You can even list all timezone names with pytz.all_timezones.
In your case it is 'Asia/Calcutta' and this should work to convert it to UTC. Just strip the timezone information from the string and add it later:
import pytz
import datetime
# String without timezone info
str = "May 15, 2015, 04.24AM"
# Parse it to naive time
dt = datetime.datetime.strptime(str, "%b %d, %Y, %I.%M%p")
# Localize it as Asia/Calcutta
dt_localized = pytz.timezone('Asia/Calcutta').localize(dt)
# Convert it to UTC
dt_utc = dt_localized.astimezone(pytz.timezone('UTC'))
And we get:
>>> dt
datetime.datetime(2015, 5, 15, 4, 24)
>>> dt_localized
datetime.datetime(2015, 5, 15, 4, 24, tzinfo=<DstTzInfo 'Asia/Calcutta' IST+5:30:00 STD>)
>>> dt_utc
datetime.datetime(2015, 5, 14, 22, 54, tzinfo=<UTC>)

First, let's strip your date string of the IST timezone and then assign it the Asia/Calcutta timezone.
import pytz
dt_str = "May 15, 2015, 04.24AM IST"
dt_str_naive = dt_str[:-4]
new_dt = (pytz.timezone('Asia/Calcutta')
.localize(dt.datetime.strptime(dt_str_naive, "%b %d, %Y, %I.%M%p")))
Now that it is timezone aware, you can assign it to the UTC timezone:
>>> new_dt.astimezone(pytz.UTC)
datetime.datetime(2015, 5, 14, 22, 54, tzinfo=<UTC>)

Display Python datetime without time

I have a date string and want to convert it to the date type:
I have tried to use datetime.datetime.strptime with the format that I want but it is returning the time with the conversion.
when = alldates[int(daypos[0])]
print when, type(when)
then = datetime.datetime.strptime(when, '%Y-%m-%d')
print then, type(then)
This is what the output returns:
2013-05-07 <type 'str'>
2013-05-07 00:00:00 <type 'datetime.datetime'>
I need to remove the time: 00:00:00.

print then.date()
What you want is a datetime.date object. What you have is a datetime.datetime object. You can either change the object when you print as per above, or do the following when creating the object:
then = datetime.datetime.strptime(when, '%Y-%m-%d').date()

If you need the result to be timezone-aware, you can use the replace() method of datetime objects. This preserves timezone, so you can do
>>> from django.utils import timezone
>>> now = timezone.now()
>>> now
datetime.datetime(2018, 8, 30, 14, 15, 43, 726252, tzinfo=<UTC>)
>>> now.replace(hour=0, minute=0, second=0, microsecond=0)
datetime.datetime(2018, 8, 30, 0, 0, tzinfo=<UTC>)
Note that this returns a new datetime object -- now remains unchanged.

>>> print then.date(), type(then.date())
2013-05-07 <type 'datetime.date'>

To convert a string into a date, the easiest way AFAIK is the dateutil module:
import dateutil.parser
datetime_object = dateutil.parser.parse("2013-05-07")
It can also handle time zones:
print(dateutil.parser.parse("2013-05-07"))
>>> datetime.datetime(2013, 5, 7, 1, 12, 12, tzinfo=tzutc())
If you have a datetime object, say:
import pytz
import datetime
now = datetime.datetime.now(pytz.UTC)
and you want chop off the time part, then I think it is easier to construct a new object instead of "substracting the time part". It is shorter and more bullet proof:
date_part datetime.datetime(now.year, now.month, now.day, tzinfo=now.tzinfo)
It also keeps the time zone information, it is easier to read and understand than a timedelta substraction, and you also have the option to give a different time zone in the same step (which makes sense, since you will have zero time part anyway).

For me, I needed to KEEP a timetime object because I was using UTC and it's a bit of a pain. So, this is what I ended up doing:
date = datetime.datetime.utcnow()
start_of_day = date - datetime.timedelta(
hours=date.hour,
minutes=date.minute,
seconds=date.second,
microseconds=date.microsecond
)
end_of_day = start_of_day + datetime.timedelta(
hours=23,
minutes=59,
seconds=59
)
Example output:
>>> date
datetime.datetime(2016, 10, 14, 17, 21, 5, 511600)
>>> start_of_day
datetime.datetime(2016, 10, 14, 0, 0)
>>> end_of_day
datetime.datetime(2016, 10, 14, 23, 59, 59)

If you specifically want a datetime and not a date but want the time zero'd out you could combine date with datetime.min.time()
Example:
datetime.datetime.combine(datetime.datetime.today().date(),
datetime.datetime.min.time())

You can use simply pd.to_datetime(then) and pandas will convert the date elements into ISO date format- [YYYY-MM-DD].
You can pass this as map/apply to use it in a dataframe/series too.

You can usee the following code:
week_start = str(datetime.today() - timedelta(days=datetime.today().weekday() % 7)).split(' ')[0]

Get timestamp for hour of current day

Ok, I need a way to get the timestamp for the current day but at a certain time.
So for example, I want the unix timestamp for today at 7:30PM - what would I do to get that value? In PHP it's possible with the strtotime() but I'm not sure how to do this in Python.
Edit: To clarify, I mean the current day not a statically written day. So if I ran this script tomorrow it would return the timestamp for 7:30PM tomorrow.

from datetime import datetime
now = datetime.utcnow() # Current time
then = datetime(1970,1,1) # 0 epoch time
ts = now - then
ts = ts.days * 24 * 3600 + ts.seconds
# alternatively, per Martijn Pieters
ts = int(ts.total_seconds())

you can use the time module :
from datetime import datetime
from time import mktime
# like said Ashoka
ts = datetime.strptime("2014-7-7 7:30","%Y-%m-%d %H:%M")
#you have now your datetime object
print mktime(ts.timetuple())
# print 1404711000.0
print int(mktime(ts.timetuple()))
# print 1404711000
be careful mktime don't care of time zone so if you want to have a UTC time zone and still use time , convert date before:
import pytz
fr = pytz.timezone('Europe/Paris')
#localize
ts = fr.localize(ts)
#timestamp in UTC
mktime(ts.astimezone(pytz.UTC).timetuple())

calendar.timegm method returns a timestamp out of passed time tuple:
import calendar
from datetime import datetime
d = datetime(year=2014, month=7, day=8, hour=7, minute=30)
calendar.timegm(d.utctimetuple())
# 1404804600
datetime.utcfromtimestamp(calendar.timegm(d.utctimetuple()))
# datetime.datetime(2014, 7, 8, 7, 30)
The important things are utctimetuple and utcfromtimestamp. You would certainly want a UTC timestamp, and not one in your local timezone.
import calendar
from datetime import datetime
from pytz import timezone, utc
tz = timezone('Europe/Warsaw')
aware = datetime(year=2014, month=7, day=8, hour=7, minute=30)
aware = tz.localize(aware)
# datetime.datetime(2014, 7, 8, 7, 30, tzinfo=<DstTzInfo 'Europe/Warsaw' CEST+2:00:00 DST>)
stamp = calendar.timegm(aware.utctimetuple())
# 1404797400
d = datetime.utcfromtimestamp(stamp)
# datetime.datetime(2014, 7, 8, 5, 30)
d = d.replace(tzinfo=utc)
d.astimezone(tz)
# datetime.datetime(2014, 7, 8, 7, 30, tzinfo=<DstTzInfo 'Europe/Warsaw' CEST+2:00:00 DST>)

How would I add the timezone to a datetime.datetime object?

I have a datetime.datetime object (datetime.datetime(2014, 4, 11, 18, 0)) and I would like to assign it a timezone using pytz. I know you can use pytz with a datetime.datetime.now() object (datetime.datetime.now(pytz.timezone('America/Los_Angeles'))) but how would I do it with a custom object?

Use the localize method:
import pytz
import datetime
la = pytz.timezone('America/Los_Angeles')
now = la.localize(datetime.datetime.now())
print(repr(now))
yields
datetime.datetime(2014, 4, 11, 21, 36, 2, 981916, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
localize is used to interpret timezone-unaware datetimes with respect to a timezone. The result is a timezone-aware datetime.
Note that some timezone-unaware datetimes, such as datetime(2002, 10, 27, 1, 30, 00), are ambiguous in certain timezones. Use the is_dst parameter to avoid the ambiguity.
astimezone is used to convert aware datetimes to other timezones.

Alternatively, you can assign timezone to os.environ['TZ'] directly.
import os
import datetime
print datetime.datetime.now()
os.environ['TZ'] = 'America/Los_Angeles'
print datetime.datetime.now()

How do you convert a naive datetime to DST-aware datetime in Python?

I'm currently working on the backend for a calendaring system that returns naive Python datetimes. The way the front end works is the user creates various calendar events, and the frontend returns the naive version of the event they created (for example, if the user selects October 5, 2020 from 3:00pm-4:00pm, the frontend returns datetime.datetime(2020, 10, 5, 15, 0, 0) as the start and datetime.datetime(2011, 10, 5, 16, 0, 0) as the end.
What I need to do is to take the naive datetime and convert it into UTC for storage in a database. Each user of the system has already specified their timezone preference, so the naive datetime is considered to be of the same timezone as their timezone preference. Obviously the datetimes need to be stored relative to UTC so that if users change their timezone, existing events will still render at the correct time that they scheduled them.
The frontend is outside my control, so I can't change the data that I'm receiving. The database design is also outside my control, so I can't change what data is being stored and how.
Here is the approximate approach I have taken so far:
import pytz
def convert_to_UTC(naive_datetime, user_tz_preference):
user_datetime = naive_datetime.replace(tzinfo=user_tz_preference)
utc_datetime = user_datetime.astimezone(pytz.utc)
The problem I ran into is related to Daylight Savings Time:
>>> from datetime import datetime
>>> import pytz
>>> user_tz_preference = pytz.timezone('US/Pacific')
>>> naive_datetime = datetime(2011, 10, 26, 12, 0, 0)
>>> user_datetime = naive_datetime.replace(tzinfo=user_tz_preference)
>>> user_datetime
datetime.datetime(2011, 10, 26, 12, 0, tzinfo=<DstTzInfo 'US/Pacific' PST-1 day, 16:00:00 STD>)
>>> received_utc = user_datetime.astimezone(pytz.utc)
>>> received_utc
datetime.datetime(2011, 10, 26, 20, 0, tzinfo=<UTC>)
>>> expected_utc = datetime(2011, 10, 26, 19, 0, tzinfo=pytz.utc)
>>> expected_utc == received_utc
False
Notice that using 'replace' sets the timezone to PST instead of PDT regardless of the date, which gives it a UTC offset of 8 hours instead of the expected 7 hours DST offset, so the time ends up being saved incorrectly.
What options do I have for converting the naive datetime to the correct PDT (or other timezone-relative DST) tzinfo?
(Also, please note that not all users live in a timezone that observes DST, or may live in a timezone that switches over at different times, so in order to do a solution like a timedelta correction before saving, I would need to know if the timezone supports DST, and on which dates it switches over).

Pytz's localize function can do this: http://pytz.sourceforge.net/#localized-times-and-date-arithmetic
from datetime import datetime
import pytz
tz = pytz.timezone('US/Pacific')
naive_dt = datetime(2020, 10, 5, 15, 0, 0)
utc_dt = tz.localize(naive_dt, is_dst=None).astimezone(pytz.utc)
# -> 2020-10-05 22:00:00+00:00

With zoneinfo from Python 3.9's standard lib:
from datetime import datetime
from zoneinfo import ZoneInfo
naive_datetime = datetime(2011, 10, 26, 12, 0, 0)
user_tz_preference = ZoneInfo("America/Los_Angeles") # former US/Pacific
# it is safe to replace the tzinfo:
user_datetime = naive_datetime.replace(tzinfo=user_tz_preference)
# ...or set it directly:
user_datetime = datetime(2011, 10, 26, 12, tzinfo=ZoneInfo("America/Los_Angeles"))
# astimezone works as before:
utc_datetime = user_datetime.astimezone(ZoneInfo("UTC"))
print(repr(user_datetime))
# datetime.datetime(2011, 10, 26, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='US/Pacific'))
print(user_datetime.isoformat())
# 2011-10-26T12:00:00-07:00
print(utc_datetime.isoformat())
# 2011-10-26T19:00:00+00:00
docs