We Keep Coding

Python Datetime strptime always to precision of millisecond

I am trying to convert strings e.g. "2010-01-01 10:09:01" into datetime to the precision of milliseconds. However even after adding 0 milliseconds at the back of the string e.g. "2010-01-01 10:09:01.000", the datetime always truncates the milliseconds part off. How do I make sure the datetime is always to the precision of millisecond even if there are 0 milliseconds in the timestamp? Thanks! :)
truncating the millisecond off

If I understood you correctly, you can use datetime.microseconds():
from datetime import datetime
a = datetime.strptime("2010-01-01 10:09:01.020", "%Y-%m-%d %H:%M:%S.%f")
print(a.microsecond) # >> 20000
print(a.microsecond//1000) # >> 20
And you can add if blocks or something that you need.
This is inside of value "a":

strptime() error - Time from sensor with more than 24 hours (e.g. 24:01:53)

I am using datetime.strptime() to convert a string containing time and date from a sensor into a datetime object.
The code sometimes fails. Minimal example:
datetime.strptime('1/9/2021 24:01:53', '%d/%m/%Y %H:%M:%S')
Output error:
ValueError: time data '1/9/2021 24:01:53' does not match format '%d/%m/%Y %H:%M:%S'
I am guessing this has to do with the fact that the time is more than 23:59:59 - which seems to me a non-realistic time (I would think that 1/9/2021 24:01:53 could potentially be 2/9/2021 00:01:53 - a time format which I have never seen).
Is this a non-standard way of representing time or possibly a hardware/software issue with the sensor acquisition system? If it is a different way of representing time, how can I convert it to a standard datetime object?
Kind regards,
D.F.

If the hour exceeds 23 in a variable representing time, a good option is to create a timedelta from it, which you can then add to a datetime object. For given example that might look like
from datetime import datetime, timedelta
def custom_todatetime(s):
"""
split date/time string formatted as 'DD/MM/YYYY hh:mm:ss' into date and time parts.
parse date part to datetime and add time part as timedelta.
"""
parts = s.split(' ')
seconds = sum(int(x) * 60 ** i for i, x in enumerate(reversed(parts[1].split(':'))))
return datetime.strptime(parts[0], "%d/%m/%Y") + timedelta(seconds=seconds)
s = '1/9/2021 24:01:53'
print(custom_todatetime(s))
# 2021-09-02 00:01:53
Note: conversion of hh:mm:ss to seconds taken from here - give a +1 there if helpful.

Finding the format of my timestamp in Python

My time format is screwy, but it seemed workable, as a string with the following format:
'47:37:00'
I tried to set a variable where:
DT = '%H:%M:%S'
So I could find the difference between two times, but it's given me the following error:
ValueError: time data '47:37:00' does not match format '%H:%M:%S'
Is it possible there are more elements to my time stamps than I thought? Or that it's formatted in minutes/seconds/milliseconds? I can't seem to find documentation that would help me determine my time format so I could set DT and do arithmetic on it.

It's because you set 47 to %H, that is not a proper value.
Here is an example:
import datetime
dt = datetime.datetime.strptime('2016/07/28 12:37:00','%Y/%m/%d %H:%M:%S')
print dt
Output: 2016-07-28 12:37:00

You wrote "I can't seem to find documentation that would help me determine my time format so I could set DT and do arithmetic on it"
Try this: https://docs.python.org/3/library/datetime.html
Way down to the bottom.
And yes, when the %H is matched with 47, you get boom the error.

Python time library: how do I preserve dst with strptime and strftime

I need to store a timestamp in a readable format, and then later on I need to convert it to epoch for comparison purposes.
I tried doing this:
import time
format = '%Y %m %d %H:%M:%S +0000'
timestamp1 = time.strftime(format,time.gmtime()) # '2016 03 25 04:06:22 +0000'
t1 = time.strptime(timestamp1, format) # time.struct_time(..., tm_isdst=-1)
time.sleep(1)
epoch_now = time.mktime(time.gmtime())
epoch_t1 = time.mktime(t1)
print "Delta: %s" % (epoch_now - epoch_t1)
Running this, instead of getting Delta of 1 sec, I get 3601 (1 hr 1 sec), CONSISTENTLY.
Investigating further, it seems that when I just do time.gmtime(), the struct has tm_isdst=0, whereas the converted struct t1 from timestamp1 string has tm_isdst=-1.
How can I ensure the isdst is preserved to 0. I think that's probably the issue here.
Or is there a better way to record time in human readable format (UTC), and yet be able to convert back to epoch properly for time diff calculation?
UPDATES:
After doing more research last night, I switched to using datetime because it preserves more information in the datetime object, and this is confirmed by albertoql answer below.
Here's what I have now:
from datetime import datetime
format = '%Y-%m-%d %H:%M:%S.%f +0000' # +0000 is optional; only for user to see it's UTC
d1 = datetime.utcnow()
timestamp1 = d1.strftime(format)
d1a = datetime.strptime(timestamp1, format)
time.sleep(1)
d2 = datetime.utcnow()
print "Delta: %s" % (d2 - d1a).seconds
I chose not to add tz to keep it simple/shorter; I can still strptime that way.

Below, first an explanation about the problem, then two possible solutions, one using time, another using datetime.
Problem explanation
The problem is on the observation that the OP made in the question: tm_isdst=-1. tm_isdst is a flag that determines whether daylight savings time is in effect or not (see for more details https://docs.python.org/2/library/time.html#time.struct_time).
Specifically, given the format of the string for the time from the OP (that complies with RFC 2822 Internet email standard), [time.strptime]4 does not store the information about the timezone, namely +0000. Thus, when the struct_time is created again according to the information in the string, tm_isdst=-1, namely unknown. The guess on how to fill in that information when making the calculation is based on the local system. For example, as if the system refers to North America, where daylight savings time is in effect, tm_isdst is set.
Solution with time
If you want to use only time package, then, the easiest way to parse directly the information is to specify that the time is in UTC, and thus adding %Z to the format. Note that time does not provide a way to store the information about the timezone in struct_time. As a result, it does not print the actual time zone associated with the time saved in the variable. The time zone is retrieved from the system. Therefore, it is not possible to directly use the same format for time.strftime. The part of the code for writing and reading the string would look like:
format = '%Y %m %d %H:%M:%S UTC'
format2 = '%Y %m %d %H:%M:%S %Z'
timestamp1 = time.strftime(format, time.gmtime())
t1 = time.strptime(timestamp1, format2)
Solution with datetime
Another solution involves the use datetime and dateutil packages, which directly support timezone, and the code could be (assuming that preserving the timezone information is a requirement):
from datetime import datetime
from dateutil import tz, parser
import time
time_format = '%Y %m %d %H:%M:%S %z'
utc_zone = tz.gettz('UTC')
utc_time1 = datetime.utcnow()
utc_time1 = utc_time1.replace(tzinfo=utc_zone)
utc_time1_string = utc_time1.strftime(time_format)
utc_time1 = parser.parse(utc_time1_string)
time.sleep(1)
utc_time2 = datetime.utcnow()
utc_time2 = utc_time2.replace(tzinfo=utc_zone)
print "Delta: %s" % (utc_time2 - utc_time1).total_seconds()
There are some aspects to pay attention to:
After the call of utcnow, the timezone is not set, as it is a naive UTC datetime. If the information about UTC is not needed, it is possible to delete both lines where the timezone is set for the two times, and the result would be the same, as there is no guess about DST.
It is not possible to use datetime.strptime because of %z, which is not correctly parsed. If the string contains the information about the timezone, then parser should be used.
It is possible to directly perform the difference from two instances of datetime and transform the resulting delta into seconds.
If it is necessary to get the time in seconds since the epoch, an explicit computation should be made, as there is no direct function that does that automatically in datetime (at the time of the answer). Below the code, for example for utc_time2:
epoch_time = datetime(1970,1,1)
epoch2 = (utc_time2 - epoch_time).total_seconds()
datetime.resolution, namely the smallest possible difference between two non-equal datetime objects. This results in a difference that is up to the resolution.

how to calculate timedelta python

What I am trying to do is to subtract 7 hours from a date. I searched stack overflow and found the answer on how to do it here. I then went to go read the documentation on timedelta because I was unable to understand what that line in the accepted answer does, rewritten here for ease:
from datetime import datetime
dt = datetime.strptime( date, '%Y-%m-%d %H:%M' )
dt_plus_25 = dt + datetime.timedelta( 0, 2*60*60 + 30*60 )
Unfortunately, even after reading the documentation I still do not understand how that line works.
What is the timedelta line doing? How does it work?
Additionally, before I found this stackoverflow post, I was working with time.struct_time tuples. I had a variable tm:
tm = time.strptime(...)
I was simply accessing the hour through tm.tm_hour and subtracting seven from it but this, for obvious reasons, does not work. This is why I am now trying to use datetime. tm now has the value
tm = datetime.strptime(...)
I'm assuming using datetime is the best way to subtract seven hours?
Note: subtracting seven hours because I want to go from UTC to US/Pacific timezone. Is there a built-in way to do this?

What is the timedelta line doing? How does it work?
It creates a timedelta object.
There are two meanings of "time".
"Point in Time" (i.e, date or datetime)
"Duration" or interval or "time delta"
A time delta is an interval, a duration, a span of time. You provided 3 values.
0 days.
2*60*60 + 30*60 seconds.

timedelta() generates an object representing an amount of time—the Greek letter delta is used in math to represent "difference". So to compute an addition or a subtraction of an amount of time, you take the starting time and add the change, or delta, that you want.
The specific call you've quoted is for generating the timedelta for 2.5 hours. The first parameter is days, and the second is seconds, so you have (0 days, 2.5 hours), and 2.5 hours in seconds is (2 hours * 60 minutes/hour * 60 seconds/minute) + (30 minutes * 60 seconds / minute).
For your case, you have a negative time delta of 0 days, 7 hours, so you'd write:
timedelta(0, -7 * 60 * 60)
... or timedelta(0, -7 * 3600) or whatever makes it clear to you what you're doing.

Note: subtracting seven hours because I want to go from UTC to US/Pacific timezone. Is there a built-in way to do this?
Yes there is: datetime has built-in timezone conversion capabilities. If you get your datetime object using something like this:
tm = datetime.strptime(date_string, '%Y-%m-%d %H:%M')
it will not have any particular timezone "attached" to it at first, but you can give it a timezone using
tm_utc = tm.replace(tzinfo=pytz.UTC)
Then you can convert it to US/Pacific with
tm_pacific = tm_utc.astimezone(pytz.all_timezones('US/Pacific'))
I'd suggest doing this instead of subtracting seven hours manually because it makes it clear that you're keeping the actual time the same, just converting it to a different timezone, whereas if you manually subtracted seven hours, it looks more like you're actually trying to get a time seven hours in the past. Besides, the timezone conversion properly handles oddities like daylight savings time.
To do this you will need to install the pytz package, which is not included in the Python standard library.