We Keep Coding

strptime() error - Time from sensor with more than 24 hours (e.g. 24:01:53)

I am using datetime.strptime() to convert a string containing time and date from a sensor into a datetime object.
The code sometimes fails. Minimal example:
datetime.strptime('1/9/2021 24:01:53', '%d/%m/%Y %H:%M:%S')
Output error:
ValueError: time data '1/9/2021 24:01:53' does not match format '%d/%m/%Y %H:%M:%S'
I am guessing this has to do with the fact that the time is more than 23:59:59 - which seems to me a non-realistic time (I would think that 1/9/2021 24:01:53 could potentially be 2/9/2021 00:01:53 - a time format which I have never seen).
Is this a non-standard way of representing time or possibly a hardware/software issue with the sensor acquisition system? If it is a different way of representing time, how can I convert it to a standard datetime object?
Kind regards,
D.F.

If the hour exceeds 23 in a variable representing time, a good option is to create a timedelta from it, which you can then add to a datetime object. For given example that might look like
from datetime import datetime, timedelta
def custom_todatetime(s):
"""
split date/time string formatted as 'DD/MM/YYYY hh:mm:ss' into date and time parts.
parse date part to datetime and add time part as timedelta.
"""
parts = s.split(' ')
seconds = sum(int(x) * 60 ** i for i, x in enumerate(reversed(parts[1].split(':'))))
return datetime.strptime(parts[0], "%d/%m/%Y") + timedelta(seconds=seconds)
s = '1/9/2021 24:01:53'
print(custom_todatetime(s))
# 2021-09-02 00:01:53
Note: conversion of hh:mm:ss to seconds taken from here - give a +1 there if helpful.

How to convert milliseconds to date and time format?

I have a value in milliseconds in a Python program. For example: 1557975599999
And I would like to convert it to a string with days and hours, minutes, seconds. How can I do this?

To convert unix timestamp to datetime, you can use datetime.fromtimestamp(). The only problem, that your timestamp is in miliseconds, but function expect timestamp in seconds. To cut miliseconds you can divide timestamp to 1000.
Code:
from datetime import datetime
a = 1557975599999
date = datetime.fromtimestamp(a // 1000)
print(date)
Output:
2019-05-16 05:59:59
Upd.
#Daniel in comments noticed that fromtimestamp() accept floats, so we can save miliseconds from original timestamp. All we need is just to remove one symbol :D
date = datetime.fromtimestamp(a / 1000)

With Pandas’ to_datetime()
import pandas as pd
pd.to_datetime(a, unit='ms')
# Or with a dataframe(column):
df['date'] = pd.to_datetime(df['Millisecond_time'], unit='ms')

Comparison between datetime and datetime64[ns] in pandas

I'm writing a program that checks an excel file and if today's date is in the excel file's date column, I parse it
I'm using:
cur_date = datetime.today()
for today's date. I'm checking if today is in the column with:
bool_val = cur_date in df['date'] #evaluates to false
I do know for a fact that today's date is in the file in question. The dtype of the series is datetime64[ns]
Also, I am only checking the date itself and not the timestamp afterwards, if that matters. I'm doing this to make the timestamp 00:00:00:
cur_date = datetime.strptime(cur_date.strftime('%Y_%m_%d'), '%Y_%m_%d')
And the type of that object after printing is datetime as well

For anyone who also stumbled across this when comparing a dataframe date to a variable date, and this did not exactly answer your question; you can use the code below.
Instead of:
self.df["date"] = pd.to_datetime(self.df["date"])
You can import datetime and then add .dt.date to the end like:
self.df["date"] = pd.to_datetime(self.df["date"]).dt.date

You can use
pd.Timestamp('today')
or
pd.to_datetime('today')
But both of those give the date and time for 'now'.
Try this instead:
pd.Timestamp('today').floor('D')
or
pd.to_datetime('today').floor('D')
You could have also passed the datetime object to pandas.to_datetime but I like the other option mroe.
pd.to_datetime(datetime.datetime.today()).floor('D')
Pandas also has a Timedelta object
pd.Timestamp('now').floor('D') + pd.Timedelta(-3, unit='D')
Or you can use the offsets module
pd.Timestamp('now').floor('D') + pd.offsets.Day(-3)
To check for membership, try one of these
cur_date in df['date'].tolist()
Or
df['date'].eq(cur_date).any()

When converting datetime64 type using pd.Timestamp() it is important to note that you should compare it to another timestamp type. (not a datetime.date type)
Convert a date to numpy.datetime64
date = '2022-11-20 00:00:00'
date64 = np.datetime64(date)
Seven days ago - timestamp type
sevenDaysAgoTs = (pd.to_datetime('today')-timedelta(days=7))
convert date64 to Timestamp and see if it was in the last 7 days
print(pd.Timestamp(pd.to_datetime(date64)) >= sevenDaysAgoTs)

Want to find difference, in days, between two dates, of different date format, in Python

I have two different dates that I am pulling from a database using a SQL query. Im looking to do transformations in Python, but the two main dates I want to work with are stored in different formats. The first date is of the date format (YYYY/MM/DD) the other is of (YYYY/MM/DD HH:MM:SS) format. I want a difference in days so the DATETIME is irrelevant on the second date. I was wondering what is the easiest way to do this in python? Ideally, I would like to automate this, where I create a DATE format of the DATETIME variable, and take the difference between the two DATES.
I've tried the following but I am also getting errors since I am dealing with Series. I am trying to get the delta for every row.
df.delta = (df.DATETIME - df.DATE)
and
df.delta = datetime.timedelta(df.DATETIME - df.DATE)

import datetime
d1 = datetime.datetime.strptime('2018/01/13', '%Y/%m/%d')
d2 = datetime.datetime.strptime('2018/01/15 18:34:02', '%Y/%m/%d %H:%M:%S')
delta = d2 - d1
print delta.total_seconds()
print delta.days

Convert your datetime object to a date object, you are then able to subtract them for a delta value.
df.delta = (df.DATETIME.date() - df.DATE)

am I using EPOCH time? python

As I need to transform the time format hh:mm to the count of minutes. Doing the maths would be:
hh*60+mm
I came across with the method mktime, which I thought it was to get the EPOCH time in minutes of a given datetimeobject. In this way I convert from datetimeto integer.
I wrote this in order to calculate the time of the day in minutes:
(the outputs in the boxes below)
a = datetime.now()
print type(a)
print a
b= '2017-07-05 00:00:00.0'
b = datetime.strptime(b, "%Y-%m-%d %H:%M:%S.%f")
type(b)
out: type 'datetime.datetime'
out: type 'datetime.datetime'
As both objects are datetime I used the method mktime that I found searching for a way to transform from timestamp to integer
timestamp = int(time.mktime(a.timetuple()))-int(time.mktime(b.timetuple()))
type(timestamp)
print timestamp
print a
print b
And the output is:
out: 62189
out: 2017-07-05 17:16:29.134026
out: 2017-07-05 00:00:00
So the value 62189is seconds? Am I using EPOCH time?
(sorry if the question is bit fool, I am quite beginner, that's why I need help)
Thanks!

I would come with a solution like:
import time
t = time.strptime("02:24", "%H:%M") # Replace "02:24" with your string parameter
nb_minutes = t.tm_hour * 60 + t.tm_min
This transforms the time format "hh:mm" to the count of minutes as requested in your OP.
Note that there is a distrinction between a date, a datetime and a time. Python offers more flexibility than Javascript Date() on this.
Basically:
date = YYYY-MM-DD
time = HH:MM:ss
datetime = YYYY-MM-DD HH:MM:ss
Depending of your calculations, using the correct object in the python library will save you efforts all while making your source code more semantically readable.