We Keep Coding

Python DateTime does not match format when use the function inside celery task

I create a function to get the different between two date time and its working fine, But when i use it in celery it returns the following error:
ValueError: time data '2023-02-16T14:38:33.301574' does not match format '%Y-%m-%d %H:%M:%S.%f'
Any advice please

I recommend not doing this. Python standard library gives you everything you need to deal with ISO 8601 formatted date/time data. I advise you to use datetime.datetime.fromisoformat and datetime.datetime.isoformat instead.

Trying to convert influxdb timestamp in a csv to date and time columns

I am migrating an InfluxDB database to mySQL. I have managed to export the influx data to a CSV file, which is great, but now I am stuck with the date and time field which has been given to me.
I have no idea what format it is in, after doing some research it tells me that it is in epoch time, but using python to try and convert the timestamp to an ISO format, it isn't recognised as a valid timestamp. Any idea how to get this converted. Ideally to separate date and time columns. The data that I have got is something like this :
time,absoluteHumidity
1578152602608558363,5.788981747966442
1578152608059500073,4.769760557208695
1578152613662193439,5.788981747966442
And the python that I was using to try and convert it, was this :
from datetime import datetime, timezone
print (datetime.fromtimestamp(1578152602608558363, timezone.utc))
Any help or suggestions would be appreciated !

According to the influxdb docs they store timestamp values with nanoseconds precision.
However the datetime.fromtimestamp method expects a floating point number and its integer part is in second precision.
So generally your approach is right you just need to divide the influx timestamp by 1e9 and it should just work:
from datetime import datetime, timezone
print(datetime.fromtimestamp(1578152602608558363 / 1e9, timezone.utc))

How do I convert a xsd:duration in date format in python?

I have been given a data set that has two rows with dates in the format xsd:duration which python accounts for as a string.
The format looks like PT3H20M (for 3h20min), or PT3H (for 3h) or PT30M (for 30m). How do you convert this format to date so that I can add the times and perform comparisons on them ?
Thanks for any help
EDIT : I'm specifically looking for any built-in package/function that I don't know about that would do that relatively easily.

I would suggest that:
You extract the numbers of hours and minutes using regular
expression in python.
Use a datetime function to convert to UNIX time or another format.
Make a comparison or sum.

sqlalchemy: Call STR_TO_DATE on column

I am moving some of my code onto sqlalchemy from using raw MySQL queries.
The current issue I am having is that the datetime was saved in a string format by a C# tool. Unfortunately, the representation does not match up with Python's (as well as that it has an extra set of single quotes), thus making filtering somewhat cumbersome.
Here is an example of the format that the date was saved in:
'2016-07-01T17:27:01'
Which I was able to convert to a usable datetime using the following MySQL command:
STR_TO_DATE(T.PredicationGeneratedTime, \"'%%Y-%%m-%%dT%%H:%%i:%%s'\")
However, I cannot find any documentation that describes how to invoke built-in functions such as STR_TO_DATE when filtering with sqlalchemy
The following Python code:
session.query(Train.Model).filter(cast(Train.Model.PredicationGeneratedTime, date) < start)
is giving me:
TypeError: Required argument 'year' (pos 1) not found
There does not seem to be a way to specify the format for the conversion.
Note: I realize the solution is to fix the way the datetime is stored, but in the mean time I'd like to run queries against the existing data.

You can try to use func.str_to_date(COLUMN, FORMAT_STRING) instead of cast

In the cast() you should be using sqlalchemy.DateTime, not (what I assume is) a datetime.date - that is the cause of the exception.
However, fixing that will not really help because of the embedded single quotes.
You are fortunate that the dates stored in your table are in ISO format. That means that lexicographic comparisons will work on the date strings themselves, without casting. As long as you use a string for start with the surrounding single quotes, it will work.
from datetime import datetime
start = "'{}'".format(datetime.now().isoformat())
session.query(Train.Model).filter(Train.Model.PredicationGeneratedTime < start)

MongoDB date and removed objects

Yesterday I had some strange experience with MongoDB. I am using twisted and txmongo - an asynchronous driver for mongodb (similar to pymongo).
I have a rest service where it receives some data and put it to mongodb. One field is timestamp in milliseconds, 13 digits.
First of all ther is no trivial way to convert millisecond timestamp into python datetime in python. I ended up with something like this:
def date2ts(ts):
return int((time.mktime(ts.timetuple()) * 1000) + (ts.microsecond / 1000))
def ts2date(ts):
return datetime.datetime.fromtimestamp(ts / 1000) + datetime.timedelta(microseconds=(ts % 1000))
The problem is that when I save the data to mongodb, retreive datetime back and convert it back to timestamp using my function I don't get the same result in milliseconds.
I did not understand why is it happening. Datetime is saved in mongodb as ISODate object. I tried to query it from shell and there is indeed difference in one second or few millisoconds.
QUESTION 1: Does anybody know why is this happening?
But this is not over. I decided not to use datetime and to save timestamp directly as long. Before that I removed all the data from collection. I was quite surprised that when I tried to save same field not as date but as long, it was represented as ISODate in shell. And when retrieved there was still difference in few milliseconds.
I tried to drop the collection and index. When it did not help I tried to drop entire database. When it did not help I tried to drop entire database and to restart mongod. And after this I guess it started to save it as Long.
QUESTION 2: Does anybody know why is this happening?
Thank you!

Python's timestamp is calculated in seconds since the Unix epoch of Jan 1, 1970. The timestamp in JavaScript (and in turn MongoDB), on the other hand, is in terms of milliseconds.
That said, if you have only have the timestamps on hand, you can multiple the Python value by 1000 to get milliseconds and store that value into MongoDB. Likewise, you can take the value from MongoDB and divide it by 1000 to make it a Python timestamp. Keep in mind that Python only seems to care for two significant digits after the decimal point instead of three (as it doesn't typically care for milliseconds) so keep that in mind if you are still having differences of < 10 milliseconds.
Normally I would suggest working with tuples instead, but the conventions for the value ranges are different for each language (JavaScript is unintuitive in that it starts days of the month at 0 instead of 1) and may cause issues down the road.

It can be the case of different timezone's. Please use the below mentioned function to rectify it.
function time_format(d, offset) {
// Set timezone
utc = d.getTime() + (d.getTimezoneOffset() * 60000);
nd = new Date(utc + (3600000*offset));
return nd;
}
searchdate = time_format(searchdate, '+5.5');
'+5.5' here is the timezone difference from the local time to GMT time.