how to convert time in which hr value is greater than 24? - python

this how my dataset looks like ;-
Timestamp
2020-11-1 0:1:28:11265
2020-11-1 0:2:3:38616
2020-11-1 0:3:30:31943
2020-11-1 0:4:25:2289
2020-11-1 0:5:4:45378
I try to convert above timestamp using below functions
import datetime
def convert(date_time):
format = '%Y-%m-%d %H:%M:%S:%f' # The format
datetime_obj = datetime.datetime.strptime(date_time, format)
datetime_str = datetime_obj.strftime("%Y-%m-%d %H:%M:%S")
return datetime.datetime.strptime(datetime_str, "%Y-%m-%d %H:%M:%S")
df['Timestamp'] = df['Timestamp'].apply(convert)
df.head()
after running above code i get below error.
ValueError: time data '2020-11-1 24:0:47:40476' does not match format '%Y-%m-%d %H:%M:%S:%f'
how do i convert the date '2020-11-1 24:0:47:40476' --> '2020-11-2 0:0:47:40476'

You can parse the date and time separately, parsing the date part into just a date with the time component set to 0:00:00, then add the time part as timedelta to it:
>>> from datetime import datetime, timedelta
>>> s = '2020-11-1 24:0:47:40476'
>>> d, t = s.split()
>>> d
'2020-11-1'
>>> t
'24:0:47:40476'
>>> ts = datetime.strptime(d, '%Y-%m-%d')
>>> ts
datetime.datetime(2020, 11, 1, 0, 0)
>>> h, m, s, ms = t.split(':')
>>> ts + timedelta(hours=int(h), minutes=int(m), seconds=int(s), milliseconds=int(ms))
datetime.datetime(2020, 11, 2, 0, 1, 27, 476000)
Note that this may or may not work as desired should DST transitions happen right during that time; it's a bit unclear how that's supposed to work.
Also note that the 40476 milliseconds (?) added up to additional minutes. It’s slightly unclear what exactly that number is supposed to represent, you may have to split that up into milli- and microseconds too before passing it to timedelta.

Related

How to convert datetime from decimal to “%y-%m-%d %H:%M:%S” given the time origin?

I did my search around but I couldn't find an answer that satisfies my problem.
I am using python 3.7 and I need to convert a series of decimal numbers into datetime object in the form %Y-%m-%d %H:%M:%S. While doing so I need to consider an origin point, for example:
t = 0.000000 equals to 2016-06-25 00:00:00
t = 0.010417 equals to ..... ?
and so on. I know that in my decimal time the integer part is day since start, decimal part is fraction of day.
I have found an answer using R here. I also think that I might need to use the class method date.fromordinal(ordinal)or something similar but I cannot figure it out how to do it.
This is what I have tried so far:
example t = 1.010416
import datetime as DT
from datetime import timedelta
day = int(x)
datetime_object = DT.datetime.strptime("2016-06-25 00:00:00", '%Y-%m-%d %H:%M:%S')
python_datetime = DT.datetime.fromordinal(day) + timedelta(days=datetime_object.day-1)
I get:
datetime.datetime(1, 1, 25, 0, 0)
But I cannot add the year 2016 nor the month. Also, for every case in which int(t) = 0, I get:
ValueError: ordinal must be >= 1
Thank you very much for your answers
Just to leave a clear answer here, taking into account my comments on the other answers:
from datetime import datetime,timedelta
base_date = datetime(2016, 6, 25)
deltas = (2.34857, 0.010417, 1.010416)
for delta in deltas:
print((base_date + timedelta(delta)).strftime('%Y-%m-%d %H:%M:%S'))
That code yields the following ouput:
>>> from datetime import datetime,timedelta
>>>
>>> base_date = datetime(2016, 6, 25)
>>> deltas = (2.34857, 0.010417, 1.010416)
>>>
>>> for delta in deltas:
... print((base_date + timedelta(delta)).strftime('%Y-%m-%d %H:%M:%S'))
...
2016-06-27 08:21:56
2016-06-25 00:15:00
2016-06-26 00:14:59
>>>
timedelta stores its data in this format: (DAYS, SECONDS) so you can calculate it easily:
import datetime
t = 2.34857
# Full days
days = int(t)
# Part of a day
part_of_day = t - int(t)
seconds = int(part_of_day * 24 * 60 * 60)
# Calculate the time delta
dt = datetime.timedelta(
days=days,
seconds=seconds
)
# Add t-delta to the first day
first_day = datetime.datetime(2016, 6, 25)
current_time = first_day + dt
current_time
will return:
datetime.datetime(2016, 6, 27, 8, 21, 56)
Then you can convert it to a string with this function:
datetime.datetime.strftime(current_time, '%Y-%m-%d %H:%M:%S')
'2016-06-27 08:21:56'
Edit 1: Instead of constructing the timedelta by days-seconds, one can use just float days as parameter (thanks to accdias!):
dt = datetime.timedelta(days=t)

python: adding any number of days to date

I am hoping someone can point me in the right direction in working with dates and timedelta.
my understanding is that to add any number (ex: 10 days) to a date, you need to convert it to a timedelta.
if that is correct, how can I add any number to a date when it is an integer?
any documentation or links would be great - thank you.
code example, my date is as follows:
x = 20100103 (formatted as YYYYMMDD)
>>> import datetime
>>> today = datetime.datetime.now().date()
>>> today
datetime.date(2016, 6, 14)
>>> today + datetime.timedelta(days=10)
datetime.date(2016, 6, 24)
There's no need to convert it to a timedelta. Just use the timedelta function, if you want to add days, use days=N, for hours, timedelta(hours=20)
x=20100103
x2 = int((datetime.datetime.strptime(str(x),"%Y%m%d") + datetime.timedelta(days=10)).strftime("%Y%m%d"))
to break it down
x=20100103
x_as_datetime = datetime.datetime.strptime(str(x),"%Y%m%d")
new_datetime = x_as_datetime + datetime.timedelta(days=10) #add your timedelta
x2 = new_datetime.strftime("%Y%m%d") # format it back how you want
int(x2) # if you just want an integer ...
from datetime import datetime
from datetime import timedelta
StartDate = "20100103"
Date = datetime.strptime(StartDate, "%Y%m%d")
EndDate = Date + timedelta(days=10)

convert time in Python

I am parsing an xml file in python and I need to convert the following date and time format to something human-friendly:
<due_date>735444</due_date>
<due_time>55800</due_time>
The format of date seems to be number of days since year 0, and time is the number of seconds since midnight.
How could I convert it into some standard format, such as 2014-07-30 15:30:00 ?
Use datetime.datetime.fromordinal() plus a timedelta() for the seconds after midnight:
import datetime
dt = datetime.datetime.fromordinal(due_date) + datetime.timedelta(seconds=due_time)
Your due_date is relative to the year 1 instead, as your values fit to produce your expected date:
>>> import datetime
>>> due_date = 735444
>>> due_time = 55800
>>> datetime.datetime.fromordinal(due_date) + datetime.timedelta(seconds=due_time)
datetime.datetime(2014, 7, 30, 15, 30)

Convert strange date format to standard date format

In Sweden we sometimes use a strange date format, for example New Year is the 31/12. If I have this format as a string ,which can be any date between 1/1 and 31/12, and if we assume that it is this year, how do I get into a standard date format using Python (format as 2012-01-01 and 2012-12-31) that can be sore in be stored as a date in a mySQL database.
Simply split the two values, map them to integers and update a datetime.date() instance:
import datetime
day, month = map(int, yourvalue.split('/'))
adate = datetime.date.today().replace(month=month, day=day)
By using datetime.date.today() we get the current year.
Demo:
>>> import datetime
>>> somevalue = '31/12'
>>> day, month = map(int, somevalue.split('/'))
>>> datetime.date.today().replace(month=month, day=day)
datetime.date(2012, 12, 31)
>>> someothervalue = '1/1'
>>> day, month = map(int, someothervalue.split('/'))
>>> datetime.date.today().replace(month=month, day=day)
datetime.date(2012, 1, 1)
Alternatively, you could use the datetime.strptime() method to parse these dates, but you'll have to manually correct the year afterwards (it'll use 1900 as the default if no year is being parsed):
adate = datetime.datetime.strptime(yourvalue, '%d/%m').date()
adate = adate.replace(year=datetime.date.today().year)
There is nothing strange about this format :)
You can use the datetime module:
import datetime
d = datetime.datetime.strptime('31/12', '%d/%m').date().replace(year=2012)
print d
>> datetime.date(2012, 12, 31)

Convert string date to timestamp in Python

How to convert a string in the format "%d/%m/%Y" to timestamp?
"01/12/2011" -> 1322697600
>>> import time
>>> import datetime
>>> s = "01/12/2011"
>>> time.mktime(datetime.datetime.strptime(s, "%d/%m/%Y").timetuple())
1322697600.0
I use ciso8601, which is 62x faster than datetime's strptime.
t = "01/12/2011"
ts = ciso8601.parse_datetime(t)
# to get time in seconds:
time.mktime(ts.timetuple())
You can learn more here.
>>> int(datetime.datetime.strptime('01/12/2011', '%d/%m/%Y').strftime("%s"))
1322683200
To convert the string into a date object:
from datetime import date, datetime
date_string = "01/12/2011"
date_object = date(*map(int, reversed(date_string.split("/"))))
assert date_object == datetime.strptime(date_string, "%d/%m/%Y").date()
The way to convert the date object into POSIX timestamp depends on timezone. From Converting datetime.date to UTC timestamp in Python:
date object represents midnight in UTC
import calendar
timestamp1 = calendar.timegm(utc_date.timetuple())
timestamp2 = (utc_date.toordinal() - date(1970, 1, 1).toordinal()) * 24*60*60
assert timestamp1 == timestamp2
date object represents midnight in local time
import time
timestamp3 = time.mktime(local_date.timetuple())
assert timestamp3 != timestamp1 or (time.gmtime() == time.localtime())
The timestamps are different unless midnight in UTC and in local time is the same time instance.
Simply use datetime.datetime.strptime:
import datetime
stime = "01/12/2011"
print(datetime.datetime.strptime(stime, "%d/%m/%Y").timestamp())
Result:
1322697600
To use UTC instead of the local timezone use .replace:
datetime.datetime.strptime(stime, "%d/%m/%Y").replace(tzinfo=datetime.timezone.utc).timestamp()
The answer depends also on your input date timezone. If your date is a local date, then you can use mktime() like katrielalex said - only I don't see why he used datetime instead of this shorter version:
>>> time.mktime(time.strptime('01/12/2011', "%d/%m/%Y"))
1322694000.0
But observe that my result is different than his, as I am probably in a different TZ (and the result is timezone-free UNIX timestamp)
Now if the input date is already in UTC, than I believe the right solution is:
>>> calendar.timegm(time.strptime('01/12/2011', '%d/%m/%Y'))
1322697600
I would give a answer for beginners (like me):
You have the date string "01/12/2011". Then it can be written by the format "%d/%m/%Y". If you want to format to another format like "July 9, 2015", here a good cheatsheet.
Import the datetime library.
Use the datetime.datetime class to handle date and time combinations.
Use the strptime method to convert a string datetime to a object datetime.
Finally, use the timestamp method to get the Unix epoch time as a float. So,
import datetime
print( int( datetime.datetime.strptime( "01/12/2011","%d/%m/%Y" ).timestamp() ) )
# prints 1322712000
A lot of these answers don't bother to consider that the date is naive to begin with
To be correct, you need to make the naive date a timezone aware datetime first
import datetime
import pytz
# naive datetime
d = datetime.datetime.strptime('01/12/2011', '%d/%m/%Y')
>>> datetime.datetime(2011, 12, 1, 0, 0)
# add proper timezone
pst = pytz.timezone('America/Los_Angeles')
d = pst.localize(d)
>>> datetime.datetime(2011, 12, 1, 0, 0,
tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
# convert to UTC timezone
utc = pytz.UTC
d = d.astimezone(utc)
>>> datetime.datetime(2011, 12, 1, 8, 0, tzinfo=<UTC>)
# epoch is the beginning of time in the UTC timestamp world
epoch = datetime.datetime(1970,1,1,0,0,0,tzinfo=pytz.UTC)
>>> datetime.datetime(1970, 1, 1, 0, 0, tzinfo=<UTC>)
# get the total second difference
ts = (d - epoch).total_seconds()
>>> 1322726400.0
Also:
Be careful, using pytz for tzinfo in a datetime.datetime DOESN'T WORK for many timezones. See datetime with pytz timezone. Different offset depending on how tzinfo is set
# Don't do this:
d = datetime.datetime(2011, 12, 1,0,0,0, tzinfo=pytz.timezone('America/Los_Angeles'))
>>> datetime.datetime(2011, 1, 12, 0, 0,
tzinfo=<DstTzInfo 'America/Los_Angeles' LMT-1 day, 16:07:00 STD>)
# tzinfo in not PST but LMT here, with a 7min offset !!!
# when converting to UTC:
d = d.astimezone(pytz.UTC)
>>> datetime.datetime(2011, 1, 12, 7, 53, tzinfo=<UTC>)
# you end up with an offset
https://en.wikipedia.org/wiki/Local_mean_time
First you must the strptime class to convert the string to a struct_time format.
Then just use mktime from there to get your float.
I would suggest dateutil:
import dateutil.parser
dateutil.parser.parse("01/12/2011", dayfirst=True).timestamp()
Seems to be quite efficient:
import datetime
day, month, year = '01/12/2011'.split('/')
datetime.datetime(int(year), int(month), int(day)).timestamp()
1.61 µs ± 120 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
you can convert to isoformat
my_date = '2020/08/08'
my_date = my_date.replace('/','-') # just to adapte to your question
date_timestamp = datetime.datetime.fromisoformat(my_date).timestamp()
You can refer this following link for using strptime function from datetime.datetime, to convert date from any format along with time zone.
https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior
just use datetime.timestamp(your datetime instanse), datetime instance contains the timezone infomation, so the timestamp will be a standard utc timestamp. if you transform the datetime to timetuple, it will lose it's timezone, so the result will be error.
if you want to provide an interface, you should write like this:
int(datetime.timestamp(time_instance)) * 1000
A simple function to get UNIX Epoch time.
NOTE: This function assumes the input date time is in UTC format (Refer to comments here).
def utctimestamp(ts: str, DATETIME_FORMAT: str = "%d/%m/%Y"):
import datetime, calendar
ts = datetime.datetime.utcnow() if ts is None else datetime.datetime.strptime(ts, DATETIME_FORMAT)
return calendar.timegm(ts.utctimetuple())
Usage:
>>> utctimestamp("01/12/2011")
1322697600
>>> utctimestamp("2011-12-01", "%Y-%m-%d")
1322697600
You can go both directions, unix epoch <==> datetime :
import datetime
import time
the_date = datetime.datetime.fromtimestamp( 1639763585 )
unix_time = time.mktime(the_date.timetuple())
assert ( the_date == datetime.datetime.fromtimestamp(unix_time) ) & \
( time.mktime(the_date.timetuple()) == unix_time )

Categories