I have a couple classes extending builtin datetime.*
Is there any good reason to not overload + (MyTime.__radd___) so MyDate + MyTime returns a MyDateTime?
This is already implemented as a class method, datetime.datetime.combine:
import datetime
d = datetime.date(2010, 12, 5)
t = datetime.time(10, 22, 15)
dt = datetime.datetime.combine(d, t)
print dt
prints
2010-12-05 10:22:15
This would generally be frowned upon because you're really combining rather than adding; this is why the actual datetime library has a combine method rather than using addition in this way.
I'm not aware of any other cases in Python where <instance of TypeA> + <instance of TypeB> produces <instance of TypeC>. Thus, the Principle of least astonishment suggests that you should simply provide a combine method rather than overload addition.
Yes, there is at least one good reason not to: the resulting instance is completely different from the two input instances. Is this important? I don't think so -- consider that date - date yields timedelta.
The way I see it:
Does adding two dates together make sense? No.
Does adding two times together make sense? No.
Does adding a date and a time together make sense? Yup!
Does adding a date and a timedelta togethor make sense? Maybe.
Does adding a time and a timedelta together make sense? Maybe.
and for subtraction
Does subtracting two dates make sense? Yes.
Does subtracting two times make sense? Yes.
Does subtracting a time from a date make sense? Nope.
Does subtracting a timedelta from a date make sense? Maybe.
Does subtracting a timedelta from a time make sense? Maybe.
Developing along the lines of what makes sense:
date + time => datetime
date + timedelta => date | datetime or exception or silently drop time portion
time + date => datetime
time + timedelta => time | wrap-around or exception
date - date => timedelta
date - timedelta => date | datetime or exception or silently drop time portion
time - time => timedelta
time - timedelta => time | wrap-around or exception
datetime + timedelta => datetime
datetime - timedelta => datetime
So, if it were me and I were designing a Date, Time, DateTime, TimeDelta framework, I would allow:
date + time
date - date
time - time
datetime + timedelta
datetime - timedelta
and for these:
date +/- timedelta
time +/- timedelta
I would default to returning the same type if the timedelta had none of the other type, and raising an exception if the timedelta did have some of the other type, but there would be a setting that would control that. The other possible behavior would be to drop the unneeded portion -- so a date combined with a timedelta that had hours would drop the hours and return a date.
Due to the existence of the date, time, and datetime cross-type addition and subtraction operators, I would think that this is fine, so long as it is well defined.
Currently (2.7.2):
date = date + timedelta
date = date - timedelta
timedelta = date - date
datetime = datetime + timedelta
datetime = datetime - timedelta
timedelta = datetime - datetime
I believe the following is also reasonable for an extension:
timedelta = time - time
datetime = date + time
I was going to suggest the following as well, but time has very specific min and max values for hour, minute, second, and microsecond, thus requiring a silent wraparound of values or returning of a different type:
time = time + timedelta
time = time - timedelta
Similarly, date cannot handle a timedelta of less than a day being added to it. Often I have been told to simply use Duck Typing with Python, because that's the intent. If that is true, then I would propose the following completed interface:
[date|datetime] = date + timedelta
[date|datetime] = date - timedelta
timedelta = date - date
[time|timedelta] = time + timedelta
[time|timedelta] = time - timedelta
timedelta = time - time
datetime = datetime + timedelta
datetime = datetime - timedelta
datetime = date + time
datetime = date - time
timedelta = datetime - datetime
timedelta = datetime - date
timedelta = timedelta + timedelta
timedelta = timedelta - timedelta
In which, given the case that date has precision loss (for timedelta's with partial days), it is promoted to datetime. Similarly, given the case that time has precision loss (for timedelta's that yield a result of more than one day, or negative time), it is promoted to timedelta. However, I'm not fully comfortable with [time|timedelta]. It makes sense given the rest of the interface from parallelism and precision views, but I do think it might be more elegant to just wraparound the time to the proper hour, thus changing all the [time|timedelta]'s to simply time, but unfortunately that leaves us with lost precision.
In my opinion, the most valuable uses of operator overloading are situations where many input values can be combined. You'd never want to deal with:
concat(concat(concat("Hello", ", "), concat("World", "!")), '\n');
or
distance = sqrt(add(add(x*x, y*y), z*z));
So we overload math symbols to create a more intuitive syntax. Another way to deal with this problem is variadic functions, like + in Scheme.
With your date + time = datetime, it doesn't make sense to add datetime + datetime, datetime + time, or datetime + date, so you could never encounter a situation like those above.
In my opinion, once again, the right thing is to use a constructor method. In a language with strong typing like C++, you'd have DateTime(const Date &d, const Time &t). With Python's dynamic typing, I guess they gave the function a name, datetime.combine(date, time), to make the code clearer when the types of the input variables are not visible in the code.
I guess most important things are functionality and efficiency. Of course using a simple + operator will be easier to use, but i am not sure about functionality.
If we compare it to datetime.combine, What combine do is:
dt = date(2011,01,01)
tm = time(20,00)
dtm = datetime.combine(dt, tm)
For dtm
If dt is a date object and tm is a time object, than date info is taken from dt, time info and tzinfo is taken from tm object
if dt is a datetime object, than its time and tzinfo attributes will be ignored.
From that point of view, working with datetime objects do not seem to be simple objects, but more compex structures with diffrent attributes, like timezone info.
Probably thats why datetime objects have some additional functions that is used for formatting object type and data structure of the object.
Python have a motto (something like that):
In python, nothing is unchangable, if you know what you are doing. If not, it is better to leave library functions as they are...
So, in my opinion, it is better you use combine that overloading + operator
Related
I am using datetime.strptime() to convert a string containing time and date from a sensor into a datetime object.
The code sometimes fails. Minimal example:
datetime.strptime('1/9/2021 24:01:53', '%d/%m/%Y %H:%M:%S')
Output error:
ValueError: time data '1/9/2021 24:01:53' does not match format '%d/%m/%Y %H:%M:%S'
I am guessing this has to do with the fact that the time is more than 23:59:59 - which seems to me a non-realistic time (I would think that 1/9/2021 24:01:53 could potentially be 2/9/2021 00:01:53 - a time format which I have never seen).
Is this a non-standard way of representing time or possibly a hardware/software issue with the sensor acquisition system? If it is a different way of representing time, how can I convert it to a standard datetime object?
Kind regards,
D.F.
If the hour exceeds 23 in a variable representing time, a good option is to create a timedelta from it, which you can then add to a datetime object. For given example that might look like
from datetime import datetime, timedelta
def custom_todatetime(s):
"""
split date/time string formatted as 'DD/MM/YYYY hh:mm:ss' into date and time parts.
parse date part to datetime and add time part as timedelta.
"""
parts = s.split(' ')
seconds = sum(int(x) * 60 ** i for i, x in enumerate(reversed(parts[1].split(':'))))
return datetime.strptime(parts[0], "%d/%m/%Y") + timedelta(seconds=seconds)
s = '1/9/2021 24:01:53'
print(custom_todatetime(s))
# 2021-09-02 00:01:53
Note: conversion of hh:mm:ss to seconds taken from here - give a +1 there if helpful.
I have a time series that I have pulled from a netCDF file and I'm trying to convert them to a datetime format. The format of the time series is in 'days since 1990-01-01 00:00:00 +10' (+10 being GMT: +10)
time = nc_data.variables['time'][:]
time_idx = 0 # first timestamp
print time[time_idx]
9465.0
My desired output is a datetime object like so (also GMT +10):
"2015-12-01 00:00:00"
I have tried converting this using the time module without much success although I believe I may be using wrong (I'm still a novice in python and programming).
import time
time_datetime = time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(time[time_idx]*24*60*60))
Any advice appreciated,
Cheers!
The datetime module's timedelta is probably what you're looking for.
For example:
from datetime import date, timedelta
days = 9465 # This may work for floats in general, but using integers
# is more precise (e.g. days = int(9465.0))
start = date(1990,1,1) # This is the "days since" part
delta = timedelta(days) # Create a time delta object from the number of days
offset = start + delta # Add the specified number of days to 1990
print(offset) # >>> 2015-12-01
print(type(offset)) # >>> <class 'datetime.date'>
You can then use and/or manipulate the offset object, or convert it to a string representation however you see fit.
You can use the same format as for this date object as you do for your time_datetime:
print(offset.strftime('%Y-%m-%d %H:%M:%S'))
Output:
2015-12-01 00:00:00
Instead of using a date object, you could use a datetime object instead if, for example, you were later going to add hours/minutes/seconds/timezone offsets to it.
The code would stay the same as above with the exception of two lines:
# Here, you're importing datetime instead of date
from datetime import datetime, timedelta
# Here, you're creating a datetime object instead of a date object
start = datetime(1990,1,1) # This is the "days since" part
Note: Although you don't state it, but the other answer suggests you might be looking for timezone aware datetimes. If that's the case, dateutil is the way to go in Python 2 as the other answer suggests. In Python 3, you'd want to use the datetime module's tzinfo.
netCDF num2date is the correct function to use here:
import netCDF4
ncfile = netCDF4.Dataset('./foo.nc', 'r')
time = ncfile.variables['time'] # do not cast to numpy array yet
time_convert = netCDF4.num2date(time[:], time.units, time.calendar)
This will convert number of days since 1900-01-01 (i.e. the units of time) to python datetime objects. If time does not have a calendar attribute, you'll need to specify the calendar, or use the default of standard.
We can do this in a couple steps. First, we are going to use the dateutil library to handle our work. It will make some of this easier.
The first step is to get a datetime object from your string (1990-01-01 00:00:00 +10). We'll do that with the following code:
from datetime import datetime
from dateutil.relativedelta import relativedelta
import dateutil.parser
days_since = '1990-01-01 00:00:00 +10'
days_since_dt = dateutil.parser.parse(days_since)
Now, our days_since_dt will look like this:
datetime.datetime(1990, 1, 1, 0, 0, tzinfo=tzoffset(None, 36000))
We'll use that in our next step, of determining the new date. We'll use relativedelta in dateutils to handle this math.
new_date = days_since_dt + relativedelta(days=9465.0)
This will result in your value in new_date having a value of:
datetime.datetime(2015, 12, 1, 0, 0, tzinfo=tzoffset(None, 36000))
This method ensures that the answer you receive continues to be in GMT+10.
How can I convert a dateutil.relativedelta object to a datetime.timedelta object?
e.g.,
# pip install python-dateutil
from dateutil.relativedelta import relativedelta
from datetime import timedelta
rel_delta = relativedelta(months=-2)
# How can I convert rel_delta to a timedelta object so that I can call total_seconds() ?
time_delta = ???(rel_delta)
time_delta.total_seconds() # call the timedelta.total_seconds() method
You can't, for one huge reason: They don't store the same information. datetime.timedelta only stores days, seconds, and milliseconds, whereas dateutil.relativedelta stores every single time component fed to it.
That dateutil.relativedelta does so is important for storing things such as a difference of 1 month, but since the length of a month can vary this means that there is no way at all to express the same thing in datetime.timedelta.
In case someone is looking to convert a relativedelta to a timedelta from a specific date, simply add and subtract the known time:
utcnow = datetime.utcnow()
rel_delta = relativedelta(months=-2)
time_delta = utcnow + rel_delta - utcnow # e.g, datetime.timedelta(days=-62)
As a commenter points out, the resulting timedelta value will differ based on what month it is.
Depending on why you want to call total_seconds, it may be possible to refactor your code to avoid the conversion altogether. For example, consider a check on whether or not a user is over 18 years old:
datetime.date.today() - user['dateOfBirth'] < datetime.timedelta(days=365*18)
This check is not a good idea, because the timedelta object does not account for things like leap years. It's tempting to rewrite as:
datetime.date.today() - user['dateOfBirth'] < dateutil.relativedelta.relativedelta(years=18)
which would require comparing a timedelta (LHS) to a relativedelta (RHS), or converting one to the other. However, you can refactor the check to avoid this conversion altogether:
user['dateOfBirth'] + dateutil.relativedelta.relativedelta(years=18) > datetime.date.today()
It looks like this has been covered somewhat in other questions, but I'm still fairly confused on how to actually do this. My lack of experience isn't helping much with that.
I have two DateTimeProperties - StartTime and EndTime. I'm subtracting StartTime from EndTime to get the Duration. From my previous question (thank you to all that answered!) it looks like this operation is producing a timedelta.
There doesn't seem to be an easy way to store timedelta directly in the GAE datastore, so this means I need to convert it either to an int in milliseconds, to a float in seconds or to time.
I will need to do other calculations on this later as well, such as figuring out avg. duration. Based on that, int seems to make the most sense to me right now.
What's the best way to do this or is there a tutorial I can play with?
Thank you!
To make this as easy as possible to work with, there's two steps: Converting the timedelta to an int or a float, and storing it in the datastore. First things first, converting a timedelta to a microtime:
def timedelta_to_microtime(td):
return td.microseconds + (td.seconds + td.days * 86400) * 1000000
You don't have to do the conversion yourself, though - you can define a custom datastore property, which will allow you to store timedeltas directly to your model:
class TimeDeltaProperty(db.Property):
def get_value_for_datastore(self, model_instance):
value = self.__get__(model_instance, model_instance.__class__)
if value is not None:
return timedelta_to_microtime(value)
def make_value_from_datastore(self, value):
if value is not None:
return datetime.timedelta(microseconds=value)
Now you can use this property like any other:
class MyModel(db.Model):
td = TimeDeltaProperty(required=True)
entity = MyModel(td=datetime.datetime.now()-some_datetime)
key = entity.put()
entity = db.get(key)
print entity.td
If you're going to store it as a datetime (which I agree is a good idea), I'd extend the DateTimeProperty - then you get various bits of parsing and validation for free.
Also, storing as timedelta as a datetime can be much easier than the other methods given here, by storing it as a datetime some distance from a reference datetime, such that the difference represents the timedelta. This is really easy thanks to the operator overloading the datetime module gives us.
from datetime import datetime, timedelta
from google.appengine.ext import db
class TimeDeltaProperty(db.DateTimeProperty):
# Use a reference datetime half way between the min and max possible
# datetimes, so that we can support both +ve and -ve timedeltas
ref_datetime = (datetime.max - datetime.min) / 2 + datetime.min
def get_value_for_datastore(self, model_instance):
# Get the timedelta instance assigned to this property
td = super(TimeDeltaProperty, self).get_value_for_datastore(model_instance)
if td is not None:
# datetime + timedelta = datetime
return self.ref_datetime + td
def make_value_from_datastore(self, dt):
if dt is not None:
# datetime - datetime = timedelta
return dt - self.ref_datetime
And here's an equivalent implementation for the NDB API, if you're that way inclined:
from datetime import datetime, timedelta
from google.appengine.ext import ndb
class TimeDeltaProperty(ndb.DateTimeProperty):
# Use a reference datetime half way between the min and max possible
# datetimes, so that we can support both +ve and -ve timedeltas
ref_datetime = (datetime.max - datetime.min) / 2 + datetime.min
def _validate(self, value):
if not isinstance(value, timedelta):
raise TypeError('expected a datetime.timedelta, got %r' % value)
def _to_base_type(self, value):
# datetime + timedelta = datetime
return self.ref_datetime + td
def _from_base_type(self, value):
# datetime - datetime = timedelta
return dt - self.ref_datetime
Accuracy
A timedelta in Python can handle deltas of roughly +/-2.7 million years. However, a datetime only covers a range of about 10,000 years. To store a greater timedelta in a datetime, you'll have to do some shifting and sacrifice some accuracy.
The approach above limits timedeltas to half this range - about +/-5000 years, because of the choice of reference datetime.
If you know your timedelta will always be positive, you can use ref_datetime = datetime.min (or if you know it'll always be negative you can use ref_datetime = datetime.max) to get the full range of about 10,000 years.
import pickle
import datetime
...
delta = end_time - start_time
for_storage = pickle.dumps(delta)
#now you have a string representation of your timedelta object that you can store
#sometime later...
delta = pickle.loads(from_storage)
You'll still need to convert the delta to a time resolution of your choice using the days, mins, seconds, and microseconds attributes of the time delta.
This ultimately worked:
delta = StartTime - EndTime
event_record.Duration = int((delta.microseconds)/1000)
basically, needed to get microseconds out of the timedelta and convert it to milliseconds.
What is the best way to handle portions of a second in Python? The datetime library is excellent, but as far as I can tell it cannot handle any unit less than a second.
In the datetime module, the datetime, time, and timedelta classes all have the smallest resolution of microseconds:
>>> from datetime import datetime, timedelta
>>> now = datetime.now()
>>> now
datetime.datetime(2009, 12, 4, 23, 3, 27, 343000)
>>> now.microsecond
343000
if you want to display a datetime with fractional seconds, just insert a decimal point and strip trailing zeros:
>>> now.strftime("%Y-%m-%d %H:%M:%S.%f").rstrip('0')
'2009-12-04 23:03:27.343'
the datetime and time classes only accept integer input and hours, minutes and seconds must be between 0 to 59 and microseconds must be between 0 and 999999. The timedelta class, however, will accept floating point values with fractions and do all the proper modulo arithmetic for you:
>>> span = timedelta(seconds=3662.567)
>>> span
datetime.timedelta(0, 3662, 567000)
The basic components of timedelta are day, second and microsecond (0, 3662, 567000 above), but the constructor will also accept milliseconds, hours and weeks. All inputs may be integers or floats (positive or negative). All arguments are converted to the base units and then normalized so that 0 <= seconds < 60 and 0 <= microseconds < 1000000.
You can add or subtract the span to a datetime or time instance or to another span. Fool around with it, you can probably easily come up with some functions or classes to do exaxtly what you want. You could probably do all your date/time processing using timedelta instances relative to some fixed datetime, say basetime = datetime(2000,1,1,0,0,0), then convert to a datetime or time instance for display or storage.
A different, non mentioned approach which I like:
from datetime import datetime
from time import sleep
t0 = datetime.now()
sleep(3)
t1 = datetime.now()
tdelta = t1 - t0
print(tdelta.total_seconds())
# will print something near (but not exactly 3)
# 3.0067
To get a better answer you'll need to specify your question further, but this should show at least how datetime can handle microseconds:
>>> from datetime import datetime
>>> t=datetime.now()
>>> t.microsecond
519943
NumPy 1.4 (in release candidate stage) has support for its own Date and DateArray objects. The one advantage is that it supports frequencies smaller than femtoseconds: http://projects.scipy.org/numpy/browser/trunk/doc/neps/datetime-proposal.rst
Otherwise I would go with the regular datetime subsecond frequencies.