I am trying call a function that triggers a report to be generated with a starting date that is either hour or days ago. The code below works fine but I would like to store the timedelta offset in a mysql database.
starting_date = datetime.today() - timedelta(days=-5)
I had hoped to store 'days=-5' in the database, extract that database column to variable 'delta_offset' and then run
starting_date = datetime.today() - timedelta(delta_offset)
It doesnt like this because delta_offset is a string. I know i could modify the function to just include the offset and store -5 in my database, like what is below. But I really wanted to store days=-5 in the database because my offset can be hours as well. I could make my offset in database always hours and store -120 in the database but was wondering if there was an elegant way where I store 'days=-5' in the database and not cause type issues
starting_date = datetime.today() - timedelta(days=delta_offset)
Instead of storing 'days=-5' in your database as a single column, you could break this into two columns named 'value' and 'unit' or similar.
Then you can pass these to timedelta in a dictionary and unpacking. Like so:
unit = 'days'
value = -5
starting_date = datetime.today() - timedelta(**{unit: value})
This will unpack the dictionary so you get the same result as doing timedelta([unit]=value).
Alternatively, if you really would like to keep 'days=-5' as a value of a single column in your database, you could split the string on '=' then take a similar approach. Here's how:
offset = 'days=-5'
unit, value = offset.split('=')
starting_date = datetime.today() - timedelta(**{unit: int(value)})
i would do it this way:
date_offset_split = date_offset.split("=")
kwargs = {date_offset_split[0]: int(date_offset_split[1])}
starting_date = datetime.today() - timedelta(**kwargs)
Related
A mysql database table has a column whose datatype is time ( http://dev.mysql.com/doc/refman/5.0/en/time.html ). When the table data is accessed, Python returns the value of this column as a datetime.timedelta object. How do I extract the time out of this? (I didn't really understand what timedelta is for from the python manuals).
E.g. The column in the table contains the value "18:00:00"
Python-MySQLdb returns this as datetime.timedelta(0, 64800)
Please ignore what is below (it does return different value) -
Added: Irrespective of the time value in the table, python-MySQLdb seems to only return datetime.timedelta(0, 64800).
Note: I use Python 2.4
It's strange that Python returns the value as a datetime.timedelta. It probably should return a datetime.time. Anyway, it looks like it's returning the elapsed time since midnight (assuming the column in the table is 6:00 PM). In order to convert to a datetime.time, you can do the following::
value = datetime.timedelta(0, 64800)
(datetime.datetime.min + value).time()
datetime.datetime.min and datetime.time() are, of course, documented as part of the datetime module if you want more information.
A datetime.timedelta is, by the way, a representation of the difference between two datetime.datetime values. So if you subtract one datetime.datetime from another, you will get a datetime.timedelta. And if you add a datetime.datetime with a datetime.timedelta, you'll get a datetime.datetime. That's how the code above works.
It seems to me that the TIME type in MySQL is intended to represent time intervals as datetime.timedelta does in Python. From the docs you referenced:
TIME values may range from '-838:59:59' to '838:59:59'. The hours part may be so large because the TIME type can be used not only to represent a time of day (which must be less than 24 hours), but also elapsed time or a time interval between two events (which may be much greater than 24 hours, or even negative).
An alternative to converting from datetime.timedelta to datetime.time would be to change the column type to DATETIME and not using the date fields.
-Insert:
tIn = datetime.datetime(
year=datetime.MINYEAR,
month=1,
day=1,
hour=10,
minute=52,
second=10
)
cursor.execute('INSERT INTO TableName (TimeColumn) VALUES (%s)', [tIn])
-Select:
cursor.execute('SELECT TimeColumn FROM TableName')
result = cursor.fetchone()
if result is not None:
tOut = result[0].time()
print 'Selected time: {0}:{1}:{2}'.format(tOut.hour, tOut.minute, tOut.second)
datetime.time() is called on a datetime object to get a time object.
I'm trying to have a start and end date as variables in a long SQL query in python that generates a dataframe. I've gone through the other posts regarding this and tried everything I know but I get errors and none of them work. I've shortened the sql query to show just the relevant part. Can anyone please give me any suggestions? I think the issue might have to do with the format of the date.
def get_dataframe():
startdate = 'input_startdate'
enddate = 'input_enddate'
query="""
where date between ? and ?
"""
params={'start':startdate, 'end':enddate}
conn = db.msSQLConnect()
df = pd.read_sql(query,conn,params)
return df
Remove quotes around startdate and enddate variable assignments.
With quotes literal string 'input_startdate' and 'input_enddate' are passed to the query, instead of the date values of the variables.
startdate = input_startdate
enddate = input_enddate
I have two different dates that I am pulling from a database using a SQL query. Im looking to do transformations in Python, but the two main dates I want to work with are stored in different formats. The first date is of the date format (YYYY/MM/DD) the other is of (YYYY/MM/DD HH:MM:SS) format. I want a difference in days so the DATETIME is irrelevant on the second date. I was wondering what is the easiest way to do this in python? Ideally, I would like to automate this, where I create a DATE format of the DATETIME variable, and take the difference between the two DATES.
I've tried the following but I am also getting errors since I am dealing with Series. I am trying to get the delta for every row.
df.delta = (df.DATETIME - df.DATE)
and
df.delta = datetime.timedelta(df.DATETIME - df.DATE)
import datetime
d1 = datetime.datetime.strptime('2018/01/13', '%Y/%m/%d')
d2 = datetime.datetime.strptime('2018/01/15 18:34:02', '%Y/%m/%d %H:%M:%S')
delta = d2 - d1
print delta.total_seconds()
print delta.days
Convert your datetime object to a date object, you are then able to subtract them for a delta value.
df.delta = (df.DATETIME.date() - df.DATE)
I have a view in Postgres which queries a master table (150 million rows) and retrieves data from the prior day (a function which returns SELECT yesterday; it was the only way to get the view to respect my partition constraints) and then joins it with two dimension tables. This works fine, but how would I loop through this query in Python? Is there a way to make the date dynamic?
for date in date_range('2016-06-01', '2017-07-31'):
(query from the view, replacing the date with the date in the loop)
My workaround was to literally copy and paste the entire view as a huge select statement format string, and then pass in the date in a loop. This worked, but it seems like there must be a better solution to utilize an existing view or to pass in a variable which might be useful in the future.
To loop day by day inside the interval on a for loop you could do something like:
import datetime
initialDate = datetime.datetime(2016, 6, 1)
finalDate = datetime.datetime(2017, 7, 31)
for day in range((finalDate - initialDate).days + 1):
current = (initialDate + datetime.timedelta(days = day)).date()
print("query from the view, replacing the date with " + current.strftime('%m/%d/%Y'))
Replacing the print with the call to your query. If the dates are strings you can do something like:
initialDate = datetime.datetime.strptime("06/01/2016", '%m/%d/%Y')
I looked at that link
It's weird because the query im doing is hit and miss.
It can't show the dates if the difference is only a few days
SQLAlchemy: how to filter date field?
model:
class UserCallsModel(db.Model):
id = db.Column(db.Integer, primary_key = True)
date = db.Column(db.String(90))
username = db.Column(db.String(90))
event_name = db.Column(db.String(90))
query:
users = UserCallsModel.query.filter(UserCallsModel.date.between("2016-1-1", "2016-1-20")).order_by(UserCallsModel.date.desc())
I've got 2 dates that fall within this range but is not getting queried?
I'm not familiar with MySQL, but I imagine it is the same as PG which I've included output below.
When you use the "between" method, you end up using the "BETWEEN" operator, like so...
SELECT * FROM my_table WHERE date BETWEEN '2016-1-1' AND '2016-1-20'
The problem is that the "between" operator does something different for dates versus strings. For example, if the value that it is testing is a string, it will see the arguments (the '2016-1-1' AND '2016-1-20' part) as strings.
mhildreth=# select '2016-1-5' between '2016-1-1' AND '2016-1-10';
?column?
----------
f
(1 row)
Meanwhile, if the value that it is testing is a date object, then it will implicitly convert the strings to date objects, essentially doing the following...
mhildreth=# select '2016-1-5'::date between '2016-1-1'::date AND '2016-1-10'::date;
?column?
----------
t
(1 row)
Thus, my guess is that you want to convert your "date" column to be a date type. If you must leave it a string, then you need to ensure that you are using a date format that also works when doing string comparison. Thus, you'll need 2016-01-01 rather than 2016-1-1.
I was under the impression that a string will actually be queried correctly as long as it was of a certain format. but nope I'm afraid it ain't so.
a better way of doing this if you have strings formatted like this:
"2016-1-5" is to simply convert the string date to a datetime.date object
python 3
import datetime
splitted_date = [int(number) for number in "2016-1-5".split("-")]
formatted_date = datetime.date(*splitted_date)