I looked at that link
It's weird because the query im doing is hit and miss.
It can't show the dates if the difference is only a few days
SQLAlchemy: how to filter date field?
model:
class UserCallsModel(db.Model):
id = db.Column(db.Integer, primary_key = True)
date = db.Column(db.String(90))
username = db.Column(db.String(90))
event_name = db.Column(db.String(90))
query:
users = UserCallsModel.query.filter(UserCallsModel.date.between("2016-1-1", "2016-1-20")).order_by(UserCallsModel.date.desc())
I've got 2 dates that fall within this range but is not getting queried?
I'm not familiar with MySQL, but I imagine it is the same as PG which I've included output below.
When you use the "between" method, you end up using the "BETWEEN" operator, like so...
SELECT * FROM my_table WHERE date BETWEEN '2016-1-1' AND '2016-1-20'
The problem is that the "between" operator does something different for dates versus strings. For example, if the value that it is testing is a string, it will see the arguments (the '2016-1-1' AND '2016-1-20' part) as strings.
mhildreth=# select '2016-1-5' between '2016-1-1' AND '2016-1-10';
?column?
----------
f
(1 row)
Meanwhile, if the value that it is testing is a date object, then it will implicitly convert the strings to date objects, essentially doing the following...
mhildreth=# select '2016-1-5'::date between '2016-1-1'::date AND '2016-1-10'::date;
?column?
----------
t
(1 row)
Thus, my guess is that you want to convert your "date" column to be a date type. If you must leave it a string, then you need to ensure that you are using a date format that also works when doing string comparison. Thus, you'll need 2016-01-01 rather than 2016-1-1.
I was under the impression that a string will actually be queried correctly as long as it was of a certain format. but nope I'm afraid it ain't so.
a better way of doing this if you have strings formatted like this:
"2016-1-5" is to simply convert the string date to a datetime.date object
python 3
import datetime
splitted_date = [int(number) for number in "2016-1-5".split("-")]
formatted_date = datetime.date(*splitted_date)
Related
I am having trouble passing a datetime.time variable into a SQLite database, I have some very basic code here to show what exactly the variable is.
import datetime as dt
time = dt.datetime.now().time()
time = time.strftime('%H:%M')
time = dt.datetime.strptime(time, '%H:%M').time()
print(time)
print(type(time))
time = dt.datetime.now().time() gets the current time in type datetime.time.
Output:
17:34:48.286215
<class 'datetime.time'>
time = time.strftime('%H:%M') is then retrieving just the hour and minute but is of type str
Output:
17:35
<class 'str'>
I then convert it back to a datetime.time with time = dt.datetime.strptime(time, '%H:%M').time() which gives the the output:
17:32:00
<class 'datetime.time'>
The column of type Time accepts the format of HH:SS as shown in the documentation (SQLite3 DateTime Documentation), so I am not sure why I am getting this error:
sqlite3.InterfaceError: Error binding parameter 11 - probably unsupported type.
From this INSERT statement:
cursor.execute("INSERT INTO booked_tickets VALUES (?,?,?,?,?,?,?,?,?,?,?,?)", (booking_ref, ticket_date, film, showing, ticket_type, num_tickets, cus_name, cus_phone, cus_email, ticket_price, booking_date, booking_time, ))
EDIT: As requested, here is a snippet of code to recreate the table with the broken columns:
import datetime as dt
import sqlite3
connection = sqlite3.connect("your_database.db")
cursor = connection.cursor()
# Get the current time
time = dt.datetime.now().time()
# Format the time as a string using the '%H:%M' format
time_str = time.strftime('%H:%M')
# Parse the string back to a time object using the '%H:%M' format
time = dt.datetime.strptime(time_str, '%H:%M').time()
# Create the table
cursor.execute("CREATE TABLE test (example_time Time)")
# Insert the time into the example_time column
cursor.execute("INSERT INTO test VALUES (?)", (time, ))
connection.commit()
connection.close()
There is no Date or Time data type in SQLite.
The documentation from the link that you have in your question clearly states that in SQLite you can store datetime in 3 ways: text in ISO-8601 format, integer unix epochs and float julian days.
If you chose the first way then you should pass strings:
booking_date = dt.datetime.now().date().strftime('%Y-%m-%d')
booking_time = dt.datetime.now().time().strftime('%H:%M:00')
sql = "INSERT INTO booked_tickets VALUES (?,?,?,?,?,?,?,?,?,?)"
cursor.execute(sql, (booking_ref, ticket_date, film, showing, ticket_type, num_tickets, cus_name, cus_phone, cus_email, ticket_price, booking_date, booking_time))
But, you could also let SQLite get the current date and/or time.
Assuming that in the columns booking_date and booking_time you want the current date and time, you can define these columns as:
booking_date TEXT NOT NULL DEFAULT CURRENT_DATE,
booking_time TEXT NOT NULL DEFAULT CURRENT_TIME
and then you don't need to pass anything for them in the INSERT statement:
sql = "INSERT INTO booked_tickets VALUES (?,?,?,?,?,?,?,?,?,?)"
cursor.execute(sql, (booking_ref, ticket_date, film, showing, ticket_type, num_tickets, cus_name, cus_phone, cus_email, ticket_price,))
Checkout the SQLite datatypes documentation
2.2. Date and Time Datatype
SQLite does not have a storage class set aside for storing dates
and/or times. Instead, the built-in Date And Time Functions of SQLite
are capable of storing dates and times as TEXT, REAL, or INTEGER
values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic
Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can choose to store dates and times in any of these
formats and freely convert between formats using the built-in date and
time functions.
Store the dates as TEXT datatypes.
The documentation you refer to mostly discusses how to format column values that representing dates and times. That is, it discusses what you can do with dates and times that already exist in your database.
It does, however, give just enough information to help you here I think. It says:
Date and time values can be stored as
text in a subset of the ISO-8601 format,
numbers representing the Julian day, or
numbers representing the number of seconds since (or before) 1970-01-01 00:00:00 UTC (the unix timestamp).
So you want to define and supply your dates and times as either full ISO-8601 date strings or as numbers. When defining a table, you indicate which of these formats you wish to use by defining a column type as a STRING, REAL or INTEGER respectively.
Here's some documentation that discusses how to store dates and times in one of these formats: https://www.sqlitetutorial.net/sqlite-date/
i have a column(scheduledStartDateTime) in database which is of type datetime and i have to search previous row of data based on user entered datetime .
my query is like this:
order = self.trips_session.query(Order).filter(
and_(
Order.driverSystemId == driver_system_id,
func.date(Order.scheduledStartDateTime) < func.date(start_date)
)).order_by(
DispatchOrder.scheduledStartDateTime.desc()).first()
my search date is 2020-01-13 07:16:06 i,e order number 5673 so ideally i am looking for order number 5677 but i am getting is 5679 . how can i compare dates based on hours minutes and seconds as well.
So you want to convert a datetime however func.date() casts the datetime to date, therefore you are missing hour/minute/seconds. You just need to perform the comparison as normal, without casting your datetimes:
order = self.trips_session.query(Order).filter(
and_(
Order.driverSystemId == driver_system_id,
Order.scheduledStartDateTime < start_date
)).order_by(
DispatchOrder.scheduledStartDateTime.desc()).first()
Alternatively, if one of the datetime's provided is not in datetime format, you can use func.datetime() to cast it/them without losing the time information.
I am trying call a function that triggers a report to be generated with a starting date that is either hour or days ago. The code below works fine but I would like to store the timedelta offset in a mysql database.
starting_date = datetime.today() - timedelta(days=-5)
I had hoped to store 'days=-5' in the database, extract that database column to variable 'delta_offset' and then run
starting_date = datetime.today() - timedelta(delta_offset)
It doesnt like this because delta_offset is a string. I know i could modify the function to just include the offset and store -5 in my database, like what is below. But I really wanted to store days=-5 in the database because my offset can be hours as well. I could make my offset in database always hours and store -120 in the database but was wondering if there was an elegant way where I store 'days=-5' in the database and not cause type issues
starting_date = datetime.today() - timedelta(days=delta_offset)
Instead of storing 'days=-5' in your database as a single column, you could break this into two columns named 'value' and 'unit' or similar.
Then you can pass these to timedelta in a dictionary and unpacking. Like so:
unit = 'days'
value = -5
starting_date = datetime.today() - timedelta(**{unit: value})
This will unpack the dictionary so you get the same result as doing timedelta([unit]=value).
Alternatively, if you really would like to keep 'days=-5' as a value of a single column in your database, you could split the string on '=' then take a similar approach. Here's how:
offset = 'days=-5'
unit, value = offset.split('=')
starting_date = datetime.today() - timedelta(**{unit: int(value)})
i would do it this way:
date_offset_split = date_offset.split("=")
kwargs = {date_offset_split[0]: int(date_offset_split[1])}
starting_date = datetime.today() - timedelta(**kwargs)
I was having trouble manipulating a time-series data provided to me for a project. The data contains the number of flight bookings made on a website per second in a duration of 30 minutes. Here is a part of the column containing the timestamp
>>> df['Date_time']
0 7/14/2017 2:14:14 PM
1 7/14/2017 2:14:37 PM
2 7/14/2017 2:14:38 PM
I wanted to do
>>> pd.set_index('Date_time')
and use the datetime and timedelta methods provided by pandas to generate the timestamp to be used as index to access and modify any value in any cell.
Something like
>>> td=datetime(year=2017,month=7,day=14,hour=2,minute=14,second=36)
>>> td1=dt.timedelta(minutes=1,seconds=58)
>>> ti1=td1+td
>>> df.at[ti1,'column_name']=65000
But the timestamp generated is of the form
>>> print(ti1)
2017-07-14 02:16:34
Which cannot be directly used as an index in my case as can be clearly seen. Is there a workaround for the above case without writing additional methods myself?
I want to do the above as it provides me greater level of control over the data than looking for the default numerical index for each row I want to update and hence will prove more efficient accordig to me
Can you check the dtype of the 'Date_time' column and confirm for me that it is string (object) ?
df.dtypes
If so, you should be able to cast the values to pd.Timestamp by using the following.
df['timestamp'] = df['Date_time'].apply(pd.Timestamp)
When we call .dtypes now, we should have a 'timestamp' field of type datetime64[ns], which allows us to use builtin pandas methods more easily.
I would suggest it is prudent to index the dataframe by the timestamp too, achieved by setting the index equal to that column.
df.set_index('timestamp', inplace=True)
We should now be able to use some more useful methods such as
df.loc[timestamp_to_check, :]
df.loc[start_time_stamp : end_timestamp, : ]
df.asof(timestamp_to_check)
to lookup values from the DataFrame based upon passing a datetime.datetime / pd.Timestamp / np.datetime64 into the above. Note that you will need to cast any string (object) 'lookups' to one of the above types in order to make use of the above correctly.
I prefer to use pd.Timestamp() - https://pandas.pydata.org/pandas-docs/stable/generated/pandas.Timestamp.html to handle datetime conversion from strings unless I am explicitly certain of what format the datetime string is always going to be in.
I am running sqlite to select data between two ranges for a sales report. To select the data from between two dates I use the following statement:
SELECT * FROM test WHERE date BETWEEN "11/1/2011" AND "11/8/2011";
This statement grabs all the dates even those outside the criteria. The date format you see entered is in the same format that I get back. I'm not sure what's wrong.
SQLite requires dates to be in YYYY-MM-DD format. Since the data in your database and the string in your query isn't in that format, it is probably treating your "dates" as strings.
Change your data to that formats to use sqlite datetime formats.
YYYY-MM-DD
YYYY-MM-DD HH:MM
YYYY-MM-DD HH:MM:SS
YYYY-MM-DD HH:MM:SS.SSS
YYYY-MM-DDTHH:MM
YYYY-MM-DDTHH:MM:SS
YYYY-MM-DDTHH:MM:SS.SSS
HH:MM
HH:MM:SS
HH:MM:SS.SSS
now
DDDDDDDDDD
SELECT * FROM test WHERE date BETWEEN '2011-01-11' AND '2011-08-11'
One more way to select between dates in SQLite is to use the powerful strftime function:
SELECT * FROM test WHERE strftime('%Y-%m-%d', date) BETWEEN "11-01-2011" AND "11-08-2011"
These are equivalent according to https://sqlite.org/lang_datefunc.html:
date(...)
strftime('%Y-%m-%d', ...)
but if you want more choice, you have it.
SELECT *
FROM TableName
WHERE julianday(substr(date,7)||'-'||substr(date,4,2)||'-'||substr(date,1,2)) BETWEEN julianday('2011-01-11') AND julianday('2011-08-11')
Note that I use the format: dd/mm/yyyy.
If you use d/m/yyyy, Change in substr().
Or you can cast your string to Date format with date function. Even the date is stored as TEXT in the DB.
Like this (the most workable variant):
SELECT * FROM test WHERE date(date)
BETWEEN date('2011-01-11') AND date('2011-08-11')
SQLite does not have a concept of dates. It only knows them as text. When you do this in SQLite you're actually doing string comparisons. You can read more from the official documentation.
When two TEXT values are compared an appropriate collating sequence is used to determine the result.
Any numeric (i.e., not using words like 'May') format for dates that is padded and in order from biggest field to smallest field will work. "2021-05-07" (May 7th) comes before "2021-05-09" (May 9th). So if you use "yyyy-mm-dd" format then you'll be set. "yyyy/mm/dd" and "yyyymmdd" work just fine too. (For a better phrasing on "sortable" date formats check out RFC 3339 section 5.1.)
A reason to use "yyyy-mm-dd" format is because that's the format that SQLite's builtin date uses.
Special thanks to Jeff and vapcguy your interactivity is really encouraging.
Here is a more complex statement that is useful when the length between '/' is unknown::
SELECT * FROM tableName
WHERE julianday(
substr(substr(date, instr(date, '/')+1), instr(substr(date, instr(date, '/')+1), '/')+1)
||'-'||
case when length(
substr(date, instr(date, '/')+1, instr(substr(date, instr(date, '/')+1),'/')-1)
)=2
then
substr(date, instr(date, '/')+1, instr(substr(date, instr(date, '/')+1), '/')-1)
else
'0'||substr(date, instr(date, '/')+1, instr(substr(date, instr(date, '/')+1), '/')-1)
end
||'-'||
case when length(substr(date,1, instr(date, '/')-1 )) =2
then substr(date,1, instr(date, '/')-1 )
else
'0'||substr(date,1, instr(date, '/')-1 )
end
) BETWEEN julianday('2015-03-14') AND julianday('2015-03-16')
Put the variable in the Where Condition and parse both dates using 'BETWEEN':
SELECT * FROM emp_master
-> if you have date formate like dd/mm/yyyy simple then,
WHERE joined_date BETWEEN '01/03/2021' AND '01/09/2021';
-> and if you have date formate like yyyy/mm/dd then,
WHERE joined_date BETWEEN '2021/03/01' AND '2021/09/01';
☻♥ Done Keep Code.
Let's say you are preparing data for some report. Then the whole ordeal will look similar to this.
--add column with date in ISO 8601
ALTER TABLE sometable ADD COLUMN DateInISO8601;
--update the date from US date to ISO8601 date
UPDATE sometable
SET DateInISO8601 = substr([DateInUSformat],length([DateInUSformat])+1, -4)
|| '-' ||
substr('00' || [DateInUSformat],instr('00' || [DateInUSformat],'/'),-2)
|| '-' ||
substr('00' || rtrim(substr([DateInUSformat],instr([DateInUSformat],'/')+1,2),'/'),-2,2);
SELECT DateInISO8601
FROM sometable
WHERE DateInISO8601 BETWEEN '2022-02-02' AND '2022-02-22';
You can of course do all that on the fly, but if you have the choice -- don't. Use the ISO date by default and convert it on the way in and out to SQLite DB.