I'm using mysqlx XDevAPI for python (NoSQL). I can't insert a date into my table.
table = my_schema.get_table("date_table")
field = "date_field"
insert = table.insert(field)
value = datetime.datetime.now()
insert.values(value)
insert.execute
I get an error:
ValueError: Expected token type 19 at pos 0 but found type 72
I'm presuming it's to do with the date/datetime format but I'm not sure how to find what tokens 19 or 72 are. If I try to insert a string or int I get the same error.
Yes. Like Rui said, Connector/Python doesn't support Python object conversion in the X DevAPI, you need to convert the datetime to a string format before inserting.
I try to extract a date from a SQL Server Table. I get my query to return it like this:
Hours = pd.read_sql_query("select * from tblAllHours",con)
Now I convert my "Start" Column in the Hours dataframe like this:
Hours['Start'] = pd.to_datetime(Hours['Start'], format='%Y-%m-%d')
then I select the row I want in the column like this:
StartDate1 = Hours.loc[Hours.Month == Sym1, 'Start'].values
Now, if I print my variable print(StartDate1) I get this result:
[datetime.date(2020, 10, 1)]
What I need is actually 2020-10-01
How can I get this result?
You currently have a column of datetimes. The format you're requesting is a string format
Use pandas.Series.dt.strftime to convert the datetime to a string
pd.to_datetime(Hours['Start'], format='%Y-%m-%d'): format tells the parser what format your dates are in, so they can be converted to a datetime, it is not a way to indicate the format you want the datetime.
Review pandas.to_datetime
If you want only the values, not the Series, use .values at the end of the following command, as you did in the question.
start_date_str = Hours.Start.dt.strftime('%Y-%m-%d')
try
print(Hours['Start'].dt.strftime('%Y-%m-%d').values)
result is a list of YYYY-MM-dd
['2020-07-03', '2020-07-02']
a bit similar to this How to change the datetime format in pandas
I run into an error that states integer is required (got type Column) - From my understanding, the column obviously contains an integer converted time string. Why am I not able to pass the column into the date parameter?
from datetime import date, timedelta
maxdate = spark.sql("select year(max(weekenddate)) as MaxYear, month(max(weekenddate)) as MaxMonth, day(max(weekenddate))as MaxDay from starsdata.sep_sdf_shipments_data a")
start_date = datetime.date(maxdate['MaxYear'],maxdate['MaxMonth'],maxdate['MaxDay'])
Without having used spark/a stack trace it seems like this is a type error. spark.sql is returning a Column object which datetime does not know how to work with. You need to get the integer values from maxdate and then pass those to datetime.date
I am running sqlite to select data between two ranges for a sales report. To select the data from between two dates I use the following statement:
SELECT * FROM test WHERE date BETWEEN "11/1/2011" AND "11/8/2011";
This statement grabs all the dates even those outside the criteria. The date format you see entered is in the same format that I get back. I'm not sure what's wrong.
SQLite requires dates to be in YYYY-MM-DD format. Since the data in your database and the string in your query isn't in that format, it is probably treating your "dates" as strings.
Change your data to that formats to use sqlite datetime formats.
YYYY-MM-DD
YYYY-MM-DD HH:MM
YYYY-MM-DD HH:MM:SS
YYYY-MM-DD HH:MM:SS.SSS
YYYY-MM-DDTHH:MM
YYYY-MM-DDTHH:MM:SS
YYYY-MM-DDTHH:MM:SS.SSS
HH:MM
HH:MM:SS
HH:MM:SS.SSS
now
DDDDDDDDDD
SELECT * FROM test WHERE date BETWEEN '2011-01-11' AND '2011-08-11'
One more way to select between dates in SQLite is to use the powerful strftime function:
SELECT * FROM test WHERE strftime('%Y-%m-%d', date) BETWEEN "11-01-2011" AND "11-08-2011"
These are equivalent according to https://sqlite.org/lang_datefunc.html:
date(...)
strftime('%Y-%m-%d', ...)
but if you want more choice, you have it.
SELECT *
FROM TableName
WHERE julianday(substr(date,7)||'-'||substr(date,4,2)||'-'||substr(date,1,2)) BETWEEN julianday('2011-01-11') AND julianday('2011-08-11')
Note that I use the format: dd/mm/yyyy.
If you use d/m/yyyy, Change in substr().
Or you can cast your string to Date format with date function. Even the date is stored as TEXT in the DB.
Like this (the most workable variant):
SELECT * FROM test WHERE date(date)
BETWEEN date('2011-01-11') AND date('2011-08-11')
SQLite does not have a concept of dates. It only knows them as text. When you do this in SQLite you're actually doing string comparisons. You can read more from the official documentation.
When two TEXT values are compared an appropriate collating sequence is used to determine the result.
Any numeric (i.e., not using words like 'May') format for dates that is padded and in order from biggest field to smallest field will work. "2021-05-07" (May 7th) comes before "2021-05-09" (May 9th). So if you use "yyyy-mm-dd" format then you'll be set. "yyyy/mm/dd" and "yyyymmdd" work just fine too. (For a better phrasing on "sortable" date formats check out RFC 3339 section 5.1.)
A reason to use "yyyy-mm-dd" format is because that's the format that SQLite's builtin date uses.
Special thanks to Jeff and vapcguy your interactivity is really encouraging.
Here is a more complex statement that is useful when the length between '/' is unknown::
SELECT * FROM tableName
WHERE julianday(
substr(substr(date, instr(date, '/')+1), instr(substr(date, instr(date, '/')+1), '/')+1)
||'-'||
case when length(
substr(date, instr(date, '/')+1, instr(substr(date, instr(date, '/')+1),'/')-1)
)=2
then
substr(date, instr(date, '/')+1, instr(substr(date, instr(date, '/')+1), '/')-1)
else
'0'||substr(date, instr(date, '/')+1, instr(substr(date, instr(date, '/')+1), '/')-1)
end
||'-'||
case when length(substr(date,1, instr(date, '/')-1 )) =2
then substr(date,1, instr(date, '/')-1 )
else
'0'||substr(date,1, instr(date, '/')-1 )
end
) BETWEEN julianday('2015-03-14') AND julianday('2015-03-16')
Put the variable in the Where Condition and parse both dates using 'BETWEEN':
SELECT * FROM emp_master
-> if you have date formate like dd/mm/yyyy simple then,
WHERE joined_date BETWEEN '01/03/2021' AND '01/09/2021';
-> and if you have date formate like yyyy/mm/dd then,
WHERE joined_date BETWEEN '2021/03/01' AND '2021/09/01';
☻♥ Done Keep Code.
Let's say you are preparing data for some report. Then the whole ordeal will look similar to this.
--add column with date in ISO 8601
ALTER TABLE sometable ADD COLUMN DateInISO8601;
--update the date from US date to ISO8601 date
UPDATE sometable
SET DateInISO8601 = substr([DateInUSformat],length([DateInUSformat])+1, -4)
|| '-' ||
substr('00' || [DateInUSformat],instr('00' || [DateInUSformat],'/'),-2)
|| '-' ||
substr('00' || rtrim(substr([DateInUSformat],instr([DateInUSformat],'/')+1,2),'/'),-2,2);
SELECT DateInISO8601
FROM sometable
WHERE DateInISO8601 BETWEEN '2022-02-02' AND '2022-02-22';
You can of course do all that on the fly, but if you have the choice -- don't. Use the ISO date by default and convert it on the way in and out to SQLite DB.
I looked at that link
It's weird because the query im doing is hit and miss.
It can't show the dates if the difference is only a few days
SQLAlchemy: how to filter date field?
model:
class UserCallsModel(db.Model):
id = db.Column(db.Integer, primary_key = True)
date = db.Column(db.String(90))
username = db.Column(db.String(90))
event_name = db.Column(db.String(90))
query:
users = UserCallsModel.query.filter(UserCallsModel.date.between("2016-1-1", "2016-1-20")).order_by(UserCallsModel.date.desc())
I've got 2 dates that fall within this range but is not getting queried?
I'm not familiar with MySQL, but I imagine it is the same as PG which I've included output below.
When you use the "between" method, you end up using the "BETWEEN" operator, like so...
SELECT * FROM my_table WHERE date BETWEEN '2016-1-1' AND '2016-1-20'
The problem is that the "between" operator does something different for dates versus strings. For example, if the value that it is testing is a string, it will see the arguments (the '2016-1-1' AND '2016-1-20' part) as strings.
mhildreth=# select '2016-1-5' between '2016-1-1' AND '2016-1-10';
?column?
----------
f
(1 row)
Meanwhile, if the value that it is testing is a date object, then it will implicitly convert the strings to date objects, essentially doing the following...
mhildreth=# select '2016-1-5'::date between '2016-1-1'::date AND '2016-1-10'::date;
?column?
----------
t
(1 row)
Thus, my guess is that you want to convert your "date" column to be a date type. If you must leave it a string, then you need to ensure that you are using a date format that also works when doing string comparison. Thus, you'll need 2016-01-01 rather than 2016-1-1.
I was under the impression that a string will actually be queried correctly as long as it was of a certain format. but nope I'm afraid it ain't so.
a better way of doing this if you have strings formatted like this:
"2016-1-5" is to simply convert the string date to a datetime.date object
python 3
import datetime
splitted_date = [int(number) for number in "2016-1-5".split("-")]
formatted_date = datetime.date(*splitted_date)