I'm trying to send today variable into SQL but it is not working.
import datetime from date
today = date.today()
stmt = "select agent_email from customer_interaction_fact where to_date(DT) >= + today + ORDER BY CONVERSATION_CREATED_TIME DESC"
You don't have to compute today's date in Python. Just use the PostgreSQL function CURRENT_DATE:
stmt = "SELECT ... WHERE TO_DATE(DT) >= CURRENT_DATE ..."
What database engine you're using? You'd need to convert the python datetime object into string with format accepted by the database.
# In case YYYY-MM-DD
today_str = str(today)
stmt = f"""select agent_email
from customer_interaction_fact
where to_date(DT) >= datetime({today}, "YYYY-MM-DD")
order by CONVERSATION_CREATED_TIME DESC"""
Another solution, assuming the client (your program) is in the same timezone as the database engine, you could use your database engine datetime.now function. In SQLite for instance datetime('now')
try like below
from datetime import date
today = date.today()
stmt = "select agent_email,aht_in_secs,queueid,EFFORTSCORE from facts.public.customer_interaction_fact where agent_email <> 'Bot' and aht_in_secs is not NULL and to_date(DT) >=" + today + "ORDER BY CONVERSATION_CREATED_TIME DESC"
Related
Below is a sample of DB table
date id name
01.02.11 4 aaaa
21.05.19 5 aaaa
31.12.12 5 aaaa
01.05.15 6 aaaa
In order to query data in the right way (avoiding duplicates), while querying I have to set a 'reporting date' which is the first month day.
The below code gives me the requested results but only for one month.
sql = 'select * from db where date = '01.03.20''
def oracle(user, pwd, dsn, sql, columns):
# Connection to databases
con = cx_Oracle.connect(user=user, password=pwd, dsn=dsn, encoding="UTF-8")
con.outputtypehandler = OutputHandler
# Cursor allows Python code to execute PostgreSQL command in a database session
cur = con.cursor()
# Check Connection
print('Connected')
# Create DF
df = pd.DataFrame(cur.execute(sql).fetchall(), columns= columns, dtype='object')[:]
print('Shape:', df.shape)
return df
Question: How can I query Data using CX_Oracle with different reporting date without doing it manually?
There are multiple way to solve this issue directly using SQL.
However, the expected solution should use 'a for loop'.
I was thinking about changing the reporting date with
for i in [str(i).zfill(2) for i in range(1,13)]:
for j in [str(j).zfill(2) for j in range(0,21)]
sql = f'select * from db where date = '01.{i}.{j}''
For eg: date = 01.01.19
The idea is to query data for this date --> store it within DF
Go to Next month 01.02.19 --> Store it in DF
And so on until reached range 21 or reached last current month (latest date)
If someone has any idea to query data using a loop with cx_Oracle and Pandas for different date thanks for helping!
How about something like this
from datetime import date, datetime, timedelta
import calendar
# Choose Start Month
start_month = date(2019, 1, 1)
# Get Current Month
current_month = date(datetime.today().year, datetime.today().month, 1)
# Create list to collect all successfully run queries
executed_sql_queries = []
# Create list for failed queries
failed_queries = []
# Create list to collect dfs
dfs = []
while start_month <= current_month:
query_date = start_month.strftime('%d.%m.%y')
sql = f"""select * from db where date = '{query_date}' """
try:
df = oracle(user, pwd, dsn, sql=sql, columns)
except sql_error as e:
print(e)
failed_queries.append(sql)
pass # move onto the next query or you can try re-running the query
else:
executed_sql_queries.append(sql)
dfs.append(df)
finally:
# Add one Month to the date for each run
days_in_month = calendar.monthrange(start_month.year, start_month.month)[1]
start_month = start_month + timedelta(days=days_in_month)
all_dfs = pd.concat(dfs)
executed_sql_queries:
["select * from db where date = '01.01.19' ",
"select * from db where date = '01.02.19' ",
"select * from db where date = '01.03.19' ",
"select * from db where date = '01.04.19' ",
"select * from db where date = '01.05.19' ",
"select * from db where date = '01.06.19' ",
"select * from db where date = '01.07.19' ",
"select * from db where date = '01.08.19' ",
"select * from db where date = '01.09.19' ",
"select * from db where date = '01.10.19' ",
"select * from db where date = '01.11.19' ",
"select * from db where date = '01.12.19' ",
"select * from db where date = '01.01.20' ",
"select * from db where date = '01.02.20' ",
"select * from db where date = '01.03.20' ",
"select * from db where date = '01.04.20' "]
When I run the following statement to query influx I get the error :
InfluxDBClientError: invalid timestamp string
date_from = '2019-12-02 T00:00:00Z'
date_to = '2019-12-02T01:00:00Z'
"""SELECT "value" FROM "location/PRESSURE_SENSOR_1" WHERE time >= """ + '\'' + date_from + '\'' + """ and time <= '2019-12-02T01:00:00Z' """
If I run the following it works fine :
client.query("""SELECT "value" FROM "location/PRESSURE_SENSOR_1" where time >= '2019-10-02 00:00:00' and time < '2019-10-03 00:00:00'"""))
The aim is to build the statement where i can insert variables but i dont seam to be able to do it. Any help appreciated?
Thanks
You should utilize bind_params in the api to accomplish this task.
query = "SELECT value FROM 'location/PRESSURE_SENSOR_1' where time >= $start_time and time < $end_time"
bind_params = {'end_time': '2019-10-03 00:00:00', 'start_time': '2019-10-02 00:00:00'}
client.query(query, bind_params=bind_params))
Find more information here:
Influx Client Documenation
I'm trying to format a MySQL statement with today's data, as well as 7 days back. I'm pretty sure the date is in the correct format, so I don't think that's the issue.
The error report is:
Warning: (1292, "Incorrect datetime value: '{} 16:00:00' for column 'run_start_date' at row 1")
result = self._query(query)
Traceback (most recent call last):
AttributeError: 'int' object has no attribute 'format'
e.g.
today = DT.date.today()
week_ago = today - DT.timedelta(days=7)
print(today.strftime('%Y-%m-%d'))
print(week_ago.strftime('%Y-%m-%d'))
cursor.execute(SELECT * FROM db WHERE run_start_date BETWEEN '{} 16:00:00' AND '{} 16:00:00'format(week_ago, today)
Using prepared statements (safe way):
qry = """
SELECT *
FROM db
WHERE run_start_date BETWEEN '%s 16:00:00' AND '%s 16:00:00'
""" #
today = DT.date.today()
week_ago = today - DT.timedelta(days=7)
today = str(today.strftime('%Y-%m-%d')) # Convert to string
week_ago = str(week_ago.strftime('%Y-%m-%d')) # Convert to string
cursor.execute(qry, [today, week_ago])
Using .format() which leaves you at risk for sql injections (if you pass user input to .format() e.g.)
qry = """
SELECT *
FROM db
WHERE run_start_date BETWEEN '{today} 16:00:00' AND '{week_ago} 16:00:00'
""" # Use named placeholders, nicer to read, prevents you having to repeat variables multiple time when calling .format()
today = DT.date.today()
week_ago = today - DT.timedelta(days=7)
today = str(today.strftime('%Y-%m-%d')) # Convert to string
week_ago = str(week_ago.strftime('%Y-%m-%d')) # Convert to string
qry = qry.format(today=today, week_ago=week_ago)
cursor.execute(qry)
I want to retrieve time in the format of "HH:MM" from datetime.timedelta object. I have a table stored in MySQL database. It has time column, which has stored the time in the format,
00:00:00
I have connected to MySQL server using PyMySQL module,
conn = pymysql.connect("localhost", "root", "cloudera", "streaming")
cursor = conn.cursor()
sql = "select * from table1 limit 5;"
cursor.execute(sql)
row = cursor.fetchone()
row[1]
Output is,
datetime.timedelta(0)
I have been through this post Python: How do I get time from a datetime.timedelta object?. But the difference from that question and mine is in output result. For that post, output is
datetime.timedelta(0, 64800)
And for me it is just,
datetime.timedelta(0)
I just don't get it why the output for me is that way. Can anyone please help me in retrieving time. Thanks in advance.
When you type row[1] python prints the repr of the variable - in this case repr of a timedelta is "datetime.timedetla(days, seconds)".
You can stringify it using str but that would give you HH:MM:SS
str(row[1])
-> "23:00:00"
To get HH:MM you can do the following:
(datetime.min + row[1]).strftime("%H:%M")
-> "23:00"
So your code should look like this:
conn = pymysql.connect("localhost", "root", "cloudera", "streaming")
cursor = conn.cursor()
sql = "select * from table1 limit 5;"
cursor.execute(sql)
row = cursor.fetchone()
timestr = (datetime.min + row[1]).strftime("%H:%M")
print(timestr)
i create a table with SQLite Date Browse App ...
when i want retrieve datetime value from timestamp column , SQLite return unicod type ...
this is my insert code :
def Insert(self,mode,path,vname,stime,ftime):
con = sqlite3.connect(PATH_DataBase) # #UndefinedVariable
con.execute('INSERT INTO SendList VALUES(?,?,?,?,?)',(mode,path,vname,stime,ftime))
con.commit()
con.close()
dt1 = datetime.datetime(2013,01,01,01,01,01,0)
dt2 = datetime.datetime(2015,01,01,01,01,01,0)
c = 0
for f in os.listdir('/home/abbas/test/'):
c += 1
slist.Insert(common.MODE_Bluetooth_JAVA, '/home/abbas/test/'+f,'flower'+str(c) , dt1, dt2)
and now this is my table :
but when i want compare starttime with datetime.now() python give me error :
TypeError: can't compare datetime.datetime to unicode
"SQLite does not have a storage class set aside for storing dates and/or times." Reference: https://www.sqlite.org/datatype3.html
Python's sqlite3 module offers "default adapters for the date and datetime types in the datetime module." Reference: https://docs.python.org/2/library/sqlite3.html#default-adapters-and-converters
The only catch is that you must be sure to define the columns appropriately. Example DDL:
import sqlite3
con = sqlite3.connect(PATH_DataBase, detect_types=sqlite3.PARSE_DECLTYPES)
con.execute('''create table if not exists SendList (
cid primary key,
mode text,
path text,
vname text,
starttime timestamp,
endtime timestamp);''')
con.commit()
con.close()
Any subsequent connections to insert or select data must pass sqlite3.PARSE_DECLTYPES as the value for the keyword argument (aka kwarg) detect_types. Example:
import datetime as dt
con = sqlite3.connect(PATH_DataBase, detect_types=sqlite3.PARSE_DECLTYPES)
cur = con.cursor()
cur.execute('''select
*
from
SendList
where
starttime between ? and ?
limit 10;''',
(dt.datetime(2013,1,1,0,0,0), dt.datetime(2014,12,31,23,59,59)))
results = cur.fetchall()