I would not call myself a newbie, but I am not terribly conversant with programming. Any help would be appreciated. I have this project that is almost done. Figured out lots of stuff, but this issue has me at a loss.
Is there a simple way to insert an acceptable date value in a postgresql query from:
start_date = raw_input('Start date: ')
end_date = raw_input('End date: ')
I want the variables above to work in the following.
WHERE (gltx.post_date > start_date AND gltx.post_date < end_date )
'YYYY-MM-DD' format works in the SELECT Query of the postgresql database through python triple quoted cursor.execute.
The postgresql column(post.date) is date format.
here is the header for the python script.
#!/usr/bin/python
import psycopg2 as dbapi2
import psycopg2.extras
import sys
import csv
For now I have been altering the query for different periods of time.
Also is there an easy way format the date returned as YYYYMMDD. Perhaps a filter that replaced dashes or hyphens with nothing. I could use that for phone numbers also.
If you are going to execute this SELECT inside a Python script, you should not be placing strings straight into your database query - else you run the risk of SQL injections. See the psycopg2 docs - the problem with query parameters.
Instead you need to use placeholders and place all your string arguments into an iterable (usually a tuple) which is passed as the second argument to cursor.execute(). Again see the docs -passing parameters to sql queries.
So you would create a cursor object, and call the execute() method passing the query string as the first argument and a tuple containing the two dates as the second. Eg
query = "SELECT to_char(gltx.post_date, 'YYYYMMDD') FROM gltx WHERE (gltx.post_date > %s AND gltx.post_date < %s)"
args = (start_date, end_date)
cursor.execute(query, args)
To format the date in Python space, you can use the strftime() method on a date object. You should probably be working with datetime objects not strings anyway, if you want to do anything more than print the output.
You also probably want to validate that the date entered into the raw_input() is a valid date too.
Use the cursor.execute method's parameter substitution
import psycopg2
query = """
select to_char(gltx.post_date, 'YYYYMMDD') as post_date
from gltx
where gltx.post_date > %s AND gltx.post_date < %s
;"""
start_date = '2014-02-17'
end_date = '2014-03-04'
conn = psycopg2.connect("dbname=cpn")
cur = conn.cursor()
cur.execute(query, (start_date, end_date))
rs = cur.fetchall()
conn.close()
print rs
Related
UPDATE! I am still looking for some way that to indicate to the user to make an update to these dates but I went with this method:
I declared a variable in my SQL file so it looks like:
declare #date date = DATE
Select * from table WHERE date = #date
and then in my python it looks like:
DATE = date(2022, 10, 31)
def sql_data(sql_path, date):
with open(sql_path) as sql_file:
sql = sql_file.read()
sql = sql.replace("DATE", "'{}'".format(date.strftime('%Y-%m-%d')))
df = sql_data('sql_path', DATE)
df.to_csv('csv path', index=False)
I have a Python script that will execute my SQL script and create a csv file. I need to run this csv file on a monthly basis -- I am trying to do two things:
Before my Python code executes and runs the SQL to create a csv file, I need to replace the date in my SQL statement first and update it to whatever month I will run this report.
Force the user who is running the Python to enter in a date that will replace the date in my SQL statement. I want a fail-safe that will not let the user move on with the code until the date is updated.
The SQL statement looks like:
select * from table as x
where x.date = '10/30/2022'
I need to replace the string 'MM/DD/YYY' to whatever date I want in the format of 'MM/DD/YYYY'. One issue is that the x.date value won't be the same since this SQL statement will be run on a monthly basis. I need the value of x.date to be based off what the user inputs.
So far I have this (ignore that I didn't establish a connection -- I just took it out of this description)
new_date = input("Enter in the reporting date here (format: 'MM/DD/YYYY')")
with open('myreport.sql', 'r') as sqlfile:
str.replace(' .date = "XXXXX", 'new_date')
I'm unsure how to replace the string within my sql file since the date value is not necessarily consistent every time i run this report.
The SQL statement looks like:
select * from table as x
where x.date = '10/31/2022' (date isn't necessarily 10/31/2022)
You still have not posted any code requested, but I still give it a try.
What you want is a parameterised query
cursor.execute("select * from table as x where x.date = (?)", (user_input))
This way you can define user_input and use it as part of the query.
You could also simply format a string:
query = f"select * from table as x where x.date = {user_input}"
But this is problematic as the user could inject unwanted code into your query.
I have a few queries in my python script which also use variables as parameters to retrieve data.
'''SELECT * FROM order_projections_daily WHERE days >= ''' + each + '''AND days < ''' + next_monday
How can I store queries like this in a separate file and call it directly from there rather than cramming them in the code?
I have tried storing the queries in a file and calling them as a string but it doesn't work with variables. it works with:
'''SELECT * FROM order_projections_daily'''
This is a very simple query, I am using much more complicated queries in the real case.
Use parameterised strings—
'''SELECT * FROM order_projections_daily WHERE days >= %(start)s AND days < %(end)s'''
Later, when executing the query, build a params dict like this:
params = {'start': ..., 'end': ...}
These params should then be passed to the DBC driver function which will take care of inserting the parameters:
cursor.execute(query, params)
Note: Do not inject format strings into your query, you're liable to SQL injection that way.
Use python string formatting.
In your separate file save query as below e.g:
query = "select * from my_persons where name = {name} and age = {age}"
In python file, format the query this way:
sql = query.format(name="jack", age=27)
You can save all your queries in a separate python file as string values and import all of them into your code anywhere. In my eg I am assuming query is saved in a separate python file.
Formatting your query
query='''SELECT * FROM order_projections_daily WHERE days >={each} AND days < {next_monday}
Format it as:
sql = query.format(each=each, next_monday=next_monday)
It's a good practice to use format method or % or maybe even join rather than using string concatenation as string concatenation creates intermediate string objects.
Ok so formatting is a bad idea then, have a look at this. Name binding in oracle https://stackoverflow.com/a/33882805/2196290
#Delete suspense window
class dWindow(QtGui.QMainWindow, Ui_dWindow):
def __init__(self, parent = None):
QtGui.QMainWindow.__init__(self, parent)
self.setupUi(self)
for row in cursor.execute("SELECT FIRSTNAME FROM Staff"):
self.comboUser.addItems(row)
con.close()
self.btnDeleteSuspense.clicked.connect(self.btnDeleteSuspense_Clicked)
def btnDeleteSuspense_Clicked(self):
user = self.comboUser.currentText() #finds selected user
date = self.dateEdit.date().toString("M/d/yyyy")
numrecord = cursor.execute() ??
Here is a sample DB and program file to further help me explain
I have created variables to hold the selection of the combobox and the dateEdit box.
The next step (the one I'm struggling with) is to then use those variables in an SQL query that will first find the count of rows with the selected user name and having a date <= than the selected date. That will populate the numrecord variable so that I can display a "This will delete 'x' rows, are you sure?"
If the user selects yes then I will then use the variable in a delete query to delete the selected rows.
I believe if I can figure out how to use the variables I have in a SQL query then the DELETE query should be rather simple.
An example of a possible DELETE query to show what I'm trying to do
cursor.execute("DELETE TO, count(*) FROM Suspense where TO = [user] and DATE = [date]")
I know that is wrong but maybe it will help clarify.
I hope I have explained my question fully and I appreciate any help provided.
Edit: Thanks so much!!
Just before I saw that you had posted this I figured it out.
What I came up with was the following:
qdate = self.dateTimeEdit.dateTime().toPyDateTime() #grabs the raw datetime from the QDateTimeEdit object and converts to python datetime
query = "SELECT DATE FROM Suspense WHERE DATE >= ?" #creates the query using ? as a placeholder for variable
cursor.execute(query, (qdate,)) #executes the query and passes qdate as a tuple to the placeholder
With this knowledge I can recreate my queries to include both variables.
As mentioned in a comment to another answer, you should be using a proper parameterized query, for example:
# assumes that autocommit=False (the default)
crsr = conn.cursor()
sql = "DELETE FROM [Suspense] WHERE [TO]=? AND [DATE]<=?"
user = self.comboUser.currentText() # as before
date = self.dateEdit.date() # Note: no .toString(...) required
params = (user, date)
crsr.execute(sql, params)
msg = "About to delete {} row(s). Proceed?".format(crsr.rowcount)
if my_confirmation_dialog(msg):
conn.commit()
else:
conn.rollback()
What I came up with was the following:
qdate = self.dateTimeEdit.dateTime().toPyDateTime() #grabs the raw datetime from the QDateTimeEdit object and converts to python datetime
query = "SELECT DATE FROM Suspense WHERE DATE >= ?" #creates the query using ? as a placeholder for variable
cursor.execute(query, (qdate,)) #executes the query and passes qdate as a tuple to the plac
With this knowledge I can now add both variables to the query as needed.
Thanks everyone for their help, especially Gord Thompson!
You use the DELETE sql command.
This assumes your DATE field is actually a date field and not a string field.
user = self.comboUser.currentText()
date = self.dateEdit.date().toString("yyyy-MM-dd")
cmd = "DELETE FROM Suspense WHERE TO = '{}' AND DATE >= '{}'".format(user, date)
cursor.execute(cmd)
Also, you may want to look into using an ORM framework (sqlalchemy is probably the most popular, but there are others). It's best to avoid manually constructing sql queries if possible.
I am using Python 3.4 and mysql.connector. I would like to to enter something like this in an insert statement:
st = 'insert into tbl(rate,inputDt) values(%s,%s)'
data = (rate, NOW())
cur.execute(st,data)
Where NOW() is the mysql function for current datetime
I can get the equivalent value by constructing a variable in Python:
today = date.today()
But how can I pass the the mysql function as above?
Not familiar with this, but I think you can pass the MySQL function as a string.
st = 'insert into tbl(rate,inputDt) values(%s,NOW())'
data = (rate)
cur.execute(st,data)
What is the syntax for inserting a SQL datetime data type into a row?
The following in Python gives a NULL value for the timestamp variable only.
timestamp = datetime.datetime.today()
print timestamp
query = "INSERT INTO table1 (name, class, time_update) VALUES('ONE','TWO',#timestamp)"
cursor.execute(query)
I'm sure it depends on which database backend you're using, but in SQLite for example, you need to send your parameter as part of the query (that's what a parameterized statement is all about):
timestamp = datetime.datetime.today()
print timestamp
query = "INSERT INTO table1 (name, class, time_update) VALUES('ONE','TWO',?)"
cursor.execute(query, (timestamp,))
It really depends on the database. For MySQL, according to this, you can specify a timestamp/datetime in several formats, mostly based on ISO8601: 'YYYY-MM-DD', 'YY-MM-DD', 'YYYYMMDD' and 'YYMMDD' (without delimiters) or if you want greater precision 'YYYY-MM-DD HH:MM:SS' or 'YY-MM-DD HH:MM:SS' ('YYYYMMDDHHMMSS' or 'YYMMDDHHMMSS').
As far as querying goes you should probably parameterise (it's safer and a very good habit) by specifying a placeholder in the query string. For MySQL you could do:
query = "INSERT INTO table1 (name, class, time_update) VALUES('ONE','TWO',%s)"
cursor.execute(query, (timestamp,))
but the syntax for placeholders varies (depending on the db interface/driver) — see the documentation for your DB/Python interface.