We Keep Coding

How to perform an SQL query with PYODBC using a column name variable?

I have written several functions to get particular items of data like this:
def get_short_desc(self, part_num):
sql = 'SELECT impShortDescription FROM Parts '
sql += 'WHERE impPartID LIKE ?'
self.cursor.execute(sql, [part_num])
item = self.cursor.fetchone().impShortDescription
return item
I want to paramaterise the functions so I am not repeating myself (DRY):
def get_part_entry_item(self, var, part_num):
sql = 'SELECT ? FROM Parts '
sql += 'WHERE impPartID LIKE ?'
self.cursor.execute(sql, [var, part_num])
item = getattr(self.cursor.fetchone(), var)
return item
The execute statement passes the value of part_num (a string) with single quotes which is correct. When the column name variable var = 'impShortDescription' the resulting SQL is effectively SELECT 'impShortDescription'... so the item returned is
('impShortDescription', )
so getattr(self.cursor.fetchone(), var) causes the error
AttributeError: 'pyodbc.Row' object has no attribute 'impShortDescription'
How do I pass a column name without the quotes?

SQL substitution only works on values, not on table or field names. Do this. Yes, this means you must be careful not to pass user data as the field name.
def get_part_entry_item(self, var, part_num):
sql = f'SELECT {var} FROM Parts '
sql += 'WHERE impPartID LIKE ?'
self.cursor.execute(sql, (part_num,))
return self.cursor.fetchone()[var]

insert into mysql database with pymysql failing to insert

I'm trying to insert dummy data into a mysql database.
The database structure looks like:
database name: messaround
database table name: test
table structure:
id (Primary key, auto increment)
path (varchar(254))
UPDATED 2 method below, and error.
I have a method to try to insert via:
def insert_into_db(dbcursor, table, *cols, **vals):
try:
query = "INSERT INTO {} ({}) VALUES ('{}')".format(table, ",".join(cols), "'),('".join(vals))
print(query)
dbcursor.execute(query)
dbcursor.commit()
print("inserted!")
except pymysql.Error as exc:
print("error inserting...\n {}".format(exc))
connection=conn_db()
insertstmt=insert_into_db(connection, table='test', cols=['path'], vals=['test.com/test2'])
However, this is failing saying:
INSERT INTO test () VALUES ('vals'),('cols')
error inserting...
(1136, "Column count doesn't match value count at row 1")
Can you please assist?
Thank you.

If you use your code:
def insert_into_db(dbcursor, table, *cols, **vals):
query = "INSERT INTO {} ({}) VALUES ({})".format(table,",".join(cols), ",".join(vals))
print(query)
insert_into_db('cursor_here', 'table_here', 'name', 'city', name_person='diego', city_person='Sao Paulo')
Python returns:
INSERT INTO table_here (name,city) VALUES (name_person,city_person)
Now with this other:
def new_insert_into_db(dbcursor, table, *cols, **vals):
vals2 = ''
for first_part, second_part in vals.items():
vals2 += '\'' + second_part + '\','
vals2 = vals2[:-1]
query = "INSERT INTO {} ({}) VALUES ({})".format(table,",".join(cols), vals2)
print(query)
new_insert_into_db('cursor_here', 'table_here', 'name', 'city', name_person='diego', city_person='Sao Paulo')
Python will return the correct SQL:
INSERT INTO table_here (name,city) VALUES ('diego','Sao Paulo')

Generally in Python you pass a parameterized query to the DB driver. See this example in PyMySQL's documentation; it constructs the INSERT query with placeholder characters, then calls cursor.execute() passing the query, and a tuple of the actual values.
Using parameterized queries is also recommended for security purposes, as it defeats many common SQL injection attacks.

you should print the sql statement which you've generated, that makes it a lot easier to see what's wrong.
But I guess you need quotes ' around string values for your ",".join(vals) (in case there are string values.
So your code is producing
insert into test (path,) values (test.com/test2,);
but it should produce
insert into test (`path`) values ('test.com/test2');
Otherwise try https://github.com/markuman/MariaSQL/ which makes it super easy to insert data to MariaDB/MySQL using pymysql.

Change your query as below
query = "INSERT INTO {} ({}) VALUES ('{}')".format(table, ",".join(cols), "'),('".join(vals))
As you are using join, the variable is expected to be a list but not a string
table = 'test'
cols = ['path']
vals = ['test.com/test2', 'another.com/anothertest']
print(query)
"INSERT INTO test (path) VALUES ('test.com/test2'),('another.com/anothertest')"
Update:
def insert_into_db(dbconnection=None, table='', cols=None, vals=None):
mycursor = dbconnection.cursor()
if not (dbconnection and table and cols and vals):
print('Must need all values')
quit()
try:
query = "INSERT INTO {} ({}) VALUES ('{}')".format(table, ",".join(cols), "'),('".join(vals))
mycursor.execute(query)
dbconnection.commit()
print("inserted!")
except pymysql.Error as exc:
print("error inserting...\n {}".format(exc))
connection=conn_db()
insertstmt=insert_into_db(dbconnection=connection, table='test', cols=['path'], vals=['test.com/test2'])

sqlite error when calling a function in the shell

I am getting the error:
builtins.NameError: name 'sqlite3' is not defined
on the line
conn = sqlite3.connect(db)
Can anyone please help me fix this?
def run_query(db, q, args=None):
"""(str, str, tuple) -> list of tuple
Return the results of running query q with arguments args on
database db."""
conn = sqlite3.connect(db)
cur = conn.cursor()
# execute the query with the given args passed
# if args is None, we have only a query
if args is None:
cur.execute(q)
else:
cur.execute(q, args)
results = cur.fetchall()
cur.close()
conn.close()
return results
def get_course_instructors(db, course):
'''Return the Course number, sections and instructors for the given course
number.'''
return (run_query(db, '''SELECT CourseNumber, sections, instructors WHERE
course = ?''', (course)))

import sqlite3 # make sure you have this
def get_course_instructors(db, course):
'''Return the Course number, sections and instructors for the given course
number.'''
Get course code, sections and instructors from Courses table, assuming your variable names are "CourseNumber, sections, instructors" in the Courses table.
return run_query(db, '''SELECT Course, Section, Name FROM Courses WHERE course = ?''', (course,))
Make sure you have a comma after (course) in the return statement.
A test case would look like this.
get_course_instructors('exams.db', 'AFSA01H3F')
[('AFSA01H3F', 'LEC01', 'S. Rockel')]

Connect MySQL with Python (GUI)

I'm trying to connect a MySQL database with python GUI. But this part of code returns a empty set. I'm not sure what's wrong with my code because there are no error codes... Please help!
def ButtonPressed (self):
print("Finding Parts!")
self.ProdNum = self.aVar.get()
print("Entry text was:", self.ProdNum)
self.db = mysql.connector.connect (user='ezhu', password='<password>', host='127.0.0.1', database='centricsit_prices')
self.query = ("SELECT sd, sy, price FROM css_hp WHERE prod_num = '%s'")
self.cursor = self.db.cursor()
self.cursor.execute (self.query, (self.ProdNum))
self.results = self.cursor.fetchall()
print (self.results)
self.cursor.close()

You are missing the comma:
self.cursor.execute (self.query, (self.ProdNum, ))
HERE^
It is quite important since the query parameters are expected to be passed as an iterable. Comma would make it a tuple. Without a comma, you are passing query parameters as a string, which is also an iterable, hence your query is parameterized with a first character of self.ProdNum, hence nothing matched by the select query.

Why does Psycopg2 return list of tuples in with Stored Procedure?

I have been using Psycopg2 to read stored procedures from Postgres successfully and getting a nice tuple returned, which has been easy to deal with. For example...
def authenticate(user, password):
conn = psycopg2.connect("dbname=MyDB host=localhost port=5433 user=postgres password=mypwd")
cur = conn.cursor()
retrieved_pwd = None
retrieved_userid = None
retrieved_user = None
retrieved_teamname = None
cur.execute("""
select "email", "password", "userid", "teamname"
from "RegisteredUsers"
where "email" = '%s'
""" % user)
for row in cur:
print row
The row that prints would give me ('user#gmail.com ', '84894531656894hashedpassword5161651165 ', 36, 'test ')
However, when I run the following code to read a row of fixtures with a Stored Procedure, I get (what looks to me like) an unholy mess.
def get_from_sql(userid):
conn = psycopg2.connect("dbname=MyDB host=localhost port=5433 user=postgres password=pwd")
fixture_cursor = conn.cursor()
callproc_params = [userid]
fixture_cursor.execute("select sppresentedfixtures(%s)", callproc_params)
for row in fixture_cursor:
print row
The resulting output:
('(5,"2015-08-28 21:00:00","2015-08-20 08:00:00","2015-08-25 17:00:00","Team ",,"Team ",,"Final ")',)
I have researched the cursor class and cannot understand why it outputs like this for a stored procedure. When executing within Postgres, the output is in a perfect Tuple. Using Psycopg2 adds onto the tuple and I don't understand why?
How do I change this so I get a tidy tuple? What am I not understanding about the request that I am making that gives me this result?
I have tried the callproc function and get an equally unhelpful output. Any thoughts on this would be great.

This is because you're SELECTing the result of the function directly. Your function returns a set of things, and each "thing" happens to be a tuple, so you're getting a list of stringified tuples back. What you want is this:
SELECT * FROM sppresentedfixtures(...)
But this doesn't work, because you'll get the error:
ERROR: a column definition list is required for functions returning "record"
The solution is to return a table instead:
CREATE OR REPLACE FUNCTION sppresentedfixtures(useridentity integer) RETURNS TABLE(
Fixture_No int,
Fixture_Date timestamp,
...
) AS
$BODY$
select
"Fixtures"."Fixture_No",
"Fixtures"."Fixture_Date",
...
from "Fixtures" ...
$BODY$ LANGUAGE sql