We Keep Coding

Returning department courses in databases

I think I have the right idea to do this function but I'm not sure why I get
this error when I test it. Can anyone please help me fix this?
cur.execute(q)
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The
current statement uses 1, and there are 0 supplied.
Current Attempt
def find_dept_courses(db, dept):
'''Return the courses from the given department. Use the "LIKE"
clause in your SQL query for the course name.'''
return run_query(db, '''SELECT DISTINCT Course FROM Courses WHERE
Course LIKE (? + 'dept%')''')
Desired output
find_dept_courses('exams.db', 'BIO')
# [('BIOA01H3F',), ('BIOA11H3F',), ('BIOB10H3F',), ('BIOB33H3F',),
# ('BIOB34H3F',), ('BIOB50H3F',), ('BIOC12H3F',), ('BIOC15H3F',),
# ('BIOC19H3F',), ('BIOC32H3F',), ('BIOC37H3F',), ('BIOC50H3F',),
# ('BIOC58H3F',), ('BIOC59H3F',), ('BIOC61H3F',), ('BIOC63H3F',),
# ('BIOD21H3F',), ('BIOD22H3F',), ('BIOD23H3F',), ('BIOD26H3F',),
# ('BIOD33H3F',), ('BIOD48H3F',), ('BIOD65H3F',)]
query function:
def run_query(db, q, args=None):
"""(str, str, tuple) -> list of tuple
Return the results of running query q with arguments args on
database db."""
conn = sqlite3.connect(db)
cur = conn.cursor()
# execute the query with the given args passed
# if args is None, we have only a query
if args is None:
cur.execute(q)
else:
cur.execute(q, args)
results = cur.fetchall()
cur.close()
conn.close()
return results

While using .execute you need to pass the argument as a list or tuple, here is the sample
# This is the qmark style:
cur.execute("insert into people values (?, ?)", (who, age))

Currently, you are using a placeholder but do not pass any parameters. Secondly, you are concatenating placeholder ? with a data value in LIKE expression.
Simply separate query statement and data value and leave ? by itself:
def find_dept_courses(db, dept):
sql = '''SELECT DISTINCT Course FROM Courses WHERE Course LIKE ?'''
return run_query(db, sql, args=(dept+'%',))

Remove the ? and resolve your sql syntax so dept is treated as a variable e.g. sql should evaluate to
SELECT DISTINCT Course FROM Courses WHERE
Course LIKE 'mydept%'
Note you may be susceptible to sql injection with this method and dept is from user input

How to remove single quotes around variables when doing mysql queries in Python?

I had a question pertaining to mysql as being used in Python. Basically I have a dropdown menu on a webpage using flask, that provides the parameters to change the mysql queries. Here is a code snippet of my problem.
select = request.form.get('option')
select_2 = request.form.get('option_2')
conn = mysql.connect()
cursor = conn.cursor()
query = "SELECT * FROM tbl_user WHERE %s = %s;"
cursor.execute(query, (select, select_2))
data = cursor.fetchall()
This returns no data from the query because there are single qoutes around the first variable, i.e.
Select * from tbl_user where 'user_name' = 'Adam'
versus
Select * from tbl_user where user_name = 'Adam'.
Could someone explain how to remove these single qoutes around the columns for me? When I hard code the columns I want to use, it gives me back my desired data but when I try to do it this way, it merely returns []. Any help is appreciated.

I have a working solution dealing with pymysql, which is to rewrite the escape method in class 'pymysql.connections.Connection', which obviously adds "'" arround your string. maybe you can try in a similar way, check this:
from pymysql.connections import Connection, converters
class MyConnect(Connection):
def escape(self, obj, mapping=None):
"""Escape whatever value you pass to it.
Non-standard, for internal use; do not use this in your applications.
"""
if isinstance(obj, str):
return self.escape_string(obj) # by default, it is :return "'" + self.escape_string(obj) + "'"
if isinstance(obj, (bytes, bytearray)):
ret = self._quote_bytes(obj)
if self._binary_prefix:
ret = "_binary" + ret
return ret
return converters.escape_item(obj, self.charset, mapping=mapping)
config = {'host':'', 'user':'', ...}
conn = MyConnect(**config)
cur = conn.cursor()

Connect MySQL with Python (GUI)

I'm trying to connect a MySQL database with python GUI. But this part of code returns a empty set. I'm not sure what's wrong with my code because there are no error codes... Please help!
def ButtonPressed (self):
print("Finding Parts!")
self.ProdNum = self.aVar.get()
print("Entry text was:", self.ProdNum)
self.db = mysql.connector.connect (user='ezhu', password='<password>', host='127.0.0.1', database='centricsit_prices')
self.query = ("SELECT sd, sy, price FROM css_hp WHERE prod_num = '%s'")
self.cursor = self.db.cursor()
self.cursor.execute (self.query, (self.ProdNum))
self.results = self.cursor.fetchall()
print (self.results)
self.cursor.close()

You are missing the comma:
self.cursor.execute (self.query, (self.ProdNum, ))
HERE^
It is quite important since the query parameters are expected to be passed as an iterable. Comma would make it a tuple. Without a comma, you are passing query parameters as a string, which is also an iterable, hence your query is parameterized with a first character of self.ProdNum, hence nothing matched by the select query.

Why does Psycopg2 return list of tuples in with Stored Procedure?

I have been using Psycopg2 to read stored procedures from Postgres successfully and getting a nice tuple returned, which has been easy to deal with. For example...
def authenticate(user, password):
conn = psycopg2.connect("dbname=MyDB host=localhost port=5433 user=postgres password=mypwd")
cur = conn.cursor()
retrieved_pwd = None
retrieved_userid = None
retrieved_user = None
retrieved_teamname = None
cur.execute("""
select "email", "password", "userid", "teamname"
from "RegisteredUsers"
where "email" = '%s'
""" % user)
for row in cur:
print row
The row that prints would give me ('user#gmail.com ', '84894531656894hashedpassword5161651165 ', 36, 'test ')
However, when I run the following code to read a row of fixtures with a Stored Procedure, I get (what looks to me like) an unholy mess.
def get_from_sql(userid):
conn = psycopg2.connect("dbname=MyDB host=localhost port=5433 user=postgres password=pwd")
fixture_cursor = conn.cursor()
callproc_params = [userid]
fixture_cursor.execute("select sppresentedfixtures(%s)", callproc_params)
for row in fixture_cursor:
print row
The resulting output:
('(5,"2015-08-28 21:00:00","2015-08-20 08:00:00","2015-08-25 17:00:00","Team ",,"Team ",,"Final ")',)
I have researched the cursor class and cannot understand why it outputs like this for a stored procedure. When executing within Postgres, the output is in a perfect Tuple. Using Psycopg2 adds onto the tuple and I don't understand why?
How do I change this so I get a tidy tuple? What am I not understanding about the request that I am making that gives me this result?
I have tried the callproc function and get an equally unhelpful output. Any thoughts on this would be great.

This is because you're SELECTing the result of the function directly. Your function returns a set of things, and each "thing" happens to be a tuple, so you're getting a list of stringified tuples back. What you want is this:
SELECT * FROM sppresentedfixtures(...)
But this doesn't work, because you'll get the error:
ERROR: a column definition list is required for functions returning "record"
The solution is to return a table instead:
CREATE OR REPLACE FUNCTION sppresentedfixtures(useridentity integer) RETURNS TABLE(
Fixture_No int,
Fixture_Date timestamp,
...
) AS
$BODY$
select
"Fixtures"."Fixture_No",
"Fixtures"."Fixture_Date",
...
from "Fixtures" ...
$BODY$ LANGUAGE sql

Database is locking but all statements are followed by commit?

I'm working on an IRC bot, forked from a modular bot called Skybot.
There are two other modules that make use of the sqlite3 database by default; they have both been removed and their tables dropped, so I know that the issue is somewhere in what I'm doing.
I only call 3 db.execute() statements in the whole thing and they're all immediately committed. This thing isn't getting hammered with queries either, but the lock remains.
Relevant code:
def db_init(db):
db.execute("create table if not exists searches"
"(search_string UNIQUE PRIMARY KEY,link)")
db.commit()
return db
def get_link(db, inp):
row = db.execute("select link from searches where"
" search_string=lower(?) limit 1",
(inp.lower(),)).fetchone()
db.commit()
return row
def store_link(db, stub, search):
db.execute("insert into searches (search_string, link) VALUES (?, ?)", (search.lower(), stub))
db.commit()
return stub
If the script only has to touch db_init() and get_link() it breezes through, but if it needs to call store_link() while the database is unlocked it will do the insert, but doesn't seem to be committing it in a way that future calls to get_link() can read it until the bot restarts.
The bot's db.py:
import os
import sqlite3
def get_db_connection(conn, name=''):
"returns an sqlite3 connection to a persistent database"
if not name:
name = '%s.%s.db' % (conn.nick, conn.server)
filename = os.path.join(bot.persist_dir, name)
return sqlite3.connect(filename, isolation_level=None)
bot.get_db_connection = get_db_connection
I did adjust the isolation_level myself, that was originally timeout=10. I am fairly stumped.
EDIT: The usages of get_db_connection():
main.py (main loop):
def run(func, input):
args = func._args
if 'inp' not in input:
input.inp = input.paraml
if args:
if 'db' in args and 'db' not in input:
input.db = get_db_connection(input.conn)
if 'input' in args:
input.input = input
if 0 in args:
out = func(input.inp, **input)
else:
kw = dict((key, input[key]) for key in args if key in input)
out = func(input.inp, **kw)
else:
out = func(input.inp)
if out is not None:
input.reply(unicode(out))
...
def start(self):
uses_db = 'db' in self.func._args
db_conns = {}
while True:
input = self.input_queue.get()
if input == StopIteration:
break
if uses_db:
db = db_conns.get(input.conn)
if db is None:
db = bot.get_db_connection(input.conn)
db_conns[input.conn] = db
input.db = db
try:
run(self.func, input)
except:
traceback.print_exc()

Send conn in your functions, along with db, as mentioned. If you wrote the code yourself, you'll know where the database actually is. Conventionally you would do something like:
db = sqlite3.connect('database.db')
conn = db.cursor()
Then for general usage:
db.execute("...")
conn.commit()
Hence, in your case:
def db_init(conn,db):
db.execute("create table if not exists searches"
"(search_string UNIQUE PRIMARY KEY,link)")
conn.commit()
return db
def get_link(conn,db, inp):
row = db.execute("select link from searches where"
" search_string=lower(?) limit 1",
(inp.lower(),)).fetchone()
conn.commit()
return row
def store_link(conn,db, stub, search):
db.execute("insert into searches (search_string, link) VALUES (?, ?)", (search.lower(), stub))
conn.commit()
return stub

On the basis that you have set the isolation_level to automatic updates:
sqlite3.connect(filename, isolation_level=None)
There is no need whatsoever for the commit statements in your code
Edit:
Wrap your execute statements in try statements, so that you at least have a chance of finding out what is going on i.e.
import sqlite3
def get_db(name=""):
if not name:
name = "db1.db"
return sqlite3.connect(name, isolation_level=None)
connection = get_db()
cur = connection.cursor()
try:
cur.execute("create table if not exists searches"
"(search_string UNIQUE PRIMARY KEY,link)")
except sqlite3.Error as e:
print 'Searches create Error '+str(e)
try:
cur.execute("insert into searches (search_string, link) VALUES (?, ?)", ("my search", "other"))
except sqlite3.Error as e:
print 'Searches insert Error '+str(e)
cur.execute("select link from searches where search_string=? limit 1", ["my search"])
s_data = cur.fetchone()
print 'Result:', s_data