I have a database, made by sqlite3:
CREATE TABLE test_table (
username TEXT,
langFrom TEXT,
langTo TEXT,
mode INT,
PRIMARY KEY (
username
));
Now I want to get a data from it by "SELECT..WHERE" command. For that, i'm making query:
def getData(self, **kwargs):
s = """SELECT * FROM test_table WHERE {} = {}"""\
.format(tuple(kwargs.keys()), tuple(kwargs.values()))
I use this method like that:
test_DB.getData(username = 'classtest', langFrom = 'ru')
And now I have query, that looks like that:
SELECT * FROM test_table WHERE ('username', 'langFrom') = ('classtest', 'ru')
But I need it like that:
SELECT * FROM test_table WHERE (username, langFrom) = ('classtest', 'ru')
So how should I format my string to do that?
First point, your query syntax is invalid, the proper SQL query would be:
SELECT * FROM test_table WHERE username='classtest' and langFrom='ru';
Second point : NEVER use string formattings for SQL queries - this is tricky as you already found out, but more over it opens your code to SQL injection attacks.
The proper way is to use your db-api connector's placeholders ('?' for sqlite3) in the query and pass the values as second argument:
q = "SELECT * FROM test_table WHERE username=? and langFrom=?"
cursor = connection.cursor()
cursor.execute(s, ("classtest", "ru"))
return cursor.fetchall()
In you case since you want to dynamically build the query:
def getData(self, **kwargs):
q = " AND ".join("{}=?".format(k) for k in kwargs)
s = "SELECT * FROM test_table WHERE {}".format(q)
c = self.connection.cursor()
c.execute(s, kwargs.values())
return c.fetchall()
but you might be better using a lightweight ORM like peewee instead of trying to reinvent the suqre wheel...
This should work
def getData(self, **kwargs):
s = """SELECT * FROM test_table WHERE ({}) = {};"""\
.format(', '.join(kwargs.keys()), tuple(kwargs.values()))
Related
I saved my data in databse where I created two columns with master_user and master_password.I inserted a value in my database. But somehow I am unable to find that master_user with my current code. error- sqlite3.OperationalError: no such column: animesh7370
def login(self):
conn = sqlite3.connect("master.db")
cur = conn.cursor()
#conn.execute("CREATE TABLE master_database (master_users TEXT NOT #NULL,master_password
#TEXT NOT NULL)")
#cur.execute("INSERT INTO master_database (master_users,master_password)
#VALUES('animesh7370','A#singh7')")
user = self.root.ids.user.text
password = self.root.ids.password.text
print(type(password))
cur.execute(f"SELECT * FROM master_database WHERE master_user = {user}")
#cur.execute("SELECT * FROM master_database ")
c=cur.fetchone()
for items in c:
print(items)
conn.commit()
conn.close()
Naming problem. You forgot the 's' to master_user.
cur.execute(f"SELECT * FROM master_database WHERE master_users = {user}")
HERE --^
cur.execute("SELECT * FROM master_database WHERE master_users =?" ,(user,))
This is because your resulting SQL looks like this (assuming that user is 'animesh7370'):
SELECT * FROM master_database WHERE master_user = animesh7370
Better use command parameters
select_stmt = "SELECT * FROM master_database WHERE master_users = %s"
cur.execute(select_stmt, (user,))
Note that command parameters are not simply inserted as a string concatenation but are passed to the query with the appropriate data type.
See: Passing parameters to SQL queries
You declared the column as master_users but referred to it as master_user in the select statement. It is usual to use column names in singular.
I am writing a function that will retrieve data from sqlite table based on the parameters user provide. This is the function so far
def database_retrieve(db_file, id):
try:
conn = sqlite3.connect(db_file)
with conn:
sql_command = "SELECT * FROM my_table WHERE id = "+id
cur = conn.cursor()
cur.execute(sql_command)
result = cur.fetchall()
return result
except Exception as e:
print(e)
db_file = 'testdb.db'
print(database_retrieve(db_file, 'subject1'))
This gives me the following error
no such column: subject1
None
When I add subject1, which is an entry under the id column in my_table, directly to the sql command like this
sql_command = "SELECT * FROM my_table WHERE id = 'subject1'"
it works fine and prints all the data.
I am new to sqlite3. Please help. Thanks in advance
These are the links I used to come this far
Python sqlite3 string variable in execute
https://www.dummies.com/programming/databases/how-to-retrieve-data-from-specific-rows-in-mysql-databases/
When you do this
sql_command = "SELECT * FROM my_table WHERE id = "+id
The value of sql_command is
"SELECT * FROM my_table WHERE id = subject1"
As you can see, subject1 is not in quotes. sqlite thinks it is a column, that's why you see that error.
Instead, do this
sql_command = "SELECT * FROM my_table WHERE id = ?"
cur.execute(sql_command, [id])
? acts as a placeholder for the variable id.
The official sqlite3 documentation mentions few others methods
https://docs.python.org/2/library/sqlite3.html
The sql_command string being generated should be something like this (Formatted string):
sql_command = "SELECT * FROM my_table WHERE id = %s AND name = %s" % (212212, 'shashank')
I want to fetch all rows from MySQL table with
query = "SELECT * FROM %s WHERE last_name=%s"
cursor.execute(query, ("employees","Smith"))
but I'm getting
You have an error in your SQL syntax. When I try
query = "SELECT * FROM employees WHERE last_name=%s"
cursor.execute(query, ("Smith",))
all is fine.
Documentation says
cursor.execute(operation, params=None, multi=False)
The parameters found in the tuple or dictionary params are bound to the variables in the operation.link on docs
The first will generate an SQL like this:
SELECT * FROM 'employees' WHERE last_name='smith'
The parameters are SQL quoted.
If you really need to have a table name as param, you must proceed in 2 steps:
table_name = 'employees'
query_tpl = "SELECT * FROM {} WHERE last_name=%s"
query = query_tpl.format(table_name)
cursor.execute(query, ("Smith",))
you need to add the quote symbol. So the query will be like
SELECT * FROM employees WHERE last_name='Smith'
Change both your query to
query = "SELECT * FROM %s WHERE last_name='%s'"
query = "SELECT * FROM employees WHERE last_name='%s'"
You can't use a parameter for the table name in the execute call.
But you can use Python string interpolation for that:
query = "SELECT * FROM %s WHERE last_name=%s" %("employees","Smith")
cursor.execute(query)
You can't use a table name as a parameter. you are generating invalid sql with your code that is putting quotes around each string. the table name cannot have quotes around it.
sql you are generating
select * from 'employees' where last_name = 'Smith'
What sql you want
select * from employees where last_name = 'Smith'
you would have to format the string first like the example below.
query = "SELECT * from {} wherre last_name ='{}'"
cursor.execute(query.format("employees","Smith"))
using code like this does open up the possibility of SQL injection. so please bear that in mind.
query="SELECT * FROM %s WHERE name=%s",(employees,smith)
cursor.execute(query)
rows = cursor.fetchall()
Try this one. Hopefully it works for you.
Have a sqlite db that I've created in python that has a DATETIME field:
import sqlite3
con = sqlite3.connect('some.db',detect_types=sqlite3.PARSE_DECLTYPES)
with con:
cur = con.cursor()
cur.execute("CREATE TABLE Table(...Date DATETIME...)")
...
Date = datetime.datetime(<a format that resolves to the correct datetime object>)
...
altogether = (..., Date, ...)
cur.execute("INSERT INTO Table VALUES(...?...)", altogether)
con.commit()
This populates correctly. I later want to be able to query this DB by datetime, and have a function to manage my queries generally:
def query_db(path, query, args=(), one=False):
connection = sqlite3.connect(path)
cur = connection.execute(query, args)
rv = [dict((cur.description[idx][0], value)
for idx, value in enumerate(row)) for row in cur.fetchall()]
return (rv[0] if rv else None) if one else rv
LOCAL_FOLDER = os.getcwd()
samplequery = "SELECT * FROM Table"
dbFile = os.path.join(LOCAL_FOLDER, "some.db")
result = query_db(dbFile, samplequery)
The above would successfully produce a resultthat gave me everything in the Table.
However, how do I structure a query that would, for instance, give me all entries in the Table table of some.db that have a Date within the past 60 days?
You can do a query like this:
SELECT *
FROM Table
where date >= date('now', '-60 day');
EDIT:
Based on your actual query:
select <field1>, <field2>, count(1) as num
FROM Table
where date >= date('now', '-60 day');
group by <field1>, <field2>;
SELECT DISTINCT is unnecessary when you are using GROUP BY.
With Python's DB API spec you can pass an argument of parameters to the execute() method. Part of my statement is a WHERE IN clause and I've been using a tuple to populate the IN. For example:
params = ((3, 2, 1), )
stmt = "SELECT * FROM table WHERE id IN %s"
db.execute(stmt, params)
But when I run into a situation where the parameter tuple is only a tuple of 1 item, the execute fails.
ProgrammingError: ERROR: syntax error at or near ")"
LINE 13: WHERE id IN (3,)
How can I get the tuple to work with clause properly?
Edit: If you think this answer circumvents the built-in protections against SQL-injection attack you're mistaken; look more closely.
Testing with pg8000 (a DB-API 2.0 compatible Pure-Python interface to the PostgreSQL database engine):
This is the recommended way to pass multiple parameters to an "IN" clause.
params = [3,2,1]
stmt = 'SELECT * FROM table WHERE id IN (%s)' % ','.join('%s' for i in params)
cursor.execute(stmt, params)
Full example:
>>> from pg8000 import DBAPI
>>> conn = DBAPI.connect(user="a", database="d", host="localhost", password="p")
>>> c = conn.cursor()
>>> prms = [1,2,3]
>>> stmt = 'SELECT * FROM table WHERE id IN (%s)' % ','.join('%s' for i in prms)
>>> c.execute(stmt,prms)
>>> c.fetchall()
((1, u'myitem1'), (2, u'myitem2'), (3, u'myitem3'))
The error is coming from the comma after the 3. Just leave it off for the single values and you're set.
params = ((3), ... )
stmt = "SELECT * FROM table WHERE id IN %s"
db.execute(stmt, params)
This may not be an answer to exactly the question you asked, but I think it may solve the problem you have.
Python's DB-API doesn't seem to give you a way to pass tuples as safely substituted parameters. The accepted answer from bernie is using the Python % operator for substitution, which is unsafe.
However, you may not have to pass tuples as parameters, particularly when the tuple you want is the result of another SQL query (as you indicated to Daniel). Instead, you can use SQL subqueries.
If the set of IDs you want in your IN clause is the result of SELECT id FROM other_table WHERE use=true, for example:
stmt = "SELECT * FROM table WHERE id IN (SELECT id FROM other_table WHERE use=true)"
db.execute(stmt)
And this can be parameterized (the safe way), too. If the IDs you want to select are the ones with a given parent_id:
stmt = "SELECT * FROM table WHERE id IN (SELECT id FROM other_table WHERE parent_id=%s)"
params = (parent_id,)
db.execute(stmt, params)
A solution with f-string.
params = [...]
stmt = f"SELECT * FROM table WHERE id IN ({','.join(['%s']*len(params ),)})"
db.execute(stmt, params)
If there is another param placeholder it will be like this
age = 18
params = [...]
stmt = f"SELECT * FROM table WHERE age>%s AND id IN ({','.join(['%s']*len(params ),)})"
db.execute(stmt, tuple([age] + params))
As the question said, the following will fail:
params = ((3, 2, 1), )
stmt = "SELECT * FROM table WHERE id IN %s"
db.execute(stmt, params)
Following the pg8000 docs the IN can be replaced with an ANY() to give the same result:
params = ((3, 2, 1), )
stmt = "SELECT * FROM table WHERE id = ANY(%s)"
db.execute(stmt, params)
This sends the query and parameters separately to the server, avoiding SQL injection attacks.
The accepted answer risks SQL injection; you should never ever pass user input directly to the database. Instead, generate a query with the correct number of placeholders, then let pg8000 do the escaping:
params = [3,2,1]
# SELECT * from table where id in (%s,%s,%s)
stmt = 'SELECT * FROM table WHERE id IN ({})'.format(','.join(['%s']*len(params)))
cursor.execute(stmt, tuple(params))