I'm new to python and I want to do a similar query to this one:
_c.execute('select * from cases where bi = ? and age = ? and
shape = ? and margin = ? and density = ?',(obj['bi'],
obj['age'], obj['margin'], obj['density']))
When some of the parameters are None, for example obj['bi'] = None, the query searches for the row when bi = 'None'. But I want it to search for the row when: 'bi is NULL'
A possible solution is to verify the values of the parameters one by one in a sequence of if-elses. For example:
query = 'select * from cases where'
if obj['bi'] is None:
query += ' bi is null'
else:
query += ' bi = ' + str(obj['bi']) + ' and '
...
# do the same if-else for the other parameters
...
_c.execute(query)
But, it doesn't seem to me as the best solution.
The question is, what is the best solution to the given problem and how to avoid SQL injections.
Okay, after firing up a python REPL and playing around with it a bit, it's simpler than I thought. The Python sqlite bindings turn a Python None into a SQL NULL, not into a string 'None' like it sounded like from your question. In SQL, = doesn't match NULL values, but IS will. So...
Given a table foo looking like:
a | b
--------------
NULL | 1
Dog | 2
Doing:
c = conn.cursor()
c.execute('SELECT * FROM foo WHERE a IS ?', (None,))
print(c.fetchone())
will return the (NULL, 1) row, and
c.execute('SELECT * FROM foo WHERE a IS ?', ('Dog',))
print(c.fetchone())
will return the ('Dog', 2) row.
In other words, use IS not = in your query.
Related
I would like to have a simple subquery. One I would like to re-use in several places. Some of those would be joins, some would not.
SQL code would be like this
SELECT IF(x, y, z) as foo, table.*
FROM TABLE
WHERE condition
And then it's used in many places, joining and where'ing by foo.
Sometimes simply like this:
SELECT * FROM
(
SELECT IF(x, y, z) as foo, table.*
FROM TABLE
WHERE condition
) WHERE (foo > 100)
Sometimes more complex, like grouping, joining.
However, I find it quite hard to do in peewee.
I figured out I can do this if I use joins
query1 = table1.select(...).where(...)
query2 = table2.select(...).join(query1, on=(...))...
This would work
query1 = table1.select(...).where(...)
query2 = query1.select(...).join(table2, on=(...))...
This would also work
However, if I just select from query1, it doesn't work. Exact code that fails:
query = tables.Payments.select(fn.IF(tables.Payments.payment_status > 0, tables.Payments.payment_amount, -tables.Payments.payment_amount).alias("x")).where(tables.Payments.payment_amount > 200)
query2 = query.select().where(query.c.x < 0)
I expect query2 to just be a select from Payments where x, calculated according to condition before, is less than 0, but instead it produces bogus SQL code
SELECT FROM `payments` AS `t1` WHERE ((`t1`.`payment_amount` > 200) AND (`t2`.`x` < 0))
Which is obviously malformed and doesn't execute
How do I do this? Is this even possible in peewee?
I know I could write "where()" and replicate my condition there, but that's bad practice, because it's copypasting code, and what if I want to change that condition later? Do I re-do it in 10 places?... Surely there's a proper way to do this
PS: As advised, I have altered my code but it produces malformed SQL query again.
My code:
query = tables.Payments.select(fn.IF(tables.Payments.payment_status > 0, tables.Payments.payment_amount, -tables.Payments.payment_amount).alias("x")).where(tables.Payments.payment_amount > 200)
query2 = query.select_from(query.c.x).where(query.c.x < 0)
Resulting query:
SELECT `t1`.`x` FROM (SELECT IF((`t2`.`payment_status` > 0), `t2`.`payment_amount`, `t2`.`payment_amount` DESC) AS `x` FROM `payments` AS `t2` WHERE (`t2`.`payment_amount` > 200)) AS `t1` WHERE (`t1`.`x` < 0)
As you see, instead of doing a minus operation, it adds DESC which is obviously not right.
How to fix this?
Here is an example of wrapping a subquery using select_from():
db = SqliteDatabase(':memory:')
class Reg(db.Model):
key = TextField()
db.create_tables([Reg])
Reg.create(key='k1')
Reg.create(key='k2')
Reg.create(key='k3')
subq = Reg.select(Reg.key.alias('foo'), Reg)
query = subq.select_from(subq.c.foo).where(subq.c.foo.in_(['k1', 'k3']))
for row in query:
print(row.foo)
# k1
# k3
Another example, this is basically what select_from() does under-the-hood:
query = Select([subq], [subq.c.foo]).bind(db)
for row in query:
print(row)
# {'foo': 'k1'}
# {'foo': 'k2'}
# {'foo': 'k3'}
For your last/most-recent edit to your issue, replace unary minus with X * -1:
query = (Payments.select(fn.IF(
Payments.payment_status > 0,
Payments.payment_amount,
Payments.payment_amount * -1).alias("x")
).where(Payments.payment_amount > 200)
query2 = query.select_from(query.c.x).where(query.c.x < 0)
I am new to python. What I am trying to achieve is to insert values from my list/tuple into my redshift table without iteration.I have around 1 million rows and 1 column. Below is the code I am using to create my list/tuple.
cursor1.execute("select domain from url limit 5;")
for record, in cursor1:
ext = tldextract.extract(record)
mylist.append(ext.domain + '.' + ext.suffix)
mytuple = tuple(mylist)
I am not sure what is best to use, tuple or list. output of print(mylist) and print(mytuple) are as follows.
List output
['friv.com', 'steep.tv', 'wordpress.com', 'fineartblogger.net',
'v56.org'] Tuple Output('friv.com', 'steep.tv', 'wordpress.com',
'fineartblogger.net', 'v56.org')
Now, below is the code I am using to insert the values into my redshift table but I am getting an error:
cursor2.execute("INSERT INTO sample(domain) VALUES (%s)", mylist) or
cursor2.execute("INSERT INTO sample(domain) VALUES (%s)", mytuple)
Error - not all arguments converted during string formatting
Any help is appreciated. If any other detail is required please let me know, I will edit my question.
UPDATE 1:
Tried using below code and getting different error.
args_str = ','.join(cur.mogrify("(%s)", x) for x in mylist)
cur.execute("INSERT INTO table VALUES " + args_str)
ERROR - INSERT has more expressions than target columns
I think you're looking for Fast Execution helpers:
mylist=[('t1',), ('t2',)]
execute_values(cursor2, "INSERT INTO sample(domain) %s", mylist, page_size=100)
what this does is it replaces the %s with 100 VALUES. I'm not sure how high you can set page_size, but that should be far more performant.
Finally found a solution. For some reason cur.mogrify was not giving me proper sql string for insert. Created my own SQl string and it works alot faster than cur.executeall()
list_size = len(mylist)
for len in range(0,list_size):
if ( len != list_size-1 ):
sql = sql + ' ('+ "'"+ mylist[len] + "'"+ ') ,'
else:
sql = sql + '('+ "'"+ mylist[len] + "'"+ ')'
cursor1.execute("INSERT into sample(domain) values " + sql)
Thanks for your help guys!
Let's suppose I have the following table :
Id (int, Primary Key) | Value (varchar)
----------------------+----------------
1 | toto
2 | foo
3 | bar
I would like to know if giving two request, the result of the first must be contained in the result of the second without executing them.
Some examples :
# Obvious example
query_1 = "SELECT * FROM example;"
query_2 = "SELECT * FROM example WHERE id = 1;"
is_sub_part_of(query_2, query_1) # True
# An example we can't know before executing the two requests
query_1 = "SELECT * FROM example WHERE id < 2;"
query_2 = "SELECT * FROM example WHERE value = 'toto' or value = 'foo';"
is_sub_part_of(query_2, query_1) # False
# An example we can know before executing the two requests
query_1 = "SELECT * FROM example WHERE id < 2 OR value = 'bar';"
query_2 = "SELECT * FROM example WHERE id < 2 AND value = 'bar';"
is_sub_part_of(query_2, query_1) # True
# An example about columns
query_1 = "SELECT * FROM example;"
query_2 = "SELECT id FROM example;"
is_sub_part_of(query_2, query_1) # True
Do you know if there's a module in Python that is able to do that, or if it's even possible to do ?
Interesting problem. I don't know of any library that will do this for you. My thoughts:
Parse the SQL, see this for example.
Define which filtering operations can be added to a query that can only result in the same or a narrower result set. "AND x" can always be added, I think, without losing the property of being a subset. "OR x" can not. Anything else you can do to the query? For example "SELECT *", vs "SELECT x", vs "SELECT x, y".
Except for that, I can only say it's an interesting idea. You might get some more input on DBA. Is this an idea you're researching or is it related to a real-world problem you are solving, like optimizing a DB query? Maybe your question could be updated with information about this, since this is not a common way to optimize queries (unless you're working on the DB engine itself, I guess).
I would like to insert records into a sqlite database with fields such that every query that specifies a value for that field does not disqualify the record.
Make Model Engine Parameter
Ford * * 1
Ford Taurus * 2
Ford Escape * 3
So a query = (database.table.Make == Ford') & (database.table.Model == 'Taurus') would return the first two records
EDIT: thanks to woot, I decided to use the following: (database.table.Make.belongs('Ford','')) & (database.table.Model.belongs('Taurus','')) which is the syntax for the IN operator in web2py
Are you looking for something like this? It won't perform well due to the ORs if you have a lot of rows.
SELECT *
FROM Cars
WHERE ( Cars.Make = 'Ford' OR Cars.Make = '*' )
AND ( Cars.Model = 'Taurus' OR Cars.Model = '*' )
Here is a SQL Fiddle example.
If you meant to use NULL, you can just replace that and replace the OR condition with OR Cars.Make IS NULL, etc.
Or to make it maybe a little less verbose:
SELECT *
FROM Cars
WHERE Cars.Make IN ('Ford','*')
AND Cars.Model IN ('Taurus','*')
But you wouldn't be able to use NULL in this case and would have to use the * token.
SQL Fiddle
Hey,
I'm trying to run the following query:
self.cursor.execute('SELECT courses.courseid, days, starttime, bldg, roomnum, '
'area, title, descrip, prereqs, endtime FROM '
'classes, courses, crosslistings, coursesprofs, profs WHERE '
'classes.courseid = courses.courseid AND '
'courses.courseid = crosslistings.courseid AND '
'courses.courseid = coursesprofs.courseid AND '
'coursesprofs.profid = profs.profid AND '
'classes.classid LIKE %s'
';',
(self.classid))
classid is an int(11) field in the db. When I set self.classid = %, it returns all the results, but as soon as I set it to say, '3454' or some other amount it returns nothing even when there is a class with that classid. Am I querying incorrectly against int fields?
Even a simpler query like
select * from classes where classes.classid = '3454'; does not work
Try:
select * from classes where classes.classid = 3454;
I resolved this on my own. Based on my db structure, I was querying the wrong fields. I was looking for values that weren't there so that's why I was always returning an empty result set. Thanks for the help on the = operator though, that was utilized.