I have a query which looks like this:
query = session.query(Item) \
.filter(Item.company_id == company_id) \
.order_by(Item.id)
It's a pretty basic query. In addition to pulling out the values for the Item, I want to append an additional value into the mix, and have it returned to me. In raw SQL, I would do this:
SELECT *, 0 as subscribed
FROM items
WHERE company_id = 34
ORDER BY id
How can I manually add that value via sqlalchemy?
You'll need to use a literal_column, which looks a bit like this:
sqlalchemy.orm.Query(Item, sqlalchemy.sql.expression.literal_column("0"))
Beware that the text argument is inserted into the query without any transformation; this may expose you to a SQL Injection vulnerability if you accept values for the text parameter from outside your application. If that's something you need, you'll want to use bindparam, which is about as easy to use; but you will have to invent a name:
sqlalchemy.orm.Query(Item, sqlalchemy.sql.expression.bindparam("zero", 0))
As mentioned in the comments of the accepted answer there's a "shorthand" for bindparam() that alleviates the need to come up with a name for the literal bindparam, literal():
Return a literal clause, bound to a bind parameter.
So one does not have to write
session.query(Item, bindparam("zero", 0).label("subscribed"))
but just
session.query(Item, literal(0).label("subscribed"))
without having to worry about quoting etc., if passing strings or such.
Related
I have some code in Python that sets a char(80) value in an sqlite DB.
The string is obtained directly from the user through a text input field and sent back to the server with a POST method in a JSON structure.
On the server side I currently pass the string to a method calling the SQL UPDATE operation.
It works, but I'm aware it is not safe at all.
I expect that the client side is unsafe anyway, so any protection is to be put on the server side. What can I do to secure the UPDATE operation agains SQL injection ?
A function that would "quote" the text so that it can't confuse the SQL parser is what I'm looking for. I expect such function exist but couldn't find it.
Edit:
Here is my current code setting the char field name label:
def setLabel( self, userId, refId, label ):
self._db.cursor().execute( """
UPDATE items SET label = ? WHERE userId IS ? AND refId IS ?""", ( label, userId, refId) )
self._db.commit()
From the documentation:
con.execute("insert into person(firstname) values (?)", ("Joe",))
This escapes "Joe", so what you want is
con.execute("insert into person(firstname) values (?)", (firstname_from_client,))
The DB-API's .execute() supports parameter substitution which will take care of escaping for you, its mentioned near the top of the docs; http://docs.python.org/library/sqlite3.html above Never do this -- insecure.
Noooo... USE BIND VARIABLES! That's what they're there for. See this
Another name for the technique is parameterized sql (I think "bind variables" may be the name used with Oracle specifically).
I have a query which looks like this:
query = session.query(Item) \
.filter(Item.company_id == company_id) \
.order_by(Item.id)
It's a pretty basic query. In addition to pulling out the values for the Item, I want to append an additional value into the mix, and have it returned to me. In raw SQL, I would do this:
SELECT *, 0 as subscribed
FROM items
WHERE company_id = 34
ORDER BY id
How can I manually add that value via sqlalchemy?
You'll need to use a literal_column, which looks a bit like this:
sqlalchemy.orm.Query(Item, sqlalchemy.sql.expression.literal_column("0"))
Beware that the text argument is inserted into the query without any transformation; this may expose you to a SQL Injection vulnerability if you accept values for the text parameter from outside your application. If that's something you need, you'll want to use bindparam, which is about as easy to use; but you will have to invent a name:
sqlalchemy.orm.Query(Item, sqlalchemy.sql.expression.bindparam("zero", 0))
As mentioned in the comments of the accepted answer there's a "shorthand" for bindparam() that alleviates the need to come up with a name for the literal bindparam, literal():
Return a literal clause, bound to a bind parameter.
So one does not have to write
session.query(Item, bindparam("zero", 0).label("subscribed"))
but just
session.query(Item, literal(0).label("subscribed"))
without having to worry about quoting etc., if passing strings or such.
I am new to this and trying to learn python. I wrote a select statement in python where I used a parameter
Select """cln.customer_uid = """[(num_cuid_number)])
TypeError: string indices must be integers
Agree with the others, this doesn't look really like Python by itself.
I will see even without seeing the rest of that code I'll guess the [(num_cuid_number)] value(s) being returned is a string, so you'll want to convert it to integer for the select statement to process.
num_cuid_number is most likely a string in your code; the string indices are the ones in the square brackets. So please first check your data variable to see what you received there. Also, I think that num_cuid_number is a string, while it should be in an integer value.
Let me give you an example for the python code to execute: (Just for the reference: I have used SQLAlchemy with flask)
#app.route('/get_data/')
def get_data():
base_sql="""
SELECT cln.customer_uid='%s' from cln
""" % (num_cuid_number)
data = db.session.execute(base_sql).fetchall()
Pretty sure you are trying to create a select statement with a "where" clause here. There are many ways to do this, for example using raw sql, the query should look similar to this:
query = "SELECT * FROM cln WHERE customer_uid = %s"
parameters = (num_cuid_number,)
separating the parameters from the query is secure. You can then take these 2 variables and execute them with your db engine like
results = db.execute(query, parameters)
This will work, however, especially in Python, it is more common to use a package like SQLAlchemy to make queries more "flexible" (in other words, without manually constructing an actual string as a query string). You can do the same thing using SQLAlchemy core functionality
query = cln.select()
query = query.where(cln.customer_uid == num_cuid_number)
results = db.execute(query)
Note: I simplified "db" in both examples, you'd actually use a cursor, session, engine or similar to execute your queries, but that wasn't your question.
Look at this answer: Case Insensitive Flask-SQLAlchemy Query
Why is it better to use SQLAlchemy's func.lower(mystring) instead of python's native mystring.lower()?
The context is important.
user = models.User.query.filter(func.lower(User.username) == func.lower("GaNyE")).first()
Here, there is no string with a lower method to call; you are composing a SQL command to execute in the database. func.lower doesn't return a lowercase string; it returns an object that represents SQL code that can be used to produce a lowercase string.
As mgilson points out, there's no need to use func.lower on actual Python values; they are constant for each row the filter will be applied against, but I'm not sure SQLAlchemy is able to detect and optimize such situations. Either of the following
user = models.User.query.filter(func.lower(User.username) == "ganye").first()
user = models.User.query.filter(func.lower(User.username) == "GaNyE".lower()).first()
would produce SQL code like lower(username) = :lower_1 instead of lower(username) = lower(:lower_1), which can make the SQL query more efficient by eliminating a call to lower for each line.
I'm trying to build a relatively complex query and would like to manipulate the where clause of the result directly, without cloning/subquerying the returned query. An example would look like:
session = sessionmaker(bind=engine)()
def generate_complex_query():
return select(
columns=[location.c.id.label('id')],
from_obj=location,
whereclause=location.c.id>50
).alias('a')
query = generate_complex_query()
# based on this query, I'd like to add additional where conditions, ideally like:
# `query.where(query.c.id<100)`
# but without subquerying the original query
# this is what I found so far, which is quite verbose and it doesn't solve the subquery problem
query = select(
columns=[query.c.id],
from_obj=query,
whereclause=query.c.id<100
)
# Another option I was considering was to map the query to a class:
# class Location(object):pass
# mapper(Location, query)
# session.query(Location).filter(Location.id<100)
# which looks more elegant, but also creates a subquery
result = session.execute(query)
for r in result:
print r
This is the generated query:
SELECT a.id
FROM (SELECT location.id AS id
FROM location
WHERE location.id > %(id_1)s) AS a
WHERE a.id < %(id_2)s
I would like to obtain:
SELECT location.id AS id
FROM location
WHERE id > %(id_1)s and
id < %(id_2)s
Is there any way to achieve this? The reason for this is that I think query (2) is slightly faster (not much), and the mapper example (2nd example above) which I have in place messes up the labels (id becomes anon_1_id or a.id if I name the alias).
Why don't you do it like this:
query = generate_complex_query()
query = query.where(location.c.id < 100)
Essentially you can refine any query like this. Additionally, I suggest reading the SQL Expression Language Tutorial which is pretty awesome and introduces all the techniques you need. The way you build a select is only one way. Usually, I build my queries more like this: select(column).where(expression).where(next_expression) and so on. The FROM is usually automatically inferred by SQLAlchemy from the context, i.e. you rarely need to specify it.
Since you don't have access to the internals of generate_complex_query try this:
query = query.where(query.c.id < 100)
This should work in your case I presume.
Another idea:
query = query.where(text("id < 100"))
This uses SQLAlchemy's text expression. This could work for you, however, and this is important: If you want to introduce variables, read the description of the API linked above, because just using format strings intead of bound parameters will open you up to SQL injection, something that normally is a no-brainer with SQLAlchemy but must be taken care of if working with such literal expressions.
Also note that this works because you label the column as id. If you don't do that and don't know the column name, then this won't work either.