I am new to this and trying to learn python. I wrote a select statement in python where I used a parameter
Select """cln.customer_uid = """[(num_cuid_number)])
TypeError: string indices must be integers
Agree with the others, this doesn't look really like Python by itself.
I will see even without seeing the rest of that code I'll guess the [(num_cuid_number)] value(s) being returned is a string, so you'll want to convert it to integer for the select statement to process.
num_cuid_number is most likely a string in your code; the string indices are the ones in the square brackets. So please first check your data variable to see what you received there. Also, I think that num_cuid_number is a string, while it should be in an integer value.
Let me give you an example for the python code to execute: (Just for the reference: I have used SQLAlchemy with flask)
#app.route('/get_data/')
def get_data():
base_sql="""
SELECT cln.customer_uid='%s' from cln
""" % (num_cuid_number)
data = db.session.execute(base_sql).fetchall()
Pretty sure you are trying to create a select statement with a "where" clause here. There are many ways to do this, for example using raw sql, the query should look similar to this:
query = "SELECT * FROM cln WHERE customer_uid = %s"
parameters = (num_cuid_number,)
separating the parameters from the query is secure. You can then take these 2 variables and execute them with your db engine like
results = db.execute(query, parameters)
This will work, however, especially in Python, it is more common to use a package like SQLAlchemy to make queries more "flexible" (in other words, without manually constructing an actual string as a query string). You can do the same thing using SQLAlchemy core functionality
query = cln.select()
query = query.where(cln.customer_uid == num_cuid_number)
results = db.execute(query)
Note: I simplified "db" in both examples, you'd actually use a cursor, session, engine or similar to execute your queries, but that wasn't your question.
I have a query which looks like this:
query = session.query(Item) \
.filter(Item.company_id == company_id) \
.order_by(Item.id)
It's a pretty basic query. In addition to pulling out the values for the Item, I want to append an additional value into the mix, and have it returned to me. In raw SQL, I would do this:
SELECT *, 0 as subscribed
FROM items
WHERE company_id = 34
ORDER BY id
How can I manually add that value via sqlalchemy?
You'll need to use a literal_column, which looks a bit like this:
sqlalchemy.orm.Query(Item, sqlalchemy.sql.expression.literal_column("0"))
Beware that the text argument is inserted into the query without any transformation; this may expose you to a SQL Injection vulnerability if you accept values for the text parameter from outside your application. If that's something you need, you'll want to use bindparam, which is about as easy to use; but you will have to invent a name:
sqlalchemy.orm.Query(Item, sqlalchemy.sql.expression.bindparam("zero", 0))
As mentioned in the comments of the accepted answer there's a "shorthand" for bindparam() that alleviates the need to come up with a name for the literal bindparam, literal():
Return a literal clause, bound to a bind parameter.
So one does not have to write
session.query(Item, bindparam("zero", 0).label("subscribed"))
but just
session.query(Item, literal(0).label("subscribed"))
without having to worry about quoting etc., if passing strings or such.
So I'm trying to use smart_query in web2py to find specific values in a db, but the only explanation I can find is in the web2py book and it's not very clear. The example GET request from the book is formatted like this:
def GET(search):
try:
rows = db.smart_query([db.person, db.pet], search).select()
return dict(result=rows)
except:
...
I'm confused as to what values I would put in place of db.person and db.pet. Here is what the book says on it:
The method db.smart_query takes two arguments:
a list of field or table that should be allowed in the query
a string containing the query expressed in natural language
I'm thinking the first value would be the database I'm searching, but then I don't know what the second value would be. The book makes it sound like it should be the string I'm searching for, but I think that that's what the variable search is for.
Could someone please help me understand what exactly each argument is supposed to do?
The first argument to smart_query is a list of DAL Table and/or Field objects (a Table object in the list will simply be expanded to include all of the table's fields). This list determines which fields can be included in the query.
The second argument is the query itself, which can include field names and comparison operators (and their natural language counterparts) as well as "and" and "or" to expression conjunctions and disjunctions. For an idea of what is allowed, you can examine the relevant code here.
The SQLFORM.grid advanced search widget generates queries that are ultimately parsed by smart_query, so to get a better idea of how to generate such queries, try creating a test SQLFORM.grid and play with the search widget in the UI to see the queries it generates.
I need to rewrite the php script that handles a POST request and queries a SQLite3 database into python. The Query now looks like this
cursor.execute("SELECT * from "+requestedProgram+" WHERE start LIKE ? ORDER BY start", (requestedDate,))
the first part until the WHERE condition works (it is not elegant but unfortunately it seems that table names cannot be parametrized).
However, I need to put quotes around the requestedDate string in order to make LIKE work. How do I get quotes around the parameter/variable?
On another note: does the ORDER BY statement even do anything considering the nature of the cursor?
Use single quotes, don't use the trailing comma behind requestedDate (it tuple-izes it), and add a trailing wildcard to the string.
cursor.execute("SELECT * from "+requestedProgram+" WHERE start LIKE '?' ORDER BY start", requestedDate + '%')
Edit: Updated per conversation thread with OP below. Details added for those who stumble across this.
Does psycopg2 have a function for escaping the value of a LIKE operand for Postgres?
For example I may want to match strings that start with the string "20% of all", so I want to write something like this:
sql = '... WHERE ... LIKE %(myvalue)s'
cursor.fetchall(sql, { 'myvalue': escape_sql_like('20% of all') + '%' }
Is there an existing escape_sql_like function that I could plug in here?
(Similar question to How to quote a string value explicitly (Python DB API/Psycopg2), but I couldn't find an answer there.)
Yeah, this is a real mess. Both MySQL and PostgreSQL use backslash-escapes for this by default. This is a terrible pain if you're also escaping the string again with backslashes instead of using parameterisation, and it's also incorrect according to ANSI SQL:1992, which says there are by default no extra escape characters on top of normal string escaping, and hence no way to include a literal % or _.
I would presume the simple backslash-replace method also goes wrong if you turn off the backslash-escapes (which are themselves non-compliant with ANSI SQL), using NO_BACKSLASH_ESCAPE sql_mode in MySQL or standard_conforming_strings conf in PostgreSQL (which the PostgreSQL devs have been threatening to do for a couple of versions now).
The only real solution is to use the little-known LIKE...ESCAPE syntax to specify an explicit escape character for the LIKE-pattern. This gets used instead of the backslash-escape in MySQL and PostgreSQL, making them conform to what everyone else does and giving a guaranteed way to include the out-of-band characters. For example with the = sign as an escape:
# look for term anywhere within title
term= term.replace('=', '==').replace('%', '=%').replace('_', '=_')
sql= "SELECT * FROM things WHERE description LIKE %(like)s ESCAPE '='"
cursor.execute(sql, dict(like= '%'+term+'%'))
This works on PostgreSQL, MySQL, and ANSI SQL-compliant databases (modulo the paramstyle of course which changes on different db modules).
There may still be a problem with MS SQL Server/Sybase, which apparently also allows [a-z]-style character groups in LIKE expressions. In this case you would want to also escape the literal [ character with .replace('[', '=['). However according to ANSI SQL escaping a character that doesn't need escaping is invalid! (Argh!) So though it will probably still work across real DBMSs, you'd still not be ANSI-compliant. sigh...
I was able to escape % by using %% in the LIKE operand.
sql_query = "select * from mytable where website like '%%.com'"
cursor.fetchall(sql_query)
If you're using a prepared statement, then the input will be wrapped in '' to prevent sql injection. This is great, but also prevents input + sql concatenation.
The best and safest way around this would be to pass in the %(s) as part of the input.
cursor.execute('SELECT * FROM goats WHERE name LIKE %(name)s', { 'name': '%{}%'.format(name)})
You can also look at this problem from a different angle. What do you want? You want a query that for any string argument executes a LIKE by appending a '%' to the argument. A nice way to express that, without resorting to functions and psycopg2 extensions could be:
sql = "... WHERE ... LIKE %(myvalue)s||'%'"
cursor.execute(sql, { 'myvalue': '20% of all'})
I found a better hack. Just append '%' to your search query_text.
con, queryset_list = psycopg2.connect(**self.config), None
cur = con.cursor(cursor_factory=RealDictCursor)
query = "SELECT * "
query += " FROM questions WHERE body LIKE %s OR title LIKE %s "
query += " ORDER BY questions.created_at"
cur.execute(query, ('%'+self.q+'%', '%'+self.q+'%'))
I wonder if all of the above is really needed. I am using psycopg2 and was simply able to use:
data_dict['like'] = psycopg2.Binary('%'+ match_string +'%')
cursor.execute("SELECT * FROM some_table WHERE description ILIKE %(like)s;", data_dict)
Instead of escaping the percent character, you could instead make use of PostgreSQL's regex implementation.
For example, the following query against the system catalogs will provide a list of active queries which are not from the autovacuuming sub-system:
SELECT procpid, current_query FROM pg_stat_activity
WHERE (CURRENT_TIMESTAMP - query_start) >= '%s minute'::interval
AND current_query !~ '^autovacuum' ORDER BY (CURRENT_TIMESTAMP - query_start) DESC;
Since this query syntax doesn't utilize the 'LIKE' keyword, you're able to do what you want... and not muddy the waters with respect to python and psycopg2.
Having failed to find a built-in function so far, the one I wrote is pretty simple:
def escape_sql_like(s):
return s.replace('\\', '\\\\').replace('%', '\\%').replace('_', '\\_')
You can create a Like class subclassing str and register an adapter for it to have it converted in the right like syntax (e.g. using the escape_sql_like() you wrote).
I made some modifications to the code above to do the following:
def escape_sql_like(SQL):
return SQL.replace("'%", 'PERCENTLEFT').replace("%'", 'PERCENTRIGHT')
def reescape_sql_like(SQL):
return SQL.replace('PERCENTLEFT', "'%").replace('PERCENTRIGHT', "%'")
SQL = "SELECT blah LIKE '%OUCH%' FROM blah_tbl ... "
SQL = escape_sql_like(SQL)
tmpData = (LastDate,)
SQL = cur.mogrify(SQL, tmpData)
SQL = reescape_sql_like(SQL)
cur.execute(SQL)
It just requires to concatenate double % before and after it. Using "ilike" instead of "like" makes it case insensitive.
query = """
select
*
from
table
where
text_field ilike '%%' || %(search_text)s || '%%'
"""
I think it would be simpler and more readable to use f-strings.
query = f'''SELECT * FROM table where column like '%%{my_value}%%' '''
cursor.execute(query)