Here is the answer for getting raw sql from insert and I am wondering whether I can get raw sql if I use session.add() such as:
session.add(ormclass).compile()
If you want SQLAlchemy to log the actual raw SQL queries you can always set echo=True on the create_engine method. I know this seems rudimentary but it's the easiest way to see executed queries.
engine = create_engine("postgresql://localhost/dbname", echo=True)
My temporal solution to this is that when an error happens, there will be an exception and in this exception there are the parameters and statement.
except Exception as inst:
print(inst.statement % inst.params)
But the problem still exists because I cannot get the statement and parameters if there are no exceptions. In addition, there are no quotation marks for strings in the print so the string cannot be executed in mysql directly.
The way that I display the raw sql (which works most [but not all] of the time):
query = [my query that I created]
from sqlalchemy.dialects import mysql
str_query = query.compile(dialect=mysql.dialect(), compile_kwargs={"literal_binds": True})
Then I can just print the sql_query statement and it will show me the query with arguments. Some queries won't display, such as bulk inserts.
Related
I have the following code:
db.execute(sql)
sql contains the character sequence %s. This makes SQLAlchemy raise an exception because I don't provide any parameters. Is it possible to make SQLAlchemy pass sql verbatim to the database without trying to substitute any parameters, so that the database also receives the character sequence %s?
Disclaimer: I am totally aware of how SQL injection works and I know what I'm doing; this isn't an issue.
Use sqlalchemy.text:
db.execute(sqlalchemy.text(sql))
I'm a postgres newbie and am having some issues querying a text field in postgresql using Python. What is the correct syntax that will allow me to search the content of column "body" from table "jivemessage" out of database "postgres"?
try:
conn = psycopg2.connect("dbname='postgres' user='postgres' host='localhost' password='<password>'")
except:
print "cannot connect"
i = 'test'
cur = conn.cursor()
cur.execute('SELECT * from jivemessage WHERE body LIKE "%'+i+'%"')
Keep getting the following error:
ProgrammingError: column "%test%" does not exist
Thanks for any help.
You are not quoting the query properly. Don't use string concatenation here, use SQL parameters instead:
cur.execute('SELECT * from jivemessage WHERE body LIKE %s', ("%{}%".format(i),))
Here, the %s placeholder signals to the database driver that the first value of the second argument should be placed there when querying.
This leaves the interpolation up to the database driver, giving the database the opportunity to optimize for the query once, even if you were to reuse the same query.
It also prevents SQL injection attacks better than you could yourself, and most of all, guarantees that the correct quoting rules are followed.
For example, when I use cursor.execute() as documented:
>>> from django.db import connection
>>> cur = connection.cursor()
>>> cur.execute("DROP TABLE %s", ["my_table"])
django.db.utils.DatabaseError: near "?": syntax error
When Django's argument substitution is not used, the query works as expected:
>>> cur.execute("DROP TABLE my_table")
django.db.utils.DatabaseError: no such table: my_table
What am I doing wrong? How can I make parameterized queries work?
Notes:
Suffixing the query with ; does not help
As per the documentation, %s should be used, not SQLite's ? (Django translates %s to ?)
You cannot use parameters in SQL statements in place of identifiers (column or table names). You can only use them in place of single values.
Instead, you must use dynamic SQL to construct the entire SQL string and send that, unparameterized, to the database (being extra careful to avoid injection if the table name originates outside your code).
You can't substitute metadata in parameterized queries.
can somebody please recomend me some python DBAL library that will best suit my requirements. I would like to write my sql statements directly, most of the logics will be in db stored procedures (postgresql), so I only need to invoke db procedures, pass arguments to them and fetch the results. The library should help me with quoting (preventing sql inject).
I played with sqlalchemy, but i think that there is no quoting helper when writing sql statement directly to engine.execute method.
Thank you
You should have given sqlalchemy a deeper look; It does a fine job of quoting placeholders:
>>> engine = sqlalchemy.create_engine("sqlite:///:memory:")
>>> engine.execute("select ?", 5).fetchall()
[(5,)]
>>> engine.execute("select ?", "; drop table users; --").fetchall()
[(u'; drop table users; --',)]
psycopg2 (via DB-API) will automatically quote to prevent SQL injection, IF you use it properly. (The python way is wrong; you have to pass the parameters as arguments to the query command itself.)
WRONG:
cur.execute('select * from table where last="%s" and first="%s"'
% (last, first))
RIGHT:
cur.execute('select * from table where last=%s and first=%s',
(last, first))
Note: you don't use %, and you don't put quotes around your values.
The syntax is slightly different for MySQLdb and sqlite3. (For example, sqlite uses ? instead of %s.)
Also, for psycopg2, always use %s even if you're dealing with numbers or some other type.
What's the best way to make psycopg2 pass parameterized queries to PostgreSQL? I don't want to write my own escpaing mechanisms or adapters and the psycopg2 source code and examples are difficult to read in a web browser.
If I need to switch to something like PyGreSQL or another python pg adapter, that's fine with me. I just want simple parameterization.
psycopg2 follows the rules for DB-API 2.0 (set down in PEP-249). That means you can call execute method from your cursor object and use the pyformat binding style, and it will do the escaping for you. For example, the following should be safe (and work):
cursor.execute("SELECT * FROM student WHERE last_name = %(lname)s",
{"lname": "Robert'); DROP TABLE students;--"})
From the psycopg documentation
(http://initd.org/psycopg/docs/usage.html)
Warning Never, never, NEVER use Python string concatenation (+) or string parameters interpolation (%) to pass variables to a SQL query string. Not even at gunpoint.
The correct way to pass variables in a SQL command is using the second argument of the execute() method:
SQL = "INSERT INTO authors (name) VALUES (%s);" # Note: no quotes
data = ("O'Reilly", )
cur.execute(SQL, data) # Note: no % operator
Here are a few examples you might find helpful
cursor.execute('SELECT * from table where id = %(some_id)d', {'some_id': 1234})
Or you can dynamically build your query based on a dict of field name, value:
query = 'INSERT INTO some_table (%s) VALUES (%s)'
cursor.execute(query, (my_dict.keys(), my_dict.values()))
Note: the fields must be defined in your code, not user input, otherwise you will be susceptible to SQL injection.
I love the official docs about this:
https://www.psycopg.org/psycopg3/docs/basic/params.html