I need to write in a txt file, sql statements with all the values escaped so I can later import it to the mysql database.
Some background info:
I recently created a program in python which reads a csv file and inserts the data to a remote mysql database.
The program except the insert statements, also performs several select statements in order to collect some information such as primary keys, max ids etc.
What I need to do in general:
Since the whole procedure is time consuming I was hoping there is a way to write all the statements in an .sql file so then I could import it to the mysql database via PhpMyAdmin.
What I tried and where I stuck:
So far I can generate sql statements by concatenating text.
But I am stuck to the point where I need to escape characters.
In SQL Server I can include any text between N'' and it will accept it as it is.. Is there anything similar in MySQL?
Example in SQL server: N'My escaped text isn't affected by single quote ->' or double quote " '
Notice I have surrounded the text with N''.
Thanks in advance!
You're going to want to learn how to use prepared statements. They solve several problems for you, including escaping characters and preventing SQL injection. Here's an example with MySQLdb:
myValues = (value1, value2)
sql = "INSERT INTO myTable (column1, column2) VALUES (%s, %s)"
cursor.execute(sql, myValues)
===
I'm sorry, I think I misunderstood your question. You don't want to write a Python script that reads a file and inserts data from it into the database. You want to use Python to generate an SQL document that you can import into your database directly.
I recommend using Python to query the database, but if you really want to make an SQL document, all your data values need to be enclosed in single quotes, and if you have single quotes in your data, escape them using double single quotes:
INSERT INTO myTable (column1, column2) VALUES ('My data', 'John''s data');
Other special characters such as semicolon (;) will be ignored if they're inside the single quotes.
If you're worried about special characters in the table and column names, you can add backticks to them:
INSERT INTO `myTable` (`column1`, `column2`) VALUES ('My data', 'John''s data');
Related
I have a need to have "special characters" in a PostgreSQL table name that I am creating dynamically as needed.
create table if not exists a#_tablename column1 integer;
It will create the table if I surround the table name with quotes "" in the sql statement, but with no quotes will throw a syntax error.
create table if not exists "a#_tablename" column1 integer;
Strangely, it doesn't have a problem with the '_' symbol, just the '#'. If I use a symbol that forces me to embed quotes, I will have to embed quotes in all future sql statements referencing this table, which I would like to avoid. I am programing this in Python 3 using psycopg2. Are there any options that will allow psycopg2/postgresql to create the table without quotes? Googling found a 2005 bug report on this.
[https://www.postgresql.org/message-id/20051125085730.271511125023#pgfoundry.org]
Is there a list of special characters allowed? Thanks for any help.
I'm working on a bit of python code to run a query against a redshift (postgres) SQL database, and I'm running into an issue where I can't strip off the surrounding single quotes from a variable I'm passing to the query. I'm trying to drop a number of tables from a list. This is the basics of my code:
def func(table_list):
drop_query = 'drop table if exists %s' #loaded from file
table_name = table_list[0] #table_name = 'my_db.my_table'
con=psycopg2.connect(dbname=DB, host=HOST, port=PORT, user=USER, password=PASS)
cur=con.cursor()
cur.execute(drop_query, (table_name, )) #this line is giving me trouble
#cleanup statements for the connection
table_list = ['my_db.my_table']
when func() gets called, I am given the following error:
syntax error at or near "'my_db.my_table'"
LINE 1: drop table if exists 'my_db.my_table...
^
Is there a way I can remove the surrounding single quotes from my list item?
for the time being, I've done it (what think is) the wrong way and used string concatenation, but know this is basically begging for SQL-injection.
This is not how psycopg2 works. You are using a string operator %s to replace with a string. The reason for this is to tokenize your string safely to avoid SQL injection, psycopg2 handles the rest.
You need to modify the query before it gets to the execute statement.
drop_query = 'drop table if exists {}'.format(table_name)
I warn you however, do not allow these table names to be create by outside sources, or you risk SQL injection.
However a new version of PSYCOPG2 kind of allows something similar
http://initd.org/psycopg/docs/sql.html#module-psycopg2.sql
from psycopg2 import sql
cur.execute(
sql.SQL("insert into {} values (%s, %s)").format(sql.Identifier('my_table')),[10, 20]
)
While reading through python sqlite3 DB-API, the first example of creating table uses thee single quotes:
c.execute('''CREATE TABLE stocks
(date text, trans text, symbol text, qty real, price real)''')
But in other examples there are double quotes:
cur.execute("create table people (name_last, age)")
So I've got 2 questions:
It this really any difference? How can it affect create table command?
Which one is better to use with parameters? ex.:
c.execute('''CREATE TABLE %s(date text, trans text, symbol text, qty
real, price real)''' % table_name)
VS.
cur.execute("create table %s (name_last, age)" % table_name)
Thanks
There is basically no rule for double or single quotes.
But three consecutive quotes start multiline quoted values.
Is this really any difference? How can it affect create table command?
Using single/double quoted string or a triple-quoted string doesn't affect the create command in any way. They are ultimately just string arguments to the create command.
Which one is better to use with parameters?
Double quotes are the norm in most of the python programs. Triple quotes are used mostly when you have a big multi-line string copy-pasted from somewhere else and you want to embed them in your program. In the link that you gave above, we can see one such example in the usage in cursor.executescript(), where an sql script (which probably existed before) is used within the program.
using three """ is like commenting out a line
#this text python doesn't read so you can write about your code
"""it allows
you to type
over multiple
lines."""
Also you can use " or ' python see them both the same as long as you close with the same style.
I'm working on a project in Python with MySQLdb. As part of this, I'm moving user details, including salted passwords from one system that generates them to a new one that simply uses them.
Single, double or triple quotes can delineate your string start and end. However, single and double quotes are part of several hashes in the 4.5k or so users I'm migrating. Both tokens appear in about 450 of those salts.
An edited version of the code is as follows
Django.execute ('INSERT INTO auth_user (password) VALUES ("' + user.password + '")')
I have tried swapping between the quote type used in this database cursor object, as and when either quote type or the other are detected, but this still leaves the 150 or so that contain both.
What work arounds can I use for this?
I have tried triple quotes, but they throw a programming error on the cursor object.
Thanks in advance
Query parameters should provide all the proper escaping, for example:
cursor.execute('INSERT INTO auth_user (password) VALUES (%s)', [password])
From the Django docs at: http://docs.djangoproject.com/en/dev/topics/db/sql/
If you're not familiar with
the Python DB-API, note that the SQL
statement in cursor.execute() uses
placeholders, "%s", rather than adding
parameters directly within the SQL. If
you use this technique, the underlying
database library will automatically
add quotes and escaping to your
parameter(s) as necessary. (Also note
that Django expects the "%s"
placeholder, not the "?" placeholder,
which is used by the SQLite Python
bindings. This is for the sake of
consistency and sanity.)
Im using python to access a MySQL database and im getting a unknown column in field due to quotes not being around the variable.
code below:
cur = x.cnx.cursor()
cur.execute('insert into tempPDBcode (PDBcode) values (%s);' % (s))
rows = cur.fetchall()
How do i manually insert double or single quotes around the value of s?
I've trying using str() and manually concatenating quotes around s but it still doesn't work.
The sql statement works fine iv double and triple check my sql query.
You shouldn't use Python's string functions to build the SQL statement. You run the risk of leaving an SQL injection vulnerability. You should do this instead:
cur.execute('insert into tempPDBcode (PDBcode) values (%s);', s)
Note the comma.
Python will do this for you automatically, if you use the database API:
cur = x.cnx.cursor()
cur.execute('insert into tempPDBcode (PDBcode) values (%s)',s)
Using the DB API means that python will figure out whether to use quotes or not, and also means that you don't have to worry about SQL-injection attacks, in case your s variable happens to contain, say,
value'); drop database; '
If this were purely a string-handling question, the answer would be tojust put them in the string:
cur.execute('insert into tempPDBcode (PDBcode) values ("%s");' % (s))
That's the classic use case for why Python supports both kinds of quotes.
However as other answers & comments have pointed out, there are SQL-specific concerns that are relevant in this case.