I have a pretty long query i need to execute, and i'm using prepared statements to insert the values to the query. As far as i know, it's suppose to work, but it's not working!
Here's the code:
cursor = connection.cursor()
rows = cursor.execute(r"""SELECT... %s.. %s.. %s...""",(val1,val2,val3,))
rows = cursor.fetchall()
rows = text_position(rows)
If i insert a simple-quote it breaks, and same thing if i insert any non-english characters. It works with double-quotes as i wrapped the statement in them. Any suggestions?
This is a general technique for debugging SQL (especially when generated from another language, like Python):
Stop the mysql server (assuming you run mysqld without logging):
sudo invoke-rc.d mysql stop
Run mysqld with logging turned on:
sudo mysqld --log=/tmp/myquery.log
Run your script.
Look in /tmp/myquery.log for the SQL command that MySQLdb sent to mysqld.
This might give you enough of a clue to solve the problem, otherwise, please post the complete SQL command in Python
cursor.execute(r"""SELECT... %s.. %s.. %s...""",(val1,val2,val3,))
and the corresponding SQL as seen by mysqld in /tmp/myquery.log.
Once you are done debugging, you can shutdown the logging mysqld and restart it with:
sudo invoke-rc.d mysql restart
Related
I have the following lines of code:
sql = "source C:\\My Dropbox\\workspace\\projects\\hosted_inv\\create_site_db.sql"
cursor.execute (sql)
When I execute my program, I get the following error:
Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'source C:\My Dropbox\workspace\projects\hosted_inv\create_site_db.sql' at line 1
Now I can copy and past the following into mysql as a query:
source C:\\My Dropbox\\workspace\\projects\\hosted_inv\\create_site_db.sql
And it works perfect. When I check the query log for the query executed by my script, it shows that my query was the following:
source C:\\My Dropbox\\workspace\\projects\\hosted_inv\\create_site_db.sql
However, when I manually paste it in and execute, the entire create_site_db.sql gets expanded in the query log and it shows all the sql queries in that file.
Am I missing something here on how mysqldb does queries? Am I running into a limitation. My goal is to run a sql script to create the schema structure, but I don't want to have to call mysql in a shell process to source the sql file.
Any thoughts? Thanks!
As others said, you cannot use the command source in MySQLdb Python API
So, instead of running that, load the file and execute it
Lets say your .sql file has
create database test;
Read the content like
sql=open("test.sql").read()
And then execute it
cursor.execute(sql);
You will get new database "test"
The source command is one of the built-in commands recognized only by the mysql command-line client. It is not supported as a statement you can execute via any API.
Some people think you can simply split an SQL script file on the ";" statement terminator and call execute() on each line you get. But there are numerous exception cases:
Statements that are built-in commands like CONNECT, SOURCE, CHARSET, WARNINGS, QUIT, etc.
Note that built-in commands don't need to terminate in ; for example DELIMITER.
Statements that contain ; but not as a terminator, like CREATE TRIGGER.
Statements that contain ; inside string literals or comments or even quoted identifiers.
Comments lines.
To load an SQL script programmatically, you'd have to duplicate a fair amount of the functionality of the mysql client. So it's best if you just fork a process to actually execute that client program with the script as input.
See also:
Loading .sql files from within PHP
is it possible to call a sql script from a stored procedure in another sql script?
composing multiple mysql scripts
Running Database scripts in C#
'source' is not an SQL command, but an internal command of the mysql command line client.
I ran into the same problem!
As a solution I installed the library sqlparse and used the sqlparse.split( sql ) results. I had to check that sql_parts don't include blank lines as solo statements... Otherwise "WOW" sqlparse is pretty great and exactly what I needed!
import sqlparse
....
sql = open("test.sql").read()
sql_parts = sqlparse.split( sql )
for sql_part in sql_parts:
if sql_part.strip() == '':
continue
cursor.execute( sql_part )
FYI: If you do not run each statement on its own you may get the error "Commands out of sync; you can't run this command now". I only got this error after I added some more queries to my sql file - not the first time around.
I believe the "source" command is specific to the mysql shell executable - it is not an sql command and cannot be interpreted correctly when executed as an sql statement.
To achieve your goal, you probably need to read your script file and parse it into individual sql statements, then execute them one at a time with your cursor.
I'm relatively new to databases, so this question may have a simple answer (I've been searching for hours).
I want to write a Python script that pulls the SQL Code stored in SQL Server Management Studio. I can successfully connect to the database using pyodbc and run queries against the database tables, but I would like to be able to pull, for example, the SQL code stored in a procedure, function, view, etc. without running it.
This seems like something that would be relatively simple. I know it can be done in Powershell, but I would prefer to use Python. Is there some sort of module or pyodbc hack that will do this?
You can use the sp_helptext command in SQL Server, which will give you the SQL Server object source code line by line:
stored_proc_text = ""
res = cursor.execute('sp_helptext my_stored_procedure')
for row in res:
stored_proc_text += row[0]
print(stored_proc_text)
Good luck!
I'm using python 3.4 (ActiveState) and pyodbc 3.0.7 on a Windows 7 box to connect to a SQL Server 2008 RC2 database running on Window NT 6.1.
The problem I'm having is that the code below fails silently. No changes are made to the database.
connection = pyodbc.connect("DRIVER={SQL Server};SERVER=(local);DATABASE=Kerb;UID=sa;PWD=password", autocommit=True)
cursor = connection.cursor()
cursor.execute('''INSERT INTO [My].[Sample] (Case) VALUES (1);''')
I've also attempted to force the insert with a commit statement (which, unless I'm mistaken, shouldn't be necessary due to the autocommit=True), this also fails with no output.
cursor.execute('''INSERT INTO [My].[Sample] (Case) VALUES (1);''')
cursor.commit()
So my solution so far has been to add a sleep, which has solved the problem. But I worry about implementing this solution in production as it doesn't take into account network lag, etc.
cursor.execute('''INSERT INTO [My].[Sample] (Case) VALUES (1);''')
time.sleep(1)
I believe my question may be related to:
pyODBC and SQL Server 2008 and Python 3
If anyone has any ideas for further debugging or has documentation regarding this bit of asynchronous behavior I would love to hear it.
Thanks!
Unfortunately it appears that PyODBC cannot execute insert statements without the use of a timeout. I have started using PyMSSQL and the timeout is no longer required for a successful commit.
I am trying to run a mysql command through python pipe
cmd='CREATE DATABASE {0} ; GRANT ALL PRIVILEGES ON {1}.* TO "{2}"#"%" identified by password "{3}";'.format(requested_database_name,requested_database_name,'xyz',requested_password)
where the password is the hash generated from mysql. I always get
["ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use near '% identified by password *ASSFS3FS2' at line 1"]
But when I see the output for the cmd I am trying to run
CREATE DATABASE db1 ; GRANT ALL PRIVILEGES ON db1.* TO "xyz"#"%" identified by password "*ASSFS3FS2";
it looks good and runs well in the mysql prompt directly.
I use pipe = pexpect.spawn(cmd)
to run these commands. I am not sure where I am going wrong in terms of formatting.
You don't show us the exact sequence of commands you use (surely you don't pexpect.spawn that specific MySQL command!) but it sure looks like you're passing cmd through a shell which, of course, is eating up those indispensable double-quote characters.
With no idea of what Python version you're using (since you chose not to tell us), it's hard to know what to suggest. In 3.3 or later, shlex.quote is good. If you're stuck on an earlier version no later than 2.7, pipes.quote might help. But probably your best bet might be simply to replace each of those " with an escaped version:
cmd = cmd.replace('"', r'\"')
and try using the new version of cmd instead of your previous one -- this one should work independently of your Python version.
I have a SQL database and I want to fill tables with a script. The following source code shows what I have done so far:
#!/bin/tcsh
mysql -h dbs1 -D my_devel -u USER --password=XYZ
insert into my_table ( col_id, type, file, result, signature) values (***,'###','+++','$$$','...');
exit
According that I pass the parameters '*','#','+','$' and '.' via a method defined in python.
But somehow this is not working for me. Can someone tell me why, or suggest an alternative?
This seems to work fine for me:
#!/bin/tcsh
# count-rows.csh
mysql -h 127.0.0.1 -D tpcw -u root <<EOF
select count(*) from $1;
exit
EOF
You need to send the commands to execute to the standard input of the mysql utility. There are several ways to do that, but using a here document is probably the easiest method.
That said, if your data comes from a Python program, you should look at using a proper MySQL connector for Python. You should see a definite performance increase, but more importantly you will be able to handle any errors properly.
Expanding on Martijn's comment, one of the easiest ways to connect to a MySQL database is using python-mysqldb.