We Keep Coding

Unable to import database into mysql using python [duplicate]

I have the following lines of code:
sql = "source C:\\My Dropbox\\workspace\\projects\\hosted_inv\\create_site_db.sql"
cursor.execute (sql)
When I execute my program, I get the following error:
Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'source C:\My Dropbox\workspace\projects\hosted_inv\create_site_db.sql' at line 1
Now I can copy and past the following into mysql as a query:
source C:\\My Dropbox\\workspace\\projects\\hosted_inv\\create_site_db.sql
And it works perfect. When I check the query log for the query executed by my script, it shows that my query was the following:
source C:\\My Dropbox\\workspace\\projects\\hosted_inv\\create_site_db.sql
However, when I manually paste it in and execute, the entire create_site_db.sql gets expanded in the query log and it shows all the sql queries in that file.
Am I missing something here on how mysqldb does queries? Am I running into a limitation. My goal is to run a sql script to create the schema structure, but I don't want to have to call mysql in a shell process to source the sql file.
Any thoughts? Thanks!

As others said, you cannot use the command source in MySQLdb Python API
So, instead of running that, load the file and execute it
Lets say your .sql file has
create database test;
Read the content like
sql=open("test.sql").read()
And then execute it
cursor.execute(sql);
You will get new database "test"

The source command is one of the built-in commands recognized only by the mysql command-line client. It is not supported as a statement you can execute via any API.
Some people think you can simply split an SQL script file on the ";" statement terminator and call execute() on each line you get. But there are numerous exception cases:
Statements that are built-in commands like CONNECT, SOURCE, CHARSET, WARNINGS, QUIT, etc.
Note that built-in commands don't need to terminate in ; for example DELIMITER.
Statements that contain ; but not as a terminator, like CREATE TRIGGER.
Statements that contain ; inside string literals or comments or even quoted identifiers.
Comments lines.
To load an SQL script programmatically, you'd have to duplicate a fair amount of the functionality of the mysql client. So it's best if you just fork a process to actually execute that client program with the script as input.
See also:
Loading .sql files from within PHP
is it possible to call a sql script from a stored procedure in another sql script?
composing multiple mysql scripts
Running Database scripts in C#

'source' is not an SQL command, but an internal command of the mysql command line client.

I ran into the same problem!
As a solution I installed the library sqlparse and used the sqlparse.split( sql ) results. I had to check that sql_parts don't include blank lines as solo statements... Otherwise "WOW" sqlparse is pretty great and exactly what I needed!
import sqlparse
....
sql = open("test.sql").read()
sql_parts = sqlparse.split( sql )
for sql_part in sql_parts:
if sql_part.strip() == '':
continue
cursor.execute( sql_part )
FYI: If you do not run each statement on its own you may get the error "Commands out of sync; you can't run this command now". I only got this error after I added some more queries to my sql file - not the first time around.

I believe the "source" command is specific to the mysql shell executable - it is not an sql command and cannot be interpreted correctly when executed as an sql statement.
To achieve your goal, you probably need to read your script file and parse it into individual sql statements, then execute them one at a time with your cursor.

How to sync data between a Google Sheet and a Mysql DB?

So I have a Google sheet that maintains a lot of data. I also have a MySQL DB with a huge junk of data. There is a vital piece of information in the Sheet that is also present in the DB. Both needs to be in sync. The information always enters the Sheet first. I had a python script with mysql queries to update my database separately.
Now the work flow has changed. Data will enter the sheet and whenever that happens the database has to updated automatically.
After some research, I found that using the onEdit function of Google AppScript (I learned from here.), I could pickup when the file has changed.
The Next step is to fetch the data from relevant cell, which I can do using this.
Now I need to connect to the DB and send some queries. This is where I am stuck.
Approach 1:
Have a python web-app running live. Send the data via UrlFetchApp.This I yet have to try.
Approach 2:
Connect to mySQL remotely through appscript. But I am not sure this is possible after 2-3 hours of reading the docs.
So this is my scenario. Any viable solution you can think of or a better approach?

Connect directly to mySQL. You likely missed reading this part https://developers.google.com/apps-script/guides/jdbc

Using JDBC within Apps Script will work if you have the time to build this yourself.
If you don't want to roll your own solution, check out SeekWell. It allows you to connect to databases and write SQL queries directly in Sheets. You can create a run a “Run Sheet” that will run multiple queries at once and schedule those queries to be run without you even opening the Sheet.
Disclaimer: I made this.

Overwrite a database

I have an online database and connect to it by using MySQLdb.
db = MySQLdb.connect(......)
cur = db.cursor()
cur.execute("SELECT * FROM YOUR_TABLE_NAME")
data = cur.fetchall()
Now, I want to write the whole database to my localhost (overwrite). Is there any way to do this?
Thanks

If I'm reading you correctly, you have two database servers, A and B (where A is a remote server and B is running on your local machine) and you want to copy a database from server A to server B?
In all honesty, if this is a one-off, consider using the mysqldump command-line tool, either directly or calling it from python.
If not, the last answer on http://bytes.com/topic/python/answers/24635-dump-table-data-mysqldb details the SQL needed to define a procedure to output tables and data, though this may well miss subtleties mysqldump does not

Using pyodbc, and html.py to generate a table

I just started to play around with python, and i really like it. I am able to connect to my database using pydobc and print the results. However, i need help taking the next step to placing the information in a table in a 'form'
import pyodbc
connection = pyodbc.connect(MY STRING)
cur = connection.cursor()
cur.execute("select top(10) May07Control.seq, May07Control.member from dbo.May07Control")
for row in cur:
print "member: %s"% row.member
print "seq: %s"% row.seq
##OR the one-line way
cur.execute("select May07Control.seq, May07Control.member from dbo.May07Control")
resultList = [(row.member, row.seq) for row in cur]

So if you're running Python through IDLE and you're aiming at a browser view/interface. The next step is to get a development server running on your local machine. All the template engines come with cookbook instructions on how to do this. Bottle is a very simple example.
I personally use the development servers that come with Google App Engine, and PyDev in Eclipse, and you could potentially use the included Python server.

If you need data-driven presentation in a browser, I (and a lot of other people) would strongly suggest Django http://www.djangoproject.com/.

SQL queries through PYODBC fail silently on one machine, works on another

I am working on a program to automate parsing data from XML files and storing it into several databases. (Specifically the USGS realtime water quality service, if anyone's interested, at http://waterservices.usgs.gov/rest/WaterML-Interim-REST-Service.html) It's written in Python 2.5.1 using LXML and PYODBC. The databases are in Microsoft Access 2000.
The connection function is as follows:
def get_AccessConnection(db):
connString = 'DRIVER={Microsoft Access Driver (*.mdb)};DBQ=' + db
cnxn = pyodbc.connect(connString, autocommit=False)
cursor = cnxn.cursor()
return cnxn, cursor
where db is the filepath to the database.
The program:
a) opens the connection to the database
b) parses 2 to 8 XML files for that database and builds the values from them into a series of records to insert into the database (using a nested dictionary structure, not a user-defined type)
c) loops through the series of records, cursor.execute()-ing an SQL query for each one
d) commits and closes the database connection
If the cursor.execute() call throws an error, it writes the traceback and the query to the log file and moves on.
When my coworker runs it on his machine, for one particular database, specific records will simply not be there, with no errors recorded. When I run the exact same code on the exact same copy of the database over the exact same network path from my machine, all the data that should be there is there.
My coworker and I are both on Windows XP computers with Microsoft Access 2000 and the same versions of Python, lxml, and pyodbc installed. I have no idea how to check whether we have the same version of the Microsoft ODBC drivers. I haven't been able to find any difference between the records that are there and the records that aren't. I'm in the process of testing whether the same problem happens with the other databases, and whether it happens on a third coworker's computer as well.
What I'd really like to know is ANYTHING anyone can think of that would cause this, because it doesn't make sense to me. To summarize: Python code executing SQL queries will silently fail half of them on one computer and work perfectly on another.
Edit:
No more problem. I just had my coworker run it again, and the database was updated completely with no missing records. Still no idea why it failed in the first place, nor whether or not it will happen again, but "problem solved."

I have no idea how to check whether
we have the same version of the
Microsoft ODBC drivers.
I think you're looking for Control Panel | Administrative Tools | Data Sources (ODBC). Click the "Drivers" tab.
I think either Access 2000 or Office 2000 shipped with a desktop edition of SQL Server called "MSDE". Might be worth installing that for testing. (Or production, for that matter.)