How can execute sql script stored in *.sql file using MySQLdb python driver. I was trying
cursor.execute(file(PATH_TO_FILE).read())
but this doesn't work because cursor.execute can run only one sql command at once. My sql script contains several sql statements instead. Also I was trying
cursor.execute('source %s'%PATH_TO_FILE)
but also with no success.
From python, I start a mysql process to execute the file for me:
from subprocess import Popen, PIPE
process = Popen(['mysql', db, '-u', user, '-p', passwd],
stdout=PIPE, stdin=PIPE)
output = process.communicate('source ' + filename)[0]
I also needed to execute a SQL file, but the catch was that there wasn't one statement per line, so the accepted answer didn't work for me.
The SQL file I wanted to execute looked like this:
-- SQL script to bootstrap the DB:
--
CREATE USER 'x'#'%' IDENTIFIED BY 'x';
GRANT ALL PRIVILEGES ON mystore.* TO 'x'#'%';
GRANT ALL ON `%`.* TO 'x'#`%`;
FLUSH PRIVILEGES;
--
--
CREATE DATABASE oozie;
GRANT ALL PRIVILEGES ON oozie.* TO 'oozie'#'localhost' IDENTIFIED BY 'oozie';
GRANT ALL PRIVILEGES ON oozie.* TO 'oozie'#'%' IDENTIFIED BY 'oozie';
FLUSH PRIVILEGES;
--
USE oozie;
--
CREATE TABLE `BUNDLE_ACTIONS` (
`bundle_action_id` varchar(255) NOT NULL,
`bundle_id` varchar(255) DEFAULT NULL,
`coord_id` varchar(255) DEFAULT NULL,
`coord_name` varchar(255) DEFAULT NULL,
`critical` int(11) DEFAULT NULL,
`last_modified_time` datetime DEFAULT NULL,
`pending` int(11) DEFAULT NULL,
`status` varchar(255) DEFAULT NULL,
`bean_type` varchar(31) DEFAULT NULL,
PRIMARY KEY (`bundle_action_id`),
KEY `I_BNDLTNS_DTYPE` (`bean_type`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
--
Some statements in the above file lie on a single line and some statements also span multiple lines (like the CREATE TABLE at the end). There are also a few SQL inline comment lines that begin with "--".
As suggested by ThomasK, I had to write some simple rules to join lines into a statement. I ended up with a function to execute a sql file:
def exec_sql_file(cursor, sql_file):
print "\n[INFO] Executing SQL script file: '%s'" % (sql_file)
statement = ""
for line in open(sql_file):
if re.match(r'--', line): # ignore sql comment lines
continue
if not re.search(r';$', line): # keep appending lines that don't end in ';'
statement = statement + line
else: # when you get a line ending in ';' then exec statement and reset for next statement
statement = statement + line
#print "\n\n[DEBUG] Executing SQL statement:\n%s" % (statement)
try:
cursor.execute(statement)
except (OperationalError, ProgrammingError) as e:
print "\n[WARN] MySQLError during execute statement \n\tArgs: '%s'" % (str(e.args))
statement = ""
I'm sure there's scope for improvement, but for now it's working pretty well for me. Hope someone finds it useful.
This worked for me:
with open('schema.sql') as f:
cursor.execute(f.read().decode('utf-8'), multi=True)
for line in open(PATH_TO_FILE):
cursor.execute(line)
This assumes you have one SQL statement per line in your file. Otherwise you'll need to write some rules to join lines together.
Another solution that allows to leverage on the MySQL interpreter without any parsing is to use the os.system command to run a MySQL prompt command directly inside python:
from os import system
USERNAME = "root"
PASSWORD = "root"
DBNAME = "pablo"
HOST = "localhost"
PORT = 3306
FILE = "file.sql"
command = """mysql -u %s -p"%s" --host %s --port %s %s < %s""" %(USERNAME, PASSWORD, HOST, PORT, DBNAME, FILE)
system(command)
It avoids any parsing error when for example you would have a string variable with a smiley ;-) in it or if you check for the ; as the last character, if you have comments afterward like SELECT * FROM foo_table; # selecting data
Many of the answers here have serious flaws...
First don't try to parse an open ended sql script yourself! If you think that is easily done, you aren't aware of how robust and complicated sql can be. Serious sql scripts certainly involve statements and procedure definitions spanning multiple lines. It is also common to explicitly declare and change delimiters the in middle of your scripts. You can also nest source commands within each other. For so many reasons, you want to run the script through the MySQL client and allow it to handle the heavy lifting. Trying to reinvent that is fraught peril and a huge waste of time. Maybe if you are the only one writing these scripts, and you are not writing anything sophisticated you could get away with that, but why limit yourself to such a degree? What about machine generated scripts, or those written by other developers?
The answer from #jdferreira is on the right track, but also has problems and weaknesses. The most significant is that a security hole is being opened up by sending the connection parameters to the process in that manner.
Here's a solution / example for your copy & paste pleasure. My extended discussion follows:
First, create a separate config file to save your user name and password.
db-creds.cfg
[client]
user = XXXXXXX
password = YYYYYYY
Slap the right file system permissions on that, so the python process can read from it, but no one can view that who should not be able to.
Then, use this Python (in my example case the creds file is adjacent to the py script):
#!/usr/bin/python
import os
import sys
import MySQLdb
from subprocess import Popen, PIPE, STDOUT
__MYSQL_CLIENT_PATH = "mysql"
__THIS_DIR = os.path.dirname( os.path.realpath( sys.argv[0] ) )
__DB_CONFIG_PATH = os.path.join( __THIS_DIR, "db-creds.cfg" )
__DB_CONFIG_SECTION = "client"
__DB_CONN_HOST = "localhost"
__DB_CONN_PORT = 3306
# ----------------------------------------------------------------
class MySqlScriptError( Exception ):
def __init__( self, dbName, scriptPath, stdOut, stdErr ):
Exception.__init__( self )
self.dbName = dbName
self.scriptPath = scriptPath
self.priorOutput = stdOut
self.errorMsg = stdErr
errNumParts = stdErr.split("(")
try : self.errorNum = long( errNumParts[0].replace("ERROR","").strip() )
except: self.errorNum = None
try : self.sqlState = long( errNumParts[1].split(")")[0].strip() )
except: self.sqlState = None
def __str__( self ):
return ("--- MySqlScriptError ---\n" +
"Script: %s\n" % (self.scriptPath,) +
"Database: %s\n" % (self.dbName,) +
self.errorMsg )
def __repr__( self ): return self.__str__()
# ----------------------------------------------------------------
def databaseLoginParms() :
from ConfigParser import RawConfigParser
parser = RawConfigParser()
parser.read( __DB_CONFIG_PATH )
return ( parser.get( __DB_CONFIG_SECTION, "user" ).strip(),
parser.get( __DB_CONFIG_SECTION, "password" ).strip() )
def databaseConn( username, password, dbName ):
return MySQLdb.connect( host=__DB_CONN_HOST, port=__DB_CONN_PORT,
user=username, passwd=password, db=dbName )
def executeSqlScript( dbName, scriptPath, ignoreErrors=False ) :
scriptDirPath = os.path.dirname( os.path.realpath( scriptPath ) )
sourceCmd = "SOURCE %s" % (scriptPath,)
cmdList = [ __MYSQL_CLIENT_PATH,
"--defaults-extra-file=%s" % (__DB_CONFIG_PATH,) ,
"--database", dbName,
"--unbuffered" ]
if ignoreErrors :
cmdList.append( "--force" )
else:
cmdList.extend( ["--execute", sourceCmd ] )
process = Popen( cmdList
, cwd=scriptDirPath
, stdout=PIPE
, stderr=(STDOUT if ignoreErrors else PIPE)
, stdin=(PIPE if ignoreErrors else None) )
stdOut, stdErr = process.communicate( sourceCmd if ignoreErrors else None )
if stdErr is not None and len(stdErr) > 0 :
raise MySqlScriptError( dbName, scriptPath, stdOut, stdErr )
return stdOut
If you want to test it out, add this:
if __name__ == "__main__":
( username, password ) = databaseLoginParms()
dbName = "ExampleDatabase"
print "MySQLdb Test"
print
conn = databaseConn( username, password, dbName )
cursor = conn.cursor()
cursor.execute( "show tables" )
print cursor.fetchall()
cursor.close()
conn.close()
print
print "-----------------"
print "Execute Script with ignore errors"
print
scriptPath = "test.sql"
print executeSqlScript( dbName, scriptPath,
ignoreErrors=True )
print
print "-----------------"
print "Execute Script WITHOUT ignore errors"
print
try : print executeSqlScript( dbName, scriptPath )
except MySqlScriptError as e :
print "dbName: %s" % (e.dbName,)
print "scriptPath: %s" % (e.scriptPath,)
print "errorNum: %s" % (str(e.errorNum),)
print "sqlState: %s" % (str(e.sqlState),)
print "priorOutput:"
print e.priorOutput
print
print "errorMsg:"
print e.errorMsg
print
print e
print
And for good measure, here's an example sql script to feed into it:
test.sql
show tables;
blow up;
show tables;
So, now for some discussion.
First, I illustrate how to use MySQLdb along with this external script execution, while storing the creds in one shared file you can use for both.
By using --defaults-extra-file on the command line you can SECURELY pass your connection parameters in.
The combination of either --force with stdin streaming the source command OR --execute running the command on the outside let's you dictate how the script will run. That is by ignoring errors and continuing to run, or stopping as soon as an error occurs.
The order in which the results comeback will also be preserved via --unbuffered. Without that, your stdout and stderr streams will be jumbled and undefined in their order, making it very hard to figure out what worked and what did not when comparing that to the input sql.
Using the Popen cwd=scriptDirPath let's you nest source commands within one another using relative paths. If your scripts will all be in the same directory (or a known path relative to it), doing this let's you reference those relative to where the top level script resides.
Finally, I threw in an exception class which carries all the info you could possibly want about what happened. If you are not using the ignoreErrors option, one of these exceptions will be thrown in your python when something goes wrong and script has stopped running upon that error.
At least MySQLdb 1.2.3 seems to allow this out of the box, you just have to call cursor.nextset() to cycle through the returned result sets.
db = conn.cursor()
db.execute('SELECT 1; SELECT 2;')
more = True
while more:
print db.fetchall()
more = db.nextset()
If you want to be absolutely sure the support for this is enabled, and/or disable the support, you can use something like this:
MYSQL_OPTION_MULTI_STATEMENTS_ON = 0
MYSQL_OPTION_MULTI_STATEMENTS_OFF = 1
conn.set_server_option(MYSQL_OPTION_MULTI_STATEMENTS_ON)
# Multiple statement execution here...
conn.set_server_option(MYSQL_OPTION_MULTI_STATEMENTS_OFF)
The accepted answer will encounter problems when your sql script contains empty lines and your query sentence spans multiple lines. Instead, using the following approach will solve the problem:
f = open(filename, 'r')
query = " ".join(f.readlines())
c.execute(query)
As mentioned in one of the comments, if you are sure that every command ends with a semi-colon, you can do this:
import mysql.connector
connection = mysql.connector.connect(
host=host,
user=user,
password=password
)
cursor = connection.cursor()
with open(script, encoding="utf-8") as f:
commands = f.read().split(';')
for command in commands:
cursor.execute(command)
print(command)
connection.close()
Load mysqldump file:
for line in open(PATH_TO_FILE).read().split(';\n'):
cursor.execute(line)
Are you able to use a different database driver?
If yes: what you want is possible with the MySQL Connector/Python driver by MySQL.
Its cursor.execute method supports executing multiple SQL statements at once by passing Multi=True.
Splitting the SQL statements in the file by semicolon is not necessary.
Simple example (mainly copy & paste from the second link, I just added reading the SQL from the file):
import mysql.connector
file = open('test.sql')
sql = file.read()
cnx = mysql.connector.connect(user='uuu', password='ppp', host='hhh', database='ddd')
cursor = cnx.cursor()
for result in cursor.execute(sql, multi=True):
if result.with_rows:
print("Rows produced by statement '{}':".format(
result.statement))
print(result.fetchall())
else:
print("Number of rows affected by statement '{}': {}".format(
result.statement, result.rowcount))
cnx.close()
I'm using this to import MySQL dumps (created in phpMyAdmin by exporting the whole database to a SQL file) from the *.sql file back into a database.
Here's a code snippet that will import a typical .sql that comes from an export. (I used it with exports from Sequel Pro successfully.) Deals with multi-line queries and comments (#).
Note 1: I used the initial lines from Thomas K's response but added more.
Note 2: For newbies, replace the DB_HOST, DB_PASS etc with your database connection info.
import MySQLdb
from configdb import DB_HOST, DB_PASS, DB_USER, DB_DATABASE_NAME
db = MySQLdb.connect(host=DB_HOST, # your host, usually localhost
user=DB_USER, # your username
passwd=DB_PASS, # your password
db=DB_DATABASE_NAME) # name of the data base
cur = db.cursor()
PATH_TO_FILE = "db-testcases.sql"
fullLine = ''
for line in open(PATH_TO_FILE):
tempLine = line.strip()
# Skip empty lines.
# However, it seems "strip" doesn't remove every sort of whitespace.
# So, we also catch the "Query was empty" error below.
if len(tempLine) == 0:
continue
# Skip comments
if tempLine[0] == '#':
continue
fullLine += line
if not ';' in line:
continue
# You can remove this. It's for debugging purposes.
print "[line] ", fullLine, "[/line]"
try:
cur.execute(fullLine)
except MySQLdb.OperationalError as e:
if e[1] == 'Query was empty':
continue
raise e
fullLine = ''
db.close()
How about using the pexpect library? The idea is, that you can start a process pexpect.spawn(...), and wait until the output of that process contains a certain pattern process.expect(pattern).
I actually used this to connect to the mysql client and execute some sql scripts.
Connecting:
import pexpect
process = pexpect.spawn("mysql", ["-u", user, "-p"])
process.expect("Enter password")
process.sendline(password)
process.expect("mysql>")
This way the password is not hardcoded into the command line parameter (removes security risk).
Executing even several sql scripts:
error = False
for script in sql_scripts:
process.sendline("source {};".format(script))
index = process.expect(["mysql>", "ERROR"])
# Error occurred, interrupt
if index == 1:
error = True
break
if not error:
# commit changes of the scripts
process.sendline("COMMIT;")
process.expect("mysql>")
print "Everything fine"
else:
# don't commit + print error message
print "Your scripts have errors"
Beware that you always call expect(pattern), and that it matches, otherwise you will get a timeout error. I needed this bit of code to execute several sql scripts and only commit their changes if no error occurred, but it is easily adaptable for use cases with only one script.
You can use something like this-
def write_data(schema_name: str, table_name: str, column_names: str, data: list):
try:
data_list_template = ','.join(['%s'] * len(data))
insert_query = f"insert into {schema_name}.{table_name} ({column_names}) values {data_list_template}"
db.execute(insert_query, data)
conn_obj.commit()
except Exception as e:
db.execute("rollback")
raise e
Related
So I've been troubleshooting this one for a minute, what's weird is that it seems to run just fine in a stand alone Python script, but when being moved to an asterisk AGI I get the below error:
cursor.execute(query, (res))
File "/usr/local/lib/python2.7/dist-packages/mysql/connector/cursor_cext.py", line 248, in execute
prepared = self._cnx.prepare_for_mysql(params)
File "/usr/local/lib/python2.7/dist-packages/mysql/connector/connection_cext.py", line 615, in prepare_for_mysql
raise ValueError("Could not process parameters")
Below is the data in question that was ran that threw this error.
#!/usr/bin/env python
import sys
from asterisk.agi import *
from os import system
import subprocess
import time
import mysql.connector
import md5
agi = AGI()
agi.verbose("python agi started")
callerId = agi.env['agi_callerid']
agi.verbose("call from %s" % callerId)
res=agi.get_variable("res")
agi.verbose("want %s br" %res)
cnx = mysql.connector.connect(user='user',password='sanitized',host='127.0.0.1',database='rental')
cursor = cnx.cursor(buffered=True)
query = ("select active,address,rent,deposit,phone,pets_allow,smoker_allow,section_8,garage,attached_garage,fenced_yard,carport,shed,patio,deck,basement,garden_area,off_street,on_street,walkin_shower,double_lavs,back_porch,front_porch,cha,ac,fireplace,central_heat,stove,dish_washer,microwave,refrigerator,washer,dryer,washer_hu,dryer_hu,garbage_dispsal,water,trash_service,gas,electric,barn,wall_furnace from activelisting where active=1 and number_br=3")
query = ("select active,address,rent,deposit,phone,calldialog,id from activelisting where active=1 and number_br=3 order by id")
cursor.execute(query, (res))
cursor.close()
cnx.close()
Note: number_br=3, should technically be number_br=%s but for the sake of troubleshooting I substituted in a static variable. This whole script worked on an older version of CentOS and I'm attempting to get it all working on a Debian Server.
EDIT: As arheops appropriately noted, this query was having issues with the res variable, as it was not a string, the resolution was ultimately to change how the variable was included in the query, see below for an example that brought about the solution
query = ("select active,address,rent,deposit,phone,pets_allow,smoker_allow,section_8,garage,attached_garage,fenced_yard,carport,shed,patio,deck,basement,garden_area,of$
query = ("select active,address,rent,deposit,phone,calldialog,id from activelisting where active=1 and number_br={} order by id".format(res))
cursor.execute(query)
cursor.execute accepting string, not touple as param.
https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html
Variable res is not string at all.
Based on Python MySQLdb execute table variable and MySQL LOAD DATA LOCAL INFILE example in python? this should work:
import pymysql, os
directory = os.path.join('path', 'to', 'directory')
filename = 'my_filename.csv'
filepath = os.path.join(directory, filename)
to_table_name = "my_table"
connection = pymysql.connect(..., local_infile=True)
with connection.cursor() as cursor:
load_statement = """
load data local infile %s
into table %s
fields terminated by ','
optionally enclosed by '"'
lines terminated by '\\n'
ignore 1 lines
"""
cursor.execute(load_statement % (filepath, to_table_name, ))
connection.commit()
connection.close
But I'm still seeing this error:
ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '/path/to/directory/my_filename.csv\n into ' at line 1")
When I run this without the parameters i.e. writing the actual filepath and table name it works.
Any help would be much appreciated.
You should use the built in ability of Execute to do your string formatting also (this avoids MYSQL Injection attacks and errors) ... Rather than passing the parameters to the load_statement using % (String Interpolation) , pass that as parameters to execute
cursor.execute(load_statement , (filepath, to_table_name ))
Notice the comma instead of a %
I have a program that uses a couple sql databases to store data. I have a class that manages the various sql functions, such as getting a value, an entire table or just updating a value. All of the processes work fine until I run a function that uses UPDATE. I execute an UPDATE command and try to commit the change and the database is always locked. Every function I have in my custom sql class has
cursor.close
database.close
So there shouldn't be any issue with the database connection still being open. Am I missing something in this syntax that is not connecting to the database correctly? I used the extra print statements in an attempt to find out where the problem is occurring, so those can be ignored.
import sqlite3 as db
import os
databaseName = "site"
class MassDb:
def __init__(self,databaseName):
super(MassDb, self).__init__()
print("Current Directory: ",os.getcwd())
self.databaseName = databaseName
def updateValue(self, location, metric, input_value):
print("OPEN CONNECTION UPDATE - running updateValue: ",location, metric, input_value)
if self.databaseName == "site":
try:
siteConn = db.connect("site_data.db")
siteCursor = siteConn.cursor()
siteCursor.execute("UPDATE sites SET " + metric + " = ? WHERE LOCATI ON = ?", (input_value, location))
siteConn.commit()
except:
print("UPDATE FAILED")
finally:
siteCursor.close
siteConn.close
elif self.databaseName == "comp":
try:
compConn = db.connect("comp_data.db")
compCursor = compConn.cursor()
compCursor.execute("UPDATE competitors SET " + metric + " = ? WHERE NAME = ?", (input_value, location))
compConn.commit()
except:
print("UPDATE FAILED")
finally:
compCursor.close
compConn.close
print("CLOSED CONNECTION UPDATE - Update Connection Closed")
else:
print("Update Error")
MassDb("site").updateValue("Location", "CURRENT_SCORE", "100")
As #roganjosh commented, my problem was that I wasn't properly closing the database. If
commit()
is used, there's no need to close the database. However,
cursor.close()
and
conn.close()
need to be written as such. Leaving off the parentheses would be as though an attribute is being referenced, rather than a method. In order to execute the close method, the () must be present. Seems obvious now, but I wasn't aware at the time. Hopefully this can help someone else if they too run across this.
Additionally, using a context manager works and eliminates the need to use close()
with conn:
#do stuff here
commit()
There is a post that tells me how to write data fom a csv-file into a sqlite database (link). Is there a way to simply copy the whole file into a database table instead of loading the data and then iterate through the rows appending them to the table as suggested in the link.
To do this in sqlite it says here to simply type
sqlite> create table test (id integer, datatype_id integer, level integer, meaning text);
sqlite> .separator ","
sqlite> .import no_yes.csv test
I am new to working with databases and sqlite3, but I thought doing something like this could work
import sqlite3
conn = sqlite3.connect('mydatabase.db')
c = conn.cursor()
def create_table(name):
c.execute("CREATE TABLE IF NOT EXISTS {} (time REAL, event INTEGER, id INTEGER, size INTERGER, direction INTEGER)".format(name))
def copy_data(file2copy,table_name):
c.executescript("""
.separator ","
.import {} {}""".format(file2copy,table_name))
conn.commit()
try:
create_table('abc')
copy_data(r'Y:\MYPATH\somedata.csv','abc')
except Exception as e:
print e
c.close()
conn.close()
But apparently it doesn't. I get the error
near ".": syntax error
EDIT: Thanks to the below suggestion to use the subprocess module, I came up with the follwing solution.
import subprocess
sqlshell = r'c:\sqlite-tools\sqlite3' # sqlite3.exe is a shell that can be obtained from the sqlite website
process = subprocess.Popen([sqlshell], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
cmd = """.open {db}
.seperator ","
.import {csv} {tb}""".format(db='C:/Path/to/database.db', csv='C:/Path/to/file.csv', tb='table_name')
stdout, stderr = process.communicate(input=cmd)
if len(stderr):
print 'something went wrong'
Importantly, you should use '/' instead of '\' in your directory names. Also, you have to be carefull with blank spaces in your directory names.
The commands you're using in copy_data that start with . are not part of sqlite3, but the interactive shell that it ships with it. You can't use them through the sqlite3 module.
You either need to manually write the insertion step or use the subprocess module to run the shell program.
I' m trying to send a text file into MySQL database. I am trying to do this with mysql connector in python 3.2. The problem is about LOAD DATA INFILE syntax. You can find my code above. My first question is is there anyway to solve this problem. Note that I have tried local-infile =1 option and Python does not allow this option. Second, is there any other way to send this data as a block into the mysql database?
from __future__ import print_function
import os
import mysql.connector
from mysql.connector import errorcode
config = {
'user':'root',
'password':'3778',
## 'host':'localhost',
# 'database':'microstructure',
# 'local-infile':'1',
}
DB_NAME = 'EURUSD'
TABLES ={}
TABLES['microstructure']=(
"CREATE TABLE `microstructure` ("
# " `p_id` int NOT NULL AUTO_INCREMENT,"
" `ticker` varchar(255),"
" `time` date,"
" `last_price` decimal(6,3)"
") ENGINE=InnoDB")
TABLES['cumulative']=(
"CREATE TABLE `cumulative` ("
" `p_id` int NOT NULL AUTO_INCREMENT,"
" `ticker` varchar(255),"
" `time` date,"
" `last_price` decimal(6,3),"
" PRIMARY KEY(`p_id`)"
") ENGINE=InnoDB")
cnx = mysql.connector.connect(**config)
cursor = cnx.cursor()
path_txt = 'C:/Users/ibrahim/Desktop/testfile.txt'
def create_database(cursor):
try:
cursor.execute(
"CREATE DATABASE IF NOT EXISTS {} DEFAULT CHARACTER SET 'utf8'".format(DB_NAME))
except mysql.connector.Error as err:
print("Failed creating database: {}".format(err))
exit(1)
try:
cnx.database = DB_NAME
except mysql.connector.Error as err:
if err.errno == errorcode.ER_BAD_DB_ERROR:
create_database(cursor)
cnx.database=DB_NAME
else:
print(err)
exit(1)
for name, ddl in TABLES.items():
try:
print("Creating table {}: ".format(name), end ='')
cursor.execute(ddl)
except mysql.connector.Error as err:
if err.errno == errorcode.ER_TABLE_EXISTS_ERROR:
print("Already exists")
else:
print(err)
else:
print("OK")
cursor.execute("SET ##global.local_infile = 1")
cursor.execute("LOAD DATA LOCAL INFILE 'testfile.txt' into table microstructure")
os.system("start")
cursor.close()
I just saw this old post, but None of theses answers solved my problem.
I saw here that there is an argument dedicated for LOCAL DATA INFILE:
allow_local_infile=True
So it is possible to do:
mysql.connector.connect(user='[username]', password='[pass]', host='[host]', allow_local_infile=True)
When in MySQLdb I use this to enable LOAD DATA LOCAL INFILE feature:
MySQLdb.connect(..., local_infile=True)
I had similar problems too. I am new to python and mysqldb. i added a conn.commit and it worked.
it seems that it protects the database by requiring a commit.
I am using:
MySQL server 8.0.25
mysql-connector-python 8.0.25
Python 3.8.10
#Maaz's answer to this question worked for me. I can't comment, so I thought I would post an answer to show activity on this page.
I also dug a little bit after the fact that adding this argument made a difference despite it shouldn't since the default should be allow_local_infile=True according to #abu and the link provided.
After a quick dive into the constructor of mysql.connector.connect, I found that the default eventually traced to [python packages]\mysql\connector\constants.py, in which it listed defaults for many of the arguments, one of which was 'allow_local_infile': False.
I hope this helps any people finding this problem in the future! :D
What version of mysql.connector are you using? I'm on 1.0.11 and I got the feature working. What I did is create data.txt with three lines (1, 2, 3), cd into it, start python 3.3.1, then ran:
import mysql.connector
conn = mysql.connector.connect(database='test', user='test', password='xx')
curs = conn.cursor()
curs.execute('''CREATE TABLE foo (id text)''')
curs.execute('''LOAD DATA INFILE '/<full path to>/data.txt' INTO TABLE foo''')
curs.execute('''SELECT * FROM FOO''')
print(curs.fetchall())
curs.execute('''DROP TABLE foo''')
My output:
[('1',), ('2',), ('3',)]
Note that you'll need to grant the FILE privilege to the 'test' user, e.g.,
GRANT FILE ON *.* TO 'test'#'localhost';
I answered this same question here: https://stackoverflow.com/a/25495823/2364773
in short, you need to add a client flag in the connection using the client_flags parameter