I have a trouble with my program. I want input database from file txt. This is my source code
import MySQLdb
import csv
db=MySQLdb.connect(user='root',passwd='toor',
host='127.0.0.1',db='data')
cursor=db.cursor()
csv_data=csv.reader(file('test.txt'))
for row in csv_data:
sql = "insert into `name` (`id`,`Name`,`PoB`,`DoB`) values(%s,%s,%s,%s);"
cursor.execute(sql,row)
db.commit()
cursor.close()
After run that program, here the error
Traceback (most recent call last):
File "zzz.py", line 9, in <module>
cursor.execute(sql,row)
File "/home/tux/.local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 187, in execute
query = query % tuple([db.literal(item) for item in args])
TypeError: not enough arguments for format string
and this is my test.txt
4
zzzz
sby
2017-10-10
Please help, and thanks in advance.
Now that you have posted the CSV file, the error should now be obvious to you - each line contains only one field, not the four that the SQL statement requires.
If that is the real format of your data file, it is not CSV data. Instead you need to read each group of four lines as one record, something like this might work:
LINES_PER_RECORD = 4
SQL = 'insert into `name` (`id`,`Name`,`PoB`,`DoB`) values (%s,%s,%s,%s)'
with open('test.txt') as f:
while True:
try:
record = [next(f).strip() for i in range(LINES_PER_RECORD)]
cursor.execute(SQL, record)
except StopIteration:
# insufficient lines available for record, treat as end of file
break
db.commit()
Related
I'm trying to get an admin account to edit a 'rank' (basically access level) for one of the profiles in my data-base. The error is:
Traceback (most recent call last):
File "U:/A-level Computor Science/Y12-13/SQL/sqlite/Databases/ork task/Python for SQL V_2.py", line 154, in <module>
main()
File "U:/A-level Computor Science/Y12-13/SQL/sqlite/Databases/ork task/Python for SQL V_2.py", line 9, in main
start_menu()
File "U:/A-level Computor Science/Y12-13/SQL/sqlite/Databases/ork task/Python for SQL V_2.py", line 22, in start_menu
login()
File "U:/A-level Computor Science/Y12-13/SQL/sqlite/Databases/ork task/Python for SQL V_2.py", line 72, in login
Mek_menu()
File "U:/A-level Computor Science/Y12-13/SQL/sqlite/Databases/ork task/Python for SQL V_2.py", line 108, in Mek_menu
where Uzaname = %s""" % (NewRank, Findaname))
sqlite3.OperationalError: unrecognized token: "0rk_D4T4B453"`
The code that seems to be the problem is:
cursor.execute(""" update 0rk_D4T4B453.Da_Boyz
set Rank = %s
where Uzaname = %s""" % (NewRank, Findaname))
Originally, it was all on one line and it didn't work, and now I've tried it on multiple lines and it still doesn't work. So I checked here to see if anyone could help.
EDIT1: Thanks for the suggestions. None of them have fixed the code, but I've narrowed the problem code to: where Uzaname = %s""" % (NewRank, Findaname))
Unless you use ATTACH, SQLite (a file-level database) does not recognize other databases. Usually server-level databases (Oracle, Postgres, SQL Server, etc.) use the database.schema.table reference. However, in SQLite the very database file you connect to is the main database in scope. But ATTACH allows you to connect to other SQLite databases and then recognizes database.table referencing.
Additionally, for best practices:
In sqlite3 and any other Python DB-APIs, use parameterization for literal values and do not format values to SQL statement.
In general Python, stop using the de-emphasized (not deprecated yet) string modulo operator, %. Use str.format or more recent F-string for string formatting. But neither is needed here.
Altogether, if you connect to the 0rk_D4T4B453 database, simply query without database reference:
conn = sqlite3.connect('/path/to/0rk_D4T4B453.db')
cursor = conn.cursor()
# PREPARED STATEMENT WITH QMARK PLACEHOLDERS
sql = """UPDATE Da_Boyz
SET Rank = ?
WHERE Uzaname = ?"""
# BIND WITH TUPLE OF PARAMS IN SECOND ARG
cursor.execute(sql, (NewRank, Findaname))
conn.commit()
If you do connect to a different database, call ATTACH. Here also, you can alias other database with better naming instead of number leading identifier.
cursor.execute("ATTACH '/path/to/0rk_D4T4B453.db' AS other_db")
sql = """UPDATE other_db.Da_Boyz
SET Rank = ?
WHERE Uzaname = ?"""
cursor.execute(sql, (NewRank, Findaname))
conn.commit()
cur.execute("DETACH other_db")
I’m trying to INSERT INTO / ON DUPLICATE KEY UPDATE taking the values from one table and inserting into another. I have the following Python code.
try:
cursor.execute("SELECT LocationId, ProviderId FROM CQCLocationDetailsUpdates")
rows = cursor.fetchall()
for row in rows:
maria_cnxn.execute('INSERT INTO CQCLocationDetailsUpdates2 (LocationId, ProviderId) VALUES (%s,%s) ON DUPLICATE KEY UPDATE ProviderId = VALUES(%s)', row)
mariadb_connection.commit()
except TypeError as error:
print(error)
mariadb_connection.rollback()
If I change this script just to INSERT INTO it work fine, the problem seems to be when I add the ON DUPLICATE KEY UPDATE. What do I have wrong? LocationId is the PRIMARY KEY
I get this error.
Traceback (most recent call last):
File "C:/Users/waynes/PycharmProjects/DRS_Dev/CQC_Locations_Update_MariaDB.py", line 228, in <module>
maria_cnxn.execute('INSERT INTO CQCLocationDetailsUpdates2 (LocationId, ProviderId) VALUES (%s,%s) ON DUPLICATE KEY UPDATE ProviderId = VALUES(%s)', row)
File "C:\Users\waynes\PycharmProjects\DRS_Dev\venv\lib\site-packages\mysql\connector\cursor.py", line 548, in execute
stmt = RE_PY_PARAM.sub(psub, stmt)
File "C:\Users\waynes\PycharmProjects\DRS_Dev\venv\lib\site-packages\mysql\connector\cursor.py", line 79, in __call__
"Not enough parameters for the SQL statement")
mysql.connector.errors.ProgrammingError: Not enough parameters for the SQL statement
Your error is because row is a 2 element tuple and your SQL statement requires three %s vars.
It is however possible to use an INSERT .. SELECT .. ON DUPLICATE KEY like:
maria_cnxn.execute('INSERT INTO CQCLocationDetailsUpdates2 (LocationId,
ProviderId)
SELECT LocationId, ProviderId
FROM CQCLocationDetailsUpdates orig
ON DUPLICATE KEY UPDATE CQCLocationDetailsUpdates2.ProviderID = orig.ProviderID')
Whenever you end up doing a loop around a SQL statement you should look to see if there is a SQL way of doing this.
I am doing this all as a test. I want to take a csv file that has headers and copy the values into a postgresql database table. The tables columns are named the same as the headers in csv file case-sensitive. table has two columns "pkey", "m". the csv just has the "m" for header. pkey is just the primary key setup to auto increment. As a test i just want to copy the "m" column in the csv file the table.
import csv
import psycopg2
database = psycopg2.connect ( database = "testing", user="**",
password="**", host="**", port="**")
ocsvf = open("sample.csv")
def merger(conn, table_name, file_object):
cursor = conn.cursor()
cursor.copy_from(file_object, table_name, sep=',', columns=('mls'))
conn.commit()
cursor.close()
try:
merger(database, 'tests', ocsvf)
finally:
database.close()
when i try to run the code i get this as a error
Traceback (most recent call last):
File "csvtest.py", line 26, in <module>
merger(database, 'tests', ocsvf)
File "csvtest.py", line 21, in merger
cursor.copy_from(file_object, table_name, sep=',', columns=('m'))
psycopg2.ProgrammingError: column "m" of relation "tests" does not exist
I am sure its something simple that i just keep over looking but i have also googled this and the one thing i found was someone said it might be the primary key is setup right but i tested it and the primary keys works fine when i do manual input from pgadmin. any help would be great thanks
In this line:
cursor.copy_from(file_object, table_name, sep=',', columns=('mls'))
The ('mls') evaluated to "mls" which eventually means that iterating over it will result in 3 items ['m','l','s'].
You should write this line as follows:
cursor.copy_from(file_object, table_name, sep=',', columns=('mls',))
The expression ('mls',) evaluated to a tuple with one item: "mls", which is what I guess you meant to do.
I'm getting the following error:
Traceback (most recent call last):
File "/home/pi/Nike/test_two.py", line 43, in <module>
do_query()
File "/home/pi/Nike/test_two.py", line 33, in do_query
for(Product,Bin,Size,Color) in records:
ValueError: too many values to unpack
Code:
def do_query():
connection = sqlite3.connect('test_db.db')
cursor = connection.cursor()
cursor.execute("SELECT * FROM TESTER ORDER BY CheckNum")
records = cursor.fetchall()
for(Product,Bin,Size,Color) in records:
row_1.append(Product)
row_2.append(Bin)
row_3.append(Size)
row_4.append(Color)
connection.commit()
cursor.close()
connection.close()
do_query()
I'm trying to load each column of a table into seperate python list. I am using Python, and sqlite3. Why am I getting this error?
You are using "SELECT *" which will return every column from the table. My guess is that the table in question contains more columns then the 4 you specified.
A better way would actually be specifying in the SQL which columns you want so that your code will not break if columns are added to the database.
Something like "SELECT col1, col2 FROM table"
You can run the sqlite3 tool on the db file and then view the table schema with ".schema <table_name>"
When I execute the code (see below) I got the next error
Traceback (most recent call last):
File "/home/test/python/main.py", line 18, in <module>
calls = tel.getCallRecordings(member['id'], member['start_date'])
File "/home/test/python/teldb.py", line 49, in getCallRecordings
'start_date': str(start_date),
File "/usr/local/lib/python2.5/site-packages/MySQL_python-1.2.3-py2.5-freebsd-8.1-RELEASE-amd64.egg/MySQLdb/cursors.py", line 159, in execute
TypeError: int argument required
The code I use is the next:
def getCallRecordings(self, member_id, start_date):
self.cursor.execute("""
SELECT var10 as filename,
var9 as duration
FROM csv_data c
LEFT JOIN transcriptions t ON
c.id=t.call_id
WHERE member_id=%(member_id)s AND
var10 IS NOT NULL AND
var9>%(min_duration)d AND
dialed_date>STR_TO_DATE(%(start_date)s, '%%Y-%%m-%%d %%H:%%i:%%s') AND
t.call_id IS NULL
ORDER BY dialed_date DESC
""", {
'member_id': member_id,
'min_duration': MIN_DURATION,
'start_date': str(start_date),
})
logging.debug("Executed query: %s" % self.cursor._executed)
return self.cursor.fetchone()
Why I got this error? Thanks.
You don't show what MIN_DURATION is, but you use %(min_duration)d, so it is required to be an int. Do you have it defined as a string or a float instead?
You don't need to use %d in a SQL query like this, the DB adapter understands the types you pass it, and will properly insert them into the query. Make MIN_DURATION an integer (perhaps with int(MIN_DURATION) in the code you have), and it will work.
BTW: the stack trace focuses you on the str(start_date), but only because that's the last source line in the executable statement that had the error, so it's misleading.
I have faced a similar issue while constructing the sql query strings involving str_to_date function. I have used .format() operator on python strings to parametrize the SQL queries. Here is an example query that would work just fine:
# Takes an input date and returns the manager's name during that period
input_string='"' + "{input_data}".format(input_data=input_data) + '"'
sql_query='''select CONCAT(e.last_name, ",", e.first_name),STR_TO_DATE(from_date,'%Y-%m-%d'),STR_TO_DATE(to_date,'%Y-%m-%d') from dept_manager dm inner join employees e on e.emp_no=dm.emp_no where STR_TO_DATE({input_string},'%Y-%m-%d')>STR_TO_DATE(from_date,'%Y-%m-%d') and STR_TO_DATE({input_string},'%Y-%m-%d')<STR_TO_DATE(to_date,'%Y-%m-%d');'''.format(input_string=input_string)