I have written a small script to create a MySQL database, create a table (previously erase it if already exists), and insert many entries. When I execute my script, it works creating the database and table, but does not write any entry to the table:
from warnings import filterwarnings
import MySQLdb as db
filterwarnings('ignore', category = db.Warning)
try:
db_name = 'chom'
con = db.connect(user='user', passwd='pass')
cur = con.cursor()
# Create new database
cur.execute('CREATE DATABASE IF NOT EXISTS ' + db_name + ';')
# Create PARAMETERS table
cur.execute('DROP TABLE IF EXISTS ' + db_name + '.PARAMETERS;')
query = ('CREATE TABLE ' + db_name + '.PARAMETERS ('
'idPARAMETERS INT(10) NOT NULL AUTO_INCREMENT, '
'Param_name VARCHAR(30) NULL DEFAULT NULL, '
'Param_value VARCHAR(255) NULL DEFAULT NULL, '
'Timestamp TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP '
'ON UPDATE CURRENT_TIMESTAMP, '
'User_id VARCHAR(20) NULL DEFAULT NULL, '
'PRIMARY KEY (idPARAMETERS) );'
)
cur.execute(query)
# Insert entries
parameters = ['param1', 'param2', 'param3',
'param4']
for i, param_name in enumerate(parameters, start=1):
cur.execute('INSERT INTO ' + db_name + '.PARAMETERS '
'(idPARAMETERS, Param_name, Param_value, User_id) '
'VALUES (' + str(i) + ', %s, %s, %s);',
(param_name, '', 'user2#localhost'))
cur.close()
con.commit()
except Exception, e:
print 'Error. Last query: ' + str(cur._last_executed)
print e
print 'DB installation script finished'
I can't see where the problem is. Any idea?
The code worked correctly, it was mysql-workbench fault, which was not showing the correct database content (while mysql console client did).
Related
I got an error in the 5th line in my code below at '"+uname+"'.
How can I create a table at runtime?
Here is my code :
name = en1.get()
uname = en2.get()
password = en3.get()
sql = "insert into register values ('" + name + "','" + uname + "','" + password + "')"
CreateTable = "CREATE TABLE '"+uname+"'(no INT AUTO_INCREMENT PRIMARY KEY,title VARCHAR(255),amount INT,date DATE,mode VARCHAR(255))"
try:
cur.execute(sql)
cur1.execute(CreateTable)
con.commit()
con1.commit()
messagebox.showinfo("Success", "Your data is registered successfully!")
except:
messagebox.showinfo("Error inserting", "Please change your username and try.")
In the statements with get(), if you get None as a return value then the below statements will fail
name = en1.get() #-- None
sql = "insert into register values ('" + name + "','" + uname + "','" + password + "')"
CreateTable = "CREATE TABLE '"+uname+"'(no INT AUTO_INCREMENT PRIMARY KEY,title VARCHAR(255),amount INT,date DATE,mode VARCHAR(255))"
Try this instead so that if any of the values is None, then below output will be obtained
without causing the error but still the sql query needs to be handled
sql = "insert into register values ('{}','{}','{}')".format(name, uname, password)
CreateTable = "CREATE TABLE '{}'(no INT AUTO_INCREMENT PRIMARY KEY,title VARCHAR(255),amount INT,date DATE,mode VARCHAR(255))".format(name)
print("sql is ", sql)
print("sql is ", CreateTable)
# Output
sql is insert into register values ('None','None','None')
sql is CREATE TABLE 'None'(no INT AUTO_INCREMENT PRIMARY KEY,title VAR
CHAR(255),amount INT,date DATE,mode VARCHAR(255))
I am trying to insert a row into my postgresql database with a table created from
CREATE TABLE public.coinbase_btc_usd
(
id bigserial primary key,
price integer NOT NULL,
buy integer NOT NULL,
sell integer NOT NULL,
"timestamp" timestamp with time zone
)
However when my python 3.6 script runs and tries to add a row using psycopg2 like this it returns an error saying "no results to fetch" and nothing is added to my db.
sql_query = "INSERT INTO coinbase_btc_usd(price, buy, sell, timestamp)" \
" VALUES (" + exchange_rate + ', ' + buy_rate + ', ' + sell_rate + ", \'2015-10-10 06:44:33.8672177\')"
print(sql_query)
cur.execute(sql_query)
I also printed the sql_query variable to see exactly what was getting attempted to execute and this was printed to the output
INSERT INTO coinbase_btc_usd(price, buy, sell, timestamp) VALUES (16392.10, 16563.40, 16235.42, '2015-10-10 06:44:33.8672177')
Make sure that you are committing the transaction:
cur.execute(sql_query)
conn.commit()
Or you can enable auto commit to commit each query immediately after execution:
conn.autocommit = True
Furthermore, it costs nothing to prevent SQL injection attack - just use parametersied queries. In fact your code will actually be cleaner as well as safer:
sql_query = "INSERT INTO coinbase_btc_usd(price, buy, sell, timestamp) VALUES (%s, %s, %s, %s)"
cur.execute(sql_query, (exchange_rate, buy_rate, sell_rate, timestamp))
conn.commit()
change the
sql_query = "INSERT INTO coinbase_btc_usd(price, buy, sell, timestamp)" \
" VALUES (" + exchange_rate + ', ' + buy_rate + ', ' + sell_rate + ", \'2015-10-10 06:44:33.8672177\')"
to:
sql_query = "INSERT INTO coinbase_btc_usd(price, buy, sell, timestamp)" \
" VALUES (" + exchange_rate + ', ' + buy_rate + ', ' + sell_rate + ", \'2015-10-10 06:44:33.8672177\') returning *"
this should fix no results to fetch in my assumption.
If you see no row added, you most probably begin transaction and never commit it.
I have tried the following, but for some reason the #check parameter gets set to the value of the constraint I'm trying to check for even if the constraint doesn't exist.
Any ideas why?
CREATE PROCEDURE AddForeignKey(
IN constraint_name varchar(64),
IN foreign_key_column_name varchar(64),
IN table_name varchar(64),
IN database_name varchar(64),
IN foreign_table_key_name varchar(64),
IN foreign_table_name varchar(64))
BEGIN
set #sql = CONCAT('ALTER TABLE ', database_name, '.', table_name,
' ADD CONSTRAINT ', constraint_name,
' FOREIGN KEY (', foreign_key_column_name, ')'
' REFERENCES ', foreign_table_name, ' (', foreign_table_key_name, ');' ) ;
set #dbname = database_name;
set #fkname = constraint_name;
set #check = '';
SELECT
CONSTRAINT_NAME
INTO
#check
FROM
INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE
CONSTRAINT_SCHEMA = #dbname
AND
CONSTRAINT_NAME = #fkname
AND
CONSTRAINT_TYPE = 'FOREIGN KEY'
LIMIT 1;
IF #check != #fkname
THEN
prepare stmt from #sql;
execute stmt;
ELSE
insert into errors(message) values (CONCAT ( 'Check: ', #check, ' DB:', #dbname, ' FK:', #fkname, ' Could not execute statement: ', #sql));
END IF;
END
You are only compare the column CONSTRAINT_NAME that is not enough to prevent a foreign key duplicate creation.
You must add REFERENTIAL_CONSTRAINTS table and add more columns to compare.
Also I changed the variable names to avoid unnecessary variable names:
CREATE PROCEDURE AddForeignKey(
IN p_constraint_name varchar(64),
IN foreign_key_column_name varchar(64),
IN p_table_name varchar(64),
IN p_database_name varchar(64),
IN foreign_table_key_name varchar(64),
IN foreign_table_name varchar(64))
BEGIN
set #sql = CONCAT('ALTER TABLE `', p_database_name, '`.`', p_table_name,
'` ADD CONSTRAINT `', p_constraint_name,
'` FOREIGN KEY (`', foreign_key_column_name, '`)'
' REFERENCES `', foreign_table_name, '` (`', foreign_table_key_name, '`);' ) ;
set #check = '';
SELECT
TABLE_CONSTRAINTS.CONSTRAINT_NAME
INTO #check
FROM
INFORMATION_SCHEMA.TABLE_CONSTRAINTS
INNER JOIN INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS
ON TABLE_CONSTRAINTS.CONSTRAINT_SCHEMA = REFERENTIAL_CONSTRAINTS.CONSTRAINT_SCHEMA
AND TABLE_CONSTRAINTS.CONSTRAINT_NAME = REFERENTIAL_CONSTRAINTS.CONSTRAINT_NAME
AND TABLE_CONSTRAINTS.TABLE_NAME = REFERENTIAL_CONSTRAINTS.TABLE_NAME
WHERE
TABLE_CONSTRAINTS.CONSTRAINT_SCHEMA = p_database_name
AND TABLE_CONSTRAINTS.CONSTRAINT_NAME = p_constraint_name
AND TABLE_CONSTRAINTS.CONSTRAINT_TYPE = 'FOREIGN KEY'
AND TABLE_CONSTRAINTS.TABLE_NAME = p_table_name
and REFERENTIAL_CONSTRAINTS.REFERENCED_TABLE_NAME = foreign_table_name
LIMIT 1;
IF #check <> p_constraint_name
THEN
prepare stmt from #sql;
execute stmt;
ELSE
insert into errors(message) values (CONCAT ( 'Check: ', #check, ' DB:', #dbname, ' FK:',
#fkname, ' Could not execute statement: ', #sql));
END IF;
END
I am trying to insert values to columns in a table of a sqlite3 DB
as follows:
c.execute("""CREATE TABLE IF NOT EXISTS keys(name TEXT, po TEXT, options TEXT, identifier TEXT, currtime TEXT)""")
c.execute('INSERT INTO keys VALUES ( ' + Customer_Name + ' , ' + Purchase_Order + ' , ' + options + ' , ' + UUID + ' , ' + currtime + ' )')
These arguments are actually passed from HTML front end. I convert them from unicode to string as follows:
options = str(request.form.get("Software_Options"))
UUID = str(request.form.get("UUID_Identifier"))
Customer_Name = str(request.form.get("Customer_Name"))
Purchase_Order = str(request.form.get("Purchase_Order"))
currtime= str(datetime.datetime.now())[:10]
If the are integers there is no error.
If the input is a string ( e.g. joseph) it gives the following error. Any idea why?
OperationalError: no such column: joseph
You seem to be obtaining the contents for one database record at a time from the web front-end. That being the case, you can build a record as a list and insert it in the database in the following way.
import sqlite3
conn = sqlite3 . connect ( 'db.sqlite' )
c = conn . cursor ( )
c.execute("""CREATE TABLE IF NOT EXISTS keys(name TEXT, po TEXT, options TEXT, identifier TEXT, currtime TEXT)""")
oneRecord = [ 'joe', 'DA123', 'nilbymouth', '0034', '1653', ]
c.execute('INSERT INTO keys VALUES (?,?,?,?,? )', oneRecord)
conn.close()
The use of this type of construction, involving the '?' character, is also considered safer that others against injection attacks.
my stored procedure works fine on its own but my python script fails to fully execute the stored procedure with my downloaded files. the purpose of the python script is to download files using ftp and store the files locally.It first compares the remote location and the local location to find new files and then download the new files to the local location. And then executes the stored procedure on each new file.
python script:
import os
import ftplib
import pyodbc
connection to sql server*
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=localhost;DATABASE=Development;UID=myid;PWD=mypassword')
cursor = conn.cursor()
ftp = ftplib.FTP("myftpaddress.com")
ftp.login("loginname", "password")
print 'ftp on'
#directory listing
rfiles = ftp.nlst()
print 'remote listing'
#save local directory listing to files
lfiles = os.listdir(r"D:\Raw_Data\myFiles")
print 'local listing'
#compare and find files in rfiles but not in lfiles
nfiles = set(rfiles) - set(lfiles)
nfiles = list(nfiles)
print 'compared listings'
#loop through the new files
#download the new files and open each file and run stored proc
#close files and disconnect to sql server
for n in nfiles:
local_filename = os.path.join(r"D:\Raw_Data\myFiles",n)
lf = open(local_filename, "wb")
ftp.retrbinary("RETR " + n, lf.write, 1024)
lf.close()
print 'file written'
cursor.execute("exec SP_my_Dailyfiles('n')")
conn.close()
lf.close()
print 'sql executed'
ftp.quit()
stored proc:
ALTER PROCEDURE [dbo].[SP_my_Dailyfiles]
-- Add the parameters for the stored procedure here
#file VARCHAR(255)
-- Add the parameters for the stored procedure here
AS
BEGIN
IF EXISTS(SELECT * FROM sysobjects WHERE name = 'myinvoice')
DROP TABLE dbo.myinvoice
----------------------------------------------------------------------------------------------------
CREATE TABLE myinvoice(
[Billing] varchar(255)
,[Order] varchar(45)
,[Item] varchar(255)
,[Quantity in pack] varchar(255)
,[Invoice] varchar(255)
,[Date] varchar(255)
,[Cost] varchar(255)
,[Quantity of pack] varchar(255)
,[Extended] varchar(255)
,[Type] varchar(25)
,[Date Due] varchar(255)
)
----------------------------------------------------------------------------------------------------
DECLARE #SourceDirectory VARCHAR(255)
DECLARE #SourceFile VARCHAR(255)
EXEC (' BULK
INSERT dbo.myinvoice
FROM ''D:\Raw_Data\myfile\'+#file+'''
WITH
(
FIRSTROW = 1,
FIELDTERMINATOR = '','',
ROWTERMINATOR = ''0x0a''
)'
)
-------------------------------------------------------------------------------------------------------------
INSERT INTO [Development].[dbo].[my_Dailyfiles](
[Billing]
,[Order]
,[Item]
,[Quantity in pack]
,[Invoice]
,[Date]
,[Cost]
,[Quantity of pack]
,[Extended]
,[Type]
,[Date Due]
,[FileName]
,[IMPORTEDDATE]
)
SELECT
replace([Billing], '"', '')
,replace([Order], '"', '')
,replace([Item], '"','')
,replace([Quantity in pack],'"','')
,replace([Invoice],'"','')
,cast(replace([Date],'"','') as varchar(255)) as date
,replace([Cost],'"','')
,replace([Quantity of pack],'"','')
,replace([Extended],'"','')
,replace([Type],'"','')
,cast(replace([Date Due],'"','') as varchar(255)) as date
,#file,
GetDate()
FROM [myinvoice] WHERE [Bill to] <> ' ' and ndc != '"***********"'
I think the problem may be that you are closing the DB connection immediately after you execute the stored procedure, whilst still in the loop.
This means the second time around the loop, the DB connection is closed when you try to execute the SP. I would actually expect an error to be thrown the second around the loop.
The way I would structure this is something like:
conn = pyodbc.connect(...)
for n in nfiles:
...
cursor = conn.cursor()
cursor.execute("exec SP_my_Dailyfiles('n')")
conn.commit()