I have to read data from Excel and insert it into Table...
For this I am using Python 2.7, pymssql and xlrd modules...
My sql connection is working fine and I am also able to read data from Excel properly.
My table structure :
CREATE TABLE MONTHLY_BUDGET
(
SEQUENCE INT IDENTITY,
TRANSACTION_DATE VARCHAR(100),
TRANSACTION_REMARKS VARCHAR(1000),
WITHDRAWL_AMOUNT VARCHAR(100),
DEPOSIT_AMOUNT VARCHAR(100),
BALANCE_AMOUNT VARCHAR(100)
)
My excel values are like this :
02/01/2015 To RD Ac no 147825000874 7,000.00 - 36,575.74
I am having problem while inserting multiple values in the table... I am not sure how to do this...
import xlrd
import pymssql
file_location = 'C:/Users/praveen/Downloads/OpTransactionHistory03-01-2015.xls'
#Connecting SQL Server
conn = pymssql.connect (host='host',user='user',password='pwd',database='Practice')
cur = conn.cursor()
# Open Workbook
workbook = xlrd.open_workbook(file_location)
# Open Worksheet
sheet = workbook.sheet_by_index(0)
for rows in range(13,sheet.nrows):
for cols in range(sheet.ncols):
cur.execute(
" INSERT INTO MONTHLY_BUDGET VALUES (%s, %s, %s, %s, %s)", <--- Not sure!!!
[(sheet.cell_value(rows,cols))])
conn.commit()
Error :
ValueError: 'params' arg () can be only a tuple or a dictionary.
The docs are here : http://pymssql.org/en/stable/pymssql_examples.html
The exception you are getting says that the "'params' arg() can be only a tuple or a dictionary" but you're passing in a list. Also, your parameter list appears to be a single tuple instead of a list with 4 values. Try changing
cur.execute(
" INSERT INTO MONTHLY_BUDGET VALUES (?, ?, ?, ?, ?)", <--- Not sure!!!
[(sheet.cell_value(rows,cols))])
to
cur.execute(
" INSERT INTO MONTHLY_BUDGET VALUES (?, ?, ?, ?, ?)", <--- Not sure!!!
(sheet.cell_value(rows,cols)))
... or maybe
cur.execute(
" INSERT INTO MONTHLY_BUDGET VALUES (?, ?, ?, ?, ?)", <--- Not sure!!!
((sheet.cell_value(rows,cols))))
NB: untested. I've always changed how the bind variables in your SQL are being called.
Related
I want to add data to my database through python, but I do not know what to do with the ID colum.
I have four colums and I only want to add the last three, the ID is counting up itself.
def add_data(temp, hum):
try:
dt = datetime.datetime.now().replace(microsecond=0).isoformat(' ')
statement = "INSERT INTO messstation (?id?,uhrzeit, luftfeuchtigkeit, raumtemperatur) VALUES (?, ?, ?, ?)"
data = (?id?, dt, hum, temp)
cursor.execute(statement, data)
connection.commit()
except database.error as e:
print(f"Error:{e}")
This should work if the id field is an auto-increment one:
statement = "INSERT INTO messstation (uhrzeit, luftfeuchtigkeit, raumtemperatur) VALUES (?, ?, ?)"
data = (dt, hum, temp)
I want to insert different variable values in SQl where the values must not be present in the table using pyodbc but gets error.
variables
for data in datafield:
id = data.Id
iname = data.name
ivalue = data.value
cursor.execute("INSERT INTO Description (SrId, FieldName, FieldValue) VALUES (?, ?, ?) WHERE IN (SELECT * from Description WHERE SrId <>id AND FieldName<>iname AND FieldValue<>ivalue)", (id, iname, ivalue))
connection.commit()
Using Pyodbc to connect the Sql server with python
Error : Syntax error
In SQL Server, there is no VALUES...WHERE syntax. Consider an insert-select with a LEFT JOIN...NULL on table value constructor to avoid duplicates:
# PREPARED STATEMENT WITH QMARK PLACEHOLDERS
sql = """INSERT INTO Description (SrId, FieldName, FieldValue)
SELECT vals.SrId, vals.FieldName, vals.FieldValue
FROM Description d
LEFT JOIN (VALUES (?, ?, ?)) AS vals(SrId, FieldName, FieldValue)
ON d.SrId = vals.SrId
AND d.FieldName = vals.FieldName
AND d.FieldValue = vals.FieldValue
WHERE d.SrId IS NULL
OR d.FieldName IS NULL
OR d.FielValue IS NULL
"""
# BIND PARAMS
cur.execute(sql, (id, iname, ivalue))
connection.commit()
You have an extra " at the end of the line.
cursor.execute("INSERT INTO Description (SrId, FieldName, FieldValue) VALUES (?, ?, ?) WHERE IN (SELECT * from Description WHERE SrId <>id AND FieldName<>iname AND FieldValue<>ivalue)", (id, iname, ivalue)")
Should be
cursor.execute("INSERT INTO Description (SrId, FieldName, FieldValue) VALUES (?, ?, ?) WHERE IN (SELECT * from Description WHERE SrId <>id AND FieldName<>iname AND FieldValue<>ivalue)", (id, iname, ivalue))
I was trying to insert the CSV file to an already existing table in the SSMS database table. I have a data column in my data. But I keep getting this error when I try to insert data. Please tell me where am I doing it wrong because server connection and extracting data from the database are fine. Below is the code.
with open("combine.csv", encoding="utf8") as f:
csvreader = csv.reader(f)
csvdata = []
for row in csvreader:
csvdata.append(row)
print(csvdata)
for row in csvdata:
# Insert a row of data
print(row)
if len(row)>=8:
data = [row[0],row[1],row[2],row[3],row[4],row[5],row[6],row[7]]
cursor.execute("INSERT INTO BILLING_COPY (DATE, DEPARTMENT_NUMBER, DEPARTMENT_NAME, DIVISION_CODE, DIVISION_NAME, O_T_AMT, R_AMT, U_AMT ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", data)
Error:
File "", line 7, in
cursor.execute("INSERT INTO BILLING_COPY (DATE, DEPARTMENT_NUMBER, DEPARTMENT_NAME, DIVISION_CODE, DIVISION_NAME, O_T_AMT, R_AMT, U_AMT ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", data)
DataError: ('22007', '[22007] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Conversion failed when converting date and/or time from character string. (241) (SQLExecDirectW)')
File "", line 7, in
cursor.execute("INSERT INTO BILLING_COPY (DATE, DEPARTMENT_NUMBER, DEPARTMENT_NAME, DIVISION_CODE, DIVISION_NAME, O_T_AMT, R_AMT, U_AMT ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", data)
I think the data type you mentioned in VALUES(?,?,? etc) is not right valid data type, try using it as %d or %s
Here is some example:
mySql_insert_query = """INSERT INTO Laptop (Id, Name, Price, Purchase_date)
VALUES
(10, 'ProductValues SP99', 6459, '2019-12-27') """
cursor = connection.cursor()
cursor.execute(mySql_insert_query)
connection.commit()
My two cents: Better to assign insert query to a variable just like data variable.
I have 2 things I needed help with:
1) I am unsure as to how I can check if a table exists in python using the sqlite3 library.
2) I am unsure as to how I can save variables from the program to a database. I want to be able to check if UserDetails exists before making the database.
I've been reading around and everyone is doing stuff differently,
Here is the section of my code that is responsible for saving the variables:
connection = sqlite3.connect("UserDetails.db")
crsr = connection.cursor()
#create table
sql_command = table_creator
crsr.execute(sql_command)
#insert values into table
data_to_insert = (username, first_name, surname, age, user_salt, password_hash, date_today)
sql_command = """INSERT INTO UserDetails VALUES ((?, ?, ?, ?, ?, ?, ?), data_to_insert);"""
crsr.execute(sql_command)
connection.commit() #save changes
connection.close() #terminate connection
and in case you want to see table_creator it looks like this:
table_creator = '''CREATE TABLE `UserDetails` (
`Username` VARCHAR(8) NOT NULL,
`Firstname` VARCHAR(10) NOT NULL,
`Surname` VARCHAR(20) NOT NULL,
`Age` INT(2) NOT NULL,
`Salt` VARCHAR(10) NOT NULL,
`Hash` VARCHAR(64) NOT NULL,
`Date` DATE NOT NULL,
PRIMARY KEY (`UserName`)
);'''
I will appreciate and feedback or support.
I am still learning to code, and my CompSci teacher doesnt teach us Python specifically, so what I know is self taught.
Oh and this is the error message I get:
Traceback (most recent call)
File "c:/Users/Arslan/Project A2/login.py", line 99, in <module>
save_details()
File "c:/Users/Arslan/Project A2/login.py", line 93, in save_details
crsr.execute(sql_command)
sqlite3.OperationalError: no such column: data_to_insert
How to check if a table exists or no :
The first way :
Use this query:
SELECT name FROM sqlite_master WHERE type='table' AND name='{table_name}';
Modify {table_name} with your table to check
There are two cases :
. If the cursor equal to 0 ==> the table does not exist
Else, the table exists
The second way:
Use :
PRAGMA table_info(table_name)
example:
The third way :
Use this query :
select 1 from table
It will return the constant 1 for every row of the table if the table exists, or nothing if not.
There are many other ways, but I listed the best in my opinion.
How to save variables from the program to a database:
To insert data into sqlite3, you can use :
cursor.execute("insert into UserDetails values (?, ?, ?, ?, ?, ?, ?)", (username, firstname, surname, age, salt, hash, date))
DON'T USE (SQL injection):
cursor.execute("insert into UserDetails values ('{1}', '{2}', '{3}', '{4}', '{5}', '{6}', '{7}')".format(username, firstname, surname, age, salt, hash, date))
Don't forget :
conn.commit()
Or you can use instead of it the connection as a context manager:
with conn:
# then cursor.execute..
1) I am unsure as to how I can check if a table exists in python using the sqlite3 library.
Use CREATE TABLE IF NOT EXISTS:
table_creator = '''CREATE TABLE IF NOT EXISTS `UserDetails` (
`Username` VARCHAR(8) NOT NULL,
`Firstname` VARCHAR(10) NOT NULL,
...
);'''
2) I am unsure as to how I can save variables from the program to a database.
You can pass variables for insert with the following syntax:
data_to_insert = (username, first_name, surname, age, user_salt, password_hash, date_today)
sql_command = '''INSERT INTO UserDetails VALUES (?, ?, ?, ?, ?, ?, ?)''';
crsr.execute(sql_command, data_to_insert )
I am trying to UPDATE or INSERT if not exist rows into sqlite table.
Can't manage to do this from python level using executemany
my input csv looks like this:
cfthostname,cftshortname,cftenv,cert_time
lx1234.pl.net,lx1234,tst,28/01/2021
plx169.net,plx169,tst,26/03/2021
sp2444445.net,sp2444445,prd,12/06/2021
my db model:
cfthostname,cftshortname,cftenv,cert_time
lx1234.pl.net,lx1234,tst,DD/MM/RRRR
plx169.net,plx169,tst,DD/MM/RRRR
sp2444445.net,sp2444445,prd,DD/MM/RRRR
what i need to do is:
1. UPDATE cert_time column if cfthostname in input csv matches cfthostname in db
2. INSERT all 4 columns if cfthostname does not exists in DB
db_update.py
import sqlite3
import csv
conn = sqlite3.connect("C:\db.sqlite3")
cursor = conn.cursor()
[...]
######---Import to DB---######
cursor.execute("CREATE TABLE IF NOT EXISTS itpassed_host (cfthostname, cftshortname, cftenv, cert_time);")
with open('C:\csv\cfthosts.csv','rt') as fin:
dr = csv.DictReader(fin)
to_db = [(i['cfthostname'], i['cftshortname'], i['cftenv'], i['cert_time']) for i in dr]
cursor.executemany("INSERT INTO itpassed_host (cfthostname, cftshortname, cftenv, cert_time) VALUES (?, ?, ?, ?) ON CONFLICT (cfthostname) DO UPDATE SET cert_time=excluded.cert_time;", to_db)
conn.commit()
conn.close()
i get
sqlite3.OperationalError: near "ON": syntax error
i have sqlite 3.7.17
Use "WHERE True" to avoid this conflict:
cursor.executemany("INSERT INTO itpassed_host (cfthostname, cftshortname, cftenv, cert_time) VALUES (?, ?, ?, ?) **WHERE True** ON CONFLICT (cfthostname) DO UPDATE SET cert_time=excluded.cert_time;", to_db)
UPSERT Parsing Ambiguity