We Keep Coding

Problem while trying to insert multiple values to Sqlite database

I have to make a request to a Brazil ZIPCODES API to get JSON data and insert it on a sqlite database using python. I'm currenctly using pycharm but I need to insert a lot of columns, but somehow the code don't insert the values. Here's the code
import requests
import sqlite3
import json
CEPC = input("Please type the zipcode:")
print("Identifying the ZIP CODE")
Requisicao = requests.get(f"https://viacep.com.br/ws/{CEPC}/json")
if Requisicao.status_code == 200:
data = Requisicao.json()
# Database
con = sqlite3.connect("Banco de dados/CEPS.db")
cur = con.cursor()
cur.execute("DROP TABLE IF EXISTS Requisicao")
cur.execute("CREATE TABLE Requisicao (cep, logradouro, bairro, uf, ddd, siafi,
validation, created json)")
cur.executemany("insert into Requisicao values (?, ?, ?, ?, ?, ?, ?, ?)", (data["cep"],
json.dumps(data)))
con.commit()
con.close()
else:
print(f"Request failed with status code {Requisicao.status_code} ")
The outpout of the zipcode is:
{
"cep": "05565-000",
"logradouro": "Avenida General Asdrúbal da Cunha",
"complemento": "",
"bairro": "Jardim Arpoador",
"localidade": "São Paulo",
"uf": "SP",
"ibge": "3550308",
"gia": "1004",
"ddd": "11",
"siafi": "7107"
}
I need to insert all of these columns: "cep, logadouro, complemento, bairro, localidade, uf, ibge, gia, ddd, siafi".When I try to run the code, It gives me the error:
Traceback (most recent call last):
File "C:\Users\Gui\PycharmProjects\pythonProject\main.py", line 19, in <module>
cur.executemany("insert into Requisicao values (?, ?, ?, ?, ?, ?, ?, ?)", (data["cep"],
json.dumps(data)))
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement
uses 8, and there are 9 supplied
When I try to put the exact same value of columns with the "?", the errors says that "uses 8, and there are 7 supplied.

This code will insert all 10 values from the JSON into the table Requisicao and 0 for both validation and created, though that can be changed.
import requests
import sqlite3
import json
CEPC = input("Please type the zipcode:")
print("Identifying the ZIP CODE")
Requisicao = requests.get(f"https://viacep.com.br/ws/{CEPC}/json")
if Requisicao.status_code == 200:
data = Requisicao.json()
# Database
con = sqlite3.connect("CEPS.db")
cur = con.cursor()
cur.execute("DROP TABLE IF EXISTS Requisicao")
cur.execute("CREATE TABLE Requisicao (cep,logradouro,complemento,bairro,localidade,uf,ibge,gia,ddd,siafi, validation, created)")
cur.execute("insert into Requisicao values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",tuple(data.values())+(0, 0))
con.commit()
con.close()
else:
print(f"Request failed with status code {Requisicao.status_code} ")

Writing Csv file to already existing table in SQL Server database using Python

I was trying to insert the CSV file to an already existing table in the SSMS database table. I have a data column in my data. But I keep getting this error when I try to insert data. Please tell me where am I doing it wrong because server connection and extracting data from the database are fine. Below is the code.
with open("combine.csv", encoding="utf8") as f:
csvreader = csv.reader(f)
csvdata = []
for row in csvreader:
csvdata.append(row)
print(csvdata)
for row in csvdata:
# Insert a row of data
print(row)
if len(row)>=8:
data = [row[0],row[1],row[2],row[3],row[4],row[5],row[6],row[7]]
cursor.execute("INSERT INTO BILLING_COPY (DATE, DEPARTMENT_NUMBER, DEPARTMENT_NAME, DIVISION_CODE, DIVISION_NAME, O_T_AMT, R_AMT, U_AMT ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", data)
Error:
File "", line 7, in
cursor.execute("INSERT INTO BILLING_COPY (DATE, DEPARTMENT_NUMBER, DEPARTMENT_NAME, DIVISION_CODE, DIVISION_NAME, O_T_AMT, R_AMT, U_AMT ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", data)
DataError: ('22007', '[22007] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Conversion failed when converting date and/or time from character string. (241) (SQLExecDirectW)')

File "", line 7, in
cursor.execute("INSERT INTO BILLING_COPY (DATE, DEPARTMENT_NUMBER, DEPARTMENT_NAME, DIVISION_CODE, DIVISION_NAME, O_T_AMT, R_AMT, U_AMT ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", data)
I think the data type you mentioned in VALUES(?,?,? etc) is not right valid data type, try using it as %d or %s
Here is some example:
mySql_insert_query = """INSERT INTO Laptop (Id, Name, Price, Purchase_date)
VALUES
(10, 'ProductValues SP99', 6459, '2019-12-27') """
cursor = connection.cursor()
cursor.execute(mySql_insert_query)
connection.commit()
My two cents: Better to assign insert query to a variable just like data variable.

Appending data from excel in existing SQL Server table using python

I have some CSV files with data which is recurring and therefore I need to update SQL Server by using this python script.
I have tried updating the Microsoft driver for SQL and that doesn't help me.
Here is my python code :
import pandas as pd
import numpy as np
import seaborn as sns
import scipy.stats as stats
import matplotlib.pyplot as plt
from datetime import time
from datetime import date
import pandas.io.sql
import pyodbc
import xlrd
server ='asd'
db = 'asd'
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=' + server + ';DATABASE=' + db + ';UID=asd ;PWD=asd')
cursor=conn.cursor()
query = """
INSERT INTO Db.table (
Emp_ID ,
Global_ID,
Emp_NAME,
Org,
SBU,
BU,
Sub_BU,
HR_Location,
Swipe_Loc,
Descp,
InOutDate,
InTime,
OutTime,
ActHrs,
ShiftCode,
AttendanceClassification,
ActualHrs
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"""
InOutDate= date.today()
InTime = time(11,11,11)
OutTime = time(11,11,11)
ActHrs = time(11,11,11)
ActualHrs = time(11,11,11)
values = ('2134123', '123213', 'Eqqwe', 'Org' , 'SBU' , 'BU ', 'Sub_BU' , 'HR_Location' ,'Swipe_Loc' ,' Descp' , InOutDate , InTime , OutTime , ActHrs , 'ShiftCode' ,'AttendanceClassification' ,ActualHrs )
cursor.execute(query, values)
conn.close()
Getting the following error when executing query:
Traceback (most recent call last):
File "update.py", line 97, in <module>
cursor.execute(query, values)
pyodbc.Error: ('HYC00', '[HYC00] [Microsoft][ODBC SQL Server Driver]Optional feature not implemented (0) (SQLBindParameter)')

Make sure the DateTime formats are compatible between python and SQL

You forgot to add cursor.commit() after execute. Execute command can be used only for some selects and read only queries. If you want to change something you shoud add cursor.commit() after.

Python insert to sql server from multiple csv

Newbie here, trying to import from multiple csv to sql server, the code did run but no data inserted into sql database.
Attached is my code. Maybe the error is lie on the loop.
Please help.
import csv
import pyodbc as p
import os
# Database Connection Info
server = "cld-077\eform"
database = "E-form"
username = "wsmeform"
password = "M1loA1s!"
connStr = (
'DRIVER={ODBC Driver 13 for SQL Server};SERVER=' + server + ';DATABASE=' + database + ';UID=' + username + ';PWD=' + password)
# Open connection to SQL Server Table
conn = p.connect(connStr)
# Get cursor
cursor = conn.cursor()
# Assign path to Excel files
print("Inserting!")
folder_to_import = 'C:/Users/ck.law/Desktop/VBFU_NOV/'
print("path")
l_files_to_import = os.listdir(folder_to_import)
print("inside loop")
for file_to_import in l_files_to_import:
if file_to_import.endswith('.csv'):
csv_files = os.path.join(folder_to_import, file_to_import)
csv_data = csv.reader(csv_files)
for row in csv_data:
if len(row) >= 19:
cursor.execute(
"INSERT INTO VesselBFUData(ShortCode,DocDT,PostDT,DocNo,LineItm,GlCode,ExpType,InvRef,VBaseCurrcy,VBaseAmt,DocCurrcy,DocAmt,VendorCode,Description,InvFilePath,InvCreateDT,InvAppvDT,InvArriDT,PoRef)" " VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
row)
print("Loop!")
cursor.close()
conn.commit()
conn.close()
print("Script has successfully run!")

('HY000', 'The SQL contains 21 parameter markers, but 1 parameters were supplied')

I'm trying to read a csv file and upload in SQL. Here is the code. I'm getting this error "pypyodbc.ProgrammingError: ('HY000', 'The SQL contains 21 parameter markers, but 1 parameters were supplied')" My csv file has 21 columns. Do you know how to resolve this issue ?
import csv
import pypyodbc
import traceback
import sys
import time
import os.path
import codecs
# Create a connection to DataBase
con = pypyodbc.connect('DRIVER={SQL Server};SERVER=c1devsql01.XXXXXX.com;DATABASE=Parameters;UID=XXXXXX;PWD=XXXX#1')
cur = con.cursor()
query = "insert into Calc_Rules_Metadata_New (Calc_Set, Calc_Set_Identifier, Dependency, Data_Subset_Keys, Calc_Step, Calc_Variable, Calc_Operator, Calc_Operand, By_Variable, Where_Clause, Source_Tracking_Columns, Source_Tracking_Rows, Revision, Tag, Notes, Updated_By, Updated_On, IsDeleted, Metadata_Type, Calculation_Summary) values(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
filename = str(sys.argv[1])
basedir = 'C:/RiskClient/InputData/Metadata/Calc'
fullpath = os.path.join(basedir, filename)
with open(fullpath, 'r') as csvfile:
next(csvfile) # Skip Header
reader = csv.reader(csvfile, delimiter=',')
for row in reader:
#for i in range(len(row)):
#if row[i] == '':
# row[i] = None
print(row)
cur.execute(query, row)
cur.commit()
Runtime Error:
['1', '1', '1b.B: Question Test 1', '', 'PFA_Unique_Identifier, Fund_Unique_Identifier; Business_Date', '1', 'Total_Borrowing', 'SUM', 'Borrowings_Data.Amount', '', "UPPER(Borrowings_Data.Commitment_Type) IN ('COMMITTED_AND_DRAWN', 'UNCOMMITTED') AND Borrowings_Data.Business_Date = &Rep_Date AND Fund_Unique_Identifier In (select Fund_Unique_Identifier from Fund_Level_Information where Applicable_PF_Sections IS NOT NULL AND PFA_Unique_Identifier = &PFA_UID AND Business_Date = &Rep_Date)", '', '', '', '', '', '', '', '', 'Test Form', '']
Traceback (most recent call last):
File "C:\RiskClient\InputData\Metadata\Calc\CalcMetadata.py", line 47, in <module>
cur.execute(query, row)
File "C:\Program Files\Python3.5.2\lib\site-packages\pypyodbc-1.3.3-py3.5.egg\pypyodbc.py", line 1470, in execute
File "C:\Program Files\Python3.5.2\lib\site-packages\pypyodbc-1.3.3-py3.5.egg\pypyodbc.py", line 1263, in _BindParams
pypyodbc.ProgrammingError: ('HY000', 'The SQL contains 20 parameter markers, but 21 parameters were supplied')

You're providing a single list instead of a full set of positional arguments. When you provide those 21 ? within your query string but just the row in cur.execute(query, row), you're effectively saying to stuff that list into the first ? and haven't provided any parameters for the remaining twenty. To resolve, you need to provide *args like so:
cur.execute(query, *row)
Make sure that your row (i.e. list in this scenario) actually has 21 items in it. Otherwise, you'll still get an error about not providing the correct number of parameters.

Thanks everyone. I fixed the issue by myself. Basically Run Time Error ('HY000', 'The SQL contains 21 parameter markers, but 1 parameters were supplied') comes when csv column count is not same with your insert query parameters. In this case, my cvs file had an Identity Column "Id" Mapped which is not suppose to be in csv file because this coulmn suppose to be generated by SQL Server and autoincremented. Removing that Column from csv file fixes the issue. Thanks.