I have the following script:
create_table_WAC = """
create table if not exists %s (
w_geocode text,
C000 text,
CFS04 text,
CFS05 text,
createdate text
)
"""
target_directory = Path(sys.argv[1]).resolve()
for file in target_directory.rglob('*.csv'):
table_name = 'opendata_uscensus_lodes_' + str(file.stem)
print(table_name)
# MAKE SURE THIS IS THE RIGHT TABLE FOR THE FILES
cur.execute(create_table_WAC, (table_name,))
with open(file,'r') as file_in:
# MAKE SURE THIS HAS THE RIGHT TABLE NAME IN THE COPY STATEMENT
cur.copy_expert("copy %s from stdin with csv header delimiter ','", table_name, file_in)
conn.commit()
conn.close()
When I run it, it throws this error related to the CREATE TABLE command. I don't understand why there are '' added -- and how do I remove them?
Here is the error:
psycopg2.ProgrammingError: syntax error at or near "'opendata_uscensus_lodes_ca_wac_SA02_JT03_2003'"
LINE 2: create table if not exists 'opendata_uscensus_lodes_ca_wac_S...
Use SQL string composition:
from psycopg2 import sql
create_table_WAC = """
create table if not exists {} ( -- note changed placeholder
w_geocode text,
C000 text,
CFS04 text,
CFS05 text,
createdate text
)
"""
# ...
cur.execute(sql.SQL(create_table_WAC).format(sql.Identifier(table_name)))
Read the comprehensive explanation in the documentation.
Related
This question already has answers here:
Passing SQLite variables in Python
(1 answer)
How to use variables in SQL statement in Python?
(5 answers)
Closed last month.
I have a small problem with a piece of code, I copied it from a web, but I have the following error:
sqlite3.OperationalError: near "(": syntax error
The code is the following:
# Import required modules
import csv
import sqlite3
# Connecting to the geeks database
connection = sqlite3.connect('isaDBCommune.db')
# Creating a cursor object to execute
# SQL queries on a database table
cursor = connection.cursor()
# Table Definition
create_table = '''CREATE TABLE IF NOT EXISTS isaCommune(
id_codedep_codecommune INTEGER NOT NULL,
nom_commune TEXT NOT NULL,
code_postal INTEGER NOT NULL,
code_commune INTEGER NOT NULL,
code_departement INTEGER NOT NULL,
nom_departement TEXT NOT NULL,
code_region INTEGER NOT NULL
)'''
# Creating the table into our
# database
cursor.execute(create_table)
# Opening the person-records.csv file
file = open('commune.csv')
# Reading the contents of the
# person-records.csv file
contents = csv.reader(file)
# SQL query to insert data into the
# person table
insert_records = "INSERT INTO isaCommune (id_codedep_codecommune, nom_commune, code_postal, code_commune, code_departement, nom_departement, code_region) VALUES ('id_codedep_codecommune', 'nom_commune', 'code_postal', 'code_commune', 'code_departement', 'nom_departement', 'code_region')"
# Importing the contents of the file
# into our person table
cursor.executemany (insert_records, contents)
# SQL query to retrieve all data from
# the person table To verify that the
# data of the csv file has been successfully
# inserted into the table
select_all = "SELECT * FROM isaCommune"
rows = cursor.execute(select_all).fetchall()
What would be the solution? I have searched all over Stack Overflow and I can't find the solution
THX
Any solution ? Or explanation to this error that for me is hidden?
New error with correction ...
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 0, and there are 1 supplied.
This will be your answer:-
import csv
import sqlite3
connection = sqlite3.connect('isaDBCommune.db')
cursor = connection.cursor()
create_table = '''CREATE TABLE IF NOT EXISTS isaCommune(
id_codedep_codecommune TEXT NOT NULL,
nom_commune TEXT NOT NULL,
code_postal TEXT NOT NULL,
code_commune TEXT NOT NULL,
code_departement TEXT NOT NULL,
nom_departement TEXT NOT NULL,
code_region TEXT NOT NULL
)'''
cursor.execute(create_table)
file = open('commune.csv')
contents = csv.reader(file)
for l in contents:
insert_records = """INSERT INTO isaCommune ('id_codedep_codecommune', 'nom_commune', 'code_postal','code_commune','code_departement','nom_departement','code_region')
VALUES(?,?,?,?,?,?,?)"""
a = (l[0],l[1],l[2],l[3],l[4],l[5],l[6],)
cursor.execute(insert_records, a)
select_all = "SELECT * FROM isaCommune"
rows = cursor.execute(select_all).fetchall()
for row in rows:
print(row)
Hope it will work now...
You need to replace the '?' by the value you want to insert in the corresponding column depending on its type INTEGER, TEXT etc..
For example:
insert_records = "INSERT INTO isaCommune VALUES(1, 'test', 1, 1, 1, 'test', 1) ('id_codedep_codecommune', 'nom_commune', 'code_postal', 'code_commune', 'code_departement', 'nom_departement', 'code_region')"
After some data manipulation I store two columns in a txt file in a csv format as following:
result.txt ->
id,avg
0,38.0
1,56.5
3,66.5
4,48.666666666666664
then I store the data in a table, which is where i find trouble, i tried running a .sql query that stores the data successfully, but executing the same query from python doesnt seem to work for some reason.
python code->
.
.
.
open('C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/result.txt', 'w').write(res)
print(res)
try:
with mysql.connector.connect(
host="localhost",
user='root',
password='tt',
database="dp",
) as connection:
clear_table_query = "drop table if exists test_db.marks;"
create_table_query = '''
create table test_db.marks (
id varchar(255) not null,
avg varchar(255) not null,
primary key (id)
);
'''
# droping the table and recreating it works fine
add_csv_query = "LOAD DATA INFILE 'C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/result.txt' INTO TABLE marks FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\\n' IGNORE 1 LINES;"
print(add_csv_query) # query is printed correctly
with connection.cursor() as cursor:
cursor.execute(clear_table_query)
cursor.execute(create_table_query)
cursor.execute(add_csv_query)
cursor.execute("SELECT * FROM test_db.marks;") # this produces -> Unread result found
except mysql.connector.Error as e:
print(e)
connection.close()
Can anybody tell me what's wrong in my program? When I run this program I get the following error message:
syntaxerror unexpected character after line continuation character
import sqlite3
sqlite_file = 'my_first_db.sqlite' # NAME OF THE SQL DATABASE FILE
table_name1 = 'my_table_1' # NAME OF THE TABLE THAT TO BE CREATED.
table_name2 = 'my_table_2' # NAME OF THE SECOND TABLE THAT TO BE CREATED.
new_filed = 'my_1st_coulmn' # NAME OF THE COULMN
filed_type = 'INTEGER' # COULMN DATA TYPE
# CONNECTING TO DATA BASE FILE
conn = sqlite3.connect(sqlite_file)
c = conn.cursor()
# CREATEING NEW SQLITE TABLE WITH 1 COULMN
c.execute('create table {tn} ({nf}) {ft})'\ .format(tn=table_name1,nf=new_filed,ft=filed_type))
# Creating a second table with 1 column and set it as PRIMARY KEY
# note that PRIMARY KEY column must consist of unique values!
c.execute('create table {tn} ({nf}) {ft} primary key)'\.format(tn=table_name2,nf=new_filed,ft=filed_type))
# Committing changes and closing the connection to the database file
conn.commit()
conn.close()
\ is used for line continuation while writing long queries.
so just remove \ before .format() at creation and execution of query if you continued the code after .
# CREATEING NEW SQLITE TABLE WITH 1 COULMN
c.execute('create table {tn} ({nf}) {ft})'.format(tn=table_name1,nf=new_filed,ft=filed_type))
and for more info read this.. https://pyformat.info/
Having problems with the following code. It basically is doing a "show table", replacing a string and executing output string.
The table definition
from macpath import join
import pyodbc
from builtins import print
import logging
import re
import codecs
#pyodbc connection
cnxn = pyodbc.connect('Driver=Teradata;DBCName=192.168.1.103;DATABASE=testdb;UID=xxx;PWD=xxx')
#create cursor
cursor = cnxn.cursor()
try:
cursor.execute("""
show table ENVIRON1.DEPT
""")
except pyodbc.Error as err:
logging.warning(err)
objectlist = cursor.fetchall()
for row in objectlist:
our_str = str(objectlist[0])
our_str = re.sub("ENVIRON1", "ENVIRON2", our_str, flags=re.I)
print(our_str)
try:
cursor.execute(our_str)
except pyodbc.Error as err:
logging.warning(err)
except pyodbc.ProgrammingError as err:
logging.warning(err)
The table definition is as follows:
CREATE SET TABLE ENVIRON1.DEPT ,NO FALLBACK ,
NO BEFORE JOURNAL,
NO AFTER JOURNAL,
CHECKSUM = DEFAULT
(
Key_Id SMALLINT NOT NULL
)
UNIQUE PRIMARY INDEX ( Key_Id );
The output of
print(ourstr)
is
('CREATE SET TABLE ENVIRON2.DEPT ,NO FALLBACK ,\r NO BEFORE JOURNAL,\r NO AFTER JOURNAL,\r CHECKSUM = DEFAULT,\r DEFAULT MERGEBLOCKRATIO\r (\r Key_Id SMALLINT NOT NULL)\rUNIQUE PRIMARY INDEX ( Key_Id );', )
The output of
cursor.execute(our_str)
is
WARNING:root:('42000', "[42000] [Teradata][ODBC Teradata Driver][Teradata Database] Syntax error, expected something like a 'SELECT' keyword or '(' or a 'NONTEMPORAL' keyword or 'AS' keyword between '(' and the string 'CREATE SET TABLE ENVIRON2.DEPT ,NO FALLBACK ,\\r NO BEFORE JOURNAL,\\r NO AFTER JOURNAL,\\r CHECKSUM = DEFAULT. (-3707) (SQLExecDirectW)")
THe problem is with the carriage return \r. The cursor.execute(our_str) statement is taking the carriage return as a literal "\r" causing a syntax error
Any tips on how i can get around this?
When you do
our_str = str(objectlist[0])
you are converting an entire "row" to a string (even though that row may contain only one column), so the string comes out as
('line1\rline2',)
If you extract the first column from the row then you should just get the undecorated string
our_str = objectlist[0][0]
which when printed should look like
line1
line2
Or, instead of fetchall() you could just do fetchone() since it looks like you only expect to retrieve one row anyway. In that case
our_str = objectlist[0]
should suffice.
I am using python to copy one table (dictionary) to another (origin_dictionary) in SQLite, and here is my code to this part:
def copyDictionaryToOrigin(self):
dropTableQueryStr = "DROP TABLE IF EXISTS origin_dictionary"
createTableQueryStr = "CREATE TABLE origin_dictionary (id INTEGER PRIMARY KEY AUTOINCREMENT, word TEXT, type TEXT)"
syncTableQueryStr = "INSERT INTO origin_dictionary (word, type) SELECT word, type FROM dictionary"
self.cur.execute(dropTableQueryStr)
self.cur.fetchone()
self.cur.execute(createTableQueryStr)
result = self.cur.fetchone()
self.cur.execute(syncTableQueryStr)
result = self.cur.fetchone()
With running this code, I can see a origin_dictionary table is created, but there is no data in the table. I could not find out the reason why the data didn't copy over to the new table. can someone please help me with this?
If you need to simply copy one table to another, why don't you use CREATE TABLE ... AS SELECT? Also, you need to commit() your statements.
Simply use code below, and it should work:
import sqlite3
conn = sqlite3.connect(example.db")
cur = conn.cursor()
cur.execute("DROP TABLE IF EXISTS origin_dictionary")
cur.execute("CREATE TABLE origin_dictionary AS SELECT * FROM dictionary")
conn.commit()
conn.close()