We Keep Coding

Cannot INSERT strings with Set Clause with mariadb connector (python)

I want to insert a new row in my table by using the python-mariadb connector. For that I prefer to use the SET clause.
For some reason it does work if I only want to save ints (i.e y=2), but when I use a string, the following error occurs
Unknown column 'myString' in 'field list'
It seems it thinks the content of the string is a column name? Any idea how to fix that (I can do it with INSERT INTO ... VALUES ..., but I want to use the SET clause here). From my understanding, it should save both an int and a str without throwing an error
Thank you.
See the code example below
def myfunction():
x = 1
y ='myString'
db = connect_db()
cur = db.cursor()
sql = "INSERT INTO Table SET col1={}, col2={}"
cur.execute(sql.format(x, y))
db.commit()
db.close()
return
Here the MariaDB Connector, but this should be fine as it works for other db functions.
import mariadb
def connect_db():
db = mariadb.connect(
user="user",
password="123",
host="localhost",
port=3306,
database="DB"
)
db.autocommit = False
return db

you are not using right syntax for insert
sql = "INSERT INTO Table (col1,col2) values({}, {})"
but if you want to update an existing row:
sql = "UPDATE Table SET col1={}, col2={} WHERE id = {}"
and probably you need a where clause

The code in question produces the SQL statement:
INSERT INTO Table SET col1=1, col2=myString;
This is incorrect syntax, and strings must be in single-quotes:
INSERT INTO Table (col1, col2) VALUES (1, 'myString');
def myfunction():
x = 1
y ='myString'
db = connect_db()
cur = db.cursor()
sql = "INSERT INTO Table (col1, COL2) VALUES ({}, '{}')"
cur.execute(sql.format(x, y))
db.commit()
db.close()
return
But the above is fragile. Don't use string building methods to create SQL statements, it is much better to use parameter binding.
def myfunction():
x = 1
y ='myString'
db = connect_db()
cur = db.cursor()
sql = "INSERT INTO Table (col1, col2) VALUES (?, ?)"
cur.execute(sql, (x, y))
db.commit()
db.close()
return
The MariaDB connector documentation explains these things.
Retrieving Data
Once you have the initial code in place you can start working with the data. The first thing you should do is try to
retrieve information from the database. Here is code for a query
against the employees database:
cur.execute(
"SELECT first_name,last_name FROM employees WHERE first_name=?",
(some_name,))
MariaDB Connector/Python uses prepared statements, sanitizing and inserting the values from the tuple into the position
of the question marks (?). This is safer than inserting through
f-strings or format specifiers when working with user provided
information.
The query results are stored in a list in the cursor object. To view
the results, you can loop over the cursor.
Adding Data
Using the same execute() method with an INSERT statement, you can add rows to the table.
cursor.execute(
"INSERT INTO employees (first_name,last_name) VALUES (?, ?)",
(first_name, last_name))

Python: SQL Server insert multiple values with datetime (Conversion failed when converting date and/or time from character string)

I'm trying to insert multiple rows in a database table (SQL server) in python:
def write_to_database(values):
conn = pyodbc.connect("Driver={SQL Server};"
"Server=xxxx.database.windows.net;"
f"Database={database};"
"UID=user;"
"PWD=password;")
cursor = conn.cursor()
cursor.executemany("insert into KeyFigures (DateTime, Parameter, CommulativeTime, Value) values (?,?,?,?)", values)
conn.commit()
return()
date = datetime.datetime.now()
atom = date.strftime("%Y-%M-%d") + " " + date.strftime("%H:%M:%S") + ".4526800"
values = ([datetime.datetime.now().isoformat(), 1, "NULL", 2],[datetime.datetime.now().isoformat(), 1, "NULL", 47],[datetime.datetime.now().isoformat(), 1, "NULL", 78])
write_to_database(values)
I tried multiple formats of datetime, string combinations etc. e.g.:
datetime.datetime.now().isoformat()
atom
"2020-02-23 11:30:53.4526800"
"2020-02-23T11:30:53.4526800"
but i keep recieving the same error:
line 50, in write_to_database
cursor.executemany("insert into KeyFigures (DateTime, Parameter, CommulativeTime, Value) values (?,?,?,?)", values)
pyodbc.DataError: ('22007', '[22007] [Microsoft][ODBC SQL Server Driver][SQL Server]Conversion failed when converting date and/or time from character string. (241) (SQLExecDirectW)')
in SSMS the folowing works:
INSERT INTO KeyFigures (DateTime, Parameter, CommulativeTime, Value) VALUES ('2020-02-23 11:30:53.4526800',2,null,21)
how can I solve this error?
***** edit ****
#Mogo thank you very much. this was already very helpfull, but does not work in my code. I still receive the same error. I also tried to insert a single row and this piece of code works (with execute instead of executemany):
def write_to_database(date,parameter,cummulativeTime,value):
conn = pyodbc.connect("Driver={ODBC Driver 17 for SQL Server};"
"Server=xxxx.database.windows.net;"
f"Database={database};"
"UID=user;"
"PWD=password;")
with conn.cursor() as cursor:
cursor.execute(f"INSERT INTO dbo.KeyFigures (DateTime, Parameter, CommulativeTime, Value) values ('{date}',{parameter},{cummulativeTime},{value})")
conn.commit()
return()
date = datetime.datetime.now()
write_to_database(date, 1, "NULL", 43)
it doesnt work without date between quotes. Is this also the problem with the executemany? when i put the questionmark between qoutes ('?' or '?') gives the error that there are only 3 parameters given instead of 4.

As I mentioned in my comment, if you use a strongly type data type (so don't convert it to a string), python and pyodbc will handle this gracefully. I also, however, recommend updating to the ODBC Driver for SQL Server rather than using the old Native SQL Server Driver. I also put the cursor into a with so that it is close gracefully.
For a table I created with the definition below this worked fine, and inserted 2 rows, with the correct date and time values:
CREATE TABLE dbo.TestDateTable (i int,dt datetime2(7));
import datetime, pyodbc
def write_to_database(values):
conn = pyodbc.connect("Driver={ODBC Driver 17 for SQL Server};"
"Server=xxxx.database.windows.net;"
"Database={database};"
"UID=user;"
"PWD=password;")
with conn.cursor() as cursor:
cursor.executemany("INSERT INTO dbo.TestDateTable (i,dt) VALUES (?,?);", values)
conn.commit()
return()
date = datetime.datetime.now()
values = ([1,date],[2,date])
write_to_database(values)

how to take a date as an input and insert it in a table while doing sql connectivity in python

I have tried doing some sql connectivity and faced a problem recently, i could'nt take isert dates into a table. I have given the code below. what should i add in this?
import mysql.connector as con
db = con.connect(host = "localhost", user = "root", password = "root", database = "vish")
if db.is_connected():
print("success")
a = int(input("roll no. : "))
b = input("name : ")
c = int(input("phone : "))
cursor = db.cursor()
q = ("insert into student values({}, '{}', {})".format(a,b,c))
cursor.execute(q)
db.commit()
how do i take date and insert it into a table

If you have a table t and a column c and you wish to insert a date literal, the format for that literal is 'YYYY-MM-DD', so for example:
INSERT INTO t(c) VALUES('2020-01-11')
But whenever you are getting input from the "outside", you are leaving yourself open to SQL Injection attacks and should be using prepared statements. If you have parsed the input date and built a string literal such as '2020-01-11' from the input, then you are safe. But if you have isolated the year, month and day and you are using a prepared statement anyway because there are other values you have not so rigorously validated, you can also use a '?' parameter for the date thus:
from datetime import date
today = date(2020, 1, 11)
cursor = db.cursor()
cursor.execute("INSERT INTO t(c) VALUES(?)", (today,))
cursor.commit()

Ask for user input and insert in DB in python with sqlite3

At the moment I try to ask the user for input in Python, to enter some information (here: expenseID, expense, categoryID, date). But I do not know were to start. No input validation is necessary at this step.
I managed to access my database and INSERT something manually. I tried several ways of the python input function but cannot use it as a placeholder in the SQL string.
import sqlite3
with sqlite3.connect('Expenses.sqlite') as conn:
# INSERT MANUALLY
script = "INSERT INTO Expense (ExpenseId, Amount, CategoryId, Date) VALUES ('103', '43625.5', '5', '2019-01-20');"
conn.execute(script) # execute the script
conn.commit() # commit changes to the file
# INSERT USER INPUT ???
pass
This is my idea:
with sqlite3.connect('Expenses.sqlite') as conn:
amount = input("What is the amount?")
script = "SELECT * FROM Category;"
conn.execute(script)
print(script)
category = input("What is the category?")
exp_ID = "SELECT LAST ExpenseId FROM Expense);"
date = datetime.date.today()
script = "INSERT INTO Expense (ExpenseId, Amount, CategoryId, Date) VALUES (exp_ID, amount, category, date);"
conn.execute(script)
conn.commit()
pass
Finally I want to achieve that the user is asked for amount of expense, and afterwards for expense category. ExpenseID and date should be added automatically. Date format is year-month-day. Thank you very much for advice.

Use the input function to retrieve the user input
user_input = input("Expense Amount: ")
Then use placeholders with sqlite3
sql = "INSERT INTO TABLE (COLUMN) VALUES (?)"
conn.execute(sql, (user_input,))
**in response to your edit
You need to add placeholders instead of the variable names.
Something like this:
script = "INSERT INTO Expense (ExpenseId, Amount, CategoryId, Date) VALUES (?, ?, ?, ?);"
conn.execute(script, (exp_ID,amount,category,date))

psycopg2 postgres database syntax error near value

I am trying to insert info from a pandas DataFrame into a database table by using a function that I wrote:
def insert(table_name="", name="", genere="", year=1, impd_rating=float(1)):
conn = psycopg2.connect("dbname='database1' user='postgres' password='postgres333' host='localhost' port=5433 ")
cur = conn.cursor()
cur.execute("INSERT INTO %s VALUES %s,%s,%s,%s" % (table_name, name, genere, year, impd_rating))
conn.commit()
conn.close()
When I try to use this function like this:
b=0
for row in DF['id']:
insert(impd_rating=float(DF['idbm_rating'][b]),
year=int(DF['year'][b]),
name=str(DF['name'][b]),
genere=str(DF['genere'][b]),
table_name='test_movies')
b = b+1
I get the following syntax error:
SyntaxError: invalid syntax
PS D:\tito\scripts\database training> python .\postgres_script.py
Traceback (most recent call last):
File ".\postgres_script.py", line 56, in <module>insert (impd_rating=float(DF['idbm_rating'][b]),year=int(DF['year'][b]),name=str(DF['name'][b]),genere=str(DF['genere'][b]),table_name='test_movies')
File ".\postgres_script.py", line 15, in insert
cur.execute("INSERT INTO %s VALUES %s,%s,%s,%s"  % (table_name ,name ,genere , year,impd_rating))
psycopg2.ProgrammingError: syntax error at or near "Avatar"
LINE 1: INSERT INTO test_movies VALUES Avatar,action,2009,7.9
I also tried to change the str replacement method from %s to .format()
but I had the same error.

The error message is explicit, this SQL command is wrong at Avatar: INSERT INTO test_movies VALUES Avatar,action,2009,7.9. Simply because values must be enclosed in parenthesis, and character strings must be quoted, so the correct SQL is:
INSERT INTO test_movies VALUES ('Avatar','action',2009,7.9)
But building a full SQL command by concatenating parameters is bad practice (*), only the table name should be directly inserted into the command because is is not a SQL parameter. The correct way is to use a parameterized query:
cur.execute("INSERT INTO %s VALUES (?,?,?,?)" % (table_name,) ,(name ,genere , year,impd_rating)))
(*) It was the cause of numerous SQL injection flaws because if one of the parameter contains a semicolumn (;) what comes after could be interpreted as a new command

Pandas has a DataFrame method for this, to_sql:
# Only needs to be executed once.
conn=psycopg2.connect("dbname='database1' user='postgres' password='postgres333' host='localhost' port=5433 ")
df.to_sql('test_movies', con=conn, if_exists='append', index=False)
This should hopefully get you going in the right direction.

In your original query
INSERT INTO %s VALUES %s,%s,%s,%s
there is a sql problem: you need braces around the values, i.e. it should be VALUES (%s, %s, %s, %s). On top of that the table name cannot be merged as a parameter, or it would be escaped as a string, which is not what you want.
You can use the psycopg 2.7 sql module to merge the table name to the query, with placeholders for the values:
from psycopg2 import sql
query = sql.SQL("INSERT INTO {} VALUES (%s, %s, %s, %s)").format(
sql.Identifier('test_movies'))
cur.execute(query, ('Avatar','action',2009,7.9))
This will make secure both merging the table name and the arguments to the query.

Hello mohamed mahrous,
First install psycopg2 package for the access access PostgreSQL database.
Try this below code,
import psycopg2
conn=psycopg2.connect("dbname='database1' user='postgres' password='postgres333' host='localhost' port=5433 ")
cur=conn.cursor()
def insert(table_name,name,genere,year,impd_rating):
query = "INSERT INTO "+table_name+"(name,genere,year,impd_rating) VALUES(%s,%s,%s,%s)"
try:
print query
cur.execute(query,(name,genere,year,impd_rating))
except Exception, e:
print "Not execute..."
conn.commit()
b=0
for row in DF['id']:
insert (impd_rating=float(DF['idbm_rating'][b]),year=int(DF['year'][b]),name=str(DF['name'][b]),genere=str(DF['genere'][b]),table_name='test_movies')
b= b+1
conn.close()
Example,
import psycopg2
conn=psycopg2.connect("dbname='database1' user='postgres' password='postgres333' host='localhost' port=5433 ")
cur=conn.cursor()
def insert(table_name,name,genere,year,impd_rating):
query = "INSERT INTO "+table_name+"(name,genere,year,impd_rating) VALUES(%s,%s,%s,%s)"
try:
print query
cur.execute(query,(name,genere,year,impd_rating))
except Exception, e:
print "Not execute"
conn.commit()
b=0
for row in DF['id']:
insert (impd_rating="7.0",year="2017",name="Er Ceo Vora Mayur",genere="etc",table_name="test_movies")
b= b+1
conn.close()
I hope my answer is helpful.
If any query so comment please.

i found a solution for my issue by using sqlalchemy and pandas to_sql method
thanks for help everyone
from sqlalchemy import *
import pandas as pd
def connect(user, password, db, host='localhost', port=5433):
'''Returns a connection and a metadata object'''
# We connect with the help of the PostgreSQL URL
# postgresql://federer:grandestslam#localhost:5432/tennis
url = 'postgresql://{}:{}#{}:{}/{}'
url = url.format(user, password, host, port, db)
# The return value of create_engine() is our connection object
con = sqlalchemy.create_engine(url, client_encoding='utf8')
# We then bind the connection to MetaData()
meta = sqlalchemy.MetaData(bind=con, reflect=True)
return con, meta
con, meta = connect('postgres','postgres333','database1')
movies= Table('test',meta,
Column('id',Integer,primary_key=True),
Column('name',String),
Column('genere',String),
Column('year',Integer),
Column('idbm_rating',REAL))
meta.create_all(con)
DF=pd.read_csv('new_movies.txt',sep=' ',engine='python')
DF.columns=('id','name' ,'genere' ,'year' ,'idbm_rating' )
DF.to_sql('movies', con=con, if_exists='append', index=False)