We Keep Coding

Issue while trying to select record in mysql using Python

Error Message
You have an error in your SQL syntax; check the manual that
corresponds to your MariaDB server version for the right syntax to use
near '%s' at line 1
MySQL Database Table
CREATE TABLE `tblorders` (
`order_id` int(11) NOT NULL,
`order_date` date NOT NULL,
`order_number` varchar(50) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
ALTER TABLE `tblorders`
ADD PRIMARY KEY (`order_id`),
ADD UNIQUE KEY `order_number` (`order_number`);
ALTER TABLE `tblorders`
MODIFY `order_id` int(11) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=4;
Code
mydb = mysql.connector.connect(host = "localhost", user = "root", password = "", database = "mydb")
mycursor = mydb.cursor()
sql = "Select order_id from tblorders where order_number=%s"
val = ("1221212")
mycursor.execute(sql, val)
Am I missing anything?

You must pass a list or a tuple as the arguments, but a tuple of a single value is just a scalar in parentheses.
Here are some workarounds to ensure that val is interpreted as a tuple or a list:
sql = "Select order_id from tblorders where order_number=%s"
val = ("1221212",)
mycursor.execute(sql, val)
sql = "Select order_id from tblorders where order_number=%s"
val = ["1221212"]
mycursor.execute(sql, val)
This is a thing about Python that I always find weird, but it makes a kind of sense.

In case you want to insert data you have to modify your SQL. Use INSERT instead of SELECT like this:
INSERT INTO tblorders (order_number) VALUES ("122121");
That statement will add new record to the table. Besides, in MariaDB you need to use ? instead of %s that works on Mysql database.
sql = "INSERT INTO tblorders (order_number) VALUES (?);"
val = "1231231"
mycursor.execute(sql, [val])

Postgresql database error: column does not exist

I'm using Postrgesql via python, but I'm getting an error on the following insertion, alleging that 'column "\ufeff61356169" does not exist'.
c_id = "\ufeff61356169"
chunk = "engine"
query = f"""
INSERT INTO company_chunks(company_id, chunk)
VALUES(`{c_id}`, `{chunk}`);
"""
c.execute(query)
>>>
DatabaseError: {'S': 'ERROR', 'V': 'ERROR', 'C': '42703', 'M': 'column "\ufeff61356169" does not exist', 'P': '88', 'F': 'parse_relation.c', 'L': '3514', 'R': 'errorMissingColumn'}
One key note: "\ufeff61356169" is the value which is to be inserted into the column. So the error confuses me. It's confusing the insertion value for the column, which should receive the insertion. Any thoughts?
Just to verify that everything else is in working order I made sure to check that my table was successfully created.
query = """
SELECT column_name
FROM information_schema.columns
WHERE table_name = 'company_chunks';
"""
c.execute(query)
c.fetchall()
>>>
(['company_id'], ['chunk'])
So the table does exist and it has the columns, which I'm trying to make insertions to. Where am I going wrong here?
Btw, I'm connecting to this database, which is stored in GCP, via the Cloud SQL Python Connector. However, this connector was able to create the table, so I believe the problem is specific to python syntax and/or Postgres.
Edit: For the sake of understanding what this table looks like, here's the creation query.
query= """
CREATE TABLE company_chunks
(
company_id VARCHAR(25) NOT NULL,
chunk VARCHAR(100) NOT NULL
);
"""
c.execute(query)
conn.commit()

better to do it this way by using %s placeholder:
sql = "INSERT INTO company_chunks(company_id, chunk) VALUES (%s, %s)"
var = (c_id,chunk)
mycursor.execute(sql,var)

Compare data between MongoDB and MySQL using python script

I am working on a Django Application that uses both MySQL and MongoDB to store its data. What I need to do is to compare the data that are stored in the MongoDB's collection and stored in the MySQL's table.
For example, my MySQL database contains the table "relation", which is created using:
CREATE TABLE relations (service_id int, beneficiary_id int, PRIMARY KEY (service_id, beneficiary_id));
My MongoDB contains a collection called "relation", which is expected to store the same data as the relation table in MySQL. The following is one document of the collection "relation":
{'_id': 0, 'service_id': 1, 'beneficiary_id': 32}
I tried to create a python script that compares the data between the relation table in MySQL and relation collection in Mongo. The script works as the following:
mysql_relations = Relations.objects.values('beneficiary_id', 'service_id')
mongo_relations_not_in_mysql = relations_mongodb.find({'$nor':list(mysql_relations)})
mongo_relations = relations_mongodb.find({}, {'_id': 0, 'beneficiary_id':1, 'service_id': 1})
filter_list = Q()
for mongo_relation in mongo_relations:
filter_list &= Q(mongo_relation)
mysql_relations_not_in_mongo = Relations.objects.exclude(filter_list)
However, this code takes forever.
I think the main problem is because of the primary key that is composed of 2 columns, which required the usage of the Q() and the '$nor'.
What do you suggest?

Just in case someone is interested, I used the following solution to optimize the data comparison.
(The Idea was to create a temporary MySQL Table to store mongo's data, then doing the comparison between the the MySQL tables). The code is below:
Get the relations From MongoDB
mongo_relations = relations_mongodb.find({}, {'_id': 0, 'service_id': 1, 'beneficiary_id': 1})
Create a temporary MySQL table to store MongoDB'S relations
cursor = connection.cursor()
cursor.execute(
"CREATE TEMPORARY TABLE temp_relations (service_id int, beneficiary_id int, INDEX `id_related` (`service_id`, `beneficiary_id`) );"
)
Insert MongoDB's relations not the temporary table just created
cursor.executemany(
'INSERT INTO temp_relations (service_id, beneficiary_id) values (%(service_id)s, %(beneficiary_id)s) ',
list(mongo_relations)
)
Get the MongoDB's relations that does not exist in MySQL
cursor.execute(
"SELECT service_id, beneficiary_id FROM temp_relations WHERE (service_id, beneficiary_id) NOT IN ("
"SELECT service_id, beneficiary_id FROM relations);"
)
mongo_relations_not_in_mysql = cursor.fetchall()
Get MySQL relations that does not exist in MongoDB
cursor.execute(
"SELECT id, service_id, beneficiary_id, date FROM relations WHERE (service_id, beneficiary_id) not IN ("
"SELECT service_id, beneficiary_id FROM temp_relations);"
)
mysql_relations_not_in_mongo = cursor.fetchall()
cursor.close() # Close the connection to MySQL

How do I insert a dataframe to an oracle table illegal/variable oracle

I have the following dataframe
df2 = {names: [PEPE, LUIS], id: [1,2], stages: [0,1], ord: [3, 2]}
but these are the required fields, the table to insert, you have more fields, which allow nulls.
And where the dataframe is equivalent to these fields in the table
df2 = {labels= :1, id= :2, stages= :3, ord= :4}
The table
CREATE TABLE customer_prf
(
names VARCHAR2(80) NOT NULL,
label VARCHAR2(80) NOT NULL,
type VARCHAR2(80) NOT NULL,
type_flag INT,
type_flag2 INT,
conc VARCHAR2(80),
id INT NOT NULL,
n_stage INT NOT NULL,
ctry INT NOT NULL,
"order" INT NOT NULL
);
How did you manage to insert the data from the data with the insert with values ​​already previously defined in the query string, which I am doing wrong, which ora-01036 sends me. Besides, does the id value auto increment itself?
My code
import cx_Oracle
import pandas as pd
...
df2.to_dict(orient='records')
print(df2)
conn = cx_Oracle.connect(connection)
cur = conn.cursor()
id = 0
insert_qry = cur.execute("insert into customer_prf values(
'references',
:1,
'text',
'',
'',
:2,
:3,
2,
:4)"
cursor.prepare(insert)
how would you insert it

You can use df2.values.tolist in order to get a parameter list, and such a code as below in order to insert the values within the dataframe :
import pandas as pd
import cx_Oracle
conn = cx_Oracle.connect('un/pwd#host:port/dbname')
try:
cursor = conn.cursor()
df2 = pd.DataFrame({'names':['Pepe','Luis'], 'stages': [0,1], 'order': [3, 2]})
col_list = df2.values.tolist()
cursor.prepare("INSERT INTO customer_prf(names,label,type,id,n_stage,ctry,\"order\") VALUES(:1,'references','text',seq_customer.nextval,:2,2,:3)")
cursor.executemany(None,col_list)
print("executed!")
except Exception as err:
print('Error', err)
else:
conn.commit()
where
prefer using executemany rather than execute as being more
performant
create a sequence (seq_customer) and add to your values list
(seq_customer.nextval) as using DB version 11g, if your database was of version 12c+, then the ID column could be identified within the create table ddl statement(eg. during table creation) such as
ID INT generated always as identity primary key

Printing out all the values from SQLite3 tables

I want to be able to add lots of data into the table (named: Hydro_RAINFALL) then be able to print it all. At the moment I am managing to insert data into the table but when I come to print it, it only prints the first values that I INSERT into the table.
cursor.execute("SELECT * FROM Hydro_RAINFALL")
print("\n01/08/2014:")
res = cursor.fetchone()
print(res)
This is the code I am using to print it all, but as I stated above, it only prints the first piece of data I INSERTed into the table.
import sqlite3
with sqlite3.connect("Weather.db") as db:
cursor = db.cursor()
cursor.execute('''
CREATE TABLE IF NOT EXISTS Hydro_RAINFALL(
wind VARCHAR (10) NOT NULL,
temp VARCHAR (10) NOT NULL,
precipitation VARCHAR (10) NOT NULL,
humidity VARCHAR (10) NOT NULL,
date VARCHAR (20) NOT NULL);
''')
cursor.execute("""
INSERT INTO Hydro_RAINFALL(wind,temp,precipitation,humidity)
VALUES ("60mp/h", "8C", "56mm", "87%")
""")
cursor.execute("""
INSERT INTO Hydro_RAINFALL(wind,temp,precipitation,humidity)
VALUES ("39mp/h", "4C", "110mm", "45%")
""")
cursor.execute("""
INSERT INTO Hydro_RAINFALL(wind,temp,precipitation,humidity)
VALUES ("69mp/h", "6.9C", "69mm", "69%")
""")
cursor.execute("""
INSERT INTO Hydro_RAINFALL(wind,temp,precipitation,humidity)
VALUES ("456mp/h", "79C", "0mm", "99%")
""")
db.commit()
cursor.execute("SELECT * FROM Hydro_RAINFALL")
print("\n01/08/2014:")
res = cursor.fetchone()
print(res)
cursor.execute("SELECT * FROM Hydro_RAINFALL")
print("\n02/08/2014:")
res = cursor.fetchone()
print(res)
cursor.execute("SELECT * FROM Hydro_RAINFALL")
print("\n03/08/2014:")
res = cursor.fetchone()
print(res)
Thanks in advance! :)

This page and this page state it should work like that.
Please try
all_rows = cursor.fetchall()
for row in all_rows:
print(row)
This will show if the data has been placed in the database correctly.
Another comments on your code. You don't have to add each row separately. You can use executemany() with placeholders:
# Larger example that inserts many records at a time
purchases = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),
('2006-04-05', 'BUY', 'MSFT', 1000, 72.00),
('2006-04-06', 'SELL', 'IBM', 500, 53.00),
]
c.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)', purchases)
(from here)
See also here