I'm a complete beginner at Python and FastAPI.
I'm using FastAPI and I have a table where the requirement is to encrypt the personal information of the user using pgcrypto module of PostgreSQL.
The raw query would be something like this which can be executed into any database client and it executes without any error
insert into customers (email, gender) values (pgm_sym_encrypt('hello#gmail.com', 'some_secret_key'), 'male')
How to execute this query using SQLAlchemy core or encode/databases?
I've tried this
from sqlalchemy import func
query = f"""
insert into customers (email, gender) values
(:email, :gender)
"""
await database.execute(query=query, values={'email': func.pgm_sys_encrypt('hello#gmail.com', 'secret_key'), 'gender': 'male'})
It didn't work.
I also tried
query = f"""
insert into customers (email, gender) values
(pgm_sys_encrypt('hello#gmail.com', 'secret_key'), :gender)
"""
await database.execute(query=query, values={'gender': 'male'})
This didn't work either. I've no idea how to execute a function in the raw query. Please help. I've tried so much but I'm totally clueless on this one still now. Thank you in advance for your help.
As it's a raw query you should be able specify it as you would in raw SQL, so this should work:
from sqlalchemy.sql import text
query = """
insert into customers (email, gender) values
(pgm_sys_encrypt(:email, :secret_key), :gender)
"""
await database.execute(query=text(query), values={'email': 'hello#gmail.com', 'secret_key': 's3cr37', 'gender': 'male'})
Related
I am trying to make a simple web budget app with a SQL database but I keep getting ValueError: parameters are of an unsupported type
from flask import Flask, render_template, request
app = Flask(__name__)
#app.route("/")
def index():
return (render_template('index.html'))
#app.route("/start", methods=['GET', 'POST'])
def start():
import sqlite3
import pandas as pd
if request.method == 'POST':
connection = sqlite3.connect('transactions.db')
cursor = connection.cursor()
amount = request.form.get('amount')
description = request.form.get('description')
category = request.form.get('category')
print(amount, description, category)
cursor.execute(f"""INSERT INTO transactions (amount, description, category)
VALUES ({amount}, '{description}', '{category}')""", connection)
cursor.close()
connection.commit()
table = pd.read_sql("""SELECT * FROM transactions """, connection)
return(render_template('start.html'), value1 == table)
else:
return(render_template('start.html'))
here is the SQL database
'-- SQLite
CREATE TABLE transactions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
amount REAL,
description TEXT,
category TEXT
);
The inputs I used were
10.50, 'Dog Food', and 'Misc.'
Quick solution, do not pass connection and use SQL placeholder binding (with ?) rather than string formatting.
cursor.execute(f"""
INSERT INTO transactions (amount, description, category)
VALUES (?, ?, ?)
""", (amount, description, category))
Longer explanation with your original code:
cursor.execute(f"""INSERT INTO transactions (amount, description, category)
VALUES ({amount}, '{description}', '{category}')""", connection)
Your origin code is passing the connection as an argument to cursor.execute, and connection is a sqlite3 instance, which is unsupported as the supported argument for cursor.execute is either a sequence or dict for parameter substitution in your query.
Also see the actual definition of the argument, and why my solution code uses ? instead of doing string substitution:
https://docs.python.org/3/library/sqlite3.html#sqlite3.Cursor.execute
execute(sql, parameters=(), /)
Execute SQL statement sql. Bind values to the statement
using placeholders that map to the sequence or dict parameters.
https://docs.python.org/3/library/sqlite3.html#sqlite3-placeholders
SQL operations usually need to use values from Python variables. However, beware of using Python’s string operations to assemble queries, as they are vulnerable to SQL injection attacks (see the xkcd webcomic for a humorous example of what can go wrong):
I am trying to write a program in Python that accepts the user inputs and queries from the MySQL database.
My database has the following tables:-
Departments(dept_no (primary key), dept_name)
Employees(emp_no(primary key), f_name, l_name, dob, hire_date)
Salaries(emp_no(primary key), salary, from_date(primary key), to_date)
When I give the following input:- Display the employees with salary greater than 20000.
Then the program should perform the following action:-
from sqlite3 import Error
from MySQLdb import connect
def mysql_code():
try:
with connect(host="localhost", user="root", password="root", database="employees") as connection:
with connection.cursor() as cursor:
cursor.execute("SELECT e.emp_no, e.first_name, e.last_name, s.salary from employees e inner join "
"salaries s on e.emp_no = s.emp_no where s.salary > '20000'")
records = cursor.fetchall()
print(records)
print("Total results found = ", cursor.rowcount)
except Error as e:
print(e)
and display the results.
Is it possible to do so or do I have to write code for each possible query?
I previously used:
cursor.execute("SELECT * FROM {} WHERE {} like %s".format(table, column), (text,))
When I defined the query and gave the user options to choose from where I wrote a query for each of the possible user inputs like
to display all records,
search records with the first name and so on. When the user chose an option the result was displayed.
Now I want the user to give inputs such as
Display employees with salaries greater than 20000 working in dept_no d002. or similar queries.
The program should accept queries in the form of a string from the user.
The code should join the tables and display the result by joining the emp_no, first_name, last_name, salary, dept_no from the tables employees, salaries and departments respectively.
you have an error in your code the like after the on comparison is wrong
from sqlite3 import Error
from MySQLdb import connect
def mysql_code():
try:
with connect(host="localhost", user="root", password="root", database="employees") as connection:
with connection.cursor() as cursor:
cursor.execute("SELECT e.emp_no, e.first_name, e.last_name, s.salary from employees e inner join "
"salaries s on e.emp_no = s.emp_no where s.salary > '20000'")
records = cursor.fetchall()
print(records)
print("Total results found = ", cursor.rowcount)
except Error as e:
print(e)
If I'm understanding your question correctly, you want to perform customizable MySQL queries in Python, creating a constant function and passing it the query.
For this I would use f strings. With this you would create a string query
cursor.execute(f"SELECT {values_to_select} \
from {table_name} \
where {filters} \
{more_stuff/extra_queries}")
This way you can pass the values to the function and it will perform the query with the values specified.
Note: As a test this is fine, but if this code is going to be used in production, giving the end user access to queries can be a huge security risk, checkout SQL injection to see how these kinds of attacks happen. I would recommend to sanitize the given strings first so they can't create custom SQL queries.
Firstly this is my first post on StackOverflow and so if I haven't structured my post properly, please let me know. Basically, I'm new to Python but I've been trying to connect an API to Python, from Python to a database that is hosted online, and then finally into a visualization package. I'm running into some problems when inserting the API data (Sheffield Solar) from Python into my database. The data does actually upload to the database but I'm struggling with an error message that I get in Python.
from datetime import datetime, date
import pytz
import psycopg2
import sqlalchemy
from pandas import DataFrame
from pvlive_api import PVLive
from sqlalchemy import create_engine, Integer, String, DATETIME, FLOAT
def insert_data():
""" Connect to the PostgreSQL database server """
# Calling the class from the pvlive_api.py file
data = PVLive()
# Gets the data between the two dates from the API and converts the output into a dataframe
dl = data.between(datetime(2019, 4, 5, 10, 30, tzinfo=pytz.utc),
datetime(2020, 4, 5, 14, 0, tzinfo=pytz.utc), entity_type="pes",
entity_id=0, dataframe=True)
# sql is used to insert the API data into the database table
sql = """INSERT INTO sheffield (pes_id, datetime_gmt, generation_mw) VALUES (%s, %s, %s)"""
uri = "Redacted"
print('Connecting to the PostgreSQL database...')
engine = create_engine(
'postgresql+psycopg2://Redacted')
# connect to the PostgreSQL server
conn = psycopg2.connect(uri)
# create a cursor that allows python code to execute Postgresql commands
cur = conn.cursor()
# Converts the data from a dataframe to an sql readable format, it also appends new data to the table, also
# prevents the index from being included in the table
into_db = dl.to_sql('sheffield', engine, if_exists='append', index=False)
cur.execute(sql, into_db)
# Commits any changes to ensure they actually happen
conn.commit()
# close the communication with the PostgreSQL
cur.close()
def main():
insert_data()
if __name__ == "__main__":
main()
The error I'm getting is as follows:
psycopg2.errors.SyntaxError: syntax error at or near "%"
LINE 1: ...eld (pes_id, datetime_gmt, generation_mw) VALUES (%s, %s, %s...
with the ^ pointing at the first %s. I'm assuming that the issue is due to me using into_db as my second argument in cur.execute(), however, as I mentioned earlier the data still uploads into my database. As I mentioned earlier I'm very new to Python and therefore it could be an easily solvable issue that I've overlooked. I've also redacted some personal connection information from the code. Any help would be appreciated, thanks.
You are getting such error because trying to execute query without any values for inserting.
If you read the doc about dl.to_sql before using of it you would know that this method writes records to database and returns None.
So, there is no needed trying to construct own sql query for inserting data.
I am using cx_Oracle module to connect to oracle database. In the script i use two variables schema_name and table_name. The below query works fine
cur1.execute("select owner,table_name from dba_tables where owner ='schema_name'")
But i need to query the num of rows of a table, where i need to qualify the table_name with the schema_name and so the query should be
SELECT count(*) FROM "schema_name"."table_name"
This does not work when using in the code, i have tried to put it in triple quotes, single quotes and other options but it does not format the query as expected and hence errors out with table does not exist.
Any guidance is appreciated.
A prepared statement containing placeholders with variables of the form ...{}.{}".format(sc,tb) might be used
sc='myschema'
tb='mytable'
cur1.execute("SELECT COUNT(*) FROM {}.{}".format(sc,tb))
print(cur1.fetchone()[0])
In this particular case, you could also try setting Connection.current_schema, see the cx_Oracle API doc
For example, if you create table in your own schema:
SQL> show user
USER is "CJ"
SQL> create table ffff (mycol number);
Table created.
SQL> insert into ffff values (1);
1 row created.
SQL> commit;
Commit complete.
Then run Python code that connects as a different user:
import cx_Oracle
import os
import sys, os
if sys.platform.startswith("darwin"):
cx_Oracle.init_oracle_client(lib_dir=os.environ.get("HOME")+"/Downloads/instantclient_19_8")
username = "system"
password = "oracle"
connect_string = "localhost/orclpdb1"
connection = cx_Oracle.connect(username, password, connect_string)
connection.current_schema = 'CJ';
with connection.cursor() as cursor:
sql = """select * from ffff"""
for r in cursor.execute(sql):
print(r)
sql = """select sys_context('USERENV','CURRENT_USER') from dual"""
for r in cursor.execute(sql):
print(r)
the output will be:
(1,)
('SYSTEM',)
The last query shows that it is not the user that is being changed, but just the first query is automatically changed from 'ffff' to 'CJ.ffff'.
I'm working on a website powered by Python at the back end. The following script receives values from JavaScript and should write the same to a database.
import cgi
import sqlite3
form = cgi.FieldStorage()
brand = form['brand'].value
model = form['model'].value
db = sqlite3.connect("dbase.db")
query = "INSERT INTO requests (brand, model) VALUES (?, ?)"
db.execute(query, (brand, model,))
db.commit()
db.close()
But invoking the script returns 500 Internal Server Error. To my surprise, the following code, run independently on terminal works perfect.
import sqlite3
brand = 'nokia'
model = 'lumia 625'
db = sqlite3.connect("dbase.db")
query = "INSERT INTO requests (brand, model) VALUES (?, ?)"
db.execute(query, (brand, model,))
db.commit()
db.close()
I'm using Python 2.7.5 and running on Lighttpd server. Also, the db.execute() portion is where the error occurs.
How can I correct the problem?
This might happen for not having database dbase and table requests .You need a database and table for inserting fields.You can create database with this query
sqlite3 dbase.db
And next you need to create a table for this like
create table requests(brand varchar(10), model varchar2(10));
Then your file will execute