I can fetch the data using this command:
connection = psg.connect( database = "Test" , host="localhost", user="postgres", password="password")
data_1 = psql.read_sql('SELECT * FROM table_1 WHERE id IN (101 , 102)', connection)
But when I run the command below, it gives me an error. A user will put the dynamic ID values and it'll show the data corresponding to the respected IDs. That's why a variable is created which will be on user interface.
connection = psg.connect( database = "Test" , host="localhost", user="postgres", password="password")
variable_p = (108 ) # 108 is the id column value.
data_1 = psql.read_sql('SELECT * FROM table_1 WHERE id IN (variable_p[0])', connection)
Error - Column variable_p does not exist.
you are not using your variable variable_p in your read sql string. You can use f-strings here:
connection = psg.connect( database = "Test" , host="localhost", user="postgres", password="password")
variable_p = 108 # don't need parentheses for single value
data_1 = psql.read_sql(f'SELECT * FROM table_1 WHERE id = {variable_p}', connection)
But here you are vulnerable to SQL injection. So try parametrising your query to make it safe.
Parametrising and using WHERE IN this might be something like (untested):
variable_p = (1, 2, 3)
data_1 = psql.read_sql('SELECT * FROM table_1 WHERE id IN %s', connection, params=variable_p)
Related
I would like to run an sql db2 query on python that will create a data table in a public schema but I'm stuck because of this error
ResourceClosedError: This result object does not return rows. It has been closed automatically.
Below is the Python code I'm using, I have deleted the private information.
import numpy as np
import pandas as pd
from sqlalchemy import create_engine
#connection string
user = "xxx"
pwd = "xxx"
host = "xxx"
port = "xxx"
db = "xxx"
conn_strng = "redshift+psycopg2://%s:%s#%s:%s/%s" %(user,pwd,host,port,db)
#establish connection
engine = create_engine(conn_strng)
with engine.connect() as conn, conn.begin():
pd.read_sql("""
drop table if exists public.fc_SER_ACC_By_DLCX_Date;
create table public.fc_SER_ACC_By_DLCX_Date as
SELECT DLCX_Date, tool_id as DLCX_Tool,Model, avg(SQZSER) as SQZSER_Mean, avg(SQZSER_BASE) as SQZSER_BASE_Mean,
avg(PRED_ACC_SMR) as PRED_ACC_SMR_Mean, avg(PRED_ACC_CMR) as PRED_ACC_CMR_Mean, count(slider_id) as Tested_Sliders
FROM (SELECT DISTINCT a.slider_id, LEFT(a.product_id,2) as Model,
a.xti_wrn1_p23 AS SQZSER,a.SER0 AS SQZSER_BASE, a.FOM2 AS PRED_ACC_SMR, a.FOM1 AS PRED_ACC_CMR,
TRUNC(c.transaction_date_time) as DLCX_Date, c.tool_id
FROM ah.param_jade_wide a
LEFT JOIN ah.param_lap_summary b ON (a.wafer_id, a.row_number) = (b.wafer_id, b.row_number)
LEFT JOIN ah.his_job c ON c.job_number = b.job_number
WHERE c.transaction_date_time > '2020-03-01'
AND LEFT(a.product_id,2) IN ('C3')
AND b.source_system_code IN ('MFG2.SLDR.LAPRUN')
AND a.xti_wrn1_p23 between -10 and 0
AND a.SER0 between -10 and 0
AND c.operation_id IN ('510150')
AND a.retest_number = 0
AND a.class_description IN ('PROD')
AND NOT c.tool_id = 0 AND NOT c.tool_id in (''))
group by DLCX_Date, DLCX_Tool, Model
Union
SELECT DLCX_Date, tool_id as DLCX_Tool,Model, avg(SQZSER) as SQZSER_Mean, avg(SQZSER_BASE) as SQZSER_BASE_Mean,
avg(PRED_ACC_SMR) as PRED_ACC_SMR_Mean,'0'PRED_ACC_CMR_Mean, count(slider_id) as Tested_Sliders
FROM (SELECT DISTINCT a.slider_id, LEFT(a.product_id,2) as Model,
a.XTI_WRN1_P19 AS SQZSER,a.XTI_WRN1_P18 AS SQZSER_BASE, a.XTI_RSVD0 AS PRED_ACC_SMR,
TRUNC(c.transaction_date_time) as DLCX_Date, c.tool_id
FROM ah.param_jade_wide a
LEFT JOIN ah.param_lap_summary b ON (a.wafer_id, a.row_number) = (b.wafer_id, b.row_number)
LEFT JOIN ah.his_job c ON c.job_number = b.job_number
WHERE c.transaction_date_time > '2020-03-01'
AND LEFT(a.product_id,2) IN ('L2','L3')
AND b.source_system_code IN ('MFG2.SLDR.LAPRUN')
AND c.operation_id IN ('510150')
AND a.XTI_WRN1_P19 between -10 and 0
AND a.XTI_WRN1_P18 between -10 and 0
AND a.retest_number = 0
AND a.class_description IN ('PROD')
AND NOT c.tool_id = 0 AND NOT c.tool_id in (''))
group by DLCX_Date, DLCX_Tool, Model
order by DLCX_Date;
commit;""", conn)
conn.close()
engine.dispose()
print("Table has been updated!")
Please help in fixing my code and thanks in advance.
Error indicates Pandas read_sql cannot import data into a data frame since you only have DDL actions: DROP TABLE and CREATE TABLE and nothing that return rows like SELECT.
If not using Pandas for data analytics, simply run your queries with SQL Alchemy transactions. And no need to close using context manager like with:
with engine.connect() as conn:
with conn.begin()
conn.execute("""DROP TABLE IF EXISTS public.fc_SER_ACC_By_DLCX_Date;""")
conn.execute("""CREATE TABLE public.fc_SER_ACC_By_DLCX_Date AS
...
""")
Or combined:
with engine.begin() as conn:
conn.execute("""DROP TABLE IF EXISTS public.fc_SER_ACC_By_DLCX_Date;""")
conn.execute("""CREATE TABLE public.fc_SER_ACC_By_DLCX_Date AS
...
""")
And if you really do need a data frame, use engine object in read_sql after transactions:
df = pd.read_sql("""SELECT * FROM public.fc_SER_ACC_By_DLCX_Date;""", engine)
I'm trying to write a python script to get a count of some tables for monitoring which looks a bit like the code below. I'm trying to get an output such as below and have tried using python multi-dimensional arrays but not having any luck.
Expected Output:
('oltptransactions:', [(12L,)])
('oltpcases:', [(24L,)])
Script:
import psycopg2
# Connection with the DataBase
conn = psycopg2.connect(user = "appuser", database = "onedb", host = "192.168.1.1", port = "5432")
cursor = conn.cursor()
sql = """SELECT COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT count(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
result = cursor.fetchall()
print('Counts:',result)
conn.close()
Current output:
[root#pgenc python_scripts]# python multi_getrcount.py
('Counts:', [(12L,)])
('Counts:', [(24L,)])
Any help is appreciated.
Thanks!
I am a bit reluctant to show this way, because best practices recommend to never build a dynamic SQL string but always use a constant string and parameters, but this is one use case where computing the string is legit:
a table name cannot be a parameter in SQL
the input only comes from the program itself and is fully mastered
Possible code:
sql = """SELECT count(*) from appuser.{}"""
tables = ['oltptransactions', 'oltpcases']
for t in tables:
cursor.execute(sql.format(t))
result = cursor.fetchall()
print("('", t, "':,", result, ")")
I believe something as below, Unable to test code because of certificate issue.
sql = """SELECT 'oltptransactions', COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT 'oltpcases', COUNT(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
for name, count in cursor:
print ("")
Or
sql = """SELECT 'oltptransactions :'||COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT 'oltpcases :'||COUNT(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
result = cursor.fetchall()
print(result)
How do you use the Pyrfc Python library to query the number of entries in an SAP R/3 database table?
I know of three methods to do this using Pyrfc. Modify the following example with your SAP R/3 server connection settings and desired table name:
from pyrfc import Connection
params = dict(
ashost="1.1.1.1",
sysnr="1",
client="100",
user="username",
passwd="password",
)
table = "MKAL"
with Connection(**params) as conn:
# Method 1
result = conn.call("RFC_GET_TABLE_ENTRIES", TABLE_NAME=table, MAX_ENTRIES=1)
entries = result["NUMBER_OF_ENTRIES"]
# Method 2
result = conn.call("EM_GET_NUMBER_OF_ENTRIES", IT_TABLES=[{"TABNAME": table}])
entries = result["IT_TABLES"][0]["TABROWS"]
# Method 3
short_field = "MANDT" # table field with short data length
result = conn.call(
"RFC_READ_TABLE",
QUERY_TABLE=table,
ROWCOUNT=0,
FIELDS=short_field,
)
entries = len(result)
I want to delete some rows from almost 500 tables in mysql database using python.
I know that the query should be something like this:
DELETE FROM (table name)
[WHERE conditions] [ORDER BY ...] [LIMIT rows]
but I am not sure what should I use for the table name when I looping over the tables!
here is my loop
from mysql.connector import (connection)
# call databasecnx = connection.MySQLConnection(user='user', password='PW',host='127.0.0.1', database= 'database')
cursor = cnx.cursor()
cursor.execute("SHOW TABLES LIKE 'options_20%'")
table_names = [tables for (tables, ) in cursor]
for t in table_names:
cursor.execute("Delete table-refs WHERE Expiration = Datadate AND UnderlyingSymbol = 'SPY'")
cnx.commit()
I got an error:
You have an error in your SQL syntax; check the manual that...etc
Test with this.... You did'nt put from on your query statement.
for table in table_names:
cursor.execute("DELETE FROM"+ table +"WHERE Expiration = Datadate AND UnderlyingSymbol = 'SPY'")
I would like to fetch an array in MySQL. Can someone please tell me how to use Python using MySQLdb to do so?
For example, this is what I would like to do in Python:
<?php
require_once('Config.php');
$q = mysql_query("SELECT * FROM users WHERE firstname = 'namehere'");
$data = mysql_fetch_array($q);
echo $data['lastname'];
?>
Thanks.
In python you have dictionary=True, I have tested in python3. This returns directory which is much similar to associative array in php.
eg.
import mysql.connector
cnx = mysql.connector.connect(user='root', password='',host='127.0.0.1',database='test1')
cursor = cnx.cursor(dictionary=True)
sql= ("SELECT * FROM `users` WHERE id>0")
cursor.execute(sql)
results = cursor.fetchall()
print(results)
You can use this (dictionary=True):
import mysql.connector
db = mysql.connector.connect(user='root', password='',host='127.0.0.1', database='test1')
cursor = db.cursor(dictionary=True)
cursor.execute("SELECT * FROM table")
for row in cursor:
print(row['column'])
Install MySQLdb (the drivers for MySQL for Python). Type pip install mysql-python
Read up on the Python DB API, which is the standard way to access databases in Python.
Then, try this:
>>> import MySQLdb
>>> connection = MySQLdb.connect(database='test')
>>> cursor = connection.cursor()
>>> cursor.execute('SELECT * FROM users WHERE firstname = %s',('somename',))
>>> results = cursor.fetchall()
>>> for i in results:
print i
I would use SQLAlchemy. Something like this would do the trick:
engine = create_engine('mysql://username:password#host:port/database')
connection = engine.connect()
result = connection.execute("select username from users")
for row in result:
print "username:", row['username']
connection.close()
Try:
import MySQLdb
connection = MySQLdb.connect(host="localhost", # your host
user="root", # username
passwd="password", # password
db="frateData") # name of the database)
cursor = connection.cursor(MySQLdb.cursors.DictCursor)
cursor.execute('SELECT * FROM users WHERE firstname = %s',['namehere'])
data = cursor.fetchall()
print data['lastname']
Please note that by initiating your cursor by passing the following parameter: "MySQLdb.cursors.DictCursor"
a list instead of an array is returned so you can reference the data with their key name, which in your case in lastname.