I'm trying to get table name for field in result set that I got from database (Python, Postgres). There is a function in PHP to get table name for field, I used it and it works so I know it can be done (in PHP). I'm looking for similar function in Python.
pg_field_table() function in PHP gets results and field number and "returns the name of the table that field belongs to". That is exactly what I need, but in Python.
Simple exaple - create tables, insert rows, select data:
CREATE TABLE table_a (
id INT,
name VARCHAR(10)
);
CREATE TABLE table_b (
id INT,
name VARCHAR(10)
);
INSERT INTO table_a (id, name) VALUES (1, 'hello');
INSERT INTO table_b (id, name) VALUES (1, 'world');
When using psycopg2 or sqlalchemy I got right data and right field names but without information about table name.
import psycopg2
query = '''
SELECT *
FROM table_a A
LEFT JOIN table_b B
ON A.id = B.id
'''
con = psycopg2.connect('dbname=testdb user=postgres password=postgres')
cur = con.cursor()
cur.execute(query)
data = cur.fetchall()
print('fields', [desc[0] for desc in cur.description])
print('data', data)
The example above prints field names. The output is:
fields ['id', 'name', 'id', 'name']
data [(1, 'hello', 1, 'world')]
I know that there is cursor.description, but it does not contain table name, just the field name.
What I need - some way to retrieve table names for fields in result set when using raw SQL to query data.
EDIT 1: I need to know if "hello" came from "table_a" or "table_b", both fields are named same ("name"). Without information about table name you can't tell in which table the value is.
EDIT 2: I know that there are some workarounds like SQL aliases: SELECT table_a.name AS name1, table_b.name AS name2 but I'm really asking how to retrieve table name from result set.
EDIT 3: I'm looking for solution that allows me to write any raw SQL query, sometimes SELECT *, sometimes SELECT A.id, B.id ... and after executing that query I will get field names and table names for fields in the result set.
It is necessary to query the pg_attribute catalog for the table qualified column names:
query = '''
select
string_agg(format(
'%%1$s.%%2$s as "%%1$s.%%2$s"',
attrelid::regclass, attname
) , ', ')
from pg_attribute
where attrelid = any (%s::regclass[]) and attnum > 0 and not attisdropped
'''
cursor.execute(query, ([t for t in ('a','b')],))
select_list = cursor.fetchone()[0]
query = '''
select {}
from a left join b on a.id = b.id
'''.format(select_list)
print cursor.mogrify(query)
cursor.execute(query)
print [desc[0] for desc in cursor.description]
Output:
select a.id as "a.id", a.name as "a.name", b.id as "b.id", b.name as "b.name"
from a left join b on a.id = b.id
['a.id', 'a.name', 'b.id', 'b.name']
Related
The user adds information here: the form
The information gets added to the shoes table.
The database: the database
I want to insert ShoeImage, ShoeName, ShoeStyle, ShoeColor, ShoePrice, and ShoeDescr, and NOT ShoeID (which is autoincrement),ShoeBrandID, and ShoeSizeID.
My insert statement:
$sql = "INSERT INTO $tblShoes VALUES (NULL, '$ShoeImage', '$ShoeName', '$ShoeStyle', '$ShoeColor',
'$ShoePrice', '$ShoeDescr')";
How to write this insert statement with inner join?
Might works.
INSERT INTO shoes
(
'ShoeImage',
'ShoeName',
'ShoeStyle',
'ShoeColor',
'ShoePrice',
'ShoeDescr',
'ShoeBrandID',
'ShoeSizeID'
)
VALUES(
NULL,
'$ShoeImage',
'$ShoeName',
'$ShoeStyle',
'$ShoeColor',
'$ShoePrice',
'$ShoeDescr',
(SELECT BrandID FROM shoebrand WHERE BrandName = '$ShoeBrand'),
(SELECT SizeID FROM shoesize WHERE Size = '$ShoeSize')
)
I need to insert multiple values into a table after checking if it doesn't exist using psycopg2.
The query am using:
WITH data(name,proj_id) as (
VALUES ('hello',123),('hey',123)
)
INSERT INTO keywords(name,proj_id)
SELECT d.name,d.proj_id FROM data d
WHERE NOT EXISTS (SELECT 1 FROM keywords u2 WHERE
u2.name=d.name AND u2.proj_id=d.proj_id)
But how to format or add the values section from tuple to ('hello',123),('hey',123) in query.
As suggested in the comment, assuming that your connection is already established as conn one of the ways would be:
from typing import Iterator, Dict, Any
def insert_execute_values_iterator(connection, keywords: Iterator[Dict[str, Any]], page_size: int = 1000) -> None:
with connection.cursor() as cursor:
psycopg2.extras.execute_values(
cursor,
""" WITH data(name,proj_id) as (VALUES %s)
INSERT INTO keywords(name,proj_id)
SELECT d.name,d.proj_id FROM data d
WHERE NOT EXISTS (SELECT 1 FROM keywords u2 WHERE
u2.name=d.name AND u2.proj_id=d.proj_id);""",
(( keyword['name'],
keyword['proj_id'] ) for keyword in keywords),
page_size=page_size)
insert_execute_values_iterator(conn,{'hello':123,'hey':123})
insert_query = """WITH data(name, proj_id) as (
VALUES (%s,%s)
)
INSERT INTO keywords(name, proj_id)
SELECT d.name,d.proj_id FROM data d
WHERE NOT EXISTS (
SELECT 1 FROM keywords u2
WHERE u2.name = d.name AND u2.proj_id = d.proj_id)"""
tuple_values = (('hello',123),('hey',123))
psycopg2.extras.execute_batch(cursor,insert_query,tuple_values)
i want to ask for a little help about my problem. I have sql query that get all the tables from some schema and put those tables in a list in Python. For example:
tablesList = ['TABLE1','TABLE2',...]
After i get this list of tables that i want i go one more time through each table in a for loop for example:
for t in range(len(tables)):
table = tables[t]
...
#here i want to check if this table exist in some db2 schema and if exist delete content
#of this table, otherwise go with next table check and don't delete content
Query for checking will be:
sql = """SELECT COUNT(*) FROM SYSIBM.SYSTABLES
WHERE TYPE = 'T'
AND CREATOR = 'MY_SCHEMA'
AND NAME = '{table}';""".format(table = table)
cursor.execute(sql)
rows_count = cursor.fetchone()
if rows_count is None:
pass
else:
delete...
Environment Details. Windows 10 - Python3.5 - Flask - SQL SERVER - PyOdbc.
cur.execute("insert into T1 (ID_CUSTOMER, ID_ENVIRONMENT) values (?, ?)",
select id from CUSTOMER where name = form['customerId'],
select id from ENVIRONMENT where name = form['environmentId'],
)
Remove the values, you can directly use INSERT SELECT.
INSERT INTO T1
(ID_CUSTOMER,
ID_ENVIRONMENT)
SELECT ID_CUSTOMER =(SELECT Max(id)
FROM customer
WHERE NAME = 'abc'),
ID_ENVIRONMENT = (SELECT Max(id)
FROM environment
WHERE NAME = 'xyz')
The Max(id) here is to make sure the inner query returns a single value
I am interested in finding the most efficient manner to query the following:
For a list of table names, return the table name if it contains at least one record that meet the conditions
Essentially, something similar to the following Python code in a single query:
dfs = [pd.read_sql('SELECT name FROM {} WHERE a=1 AND b=2'.format(table), engine) for table in tables]
tables = [table for table, df in zip(tables, dfs) if not df.empty]
Is this possible in MySQL?
Assuming you trust the table names in tables not to contain any surprises leading to SQL injection, you could device something like:
from sqlalchemy import text
selects = [f'SELECT :table_{i} FROM {table} WHERE a = 1 AND b = 2'
for i, table in enumerate(tables)]
stmt = ' UNION '.join(selects)
stmt = text(stmt)
results = engine.execute(
stmt, {f'table_{i}': table for i, table in enumerate(tables)})
or you could use SQLAlchemy constructs to build the same query safely:
from sqlalchemy import table, column, union, and_, select, Integer, literal
tbls = [table(name,
column('a', Integer),
column('b', Integer)) for name in tables]
stmt = union(*[select([literal(name).label('name')]).
select_from(tbl).
where(and_(tbl.c.a == 1, tbl.c.b == 2))
for tbl, name in zip(tbls, tables)])
results = engine.execute(stmt)
You can use a UNION of queries that search each table.
(SELECT 'table1' AS table_name
FROM table1
WHERE a = 1 AND b = 2
LIMIT 1)
UNION
(SELECT 'table2' AS table_name
FROM table2
WHERE a = 1 AND b = 2
LIMIT 1)
UNION
(SELECT 'table3' AS table_name
FROM table3
WHERE a = 1 AND b = 2
LIMIT 1)
...