Parse SQL Script to extract table and column names - python

If I have a SQL script is there a way to parse and extract the columns and tables referenced in the script into a table like structure :
Script:
Select t1.first, t1.last, t2.car, t2.make, t2.year
from owners t1
left join cars t2
on t1.owner_id = t2.owner_id
Output:
Table Column
owners first
owners last
owners owner_id
cars car
cars make
cars year
cars owner_id

Old question but interesting so here it goes - turn your script temporarily into a stored procedure forcing SQL Server to map the dependencies and then you can retrieve them by using:
SELECT referenced_entity_name ,referenced_minor_name FROM sys.dm_sql_referenced_entities('dbo.stp_ObjectsToTrack', 'Object')

This is what you want in SQL Server:
select t.name as [Table], c.name as [Column]
from sys.columns c
inner join sys.tables t
on c.object_id = t.object_id

Related

Filter rows with remaining only the latest record in SQLAlchemy

I have been trying to write the SQLAlchemy code that should function as the following SQL query.
SELECT * FROM events AS ev
INNER JOIN event_types AS et1 on ev.event_type_id = et1.id
INNER JOIN (
SELECT event_type, MAX(created_at) AS LatestCreatedAt
FROM event_types et GROUP BY event_type
) AS et2
ON
et2.event_type = et1.event_type
AND
et2.LatestCreatedAt = et1.created_at
What I'm trying to do is to
Get all columns from the events table
Inner join the event_type table (et1) on the event table
Group by the event_type with only the rows that have the latest record (i.e. Filter out old event types by looking at created_at if duplicated)
Inner join the grouped event_type (et2) on the event_type table (et1)
What I wrote for the SQL Alchemy version of the above is
from sqlalchemy import func
subquery = session.query(EventTypeTable.event_type,
func.max(EventTypeTable.created_at).group_by(EventTypeTable.event_type)).all()
events = (session.query(EventTable)
.join(EventTypeTable)
.join(subquery)
.all())
However, I get the following error.
Neither 'max' object nor 'Comparator' object has an attribute 'group_by'
It seems to complain that I can not use group_by with max function. Is there any other way to get the query results while leaving only the latest record on the created_at column in the event_type table in SQL Alchemy?
Any help or comments are appreciated. Thank you!

Separating Destination tables and Source tables from a query

I do have lot of queries that need to separate insert into (destination) table names as well as from (Source) table names.
Do you have any idea or a python code to separate them? I do have really large oracle stored procedure list.
Doing it manually is really time consuming. If someone has any clue for this, would be highly appreciated.
I need to separate only the destination tables and source tables..
Below is a sample query to work on
Create or replace procedure sfa.dlm_upload
BEGIN
INSERT INTO SFL.DFV_ALERT_INT
SELECT A.PROFILE_ID, A.AGENT_NAME, B.CONTACTR SRC_MSD,
C.PROFILE_ID, B.DSR_PROFILE_ID
FROM EDW_TRD.RETAILER_SO1_DATA A, SFL.SFL_AGENT_DTL B,
SFL.SFL_AGENT_DTL_TEMP C
WHERE A.PROFILE_ID = B.PROFILE_ID
AND B.DSR_ID = C.AGENT_ID
AND C.AGENT_STATUS = 'Active'
AND MONTH_KEY = (SELECT MAX(MONTH_KEY) FROM
EDW_TRD.RETAILER_SO1_DATAMART)
;
INSERT OVERWRITE INTO SFL.MLV_ALERT_INTER
SELECT PROFILE_ID, TRUNC(PROFILE_CREATED_DATE) DATE_,
COUNT(DISTINCT CONTRACT_ID)
FROM
(SELECT PROFILE_ID,PROFILE_CREATED_DATE, CONTRACT_ID
FROM MDW.RTV_PRE_CHANE_SALES
WHERE TRUNC(PROFILE_DATE,'MM') >=
ADD_MONTHS(TRUNC(SYSDATE,'MM'),-2)
UNION ALL
SELECT TO_NUMBER(PMS_ID), PROFILE_CREATED_DATE, CONTRACT_ID
FROM MDW.MTV_POST_CHAN_SALES
WHERE TRUNC(PROFILE_CREATED_DATE,'MM') >=
ADD_MONTHS(TRUNC(SYSDATE,'MM'),-2))
GROUP BY PROFILE_ID, TRUNC(PROFILE_CREATED_DATE);
END;
OUTPUT -
Destination tables
SFL.DFV_ALERT_INT
SFL.MLV_ALERT_INTER
Source tables
EDW_TRD.RETAILER_SO1_DATA
SFL.SFL_AGENT_DTL
SFL.SFL_AGENT_DTL_TEMP
MDW.RTV_PRE_CHANE_SALES
MDW.MTV_POST_CHAN_SALES
Can anyone help me on this?

SQLite Query with COUNT and ID string

First... i have a SQLite database:
I have an user table tbl_members
member_id
name
and an order table tbl_orders
order_id
member_ids
name
An order can be edited by more than one member and this members are stored in tbl_orders member_ids in this fashion 1,2,34,23,65,
I need a query that returns:
tbl_members.member_id, tbl_members.name and a COUNT(tbl_orders.order_id) of the orders where the tbl.members.member_id is in tbl.orders.member_ids
I can't get it... can anyone give me a hint?
is this your expected answer?
SELECT tm.member_id, tm.name, COUNT(to.order_id)
FROM tbl_members as tm
LEFT JOIN tbl_orders as to on tm.member_id = to.member_id
GROUP BY tm.member.id, tm.name
I got it!
SELECT tm.member_id, tm.name, COUNT(to.order_id)
FROM tbl_members tm
LEFT JOIN tbl_orders to ON (to.member_ids LIKE '%,'||tm.member_id||'%')
GROUP BY tm.member_id
Works for me

SQLAlchemy Joining 2 tables using Junction table

I am learning SQL-Python using SQLAlchemy and will appreciate much help on this.
I have 3 tables,
Table 1 (Actors) : nconst (primary key), names
Table 2 (Movies) : tconst (primary key) , titles
Table 3 (Junction table) : nconst (from Actors table) , tconst(from Movies table)
I am trying to obtain 10 rows of actors that acted in particular movies. Hence I am trying to do an inner join of Actors on Junction table (using nconst) and then another inner join onto Movies table.
In SQL, this means
FROM principals INNER JOIN actors
ON principals.nconst=actors.nconst INNER JOIN
movies ON principals.tconst=movies.tconst
In SQLAlchemy, my current code is:
mt = list(session.query(Movies, Principals, Actors).select_from(
join(Movies, Principals, Movies.tconst == Principals.tconst)
.join(Actors, Principals, Actors.nconst == Principals.nconst
).with_entities(
Movies.title, # Select clause
))
Alternatively, I am trying
from sqlalchemy.orm import join
mv = list(session.query(Actors).select_from(
join(Movies, Principals, Actors, Movies.tconst == Principals.tconst,
Actors.nconst == Principals.nconst) # Join clause
).with_entities(
Actors.name, # Select clause
Movies.title,
))
mv
The error I am getting is an Attribute Error, "Actor type object 'Actors' has no attribute '_from_objects'
Appreciate much help on this. Thank you very much.

loop over all tables in mysql databases

I am new with MySQL and I need some help please. I am using MySQL connector to write scripts.
I have database contain 7K tables and I am trying to select some values from some of these tables
cursor.execute( "SELECT SUM(VOLUME) FROM stat_20030103 WHERE company ='Apple'")
for (Volume,) in cursor:
print(Volume)
This works for one table e.g (stats_20030103). However I want to sum all volume of all tables .startwith (stats_2016) where the company name is Apple. How I can loop over my tables?
I'm not an expert in MySQL, but here is something quick and simple in python:
# Get all the tables starting with "stats_2016" and store them
cursor.execute("SHOW TABLES LIKE 'stats_2016%'")
tables = [v for (v, ) in cursor]
# Iterate over all tables, store the volumes sum
all_volumes = list()
for t in tables:
cursor.execute("SELECT SUM(VOLUME) FROM %s WHERE company = 'Apple'" % t)
# Get the first row as is the sum, or 0 if None rows found
all_volumes.append(cursor.fetchone()[0] or 0)
# Return the sum of all volumes
print(sum(all_volumes))
You can probably use select * from information_schema.tables to get all tables name into your query.
I'd try to left-join.
SELECT tables.*, stat.company, SUM(stat.volume) AS volume
FROM information_schema.tables AS tables LEFT JOIN mydb.stat_20030103 AS stat
WHERE tables.schema = "mydb" GROUP BY stat.company;
This will give you all results at once. Maybe MySQL doesn't support joining from metatables, in which case you might select it into a temporary table.
CREATE TEMPORARY TABLE mydb.tables SELECT name FROM information_schema.tables WHERE schema = "mydb"
See MySQL doc on information_schema.table.

Categories