I would like to know wether there is a way to combine joining two tables, and retrieving the two entities only with their relevant columns.
I want to avoid doing a thing such select * after joining two tables and getting only column id from Table A and column address from table B.
Working with python2.7
I heard of sqlalchemy feature called with_entities, but not sure how can it be mixed with this kind of join, lets work on this example from sqlalchemy tutorial:
u, a = session.query(User, Address).\
filter(User.id==Address.user_id).\
filter(Address.email_address=='jack#google.com').\
first():
Check out Query.join()
id, address = session.query(A.id, B.address).\
join(B, B.user_id == A.id).\
filter(A.email_address=='jack#google.com').\
first()
This can be done using a join or outerjoin depending on the use case, joins can be implicit or explicit. The .join second argument is the explicit join statement.
Related
I was following a tutorial to make my first Flask API (https://medium.com/#dushan14/create-a-web-application-with-python-flask-postgresql-and-deploy-on-heroku-243d548335cc) I did it but now I want to do queries more custom with SQLAlchemy and PostgreSQL. My question is how I could do something like this:
query = text("""SELECT enc.*, persona."Persona_Nombre", persona."Persona_Apellido", metodo."MetEnt_Nombre", metodo_e."MetPag_Descripcion"
FROM "Ventas"."Enc_Ventas" AS enc
INNER JOIN "General"."Persona" AS persona ON enc."PersonaId" = persona."PersonaId"
INNER JOIN "Ventas"."Metodo_Entrega" AS metodo ON enc."MetodoEntregaId" = metodo."MetodoEntregaId"
INNER JOIN "General"."Metodo_Pago" AS metodo_e ON enc."MetodoPagoId" = metodo_e."MetodoPagoId"
INNER JOIN "General"."Estatus" AS estado ON enc. """)
but with SQLAlchemy in order to use the models that I created previously. Thanks in advance for any answer!!
Edit:
The columns that I wish to see at the final result are: enc.*, persona."Persona_Nombre", persona."Persona_Apellido", metodo."MetEnt_Nombre", metodo_e."MetPag_Descripcion"
I really wish I could share more info but sadly I can't at the moment.
Doing this from the ORM layer, you would reference model names (I match the names of your query above, but I'm sure some of the model/table names are off - now adjusted slightly).
Now revised to include the specific columns you only want to see (note that I ignore your SQL aliases, ORM layer handles the actual query construction):
selection = session.query(Enc_Ventas, Persona.Persona_Nombre, Persona.Persona_Apellido, Metodo_Entrega.MetEnt_Nombre, Metodo_Pago.MetPag_Descripcion).\
join(Persona, Enc_Ventas.PersonaId == Persona.PersonaId).
join(Metodo_Entrega, Enc_Ventas.MetodoEntregaId == Metodo_Entrega.MetodoEntregaId).\
join(Metodo_Pago, Enc_Ventas.MetodoPagoId == Metodo_Pago.MetodoPagoId).\
join(Estatus).all()
Referencing the selection collection would be by iteration through the rows of tuples. A more robust and stable solution would be to transform each output row into a dict.
Otherwise, by including whole models, the collection of rows returned can be individually accessed by referencing as dot notation the model names in the query().
If you need further access to the columns in the related tables, use the ORM technique of .options(joinedload(myTable)), which in a single database query will bring in those additional columns, using the relationship name, also as dot notation.
You also need to define sqlalchemy relationships within your models for this to work, as well as defining the underlying SQL foreign keys.
Much more detail and/or a more specific question is needed to help further, imo.
I am trying to find out how to see if two table are the same in sql. The tables should be the exact same but the order of the columns is slightly different. My initial though is to subtract the two tables from eachother and verify that the result is 0 records but I do not think this will work because of the difference in the order of columns. Any suggestions? Your help is much appreciated,
Thank you,
John
If what you want is to compare tables you can try the following; minus and except depends on the SQL used by yout Data Base Management System.
select * from tableA
minus
select * from tableB
If no rows are return by query then data is the same for both tables,now this will work if both have same amount of rows.
When comparing different entries in two different columns size you will need to use
LEFT JOIN
There is some source material you can use for reference:
Union! SQL Server
Compare two tables with different entries and column
Might not be the answer you are looking for, but you could create a view and select the columns in the matching order from there. Then the comparison should be easier.
You can perform a full outer join on each field, then compare the length of the result to the length both original tables.
SELECT * from
Table_A FULL OUTER JOIN
Table_B on
A.Field_X = B.Field_X
A.Field_Y = B.Field_Y
If the view returns the same number of results as the length of both original tables, then the two tables are the same for those fields.
There are several parts to this question. I am working with sqlite3 in Python 2.7, but I am less concerned with the exact syntax, and more with the methods I need to use. I think the best way to ask this question is to describe my current database design, and what I am trying to accomplish. I am new to databases in general, so I apologize if I don't always use correct nomenclature.
I am modeling refrigeration systems (using Modelica--not really important to know), and I am using the database to manage input data, results data, and models used for that data.
My top parent table is Model, which contains the columns:
id, name, version, date_created
My child table under Model is called Design. It is used to create a unique id for each combination of design input parameters and the model used. the columns it contains are:
id, model_id, date_created
I then have two child tables under Design, one called Input, and the other called Result. We can just look at Input for now, since one example should be enough. The columns for input are:
id, value, design_id, parameter_id, component_id
parameter_id and component_id are foreign keys to their own tables.The Parameter table has the following columns:
id, name, units
Some example rows for Parameter under name are: length, width, speed, temperature, pressure (there are many dozens more). The Component table has the following columns:
id, name
Some example rows for Component under name are: compressor, heat_exchanger, valve.
Ultimately, in my program I want to search the database for a specific design. I want to be able to search a specific design to be able to grab specific results for that design, or to know whether or not a model simulation with that design has already been run previously, to avoid re-running the same data point.
I also want to be able to grab all the parameters for a given design, and insert it into a class I have created in Python, which is then used to provide inputs to my models. In case it helps for solving the problem, the classes I have created are based on the components. So, for example, I have a compressor class, with attributes like compressor.speed, compressor.stroke, compressor.piston_size. Each of these attributes should have their own row in the Parameter table.
So, how would I query this database efficiently to find if there is a design that matches a long list (let's assume 100+) of parameters with specific values? Just as a side note, my friend helped me design this database. He knows databases, but not my application super well. It is possible that I designed it poorly for what I want to accomplish.
Here is a simple picture trying to map a certain combination of parameters with certain values to a design_id, where I have taken out component_id for simplicity:
Picture of simplified tables
Simply join the necessary tables. Your schema properly reflects normalization (separating tables into logical groupings) and can scale for one-to-many relationships. Specifically, to answer your question --So, how would I query this database efficiently to find if there is a design that matches a long list (let's assume 100+) of parameters with specific values?-- consider below approaches:
Inner Join with Where Clause
For handful of parameters, use an inner join with a WHERE...IN() clause. Below returns design fields joined by input and parameters tables, filtered for specific parameter names where you can have Python pass as parameterized values even iteratively in a loop:
SELECT d.id, d.model_id, d.date_created
FROM design d
INNER JOIN input i ON d.id = i.design_id
INNER JOIN parameters p ON p.id = i.parameter_id
WHERE p.name IN ('param1', 'param2', 'param3', 'param4', 'param5', ...)
Inner Join with Temp Table
Should values be over 100+ in a long list, consider a temp table that filters parameters table to specific parameter values:
# CREATE EMPTY TABLE (SAME STRUCTURE AS parameters)
sql = "CREATE TABLE tempparams AS SELECT id, name, units FROM parameters WHERE 0;"
cur.execute(sql)
db.commit()
# ITERATIVELY APPEND TO TEMP
for i in paramslist: # LIST OF 100+ ITEMS
sql = "INSERT INTO tempparams (id, name, units) \
SELECT p.id, p.name, p.units \
FROM parameters p \
WHERE p.name = ?;"
cur.execute(sql, i) # CURSOR OBJECT COMMAND PASSING PARAM
db.commit() # DB OBJECT COMMIT ACTION
Then, join main design and input tables with new temp table holding specific parameters:
SELECT d.id, d.model_id, d.date_created
FROM design d
INNER JOIN input i ON d.id = i.design_id
INNER JOIN tempparams t ON t.id = i.parameter_id
Same process can work with components table as well.
*Moved picture to question section
Is there direct syntax to undefer all columns in a query object?
I know there is a way of undefering groups, but I have a query touching many tables with many groups deferred that I'd like to undefer. It's getting a bit verbose.
I know you can undefer all columns in a single Table, but what about doing it for every Table:
# undefer all columns specific to a single class using Load + *
session.query(MyClass, MyOtherClass).options(
Load(MyClass).undefer("*"))
http://docs.sqlalchemy.org/en/latest/orm/loading_columns.html#sqlalchemy.orm.undefer
I have an error, when SQLAlchemy produced wrong SQL query, but I can't determine conditions.
I use Flask-SQLAlchemy and initially it's a just MyModel.query and it represented by simple SELECT with JOINs. But when .limit() method is applied, it transforms and uses subquery for fetch main objects and only then apply JOINs. The problem is in ORDER BY statement, which remains the same and ignores the subquery definition.
Here's example and I've simplify select fields:
-- Initially
SELECT *
FROM customer_rates
LEFT OUTER JOIN seasons AS seasons_1 ON seasons_1.id = customer_rates.season_id
LEFT OUTER JOIN users AS users_1 ON users_1.id = customer_rates.customer_id
-- other joins ...
ORDER BY customer_rates.id, customer_rates.id
-- Then .limit()
SELECT anon_1.*, *
FROM (
SELECT customer_rates.*
FROM customer_rates
LIMIT :param_1) AS anon_1
LEFT OUTER JOIN seasons AS seasons_1 ON seasons_1.id = anon_1.customer_rates_season_id
LEFT OUTER JOIN users AS users_1 ON users_1.id = anon_1.customer_rates_customer_id
-- other joins
ORDER BY customer_rates.id, customer_rates.id
And this query gives following error:
ProgrammingError: (psycopg2.ProgrammingError) missing FROM-clause entry for table "customer_rates"
The last line in query should be:
ORDER BY anon_1.customer_rates_id
The code, that produces this queries is a part of large application. I've tried to implement this from scratch in a small flask application, But I can't reproduce it. In small application it always uses a JOIN.
So I need to know, when SQLAlchemy decides to use subquery.
I use python 2.7 and PostgreSQL 9
The answer is pretty straightforward. It uses subquery when it joined table has many-to-one relations with queried model. So for producing correct number of results it limits the queried rows in the subquery