Imagine one has two SQL tables
objects_stock
id | number
and
objects_prop
id | obj_id | color | weight
that should be joined on objects_stock.id=objects_prop.obj_id, hence the plain SQL-query reads
select * from objects_prop join objects_stock on objects_stock.id = objects_prop.obj_id;
How can this query be performed with SqlAlchemy such that all returned columns of this join are accessible?
When I execute
query = session.query(ObjectsStock).join(ObjectsProp, ObjectsStock.id == ObjectsProp.obj_id)
results = query.all()
with ObjectsStock and ObjectsProp the appropriate mapped classes, the list results contains objects of type ObjectsStock - why is that? What would be the correct SqlAlchemy-query to get access to all fields corresponding to the columns of both tables?
Just in case someone encounters a similar problem: the best way I have found so far is listing the columns to fetch explicitly,
query = session.query(ObjectsStock.id, ObjectsStock.number, ObjectsProp.color, ObjectsProp.weight).\
select_from(ObjectsStock).join(ObjectsProp, ObjectsStock.id == ObjectsProp.obj_id)
results = query.all()
Then one can iterate over the results and access the properties by their original column names, e.g.
for r in results:
print(r.id, r.color, r.number)
A shorter way of achieving the result of #ctenar's answer is by unpacking the columns using the star operator:
query = (
session
.query(*ObjectsStock.__table__.columns, *ObjectsProp.__table__.columns)
.select_from(ObjectsStock)
.join(ObjectsProp, ObjectsStock.id == ObjectsProp.obj_id)
)
results = query.all()
This is useful if your tables have many columns.
Related
I have got 2 column properties which use the same query, but just return different columns:
action_time = column_property(
select([Action.created_at]).where((Action.id == id)).order_by(desc(Action.created_at)).limit(1)
)
action_customer = column_property(
select([Action.customer_id]).where((Action.id == id)).order_by(desc(Action.created_at)).limit(1)
)
SQL query that is produced will have 2 subqueries for each of the properties. So it mean if I'd like to add a few more similar properties, SQL query will end up with N subqueries.
I am wondering whether it is possible to have one LEFT OUTER JOIN which will be used for multiple column_property (ies)?
I want to get all the columns of a table with max(timestamp) and group by name.
What i have tried so far is:
normal_query ="Select max(timestamp) as time from table"
event_list = normal_query \
.distinct(Table.name)\
.filter_by(**filter_by_query) \
.filter(*queries) \
.group_by(*group_by_fields) \
.order_by('').all()
the query i get :
SELECT DISTINCT ON (schema.table.name) , max(timestamp)....
this query basically returns two columns with name and timestamp.
whereas, the query i want :
SELECT DISTINCT ON (schema.table.name) * from table order by ....
which returns all the columns in that table.Which is the expected behavior and i am able to get all the columns, how could i right it down in python to get to this statement?.Basically the asterisk is missing.
Can somebody help me?
What you seem to be after is the DISTINCT ON ... ORDER BY idiom in Postgresql for selecting greatest-n-per-group results (N = 1). So instead of grouping and aggregating just
event_list = Table.query.\
distinct(Table.name).\
filter_by(**filter_by_query).\
filter(*queries).\
order_by(Table.name, Table.timestamp.desc()).\
all()
This will end up selecting rows "grouped" by name, having the greatest timestamp value.
You do not want to use the asterisk most of the time, not in your application code anyway, unless you're doing manual ad-hoc queries. The asterisk is basically "all columns from the FROM table/relation", which might then break your assumptions later, if you add columns, reorder them, and such.
In case you'd like to order the resulting rows based on timestamp in the final result, you can use for example Query.from_self() to turn the query to a subquery, and order in the enclosing query:
event_list = Table.query.\
distinct(Table.name).\
filter_by(**filter_by_query).\
filter(*queries).\
order_by(Table.name, Table.timestamp.desc()).\
from_self().\
order_by(Table.timestamp.desc()).\
all()
I have been performing a query to count how many times in my sqlite3 database table (Users), within the column "country", the value "Australia" occurs.
australia = db.session.query(Users.country).filter_by(country="Australia").count()
I need to do this in a more dynamic way for any country value that may be within this column.
I have tried the following but unfortunately I only get a count of 0 for all values that are passed in the loop variable (each).
country = list(db.session.query(Users.country))
country_dict = list(set(country))
for each in country_dict:
print(db.session.query(Users.country).filter_by(country=(str(each))).count())
Any assistance would be greatly appreciated.
The issue is that country is a list of result tuples, not a list of strings. The end result is that the value of str(each) is something along the lines of ('Australia',), which should make it obvious why you are getting counts of 0 as results.
For when you want to extract a list of single column values, see here. When you want distinct results, use DISTINCT in SQL.
But you should not first query distinct countries and then fire a query to count the occurrence of each one. Instead use GROUP BY:
country_counts = db.session.query(Users.country, db.func.count()).\
group_by(Users.country).\
all()
for country, count in country_counts:
print(country, count)
The main thing to note is that SQLAlchemy does not hide the SQL when using the ORM, but works with it.
If you can use the sqlite3 module with direct SQL it is a simple query:
curs = con.execute("SELECT COUNT(*) FROM users WHERE country=?", ("Australia",))
nb = curs.fetchone()[0]
I am new to sqlalchemy. So any help is appreciated.
I have a function that constructs my queries for my application. I pass it a list of tables to join.
Here are the relevant code snippets.
class Scope(Base):
entry = Column(String(512))
location_id = Column(Integer, ForeignKey('location_id'))
type = Column(String(128))
class Location(Base):
id = Column(Integer, primary_key=True)
name = Column(String(512)
modified_by = Column(String(128))
instances = [Scope, Location]
join_classes = [Location]
queryset = session.query(*instances).join(*join_classes).all()
Here is the SQL query that runs (when I print queryset to screen before the .all()):
queryset: SELECT scope.id AS scope_id, scope.location_id AS scope_location_id, scope.entry AS scope_entry, scope.type AS scope_type, location.name AS location_name, location.modified_by AS location_modified_by,
FROM scope JOIN location ON location.id = scope.location_id
My end result I want is: a list of dictionaries for all columns (from both tables - like regular inner join gives a single table).
However, I am getting the a list when I type(queryset) and when I just try to do [u._asdict() for u in queryset] which is how I return a list of dictionaries in queries that don't have a join, it only returns a list of dictionaries for 1 column from each table (the column in the __repr__.
I need all columns from both tables to be returned.
Right now this is how what is what I get:
[{'Scope': 192.168.0.0/24, 'Location': main}, ...]
I need something like, where all columns from the join are returned in a list of dictionaries:
[{'Scope.entry': 192.168.0.0/24, 'Scope.type': 'virtual', 'Location.name': main, 'Location.modified_by': 'jim'}, ...]
In my code the instances & join_classes are dynamically passed and not hard coded as different functions pass the table models to join on (with the 1st model table being the table that all proceeding join on). I need this to work with a join on multiple tables (but all tables will be joined to the 1st model table, Scope in this example.)
Edit: I finally realized I was getting a list of sqlalchemy table objects back. That is why I was getting the __repr__ values when displaying.
records = DBSession.query(GeneralLedger, ConsolidatedLedger).join(ConsolidatedLedger, GeneralLedger.invoiceId == ConsolidatedLedger.invoiceId).all()
this sort of query return both table data
Well, writing something done definitely helps you figure out the answer.
For anyone that might benefit this is what I did. I am sure there is a more eloquent way to do this so please let me know if so.
I finally read my output correctly and realized that it was giving me 2 table model objects (1 per table joined). I then iterated over each, converted each iteration to a list of dictionaries and then merged those dictionaries appropriately so that i had one list of dictionaries like a inner join table would give me.
Here is some of my code:
for obj in queryset:
result.append(queryset_to_dict(obj))
for r in result:
new_dict = {}
for inner in r:
new_dict = {**new_dict, **inner}
new_result.append(new_dict)
Note the queryset_to_dict is a function I created for converting sqlalchemy table model objects to list of dictionaries.
I am trying to query a table in an existing sqlite database. The data must first be subsetted as such, from a user input:
query(Data.num == input)
Then I want to find the max and min of another field: date in this subset.
I have tried using func.min/max, as well as union, but received an error saying the columns do not match. One of the issues here is that func.min/max need to be used as query arguments, not filter.
ids = session.query(Data).filter(Data.num == input)
q = session.query(func.max(Data.date),
func.min(Data.date))
ids.union(q).all()
ArgumentError: All selectables passed to CompoundSelect must have identical numbers of columns; select #1 has 12 columns, select #2 has 2
Similarly, if I use func.max and min separately, the error says #2 has 1 column.
I think seeing this query in SQL might help as well.
Thanks
The following solution works. You first need to set up the query, then filter the data down afterwards.
query = session.query(Data.num, func.min(Data.date),
func.max(Data.date), Data.date)
query = query.filter(Data.num == input)
query = query.all()