We Keep Coding

return populated fields from a join table in SQLalchemy Flask

UserService is a join table connecting Users and Services tables. I have a query that returns all the tables that have a user_id = to the id passed in the route.
#bp.route('/user/<id>/services', methods=['GET'])
def get_services_from_user(id):
user = db_session.query(User).filter(id == User.id).first()
if not user:
return jsonify({'Message': f'User with id: {id} does not exist', 'Status': 404})
user_services = db_session.query(UserService).filter(user.id == UserService.user_id).all()
result = user_services_schema.dump(user_services)
for service in result:
user_service = db_session.query(Service).filter(service['service_id'] == Service.id).all()
result = services_schema.dump(user_service)
return jsonify(result)
result holds a list that looks as such:
[
{
"id": 1,
"service_id": 1,
"user_id": 1
},
{
"id": 2,
"service_id": 2,
"user_id": 1
}
]
how could I then continue this query or add another query to get all the actual populated services (Service class) instead of just the service_id and return all of them in a list? The for loop is my attempt at that but currently failing. I am only getting back a list with one populated service, and not the second.

You could try something like this:
userServies = db_session.query(Users, Services).filter(Users.id == Services.id).all()
userServices would be an iterable. You should use:
for value, index in userServices:
to iterate through it. (Could be index, value I'm not 100% sure of the order)
There is another way using .join() and adding the columns that you need with .add_columns().
There is also another way using
db_session.query(Users.id, Services.id, ... 'all the columns that you need' ...).filter(Users.id == Services.id).all()

How to filter from joined table in sqlalchemy?

Hello everyone, I'm very new with sqlalchemy and I try to do a searching module from any field that I input from the user by the below code.
filters = []
if 'inputVendorName' in inputfield:
filters.append(Vendors.vendor_name.contains(inputfield['inputVendorName']))
if 'inputProductName' in inputfield:
filters.append(Product.product_name.contains(inputfield['inputProductName']))
if 'inputCustomerName' in inputfield:
filters.append(Customers.customer_name.contains(inputfield['inputCustomerName']))
if 'inputSalePrice' in inputfield:
filters.append(Sales.price.contains(inputfield['inputSalePrice']))
# jointable --> how to join table
results = jointable.query.filter(db.or_(*filters)).all()
Begin with fiters is a list that contains any input value from the user, and I want to use these values in a list to filter from my join table.
For example, the user has input some product_name and I want to use this product_name to filter and get any record value in Products table that matches to product_name and also gets the other record from another table (Vendors, Sales, Customers) that related to this 'product_name'.
So how can I do that?

Here's a piece of code that runs a query, based on a set of 'dynamic' filters.
filters = []
# this is an example:
inputfield = {
"inputVendorName": "J",
"inputProductName": "Pen",
"MinPrice": 10
}
if 'inputVendorName' in inputfield:
filters.append(Vendor.vendor_name.contains(inputfield["inputVendorName"]))
if 'inputProductName' in inputfield:
filters.append(Product.product_name.contains(inputfield["inputProductName"]))
if 'MinPrice' in inputfield:
filters.append(Sale.price > inputfield["MinPrice"])
base_query = session.query(Customer, Product, Vendor, Sale).filter(
Sale.customer_id == Customer.customer_id, Vendor.vendor_id == Product.vendor_id, Sale.product_id == Product.product_id)
for res in base_query.filter(*filters).all():
print(res)

How to (Python Iterate from a json array of objects)

What is the best way to iterate from an array of objects and use that data to update selected rows in a database table ?
I wanted to update data from the database where id = tasklist,
id from the json I've provided below, and set
attached_document_ins.is_viewed = checked value from the json with that ID .
for example, if id == 35 then attached_document_ins.is_viewed = True cause the checked value of id 35 is True.
What is the best algo for that ?
I have provided my code below.
#Code
def post(self, request):
data = request.data
print("Response Data :" , data)
try:
attached_document_ins = DocumentTask.objects.filter(id=tasklist_id)
for attached_document_ins in attached_document_ins:
attached_document_ins.is_viewed = True
attached_document_ins.save()
return Response("Success", status=status.HTTP_200_OK)
except DocumentTask.DoesNotExist:
return Response("Failed.", status=status.HTTP_400_BAD_REQUEST)
Json(data)
{
'tasklist':[
{
'files':[
],
'checked':True,
'company':6,
'task':'s',
'applicant':159,
'id':35
},
{
'files':[
],
'checked':True,
'company':6,
'task':'ss',
'applicant':159,
'id':36
},
{
'files':[
],
'checked':True,
'company':6,
'task':'sss',
'applicant':159,
'id':37
}
]
}

Here is one way you could do it:
for task in data['tasklist']:
if task['checked']:
document = DocumentTask.objects.get(id=task['id'])
document.is_viewed = True
document.save()

Django queryset result is wrong for the test

My model is:
class AndroidOffer(models.Model):
name = models.CharField(max_length=128, db_index=True)
# ...
countries = models.ManyToManyField(Country)
And the following code (I skipped previous filtering):
active_offers = active_offers.filter(countries__in=[country])
It generates this SQL query:
SELECT "offers_androidoffer"."id", "offers_androidoffer"."name", "offers_androidoffer"."title", "offers_androidoffer"."is_for_android", "offers_androidoffer"."is_for_ios", "offers_androidoffer"."url", "offers_androidoffer"."icon", "offers_androidoffer"."cost", "offers_androidoffer"."quantity", "offers_androidoffer"."hourly_completions", "offers_androidoffer"."is_active", "offers_androidoffer"."description", "offers_androidoffer"."comment", "offers_androidoffer"."priority", "offers_androidoffer"."offer_type", "offers_androidoffer"."package_name", "offers_androidoffer"."is_search_install", "offers_androidoffer"."search_query", "offers_androidoffer"."launches" FROM "offers_androidoffer" INNER JOIN "offers_androidoffer_platform_versions" ON ("offers_androidoffer"."id" = "offers_androidoffer_platform_versions"."androidoffer_id") INNER JOIN "offers_androidoffer_countries" ON ("offers_androidoffer"."id" = "offers_androidoffer_countries"."androidoffer_id") WHERE ("offers_androidoffer"."is_active" = True AND "offers_androidoffer"."quantity" > 0 AND NOT ("offers_androidoffer"."id" IN (SELECT U0."offer_id" FROM "offers_androidofferstate" U0 WHERE (U0."device_id" = 1 AND (U0."state" = 3 OR U0."state" = 4)))) AND NOT ("offers_androidoffer"."package_name" IN (SELECT V0."package_name" FROM "applications_app" V0 INNER JOIN "applications_deviceapp" V1 ON (V0."id" = V1."app_id") WHERE (V1."device_id" IN (SELECT U0."device_id" FROM "users_userdevice" U0 WHERE U0."user_id" = 2) AND NOT (V0."package_name" IN (SELECT U2."package_name" FROM "offers_androidofferstate" U0 INNER JOIN "offers_androidoffer" U2 ON (U0."offer_id" = U2."id") WHERE (U0."device_id" = 1 AND (U0."state" = 0 OR U0."state" = 1 OR U0."state" = 2))))))) AND "offers_androidoffer_platform_versions"."platformversion_id" IN (14) AND "offers_androidoffer_countries"."country_id" IN (6252001)) ORDER BY "offers_androidoffer"."priority" DESC;
If I run this query in Postgresql console, it will return 0 rows, but active_offers has 4 results (all rows in table), like if I remove AND "offers_androidoffer_countries"."country_id" IN (6252001) statement.
I run this code from tests (APITestCase.client -> DRF view -> filter queryset). Django version is 2.0.2.
Why it ignores country filtering?
UPD. I've just checked with simple TestCase (test -> filter queryset) test and it returns correct number of rows. So, problem exists only with DRF testing.
UPD 2. Testcase where it works incorrectly:
class AndroidOffersListTests(APITestCase):
fixtures = [
'geo/fixtures/cities.json',
'offers/fixtures/users.json',
'offers/fixtures/devices.json',
'offers/fixtures/geo.json',
'offers/fixtures/apps.json',
'offers/fixtures/offers.json',
]
def test_list_offers_1(self):
user_device = UserDevice.objects.get(pk=1)
token = AndroidOffersListTests.get_token_for_device(user_device)
self.client.credentials(HTTP_AUTHORIZATION='Token {}'.format(token))
url = AndroidOffersListTests.get_url(user_device)
response = self.client.get(url)
self.assertEqual(status.HTTP_200_OK, response.status_code)
self.assertEqual(0, len(response.data)) # result is 4
View code:
class AndroidOffersView(ListAPIView):
model = AndroidOffer
serializer_class = AndroidOffersSerializer
permission_classes = (IsAuthenticated,)
def get_queryset(self):
device = UserDevice.get_from_request(self.request)
if device is None:
raise PermissionDenied()
return AndroidOffer.get_offers_for_device(device)
get_offers_for_device:
#staticmethod
def get_offers_for_device(user_device):
active_offers = AndroidOffer.get_active_offers()
# Filter completed
completed_states = AndroidOfferState.get_completed_for_device(user_device)
completed_offers_ids = completed_states.values_list('offer__pk', flat=True)
active_offers = active_offers.exclude(pk__in=completed_offers_ids)
# Filter apps already installed on the user's devices
apps = user_device.user.apps
# Remove packages that are in progress
in_progress_states = AndroidOfferState.get_in_progress_for_device(user_device)
in_progress_packages = in_progress_states.values_list('offer__package_name', flat=True)
apps = apps.exclude(package_name__in=in_progress_packages)
packages = apps.values_list('package_name', flat=True)
active_offers = active_offers.exclude(package_name__in=packages)
# Filter by platform version
active_offers = active_offers.filter(platform_versions__in=[user_device.device.version])
# Filter by country
country = user_device.last_geo_record.country
if country is not None:
active_offers = active_offers.filter(countries__in=[country])
return active_offers
Test case where it works fine:
class AndroidOffersListTests(TestCase):
fixtures = [
'geo/fixtures/cities.json',
'offers/fixtures/users.json',
'offers/fixtures/devices.json',
'offers/fixtures/geo.json',
'offers/fixtures/apps.json',
'offers/fixtures/offers.json',
]
def test_list_offers_1(self):
user_device = UserDevice.objects.get(pk=1)
offers = AndroidOffer.get_offers_for_device(user_device)
self.assertEqual(0, offers.count()) # 0 — thats ok
UPD 3: when I'm running the same request in browser, it works fine:

You said this response is incorrect:
self.assertEqual(0, len(response.data)) # result is 4
But you also say this JSON response is correct:
{
"count": 0,
"next": null,
"previous": null,
"results": []
}
You're using a paginated API here. The length of 4 is due to the number of keys present in the deserialized json:
>>> len(json.loads('{"count": 0, "next": null, "previous": null, "results": []}'))
4
Note that you don't need to actually call json.loads yourself, the DRF framework has already handled that for you when preparing the response - i.e. response.data will be a dict already.
In the "Test case where it works fine", you're dealing with the queryset directly:
self.assertEqual(0, offers.count()) # 0 — thats ok
^
|____ here you go to the database, no serializer!
If you want to check the number of results, from the paginated JSON api, then you'll need to drill down that page:
len_results = len(response.data['results'])
For a test that is expected to return 0 results, this is sufficient. But take care - if you ever have tests which you expect to generate more results than the page size (configured in the settings), you may also want to check the count, and next values. You'll have to make additional requests to subsequent pages to collect all results.

field__in checks if the field is in the list that you pass in to it.
You can get your desired behavior with just this
active_offers = active_offers.filter(countries=country)

Is there a dynamic query builder for Flask using Sqlalchemy?

A simple query looks like this
User.query.filter(User.name == 'admin')
In my code, I need to check the parameters that are being passed and then filter the results from the database based on the parameter.
For example, if the User table contains columns like username, location and email, the request parameter can contain either one of them or can have combination of columns. Instead of checking each parameter as shown below and chaining the filter, I'd like to create one dynamic query string which can be passed to one filter and can get the results back. I'd like to create a separate function which will evaluate all parameters and will generate a query string. Once the query string is generated, I can pass that query string object and get the desired result. I want to avoid using RAW SQL query as it defeats the purpose of using ORM.
if location:
User.query.filter(User.name == 'admin', User.location == location)
elif email:
User.query.filter(User.email == email)

You can apply filter to the query repeatedly:
query = User.query
if location:
query = query.filter(User.location == location)
if email:
query = query.filter(User.email == email)
If you only need exact matches, there’s also filter_by:
criteria = {}
# If you already have a dict, there are easier ways to get a subset of its keys
if location: criteria['location'] = location
if email: criteria['email'] = email
query = User.query.filter_by(**criteria)
If you don’t like those for some reason, the best I can offer is this:
from sqlalchemy.sql.expression import and_
def get_query(table, lookups, form_data):
conditions = [
getattr(table, field_name) == form_data[field_name]
for field_name in lookups if form_data[field_name]
]
return table.query.filter(and_(*conditions))
get_query(User, ['location', 'email', ...], form_data)

Late to write an answer but if anyone is looking for the answer then sqlalchemy-json-querybuilder can be useful. It can be installed as -
pip install sqlalchemy-json-querybuilder
e.g.
filter_by = [{
"field_name": "SomeModel.field1",
"field_value": "somevalue",
"operator": "contains"
}]
order_by = ['-SomeModel.field2']
results = Search(session, "pkg.models", (SomeModel,), filter_by=filter_by,order_by=order_by, page=1, per_page=5).results

https://github.com/kolypto/py-mongosql/
MongoSQL is a query builder that uses JSON as the input.
Capable of:
Choosing which columns to load
Loading relationships
Filtering using complex conditions
Ordering
Pagination
Example:
{
project: ['id', 'name'], // Only fetch these columns
sort: ['age+'], // Sort by age, ascending
filter: {
// Filter condition
sex: 'female', // Girls
age: { $gte: 18 }, // Age >= 18
},
join: ['user_profile'], // Load the 'user_profile' relationship
limit: 100, // Display 100 per page
skip: 10, // Skip first 10 rows
}