We Keep Coding

flask-smorest response and return type different

I am learning/working on a Rest Api suing flask-smorest and adding the schema using marshmallow.
Below is the code that I am confused with and have a question.
Schemas.py
class ChildAddressDetailsSchema(Schema):
class Meta:
unknown = EXCLUDE
address_id = fields.String(required=True)
address_type = fields.String(required=True)
is_primary = fields.Boolean(required=True)
class ChildAddressDetailsSchemaList(Schema):
class Meta:
unknown = EXCLUDE
person_list = fields.List(fields.Nested(ChildAddressDetailsSchema))
Endpoint Implementation
#address_blueprint.response(status_code=200, schema=ChildAddressDetailsSchema)
#address_blueprint.get('/child/address/<string:person_id>/list')
def get_child_address(person_id):
person_address_list = PersonAddressModel.query.filter_by(person_id=person_id).all()
person_address_dict = [{'address_id': person_address.address_id,
'address_type': person_address.address_type,
'is_primary': person_address.is_primary} for person_address in person_address_list]
return person_address_dict
The part where I have doubt is even though the schema defined in response of blueprint is
ChildAddressDetailsSchema which is not a list , still I get a valid response.Below is the screenshot of the Insomnia from where I am testing the api.
I was expecting an empty response or a error since the return of the get function get_child_address is a list of dictionary which is not as per the schema. Could someone please help me figuring out on to how to fix the issue and return type is strictly informed. Is this something that needs to be coded or does marshmallow handles this.

It's because you called Blueprint.response() before Blueprint.get(). So do like this.
#address_blueprint.get('/child/address/<string:person_id>/list')
#address_blueprint.response(status_code=200, schema=ChildAddressDetailsSchema)
def get_child_address(person_id):
...
A Python decorator returns a new function that calls the original function. So the order of decorators matters in general. In this case, the implementation of the ResponseMixin.response() of Flask Smorest does not work correctly if the Blueprint.route()(which is equivalent to the Scaffold.get()) is not called before.
You can see that on this and this. If the ResponseMixin.response() is called before the Blueprint.route(), the closure wrapper(created at the decorator() inside the ResponseMixin.response()) will be ignored, because the add_url_rule() will be called with the original endpoint function not the wrapper, at the decorator() inside the Blueprint.route().

Simple request parsing without reqparse.RequestParser()

flask_restful.reqparse has been deprecated (https://flask-restful.readthedocs.io/en/latest/reqparse.html):
The whole request parser part of Flask-RESTful is slated for removal and will be replaced by documentation on how to integrate with other packages that do the input/output stuff better (such as marshmallow). This means that it will be maintained until 2.0 but consider it deprecated. Don’t worry, if you have code using that now and wish to continue doing so, it’s not going to go away any time too soon.
I've looked briefly at Marshmallow and still a bit confused about how to use it if I wanted to replace reqparse.RequestParser(). What would we write instead of something like the following:
from flask import Flask, request, Response
from flask_restful import reqparse
#app.route('/', methods=['GET'])
def my_api() -> Response:
parser = reqparse.RequestParser()
parser.add_argument('username', type=str, required=True)
args = parser.parse_args()
return {'message': 'cool'}, 200
(after half an hour of reading some more documentation…)
RequestParser looks at the MultiDict request.values by default (apparently query parameters, then form body parameters according to https://stackoverflow.com/a/16664376/5139284). So then we just need to validate the data in request.values somehow.
Here's a snippet of some relevant code from Marshmallow. It seems a good deal more involved than reqparse: first you create a schema class, then instantiate it, then have it load the request JSON. I'd rather not have to write a separate class for each API endpoint. Is there something more lightweight similar to reqparse, where you can write all the types of the argument validation information within the function defining your endpoint?
from flask import Flask, request, Response
from flask_restful import reqparse
from marshmallow import (
Schema,
fields,
validate,
pre_load,
post_dump,
post_load,
ValidationError,
)
class UserSchema(Schema):
id = fields.Int(dump_only=True)
email = fields.Str(
required=True, validate=validate.Email(error="Not a valid email address")
)
password = fields.Str(
required=True, validate=[validate.Length(min=6, max=36)], load_only=True
)
joined_on = fields.DateTime(dump_only=True)
user_schema = UserSchema()
#app.route("/register", methods=["POST"])
def register():
json_input = request.get_json()
try:
data = user_schema.load(json_input)
except ValidationError as err:
return {"errors": err.messages}, 422
# etc.

If your endpoints share any commonalities in schema, you can use fields.Nested() to nest definitions within each Marshmallow class, which may save on code writing for each endpoint. Docs are here.
For example, for operations that update a resource called 'User', you would likely need a standardised subset of user information to conduct the operation, such as user_id, user_login_status, user_authorisation_level etc. These can be created once and nested in new classes for more specific user operations, for example updating a user's account:
class UserData(Schema):
user_id = fields.Int(required=True)
user_login_status = fields.Boolean(required=True)
user_authentication_level = fields.Int(required=True)
# etc ....
class UserAccountUpdate(Schema):
created_date = fields.DateTime(required=True)
user_data = fields.Nested(UserData)
# account update fields...

Using JSON data in views

I've been working on Django-rest-framework for past few months. I did not quite understood how the json request can be used.Can you please help me, I got stuck here for months.
I have a third-party html snippet on another website and lets assume it sends this json data
[{"idcomment":1,"isFlagged":false,"isDeleted":false,"isApproved":true,"createdAt":"2015-11-22T12:39:33Z","numReports":0,"isEdited":false,"message":"xcvvzvc","isSpam":false,"isHighlighted":false,"ip_address":"","is_public":true,"tree_path":"0000000001","tone":"Neutral","forum":1,"parent":null,"topic":1,"last_child":null,"user":1}][{"idcomment":1,"isFlagged":false,"isDeleted":false,"isApproved":true,"createdAt":"2015-11-22T12:39:33Z","numReports":0,"isEdited":false,"message":"xcvvzvc","isSpam":false,"isHighlighted":false,"ip_address":"","is_public":true,"tree_path":"0000000001","tone":"Neutral","forum":1,"parent":null,"topic":1,"last_child":null,"user":1}]
My question: How can I use this json request data and do some verification like if the comment belongs to the correct topic.
I couldn't find any examples where its done. So I figured its possible in Flask by using something like this.
mod.route("/create/", methods=["POST"])
def create():
json = getJson(request)
check_required(json, ['date', 'thread', 'message', 'user', 'forum'])
uid = id_by_email(json['user'])
fid = id_by_sname(json['forum'])
if uid < 0 or fid < 0:
return send_resp(json)
if 'parent' in json:
parent = json['parent']
else:
parent = None
json['parent'] = None
if 'isApproved' in json:
approved = json['isApproved']
else:
approved = 0
json['isApproved'] = 0
if 'isHighlighted' in json:
highlighted = json['isHighlighted']
else:
highlighted = 0
json['isHighlighted'] = 0
if 'isEdited' in json:
edited = json['isEdited']
else:
edited = 0
json['isEdited'] = 0
if 'isSpam' in json:
spam = json['isSpam']
else:
spam = 0
json['isSpam'] = 0
if 'isDeleted' in json:
deleted = json['isDeleted']
else:
deleted = 0
json['isDeleted'] = 0
db.insert("""INSERT INTO posts (date,thread_id,message,user_id,forum_id,parent,approved,highlighted,edited,spam,deleted)
values (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)""", (
json['date'], json['thread'], json['message'], uid, fid, parent, approved, highlighted, edited, spam, deleted))
pid = db.query("SELECT LAST_INSERT_ID() as id")[0]['id']
json['id'] = pid
return send_resp(json)
What is the alternative for something like this in django-rest-framework.
I am quite new, so please explain in simple language. I don't need any code, just searching how can I use json requests I get to the server.
It's a no-brainer I read the documentation several times
Disclaimer: I have also gonna all over through the django-rest-framework code

This job is handled by your Serializer Class :
The first thing we need to get started on our Web API is to provide a
way of serializing and deserializing the snippet instances into
representations such as json. We can do this by declaring serializers
that work very similar to Django's forms.
A serializer class is very similar to a Django Form class, and
includes similar validation flags on the various fields, such as
required, max_length and default.
If your serializer inherit of serializer.ModelSerializer then it use your model do validate your data like form class that inherit from form.ModelForm. And yes, you don't re-write any code to validate your data, and it seems to be magical. (DRY concept) But, of-course, you can define and override models validators in your serializer class.
I recommend you to re-read Django-Rest-Framework and specially Serializer part. It explain all serializers class that DRF provides with use cases.

Update row (SQLAlchemy) with data from marshmallow

I'm using Flask, Flask-SQLAlchemy, Flask-Marshmallow + marshmallow-sqlalchemy, trying to implement REST api PUT method. I haven't found any tutorial using SQLA and Marshmallow implementing update.
Here is the code:
class NodeSchema(ma.Schema):
# ...
class NodeAPI(MethodView):
decorators = [login_required, ]
model = Node
def get_queryset(self):
if g.user.is_admin:
return self.model.query
return self.model.query.filter(self.model.owner == g.user)
def put(self, node_id):
json_data = request.get_json()
if not json_data:
return jsonify({'message': 'Invalid request'}), 400
# Here is part which I can't make it work for me
data, errors = node_schema.load(json_data)
if errors:
return jsonify(errors), 422
queryset = self.get_queryset()
node = queryset.filter(Node.id == node_id).first_or_404()
# Here I need some way to update this object
node.update(data) #=> raises AttributeError: 'Node' object has no attribute 'update'
# Also tried:
# node = queryset.filter(Node.id == node_id)
# node.update(data) <-- It doesn't if know there is any object
# Wrote testcase, when user1 tries to modify node of user2. Node doesn't change (OK), but user1 gets status code 200 (NOT OK).
db.session.commit()
return jsonify(), 200

UPDATED, 2022-12-08
Extending the ModelSchema from marshmallow-sqlalchemy instead of Flask-Marshmallow you can use the load method, which is defined like this:
load(data, *, session=None, instance=None, transient=False, **kwargs)
Putting that to use, it should look like that (or similar query):
node_schema.load(json_data, session= current_app.session, instance=Node().query.get(node_id))
And if you want to load without all required fields of Model, you can add the partial=True argument, like this:
node_schema.load(json_data, instance=Node().query.get(node_id), partial=True)
See the docs for more info (does not include definition of ModelSchema.load).
See the code for the load definition.

I wrestled with this issue for some time, and in consequence came back again and again to this post. In the end what made my situation difficult was that there was a confounding issue involving SQLAlchemy sessions. I figure this is common enough to Flask, Flask-SQLAlchemy, SQLAlchemy, and Marshmallow, to put down a discussion. I certainly, do not claim to be an expert on this, and yet I believe what I state below is essentially correct.
The db.session is, in fact, closely tied to the process of updating the DB with Marshmallow, and because of that decided to to give the details, but first the short of it.
Short Answer
Here is the answer I arrived at for updating the database using Marshmallow. It is a different approach from the very helpful post of Jair Perrut. I did look at the Marshmallow API and yet was unable to get his solution working in the code presented, because at the time I was experimenting with his solution I was not managing my SQLAlchemy sessions properly. To go a bit further, one might say that I wasn't managing them at all. The model can be updated in the following way:
user_model = user_schema.load(user)
db.session.add(user_model.data)
db.session.commit()
Give the session.add() a model with primary key and it will assume an update, leave the primary key out and a new record is created instead. This isn't all that surprising since MySQL has an ON DUPLICATE KEY UPDATE clause which performs an update if the key is present and creates if not.
Details
SQLAlchemy sessions are handled by Flask-SQLAlchemy during a request to the application. At the beginning of the request the session is opened, and when the request is closed that session is also closed. Flask provides hooks for setting up and tearing down the application where code for managing sessions and connections may be found. In the end, though, the SQLAlchemy session is managed by the developer, and Flask-SQLAlchemy just helps. Here is a particular case that illustrates the management of sessions.
Consider a function that gets a user dictionary as an argument and uses that with Marshmallow() to load the dictionary into a model. In this case, what is required is not the creation of a new object, but the update of an existing object. There are 2 things to keep in mind at the start:
The model classes are defined in a python module separate from any code, and these models require the session. Often the developer (Flask documentation) will put a line db = SQLAlchemy() at the head of this file to meet this requirement. This in fact, creates a session for the model.
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
In some other separate file there may be a need for a SQLAlchemy session as well. For example, the code may need to update the model, or create a new entry, by calling a function there. Here is where one might find db.session.add(user_model) and db.session.commit(). This session is created in the same way as in the bullet point above.
There are 2 SQLAlchemy sessions created. The model sits in one (SignallingSession) and the module uses its own (scoped_session). In fact, there are 3. The Marshmallow UserSchema has sqla_session = db.session: a session is attached to it. This then is the third, and the details are found in the code below:
from marshmallow_sqlalchemy import ModelSchema
from donate_api.models.donation import UserModel
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class UserSchema(ModelSchema):
class Meta(object):
model = UserModel
strict = True
sqla_session = db.session
def some_function(user):
user_schema = UserSchema()
user['customer_id'] = '654321'
user_model = user_schema.load(user)
# Debug code:
user_model_query = UserModel.query.filter_by(id=3255161).first()
print db.session.object_session(user_model_query)
print db.session.object_session(user_model.data)
print db.session
db.session.add(user_model.data)
db.session.commit()
return
At the head of this module the model is imported, which creates its session, and then the module will create its own. Of course, as pointed out there is also the Marshmallow session. This is entirely acceptable to some degree because SQLAlchemy allows the developer to manage the sessions. Consider what happens when some_function(user) is called where user['id'] is assigned some value that exists in the database.
Since the user includes a valid primary key then db.session.add(user_model.data) knows that it is not creating a new row, but updating an existing one. This behavior should not be surprising, and is to be at least somewhat expected since from the MySQL documentation:
13.2.5.2 INSERT ... ON DUPLICATE KEY UPDATE Syntax
If you specify an ON DUPLICATE KEY UPDATE clause and a row to be inserted would cause a duplicate value in a UNIQUE index or PRIMARY KEY, an UPDATE of the old row occurs.
The snippet of code is then seen to be updating the customer_id on the dictionary for the user with primary key 32155161. The new customer_id is '654321'. The dictionary is loaded with Marshmallow and a commit done to the database. Examining the database it can be found that it was indeed updated. You might try two ways of verifying this:
In the code: db.session.query(UserModel).filter_by(id=325516).first()
In MySQL: select * from user
If you were to consider the following:
In the code: UserModel.query.filter_by(id=3255161).customer_id
You would find that the query brings back None. The model is not synchronized with the database. I have failed to manage our SQLAlchemy sessions correctly. In an attempt to bring clarity to this consider the output of the print statements when separate imports are made:
<sqlalchemy.orm.session.SignallingSession object at 0x7f81b9107b90>
<sqlalchemy.orm.session.SignallingSession object at 0x7f81b90a6150>
<sqlalchemy.orm.scoping.scoped_session object at 0x7f81b95eac50>
In this case the UserModel.query session is different from the Marshmallow session. The Marshmallow session is what gets loaded and added. This means that querying the model will not show our changes. In fact, if we do:
db.session.object_session(user_model.data).commit()
The model query will now bring back the updated customer_id! Consider the second alternative where the imports are done through flask_essentials:
from flask_sqlalchemy import SQLAlchemy
from flask_marshmallow import Marshmallow
db = SQLAlchemy()
ma = Marshmallow()
<sqlalchemy.orm.session.SignallingSession object at 0x7f00fe227910>
<sqlalchemy.orm.session.SignallingSession object at 0x7f00fe227910>
<sqlalchemy.orm.scoping.scoped_session object at 0x7f00fed38710>
And the UserModel.query session is now the same as the user_model.data (Marshmallow) session. Now the UserModel.query does reflect the change in the database: the Marshmallow and UserModel.query sessions are the same.
A note: the signalling session is the default session that Flask-SQLAlchemy uses. It extends the default session system with bind selection and modification tracking.

I have rolled out own solution. Hope it helps someone else. Solution implements update method on Node model.
Solution:
class Node(db.Model):
# ...
def update(self, **kwargs):
# py2 & py3 compatibility do:
# from six import iteritems
# for key, value in six.iteritems(kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
class NodeAPI(MethodView):
decorators = [login_required, ]
model = Node
def get_queryset(self):
if g.user.is_admin:
return self.model.query
return self.model.query.filter(self.model.owner == g.user)
def put(self, node_id):
json_data = request.get_json()
if not json_data:
abort(400)
data, errors = node_schema.load(json_data) # validate with marshmallow
if errors:
return jsonify(errors), 422
queryset = self.get_queryset()
node = queryset.filter(self.model.id == node_id).first_or_404()
node.update(**data)
db.session.commit()
return jsonify(message='Successfuly updated'), 200

Latest Update [2020]:
You might facing the issue of mapping keys to the database models. Your request body have only updated fields so, you want to change only those without affecting others. There is an option to write multiple if conditions but that's not a good approach.
Solution
You can implement patch or put methods using sqlalchemy library only.
For example:
YourModelName.query.filter_by(
your_model_column_id = 12 #change 12: where condition to find particular row
).update(request_data)
request_data should be dict object. For ex.
{
"your_model_column_name_1": "Hello",
"your_model_column_name_2": "World",
}
In above case, only two columns will be updated that is: your_model_column_name_1 and your_model_column_name_2
Update function maps request_data to the database models and creates update query for you. Checkout this: https://docs.sqlalchemy.org/en/13/core/dml.html#sqlalchemy.sql.expression.update

Previous answer seems to be outdated as ModelSchema is now deprecated.
You should instead SQLAlchemyAutoSchema with the proper options.
class NodeSchema(SQLAlchemyAutoSchema):
class Meta:
model = Node
load_instance = True
sqla_session = db.session
node_schema = NodeSchema()
# then when you need to update a Node orm instance :
node_schema.load(node_data, instance=node, partial=True)
db.session.update()

Below is my solution with Flask-Marshmallow + marshmallow-sqlalchemy bundle as the author requested initially.
schemas.py
from flask import current_app
from flask_marshmallow import Marshmallow
from app.models import Node
ma = Marshmallow(current_app)
class NodeSchema(ma.SQLAlchemyAutoSchema):
class Meta:
model = Node
load_instance = True
load_instance is a key point here to make an update further.
routes.py
from flask import jsonify, request
from marshmallow import ValidationError
from app import db
#bp.route("/node/<node_uuid>/edit", methods=["POST"])
def edit_node(node_uuid):
json_data = request.get_json(force=True, silent=True)
node = Node.query.filter_by(
node_uuid=node_uuid
).first()
if node:
try:
schema = NodeSchema()
json_data["node_uuid"] = node_uuid
node = schema.load(json_data, instance=node)
db.session.commit()
return schema.jsonify(node)
except ValidationError as err:
return jsonify(err.messages), 422
else:
return jsonify("Not found"), 404
You have to check for existence of Node first, otherwise the new instance will be created.

Get model object from tastypie uri?

How do you get the model object of a tastypie modelresource from it's uri?
for example:
if you were given the uri as a string in python, how do you get the model object of that string?

Tastypie's Resource class (which is the guy ModelResource is subclassing ) provides a method get_via_uri(uri, request). Be aware that his calls through to apply_authorization_limits(request, object_list) so if you don't receive the desired result make sure to edit your request in such a way that it passes your authorisation.
A bad alternative would be using a regex to extract the id from your url and then use it to filter through the list of all objects. That was my dirty hack until I got get_via_uri working and I do NOT recommend using this. ;)
id_regex = re.compile("/(\d+)/$")
object_id = id_regex.findall(your_url)[0]
your_object = filter(lambda x: x.id == int(object_id),YourResource().get_object_list(request))[0]

You can use get_via_uri, but as #Zakum mentions, that will apply authorization, which you probably don't want. So digging into the source for that method we see that we can resolve the URI like this:
from django.core.urlresolvers import resolve, get_script_prefix
def get_pk_from_uri(uri):
prefix = get_script_prefix()
chomped_uri = uri
if prefix and chomped_uri.startswith(prefix):
chomped_uri = chomped_uri[len(prefix)-1:]
try:
view, args, kwargs = resolve(chomped_uri)
except Resolver404:
raise NotFound("The URL provided '%s' was not a link to a valid resource." % uri)
return kwargs['pk']
If your Django application is located at the root of the webserver (i.e. get_script_prefix() == '/') then you can simplify this down to:
view, args, kwargs = resolve(uri)
pk = kwargs['pk']

Are you looking for the flowchart? It really depends on when you want the object.
Within the dehydration cycle you simple can access it via bundle, e.g.
class MyResource(Resource):
# fields etc.
def dehydrate(self, bundle):
# Include the request IP in the bundle if the object has an attribute value
if bundle.obj.user:
bundle.data['request_ip'] = bundle.request.META.get('REMOTE_ADDR')
return bundle
If you want to manually retrieve an object by an api url, given a pattern you could simply traverse the slug or primary key (or whatever it is) via the default orm scheme?