We Keep Coding

Simple request parsing without reqparse.RequestParser()

flask_restful.reqparse has been deprecated (https://flask-restful.readthedocs.io/en/latest/reqparse.html):
The whole request parser part of Flask-RESTful is slated for removal and will be replaced by documentation on how to integrate with other packages that do the input/output stuff better (such as marshmallow). This means that it will be maintained until 2.0 but consider it deprecated. Don’t worry, if you have code using that now and wish to continue doing so, it’s not going to go away any time too soon.
I've looked briefly at Marshmallow and still a bit confused about how to use it if I wanted to replace reqparse.RequestParser(). What would we write instead of something like the following:
from flask import Flask, request, Response
from flask_restful import reqparse
#app.route('/', methods=['GET'])
def my_api() -> Response:
parser = reqparse.RequestParser()
parser.add_argument('username', type=str, required=True)
args = parser.parse_args()
return {'message': 'cool'}, 200
(after half an hour of reading some more documentation…)
RequestParser looks at the MultiDict request.values by default (apparently query parameters, then form body parameters according to https://stackoverflow.com/a/16664376/5139284). So then we just need to validate the data in request.values somehow.
Here's a snippet of some relevant code from Marshmallow. It seems a good deal more involved than reqparse: first you create a schema class, then instantiate it, then have it load the request JSON. I'd rather not have to write a separate class for each API endpoint. Is there something more lightweight similar to reqparse, where you can write all the types of the argument validation information within the function defining your endpoint?
from flask import Flask, request, Response
from flask_restful import reqparse
from marshmallow import (
Schema,
fields,
validate,
pre_load,
post_dump,
post_load,
ValidationError,
)
class UserSchema(Schema):
id = fields.Int(dump_only=True)
email = fields.Str(
required=True, validate=validate.Email(error="Not a valid email address")
)
password = fields.Str(
required=True, validate=[validate.Length(min=6, max=36)], load_only=True
)
joined_on = fields.DateTime(dump_only=True)
user_schema = UserSchema()
#app.route("/register", methods=["POST"])
def register():
json_input = request.get_json()
try:
data = user_schema.load(json_input)
except ValidationError as err:
return {"errors": err.messages}, 422
# etc.

If your endpoints share any commonalities in schema, you can use fields.Nested() to nest definitions within each Marshmallow class, which may save on code writing for each endpoint. Docs are here.
For example, for operations that update a resource called 'User', you would likely need a standardised subset of user information to conduct the operation, such as user_id, user_login_status, user_authorisation_level etc. These can be created once and nested in new classes for more specific user operations, for example updating a user's account:
class UserData(Schema):
user_id = fields.Int(required=True)
user_login_status = fields.Boolean(required=True)
user_authentication_level = fields.Int(required=True)
# etc ....
class UserAccountUpdate(Schema):
created_date = fields.DateTime(required=True)
user_data = fields.Nested(UserData)
# account update fields...

Extend django-import-export's import form to specify fixed value for each imported row

I am using django-import-export 1.0.1 with admin integration in Django 2.1.1. I have two models
from django.db import models
class Sector(models.Model):
code = models.CharField(max_length=30, primary_key=True)
class Location(models.Model):
code = models.CharField(max_length=30, primary_key=True)
sector = ForeignKey(Sector, on_delete=models.CASCADE, related_name='locations')
and they can be imported/exported just fine using model resources
from import_export import resources
from import_export.fields import Field
from import_export.widgets import ForeignKeyWidget
class SectorResource(resources.ModelResource):
code = Field(attribute='code', column_name='Sector')
class Meta:
model = Sector
import_id_fields = ('code',)
class LocationResource(resources.ModelResource):
code = Field(attribute='code', column_name='Location')
sector = Field(attribute='sector', column_name='Sector',
widget=ForeignKeyWidget(Sector, 'code'))
class Meta:
model = Location
import_id_fields = ('code',)
and import/export actions can be integrated into the admin by
from django.contrib import admin
from import_export.admin import ImportExportModelAdmin
class SectorAdmin(ImportExportModelAdmin):
resource_class = SectorResource
class LocationAdmin(ImportExportModelAdmin):
resource_class = LocationResource
admin.site.register(Sector, SectorAdmin)
admin.site.register(Location, LocationAdmin)
For Reasons™, I would like to change this set-up so that a spreadsheet of Locations which does not contain a Sector column can be imported; the value of sector (for each imported row) should be taken from an extra field on the ImportForm in the admin.
Such a field can indeed be added by overriding import_action on the ModelAdmin as described in Extending the admin import form for django import_export. The next step, to use this value for all imported rows, is missing there, and I have not been able to figure out how to do it.

EDIT(2): Solved through the use of sessions. Having a get_confirm_import_form hook would still really help here, but even better would be having the existing ConfirmImportForm carry across all the submitted fields & values from the initial import form.
EDIT: I'm sorry, I thought I had this nailed, but my own code wasn't working as well as I thought it was. This doesn't solve the problem of passing along the sector form field in the ConfirmImportForm, which is necessary for the import to complete. Currently looking for a solution which doesn't involve pasting the whole of import_action() into an ImportMixin subclass. Having a get_confirm_import_form() hook would help a lot here.
Still working on a solution for myself, and when I have one I'll update this too.
Don't override import_action. It's a big complicated method that you don't want to replicate. More importantly, as I discovered today: there are easier ways of doing this.
First (as you mentioned), make a custom import form for Location that allows the user to choose a Sector:
class LocationImportForm(ImportForm):
sector = forms.ModelChoiceField(required=True, queryset=Sector.objects.all())
In the Resource API, there's a before_import_row() hook that is called once per row. So, implement that in your LocationResource class, and use it to add the Sector column:
def before_import_row(self, row, **kwargs):
sector = self.request.POST.get('sector', None)
if contract:
self.request.session['import_context_sector'] = sector
else:
# if this raises a KeyError, we want to know about it.
# It means that we got to a point of importing data without
# contract context, and we don't want to continue.
try:
sector = self.request.session['import_context_sector']
except KeyError as e:
raise Exception("Sector context failure on row import, " +
f"check resources.py for more info: {e}")
row['sector'] = sector
(Note: This code uses Django sessions to carry the sector value from the import form to the import confirmation screen. If you're not using sessions, you'll need to find another way to do it.)
This is all you need to get the extra data in, and it works for both the dry-run preview and the actual import.
Note that self.request doesn't exist in the default ModelResource - we have to install it by giving LocationResource a custom constructor:
def __init__(self, request=None):
super()
self.request = request
(Don't worry about self.request sticking around. Each LocationResource instance doesn't persist beyond a single request.)
The request isn't usually passed to the ModelResource constructor, so we need to add it to the kwargs dict for that call. Fortunately, Django Import/Export has a dedicated hook for that. Override ImportExportModelAdmin's get_resource_kwargs method in LocationAdmin:
def get_resource_kwargs(self, request, *args, **kwargs):
rk = super().get_resource_kwargs(request, *args, **kwargs)
rk['request'] = request
return rk
And that's all you need.

Single model dynamic database settings in Django

For example assume that I have 100 clients who uses WordPress and I have to write a service in Django which should return list of posts from WordPress's MySQL DB. The problem is 100 clients are having different database connection settings.
I know that I can use DatabaseRouter to switch databases which are already loaded in settings. But I don't know how to make a singe model class to use different database settings.
I have tried mutating settings.
I also tried mutating model's app_label.
But I later understood that mutating anyting in Django is meaning less.
My Requirements
I want to create a model and dynamically change database connection. List of connection can be in a managed database table. But I don't want to unnecessarily load all the connection settings or create multiple models.

I made something like that, but to change mongodb connections.
I created a GenericView that select the connection and use it on the get_queryset.
I'm using django rest framework, so I made something like this:
class SwitchDBMixinView(object):
model = None
fields = None
def initial(self, request, *args, **kwargs):
result = super().initial(request, *args, **kwargs)
if request.user.is_authenticated():
request.user.database_connection.register()
return result
def get_object(self, *args, **kwargs):
return super().get_object(*args, **kwargs).switch_db(self.get_db_alias())
def get_db_alias(self):
if self.request is None or not self.request.user.is_authenticated():
return DEFAULT_CONNECTION_NAME
return self.request.user.database_connection.name
def get_queryset(self):
return self.model.objects.using(self.get_db_alias()).all()
def perform_destroy(self, instance):
instance.switch_db(self.get_db_alias()).delete()
The model:
from mongoengine.connection import register_connection, get_connection
AUTH_USER_MODEL = getattr(settings, 'AUTH_USER_MODEL')
class Connection(models.Model):
class Meta:
pass
owner = models.OneToOneField(
AUTH_USER_MODEL,
related_name='database_connection',
)
uri = models.TextField(
default=DefaultMongoURI()
)
def register(self):
register_connection(
self.name,
host=self.uri,
tz_aware=True,
)
get_connection(
self.name,
reconnect=True
)
def get_name(self):
return 'client-%d' % self.owner.pk
name = property(get_name)
def __str__(self):
return self.uri

You may want to have a look at django.db.connections (in django/db/__init__.py) and django.db.utils.ConnectionHandler (which django.db.connections is an instance of). This should let you dynamically add new db configs without hacking settings.DATABASES (actually ConnectionHandler builds it's _databases attribute from settings.DATABASES). I can't tell for sure since I never tried but it should mostly boils down to
from django import db
def add_db(alias, connection_infos):
databases = db.connections.databases
if alias in databases:
either_raise_or_log_and_ignore(your choice)
db.connections.databases[alias] = connection_infos
where connection_infos is a mapping similar to the ones expected in settings.DATABASES.
Then it's mostly a matter of using Queryset.using(alias) for your queries, ie:
alias = get_alias_for_user(request.user)
posts = Post.objects.using(alias).all()
cf https://docs.djangoproject.com/en/1.11/topics/db/multi-db/#manually-selecting-a-database
The main problem with this IMHO (assuming you manage to make something that works out of the untested suggestion above) is that you will have to store databases users/password in clear somewhere which can be a major security issue. I don't know how much control you have on the databases admin part but it would be better if you could add a 'django' user with a same password (and appropriate permissions of course) on all those databases so you can keep the password in your settings file instead of having to keep it in your main db.

Graphene Django without Django Model?

I've successfully used Graphene-Django to successfully build several GraphQL calls. In all of those cases I populated, in whole or in part, a Django model and then returned the records I populated.
Now I have a situation where I'd like to return some data that I don't wish to store in the Django model. Is this possible to do with Graphene?
Robert

Robert_LY answered his own question perfectly in the comments, I'd just like to expand his solution.
My database-less model WordForm is generated automatically, without storing it in a database. I define it as a Django model as follows:
from django.db import models
class WordForm(models.Model):
value = models.CharField(max_length=100)
attributes = models.CharField(max_length=100)
In the schema I define the node and query like this:
class WordFormNode(DjangoObjectType):
class Meta:
model = WordForm
interfaces = (relay.Node, )
class Query(AbstractType):
word_forms = List(WordFormNode,query=String(),some_id=String())
def resolve_word_forms(self, args, context, info):
query= args['query']
some_id = from_global_id(args['some_id'])[1]
word_forms = []
# some logic to make WordForm objects with
# WordForm(value=value,attributes=attributes),
# then append them to list word_forms
return word_forms
You can pass as many arguments as you like to the List and access them in resolve_word_forms.

When you map your Django model to a GraphQL, it create a new model with GraphQL object types from the introspection of the Django model..
And nothing prevent you to combine this model with with plain GraphQL objects types, or mapped from an other third party persistence model

Returning non-predefined fields via. API with Tastypie in Django

I am using Tastypie for non-ORM data source (Amazon Dynamodb). I have gone through the official documentation for non-ORM source and found the following code:
class MessageResource(Resource):
# Just like a Django ``Form`` or ``Model``, we're defining all the
# fields we're going to handle with the API here.
uuid = fields.CharField(attribute='uuid')
user_uuid = fields.CharField(attribute='user_uuid')
message = fields.CharField(attribute='message')
created = fields.IntegerField(attribute='created')
I am new to Tastypie and what I understand is that fields uuid, message, created.. which are returned by API are defined over here. Is there any way that I return those fields that are not defined here i.e. all those fields returned by the dictionary in obj_get_list or obj_get.

You can use the dehydrade method. Simply add a new key to bundle.data.
def dehydrate(self, bundle):
for item in bundle.obj.iteritems():
bundle.data["new_key"] = "new_value"
return bundle