You can see the code here
The concrete problem that I'm trying to solve is this.
Say that I need to provide REST interface to some entities modeled (sqlalchemy in my case) with some tool stored in database. Say that this collection is called parents.
I would need handlers like this
GET /parents
GET /parents/some_id
POST /parents
DELETE /parents/some_id
Conceptually all this handlers are very similar.
They all take ids from url, then create appropriate query. Then they fetching data with that query, then turn this data into dict and then call jsonify to create correct http response.
So with OOP I could design this like that.
class AbstractHandler():
def __init__(serializer, **specs):
self.specs = specs
def handle_request(self, spec_data, *_ids):
q = self.create_query(_ids)
d = self.fetch_data(self.specs[spec_data['name']](**(spec_data['args'] + (query, ))
out = serializer(d)
return jsonify(out)
The spec is a function that takes some parameters and query and produce more refined query based of this parameters.
So for example
GET /parents?spec={'name': 'by_name', 'args': ['adam'}
would return parent named Adam from collection.
This code has some flaws but I hope you see the point how template method makes flow of control here and subclasses can change how they would create query, how they would fetch data (item handler would need to call query.one() and collection handler would need to call query.all() for example)
So I can replace create_query, fetch_data with dependency injection instead. But that would create a problem that someone could create wrong configuration by giving the wrong dependency. That's basically what I've done, but with using partial functions instead.
So what I'm thinking right now is that I can solve this problem by creating factory functions for every type of handler that I need, that would give appropriate dependency to handler.
That's very much alike with template method solution I think. The difference basically is that in template method correctness dependencies is guarantied by object type and in my solution it's guarantied by type of factory function.
So enough with what I think, I'd like to know what you think about that?
How people in functional world solve this kinds of problem?
Related
So I am creating an API class in python, the goal of which is to allow me to create custom API routes that I can call methods on to make calls to APIs (as well as other helpful operations). However, I want this to be very general purpose and safe, as some of the API calls look a little different than others. What I want to know is, what is the best way to fill in an API call to a URL as a string using an OO approach? Take the following example:
#
# ...defined apiObject class up here somewhere (name tbd)...
#
myAPI = apiObject('https://jsonplaceholder.typicode.com/{}/{}')
# Note the {} as placeholders for the API calls
# Below is an example of calling the API
myAPI.callAPI('comments', '123')
# This would form the url 'https://jsonplaceholder.typicode.com/comments/123' make the call and do some stuff
# Similarly:
myAPI2 = apiObject('https://anotherrestapi.test.com/{}/{}/{}')
myAPI.callAPI('2020', 'Feb', '13')
# Again, would form the url 'https://anotherrestapi.test.com/2020/Feb/13' make the call and do some stuff
This is not a problem I can't fix, moreover, its a question about best programming practices in Python. My initial reaction is to use **kwargs in the callAPI function as I could then name the placeholders and fill in the url as needed.
Alternatively, is there a way I should form my API class such that this is more set in stone?
Or last of all, do you have any library recommendations for me to check out as an alternative to doing this myself?
I am not as familiar with Python as I am with other languages, so I thought I'd ask and see what other people think. Thanks in advance for taking the time to think about it.
To build a dynamic user update route on Flask, I have iterate over the user Flask SQLAlchemy object using the dunder __dict__:
parameters = ['name'] # Valid parameters after the Regex filter was applied
for parameter in parameters:
user.__dict__[parameter] = request.form.get(parameter)
I have done this to avoid the usage of ifs. To ensure that only valid parameters are present in parameters, I have applied a Regex pattern that filters the valid parameters received in the request for the user route, and I have documented this aspect on the doc string.
I'm asking if iterate over a Flask SQLAlchemy object using __dict__ is it a bad practice because if I print the user.__dict__, I receive all parameters, even those that aren't on the Regex filter, i.g, password, date created, etc; and should never be updated by this route.
I have found another approach that uses get all columns in SQLAlchemy, but I think that at the end its similar to the approach that I'm using...
Obs: the implemented route can update specific attributes from user or all of them, using the same route
I'd recommend looking into marshmallow-sqlalchemy to manage this sort of thing. I've found that there are very few use-cases where there is a problem, and __dict__ is the best solution.
Here's an example application using it: How do I produce nested JSON from database query with joins? Using Python / SQLAlchemy
I need to intercept all queries that concern a model in SQLAlchemy, in a way that I can inspect it at the point where any of the query methods (all(), one(), scalar(), etc.) is executed.
I have thought about the following approaches:
1. Subclass the Query class
I could subclass sqlalchemy.orm.Query and override the execution code, starting basically from something like this.
However, I am writing a library that can be used in other SQLAlchemy applications, and thus the creation of the declarative base, let alone engines and sessions, is outside my scope.
Maybe I have missed something and it is possible to override the Query class for my models without knowledge of the session?
2. Use the before_execute Core Event
I have also thought of hooking into execution with the before_execute event.
The problem is thatit is bound to an engine (see above). Also, I need to modify objects in the session, and I got the impression that I do not have access to a session from within this event.
What I want to be able to do is something like:
session.query(MyModel).filter_by(foo="bar").all() is executed.
Intercept that query and do something like storing the query in a log table within the same database (not literally that, but a set of different things that basically need the exact same functionality as this example operation)
Let the query execute like normal.
What I am trying to do in the end is inject items from another data store into the SQLAlchemy database on-the-fly upon querying. While this seems stupid - trust me, it might be less stupid than it sounds (or even more stupid) ;).
The before_compile query event might be useful for you.
from weakref import WeakSet
from sqlalchemy import event
from sqlalchemy.orm import Query
visited_queries = WeakSet()
#event.listens_for(Query, 'before_compile')
def log_query(query):
# You can get the session
session = query.session
# Prevent recursion if you want to compile the query to log it!
if query not in visited_queries:
visited_queries.add(query)
# do something with query.statement
You can look at query.column_descriptions to see if your model is being queried.
In MVC frameworks, where do you usually embed SQL Alchemy Code, is it ideal to put the query in Controller Methods or just use the Model Methods?
query = session.query(User, Document, DocumentsPermissions).join(Document).join(DocumentsPermissions).filter_by(Document.name=="Something")
Or I just delegate this to a Model Method which takes a args? What is the preferred way to do this? One of the benefits of the latter is that it can be re-used and it almost presents a view for the API programmers. Another advantage is that I can easily over-ride this if I make it a class method. This is usually helpful in customizations especially in commercial softwares.
#Ctrl.py
self.getdocument("Foo")
#Mdl.py
def getdocument(name):
query = session.query(User, Document, DocumentsPermissions).join(Document).join(DocumentsPermissions).filter_by(Document.name=="Something")
TL;DR: Isn't the concept of "M" in MVC blurred when you use ORM's like SQL Alchemy? I didn't have any problems with Model View Controller design patterns.
[PS: I am not sure if this belongs to Code Review Site, if so please let me know, I can just transfer over.]
I strongly prefer the second approach. It has a few advantages:
Your controller code can be dumb. This is good. Controllers that just fetch data from the backend, possibly reformat it a little bit, and pass it on to views are very easy to reason about.
It's easier to test that method in isolation. You can run getdocument('valid_name'), getdocument(None), getdocument(123), etc. to ensure they all work or fail as expected without dealing with all the surrounding controller code.
It's easier to test the controller. You can write a mock for getdocument() so that it always returns a known value and test that your controller processes it correctly.
I tend to put database query code in the Controller rather than the Model. As my understanding goes, Model methods are used to transform the data of the model into something else.
For example, a UserModel may have a FullName() method to return the concatenation of the user's first and last names.
Whereas, a UserController contains a GetAll() method to get a list of all users, which is where the database query code is found.
EDIT:
I have added [MVC] and [design-patterns] tags to expand the audience for this question as it is more of a generic programming question than something that has direclty to do with Python or SQLalchemy. It applies to all applications with business logic and an ORM.
The basic question is if it is better to keep business logic in separate modules, or to add it to the classes that our ORM provides:
We have a flask/sqlalchemy project for which we have to setup a structure to work in. There are two valid opinions on how to set things up, and before the project really starts taking off we would like to make our minds up on one of them.
If any of you could give us some insights on which of the two would make more sense and why, and what the advantages/disadvantages would be, it would be greatly appreciated.
My example is an HTML letter that needs to be sent in bulk and/or displayed to a single user. The letter can have sections that display an invoice and/or a list of articles for the user it is addressed to.
Method 1:
Split the code into 3 tiers - 1st tier: web interface, 2nd tier: processing of the letter, 3rd tier: the models from the ORM (sqlalchemy).
The website will call a server side method in a class in the 2nd tier, the 2nd tier will loop through the users that need to get this letter and it will have internal methods that generate the HTML and replace some generic fields in the letter, with information for the current user. It also has internal methods to generate an invoice or a list of articles to be placed in the letter.
In this method, the 3rd tier is only used for fetching data from the database and perhaps some database related logic like generating a full name from a users' first name and last name. The 2nd tier performs most of the work.
Method 2:
Split the code into the same three tiers, but only perform the loop through the collection of users in the 2nd tier.
The methods for generating HTML, invoices and lists of articles are all added as methods to the model definitions in tier 3 that the ORM provides. The 2nd tier performs the loop, but the actual functionality is enclosed in the model classes in the 3rd tier.
We concluded that both methods could work, and both have pros and cons:
Method 1:
separates business logic completely from database access
prevents that importing an ORM model also imports a lot of methods/functionality that we might not need, also keeps the code for the model classes more compact.
might be easier to use when mocking out ORM models for testing
Method 2:
seems to be in line with the way Django does things in Python
allows simple access to methods: when a model instance is present, any function it
performs can be immediately called. (in my example: when I have a letter-instance available, I can directly call a method on it that generates the HTML for that letter)
you can pass instances around, having all appropriate methods at hand.
Normally, you use the MVC pattern for this kind of stuff, but most web frameworks in python have dropped the "Controller" part for since they believe that it is an unnecessary component. In my development I have realized, that this is somewhat true: I can live without it. That would leave you with two layers: The view and the model.
The question is where to put business logic now. In a practical sense, there are two ways of doing this, at least two ways in which I am confrontet with where to put logic:
Create special internal view methods that handle logic, that might be needed in more than one view, e.g. _process_list_data
Create functions that are related to a model, but not directly tied to a single instance inside a corresponding model module, e.g. check_login.
To elaborate: I use the first one for strictly display-related methods, i.e. they are somehow concerned with processing data for displaying purposes. My above example, _process_list_data lives inside a view class (which groups methods by purpose), but could also be a normal function in a module. It recieves some parameters, e.g. the data list and somehow formats it (for example it may add additional view parameters so the template can have less logic). It then returns the data set to the original view function which can either pass it along or process it further.
The second one is used for most other logic which I like to keep out of my direct view code for easier testing. My example of check_login does this: It is a function that is not directly tied to display output as its purpose is to check the users login credentials and decide to either return a user or report a login failure (by throwing an exception, return False or returning None). However, this functionality is not directly tied to a model either, so it cannot live inside an ORM class (well it could be a staticmethod for the User object). Instead it is just a function inside a module (remember, this is Python, you should use the simplest approach available, and functions are there for something)
To sum this up: Display logic in the view, all the other stuff in the model, since most logic is somehow tied to specific models. And if it is not, create a new module or package just for logic of this kind. This could be a separate module or even a package. For example, I often create a util module/package for helper functions, that are not directly tied for any view, model or else, for example a function to format dates that is called from the template but contains so much python could it would be ugly being defined inside a template.
Now we bring this logic to your task: Processing/Creation of letters. Since I don't know exactly what processing needs to be done, I can only give general recommendations based on my assumptions.
Let's say you have some data and want to bring it into a letter. So for example you have a list of articles and a costumer who bought these articles. In that case, you already have the data. The only thing that may need to be done before passing it to the template is reformatting it in such a way that the template can easily use it. For example it may be desired to order the purchased articles, for example by the amount, the price or the article number. This is something that is independent of the model, the order is now only display related (you could have specified the order already in your database query, but let's assume you didn't). In this case, this is an operation your view would do, so your template has the data ready formatted to be displayed.
Now let's say you want to get the data to create a specifc letter, for example a list of articles the user bough over time, together with the date when they were bought and other details. This would be the model's job, e.g. create a query, fetch the data and make sure it is has all the properties required for this specifc task.
Let's say in both cases you with to retrieve a price for the product and that price is determined by a base value and some percentages based on other properties: This would make sense as a model method, as it operates on a single product or order instance. You would then pass the model to the template and call the price method inside it. But you might as well reformat it in such a way, that the call is made already in the view and the template only gets tuples or dictionaries. This would make it easier to pass the same data out as an API (see below) but it might not necessarily be the easiest/best way.
A good rule for this decision is to ask yourself If I were to provide a JSON API additionally to my standard view, how would I need to modify my code to be as DRY as possible?. If theoretical is not enough at the start, build some APIs for the templates and see where you need to change things to the API makes sense next to the views themselves. You may never use this API and so it does not need to be perfect, but it can help you figure out how to structure your code. However, as you saw above, this doesn't necessarily mean that you should do preprocessing of the data in such a way that you only return things that can be turned into JSON, instead you might want to make some JSON specifc formatting for the API view.
So I went on a little longer than I intended, but I wanted to provide some examples to you because that is what I missed when I started and found out those things via trial and error.