Basically, I am trying to share as much database layer code as possible between a Flask application (for a REST API) and a non-Flask API.
Is it a good idea to use the same Flask-SQLAlchemy layer in both the pure Python API (intended to be imported in non-web Python applications) and in the REST API Flask daemon?
I guess another way to phrase, though I am not sure on the terminology, "how do I best share database model between Flask app and a separate Python import library?"
Or from another angle, is there any point in using Flask-SQLAlchemy in a Flask REST API, if you also want to share the SQL abstraction with an import library. Is it better then to use plain SQLAlchemy?
Use case: we have a large database with many tables, and want to build both a REST API (for customer access) and a Python import library (for performant internal tools) for accessing the database, but of course share as much code between them as possible.
Related:
Using Flask-SQLAlchemy in Blueprint models without reference to the app
What's your folder layout for a Flask app divided in modules?
SQLAlchemy with Flask -- hybrid models?
Using Flask-SQLAlchemy models out of a web context is a matter of creating a Flask application and call
app.test_request_context().push()
The point here is what you will do with your "non web" library.
If it's not a problem to have the whole Flask library installed when you need to use the library then there's no problem at all using it in that way.
If you plan to make performance improvements in the library data accessing code, like using different sessions, concurrency and such, then you're modifying your initial code so it's a totally different scenario. In this case a pure-SQLAlchemy approach might be better but it really depends on the differences between the two patterns.
Normally with models comes methods and using 2 different ORM patterns (Flask-SQLAlchemy wrapper models and pure SQLAlchemy) means duplicating code.
Related
I have a python script that does several analysis and develop and update a database on a regular basis non-stop.
On the other hand, I am using a Django application to show the database online.
What is your suggestion for the best way to shake hands between these two parts?
I think of a simple text file or CSV file, but is there any better way to do this?
I have two ideas for you.
Django can access multiple different databases at at time
If your database can easily be connected to your Django app, then you could just connect it directly. For more information, take a look at the official documentation on this topic: https://docs.djangoproject.com/en/3.1/topics/db/multi-db/. You are talking about using a CSV file, so I assume that these apps are running on the same server, which is why this choice would probably make the most sense.
A microservice
In my job, I build deployable machine learning projects. That requires a database which holds the models and the data for the models as well as a database that handles all the user management, accounts, email information, etc. What we do there is attach a Flask app that just allows us to query the ML database as required from the main Django app.
I noticed that most examples for accessing mysql from flask suggest using a plugin that calls init_app(app).
I was just wondering why that is as opposed to just using a mysql connector somewhere in your code as you need it?
Is it that flask does better resource management with request life cycles?
Using packages like flask-mysql or Flask-SQLAlchemy, they provided useful defaults and extra helpers that make it easier to accomplish common CRUD tasks.
All of such package are good at handling relationships between objects. You only need to create the objects and then the objects contain all the functions and helpers you needed to deal with the database, you don't have to implement such code by yourself and you don't need to worry about the performance of the queries.
I had worked on a Django project（I believe the theory in Flask is similar） and its ORM is really amazing, all i need to do is writing Models and encapsulate business logic. All CRUD commands are handled by the built-in ORM, as a developer we don't worry about the SQL statements.
Another benefit is that it makes database migration much easier. You can switch it from MySQL to PostgresSQL with minimal code modifications which will speed up development.
I've begin building a web application using Flask and Flask-SQLAlchemy. My experience with building web applications is very limited so I apologize before hand if I am being ambiguous.
The database for the web application is already built by another program running on the server. So all the tables are already predefined in the database. My web app and the other program running on the server will have read & write access to the database.
Recently I've decided to restructure my application and cater toward the model-view-controller schema, similar to Django, and to use blueprints for making things more manageable and neater. Similar to this https://github.com/mitsuhiko/flask/wiki/Large-app-how-to
...
app_name/
model.py
view.py
controller.py
__init__.py
...
So I've done a few tutorials and read up on Flask-SQLAlchemy but what I can't make sense for the life of me is how do I define my models in the application if my database is already predefined by another program??. and How can I use the SQLAlchemy ORM with a predefined database?
Previously, before I decided to restructure, I just used Pyhton's sqlite3 library for grabbing stuff from the database and displaying it on the web page. It wasn't to pretty and the way I was structuring things things would have gotten out of hand fast.
Does the other program also use sqlalchemy so that you can lift/share its models?
If not, you could try reflection: http://docs.sqlalchemy.org/en/latest/core/reflection.html
Or this experimental automap: http://docs.sqlalchemy.org/en/latest/orm/extensions/automap.html
And your question wasn't about this, but sqlite3 is a file database, not a client-server database. Not saying you will, but you may run into concurrency issues:
sqlite3 concurrent access
http://www.sqlite.org/lockingv3.html
The OS and filesystem plays a role in this.
I am writing a library for Pyramid. One aspect involves the library being provided with a model class, then retrieving all instances of the model from the database. However, I am unable to interact with the DB without access to the session factory.
In Django this was taken care of behind the scenes. With Pyramid & SQLAlchemy this is not the case.
Is there a standard way for me to get hold of the current thread's DB Session within Pyramid, and without having any knowledge of how a particular project is setup (as this is a reusable library)?
PS. I'm still getting my head around this area of SQLAlchemy, so please excuse any confusion.
You could specify sqla session as attribute in inherited classes, like factory-boy does:
https://factoryboy.readthedocs.org/en/latest/orms.html?highlight=sqlalchemy#sqlalchemy
Another way is create interface and require to register sqlalchemy session in application registry as utility, before "config.include" your extension. Maybe pyramid_jinja2 will clarify this solution.
I recommend reading about
a convention to add session to the request object. Your library makes just an assumption about that and write that in your package docs.
http://blog.safaribooksonline.com/2014/01/07/building-pyramid-applications/
Global vs. Non-Global session
http://docs.pylonsproject.org/projects/pyramid_cookbook/en/latest/database/sqlalchemy.html
A offical tutorial showing those concepts
http://docs.pylonsproject.org/projects/pyramid/en/master/quick_tutorial/databases.html
A lot of pyramid applications use the package zope.sqlalchemy to integrate application transaction management and DB session management. This approach is even recommended as one of many options by SQLAlchemy docs. Docs of zope.sqlalchemy are a bit confusing at least to me. The topic as a whole is a constant source of confusion to people starting with pyramid and thread-local sessions using SQLAlchemy.
To see a full-featured pyramid app that makes use of these packages look at ToDoPyramid - one of the sample application listed on pyramid docs pages
I cloned the project to make the database-related code at least more testable and readable to me. I found the concepts work very well - if the environment targetting the database is setup up properly.
I have two apps that both access the same database. The first has clients connecting via TCP and writes to the db using SQLAlchemy. The second is a more typical webapp using Django. Both have read/write requirements.
I would like to unify the database access layer, but picking just SQLAlchemy or just Django is unattractive because:
I would like to use django auth, permissions, and maybe third party plugins, which require the Django ORM (correct me if I'm wrong).
For the first app, using SQLAlchemy (so far) is much simpler than trying to use the Django ORM outside of a Django app - it is a TCP/IP server app, not a HTTP/web app.
Are there any problems with mixing these two ORMs on the same database?
In which system (Django, SQLA) should I create the models, vs using some kind of introspection such as Django inspectdb?
Firstly - it's not very hard to use Django ORM outside a manage.py, WSGI handlers and other HTTP related stuff. You can use it any python script, but it needs some initialization (example).
Secondly - SQLA is a very powerfull tool and it's able to do stuff which is very hard to achive in Django ORM (like genuine polymorphism and polymorphic queries). If I had to choose, I'd personally choose to use Django ORM as a platform to create models, then manually map them in SQLA since it is much more flexibile and hopefully will be able to adopt. Which may not work in the opposite case.
Finally, since you can use Django ORM on both sides, and you just have to use a Django ORM because of the plugins, I suggest to abandon the SQLA. It's a powerfull tool, but also rather complicated. Having two different ORMs operating on one database may result in unexpected problems in the future and increases the complexity of your app, so it'll be harder to maintenance.