I've begin building a web application using Flask and Flask-SQLAlchemy. My experience with building web applications is very limited so I apologize before hand if I am being ambiguous.
The database for the web application is already built by another program running on the server. So all the tables are already predefined in the database. My web app and the other program running on the server will have read & write access to the database.
Recently I've decided to restructure my application and cater toward the model-view-controller schema, similar to Django, and to use blueprints for making things more manageable and neater. Similar to this https://github.com/mitsuhiko/flask/wiki/Large-app-how-to
...
app_name/
model.py
view.py
controller.py
__init__.py
...
So I've done a few tutorials and read up on Flask-SQLAlchemy but what I can't make sense for the life of me is how do I define my models in the application if my database is already predefined by another program??. and How can I use the SQLAlchemy ORM with a predefined database?
Previously, before I decided to restructure, I just used Pyhton's sqlite3 library for grabbing stuff from the database and displaying it on the web page. It wasn't to pretty and the way I was structuring things things would have gotten out of hand fast.
Does the other program also use sqlalchemy so that you can lift/share its models?
If not, you could try reflection: http://docs.sqlalchemy.org/en/latest/core/reflection.html
Or this experimental automap: http://docs.sqlalchemy.org/en/latest/orm/extensions/automap.html
And your question wasn't about this, but sqlite3 is a file database, not a client-server database. Not saying you will, but you may run into concurrency issues:
sqlite3 concurrent access
http://www.sqlite.org/lockingv3.html
The OS and filesystem plays a role in this.
Related
I have a python script that does several analysis and develop and update a database on a regular basis non-stop.
On the other hand, I am using a Django application to show the database online.
What is your suggestion for the best way to shake hands between these two parts?
I think of a simple text file or CSV file, but is there any better way to do this?
I have two ideas for you.
Django can access multiple different databases at at time
If your database can easily be connected to your Django app, then you could just connect it directly. For more information, take a look at the official documentation on this topic: https://docs.djangoproject.com/en/3.1/topics/db/multi-db/. You are talking about using a CSV file, so I assume that these apps are running on the same server, which is why this choice would probably make the most sense.
A microservice
In my job, I build deployable machine learning projects. That requires a database which holds the models and the data for the models as well as a database that handles all the user management, accounts, email information, etc. What we do there is attach a Flask app that just allows us to query the ML database as required from the main Django app.
I am setting a web server based on Django Framework with PostgreSQL. I need to update the models in my Django app every hour and I am wondering if is it better way to update the database from another script file using python and psycopg2 or using script that directly connects to Django models and update them using Django API for models? I use this Database only in Django web server.
Summarizing... What is better logic?
update_db.py that contains:
connect to database via psycopg2, update the tables
update_db.py that contains:
connect to Django app models, update them using Django API for models
I will run script every hour using Cron, and the Django Project is already connected to that specific database and it works fine. I am just asking for better performance and better way in logic sense. Data won't be updated very often but it would have big jsons.
Unless you have very compelling reasons to do otherwise, the obvious solution here is to use a custom management command and the ORM - your code will be at the obvious place (for someone else having to work on the project), can be tested as part of your whole project's test suite, and won't require having to mentally "translate" from raw SQL to Django ORM code (not to mention that you'll have more chance to catch a mismatch between your Django models and your script's code when your schema changes - from experience, raw sql scripts that aren't under test are usually forgotten when doing a schema migration).
FWIW, optimizing "for performances" doesn't really makes sense in your case, unless your updates are long and complex and do block access to the site - but even then the overhead of the ORM is certainly not going to be the main bottleneck (just make sure you use properly).
I noticed that most examples for accessing mysql from flask suggest using a plugin that calls init_app(app).
I was just wondering why that is as opposed to just using a mysql connector somewhere in your code as you need it?
Is it that flask does better resource management with request life cycles?
Using packages like flask-mysql or Flask-SQLAlchemy, they provided useful defaults and extra helpers that make it easier to accomplish common CRUD tasks.
All of such package are good at handling relationships between objects. You only need to create the objects and then the objects contain all the functions and helpers you needed to deal with the database, you don't have to implement such code by yourself and you don't need to worry about the performance of the queries.
I had worked on a Django project（I believe the theory in Flask is similar） and its ORM is really amazing, all i need to do is writing Models and encapsulate business logic. All CRUD commands are handled by the built-in ORM, as a developer we don't worry about the SQL statements.
Another benefit is that it makes database migration much easier. You can switch it from MySQL to PostgresSQL with minimal code modifications which will speed up development.
Basically, I am trying to share as much database layer code as possible between a Flask application (for a REST API) and a non-Flask API.
Is it a good idea to use the same Flask-SQLAlchemy layer in both the pure Python API (intended to be imported in non-web Python applications) and in the REST API Flask daemon?
I guess another way to phrase, though I am not sure on the terminology, "how do I best share database model between Flask app and a separate Python import library?"
Or from another angle, is there any point in using Flask-SQLAlchemy in a Flask REST API, if you also want to share the SQL abstraction with an import library. Is it better then to use plain SQLAlchemy?
Use case: we have a large database with many tables, and want to build both a REST API (for customer access) and a Python import library (for performant internal tools) for accessing the database, but of course share as much code between them as possible.
Related:
Using Flask-SQLAlchemy in Blueprint models without reference to the app
What's your folder layout for a Flask app divided in modules?
SQLAlchemy with Flask -- hybrid models?
Using Flask-SQLAlchemy models out of a web context is a matter of creating a Flask application and call
app.test_request_context().push()
The point here is what you will do with your "non web" library.
If it's not a problem to have the whole Flask library installed when you need to use the library then there's no problem at all using it in that way.
If you plan to make performance improvements in the library data accessing code, like using different sessions, concurrency and such, then you're modifying your initial code so it's a totally different scenario. In this case a pure-SQLAlchemy approach might be better but it really depends on the differences between the two patterns.
Normally with models comes methods and using 2 different ORM patterns (Flask-SQLAlchemy wrapper models and pure SQLAlchemy) means duplicating code.
I have two apps that both access the same database. The first has clients connecting via TCP and writes to the db using SQLAlchemy. The second is a more typical webapp using Django. Both have read/write requirements.
I would like to unify the database access layer, but picking just SQLAlchemy or just Django is unattractive because:
I would like to use django auth, permissions, and maybe third party plugins, which require the Django ORM (correct me if I'm wrong).
For the first app, using SQLAlchemy (so far) is much simpler than trying to use the Django ORM outside of a Django app - it is a TCP/IP server app, not a HTTP/web app.
Are there any problems with mixing these two ORMs on the same database?
In which system (Django, SQLA) should I create the models, vs using some kind of introspection such as Django inspectdb?
Firstly - it's not very hard to use Django ORM outside a manage.py, WSGI handlers and other HTTP related stuff. You can use it any python script, but it needs some initialization (example).
Secondly - SQLA is a very powerfull tool and it's able to do stuff which is very hard to achive in Django ORM (like genuine polymorphism and polymorphic queries). If I had to choose, I'd personally choose to use Django ORM as a platform to create models, then manually map them in SQLA since it is much more flexibile and hopefully will be able to adopt. Which may not work in the opposite case.
Finally, since you can use Django ORM on both sides, and you just have to use a Django ORM because of the plugins, I suggest to abandon the SQLA. It's a powerfull tool, but also rather complicated. Having two different ORMs operating on one database may result in unexpected problems in the future and increases the complexity of your app, so it'll be harder to maintenance.