We're using two backends languages, JAVA and Python (Django), to handle different modules in the same project.
As shown above, in the database, pl_**** tables work for the JAVA part and qa_**** tables work for my Django part.
Right now, my Django part want to use the pl_user table, what should I do to access the data? Do I need add anything in the models.py? Or, do I need make some relationships between pl_user table and auth_user table?
Yes, you need to define models, which inspectdb can do for you automatically. You will need to mark them as managed = False so that migrations won't be generated for them.
Related
I'm a few weeks into Python/Django and encountering an annoying problem. I have some existing databases set up in settings.py, everything looks good, I've even accessed the databases using connections[].cursor()
But the databases (and data) are not making their way into models that I want to use, despite doing the makemigrations and migrate commands. I was able to use py manage.py inspectdb --database-dbname and copied that class information manually into my models.py, but that didn't work either (typing py manage.py inspectdb on its own does not pull up these databases, I was only able to view by that --database extension). So I'm stumped, as it seems I'm doing all the right steps but not able to use these existing databases in Django.
Any other hints and steps I can take are welcome!
(Almost) all the tutorials, examples, and third-party app you'll find on the internet, and most of the Django documentation assume you use one database for your app. That's because it's fairly tricky and unusual to use multiple databases in one app.
But it's not impossible to use multiple databases and the documentation contains instructions on how to do this and what changes you'll need to make to make it work.
IMO, these are the pre-conditions to use multiple databases in one project:
The databases contain explicitly unrelated information, i.e. you won't have SQL relationships between tables in different databases. One database may contain a table with a column that maps to a column in a table in another database, but they aren't explicit (no ForeignKey or ManyToManyField in your models).
You don't need to mix databases in one query: This basically derives from the previous condition. It just means that if you need to get objects from one database that depend on the rows coming from another database, you establish the relationship in python. E.g. fetching as list of names from one database and using that list to filter a queryset on the other database.
For example, if you have an existing database that contains Strava routes (which are regularly updated via some external mechanism) and your app is a broader app that helps users getting to know their neighbourhood where they can recommend locations and things to do, being able to offer a list of routes with a starting point nearby might be something you'd want to show.
Now that you know this, the way to go is described in the doc linked above:
Create a database router so that queries for certain models are automatically routed to the correct database. E.g. Route.objects.filter(start_city=city) would automatically fetch routes from your Strava routes database.
If you need to save information about a route in your app, save it in a model in the default database and use a unique identifier of the route that will map to the strava database. Use separate queries (no relationships) to fetch information about a specific route.
That being said, if the Strava database is not regularly updated via 3rd channels and its purpose is just to pre-populate your default database, then export the data from the Strava database as json and import it into your django db using manage.py loaddata or a migration file, the latter being more flexible as to the structure of the json file.
I have a typical Django project with one primary database where I keep all the data I need.
Suppose there is another DB somewhere with some additional information. That DB isn't directly related to my Django project so let's assume I do not even have a control under it.
The problem is that I do ont know if I need to create and maintain a model for this external DB so I could use Django's ORM. Or maybe the best solution is to use raw SQL to fetch data from external DB and then use this ifo to filter data from primary DB using ORM, or directly in views.
The solution with creating a model seems to be quite ok but the fact that DB isn't a part of my project means I am not aware of possible schema changes and looks like it's a bad practice then.
So in the end if I have some external resources like DBs that are not related to but needed for my project should I:
Try to create django models for them
Use raw SQL to get info from external DB and then use it for filtering data from the primary DB with ORM as well as using data directly in views if needed
Use raw SQL both for a primary and an external DB where they intersect in app's logic
An alternative is to use SQLAlchemy for the external database. It can use reflection to generate the SQLAlchemy-equivalent of django models during runtime.
It still won't be without issues. If your code depends on a certain column, it would still break if that column is removed or changed in an incompatible way. However, it will add a bit more flexibility to your database interactions, e.g. a Django model would definitely break if an int column is changed to a varchar column, but using database reflection, it will only break if your code depends on the fact that it is an int. If you simply display the data or something, it will remain fully functional. However, there is always a chance that a change doesn't break the system, but causes unexpected behaviour.
If, like Benjamin said, the external system has an API, that would be the preferred choice.
I suggest you to read about inspectdb and database routers. It's possible to use the django ORM to manipulate a external DB.
https://docs.djangoproject.com/en/1.7/ref/django-admin/#inspectdb
I would create the minimal django models on the external databases => those that interact with your code:
Several outcomes to this
If parts of the database you're not interested in change, it won't have an impact on your app.
If the external models your using change, you probably want to be aware of that as quickly as possible (your app is likely to break in that case too).
All the relational databases queries in your code are handled by the same ORM.
Django: If I added new tables to database, how can I query them?
Do I need to create the relevant models first? Or django creates it by itself?
More specifically, I installed another django app, it created several database tables in database, and now I want to get some specific data from them? What are the correct approaches? Thank you very much!
I suppose another django app has all model files needed to access those tables, you should just try importing those packages and use this app's models.
Django doen't follow convention over configuration philosophy. you have to explicitly create the backing model for the table and in the meta tell it about the table name...
How do I make syncdb execute SQL queries (for table creation) defined by me, rather then generating tables automatically.
I'm looking for this solution as some particular models in my app represent SQL-table-views for a legacy-database table.
So, I've created their SQL-views in my django-DB like this:
CREATE VIEW legacy_series AS SELECT * FROM legacy.series;
I have a reverse engineered model that represents the above view/legacytable. But whenever I run syncdb, I have to create all the views first by running sql scripts, otherwise syncdb simply creates tables for them (if a view is not found).
How do I make syncdb run the above mentioned SQL?
There are 2 possible approaches I know of to adapt your models to a legacy database table (without using views that is):
1) Run python manage.py inspectdb within your project. This will generate models for existing database tables, you can then continue to work with those.
2) Modify your tables with some specific settings. First of all you define the table name in your model by setting the db_table option in your meta options. Secondly you define for each field the column name to match your legacy database by setting the db_column option. Note there are other db_ options listed you possibly could use to match your legacy database.
If you really want the views approach an (ugly) workaround is possible, you can define custom sql commands per application model. This file is found in "application"/sql/"model".sql . Django will call this sql's after it created all tables. You can try to specify DROP statements for the generated tables followed by your view create statement in this file. Note that this will be a bit tricky for the tables with foreign keys as django guarantees no order of execution of these files (so stuffing all statements in one .sql will be the easiest way I think, I've never tried this before).
You could use unmanaged models for your reverse-engineered models, and initial SQL scripts for creating your views.
EDIT:
A bit more detailed answer. When you use unmanaged models, syncdb will not create your database tables for you, so you have to take care of it yourself. An important point is the table name, and how django maps Model classes to table names, I suggest you read the doc on that point.
Basically, your Series model will look like that :
class Series(models.Model):
# model fields...
...
class Meta:
managed = False
db_table = "legacy_series"
Then, you can put your SQL commands, in the yourapp/sql/series.sql file :
### yourapp/sql/series.sql
CREATE VIEW legacy_series AS SELECT * FROM legacy.series;
You can then syncdb as usual, and start using your legacy models.
I have some models I'm working with in a new Django installation. Is it possible to change the fields without losing app data?
I tried changing the field and running python manage.py syncdb. There was no output from this command.
Renavigating to admin pages for editing the changed models caused TemplateSyntaxErrors as Django sought to display fields that didn't exist in the db.
I am using SQLite.
I am able to delete the db file, then re-run python manage.py syncdb, but that is kind of a pain. Is there a better way to do it?
Django does not ever alter an existing database column. Syncdb will create tables, but it does not do 'migrations' as found in Rails, for instance. If you need something like that, check out Django South.
See the docs for syndb:
Syncdb will not alter existing tables
syncdb will only create tables for models which have not yet been installed. It will never issue ALTER TABLE statements to match changes made to a model class after installation. Changes to model classes and database schemas often involve some form of ambiguity and, in those cases, Django would have to guess at the correct changes to make. There is a risk that critical data would be lost in the process.
If you have made changes to a model and wish to alter the database tables to match, use the sql command to display the new SQL structure and compare that to your existing table schema to work out the changes.
You have to change the column names in your DB manually through whatever administration tools sqlite provides. I've done this with MySQL, for instance, and since MySQL lets you change column names without affecting your data, it's no problem.
Of course there is.
Check out South
You'll have to manually update the database schema/layout, if you're only talking about adding/removing columns.
If you're attempting to rename a column, you'll have to find another way.
You can use the python manage.py sql [app name] (http://docs.djangoproject.com/en/dev/ref/django-admin/#sql-appname-appname) command to see what the new SQL should look like, to see what columns, of what type/specification Django would have you add, and then manually run corresponding ALTER TABLE commands.
There are some apps/projects that enable easier model/DB management, but Django doesn't support this out of the box, on purpose/by design.