We have two different groups working on the same Django database:
The web team
A team of data scientists
We'd like to make it easy for the data scientists to read/write to a subset of a tables using Django models but without giving them write/delete access to the rest of the tables.
Our current thought is that we'd like to lock down the tables in the data layer (Postgres) with GRANT and REVOKE style SQL statements, but we'd like to manage those permissions in the models.
We'd have two ROLES:
data_scientists_rw
web_team_admin
Instead of manually writing GRANT and REVOKE permissions in each migration, we'd like to have a decorator or a Meta class variable on a model so that when we makemigrations, it will automatically generate the correct SQL.
# data_scientists/models.py (pseudocode)
class DataScientistModel(models.Model):
...
my_special_number = models.FloatField()
...
class Meta:
data_science_team_editable = True
Make the migration:
$ ./manage.py makemigrations
Auto-generated model (pseudocode):
# migrations/0008_new_data_scientist_model.py
...
if data_science_team_editable:
RunSQL('GRANT INSERT, UPDATE, DELETE ON {tablename} TO data_scientists_rw;')
...
Questions:
Does this approach seem sensible?
How I do hook into makemigrations so that I can auto-generate the RunSQL code?
Related
I have a DB (lets call it data_db) containing some tables. I want to create a dashboard to present data from data_db, so I created a Django project for that purpose.
When I want to get data from one of the tables in data_db, is there a way to do it with Models? (I want Django security management with the DB) or do I have to use raw SQL?
One note: there is existing data in the data_db's table, and I don't want to create a new table with the same exact data on the default Django DB. I also use 2 DBs, Django default's and data_db and I created a database router for data_db to prevent Django from creating all its tables in there.
Thanks.
Yes. In fact Django can even help you create the models. Models that you do not migrate with the help of Django are unmanaged models. These have a managed = False attribute in the Meta class, so something like:
class MyModel(models.Model):
# … fields …
class Meta:
managed = False
If you thus write these unmanaged models, you can make queries with the Django ORM, without Django trying to create new models for these tables.
Of course, specifying models that match with the database is cumbersome. Therefore Django can often construct models based on the tables. You can generate the models with the inspectdb command [Django-doc].
You can generate these models on the stdout with:
python3 manage.py inspectdb
or you can save these to a file through I/O redirection:
python3 manage.py inspectdb > app_name/models.py
I saw lots of information about using multiple databases with one server but I wasn't able to find contents about sharing one database with multiple servers.
Using Micro Service Architectures, If I define a database and models in a django server, named Account, How can I use the database and models in Account server from another server named like Post??
What I'm thinking is to write same models.py in both servers and use the django commands --fake
Then, type these commands
python manage.py makemigrations
python manage.py migrate
and in another server
python manage.py makemigrations
python manage.py migrate --fake
I'm not sure if this would work and I wonder whether there is any good ways.
I doubt this is the best approach, but if you want two separate Django projects to use the same database you could probably create the first like normal then, in the second project, copy over all of the models.py and migration files. Django creates a database table behind the scenes to track which migrations have been applied, so as long as the apps, models, and migration files are identical in the second app it should work without having to fake any migrations.
That said, this sounds like a mess to maintain going forward. I think what I would do is create a single Django project that talks to the database, then create an API in that first project that all other apps can interface with to communicate with the database. That way you avoid duplicating code or having to worry about keeping multiple projects in sync.
When using additional Django servers with the same database that is already managed by the initial Django server, the tables don't need to be managed by the additional servers.
So you can add into the Meta for the models that managed = False and Django will not need to touch them, but can still use them. You will need to copy your models across to the additional servers, or use inspectdb (see below).
from django.db import models
class ExampleModel(models.Model):
id = models.IntegerField(db_column='db', primary_key=True)
created = models.DateTimeField(auto_now_add=True)
modified = models.DateTimeField(auto_now=True)
class Meta:
managed = False
db_table = 'example_table'
You will probably need to state the name of the table being referenced in the Meta as well, otherwise Django may generate a name that doesn't match the database.
You can even cut down the models when using them unmanaged.
It's not necessary to declare all the fields, just the ones you're using.
You can also use python manage.py inspectdb to automatically generate unmanaged models for all the tables in your database, saving time and ensuring the model fields conform to the actual database setup.
This is detailed in the documentation:
https://docs.djangoproject.com/en/4.0/ref/models/options/#managed
https://docs.djangoproject.com/en/4.0/ref/django-admin/#inspectdb
In my project, I have the same case that I have 2 Django servers and 1 database.
I did that I run on server 1
python manage.py makemigrations
and
python manage.py migrate
and on server 2 I just run:
python manage.py makemigrations
I did not run migrate commands on server 2
Now if there is any change on model then I run makemigrations command on both servers and migrate command on any of one server. I am using only one database
I'm currently working on a project with Django, I have designed a model like
Class Item(models.Model):
id
name
...
And I've already had a sqlite database with data like
Id, name, ...
1, a, ...
2, b, ...
Now, the question is how I can push this database to django? Thanks
You usually don't create any databases tables in the database per se, you just need to create the Django models and then run and apply the migrations. So, for instance, if you have the following model:
class Item(models.Model):
name = models.CharField(max_length=255)
You just need to go to the terminal and type
python manage.py makemigrations (This will create a migration file)
python manage.py migrate (This will create the actual tables/columns in your database)
If you then open a shell:
python manage.py shell
and create an object via Django:
from yourapp.models import Item
Item.objects.create(name='Hello')
That Item should be saved to your DB.
I have created a UserProfile field in order to add a favorites functionality to my site. Using Django's recommendation, I created a UserProfile model as follows at the bottom
Unfortunately, I already had the rest of my database created, and so I need to either use a migration utility or manually edit my database. However, I do not have sufficient permissions to utilize a migration utility, so I have to edit the database directly, and am struggling to do so.
This answer is similar to what I want to accomplish, but I can't quite get the syntax to work in my case.
MySQL - One To One Relation?
class UserProfile(models.Model):
user = models.ForeignKey(User, unique=True)
favorites = models.ManyToManyField(Media, related_name='favorited_by')
In my experience, the best migration utility is South. Once you've installed and added it to your settings, you'll need to create initial migrations for your existing modules using
./manage.py schemamigration --initial my_module,
which will include the one containing your UserProfile model, then from there you can migrate using
manage.py migrate my_module.
The real power in using a utility like this is portability and reversibility. You can migrate forward and backward as needed, and you'll be able to bring your schema to virtually any SQL database without all the fuss of rebuilding using SQL directly.
I would certainly agree with Steves recommendation to use South.
However if you for some reason wouldn't want to, you can issue the following command:
python manage.py sql <appname>
This will output the SQL statements which django will use to create your tables. This can then be used to manually modify the database.
How do I make syncdb execute SQL queries (for table creation) defined by me, rather then generating tables automatically.
I'm looking for this solution as some particular models in my app represent SQL-table-views for a legacy-database table.
So, I've created their SQL-views in my django-DB like this:
CREATE VIEW legacy_series AS SELECT * FROM legacy.series;
I have a reverse engineered model that represents the above view/legacytable. But whenever I run syncdb, I have to create all the views first by running sql scripts, otherwise syncdb simply creates tables for them (if a view is not found).
How do I make syncdb run the above mentioned SQL?
There are 2 possible approaches I know of to adapt your models to a legacy database table (without using views that is):
1) Run python manage.py inspectdb within your project. This will generate models for existing database tables, you can then continue to work with those.
2) Modify your tables with some specific settings. First of all you define the table name in your model by setting the db_table option in your meta options. Secondly you define for each field the column name to match your legacy database by setting the db_column option. Note there are other db_ options listed you possibly could use to match your legacy database.
If you really want the views approach an (ugly) workaround is possible, you can define custom sql commands per application model. This file is found in "application"/sql/"model".sql . Django will call this sql's after it created all tables. You can try to specify DROP statements for the generated tables followed by your view create statement in this file. Note that this will be a bit tricky for the tables with foreign keys as django guarantees no order of execution of these files (so stuffing all statements in one .sql will be the easiest way I think, I've never tried this before).
You could use unmanaged models for your reverse-engineered models, and initial SQL scripts for creating your views.
EDIT:
A bit more detailed answer. When you use unmanaged models, syncdb will not create your database tables for you, so you have to take care of it yourself. An important point is the table name, and how django maps Model classes to table names, I suggest you read the doc on that point.
Basically, your Series model will look like that :
class Series(models.Model):
# model fields...
...
class Meta:
managed = False
db_table = "legacy_series"
Then, you can put your SQL commands, in the yourapp/sql/series.sql file :
### yourapp/sql/series.sql
CREATE VIEW legacy_series AS SELECT * FROM legacy.series;
You can then syncdb as usual, and start using your legacy models.