We Keep Coding

Django set privacy options per model field

I have gone through the question, best way to implement privacy on each field in model django and Its answers doesn't seem solve my problem so I am asking some what related question here,
well, I have a User model. I want the user to make possible to control the privacy of each and every field of their profile (may be gender, education, interests etc . ..).
The privacy options must not to be limited to just private or public, but as descriptive as
public
friends
only me
friend List 1 (User.friendlist.one)
friend List 2 (User.friendlist.two)
friend List 3 (User.friendlist.three)
another infinte lists that user may create.
I also don't want these privacy options to be saved on another model, but the same so that with one query I could get the user object along with the privacy options.
so If I have the UserModel,
class User(models.Model):
name = models.CharField()
email = models.EmailField()
phone = models.CharField()
How do I setup a privacy setting here? I am using postgres, can I map a JSON field or Hstore even an ArrayField?
what is the best solution that people used to do with Django with same problem?
update:
I have n model fields. What I really want is to store the privacy settings of each instance on itself or some other convenient way.

I have worked on my issue, tried solutions with permissions and other relations. I have a Relationship Model and all other relationship lists are derived from the Relationship model, so I don't want to maintain a separate list of Relationships.
So my pick was to go with a Postgres JSONField or HStoreField. Since Django has good support for postgres freatures, I found these points pro for the choice I made.
JSON/HashStore can be queried with Django ORM.
The configurations are plain JSON/HashStore which are easy to edit and maintain than permissions and relations.
I found database query time taken are larger with permissions than with JSON/HStore. (hits are higher with permissions)
Adding and validating permissions per field are complex than adding/validating JSON.
At some point in future if comes a more simple or hassle free solution, I can migrate to it having whole configuration at a single field.
So My choice was to go with a configuration model.
class UserConfiguration(models.Model):
user = # link to the user model
configuration = #either an HStore of JSONFeild
Then wrote a validator to make sure configuration data model is not messed up while saving and updating. I grouped up the fields to minimize the validation fields. Then wrote a simple parser that takes the users and finds the relationship between them, then maps with the configuration to return the allowed field data (logged at 2-4ms in an unoptimized implementation, which is enough for now). (With permission's I would need a separate list of friends to be maintained and should update all the group permissions on updation of privacy configuration, then I still have to validate the permissions and process it, which may take lesser time than this, but for the cost of complex system).
I think this method is scalable as well, as most of the processing is done in Python and database calls are cut down to the least as possible.
Update
I have skinned down database queries further. In the previous implementation the relations between users where iterated, which timed around 1-2ms, changing this implementation to .value_list('relations', flat=True) cut down the query time to 400-520µs.

I also don't want these privacy options to be saved on another model, but the same so that with one query I could get the user object along with the privacy options.
I would advice you to decouple the privacy objects from the UserModel, to not mess your users data together with those options. To minimize the amount of database queries, use djangos select_related and prefetch_related.
The requirements you have defined IMO lead to a set of privacy related objects, which are bound to the UserModel. django.contrib.auth is a good point to start with in this case. It is build to be extendable. Read the docs on that topic.
If you expect a large amount of users and therefore also an even larger amount of groups you might want to consider writing the permissions resolved for one user in a redis based session to be able to fetch them quickly on each page load.
UPDATE:
I thought a little more about your requirements and came to the conclusion that you need per object permission as implemented in django-guardian. You should start reading their samples and code first. They build that on top of django.contrib.auth but without depending on it, which makes it also usable with custom implementations that follow the interfaces in django.contrib.auth.

What about something like this?
class EditorList(models.Model):
name = models.CharField(...)
user = models.ForeignKey(User)
editor = models.ManyToManyField(User)
class UserPermission(models.Model):
user = models.ForeignKey(User)
name = models.BooleanField(default=False)
email = models.BooleanField(default=False)
phone = models.BooleanField(default=False)
...
editor = models.ManyToManyField(User)
editor_list = models.ManyToManyField(EditorList)
If a user wants to give 'email' permissions to public, then she creates a UserPermission with editor=None and editor_list=None and email=True.
If she wants to allow user 'rivadiz' to edit her email, then she creates a UserPermission with editor='rivadiz' and email=True.
If she wants to create a list of friends that can edit her phone, then she creates and populates an EditorList called 'my_friends', then creates a UserPermission with editor_list='my_friends' and phone=True
You should then be able to query all the users that have permission to edit any field on any user.
You could define some properties in the User model for easily checking which fields are editable, given a User and an editor.
You would first need to get all the EditorLists an editor belonged to, then do something like
perms = UserPermissions.objects.filter(user=self).filter(Q(editor=editor) | Q(editor_list=editor_list))

First of all, in my opinion you should go for multiple models and for making the queries faster, as already mentioned in other answers, you can use caching or select_related or prefetch_related as per your usecase.
So here is my proposed solution:
User model
class User(models.Model):
name = models.CharField()
email = models.EmailField()
phone = models.CharField()
...
public_allowed_read_fields = ArrayField(models.IntegerField())
friends_allowed_read_fields = ArrayField(models.IntegerField())
me_allowed_read_fields = ArrayField(models.IntegerField())
friends = models.ManyToManyField(User)
part_of = models.ManyToManyField(Group, through=GroupPrivacy)
Group(friends list) model
class Group(models.Model):
name = models.CharField()
Through model
class GroupPrivacy(models.Model):
user = models.ForeignKey(User)
group = models.ForeignKey(Group)
allowed_read_fields = ArrayField(models.IntegerField())
User Model fields mapping to integers
USER_FIELDS_MAPPING = (
(1, User._meta.get_field('name')),
(2, User._meta.get_field('email')),
(3, User._meta.get_field('phone')),
...
)
HOW DOES THIS HELPS??
for each of public, friends and me, you can have a field in the User model itself as already mentioned above i.e. public_allowed_read_fields, friends_allowed_read_fields and me_allowed_read_fields respectively. Each of this field will contain a list of integers mapped to the ones inside USER_FIELDS_MAPPING(explained in detail below)
for friend_list_1, you will have group named friend_list_1. Now the point is the user wants to show or hide a specific set of fields to this friends list. That's where the through model, GroupPrivacy comes into the play. Using this through model you define a M2M relation between a user and a group with some additional properties which are unique to this relation. In this GroupPrivacy model you can see allowed_read_fields field, it is used to store an array of integers corresponding to the ones in the USER_FIELDS_MAPPING. So lets say, for group friend_list_1 and user A, the allowed_read_fields = [1,2]. Now, if you map this to USER_FIELDS_MAPPING, you will know that user A wants to show only name and email to the friends in this list. Similarly different users in friend_list_1 group will have different values in allowed_read_fields for their corresponding GroupPrivacy model instance.
This will be similar for multiple groups.

This will be much more cumbersome without a separate permissions model. The fact that you can associate a given field of an individual user's profile with more than one friend list implies a Many to Many table, and you're better off just letting Django handle that for you.
I'm thinking something more like:
class Visibility(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
field = models.CharField(max_length=32)
public = models.BooleanField(default=False)
friends = models.BooleanField(default=False)
lists = models.ManyToManyField(FriendList)
#staticmethod
def visible_profile(request_user, profile_user):
"""Get a dictionary of profile_user's profile, as
should be visible to request_user..."""
(I'll leave the details of such a method as an exercise, but it's not
too complex.)
I'll caution that the UI involved for a user to set those permissions is likely to be a challenge because of the many-to-many connection to friend lists. Not impossible, definitely, but a little tedious.
A key advantage of the M2M table here is that it'll be self-maintaining if the user or any friend list is removed -- with one exception. The idea in this scheme is that without any Visibility records, all data is private (to allow everyone to see your name, you'd add a Visibility record with user=(yourself), field="name", and public=True. Since a Visibility record where public=False, friends=False, and lists=[] is pointless, I'd check for that situation after the user edits it and remove that record entirely.
Another valid strategy is to have two special FriendList records: one for "public", and one for "all friends". This simplifies the Visibility model quite a bit at the expense of a little more code elsewhere.

Where do I put a function to count foreign keys in Django?

I have a Django model object, Record, which has foreign keys to two other models RecordType and Source:
class Record(models.Model):
title = models.CharField(max_length=200)
record_type = models.ForeignKey(RecordType)
source = models.ForeignKey(Source)
Question: If I want to count the number of Record objects which refer to RecordType with id "x" and Source with id "y", where is the appropriate area of code to put that function?
Right now I have it in views.py and I feel that is a violation of best practices for "fat model, thin views", so I want to move it from views.py. But I'm not entirely sure if this is a row-based or table-based type of operation, so I'm not sure if it should be implemented as a model method, or instead as a manager.
Here's the current (working) logic in views.py:
record_count = Record.objects.filter(record_type__id=record_type_.id, source__id=source_.id).count()
Just to be clear, this isn't a question of how to get the count, but simply in which area of code to put the function.
Here's a similar question, but which was addressing "how to" not "where":
Counting and summing values of records, filtered by a dictionary of foreign keys in Django

If the result involves multiple rows, it is a table-related method, and according to Django conventions, should be a manager method.
From the Django docs:
Adding extra Manager methods is the preferred way to add “table-level” functionality to your models. (For “row-level” functionality – i.e., functions that act on a single instance of a model object – use Model methods, not custom Manager methods.)

App Engine: Structured Property vs Reference Property for one-to-many relationship

My background with designing data stores comes from Core Data on iOS, which supports properties having a one-to-many relationship with another entity.
I'm working on an App Engine project which currently has three entity types:
User, which represents a person using the app.
Project, which represents a project. A User may be associated with many projects.
Post, which is the main content behind a Project. A Project may have many posts.
Currently, User has a property, projects, that is a one-to-many relationship to Project entities. Project has a property, posts, that is a one-to-many relationship to Post entities.
In this case, is Datastore's Reference Property or NDB's Structured Property better for the job (and how are the two conceptually different)? Is there a better way to structure my data?

By reference property you probably mean Key Property. This is a reference to another datastore entity. It is present in both db and ndb APIs. Using these, you can model a many to one relationship by pointing many entities to the key of another entity.
Structured property is a completely different beast. It allows you to define a data structure, and then include it within another entity.
Here's an example from the docs where you include multiple addresses for a single contact:
class Address(ndb.Model):
type = ndb.StringProperty() # E.g., 'home', 'work'
street = ndb.StringProperty()
city = ndb.StringProperty()
class Contact(ndb.Model):
name = ndb.StringProperty()
addresses = ndb.StructuredProperty(Address, repeated=True)
guido = Contact(name='Guido',
addresses=[Address(type='home',
city='Amsterdam'),
Address(type='work',
street='Spear St',
city='SF')])
guido.put()
For your specific application I'd recommend using NDB (it's always best to use the latest version of the api available), with the following:
Post model included under Project model as a repeated structured property.
Users include a repeated KeyProperty that contains the keys of the Projects they have permissions to.
To make it a bit more complex, you can create a another model to represent projects and permissions/roles, and then include that as a repeated structured property within the user model.
The main reason you want to hang on to the keys, is to keep the data accessible in light of HRDs eventual consistency.
Let me know if you need any more help on this.
EDIT:
To clarify, here's the proposed structure:
Models:
User
User-Project-Mapping (optional, needed to handle permissions)
Project
Post
User model should contain User-Project-Mapping as repeated structured property.
Project model should contain Post as repeated structured property.
User-Project-Mapping only needs to contain Key reference to the Project and relevant permissions representation.
Since this sounds like a commercial project, if you'd like further help with this, I'll gladly consult for you. Hope you have enough to succeed!

There is another point that was not mentioned and might be relevant: entities inserted in a StructuredProperty "are not full-fledged entities", as mentioned in this part of the docs. Below is the complete quote (it refers to the same example mentioned in the answer by #Sologoub):
Although the Address instances are defined using the same
syntax as for model classes, they are not full-fledged entities. They
don't have their own keys in the Datastore. They cannot be retrieved
independently of the Contact entity to which they belong.
This may cast some limitations in the design given that you cannot reuse an entity's property without duplicating data. The KeyProperty, on the other side, refers to another entity's key and therefore represents entities relationship in a more "relational" way. And KeyProperties can also be repeated: just include the repeated=True parameter.

django: creating a model for Person who can be Writer or Actor or both

i am new to both python and Django, trying to create a database that hold general information about people, so i have this:
a model for a Person (contain general info).
a model for a Category that a person belongs to (a person can be in multiple categories at the same time).
each category contain its own extra data (a person who is in "Writers" category might have some books that we want to store in DB)
i thought about model sub-classing, but it seems that this will not work if a person can be in multiple categories the same time (especially dynamically)
another thought is creating profiles that have OneToOne relationship with the Person model, but i am not sure if it is the best way
what is the best/other ways to tackle this?

It sounds like you want each category to have a ForeignKey to Person.

add ManyToManyField(Category) to Person class.
Django permissions in User model are solved the same way (via Group class)
edit:
you're right, I'm sorry about my useless answer.
My solution would looks like this:
Class Person(Model):
#property
def extras(self):
extra_data = {}
for category in self.categories.all():
category_model_class = CATEGORIES_DATA_MAP[category.name]
extra_data[category.name] = category_model_class.objects.filter(user=self.pk)
return extra_data
... where CATEGORIES_DATA_MAP is dictionary with category/model relation map

The multiple 1to1 rel is the way to go.
Define an ActorProfile model, a WriterProfile model etc, with each having a fk to User. Use some orm magic to load then when necessary.
It's basic that the FK is in the profile model, otherwise you'll need to add a new column on the user table each time you need a new kind of profile.
As the profiles will be probably somewhat overlapping, I'd suggest to use composition (instead of inheritance), do more profile types which store the common data, and keep the specific profiles for very specific data.
EDIT
Use a document-based database (mongodb?) and forget about migrations, fixed schemata, artificial joins... You have to think only about structuring the data the way you really need, but the advantages are worth considering.

User-specific model in Django

I have a model containing items, which has many different fields. There is another model which assigns a set of this field to each user using a m2m-relation.
I want to achieve, that in the end, every user has access to a defined set of fields of the item model, and he only sees these field in views, he can only edit these field etc.
Is there any generic way to set this up?

One way to do this would be to break the Item model up into the parts that are individually assignable to a user. If you have fixed user types (admin, customer, team etc.) who can always see the same set of fields, these parts would be whole groups of fields. If it's very dynamic and you want to be able to set up individual fields for each user, each field is a part of its own.
That way, you would have a meta-Item which consists solely of an Id that the parts can refer to. This holds together the parts. Then, you would map a user not to the Item but to the parts and reconstruct the item view from the common Id of the parts.

A second approach would be to not include the filtering in the model layer. I. e., you leave the mapping on the model layer as it is and retrieve the full set of item fields for each user. Then you pass the items through a filter that implements the rules.
Which approach is better for you depends on how you want to filter. If it's fixed types of users, I would probably implement a rules-based post-processor, if it's very dynamic, I would suggest the approach from my earlier answer. Another reason to put the filtering rules in the model would be if you want to reuse the model in applications that couldn't reuse your filter engine (for example if you have applications in different languages sharing the same database).