It's more of a general question but I am trying to implement this using Python on top of Peewee as ORM. What is a decent OO way of abstracting the DB out of a python program?
In Peewee , classes are defined which inherit from peewee.Model and have Peewee fields as attributes.For example:
class Person(peewee.Model):
class Meta:
database = db
name = peewee.TextField()
height = peewee.DecimalField()
In an OO implementation we would like to have methods such as grow(size), die(),.. to be part of the objects. Is it best to build a class on top of these Peewee models to contain such functionality or should this be put in the model itself?
I can remember in a Java EE program that we used to have a DAO (Data access object) and DTO (Data transfer object). The peewee model object is a DAO or can it be both? Is there some sort of pattern that can be applied here?
Peewee is an ActiveRecord ORM, so there's no distinction between the data access and the object representation. This means when you execute queries the data is returned to you as model instances. Given that this is the case, it's common to put methods on the model itself, since you're using it anyways.
Whether you want to build a service layer on top of your models is entirely up to you. If you have mutually-dependent models this may make sense.
Related
We are developing a collection management project using Django, usable for different types of collections.
This problem quite naturally divides itself in two:
The common part, that will be shared by all collections.
The specializations, different for each collection type.
Example
To illustrate this a bit further, let's take a simplified example in pseudocode.
Common part
class ItemBase: # ideally abstract
name = CharField()
class Rental
item = ForeignKey("Item")
rented_to_person = CharField()
Specialization for a collection of cars
class ItemSpecialization
horse_power = Int()
The problem
The question is how we could organize the code in order to allow reuse of the common part without duplicating its content ?
We would imagine it would be best to have the common part as a non-installed application, and have each specialized configuration as a separate installed application. But this would cause a problem with the Rental concrete class, because it resides in the common-part application.
Any advices on how we could proceed ?
It really depends on what you want, you may use an abstract model class for common stuff, and inherit from that in specialized model classes.
Otherwise, if you really want one table for all common data, typically to be able to relate to it, then you'll need your specialized model to have a relation to the common model. It can be a foreign key, or you can use model inheritance, in which case the foreign key in question will be managed for you by django, but it'll be harder to use.
It sounds like you're looking for a OneToOneField field relationship. Based on your example:
class ItemBase:
name = models.CharField(max_length=50)
class Rental:
item = models.OneToOneField(ItemBase)
rented_to_person = models.CharField(max_length=50)
class ItemSpecialization
item = models.OneToOneField(ItemBase)
horse_power = models.IntegerField()
With this model hierarchy, you could fetch Rental or ItemSpecialzation objects and also gain access to ItemBase fields. It's basically OO inheritance for Django models. More details in the docs: https://docs.djangoproject.com/en/1.9/topics/db/examples/one_to_one/
I am currently using Django framework including its Models mechanism to abstract the database schema declaration and general db access, which is working fine for most scenarios.
However, my application also requires tables to be created and accessed dynamically during runtime, which as far as I can see, is not supported by Django out of the box.
These tables usually have an identical structure, and can basically be abstracted by the same Model class, but Django doesn't let you change the underlying db_table of a certain model query, as it is declared on the Model class and not on the Manager.
My solution for this is to do this process whenever I need a new table to be created, populated and accessed:
Create and populate the table using raw sql
Add indexes to the table using raw sql
When I need to access the table (using django queryset api), I declare a new type dynamically and return it as the model for the query, by using this code:
table_name = # name of the table created by sql
model_name = '%d_%s' % (connection.tenant.id, table_name)
try:
model = apps.get_registered_model('myapp', model_name)
return model
except LookupError:
pass
logger.debug("no model exists for model %s, creating one" % model_name)
class Meta:
db_table = table_name
managed = False
attrs = {
'field1' : models.CharField(max_length=200),
'field2' : models.CharField(max_length=200),
'field3' : models.CharField(max_length=200)
'__module__': 'myapp.models',
'Meta':Meta
}
model = type(str(model_name), (models.Model,), attrs)
return model
Note that I do check if the model is already registered in django and I'm using an existing model in case it does. The model name is always unique for each table. Since I'm using multi tenants, the tenant name is also part of the model name to avoid conflict with similar tables declared on different schemas.
In case it's not clear: the tables created dynamically will and should be persisted permanently for future sessions.
This solution works fine for me so far.
However, the application will need to support a large number of these tables. i.e. 10,000 - 100,000 such tables(and corresponding model classes), with up to a million rows per table.
Assuming the underlying db is fine with this load, my questions are:
Do you see any problem with this solution, with and without regards to the expected scale ?
Anybody has a better solution for this scenario ?
Thanks.
There is a wiki page on creating models dynamically, although it has been a while since it was last updated:
DynamicModels Django
There are also a few apps that are designed for this use case, but I don't think any of them is being actively maintained:
Django Packages: Dynamic models
I understand that if you are already committed to Django this isn't very helpful, but this a use case for which Django isn't really good. It might be more costly to fight against the abstractions provided by Django's model layer, than to just use psycopg2 or whatever other adapter is appropriate for your data.
Depending on what sort of operations you are going to perform on your data, it may be also more reasonable to use a single model with an indexed field that allows you to distinguish in which table that row would be and then sharding the data by that column.
If you still need to do this, the general idea would be:
Create a metaclass that extends Django's ModelBase. This metaclass you would use as a factory for your actual models.
Consider stuff mentioned on that wiki page, like circumventing the app_label issue.
Generate and execute the sql for the creation of the model as also shown on the wiki page.
In my GAE project, I have a base class called Part.
From this class I derive other classes such as Motor and Battery.
If I run the following:
motors = Motor.query().fetch()
batterys = Battery.query().fetch()
I will get all the parts, but I am looking for something more elegant.
If I run:
parts = Part.query().fetch()
I get an empty list [ ].
How can I run the above query and get all results in one list?
Thank you
You can do this, but all your Parts classes must inherit from PolyModel
https://developers.google.com/appengine/docs/python/ndb/polymodelclass
So
class Part(ndb.PolyModel):
#stuff
class Motor(Part):
# stuff
class Wheel(Part):
# stuff
parts = Part.query().fetch()
However all items are stored in the datastore as Part, they have an additional classes attribute which names each of the classes in it's inheritance heirarchy. Then when the entity is retrieved the correct sub class is instantiated.
Another potential downside, the property names in the stored model are a union of all subclasses, if you have default values for any of the properties. I haven't checked to see how far this goes. So if you have lots of very different properties in all the subclasses you could be storing a lot of empty properties, and incur the cost of storing all the property names in each entity.
The datastore has no concept of inheritance, and doesn't know that your entity types derive from Part.
There isn't really any way of doing this sort of thing with GAE: ancestor keys are not really the answer, as they would have all Motor/Battery entities descending from a single Part, which would severely limit update rates.
The best way to model this kind of relationship would really be to drop the separate models and have a single Part model, with a part_type field that can be "motor" or "battery".
I am beginning to learn Python and Django. I want to know how if I have a simple class of "player" with some properties, like: name, points, inventory, how would I make the class also write the values to the database if they are changed. My thinking is that I create Django data models and then call the .save method within my classes. Is this correct?
You are correct that you call the save() method to save models to your db, But you don't have to define the save method within your model classes if you don't want to. It would be extremely helpful to go through the django tutorial which explains all.
https://docs.djangoproject.com/en/dev/intro/tutorial01/
https://docs.djangoproject.com/en/dev/topics/db/models/
Explains django models
django uses its own ORM (object-relational mapping)
This does exacxtly what it sounds like maps your django/python objects (models) to your backend.
It provides a sleek, intuitive, pythonic, very easy to use interface for creating models (tables in your rdbms) adding data and retrieving data.
First you would define your model
class Player(models.Model):
points = models.IntegerField()
name = models.CharField(max_length=255)
django provides commands for chanign this python object into a table.
python manage.py syncdb
you could also use python manage.py sql <appname> to show the actual sql that django is generating to turn this object into a table.
Once you have a storage for this object you can create new ones in the same manner you would create python objects
new_player = Player(points=100, name='me')
new_player.save()
Calling save() actually writes the object to your backend.
You're spot on...
Start at https://docs.djangoproject.com/en/dev/intro/tutorial01/
Make sure you have the python bindings for MySQL and work your way through it... Then if you have specific problems, ask again...
I tried Django-nonrel(which is a NoSQL fork of Django project) with MongoDB. In it's ORM, class can only inherit from abstract class, so even two class are inherit from the same base class, they are saved into different collections in MongoDB.
This design added a pre-defined schema to the schema-less database, every collection has its unique schema now.
What I want is, if ClassA and ClassB are inherited from BaseClass, every instance of ClassA and ClassB should be put in the same collection ( or something similar in NoSQL other than MongoDB )
The goal of Django-nonrel is to maintain the same behaviors as Django's ORM has, but to allow use of non-relational storage backends. Since Django doesn't permit multiple different models to be stored in the same (relational) table, I'm not surprised to learn that Django-nonrel doesn't either.
If you want the behavior you have described, you might check out Mongoengine, an ODM (object-document mapper) for MongoDB, which is inspired by and has a similar feel to the Django ORM, but which allows more flexible schemas and behaviors, like the one you've described.
In particular, when using Mongoengine, you can create multiple classes whose records are stored in the same collection like:
class BlogPost(mongoengine.Document):
create_date = mongoengine.DateTimeField()
mod_date = mongoengine.DateTimeField()
title = mongoengine.StringField()
meta = {
'abstract': True,
}
def save(self):
if not self.pk:
self.create_date = datetime.utcnow()
self.mod_date = datetime.utcnow()
super(BlogPost, self).save()
class TextBlogPost(BlogPost):
body = mongoengine.StringField()
class ImageBlogPost(BlogPost):
image = mongoengine.FileField()
caption = mongoengine.StringField()
Since TextBlogPost and ImageBlogPost both inherit from BlogPost, they will both be saved in the same collection (by default, named "blog_post").