I was going through code of https://github.com/hit9/CURD.py/blob/master/CURD.py which is a simple orm that performs normal curd operations .. and i could not understood part of code which goes like this(on line number 616):
.....#smthing #..
for name, attr in cls.__dict__.iteritems():
if isinstance(attr, Field):
attr.describe(name, cls)
fields[name] = attr
what does attr.describe(attr, Field) do ? I googled it out but found nothing.
It's not a Python language feature, it's a method on that library. You can see the definition here:
https://github.com/hit9/CURD.py/blob/master/CURD.py#L251
class Field(Leaf):
"""
Field object.
Field examples: User.name, User.age ..
"""
def __init__(self, is_primarykey=False, is_foreignkey=False):
self.is_primarykey = is_primarykey
self.is_foreignkey = is_foreignkey
# describe model's attr
def describe(self, name, model):
self.name = name
self.model = model
# fullname e.g. : User.id 's fullname is "user.id"
self.fullname = self.model.table_name + "." + self.name
# describe the attribute, reload its access control of writing, reading
setattr(model, name, FieldDescriptor(self))
setattr sets an attribute on an object. So if I call describe("field_name", myObject), it will set myObject.field_name to the description of the model. Or something like that.
That is not a python standard thing.
The loop iterates through the names and values of a class, and the attributes of that class that are instances of a Field type are added to a dictionary.
Field is not part of python standard library, you should search that project for the Field class.
Related
I've created decorators that wrap functions before, but in this instance, I don't need to wrap, so I'm guessing I'm using the wrong paradigm, so maybe somebody can help me figure this out and solve my ultimate goal.
What I imagined was a decorator that, when called (during compile time), it takes 3 arguments:
The decorated function (that resides inside a Model class)
The name of a data member of the class (i.e. a database field, e.g. a name field of type CharField)
The name of a parent key data member in the class (e.g. parent of type ForeignKey)
My decorator code would register the function, the field, and related key associated with it in a global list variable.
I would then have a class that inherits from Model that over-rides save() and delete(). If would cycle through the global list to update the associated fields using the output of the function and then call the parent model's .save() method so that it would update its decorated fields as well.
I quickly realized though that the decorator isn't passing the function that has the decorator, because I get an exception I created for when there isn't a field or a parent key supplied to the decorator during compile time.
In case this isn't clear, here's the code I have:
updater_list: Dict[str, List] = {}
def field_updater_function(fn, update_field_name=None, parent_field_name=None):
"""
This is a decorator for functions in a Model class that are identified to be used to update a supplied field and
fields of any linked parent record (if the record is changed). The function should return a value compatible with
the field type supplied. These decorators are identified by the MaintainedModel class, whose save and delete
methods override the parent model and call the given functions to update the supplied field. It also calls linked
dependent models (if supplied) update methods.
"""
if update_field_name is None and parent_field_name is None:
raise Exception(
"Either an update_field_name or parent_field_name argument is required."
)
# Get the name of the class the function belongs to
class_name = fn.__qualname__.split(".")[0]
func_dict = {
"function": fn.__name__,
"update_field": update_field_name,
"parent_field": parent_field_name,
}
if class_name in updater_list:
updater_list[class_name].append(func_dict)
else:
updater_list[class_name] = [func_dict]
if settings.DEBUG:
print(f"Added field_updater_function decorator to function {fn.__qualname__}")
return fn
class MaintainedModel(Model):
"""
This class maintains database field values for a django.models.Model class whose values can be derived using a
function. If a record changes, the decorated function is used to update the field value. It can also propagate
changes of records in linked models. Every function in the derived class decorated with the
`#field_updater_function` decorator (defined above, outside this class) will be called and the associated field
will be updated. Only methods that take no arguments are supported. This class overrides the class's save and
delete methods as triggers for the updates.
"""
def save(self, *args, **kwargs):
# Set the changed value triggering this update
super().save(*args, **kwargs)
# Update the fields that change due to the above change (if any)
self.update_decorated_fields()
# Now save the updated values (i.e. save again)
super().save(*args, **kwargs)
# Percolate changes up to the parents (if any)
self.call_parent_updaters()
def delete(self, *args, **kwargs):
# Delete the record triggering this update
super().delete(*args, **kwargs) # Call the "real" delete() method.
# Percolate changes up to the parents (if any)
self.call_parent_updaters()
def update_decorated_fields(self):
"""
Updates every field identified in each field_updater_function decorator that generates its value
"""
for updater_dict in self.get_my_updaters():
update_fun = getattr(self, updater_dict["function"])
update_fld = updater_dict["update_field"]
if update_fld is not None:
setattr(self, update_fld, update_fun())
def call_parent_updaters(self):
parents = []
for updater_dict in self.get_my_updaters():
parent_fld = getattr(self, updater_dict["parent_field"])
if parent_fld is not None and parent_fld not in parents:
parents.append(parent_fld)
for parent_fld in parents:
parent_instance = getattr(self, parent_fld)
if isinstance(parent_instance, MaintainedModel):
parent_instance.save()
elif isinstance(parent_instance, ManyToManyField) and :
parent_instance.all().save()
else:
raise Exception(
f"Parent class {parent_instance.__class__.__name__} or {self.__class__.__name__} must inherit "
f"from {MaintainedModel.__name__}."
)
#classmethod
def get_my_updaters(cls):
"""
Convenience method to retrieve all the updater functions of the calling model.
"""
if cls.__name__ in updater_list:
return updater_list[cls.__name__]
else:
return []
class Meta:
abstract = True
And here's the first decorator I applied that triggers the exception at compiletime:
class Tracer(models.Model, TracerLabeledClass):
id = models.AutoField(primary_key=True)
name = models.CharField(
max_length=256,
unique=True,
help_text="A unique name or lab identifier of the tracer, e.g. 'lysine-C14'.",
)
compound = models.ForeignKey(
to="DataRepo.Compound",
on_delete=models.RESTRICT,
null=False,
related_name="tracer",
)
class Meta:
verbose_name = "tracer"
verbose_name_plural = "tracers"
ordering = ["name"]
def __str__(self):
return str(self._name())
#field_updater_function("name", "infusates")
def _name(self):
# format: `compound - [ labelname,labelname,... ]` (but no spaces)
if self.labels is None or self.labels.count() == 0:
return self.compound.name
return (
self.compound.name
+ "-["
+ ",".join(list(map(lambda l: str(l), self.labels.all())))
+ "]"
)
And my exception:
...
File ".../tracer.py", line 31, in Tracer
#field_updater_function("name")
File ".../maintained_model.py", line 19, in field_updater_function
raise Exception(
Exception: Either an update_field_name or parent_field_name argument is required.
The basic idea is we have a bunch of fields in the database that can be derived fully from other fields in the database. We'd started out originally with cached_properties, but they provided virtually no speedup, so we'd rather just save the computed values in the database.
I'd written a caching mechanism which auto-refreshes the cache using an override of .save and .delete, and that works great, but has various drawbacks.
We could custom code an override of .save() that explicitly calls the function to update every field, but I wanted something that made the overhead of maintaining field values as simple as applying decorators to the functions that perform the updates, and just supply the fields they compute the values for and the links to other affected fields up the hierarchy. Such as:
#field_updater_function("name", "infusates")
def _name(self):
...
Is there something other than decorators I should be using to accomplish this? I could just make a dummy decorator using functools.wraps that just returns the supplied function as is (I think), but that just feels wrong.
You need to make a decorator factory. That is, a function you call with arguments that returns a decorator function that gets passed the function to be decorated.
A typical way to do that is with nested functions. A function defined within another function can access the variables in the enclosing function's namespace. Here's what I think it would look like for your code:
def field_updater_function(update_field_name=None, parent_field_name=None): # factory
# docstring omitted for brevity
if update_field_name is None and parent_field_name is None:
raise Exception(
"Either an update_field_name or parent_field_name argument is required."
)
def decorator(fn): # the actual decorator
class_name = fn.__qualname__.split(".")[0]
func_dict = {
"function": fn.__name__,
"update_field": update_field_name, # you can still access variables
"parent_field": parent_field_name, # from the enclosing namespace
}
if class_name in updater_list:
updater_list[class_name].append(func_dict)
else:
updater_list[class_name] = [func_dict]
if settings.DEBUG:
print(f"Added field_updater_function decorator to function {fn.__qualname__}")
return fn
return decorator # here the factory returns the decorator
A little background: You'll notice my comments describe what I'll go through later. Let's say I have the following object...
#!/usr/bin/python
import os
import sys
class ContainerField(object):
''' An attribute/object storage device '''
def __init__(self, field=None, value=None):
self.m_field = field
self.m_value = value
def __getattr__(self, key):
'''
What can we do here that runs the .get() command but -only- if the key
does not exist.
'''
# super(ContainerField, self).__getattr__(key)
def __call__(self):
return self.get()
def value(self):
return self.m_value
def setValue(self, value):
self.m_value = value
def _recurseSetAttr(self, attr, values):
'''Generate our attributes/objects and store them succinctly.'''
# Container
# \_Container
# \_Container
# \_Container...
for field, value in values.items():
if not hasattr(attr, field):
setattr(attr,
field,
# field type is known from model caching
ContainerField(value=value, field=field_type(field)))
fdbf = getattr(attr, field)
if isinstance(value, dict):
self._recurseSetAttr(fdbf, value)
else:
fdbf.setValue(value)
def get(self):
# Create the new object from scratch and proliferate it's
# attributes recursively. 'values' come in the form of a
# dictionary that we can then use to setattr().
# So... Create container, set value, find keys for this
# and create containers that hold the values of those keys
# and repeate...
self._recurseSetAttr(self, attr, values)
Now, when generating the objects I can have a dict that looks something like this: {"myContainer" : { "id" : 2, "foo" : { "id" : 3, "bar" : 1 } }} that, once created, can be called like this: myContainer.foo.id.value()
In the scenario there's the self.m_field which tells the application what data type the object really is. This is referencing off of Django models but any python could apply.
All containers will have an id (or pk) key to them as part of their instantiation. This is mandatory.
The Rub
Ideally, we fill our the top level attributes and only when the user requests for the attributes that lie underneath it do we construct them based off the id value and the field type.
So finally, let's say the myContainer.foo.bar attribute has a foreign key field type. If we call myContainer.foo.bar.newkey.value() the app should understand that the 'newkey' attribute does not exist, query against our django instance, store the bar attribute as the now more filled out Container, and return the newkey value that's been put to memory.
The Python Pitfall
I'd hoped it would be a simple hasattr() but Python seems to just use getattr() with a default None (The recursion is real!). I've also had loads of trouble getting a try: except: to work.
As I write this I'm realizing how much more complicated it may be due to the recursive attribute setting relying on getattr() and hasattr() Any suggestions would be greatly appreciated. - Cheers
So to answer the first part of the question: how to have __getattr__ call self.get() only when the attribute is not defined already. There are two attribute access methods in python classes: __getattribute__ and __getattr__. The first is called every time an attribute lookup is attempted, the second is called only when the normal attribute lookup system fails (including lookups in superclasses). Since you're defining __getattr__, which is only called when the attribute doesn't already exist, you can simply proxy it to a call to .get. Where you run into recursion issues is if you try to look up another attribute of self, that also doesn't yet exist, inside of __getattr__. The way to avoid this is to have a list of keys that require special handling and check if the current attribute requested is one of them. This typically is only needed when implementing __getattribute__.
Note that your .get method has a problem: attr and values are undefined. I'd give a slightly more concrete answer for what to put in __getattr__ if I knew what values for .get's attr and values want.
You could consider using the #property decorator with private internal fields. The idea would be something like:
class ContainerField(object):
def __init__(self, field=None, value=None):
self._m_field = field
self._m_value = value
#property
def m_field(self):
if self._m_field is None:
self._m_field = self.function_to_populate_m_field()
return self._m_field
#property
def m_value(self):
if self._m_value is None:
self._m_value = self.function_to_populate_m_value()
return self._m_value
...
Check this out:
class Test(object):
def __init__(self):
self.a = 5
self.b = 6
def __getattr__(self, key):
return 'created a new key: {}'.format(key)
obj = Test()
print(obj.a, obj.b)
print(obj.c)
Here, instead of returning 'created a new key...', you create a new attribute and return it.
I am working in Python 2.7 and trying to create a model structure which is something similar to django models but without database. Consider the following situation:
I want to create a model for Products (e.g CarryBag) which will have some predefined attributes of basic type (string, int, boolean etc), using which instances can be created. This model must inherit from an abstract model say 'GenericProduct' which will have certain defined functionality related to a product:
class GenericProduct:
name = StringType(required=True)
id = StringType(required=True)
def to_json(self):
....
def to_xml(self):
...
def some_other_function(self):
...
class CarryBag(GenericProduct):
material = StringType(required=True, default="Polyester", help_text="Type of material used")
price = IntType(required=True)
available = BooleanType(default=False)
So, challenges are:
This modelling structure must work very much similar to Django models but without database interaction.
How do I define abstract class 'GenericProduct'
How do I create a class that defines datatype classes 'IntType', 'StringType' & 'BooleanType' which can be used to define attributes.
What is the best way to achieve this keeping in mind python constructs. I am trying to go through django db model code to get some sense about it. Following is my code for one of the datatype classes:
class StringType(object):
def __init__(self, default=None, required=False, help_text=None):
self.required = required
self.help_text = help_text
if default:
self.__set__(self,default)
def __get__(self, instance, owner):
return self.value
def __set__(self, instance, value):
if not isinstance(value, str):
raise TypeError("'{}' must be a string.".format(value))
self.value = value
And this is how I was using it:
class GenericProduct:
name = StringType(required=True) # WRONG: This creates a static member of the class
.....
I have a custom type in dexterity (schema driven) without title or description fields.
class IAnimal(model.Schema):
name = schema.TextLine(title=_(u"Name"),required=True,)
specie = schema.Choice(title=_(u"Specie"),vocabulary=animalSpecies)
weight = schema.TextLine(title=_(u"Weight"))
(etc)
I really don't need the title field on my model, but when I create some content, on folder listing is displaying:
— by admin — last modified Oct 17, 2015 02:27 PM
I created this product with mr.bob and didn't override any forms yet. This can be accomplished by override any forms, a custom behavior (like plone.app.content.interfaces.INameFromTitle) or what?
I just want the "name" field as "title" without changing "name" for "title", even that I must have to hide the "title" field in the model.
Looking at some old archetypes products, it was a method like:
def at_post_create_script(self):
title = self.name
plone_utils = getToolByName(self, 'plone_utils', None)
new_id = plone_utils.normalizeString(title)
self.setTitle(new_id)
I didn't get why you can't simply provide a "title" field named "Name" (easiest way), however you can also override the title attribute and Title method of the base class.
If you used mr.bob you probably don't have a base class you can customize.
Check your type definition XML: in the klass property you probably have plone.dexterity.content.Item. Change it to a new dexterity base class you must add to you product (example: https://github.com/collective/wildcard.media/blob/84b82994dc424fe40b92b1c9af8d48edb558a78d/wildcard/media/content.py#L6)
When you have the base class you can add the Title method and a title attribute, something like:
class MyType(Item):
implements(IMyType)
def Title(self):
return self.name
#property
def title(self)
return self.name
I have a django model that I want to attach an extra piece of information to, depending on the environment the instance is in (which user is logged in). For this reason, I don't want to do it at the database level.
Is this okay to do? Or are there problems that I don't foresee?
in models.py
class FooOrBar(models.Model):
"""Type is 'foo' or 'bar'
"""
def __init__(self, type):
self.type = type
in views.py
class FooCheck(FooOrBar):
"""Never saved to the database
"""
def __init__(self, foo_or_bar):
self.__dict__ = foo_or_bar.__dict__.copy()
def check_type(self, external_type):
if external_type == 'foo':
self.is_foo = True
else:
self.is_foo = False
foos_or_bars = FooOrBar.objects.all()
foochecks = map(FooCheck, foos_or_bars)
for foocheck in foochecks:
foocheck.check_type('foo')
extra credit question: Is there a more efficient way of calling a method on multiple objects i.e. replacing the last forloop with something clever?
Okay, this does not work. Trying to delete a FooOrBar objects throws a complaint about
OperationalError at /
no such table: test_FooCheck
To get around this I'm just not going to inherit from FooOrBar, but if anyone has a suggestion on a better way to do it I'd be interested in hearing it
I had a similar issue, I did something like:
class Foo(models.Model):
# specific info goes here
class Bar(models.Model):
# specific info goes here
class FooBar(models.Model):
CLASS_TYPES = {
"foo":Foo,
"bar":Bar
}
type = models.CharField(choices=CLASS_TYPES)
id = models.IntegerField()
#field to identify FooBar
then you can get the object back using
object = FooBar.CLASS_TYPES[instance.type].objects.get(id=instance.id)
where instance is the FooBar instance