I have a declared model where the table stores a "raw" path identifier of an object. I then have a #hybrid_property which allows directly getting and setting the object which is identified by this field (which is not another declarative model). Is there a way to query directly on this high level?
I can do this:
session.query(Member).filter_by(program_raw=my_program.raw)
I want to be able to do this:
session.query(Member).filter_by(program=my_program)
where my_program.raw == "path/to/a/program"
Member has a field program_raw and a property program which gets the correct Program instance and sets the appropriate program_raw value. Program has a simple raw field which identifies it uniquely. I can provide more code if necessary.
The problem is that currently, SQLAlchemy simply tries to pass the program instance as a parameter to the query, instead of its raw value. This results in a Error binding parameter 0 - probably unsupported type. error.
Either, SQLAlchemy needs to know that when comparing the program, it must use Member.program_raw and match that against the raw property of the parameter. Getting it to use Member.program_raw is done simply using #program.expression but I can't figure out how to translate the Program parameter correctly (using a Comparator?), and/or
SQLAlchemy should know that when I filter by a Program instance, it should use the raw attribute.
My use-case is perhaps a bit abstract, but imagine I stored a serialized RGB value in the database and had a property with a Color class on the model. I want to filter by the Color class, and not have to deal with RGB values in my filters. The color class has no problems telling me its RGB value.
Figured it out by reading the source for relationship. The trick is to use a custom Comparator for the property, which knows how to compare two things. In my case it's as simple as:
from sqlalchemy.ext.hybrid import Comparator, hybrid_property
class ProgramComparator(Comparator):
def __eq__(self, other):
# Should check for case of `other is None`
return self.__clause_element__() == other.raw
class Member(Base):
# ...
program_raw = Column(String(80), index=True)
#hybrid_property
def program(self):
return Program(self.program_raw)
#program.comparator
def program(cls):
# program_raw becomes __clause_element__ in the Comparator.
return ProgramComparator(cls.program_raw)
#program.setter
def program(self, value):
self.program_raw = value.raw
Note: In my case, Program('abc') == Program('abc') (I've overridden __new__), so I can just return a "new" Program all the time. For other cases, the instance should probably be lazily created and stored in the Member instance.
Related
I'm using MongoEngine and trying to create a field that works like SQLAlchemy's PickleType field. Basically, I just need to pickle objects before they're written to the database, and unpickle them when they're loaded.
However it looks like MongoEngine's fields don't provide proper conversion methods I could override, instead having two coercion methods (to_python and to_mongo). If I understand correctly, these functions can be called anytime, that is, a call to to_python(v) does not guarantee that v comes from the database. I've thought of writing something like this:
class PickleField(fields.BinaryField):
def to_python(self, value):
value = super().to_python(value)
if <<value was pickled by the field>>
return pickle.loads(value)
else:
return value
Unfortunately, if I want to be as general as possible, I don't see a way to check whether the value should be unpickled or not. For instance,
a = pickle.dumps(x)
PickleField().to_python(a) # should return a, will return x
I also don't think I can store any state in the PickleField, since that's shared by all instances.
Is there a way around this?
Python's peewee orm library allows me to query database using the get method like this:
grandma = Person.get(Person.name == 'Grandma L.')
This query will filter the resultset based on Person's name doing a sql where behind the scenes.
I know the expression Person.name == 'Grandma L.' is evaluated first and the get method receives just a boolean value.
How does the get method inspect it's arguments to detect that the filter needs to be applyed to the 'name' field ?
PS: I read peewee's source but couldn't have any clue how it does that.
I know the expression Person.name == 'Grandma L.' is evaluated first and the get method receives just a boolean value.
I don't know Peewee specifically but I do know how this sort of thing is done.
Basically, the second part of your sentence isn't true. name is of a custom type. Its __eq__ method doesn't return just a Boolean value, but rather an object that contains information on what comparison was actually done. (This class might even derived from bool so it works like a Boolean in other contexts.) Its other rich comparison methods are similar.
But how does Person.name know that its name is name? The answer to that is that Person probably doesn't actually have a name attribute. Instead, it has a __getattr__() method that returns the name object, which is the custom class that has the __eq__ method I just described.
Since Person.__getattr__() receives the name of the attribute, it can bake that name into the value it returns for name. Then the custom __eq__ method on name returns a Boolean-like object that contains some representation of name == 'Grandma L.' inside it. And Person.get() uses this representation to put together the query.
I have an sqlalchemy mixin and I want to set a default value for the mixin's column, but I haven't found a direct answer and have resorted the the trial and error of the sort of scientific method variety. This is my first attempt, and I haven't had a chance to use it yet:
class ExampleMixin(object):
"""Mixin for `AnExample`"""
def __init__(self, example_id):
self.example_id = HMAC("{}".format(urandom(32).encode('base_64'))).hexdigest()
super(ExampleMixin, self).__init__(self.example_id)
example_id = Column(String(255))
Admittedly, I do not understand fully the use of super yet, but I need a way to provide a default value for a mixin without putting it in the model receiving the mixin. I'm sure the above isn't fully correct, but is where I'm starting.
How do you provide default values that are only initialized once on creation?
According to Mixing in Columns and Column Insert/Update Defaults, the code below should achieve what you desire:
def _example_id_default():
return HMAC("{}".format(urandom(32).encode('base_64'))).hexdigest()
class ExampleMixin(object):
"""Mixin for `AnExample`"""
example_id = Column(String(255), default=_example_id_default)
Is there a way to perform validation on an object after (or as) the properties are set but before the session is committed?
For instance, I have a domain model Device that has a mac property. I would like to ensure that the mac property contains a valid and sanitized mac value before it is added to or updated in the database.
It looks like the Pythonic approach is to do most things as properties (including SQLAlchemy). If I had coded this in PHP or Java, I would probably have opted to create getter/setter methods to protect the data and give me the flexibility to handle this in the domain model itself.
public function mac() { return $this->mac; }
public function setMac($mac) {
return $this->mac = $this->sanitizeAndValidateMac($mac);
}
public function sanitizeAndValidateMac($mac) {
if ( ! preg_match(self::$VALID_MAC_REGEX) ) {
throw new InvalidMacException($mac);
}
return strtolower($mac);
}
What is a Pythonic way to handle this type of situation using SQLAlchemy?
(While I'm aware that validation and should be handled elsewhere (i.e., web framework) I would like to figure out how to handle some of these domain specific validation rules as they are bound to come up frequently.)
UPDATE
I know that I could use property to do this under normal circumstances. The key part is that I am using SQLAlchemy with these classes. I do not understand exactly how SQLAlchemy is performing its magic but I suspect that creating and overriding these properties on my own could lead to unstable and/or unpredictable results.
You can add data validation inside your SQLAlchemy classes using the #validates() decorator.
From the docs - Simple Validators:
An attribute validator can raise an exception, halting the process of mutating the attribute’s value, or can change the given value into something different.
from sqlalchemy.orm import validates
class EmailAddress(Base):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
email = Column(String)
#validates('email')
def validate_email(self, key, address):
# you can use assertions, such as
# assert '#' in address
# or raise an exception:
if '#' not in address:
raise ValueError('Email address must contain an # sign.')
return address
Yes. This can be done nicely using a MapperExtension.
# uses sqlalchemy hooks to data model class specific validators before update and insert
class ValidationExtension( sqlalchemy.orm.interfaces.MapperExtension ):
def before_update(self, mapper, connection, instance):
"""not every instance here is actually updated to the db, see http://www.sqlalchemy.org/docs/reference/orm/interfaces.html?highlight=mapperextension#sqlalchemy.orm.interfaces.MapperExtension.before_update"""
instance.validate()
return sqlalchemy.orm.interfaces.MapperExtension.before_update(self, mapper, connection, instance)
def before_insert(self, mapper, connection, instance):
instance.validate()
return sqlalchemy.orm.interfaces.MapperExtension.before_insert(self, mapper, connection, instance)
sqlalchemy.orm.mapper( model, table, extension = ValidationExtension(), **mapper_args )
You may want to check before_update reference because not every instance here is actually updated to the db.
"It looks like the Pythonic approach is to do most things as properties"
It varies, but that's close.
"If I had coded this in PHP or Java, I would probably have opted to create getter/setter methods..."
Good. That's Pythonic enough. Your getter and setter functions are bound up in a property; that's pretty good.
What's the question?
Are you asking how to spell property?
However, "transparent validation" -- if I read your example code correctly -- may not really be all that good an idea.
Your model and your validation should probably be kept separate. It's common to have multiple validations for a single model. For some users, fields are optional, fixed or not used; this leads to multiple validations.
You'll be happier following the Django design pattern of using a Form for validation, separate form the model.
Is there a way to get the key (or id) value of a db.ReferenceProperty, without dereferencing the actual entity it points to? I have been digging around - it looks like the key is stored as the property name preceeded with an _, but I have been unable to get any code working. Examples would be much appreciated. Thanks.
EDIT: Here is what I have unsuccessfully tried:
class Comment(db.Model):
series = db.ReferenceProperty(reference_class=Series);
def series_id(self):
return self._series
And in my template:
more
The result:
more
Actually, the way that you are advocating accessing the key for a ReferenceProperty might well not exist in the future. Attributes that begin with '_' in python are generally accepted to be "protected" in that things that are closely bound and intimate with its implementation can use them, but things that are updated with the implementation must change when it changes.
However, there is a way through the public interface that you can access the key for your reference-property so that it will be safe in the future. I'll revise the above example:
class Comment(db.Model):
series = db.ReferenceProperty(reference_class=Series);
def series_id(self):
return Comment.series.get_value_for_datastore(self)
When you access properties via the class it is associated, you get the property object itself, which has a public method that can get the underlying values.
You're correct - the key is stored as the property name prefixed with '_'. You should just be able to access it directly on the model object. Can you demonstrate what you're trying? I've used this technique in the past with no problems.
Edit: Have you tried calling series_id() directly, or referencing _series in your template directly? I'm not sure whether Django automatically calls methods with no arguments if you specify them in this context. You could also try putting the #property decorator on the method.