Consider this simplified scenario. Master-detail tables:
CREATE TABLE queue (
id bigint PRIMARY KEY,
num text NOT NULL UNIQUE
);
CREATE TABLE queue_device (
id bigint PRIMARY KEY,
queue_id bigint NOT NULL REFERENCES queue ON DELETE CASCADE,
device text NOT NULL,
UNIQUE (queue_id,device)
);
When adding devices, users obviously don't know the id, they enter num instead. So I tried this validation schema:
SCHEMA = {
'queue': {
'type': 'string',
'empty': False,
'required': True,
'rename': 'queue_id',
'coerce': 'queue_id'
},
'device': {
'type': 'string',
'empty': False,
'required': True
}
}
I wanted to rename the field and coerce it to proper value, but custom coercer does not get executed. I am sure there is a rationale for doing renaming before coercing, but I for one don't see it. This way, you effectively can't have both rename and coerce rules on same field.
OK, so I tried to set coercer on a renamed field, marking it readonly because users must not set it directly.
SCHEMA = {
'queue': {
'type': 'string',
'empty': False,
'required': True,
'rename': 'queue_id'
},
'device': {
'type': 'string',
'empty': False,
'required': True
},
'queue_id': {
'readonly': True,
'coerce': 'queue_id'
}
}
I do the validation first, then normalization.
if not validator.validate(document, normalize=False):
raise ValidationError('Document validation failed.', validator.errors)
document = validator.normalized(document)
This fails because of the readonly rule. Again, I wonder what is the rationale for checking readonly during normalization, as this is a validation, not normalization rule.
I keep hitting a wall. What is the proper way to write a validation schema in this case?
Related
Consider the following Cerberus schema:
{
'employee': {
'type': 'list',
'schema': {
'type': 'dict',
'schema': {
'id': {'required': True, 'type': 'integer'},
'name': {'required': True, 'type': 'string'}
}
}
},
'ceo-employee-id': {'required': True, 'type': 'integer'}
}
1) How can I validate that the ceo-employee-id matches one of the id values in the employee list? (Referential integrity)
2) How can I validate that each id in the employee list is unique (i.e. no duplicate employee ids)?
I realize I can do this at run-time after validating and parsing the config as suggested by #rafael below. I am wondering if I can do it with the Cerberus validation features.
You'll need to make use of a custom validator that implements check_with methods, use the document property in these, and amend your schema to include these:
from cerberus import Validator
class CustomValidator(Validator):
def _check_with_ceo_employee(self, field, value):
if value not in (x["id"] for x in self.document["employee"]):
self._error(field, "ID is missing in employee list.")
def _check_with_employee_id_uniqueness(self, field, value):
all_ids = [x["id"] for x in self.document["employee"]]
if len(all_ids) != len(set(all_ids)):
self._error(field, "Employee IDs are not unique.")
validator = CustomValidator({
'employee': {
'type': 'list',
'schema': {
'type': 'dict',
'schema': {
'id': {'required': True, 'type': 'integer'},
'name': {'required': True, 'type': 'string'}
},
},
'check_with': 'employee_id_uniqueness'
},
'ceo-employee-id': {'required': True, 'type': 'integer', 'check_with': 'ceo_employee'}
})
The referenced document contains hints on all the parts used here.
(I apologize for any indentation error that might have slipped into the example.)
assuming that you already have validated the schema of your json you can easily check your two conditions like this.
Let doc be your json document.
employee_ids = [employee['id'] for employee in doc['employee']]
ceo_employee_id = doc['ceo-employee-id']
1) How can I validate that the ceo-employee-id matches one of the id values in the employee list? (Referential integrity)
ceo_id_exists_in_employees = any([employee_id == ceo_employee_id for employee_id in employee_ids])
2) How can I validate that each id in the employee list is unique (i.e. no duplicate employee ids)?
employee_id_is_unique = len(set(employee_ids)) == len(employee_ids)
3) Assert that both values are True
if ceo_id_exists_in_employees and employee_id_is_unique:
print('passed')
else:
print('failed')
currently when I DELETE an item which is being referenced by. I can still delete it. I would like to prohibit this giving an error. I couldn't find anything on http://docs.python-eve.org or google.
say I have the following two schemas:
foos = {
...
'schema' : {
...,
'bar': {
'type': 'string',
'required': true,
'data_relation': {
'resource': 'bars',
'field': 'name',
'embeddable': True
}
}
}
}
bars = {
...
'schema' : {
...,
'name': {
'type': 'string',
'required': true,
}
}
}
if I delete now an item from bars which is still being referenced by >=1 items in foos. How could I return an error instead of deleting the bars item?
Would i need to write a
on_delete_item
def event(resource_name, item)
and how? Any help is appreciated.
I want to change the default type from dict to string for a particular user.
DOMAIN = {
'item': {
'schema': {
'profile':{
'type': 'dict'
},
'username': {
'type': 'string'
}
}
}
}
suppose if I get a request from x user type should not change. If I get a request from y user type should change from dict to string. How to change for a particular item resource without affecting others.
TIA.
Your best approach would probably be to set up two different API endpoints, one for users of type X, and another for users of type Y. Both endpoints would consume the same underlying datasource (same DB collection being updated). You achieve that by setting the datasource for your endpoint, like so:
itemx = {
'url': 'endpoint_1',
'datasource': {
'source': 'people', # actual DB collection consumed by the endpoint
'filter': {'usertype': 'x'} # optional
'projection': {'username': 1} # optional
},
'schema': {...} # here you set username to dict, or string
}
Rinse and repeat for the second endpoint. See the docs for more info.
I have the following resource defined:
item = {
'wrapper': {
'type': 'dict',
'schema': {
'element': {
'type': 'objectid',
'data_relation': {
'resource': 'code',
'field': '_id',
'embeddable': True,
},
},
},
},
}
When I try to query using the objectid, I get empty list.
http://127.0.0.1:5000/item?where={"wrapper.element":"5834987589b0dc353b72c27d"}
5834987589b0dc353b72c27d is the valid _id for the element.
If I move the data relation out of the embedded document I can query it as expected
Is there anyway to do this with an embedded data relation?
I have just tested with eve==0.7.1 and it works as expected by filtering with ?where={"wrapper.element" : "<your_objectid>"}, as you said.
I had a problem where the _id was being stored as a string rather than an ObjectId(), this broke the query
In Python Eve framework, is it possible to have a condition which checks combination of two fields to be unique?
For example the below definition restricts only firstname and lastname to be unique for items in the resource.
people = {
# 'title' tag used in item links.
'item_title': 'person',
'schema': {
'firstname': {
'type': 'string',
'required': True,
'unique': True
},
'lastname': {
'type': 'string',
'required': True,
'unique': True
}
}
Instead, is there a way to restrict firstname and lastname combination to be unique?
Or is there a way to implement a CustomValidator for this?
You can probably achieve what you want by overloading the _validate_unique and implementing custom logic there, taking advantage of self.document in order to retrieve the other field value.
However, since _validate_unique is called for every unique field, you would end up performing your custom validation twice, once for firstname and then for lastname. Not really desirable. Of course the wasy way out is setting up fullname field, but I guess that's not an option in your case.
Have you considered going for a slighty different design? Something like:
{'name': {'first': 'John', 'last': 'Doe'}}
Then all you need is make sure that name is required and unique:
{
'name': {
'type':'dict',
'required': True,
'unique': True,
'schema': {
'first': {'type': 'string'},
'last': {'type': 'string'}
}
}
}
Inspired by Nicola and _validate_unique.
from eve.io.mongo import Validator
from eve.utils import config
from flask import current_app as app
class ExtendedValidator(Validator):
def _validate_unique_combination(self, unique_combination, field, value):
""" {'type': 'list'} """
self._is_combination_unique(unique_combination, field, value, {})
def _is_combination_unique(self, unique_combination, field, value, query):
""" Test if the value combination is unique.
"""
if unique_combination:
query = {k: self.document[k] for k in unique_combination}
query[field] = value
resource_config = config.DOMAIN[self.resource]
# exclude soft deleted documents if applicable
if resource_config['soft_delete']:
query[config.DELETED] = {'$ne': True}
if self.document_id:
id_field = resource_config['id_field']
query[id_field] = {'$ne': self.document_id}
datasource, _, _, _ = app.data.datasource(self.resource)
if app.data.driver.db[datasource].find_one(query):
key_names = ', '.join([k for k in query])
self._error(field, "value combination of '%s' is not unique" % key_names)
The way I solved this issue is by creating a dynamic field using a combination of functions and lambdas to create a hash that will use
which ever fields you provide
def unique_record(fields):
def is_lambda(field):
# Test if a variable is a lambda
return callable(field) and field.__name__ == "<lambda>"
def default_setter(doc):
# Generate the composite list
r = [
str(field(doc)
# Check is lambda
if is_lambda(field)
# jmespath is not required, but it enables using nested doc values
else jmespath.search(field, doc))
for field in fields
]
# Generate MD5 has from composite string (Keep it clean)
return hashlib.md5(''.join(r).encode()).hexdigest()
return {
'type': 'string',
'unique': True,
'default_setter': default_setter
}
Practical Implementation
My use case was to create a collection that limits the amount of key value pairs a user can create within the collection
domain = {
'schema': {
'key': {
'type': 'string',
'minlength': 1,
'maxlength': 25,
'required': True,
},
'value': {
'type': 'string',
'minlength': 1,
'required': True
},
'hash': unique_record([
'key',
lambda doc: request.USER['_id']
]),
'user': {
'type': 'objectid',
'default_setter': lambda doc: request.USER['_id'] # User tenant ID
}
}
}
}
The function will receive a list of either string or lambda function for dynamic value setting at request time, in my case the user's "_id"
The function supports the use of JSON query with the JMESPATH package, this isn't mandatory, but leave the door open for nested doc flexibility in other usecases
NOTE: This will only work with values that are set by the USER at request time or injected into the request body using the pre_GET trigger pattern, like the USER object I inject in the pre_GET trigger which represents the USER currently making the request