We Keep Coding

How to pass a class name with a string in Django models

I want to get all of objects that are related to an instance of models.
Because my code is kinda generic, I pass the related table as an string and use eval() function to convert it to the related table class. But I got an error.
Suppose that we have an instance of a table like self.casefile; this is a part of my code:
def related_models_migration(self):
opts = self.casefile._meta
table_name = 'Files'
for f in opts.many_to_many:
name = ''.join(f.name.split('_'))
table_name += name.capitalize()
objects = self.casefile.eval(table_name).all()
and I got this error:
AttributeError Traceback (most recent call last)
<ipython-input-6-025484eeba97> in <module>
----> 1 obj.related_models_migration()
~/Documents/kangaroo/etl/data_migration.py in related_models_migration(self)
28 name = ''.join(f.name.split('_'))
29 table_name += name.capitalize()
---> 30 objects = self.casefile.eval(table_name).all()
31
32 for d in dir(etl.models):
AttributeError: 'FilesCasefiles' object has no attribute 'eval'
How can I pass the class name?

You can not use eval(..) for that. What you probably want to use here is getattr(..):
def related_models_migration(self):
opts = self.casefile._meta
table_name = 'Files'
for f in opts.many_to_many:
name = ''.join(f.name.split('_'))
table_name += name.capitalize()
objects = getattr(self.casefile, table_name).all()
I am not sure you should use table_name += … here however, since it will each time add more content to the table_name. You likely want to use something like table_name = 'Files{}'.format(name.capitalize()).
Note: normally related fields are not capitalized. One writes users or user_set, not Users.

Django provides a way to do this, although you do need to specify the name of the app in which the moodel is defined (because it's possible to have two models with the same name in different apps).
apps.get_model(app_label, model_name, require_ready=True)¶
Returns the Model with the given app_label and model_name.
As a shortcut, this method also accepts a single argument in the form
app_label.model_name. model_name is case-insensitive.

flask sqlalchemy query with keyword as function argument

I am stuck with a query in a function. here is my code:
def action(changePin, action):
pins = Pins.query.all()
changePin = int(changePin)
deviceName = Pins.query.filter_by(pin=changePin, name)
if action == "on":
GPIO.output(changePin, GPIO.HIGH)
print("turned ", deviceName , " on")
if action =="off":
GPIO.output(changePin, GPIO.LOW)
print("turned ", deviceName , " off")
for pin in pins:
db.session.commit()
The error for this is
File "<stdin>", line 4
SyntaxError: positional argument follows keyword argument
In line 4 I want to find the name of the pin relating to the pin "changePin", this is adapted code from a tutorial, here is the origional code where a dictionary holds the pin information not a database, code:
deviceName = pins[changePin]['name']
I have tried numerous different ways but none work, here is a list of the different versions of line 4:
deviceName = Pins.query.filter_by(changePin=pin).name
deviceName = Pins.query.filter_by(changePin, name=name)
deviceName = Pins.query.filter_by(Pins.pin=changePin, Pins.Name)
deviceName = Pins.query(Pins.pin=changePin, Pins.Name)
deviceName = Pins.query(**changePin, Pins.name)
deviceName = Pins.query(**changePin)
deviceName = db.session.filter_by(Pins.changePin)
deviceName = db.session(Pins).filter_by(pin=changePin)
and many other variations, I have read the sqlalchemy docs and the flask docs, but I have not seen any comparisons, I have looked at and tried this; flask sqlalchemy query with keyword as variable
but got this;
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in action
TypeError: BaseQuery object argument after ** must be a mapping, not int
this is my models.py code;
class Pins(db.Model):
id = db.Column(db.Integer, primary_key=True)
pin = db.Column(db.Integer, index=True, unique=True)
name = db.Column(db.String(64))
upDown = db.Column(db.String(4))
state = db.Column(db.String(9))
def __repr__(self):
return '<Valves {}>'.format(self.pin)
Querying with function on Flask-SQLAlchemy model gives BaseQuery object is not callable error
Dear Menno
Thank you for comming back to me I have followed your advice and it works!!!
rows = Pins.query.filter_by(pin=changePin).all()
deviceName = rows[0].name
I don't understand how the "rows[0].name" part works but it does,
thank you thank you
help
regards
Paul

A filter_by expression selects rows, not fields. From the documentation: "apply the given filtering criterion to a copy of this Query, using keyword expressions.". Name is not a keyword expression, just a name.
Also, after building the query, you have to run it. You do that by calling all() on it:
deviceName = Pins.query.filter_by(pin=changePin).all()
Letś also change the name of the outcome:
rows = Pins.query.filter_by(pin=changePin).all()
Calling that returns a list with zero or more rows, hence the name. rows[0] (the first row) is what you want. After you have a row, a field becomes an attribute of the row: rows[0].name. You can also use one() to get a unique row.
If you still need that: To limit the columns that are returned use load only cols

What can cause failure to store an `int` to an `IntField`?

I have the following MongonEngine models:
from app import db
from datetime import datetime
from mongoengine import signals
class PathEmbedded(db.EmbeddedDocument):
"""
To be embedded.
"""
_id = db.ObjectIdField(required=False)
distance = db.IntField(required=False, min_value=0, default=0)
meta = {
"allow_inheritance": True,
}
def __unicode__(self):
return "Path '%s': %d m" % (self.id, self.distance)
class Path2(PathEmbedded, db.Document):
"""
Same as above, but standalone version to be stored in its own collection.
"""
_id = db.ObjectIdField()
orig = db.ObjectIdField(required=True)
dest = db.ObjectIdField(required=True)
updateStamp = db.DateTimeField(required=True)
ok_to_use = db.BooleanField(required=True, default=False)
meta = {
'indexes': [
{
'fields': ['ok_to_use', 'orig', 'dest'],
'cls': False, # does this affect performance?!
},
],
}
#classmethod
def pre_save(cls, sender, document, **kwargs):
document.updateStamp = datetime.utcnow()
def to_embedded(self):
"""
Converts the standalone Path instance into an embeddadle PathEmbedded instance.
"""
import json
temp = json.loads(self.to_json())
#remove the {"_cls": "Location"} key.
#If we don't do this, the output will be a 'Location' instance, not a 'LocationEmbedded' instace
temp.pop('_cls')
return PathEmbedded().from_json(json.dumps(temp))
def get_from_gmaps(self):
"""
Get distance from Google maps using the directions API and append to the 'paths' list.
Return False on error or True on success.
"""
try:
self.distance = 10,
self.save()
except Exception, e:
print str(e)
return False
else:
return True
# connect event hooks:
signals.pre_save.connect(Path2.pre_save, sender=Path2)
So, at some point I'm updating a path instance by calling get_from_gmaps():
from app.models.Path2 import Path2 as P
from bson import ObjectId
p=P(orig=ObjectId(), dest=ObjectId())
p.save()
p.get_from_gmaps()
which raises:
>>> p.get_from_gmaps()
ValidationError (Path2:54d34b97362499300a6ec3be) (10 could not be converted to int: ['distance'])
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "[...]app/models/Path2/get_from_gmaps.py", line 18, in get_from_gmaps
self.save()
File "[...]venv/local/lib/python2.7/site-packages/mongoengine/document.py", line 224, in save
self.validate(clean=clean)
File "[...]venv/local/lib/python2.7/site-packages/mongoengine/base/document.py", line 323, in validate
raise ValidationError(message, errors=errors)
ValidationError: ValidationError (Path2:54d34b97362499300a6ec3be) (10 could not be converted to int: ['distance'])
Originally I was storing an integer parsed from some json and converted to int, and thought somthing was wrong there, but i replaced it with an int value for debugging and now get this. I really don't know where to start o.O
EDIT: expanded code to provide complete [non]working example.

There's an extra comma after the 10:
self.distance = 10,
^
You are setting distance to a tuple containing an int, instead of an int.
HINT: The reason why your are seeing such an unhelpful message is that MongoEngine is using %s format string improperly. In fact, the result of "%s" % something depends on the type of something, as tuples are special cased. Compare:
>>> '%s' % 10
'10'
>>> '%s' % (10,)
'10'
>>> '%s' % (10, 11)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: not all arguments converted during string formatting
>>> '%s' % ((10,),) # This is the correct way of formatting strings
'(10,)' # when the type of the argument is unknown.
This is a MongoEngine's problem of course, but if you want to avoid the same kind of mistake in your code, remember to always use tuples at the right of the % operator, or even better use the .format() method.

Are you sure the self model you send is the right one?
This ValidationError is thrown when you have declare a ReferenceField in a document, and you try to save this document before saving the referenced document (Mongoengine represents a reference field in MongoDB as an dictionnay containing the class and the ObjectId of the reference).

serializing sqlalchemy class to json

I'm trying to serialize the result (a list) of an sqlalchemy query to json.
this is the class:
class Wikilink(Base):
__tablename__='Wikilinks'
__table_args__={'extend_existing':True}
id = Column(Integer,autoincrement=True,primary_key=True)
title = Column(Unicode(350))
user_ip = Column(String(50))
page = Column(String(20))
revision = Column(String(20))
timestamp = Column(String(50))
and I guess my problem is with the __repr__(self): function.
I tried something like:
return '{{0}:{"title":{1}, "Ip":{2}, "page":{3} ,"revision":{4}}}'.format(self.id,self.title.encode('utf-8'),self.user_ip,self.page,self.revision)
or:
return '{"id"={0}, "title"={1}, "Ip"={2}}'.format(self.id,self.title.encode('utf-8'),self.user_ip.encode('utf-8'),self.page,self.revision)
and I got:
TypeError(repr(o) + " is not JSON serializable")
ValueError: Single '}' encountered in format string
I tried:
return '{id=%d, title=%s, Ip=%s}'%(self.id,self.title.encode('utf-8'),self.user_ip.encode('utf-8'))
and I got:
TypeError: {id=8126, title=1 בדצמבר, Ip=147.237.70.106} is not JSON serializable
adding "" around (according to the JSON formatting) like this: "id"="%d", "title"="%s", "Ip"="%s" didn't help either.
I know this is supposed to be dead simple but I just can't get this right
actually bottle is handling the jsonification part automatically, but trying to call json.dumps on the result gives me the same errors.

Instead of trying to convert to json a string, you could define, for example, your own to_dict method that returns the dictionary structure it seems you're trying to create and, after that, generate the json from that structure:
>>> import json
>>> d = {'id':8126, 'title':u'1 בדצמבר', 'ip':'147.237.70.106'}
>>> json.dumps(d)
'{"ip": "147.237.70.106", "id": 8126, "title": "1 \\u05d1\\u05d3\\u05e6\\u05de\\u05d1\\u05e8"}'

I'm not sure I understand what you tried. Couldn't you build the dict and let json.dumps() do the work for you?
Something like:
>>> class Foo:
... id = 1
... title = 'my title'
... to_jsonize = ['id', 'title']
>>>
>>> dct = {name: getattr(Foo,name) for name in Foo.to_jsonize}
>>> import json
>>> json.dumps(dct)
'{"id": 1, "title": "my title"}'

SQLAlchemy - MappedCollection problem

I have some problems with setting up the dictionary collection in Python's SQLAlchemy:
I am using declarative definition of tables. I have Item table in 1:N relation with Record table. I set up the relation using the following code:
_Base = declarative_base()
class Record(_Base):
__tablename__ = 'records'
item_id = Column(String(M_ITEM_ID), ForeignKey('items.id'))
id = Column(String(M_RECORD_ID), primary_key=True)
uri = Column(String(M_RECORD_URI))
name = Column(String(M_RECORD_NAME))
class Item(_Base):
__tablename__ = 'items'
id = Column(String(M_ITEM_ID), primary_key=True)
records = relation(Record, collection_class=column_mapped_collection(Record.name), backref='item')
Now I want to work with the Items and Records. Let's create some objects:
i1 = Item(id='id1')
r = Record(id='mujrecord')
And now I want to associate these objects using the following code:
i1.records['source_wav'] = r
but the Record r doesn't have set the name attribute (the foreign key). Is there any solution how to automatically ensure this? (I know that setting the foreign key during the Record creation works, but it doesn't sound good for me).
Many thanks

You want something like this:
from sqlalchemy.orm import validates
class Item(_Base):
[...]
#validates('records')
def validate_record(self, key, record):
assert record.name is not None, "Record fails validation, must have a name"
return record
With this, you get the desired validation:
>>> i1 = Item(id='id1')
>>> r = Record(id='mujrecord')
>>> i1.records['source_wav'] = r
Traceback (most recent call last):
[...]
AssertionError: Record fails validation, must have a name
>>> r.name = 'foo'
>>> i1.records['source_wav'] = r
>>>

I can't comment yet, so I'm just going to write this as a separate answer:
from sqlalchemy.orm import validates
class Item(_Base):
[...]
#validates('records')
def validate_record(self, key, record):
record.name=key
return record
This is basically a copy of Gunnlaugur's answer but abusing the validates decorator to do something more useful than exploding.

You have:
backref='item'
Is this a typo for
backref='name'
?