We Keep Coding

'ReverseManyToOneDescriptor' object has no attribute 'latest'

I have received this error when trying to run a function. This is my first django/python project so I am not experienced in this. I have searched for this error but not found anything similar.
def getpriority(chunks):
p = 0
for chunk in chunks:
a = chunk.result_set.all()
l = a.latest()
if pytz.utc.localize(datetime.now()) - l.timestamp > datetime.timedelta(days=3):
x = getresult(chunk)
print(x)
I am trying to assign priorities to my Chunk model so I can select the Chunk with the highest priority to use the object.
I believe that my error is in calling latest() on 'a'.
When I run Chunk.result_set.latest() in a django shell I get the following error:
Traceback (most recent call last):
File "<console>", line 1, in <module>
AttributeError: 'ReverseManyToOneDescriptor' object has no attribute 'latest'
In my Result model, I have set get_latest_by which I believe is required to run .latest():
class Result(models.Model):
rel_chunk = models.ForeignKey(Chunk, on_delete=models.CASCADE)
score = models.IntegerField()
timestamp = models.DateTimeField(auto_now_add=True)
class Meta:
get_latest_by = 'timestamp'
I believe that the error lies in the fact that I'm calling latest on a related object set but if this can't be called on a related object set then how can I find the latest related Result?

It should be chunk.result_set.latest() not Chunk.result_set.latest()
note that chunk should be a instance not a class model.

The error indicates that you're accessing the result_set attribute on the model class. You need to access it on a model instance, in which case it will return a proper manager object instead of a ReverseManyToOneDescriptor:
chunk = Chunk.objects.get(...)
chunk.result_set.latest()
Here, chunk.result_set is a manager that behaves exactly like Result.objects, except it filters the queryset to only include results related to the chunk instance. The manager also has some added methods to manipulate the relation between chunk and the results, such as chunk.result_set.add() and chunk.result_set.remove() to add/remove results from the result set related to this chunk instance.
As this all applies to a specific instance of Chunk, it makes no sense to access the related manager on a class level. That's why you need a Chunk instance to use the related manager.
Since chunk.result_set is a manager instance, it proxies all queryset methods*. There is no reason why you would need chunk.result_set.all().latest() instead of just chunk.result_set.latest().
*) delete() is disabled on managers so you don't accidentally delete all objects. as_manager() is disabled as it makes no sense on a manager.

odd behavior of the "type" built-in python function when creating classes on the fly

I am working on a project in which I am using the API of some third party closed-source Django application in order to write a plugin that extends its functionality.
The plugin consists of defining resource classes by inheriting from pre-made existing ones (defined in the Django application) and then writing the required logic to create those new resources and afterwards updating them on a regular basis
The straight forward way to create the custom resource classes is by using the class keyword as in the following code snippet that defines a new resource called Volume
from chroma_core.lib.storage_plugin.api import attributes, identifiers, plugin, relations, resources, statistics
class Volume(resources.PhysicalDisk):
name = attributes.String()
storagePool = attributes.String()
wwn = attributes.String()
health = attributes.String()
class Meta:
identifier = identifiers.ScopedId('serialNumber')
the instantiation of object resources from the custom resource defined above is done by using a method from the given API that has the following signature:
Plugin.update_or_create(ResourceClass, parents=[], **attrs) which is defined by the API
Doing things in this way, everything is good and there are no problems at all.
However, for many reasons and requirement constraints I have to create the custom resources on the fly, so I moved to use the "type" built-in function as folowes:
Meta = type('Meta', (), {'identifier': identifiers.ScopedId('serialNumber')})
superclasses = (resources.PhysicalDisk, )
attrs = {'name': attributes.String(), 'storagePool': attributes.String(), 'wwn': attributes.String(), 'health': attributes.String(), 'Meta': Meta}
Volume = type('Volume', superclasses, attrs)
unfortunately, when creating new objects from the custom resource "Volume" defined on the fly I get an error:
[2014-07-21 09:30:26,916: WARNING/scan_daemon] Backtrace: Traceback (most recent call last):
File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/scan_daemon.py", line 192, in run
File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/scan_daemon.py", line 222, in _scan_loop
File "/usr/share/chroma-manager/chroma_core/lib/storage_plugin/base_plugin.py", line 193, in do_initial_scan
File "/usr/share/chroma-manager/chroma_core/lib/storage_plugin/base_plugin.py", line 210, in _initial_populate
File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/resource_manager.py", line 327, in session_open
File "/usr/lib/python2.6/site-packages/django/db/transaction.py", line 224, in inner
return func(*args, **kwargs)
File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/resource_manager.py", line 1361, in _persist_new_resources
File "/usr/share/chroma-manager/chroma_core/services/plugin_runner/resource_manager.py", line 1340, in order_by_references
File "/usr/share/chroma-manager/chroma_core/lib/storage_plugin/manager.py", line 229, in get_plugin_resource_class
PluginNotFound: PluginNotFound: Plugin chroma_core.lib.storage_plugin.base_resource not found (not one of ['pluginmngr', 'linux_network', 'linux'])
I did a lot of debugging, and by inspecting the pluginmngr.py module with inspect.getmembers I figured out a very odd behavior that I do not understand:
here below is the inspecting code:
module_obj = sys.modules[__name__]
inspect.getmembers(module_obj)
when the Volume resource is defined statically by the class keyword then inspection shows that Volume is reported as expected
'Volume': <class 'pluginmngr.Volume'>
while when creating the Volume on the fly using the "type" built-in function as showed above, then inspecting the module shows that Volume is reported as
'Volume': <class 'chroma_core.lib.storage_plugin.base_resource.Volume'>
hence, the custom resource Volume oddly enough was associated to some strange module that belongs to the django application (as stated before, it is closed source and I do not have access to the source code of that module).
for the sake of trying to understand better what's going on, I removed the inheritance from Volume = type('Volume', superclasses, attrs) and created the volume just as Volume = type('Volume',(), attrs)
then inspecting the module in this case shows that Volume is correctly associated to pluginmngr (pluginmngr.Volume) as one expects and not to chroma_core.lib.storage_plugin.base_resource
However, inheriting from some built-in resource class is compulsory and the problem is still there
so to summarize, could anyone explain why defining the Volume resource class in two different ways firstly by the class keyword, and secondly by the type built-in function (that should be perfectly interchangeable) leads in the first case to pluginmngr.Volume and to chroma_core.lib.storage_plugin.base_resource.Volume in the second case?
best regards

python cherrypy session usage example of cherrypy.lib.sessions

I am total newbie with cherrypy.
My setup: Arch Linux, Python 3.3, tornado, cherrypy 3.2
Trying to implement session handling for a web app using cherrypy.lib.sessions (for some reason often referred to as cherrypy.sessions in various forums, might be another version)
I am looking for an example of the following:
instantiate a session object
set a value of an arbitrarily named attribute
write session into a session file
read session info from session file
access the the value of the modified attribute
My (relevant) code:
import cherrypy
class RequestHandlerSubmittedRequest(tornado.web.RequestHandler):
def get(self):
SetState(self)
def SetState(self):
cherrypy.config.update({'tools.sessions.on': True})
cherrypy.config.update({'tools.sessions.storage_type': 'file'})
#directory does exist
cherrypy.config.update({'tools.sessions.storage_path': '/tmp/cherrypy_sessions'})
cherrypy.config.update({'tools.sessions.timeout': 60})
cherrypy.config.update({'tools.sessions.name': 'hhh'})
So far so good. Now:
obj_session = cherrypy.lib.sessions.FileSession
Here I get the first snag (or misunderstanding).
The returned obj_session contains no session ID of any kind, just an empty object frame. Also: no file is created at this point in /tmp/cherrypy_sessions -
should not it be there now? I would expect it to be created and named after its session ID.
OK, no ID in the object, let's assign one:
session_id = obj_session.generate_id(self)
This returns a long random string as it should I guess
And now I don't know how to proceed with assignments and saving calling obj_session.save() or obj_session.load() with several variations of input gives "AttributeError: 'module' object has no attribute X" where X can be "load" and couple of other keywords. Passing self or obj_session itself to the methods does not help, just changes the wording of the error. I must be going in a very wrong direction in general.
So, is there an example of those five steps above? I could not find one anywhere.
Thanks.
Igor

SQLAlchemy DetachedInstanceError with regular attribute (not a relation)

I just started using SQLAlchemy and get a DetachedInstanceError and can't find much information on this anywhere. I am using the instance outside a session, so it is natural that SQLAlchemy is unable to load any relations if they are not already loaded, however, the attribute I am accessing is not a relation, in fact this object has no relations at all. I found solutions such as eager loading, but I can't apply to this because this is not a relation. I even tried "touching" this attribute before closing the session, but it still doesn't prevent the exception. What could be causing this exception for a non-relational property even after it has been successfully accessed once before? Any help in debugging this issue is appreciated. I will meanwhile try to get a reproducible stand-alone scenario and update here.
Update: This is the actual exception message with a few stacks:
File "/home/hari/bin/lib/python2.6/site-packages/SQLAlchemy-0.6.1-py2.6.egg/sqlalchemy/orm/attributes.py", line 159, in __get__
return self.impl.get(instance_state(instance), instance_dict(instance))
File "/home/hari/bin/lib/python2.6/site-packages/SQLAlchemy-0.6.1-py2.6.egg/sqlalchemy/orm/attributes.py", line 377, in get
value = callable_(passive=passive)
File "/home/hari/bin/lib/python2.6/site-packages/SQLAlchemy-0.6.1-py2.6.egg/sqlalchemy/orm/state.py", line 280, in __call__
self.manager.deferred_scalar_loader(self, toload)
File "/home/hari/bin/lib/python2.6/site-packages/SQLAlchemy-0.6.1-py2.6.egg/sqlalchemy/orm/mapper.py", line 2323, in _load_scalar_attributes
(state_str(state)))
DetachedInstanceError: Instance <ReportingJob at 0xa41cd8c> is not bound to a Session; attribute refresh operation cannot proceed
The partial model looks like this:
metadata = MetaData()
ModelBase = declarative_base(metadata=metadata)
class ReportingJob(ModelBase):
__tablename__ = 'reporting_job'
job_id = Column(BigInteger, Sequence('job_id_sequence'), primary_key=True)
client_id = Column(BigInteger, nullable=True)
And the field client_id is what is causing this exception with a usage like the below:
Query:
jobs = session \
.query(ReportingJob) \
.filter(ReportingJob.job_id == job_id) \
.all()
if jobs:
# FIXME(Hari): Workaround for the attribute getting lazy-loaded.
jobs[0].client_id
return jobs[0]
This is what triggers the exception later out of the session scope:
msg = msg + ", client_id: %s" % job.client_id

I found the root cause while trying to narrow down the code that caused the exception. I placed the same attribute access code at different places after session close and found that it definitely doesn't cause any issue immediately after the close of query session. It turns out the problem starts appearing after closing a fresh session that is opened to update the object. Once I understood that the state of the object is unusable after a session close, I was able to find this thread that discussed this same issue. Two solutions that come out of the thread are:
Keep a session open (which is obvious)
Specify expire_on_commit=False to sessionmaker().
The 3rd option is to manually set expire_on_commit to False on the session once it is created, something like: session.expire_on_commit = False. I verified that this solves my issue.

We were getting similar errors, even with expire_on_commit set to False. In the end it was actually caused by having two sessionmakers that were both getting used to make sessions in different requests. I don't really understand what was going on, but if you see this exception with expire_on_commit=False, make sure you don't have two sessionmakers initialized.

I had a similar problem with the DetachedInstanceError: Instance <> is not bound to a Session;
The situation was quite simple, I pass the session and the record to be updated to my function and it would merge the record and commit it to the database. In the first sample I would get the error, as I was lazy and thought that I could just return the merged object so my operating record would be updated (ie its is_modified value would be false). It did return the updated record and is_modified was now false but subsequent uses threw the error. I think this was compounded because of related child records but not entirely sure of that.
def EditStaff(self, session, record):
try:
r = session.merge(record)
session.commit()
return r
except:
return False
After much googling and reading about sessions etc, I realized that since I had captured the instance r before the commit and returned it, when that same record was sent back to this function for another edit/commit it had lost its session.
So to fix this I just query the database for the record just updated and return it to keep it in session and mark its is_modified value back to false.
def EditStaff(self, session, record):
try:
session.merge(record)
session.commit()
r = self.GetStaff(session, record)
return r
except:
return False
Setting the expire_on_commit=False also avoided the error as mentioned above, but I don't think it actually addresses the error, and could lead to many other issues IMO.

To throw my cause & solution into the ring, I use flask and flask-sqlalchemy to manage all my session stuff. This is fine when I'm doing things via the site, but when doing things via command line and scripts, you have to ensure that anything that's doing flask-y things has to do it with the flask context.
So, in my situation, I needed to get things from a database (using flask-sqlalchemy), then render them to templates (using flask's render_template), then email them (using flask-mail).
In code, what I'd done was something like,
def render_obj(db_obj):
with app.app_context():
return render_template('template_for_my_db_obj.html', db_obj=db_obj
def get_renders():
my_db_objs = MyDbObj.query.all()
renders = []
for day, _db_objs in itertools.groupby(my_db_objs, MyDbObj.get_date):
renders.extend(list(map(render_obj, _db_obj)))
return renders
def email_report():
renders = get_renders()
report = '\n'.join(renders)
with app.app_context():
mail.send(Message('Subject', ['me#me.com'], html=report))
(this is basically pseudocode, I was doing other things in the grouping section)
And when I was running, I'd get through the first _db_obj, but then I'd get the error on any run after.
The culprit? with app.app_context().
Basically it does a few things when you come out of that context, including kinda freshening up the db connections. One of the things that comes from that is getting rid of the last session that was around, which was the session that all the my_db_objs were associated with.
There's a few different options for solutions, but I went with a variant of,
def render_obj(db_obj):
return render_template('template_for_my_db_obj.html', db_obj=db_obj
def get_renders():
my_db_objs = MyDbObj.query.all()
renders = []
for day, _db_objs in itertools.groupby(my_db_objs, MyDbObj.get_date):
renders.extend(list(map(render_obj, _db_obj)))
return renders
def email_report():
with app.app_context():
renders = get_renders()
report = '\n'.join(renders)
mail.send(Message('Subject', ['me#me.com'], html=report))
Only 1 with app.app_context() which wraps them all. The main thing you need to do (if you've a setup like mine) is ensure any dB object you're using to be "inside" any app_context you're using. If you do what I did in the first iteration, all your dB objects will lose their session, ending in DetachedInstanceError like me.

My solution was a simple oversight;
I created an object, added and ,committed it to the db and after that I tried to access on of the original object attributes without refreshing session session.refresh(object)
user = UserFactory()
session.add(user)
session.commit()
# missing session.refresh(user) and causing the problem
print(user.name)

As for me (newbie), I made a mistake on the indent and close the session inside my loop, in which I loop each row, do some operation and commit each time.
So for those newbie like me, check your code before setting things like expire_on_commit=False, it may lead your to another trap.

My solution to this error was also a simple oversight, which I don't think any of the other answers cover.
My function is fetching object x, modifying it, then returning the original x, because I would like the older version.
Before committing and returning x, I was calling expunge_all, but it was "too late", as the object was already marked dirty.
The solution was simply to expunge the object as early as possible.
# pseudo code
x = session.fetch_x()
# adding the following line fixed it
session.expunge(x)
y = session.update(x)
return x

I have a similar problem in my current project and this fix works for me. Please check in your DB relationship for options lazy=True and change it to lazy='dynamic'.

Python what does it mean "AttributeError: 'unicode' object has no attribute 'has_key' "

I would like to ask what does it mean "AttributeError: 'unicode' object has no attribute 'has_key'"
Here is the full stack trace:
Traceback (most recent call last):
File "D:\Projects\GoogleAppEngine\google_appengine\google\appengine\ext\webapp\__init__.py", line 509, in __call__
handler.post(*groups)
File "D:\Projects\workspace\foo\src\homepage.py", line 71, in post
country=postedcountry
File "D:\Projects\GoogleAppEngine\google_appengine\google\appengine\ext\db\__init__.py", line 656, in __init__
prop.__set__(self, value)
File "D:\Projects\GoogleAppEngine\google_appengine\google\appengine\ext\db\__init__.py", line 2712, in __set__
value = self.validate(value)
File "D:\Projects\GoogleAppEngine\google_appengine\google\appengine\ext\db\__init__.py", line 2742, in validate
if value is not None and not value.has_key():
AttributeError: 'unicode' object has no attribute 'has_key'
Let me describe a little bit more about the situation:
First I created the models.py that has the db.Model for CMSRequest which has an attribute country that reference to the CMSCountry class
class CMSRequest(db.Model):
country = db.ReferenceProperty(CMSCountry, required=True)
class CMSCountry(db.Model):
name = db.StringProperty(required=True)
Then, I created a bulkloader class to import the data into the CMSCountry
In the form, user can select the country from the drop down list box, the results are posted back and save to the CMSRequest object
def post(self):
postedcountry = self.request.get('country')
cmsRequest = models.CMSRequest(postedcountry)
Maybe I have found the solution to my question, it is because I have not converted the posted key of CMSCountry back to save to the CMSRequest.
Thank you everyone!

In this line:
if value is not None and not value.has_key():
value is a unicode string. It looks like the code is expecting it to be a db.Model,
(From what I can see, has_key is a method of db.Model, as well as a method of Python dictionaries, but this must be the db.Model one because it's being called with no arguments.)
Are you passing a string to a GAE API that expects a db.Model?

Your problem is that postedcountry is a string and not a country object. Values retrieved from self.request.get are the string values of variables passed by the browser.
You need to look up a country object using some GQL. Exactly how you do that will depend on what exactly the country field of your HTML form is returning , Object Key?, country name?
def post(self):
postedcountry = self.request.get('country')
# <-------- Need to look up a country here !!!!!
cmsRequest = models.CMSRequest(country=postedcountry)

Note: normally "mapping" types in Python (dictionaries and dictionary like classes ... such as various types of dbm (indexed file) and some DBMS/ORM interfaces ... will implement a has_key() method.
Somehow you have gotten a Unicode (string) object into this statement when you were expecting to have some sort of dictionary or other mapping object reference.
In general AttributeError means that you have tangled up your object bindings (variable assignments). You've given a name to some object other than the type that you intended. (Or sometimes it means you have a typo like ".haskey()" instead of has_key() ... etc).
BTW: the use of has_key() is somewhat dated. Normally it's better to test your containers with the Python in operator (which implicitly calls __contains__() --- and which works on lists, strings, and other sequences as well as mapping types).
Also value.has_key() would raise an error even if value were a dictionary since the .has_key() method requires an argument.
In your code I would either explicitly test for if postedcountry is not None: ... or I'd supply your .get() with an (optional) default for "postedcountry."
How do you want to handle the situation where request had no postedcountry? Do you want to assume it's being posted from some particular default? Force a redirection to some page that requires the user to supply a value for that form element? Alert the Spanish Inquisition?

If you read the traceback, it'll tell you exactly what is going on:
if value is not None and not value.has_key():
AttributeError: 'unicode' object has no attribute 'has_key'
What this says is the the value variable which you're using doesn't have the has_key attribute. And, what it's telling you is that your value variable isn't a dictionary, as it looks like you're expecting...instead, it's a unicode object, which is basically a string.

You're attempting to set a string to a ReferenceProperty. The 'country' field of CMSCountry is a db.ReferenceProperty, which takes a db.Key or a CMSCountry object, but you're attempting to set it to a string.
You should be doing something like this:
def post(self):
postedcountry = self.request.get('country')
country = models.CMSCountry.all().filter('name =', postedcountry)
if not country:
# Couldn't find the country
else:
cmsRequest = models.CMSRequest(country=country)
Please do file a bug about the rather unhelpful error message, though.

The dictionary object's has_key() method is deprecated in 3.0 - use the "in" expression instead. If you are using the old library in 3.x, you must make code changes to accommodate it.