SQLAlchemy - Making a hybrid_property which utilizes multiple relationships queryable - python

I have a Flask-SQLAlchemy model that contains several relationships to tables for holding quantities in decimal, fraction, and integer:
class Packet(db.Model):
# ...
qty_decimal_id = db.Column(db.Integer, db.ForeignKey('qty_decimals.id'))
_qty_decimal = db.relationship('QtyDecimal', backref='_packets')
qty_fraction_id = db.Column(db.Integer, db.ForeignKey('qty_fractions.id'))
_qty_fraction = db.relationship('QtyFraction', backref='_packets')
qty_integer_id = db.Column(db.Integer, db.ForeignKey('qty_integers.id'))
_qty_integer = db.relationship('QtyInteger', backref='_packets')
# ...
The tables each contain a column named 'value' that contains the actual value, such that if I want to store an integer quantity of '100', I would store it in ._qty_integer.value. I have created a hybrid_property that currently gets whichever of these relationships is not null, and sets to a relevant relationship depending on what kind of data is detected by the setter:
#hybrid_property
def quantity(self):
retqty = None
for qty in [self._qty_decimal, self._qty_fraction, self._qty_integer]:
if qty is not None:
if retqty is None:
retqty = qty
else:
raise RuntimeError('More than one type of quantity'
' was detected for this packet. Only'
' one type of quantity may be set!')
return retqty.value
#quantity.setter
def quantity(self, value):
if is_decimal(value):
self.clear_quantity()
self._qty_decimal = QtyDecimal.query.filter_by(value=value).\
first() or QtyDecimal(value)
elif is_fraction(value):
if is_480th(value):
self.clear_quantity()
self._qty_fraction = QtyFraction.query.filter_by(value=value).\
first() or QtyFraction(value)
else:
raise ValueError('Fractions must have denominators'
' 480 is divisible by!')
elif is_int(value):
self.clear_quantity()
self._qty_integer = QtyInteger.query.filter_by(value=value).\
first() or QtyInteger(value)
else:
raise ValueError('Could not determine appropriate type for '
'quantity! Please make sure it is a integer, '
'decimal number, or fraction.')
The getter and setter for .quantity work perfectly as far as I can tell (it's always possible I missed some edge cases in my test suite) but I cannot, for the life of me, figure out how to implement a comparator or expression such that one could, for example, get all packets that have a quantity of 100, or all packets with a quantity of 1/4. (Units are implemented too, but irrelevant to this question.) As far as I can tell it's not doable as an expression, but it should be doable as a comparator. I can't figure out how to pull it off, though. So far this is my best attempt at a comparator:
class QuantityComparator(Comparator):
def __init__(self, qty_decimal, qty_fraction, qty_integer):
self.qty_decimal = qty_decimal
self.qty_fraction = qty_fraction
self.qty_integer = qty_integer
def __eq__(self, other):
if is_decimal(other):
return self.qty_decimal.value == other
elif is_fraction(other):
if is_480th(other):
return self.qty_fraction.value == other
else:
raise ValueError('Cannot query using a fraction with a '
'denominator 480 is not divisible by!')
elif is_int(other):
return self.qty_fraction.value == other
else:
raise ValueError('Could not parse query value'
' as a valid quantity!')
#quantity.comparator
def quantity(self):
return self.QuantityComparator(self._qty_decimal,
self._qty_fraction,
self._qty_integer)
Unsurprisingly, this does not work, when I try to run Packet.query.filter_by(quantity=) it raises an exception:
AttributeError: Neither 'InstrumentedAttribute' object nor 'Comparator' object associated with Packet._qty_fraction has an attribute 'value'
If the solution to getting this use case to work is in the SQLAlchemy docs, I've either missed it or (more likely) haven't wrapped my head around enough of SQLAlchemy yet to figure it out.
I have come up with a stopgap solution that at least lets users get Packets based on quantity:
#staticmethod
def quantity_equals(value):
if is_decimal(value):
return Packet.query.join(Packet._qty_decimal).\
filter(QtyDecimal.value == value)
elif is_fraction(value):
if is_480th(value):
return Packet.query.join(Packet._qty_fraction).\
filter(QtyFraction.value == value)
else:
raise ValueError('Fraction could not be converted to 480ths!')
elif is_int(value):
return Packet.query.join(Packet._qty_integer).\
filter(QtyInteger.value == value)
This works and gets me what looks to be the correct Query object, as shown by this test:
def test_quantity_equals(self):
pkt1 = Packet()
pkt2 = Packet()
pkt3 = Packet()
db.session.add_all([pkt1, pkt2, pkt3])
pkt1.quantity = Decimal('3.14')
pkt2.quantity = Fraction(1, 4)
pkt3.quantity = 100
db.session.commit()
qty_dec_query = Packet.quantity_equals(Decimal('3.14'))
self.assertIs(pkt1, qty_dec_query.first())
self.assertEqual(qty_dec_query.count(), 1)
qty_frac_query = Packet.quantity_equals(Fraction(1, 4))
self.assertIs(pkt2, qty_frac_query.first())
self.assertEqual(qty_frac_query.count(), 1)
qty_int_query = Packet.quantity_equals(100)
self.assertIs(pkt3, qty_int_query.first())
self.assertEqual(qty_int_query.count(), 1)
I can easily make similar dirty methods to substitute for other comparison operators, but I would think it's possible to do it in a custom comparator such as the aforementioned QuantityComparator, or to otherwise achieve the desired ability to use the .quantity property in query filters.
Can anybody help me get the QuantityComparator working, or point me in the right direction for figuring it out myself?
Edit: Solution
While I haven't actually solved the explicit question of making a hybrid_property with multiple relationships queryable, I have solved the core issue of representing a quantity that could either be int, float (I use float instead of Decimal here because no arithmetic is done the quantity) or Fraction in the database, and making it possible to use values in queries as they would be used by the value setter. The solution was to create a quantities table with rows for a floating point value, an integer numerator, an integer denominator, and a boolean representing whether or not the stored value should be interpreted as a decimal number (as opposed to a fraction) and create an sqlalchemy.ext.hybrid.Comparator for the hybrid property which compares against Quantity._float and Quantity.is_decimal:
class Quantity(db.Model):
# ...
_denominator = db.Column(db.Integer)
_float = db.Column(db.Float)
is_decimal = db.Column(db.Boolean, default=False)
_numerator = db.Column(db.Integer)
# ...
#staticmethod
def for_cmp(val):
"""Convert val to float so it can be used to query against _float."""
if Quantity.dec_check(val): # True if val looks like a decimal number
return float(val)
elif isinstance(val, str):
frac = Quantity.str_to_fraction(val)
else:
frac = Fraction(val)
return float(frac)
#hybrid_property
def value(self):
if self._float is not None:
if self.is_decimal:
return self._float
elif self._denominator == 1:
return self._numerator
else:
return Fraction(self._numerator, self._denominator)
else:
return None
class ValueComparator(Comparator):
def operate(self, op, other):
return and_(op(Quantity._float, Quantity.for_cmp(other)),
Quantity.is_decimal == Quantity.dec_check(other))
#value.comparator
def value(cls):
return Quantity.ValueComparator(cls)
#value.setter
def value(self, val):
if val is not None:
if Quantity.dec_check(val):
self.is_decimal = True
self._float = float(val)
self._numerator = None
self._denominator = None
else:
self.is_decimal = False
if isinstance(val, str):
frac = Quantity.str_to_fraction(val)
else:
frac = Fraction(val)
self._numerator = frac.numerator
self._denominator = frac.denominator
self._float = float(frac)
else:
self.is_decimal = None
self._numerator = None
self._denominator = None
self._float = None
Since I don't have the original model I asked this question for any longer, I can't readily go back and answer this question properly, but I'd imagine it could be done using join or select.

Related

how to fix the _getitem__ method

def pnamedtuple(type_name, field_names, mutable=False):
pass
class type_name:
def __init__(self, x, y):
self.x = x
self.y = y
self._fields = ['x','y']
self._mutable = False
def get_x(self):
return self.x
def get_y(self):
return self.y
def __getitem__(self,i):
if i > 1 or i <0:
raise IndexError
if i == 0 or i == 'x':
return self.get_x():
if i == 1 or i == 'y':
return self.get_y():
the getitem method to overload the [] (indexing operator) for this class: an index of 0 returns the value of the first field name in the field_names list; an index of 1 returns the value of the second field name in the field_names list, etc. Also, the index can be a string with the named field. So, for p = Point(1,2) writing p.get_x(), or p[0]), or p['x'] returns a result of 1. Raise an IndexError with an appropriate message if the index is out of bounds int or a string that does not name a field.
I am not sure how to fix the getitme function. below is the bsc.txt
c-->t1 = Triple1(1,2,3)
c-->t2 = Triple2(1,2,3)
c-->t3 = Triple3(1,2,3)
# Test __getitem__ functions
e-->t1[0]-->1
e-->t1[1]-->2
e-->t1[2]-->3
e-->t1['a']-->1
e-->t1['b']-->2
e-->t1['c']-->3
^-->t1[4]-->IndexError
^-->t1['d']-->IndexError
^-->t1[3.2]-->IndexError
can someone tell how to fix my _getitem _ function to get the output in bsc.txt? many thanks.
You've spelled __getitem__ incorrectly. Magic methods require two __ underscores before and after them.
So you haven't overloaded the original __getitem__ method, you've simply created a new method named _getitem_.
Python 3 does not allow strings and integers to be compared with > or <; it's best to stick with == if you don't yet know the type of i. You could use isinstance, but here you can easily convert the only two valid integer values to strings (or vice versa), then work only on strings.
def __getitem__(self, i):
if i == 0:
i = "x"
elif i == 1:
i = "y"
if i == "x":
return self.get_x()
elif i == "y":
return self.get_y()
else:
raise IndexError("Invalid key: {}".format(i))
your function is interesting, but there are some issues with it:
In python 3 you can't compare string with numbers, so you first should check with == against know values and or types. For example
def __getitem__(self,i):
if i in {0,"x"}:
return self.x
elif i in {1,"y"}:
return self.y
else:
raise IndexError(repr(i))
But defined like that (in your code or in the example above) for an instance t1 this t1[X] for all string X others than "x" or "y" will always fail as you don't adjust it for any other value. And that is because
pnamedtuple looks like you want for it to be a factory like collections.namedtuple, but it fail to be general enough because you don't use any the arguments of your function at all. And no, type_name is not used either, whatever value it have is throw away when you make the class declaration.
how to fix it?
You need other ways to store the value of the fields and its respective name, for example a dictionary lets call it self._data
To remember how you called yours field, use the argument of your function, for instance self._fields = field_names
To accept a unknown number of arguments use * like __init__(self, *values) then verify that you have the same numbers of values and fields and build your data structure of point 1 (the dictionary)
Once that those are ready then __getitem__ become something like:
def __getitem__(self, key):
if key in self._data:
return self._data[key]
elif isintance(key,int) and 0 <= key < len(self._fields):
return self._data[ self._fields[key] ]
else:
raise IndexError( repr(key) )
or you can simple inherit from a appropriate namedtuple and the only thing you need to do is overwrite its __getitem__ like
def __getitem__(self,key):
if key in self._fields:
return getattr(self,key)
return super().__getitem__(key)

How to create a Tree-based Map with keys

For my intro to computer science class we have a tree based map problem. I'm getting really confused on how to make the tree in the fashion they are asking it.
What I have so far:
class EmptyMap():
__slots__ = ()
class NonEmptyMap():
__slots__ = ('left','key','value','right')
def mkEmptyMap():
m = EmptyMap()
return m
def mkNonEmptyMap(map1, key, value, map2):
m = NonEmptyMap()
m.left = map1
m.key = key
m.value = value
m.right = map2
return m
def mapInsert(key, value, map1):
if isinstance(map1, EmptyMap):
else:
I'm getting stuck on the mapInsert function which is supposed to be recursive. Our tutoring lab doesnt have any tutors in it now so any help is appreciated.
Link to homework file http://www.cs.rit.edu/~vcss241/Homeworks/08/TreeMap-stu.pdf
Thanks!
I have never written or seen Python, but try this:
def mapInsert(key, value, map1):
if isinstance(map1, EmptyMap):
return mkNonEmptyMap(mkEmptyMap(), key, value, mkEmptyMap())
else:
if map1.key == key:
map1.value = value;
else if map1.key > key:
return map1.left = mapInsert(key, value, map1.left)
else:
return map1.right = mapInsert(key, value, map1.right)

Elegant way to avoid .put() on unchanged entities

A reoccurring pattern in my Python programming on GAE is getting some entity from the data store, then possibly changing that entity based on various conditions. In the end I need to .put() the entity back to the data store to ensure that any changes that might have been made to it get saved.
However often there were no changes actually made and the final .put() is just a waste of money. How to easily make sure that I only put an entity if it has really changed?
The code might look something like
def handle_get_request():
entity = Entity.get_by_key_name("foobar")
if phase_of_moon() == "full":
entity.werewolf = True
if random.choice([True, False]):
entity.lucky = True
if some_complicated_condition:
entity.answer = 42
entity.put()
I could maintain a "changed" flag which I set if any condition changed the entity, but that seems very brittle. If I forget to set it somewhere, then changes would be lost.
What I ended up using
def handle_get_request():
entity = Entity.get_by_key_name("foobar")
original_xml = entity.to_xml()
if phase_of_moon() == "full":
entity.werewolf = True
if random.choice([True, False]):
entity.lucky = True
if some_complicated_condition:
entity.answer = 42
if entity.to_xml() != original_xml: entity.put()
I would not call this "elegant". Elegant would be if the object just saved itself automatically in the end, but I felt this was simple and readable enough to do for now.
Why not check if the result equals (==) the original and so decide whether to save it. This depends on a correctly implemented __eq__, but by default a field-by-field comparison based on the __dict__ should do it.
def __eq__(self, other) :
return self.__dict__ == other.__dict__
(Be sure that the other rich comparison and hash operators work correctly if you do this. See here.)
One possible solution is using a wrapper that tracks any attribute change:
class Wrapper(object):
def __init__(self, x):
self._x = x
self._changed = False
def __setattr__(self, name, value):
if name[:1] == "_":
object.__setattr__(self, name, value)
else:
if getattr(self._x, name) != value:
setattr(self._x, name, value)
self._changed = True
def __getattribute__(self, name):
if name[:1] == "_":
return object.__getattribute__(self, name)
return getattr(self._x, name)
class Contact:
def __init__(self, name, address):
self.name = name
self.address = address
c = Contact("Me", "Here")
w = Wrapper(c)
print w.name # --> Me
w.name = w.name
print w.name, w._changed # --> Me False
w.name = "6502"
print w.name, w._changed # --> 6502 True
This answer is a part of an question i posted about a Python checksum of a dict
With the answers of this question I developed a method to generate checksum from
a db.Model.
This is an example:
>>> class Actor(db.Model):
... name = db.StringProperty()
... age = db.IntegerProperty()
...
>>> u = Actor(name="John Doe", age=26)
>>> util.checksum_from_model(u, Actor)
'-42156217'
>>> u.age = 47
>>> checksum_from_model(u, Actor)
'-63393076'
I defined these methods:
def checksum_from_model(ref, model, exclude_keys=[], exclude_properties=[]):
"""Returns the checksum of a db.Model.
Attributes:
ref: The reference og the db.Model
model: The model type instance of db.Model.
exclude_keys: To exclude a list of properties name like 'updated'
exclude_properties: To exclude list of properties type like 'db.DateTimeProperty'
Returns:
A checksum in signed integer.
"""
l = []
for key, prop in model.properties().iteritems():
if not (key in exclude_keys) and \
not any([True for x in exclude_properties if isinstance(prop, x)]):
l.append(getattr(ref, key))
return checksum_from_list(l)
def checksum_from_list(l):
"""Returns a checksum from a list of data into an int."""
return reduce(lambda x,y : x^y, [hash(repr(x)) for x in l])
Note:
For the base36 implementation: http://en.wikipedia.org/wiki/Base_36#Python_implementation
Edit:
I removed the return in base36, now these functions run without dependences. (An advice from #Skirmantas)
Didn't work with GAE but in same situation I'd use something like:
entity = Entity.get_by_key_name("foobar")
prev_entity_state = deepcopy(entity.__dict__)
if phase_of_moon() == "full":
entity.werewolf = True
if random.choice([True, False]):
entity.lucky = True
if some_complicated_condition:
entity.answer = 42
if entity.__dict__ == prev_entity_state:
entity.put()

Subclass/Child class

I had this class and subclass :
class Range:
def __init__(self, start, end):
self.setStart(start)
self.setEnd(end)
def getStart(self):
return self.start
def setStart(self, s):
self.start = s
def getEnd(self):
return self.end
def setEnd(self, e):
self.end = e
def getLength(self):
return len(range(self.start, self.end))
def overlaps(self, r):
if (r.getStart() < self.getEnd() and r.getEnd() >= self.getEnd()) or \
(self.getStart() < r.getEnd() and self.getEnd() >= r.getEnd()) or \
(self.getStart() >= r.getStart() and self.getEnd() <= r.getEnd()) or \
(r.getStart() >= self.getStart() and r.getEnd() <= self.getEnd()):
return True
else:
return False
class DNAFeature(Range):
def __init__(self, start, end):
self.setStart(start)
self.setEnd(end)
self.strand = none
self.sequencename = none
def getSeqName(self, s):
return self.SeqName
def setSeqName(self, s):
self.sequencename = s
def getStrand(self):
if self.SeqName == 'plus':
return 1
elif self.SeqName == 'minus':
return -1
else:
return 0
def setStrand(self, s):
self.strand = s
And here is what I have to do:
Create
 a 
new 
class
– 
GeneModel
‐
 that 
contains 
a 
group
 of 
DNAFeature 
objects
 representing
 exons
 and
 is 
a 
child 
class 
of 
DNAFeature. 
It
 should 
implement 
the
 following 
methods:


getFeats() 
–
returns 
a 
list
 of
 DNAFeature 
objects,
sorted
 by
 start 
position
addFeat(feat)
–
 accepts 
a 
DNAFeature 
feat 
and
 adds 
it 
to 
its 
internal 
group
 of
 DNAFeature 
objects
setTranslStart(i)
– 
accepts 
a 
non‐negative 
int,
sets 
the
 start
 position 
of
 the
 initiating 
ATG
 codon
getTranslStart()
–
returns 
an 
int, 
the
 start 
position 
of 
the
 initiating 
ATG
 codon
setTranslStop(i)
– 
accepts
 a 
positive 
int,
sets
 the
 end
 position 
for 
the
 stop
 codon
getTranslStop()
–
 returns
 an 
int,
the
 end 
position 
for
 the
 stop 
codon
setDisplayId(s) 
–
sets 
the
 name
 of
 the
 gene
 model; 
s 
is 
a 
string
getDisplayId()
– 
return
 the
 name 
of 
the 
gene
 model,
 returns 
a 
string,
e.g.,
AT1G10555.1
 
 GeneModel
 should 
raise
 appropriate 
ValueError 
and 
TypeError 
exceptions 
when
 users
 pass
 incorrect 
types 
and
 values 
to 
constructors 
and 
“set” 
methods.
I have tried to write whatever comes to my mind, and read the books as well as searching the way to put codes together, but I am so new to programming and hardly can understand how to write the codes correctly. To be honest, this is the first time I ever do a programming class. So if I make any funny mistake in my codes, please forgive me. I haven't finish my codes yet and still reading the books to see where I am doing wrong and right with my codes. However, I really need your help to guide me to the right path. Thank you guys very much. Below is my codes:
class GeneModel(DNAFeature):
def __init__(self, translstart, translend, displayid):
self.setTranslStart(translstart)
self.setTranslStop(translend)
setDisplayId(displayid)
def getFeats():
result = []
sort.self.getStart()
return result
def addFeat(feat):
self.addFeat = feat
return self.getStart+self.getEnd
def setTranslStart(i):
self.translstart = self.setStart
self.translstart = non-negative int
def getTranslStart():
return self.translstart
def setTranslStop(i):
self.translend = self.setEnd
self.translend = "+" int
def getTranslStop():
return self.translend
def setDisplayId(s):
self.displayid = re.compile('r'\AT1G[0-9]{5,5}\.[0-9]{,1}, IGNORECASE')
def getDisplayId():
return self.displayid
I don't understand what the name of the gene model is. I think it's subject specific, but I think this will work for you:
class GenoModel(DNAFeature):
def __init__(self, start, end):
self.setStart(start)
self.setEnd(end)
self.strand = None
self.sequencename = None
self.exons = []
self.translStart = None
self.translStop = None
self.displayId = None
def getFeats(self):
self.exons.sort(cmp=self.start)
return self.exons
def addFeat(self, f):
if type(f) == DNAFeature:
self.exons.append(f)
else:
raise TypeError("Cannot add feature as it is not of type DNAFeature")
def setTranslStart(self, i):
if type(i) != int:
raise TypeError("Cannot set translStart as it is not of type int")
elif i < 0:
raise ValueError("Cannot set tanslStart to a negative int")
else:
self.translStart = i
def getTranslStart(self):
return self.translStart
def setTranslStop(self, i):
if type(i) != int:
raise TypeError("Cannot set translStop as it is not of type int")
elif i <= 0:
raise ValueError("Cannot set tanslStop to anything less than 1")
else:
self.translStop = i
def getTranslStop(self):
return self.translStop
def setDisplayId(self, s):
if type(s) != str:
raise TypeError("Cannot set desiplayId as it is not of type string")
else:
self.displayId = s
def getDisplayId(self):
return self.displayId
Hope this helps.
First, a little bit of cleanup. I'm not completely convinced that your original class, DNAFeature, is actually correct. DNAFeature seems to be inheriting from some other class, named Range, that we're missing here so if you have that code please offer it as well. In that original class, you need to define the variable SeqName (also, its preferable to keep variables lower-cased) since otherwise self.SeqName will be meaningless. Additionally, unless they're inherited from the Range class, you should also define the methods "setStart" and "setEnd". You're getter should not any additional variables, so feel free to change it to "def getSeqName(self)" instead of adding "s". I'm not sure what else your code is really supposed to do, so I'll hold any further comment.
Additionally, though you stated otherwise in your comment, I have to believe from the naming conventions (and what little I remember from bio) that you actually want GeneModel to be a container for a set of DNAFeature instances. That's different from GeneModel subclassing DNAFeature. If I'm right, then you can try:
class GeneModel(object):
def __init__(dnafeatures):
self.dnafeatures = dnafeatures
def get_features(self):
return self.dnafeatures
def add_feature(self, feature):
self.dnafeatures.append(feature)
Here dnafeatures would just be a list of dnafeature instances. This would then allow you to write methods to access these features and do whatever fun stuff you need to do.
My advice would be to make sure your DNAFeature class is correct and that your model of how you want your problem solved (in terms of what your classes do) and try asking again when its a little clearer. Hope this helps!

python list mysteriously getting set to something within my django/piston handler

Note: (I've updated this since the first two suggestions... you can view the old post in txt form here: http://bennyland.com/old-2554127.txt). The update I made was to better understand what was going wrong - and now I at least sort of know what's happening but I have no clue how to fix it.
Anyway, using Django and Piston, I've set up a new BaseHandler class named BaseApiHandler which does most of the work I was doing across all of my handlers. This worked great until I added the ability to limit the filters being applied to my results (for instance 'give me the first result only').
Examples (had to remove ":" because i can't submit more urls):
- http//localhost/api/hours_detail/empid/22 gives me all hours_detail rows from employee # 22
- http//localhost/api/hours_detail/empid/22/limit/first gives me the first hours_detail row from employee # 22
What's happening is that when I run /limit/first several times in succession, the first example is then broken, pretending it's a /limit/ url when it isn't.
Right now I'm storing whether or not it's a limit and what the limit is in a new class - prior to this stackoverflow edit, I was just using a list with two entries (limit = [] when initialized, limit = [0,1] when set). Prior to this stackoverflow edit, once you spammed /limit/first, when going to the first example 'limit' would be pre-set to [0,1] and the handler would then limit the query because of this. With the debug data I've added, I can say for certain that the list was pre-set and not getting set during the execution of the code.
I'm adding debug info into my response so I can see what's happening. Right now when you first ask for Example 1's url, you get this CORRECT statusmsg response:
"statusmsg": "2 hours_detail found with query: {'empid':'22','datestamp':'2009-03-02',}",
When you ask for Example 2's url, you get this CORRECT statusmsg response:
"statusmsg": "1 hours_detail found with query: {'empid':'22','datestamp':'2009-03-02','limit','first',with limit[0,1](limit,None... limit set 1 times),}",
However, if you refresh a bunch of times, the limit set value starts increasing (incrementing this value was something a friend of mine suggested to see if this variable was somehow getting kept around)
"statusmsg": "1 hours_detail found with query: {'empid':'22','datestamp':'2009-03-02','limit','first',with limit[0,1](limit,None... limit set 10 times),}",
Once that number goes above '1 times', you can start trying to get Example 1's url. Each time I now refresh example 1, i get odd results. Here are 3 different status messages from different refreshes (Notice that from each one, 'limit':'first' is CORRECTLY missing from the kwarg's debug output while the actual value of islimit is hovering between 8 and 10):
"statusmsg": "1 hours_detail found with query: {'empid':'22','datestamp':'2009-03-02',with limit[0,1](limit,None... limit set 10 times),}",
"statusmsg": "1 hours_detail found with query: {'empid':'22','datestamp':'2009-03-02',with limit[0,1](limit,None... limit set 8 times),}",
"statusmsg": "1 hours_detail found with query: {'empid':'22','datestamp':'2009-03-02',with limit[0,1](limit,None... limit set 9 times),}",
So it would appear that this object is getting cached. Prior to changing 'limit' form a list to a class, it also appeared that the list version of 'limit' was getting cached as after going to Example 2's url, i would sometimes have [0,1] as the limit.
Here are the updated snippets of the code (remember, you can view the first post here: bennyland.com/old-2554127.txt)
URLS.PY - inside 'urlpatterns = patterns('
#hours_detail/id/{id}/empid/{empid}/projid/{projid}/datestamp/{datestamp}/daterange/{fromdate}to{todate}
#empid is required
url(r'^api/hours_detail/(?:' + \
r'(?:[/]?id/(?P<id>\d+))?' + \
r'(?:[/]?empid/(?P<empid>\d+))?' + \
r'(?:[/]?projid/(?P<projid>\d+))?' + \
r'(?:[/]?datestamp/(?P<datestamp>\d{4,}[-/\.]\d{2,}[-/\.]\d{2,}))?' + \
r'(?:[/]?daterange/(?P<daterange>(?:\d{4,}[-/\.]\d{2,}[-/\.]\d{2,})(?:to|/-)(?:\d{4,}[-/\.]\d{2,}[-/\.]\d{2,})))?' + \
r')+' + \
r'(?:/limit/(?P<limit>(?:first|last)))?' + \
r'(?:/(?P<exact>exact))?$', hours_detail_resource),
HANDLERS.PY
class ResponseLimit(object):
def __init__(self):
self._min = 0
self._max = 0
self._islimit = 0
#property
def min(self):
if self.islimit == 0:
raise LookupError("trying to access min when no limit has been set")
return self._min
#property
def max(self):
if self.islimit == 0:
raise LookupError("trying to access max when no limit has been set")
return self._max
#property
def islimit(self):
return self._islimit
def setlimit(self, min, max):
self._min = min
self._max = max
# incrementing _islimit instead of using a bool so I can try and see why it's broken
self._islimit += 1
class BaseApiHandler(BaseHandler):
limit = ResponseLimit()
def __init__(self):
self._post_name = 'base'
#property
def post_name(self):
return self._post_name
#post_name.setter
def post_name(self, value):
self._post_name = value
def process_kwarg_read(self, key, value, d_post, b_exact):
"""
this should be overridden in the derived classes to process kwargs
"""
pass
# override 'read' so we can better handle our api's searching capabilities
def read(self, request, *args, **kwargs):
d_post = {'status':0,'statusmsg':'Nothing Happened'}
try:
# setup the named response object
# select all employees then filter - querysets are lazy in django
# the actual query is only done once data is needed, so this may
# seem like some memory hog slow beast, but it's actually not.
d_post[self.post_name] = self.queryset(request)
s_query = ''
b_exact = False
if 'exact' in kwargs and kwargs['exact'] <> None:
b_exact = True
s_query = '\'exact\':True,'
for key,value in kwargs.iteritems():
# the regex url possibilities will push None into the kwargs dictionary
# if not specified, so just continue looping through if that's the case
if value is None or key == 'exact':
continue
# write to the s_query string so we have a nice error message
s_query = '%s\'%s\':\'%s\',' % (s_query, key, value)
# now process this key/value kwarg
self.process_kwarg_read(key=key, value=value, d_post=d_post, b_exact=b_exact)
# end of the kwargs for loop
else:
if self.limit.islimit > 0:
s_query = '%swith limit[%s,%s](limit,%s... limit set %s times),' % (s_query, self.limit.min, self.limit.max, kwargs['limit'],self.limit.islimit)
d_post[self.post_name] = d_post[self.post_name][self.limit.min:self.limit.max]
if d_post[self.post_name].count() == 0:
d_post['status'] = 0
d_post['statusmsg'] = '%s not found with query: {%s}' % (self.post_name, s_query)
else:
d_post['status'] = 1
d_post['statusmsg'] = '%s %s found with query: {%s}' % (d_post[self.post_name].count(), self.post_name, s_query)
except:
e = sys.exc_info()[1]
d_post['status'] = 0
d_post['statusmsg'] = 'error: %s %s' % (e, traceback.format_exc())
d_post[self.post_name] = []
return d_post
class HoursDetailHandler(BaseApiHandler):
#allowed_methods = ('GET', 'PUT', 'POST', 'DELETE',)
model = HoursDetail
exclude = ()
def __init__(self):
BaseApiHandler.__init__(self)
self._post_name = 'hours_detail'
def process_kwarg_read(self, key, value, d_post, b_exact):
# each query is handled slightly differently... when keys are added
# handle them in here. python doesn't have switch statements, this
# could theoretically be performed using a dictionary with lambda
# expressions, however I was affraid it would mess with the way the
# filters on the queryset work so I went for the less exciting
# if/elif block instead
# querying on a specific row
if key == 'id':
d_post[self.post_name] = d_post[self.post_name].filter(pk=value)
# filter based on employee id - this is guaranteed to happen once
# per query (see read(...))
elif key == 'empid':
d_post[self.post_name] = d_post[self.post_name].filter(emp__id__exact=value)
# look for a specific project by id
elif key == 'projid':
d_post[self.post_name] = d_post[self.post_name].filter(proj__id__exact=value)
elif key == 'datestamp' or key == 'daterange':
d_from = None
d_to = None
# first, regex out the times in the case of range vs stamp
if key == 'daterange':
m = re.match('(?P<daterangefrom>\d{4,}[-/\.]\d{2,}[-/\.]\d{2,})(?:to|/-)(?P<daterangeto>\d{4,}[-/\.]\d{2,}[-/\.]\d{2,})', \
value)
d_from = datetime.strptime(m.group('daterangefrom'), '%Y-%m-%d')
d_to = datetime.strptime(m.group('daterangeto'), '%Y-%m-%d')
else:
d_from = datetime.strptime(value, '%Y-%m-%d')
d_to = datetime.strptime(value, '%Y-%m-%d')
# now min/max to get midnight on day1 through just before midnight on day2
# note: this is a hack because as of the writing of this app,
# __date doesn't yet exist as a queryable field thus any
# timestamps not at midnight were incorrectly left out
d_from = datetime.combine(d_from, time.min)
d_to = datetime.combine(d_to, time.max)
d_post[self.post_name] = d_post[self.post_name].filter(clock_time__gte=d_from)
d_post[self.post_name] = d_post[self.post_name].filter(clock_time__lte=d_to)
elif key == 'limit':
order_by = 'clock_time'
if value == 'last':
order_by = '-clock_time'
d_post[self.post_name] = d_post[self.post_name].order_by(order_by)
self.limit.setlimit(0, 1)
else:
raise NameError
def read(self, request, *args, **kwargs):
# empid is required, so make sure it exists before running BaseApiHandler's read method
if not('empid' in kwargs and kwargs['empid'] <> None and kwargs['empid'] >= 0):
return {'status':0,'statusmsg':'empid cannot be empty'}
else:
return BaseApiHandler.read(self, request, *args, **kwargs)
I would say that there is a basic flaw in your code, if has_limit() can return True when limit is a list of length 2, but this line will fail if limit is shorter than 3 elements long:
s_query = '%swith limit[%s,%s](limit,%s > traceback:%s),' %
(s_query, self.limit[0], self.limit[1], kwargs['limit'],
self.limit[2])
Why are you initializing self.limit to an invalid length list? You could also make this code a little more defensive:
if self.has_limit():
s_query += 'with limit[%s,%s]' % self.limit[0:1]
if 'limit' in kwargs and len(self.limit) > 2:
s_query += '(limit,%s > traceback:%s),' %
(kwargs['limit'], self.limit[2])
I think you may be creating an alias to your internal limit list, via the get_limit property accessor. Try removing (or at least adding a print statement) inside this accessor. If you have code externally that binds a local list to get_limit, then it can update the contents using append, del, or assignment to slices, such as [:]. Or try this:
def get_limit(self):
return self._limit[:]
Instead of binding your internal list to an external name, it will make a copy of your internal list.

Categories