We Keep Coding

Django queryset interpolation for a previous / next instance function

Writing a dry function that returns either previous or next instances of a given instance.
This function return previous instances:
def previous(instance):
try:
return Picture.objects.filter(id__lt=instance.id).first()
except Picture.DoesNotExist:
return instance
I want to create an abstracted function which returns either the previous or the next instance using an additional gt_or_lt argument. The problem lies in interpolating that argument into the filter(id__gt_or_lt).
def seek_instance(gt_or_lt, instance):
try:
return Picture.objects.filter(id__gt_or_lt=instance.id).first()
except Picture.DoesNotExist:
return instance
I've tried:
return Picture.objects.filter(id__gt_or_lt = instance.id).first()
seek_instance("gt", instance)
return Picture.objects.filter(id__f"{gt_or_lt}" = instance.id).first()
seek_instance("gt", instance)
return Picture.objects.filter(f"{gt_or_lt}" = instance.id).first()
return Picture.objects.filter(gt_or_lt = instance.id).first()
seek("id__gt", instance)
All fail with their respective errors.

Use a dictionary with kwargs expansion.
return Picture.objects.filter(**{f"id__{gt_or_lt}": instance.id})

You can use dictionary expansion, like #DanielRoseman suggests. But that will still not per se render the previous, or next item. If for example the model has an ordering option [Django-doc], then it is possible that the order is different than on the id. Furthermore, for the previous one, you will need to reverse the ordering.
Furthermore depending on the situation, you might want to prevent that seek_instance can be given a different lookup, like 'in' for example.
We can thus use an if … elif … else here to branch on the item we wish to retrieve, and raise a ValueError in case you use some other lookup:
def seek_instance(lt_or_gt, instance):
try:
if lt_or_gt == 'lt':
return Picture.objects.filter(pk__lt=instance.pk).order_by('-pk').first()
elif lt_or_gt == 'gt':
return Picture.objects.filter(pk__gt=instance.pk).order_by('pk').first()
else:
raise ValueError("Should be 'lt' or 'gt'")
except Picture.DoesNotExist:
return instance

What is the "pythonic" way to create a list with one element or just keep it empty?

Let's say a have Person instance person. person may have an attribute id.
I want to write a function that gets me either an empty list [] when the attribute is None or missing or the list [12, ] when the attribute id is 12.
def get_id_list(person):
try:
return [getattr(person, 'id'), ]
except AttributeError:
return []
This works perfectly but is there a "pythonic" way of doing this without the try-except block and maybe in one line?

I would go for
def get_id_list(person):
_id = getattr(person, 'id', None)
return [] if _id is None else [_id]
However, it is good practice to make sure that attributes are always defined so you don't have to use getattr with a default or use hasattr to check for existence.

You may use hasattr() to check for attribute as:
def get_id_list(person):
if hasattr(person, 'id'):
return [person.id] # better then getattr(person, 'id')
else:
return []
which could be written in one line as:
def get_id_list(person):
return [person.id] if hasattr(person, 'id') else []

There are two basic ways of going about this:
EAFP (It's Easier to Ask Forgiveness than Permission)- which is what you have:
def get_id_list(person):
try:
return [person.id]
except AttributeError:
return []
LBYL (Look Before You Leap)
def get_id_list(person):
if hasattr(person, 'id'):
return [person.id]
else:
return []
Generally, EAFP is "pythonic", but really, it depends on the specifics of your use-case. If you know that person will usually have the id attribute, then you should use EAFP. That is to say, if it is an exceptional case for person not to have id, use exceptions! If, on the other hand, it is common for person to lack id, then LBYL will be more efficient.
If you simply want a one-liner, you could use a conditional expression:
return [person.id] if hasattr(person,'id') else []

Your approach is pythonic, just do some few tweaking.
First off, don't return inside the try block. You can preserve the id value in a variable and return it properly within else clause. The reason is that you are doing multiple operation like getting attribute and converting to list (and maybe more(in other cases)) while you are just catching the AttributeError. And when you use else you can handle other exceptions easily, also it's more elegant from coding style's perspective.
Secondly you don't need to use getattr in this case, you can simply use direct attribute access.
Also instead of using the try-except you might want to check if the object has the attribute then return its value but due to It's easier to ask forgiveness than it is to get permission. principal the try-exepct is better.
def get_id_list(person):
try:
id = person.id
except AttributeError:
return []
else:
return [id]

getattr() allows you to specify a default if the attribute isn't there so you could use that if you want to avoid a try/catch block.
If the named attribute does not exist, default is returned if provided, otherwise AttributeError is raised.
id = getattr(person, 'id', None)
if id is not None:
return [id]
return []

You can provide a default value (a third argument) to getattr() function to be returned if an attribute does not exist:
def get_id_list(person):
id_ = getattr(person, 'id', None)
return [id_] if id_ is not None else []
(assuming here that None is not a valid value for the id attribute)

according to the Python documentation:
EAFP:
Easier to ask for forgiveness than permission. This common Python
coding style assumes the existence of valid keys or attributes and
catches exceptions if the assumption proves false. This clean and fast
style is characterized by the presence of many try and except
statements. The technique contrasts with the LBYL style common to many
other languages such as C.
That means that your code follows the "official" pythonic way, checking if the attribute exists would be less pythonic.
However, performance needs could eventually override pythonic consideration according to the frequence where person object will not have an id attribute, because raising an exception takes more time than evaluating a simple condition.
Consider following code:
import os
from timeit import timeit
def get_id_list_try(person):
try:
return [person.id]
except AttributeError:
return []
def get_id_list_if(person):
if hasattr(person, 'id'):
return [person.id]
else:
return []
class Person(object):
def __init__(self, id):
self.id = id
person_with_id = Person(1664)
person_no_id = object()
print("try with id: {}".format(
timeit("get_id_list_try(person_with_id)", number=1000000,
setup="from __main__ import get_id_list_try, person_with_id")))
print("try no id: {}".format(
timeit("get_id_list_try(person_no_id)", number=1000000,
setup="from __main__ import get_id_list_try, person_no_id")))
print("if with id: {}".format(
timeit("get_id_list_if(person_with_id)", number=1000000,
setup="from __main__ import get_id_list_if, person_with_id")))
print("if no id: {}".format(
timeit("get_id_list_if(person_no_id)", number=1000000,
setup="from __main__ import get_id_list_if, person_no_id")))
It tests the performance of the try/catch and the if/else methods with and without an id. It prints this:
try with id: 0.25232274121
try no id: 2.32747888986
if with id: 0.364873724104
if no id: 0.728008592266
As you can see, the try/catch method is a bit faster when an id exists; but when the id does not exists the if/else method is 3 times faster than try/catch method.

Checking for None when accessing nested attributes

I am currently implementing an ORM that stores data defined in an XSD handled with a DOM generated by PyXB.
Many of the respective elements contain sub-elements and so forth, which each have a minOccurs=0 and thus may resolve to None in the DOM.
Hence when accessing some element hierarchy containing optional elements I now face the problem whether to use:
with suppress(AttributeError):
wanted_subelement = root.subelement.sub_subelement.wanted_subelement
or rather
if root.subelement is not None:
if root.subelement.sub_subelement is not None:
wanted_subelement = root.subelement.sub_subelement.wanted_subelement
While both styles work perfectly fine, which is preferable? (I am not Dutch, btw.)

This also works:
if root.subelement and root.subelement.sub_subelement:
wanted_subelement = root.subelement.sub_subelement.wanted_subelement
The if statement evaluates None as False and will check from left to right. So if the first element evaluates to false it will not try to access the second one.

If you have quite a few such lookups to perform, better to wrap this up in a more generic lookup function:
# use a sentinel object distinct from None
# in case None is a valid value for an attribute
notfound = object()
# resolve a python attribute path
# - mostly, a `getattr` that supports
# arbitrary sub-attributes lookups
def resolve(element, path):
parts = path.split(".")
while parts:
next, parts = parts[0], parts[1:]
element = getattr(element, next, notfound)
if element is notfound:
break
return element
# just to test the whole thing
class Element(object):
def __init__(self, name, **attribs):
self.name = name
for k, v in attribs.items():
setattr(self, k, v)
e = Element(
"top",
sub1=Element("sub1"),
nested1=Element(
"nested1",
nested2=Element(
"nested2",
nested3=Element("nested3")
)
)
)
tests = [
"notthere",
"does.not.exists",
"sub1",
"sub1.sub2",
"nested1",
"nested1.nested2",
"nested1.nested2.nested3"
]
for path in tests:
sub = resolve(e, path)
if sub is notfound:
print "%s : not found" % path
else:
print "%s : %s" % (path, sub.name)

How to identify a column name by a string? django/python

I am writing a basic function that takes three arguments, request, field, and user_id.
The idea is, when you pass through the info, the function returns the result which would be the column (identified by argument "field"), the row (identified by the argument "user_id").
this is my function:
def get_user_field(request, user_id, field):
result = Users.objects.raw("SELECT id, %s FROM blog_users WHERE id = %s", [field, user_id])[0]
#return result.??????
what I do not know how to do is to replace those question marks with what to return the corresponding column. If i try
return result.field
It will pass a string where "field" is. And of course a string cannot be put there.
So how can i achieve a function that works pretty much exactly like this and returns one result?
Thanks!

This can be done with the getattr Python builtin:
return getattr(result, field)
But it would be better to do it entirely differently:
def get_user_field(request, user_id, field):
return User.objects.filter(id=user_id).values_list(field, flat=True)[0]
Or, to allow for the possibility of the user not existing:
def get_user_field(request, user_id, field):
result = User.objects.filter(id=user_id).values_list(field, flat=True)
if result:
return result[0]
else:
return None # or raise an exception or whatever you like

Django: What is a clean way to populate a list of optional object attributes that need to be wrapped in try/except? Is this a good place for eval()?

I repeatedly find myself in a position where I'm writing specific django model instance fields into a list for various reasons (exporting to CSV, logging) and I'd imagine the same is true for many other people.
Generating a report could require traversing through foreign keys IF they exist. The larger the report, the more unreadable the code gets as I wrap attribute getter attempts in try/except blocks.
Optional foreign keys are also problems: item.optional_fk.optional_date.method()
for item in django_model_instances:
try:
date_created = item.order.date_created.strftime('%Y/%m/%d')
except AttributeError:
date_created = ''
try:
date_complete = item.order.date_complete.strftime('%Y/%m/%d')
except AttributeError:
date_complete = ''
# perhaps more try/except...
writer.writerow([
item.optional_fk.optional_field.strtime('%Y'),
item.optional_fk.method(),
item.bar,
date_created,
# other attributes...
date_complete,
# other attributes...
])
As you have more columns to write the code starts to look like a monster.
I like the readability of using eval() wrapped in try/except but I read I should avoid eval like the plague.
Is using eval in Python a bad practice?
There is almost always a better way to do it - trying to find a better way without writing too much code :)
Very dangerous and insecure - the strings are hard coded
Makes debugging difficult - True
Slow - code is for generating reports, it can be slow.
.
def no_exceptions_getter(item, statement):
try:
return eval(statement)
except AttributeError, e:
log.debug(e)
return ''
for item in django_model_instances:
writer.writerow([no_exceptions_getter(item, x) for x in (
'item.foo',
'item.bar',
'item.date_created.strftime("%Y/%m/%d")',
'item.date_complete.strftime("%Y/%m/%d")',
'item.optional_foreign_key.foo',
# more items in a readable list format
)])
I don't see the security vulnerability, speed or debugging problems being an issue.
So my question for you experts out there is: is this an OK use of eval?

Why aren't you just using getattr?
for item in django_model_instances:
date_created = getattr(item.order, 'date_created', '')
if date_created:
date_created = date_created.strftime('%Y/%m/%d')
or a simple wrapper, if this particular pattern is used a lot:
def get_strftime(object, attr):
value = getattr(object, attr, None)
if value is None:
return ''
return value.strftime('%Y/%m/%d')
writer.writerow([
item.foo,
item.bar,
get_strftime(item.order, 'date_created'),
get_strftime(item.order, 'date_complete'),
])

I assume in your example that date_created might not exist because you're not always passing the same model class to your loop. The solution may then appear to be to put the logic in the object via the model class definition, and write a 'getrow' method function for any classes you might want to do this to. For classes that have a 'date_created' method, they return that, otherwise return '' or None.

How about
def getattrchain(obj, attrStr, fnArgs=None, defaultResult=''):
try:
for attr in attrStr.split('.'):
obj = getattr(obj,attr)
except AttributeError:
return defaultResult
if callable(obj) and fnArgs is not None:
return obj(*fnArgs)
else:
return obj
for item in django_model_instances:
writer.writerow([getchainattr(item, x, args) for x,args in (
('foo', None),
('bar', None),
('date_created.strftime', [r'%Y/%m/%d']),
('date_complete.strftime', [r'%Y/%m/%d']),
('optional_foreign_key.foo', None),
)
])