We Keep Coding

What is the "pythonic" way to create a list with one element or just keep it empty?

Let's say a have Person instance person. person may have an attribute id.
I want to write a function that gets me either an empty list [] when the attribute is None or missing or the list [12, ] when the attribute id is 12.
def get_id_list(person):
try:
return [getattr(person, 'id'), ]
except AttributeError:
return []
This works perfectly but is there a "pythonic" way of doing this without the try-except block and maybe in one line?

I would go for
def get_id_list(person):
_id = getattr(person, 'id', None)
return [] if _id is None else [_id]
However, it is good practice to make sure that attributes are always defined so you don't have to use getattr with a default or use hasattr to check for existence.

You may use hasattr() to check for attribute as:
def get_id_list(person):
if hasattr(person, 'id'):
return [person.id] # better then getattr(person, 'id')
else:
return []
which could be written in one line as:
def get_id_list(person):
return [person.id] if hasattr(person, 'id') else []

There are two basic ways of going about this:
EAFP (It's Easier to Ask Forgiveness than Permission)- which is what you have:
def get_id_list(person):
try:
return [person.id]
except AttributeError:
return []
LBYL (Look Before You Leap)
def get_id_list(person):
if hasattr(person, 'id'):
return [person.id]
else:
return []
Generally, EAFP is "pythonic", but really, it depends on the specifics of your use-case. If you know that person will usually have the id attribute, then you should use EAFP. That is to say, if it is an exceptional case for person not to have id, use exceptions! If, on the other hand, it is common for person to lack id, then LBYL will be more efficient.
If you simply want a one-liner, you could use a conditional expression:
return [person.id] if hasattr(person,'id') else []

Your approach is pythonic, just do some few tweaking.
First off, don't return inside the try block. You can preserve the id value in a variable and return it properly within else clause. The reason is that you are doing multiple operation like getting attribute and converting to list (and maybe more(in other cases)) while you are just catching the AttributeError. And when you use else you can handle other exceptions easily, also it's more elegant from coding style's perspective.
Secondly you don't need to use getattr in this case, you can simply use direct attribute access.
Also instead of using the try-except you might want to check if the object has the attribute then return its value but due to It's easier to ask forgiveness than it is to get permission. principal the try-exepct is better.
def get_id_list(person):
try:
id = person.id
except AttributeError:
return []
else:
return [id]

getattr() allows you to specify a default if the attribute isn't there so you could use that if you want to avoid a try/catch block.
If the named attribute does not exist, default is returned if provided, otherwise AttributeError is raised.
id = getattr(person, 'id', None)
if id is not None:
return [id]
return []

You can provide a default value (a third argument) to getattr() function to be returned if an attribute does not exist:
def get_id_list(person):
id_ = getattr(person, 'id', None)
return [id_] if id_ is not None else []
(assuming here that None is not a valid value for the id attribute)

according to the Python documentation:
EAFP:
Easier to ask for forgiveness than permission. This common Python
coding style assumes the existence of valid keys or attributes and
catches exceptions if the assumption proves false. This clean and fast
style is characterized by the presence of many try and except
statements. The technique contrasts with the LBYL style common to many
other languages such as C.
That means that your code follows the "official" pythonic way, checking if the attribute exists would be less pythonic.
However, performance needs could eventually override pythonic consideration according to the frequence where person object will not have an id attribute, because raising an exception takes more time than evaluating a simple condition.
Consider following code:
import os
from timeit import timeit
def get_id_list_try(person):
try:
return [person.id]
except AttributeError:
return []
def get_id_list_if(person):
if hasattr(person, 'id'):
return [person.id]
else:
return []
class Person(object):
def __init__(self, id):
self.id = id
person_with_id = Person(1664)
person_no_id = object()
print("try with id: {}".format(
timeit("get_id_list_try(person_with_id)", number=1000000,
setup="from __main__ import get_id_list_try, person_with_id")))
print("try no id: {}".format(
timeit("get_id_list_try(person_no_id)", number=1000000,
setup="from __main__ import get_id_list_try, person_no_id")))
print("if with id: {}".format(
timeit("get_id_list_if(person_with_id)", number=1000000,
setup="from __main__ import get_id_list_if, person_with_id")))
print("if no id: {}".format(
timeit("get_id_list_if(person_no_id)", number=1000000,
setup="from __main__ import get_id_list_if, person_no_id")))
It tests the performance of the try/catch and the if/else methods with and without an id. It prints this:
try with id: 0.25232274121
try no id: 2.32747888986
if with id: 0.364873724104
if no id: 0.728008592266
As you can see, the try/catch method is a bit faster when an id exists; but when the id does not exists the if/else method is 3 times faster than try/catch method.

One-liner to save value of if statement?

Is there a smart way to write the following code in three or four lines?
a=l["artist"]
if a:
b=a["projects"]
if b:
c=b["project"]
if c:
print c
So I thought for something like pseudocode:
a = l["artist"] if True:

How about:
try:
print l["artist"]["projects"]["project"]
except KeyError:
pass
except TypeError:
pass # None["key"] raises TypeError.
This will try to print the value, but if a KeyError is raised, the except block will be run. pass means to do nothing. This is known and EAFP: it’s Easier to Ask Forgiveness than Permission.

I don't necessarily think that this is better but you could do:
try:
c = l["artist"]["projects"]["project"]
except (KeyError, TypeError) as e:
print e
pass

p = l.get('artist') and l['artist'].get('projects') and l['artist']['projects'].get('project')
if p:
print p
You can also make a more general function for this purpose:
def get_attr(lst, attr):
current = lst
for a in attr:
if current.get(a) is not None:
current = current.get(a)
else:
break
return current
>>> l = {'artist':{'projects':{'project':1625}}}
>>> get_attr(l,['artist','projects','project'])
1625

One-liner (as in the title) without exceptions:
if "artist" in l and l["artist"] and "projects" in l["artist"] and l["artist"]["projects"] and "project" in l["artist"]["projects"]: print l["artist"]["projects"]["project"]

Since you're dealing with nested dictionaries, you might find this generic one-liner useful because it will allow you to access values at any level just by passing it more keys arguments:
nested_dict_get = lambda item, *keys: reduce(lambda d, k: d.get(k), keys, item)
l = {'artist': {'projects': {'project': 'the_value'}}}
print( nested_dict_get(l, 'artist', 'projects', 'project') ) # -> the_value
Note: In Python 3, you'd need to add a from functools import reduce at the top.

how to set an attribute in a nested object in Python?

I'm making my first attempts at Python.
I need to loop over a log, parse log entries and then update an object, which includes nested objects for machines listed in the log.
This is what I have:
import re
format_pat= re.compile(
r"(?P<host>(?:[\d\.]|[\da-fA-F:])+)\s"
r"(?P<identity>\S*)\s"
r"(?P<user>\S*)\s"
r"\[(?P<time>.*?)\]\s"
r'"(?P<request>.*?)"\s'
r"(?P<status>\d+)\s"
r"(?P<bytes>\S*)\s"
r'"(?P<referer>.*?)"\s'
r'"(?P<user_agent>.*?)"\s*'
)
from json import JSONEncoder
class MyEncoder(JSONEncoder):
def default(self, o):
return o.__dict__
# JSON response object
class ResponseObject(object):
def __init__(self, dict):
self.__dict__ = dict
# check for JSON response object
try:
obj
except NameError:
obj = ResponseObject({})
test = ['2001:470:1f14:169:15f3:824f:8a61:7b59 - SOFTINST [14/Nov/2012:09:32:31 +0100] "POST /setComputer HTTP/1.1" 200 4 "-" "-" 102356']
# log loop
for line in test:
try:
# try to create object from log entry
m = format_pat.match(line)
if m:
res = m.groupdict()
res["status"] = int(res["status"])
# register machine if not done
if not hasattr(obj, res["user"]):
setattr(obj, res["user"], {"downtime":"0","flag":"false","downstart":"0","init":res["time"],"last":"","uptime":"","downtime":"","totaltime":""})
machine = getattr(obj, res["user"])
flag = machine["flag"]
start = machine["downstart"]
down = machine["downtime"]
last = machine["last"]
print "done"
# set last
last = res["time"]
# PROBLEM this does not work
setattr(machine, last, res["time"])
print machine
else:
print "nope"
except:
print "nope base"
print MyEncoder().encode(obj)
The error I'm getting when trying to setattr() is
AttributeError: 'dict' object has no attribute ''
but I was afraid it was not as easy as this...
Question:
How do I update the last value in my nested object using 'setattr'? Or is there another way to update nested object attributes?

I think you need to do this:
setattr(machine, 'last', res["time"])
As setattr needs a string of the name of the attribute to be set

Do not use setattr. Just assign a value to the "last" key for each machine dictionary.
(actually you answered your own question!)

I don't understand why, but I can set the value of last like this:
print machine
print machine["last"]
print res["time"]
# this works
machine["last"] = res["time"]
print machine
If someone can explain, would be nice :-)

Python Function with "Pointers"

Ok, I recently started programming in Python, and I really like it.
However, I have run into a little issue.
I want to be able to define a function to take in some data and assign it to a variable that I designate, rather than have to perform the operation every time I want to submit the value.
Here is a code fragment:
try:
if elem.virtual.tag:
virt = True
temp_asset.set_virtual(True)
except AttributeError:
temp_asset.set_virtual(False)
if virt: #if virtual, get only faction, value, and range for presence
try:
fac = elem.presence.faction #an xml tag (objectified)
except AttributeError:
fac = "faction tag not found"
temp_asset.misload = True
try:
val = elem.presence.value
except AttributeError:
val = "value tag not found"
temp_asset.misload = True
try:
rang = elem.presence.range
except AttributeError:
rang = "range tag not found"
temp_asset.misload = True
#Set presence values
temp_asset.set_presence(fac, val, rang)
The functions set the values, but I want to be able to perform the error checking with something like this:
def checkval(self, variable_to_set, tag_to_use)
try:
variable_to_set = tag_to_use
except AttributeError:
variable_to_set = "tag not found"
temp_asset.misload = True
Is this doable? Let me know if I need to show more code.
Edit: I don't need pointers per se, just anything that works this way and saves typing.
Edit 2: Alternatively, I need a solution of how to check whether an objectified xml node exists (lxml).

Have you tried/looked into the getattr and setattr functions?
For example, assuming these "variables" are object attributes:
def checkval(self, attr, presence, tagstr):
tag = getattr(presence, tagstr, None) # tag = presence."tagstr" or None
setattr(self, attr, tag or 'tag not found') # ?? = presence."tagstr" or 'tag not found'
if tag is None:
self.temp_asset.misload = True
You call it like,
your_object.checkval('fac', elem.presence, 'faction')
Alternatively, you can pre-define these variables and set them default values before you attempt to look up the tags. For example:
class YourObject(object):
_attrmap = {
'fac': 'faction',
'val': 'value',
'rang': 'range',
}
def __init__(self):
# set default values
for attr, tagstr in self._attrmap.items():
setattr(self, attr, '%s tag not found' % tagstr)
def checkval(self, attr, presence):
for attr, tagstr in self._attrmap.items():
tag = getattr(presence, tagstr, None)
if tag is not None:
setattr(self, attr, tag)
else:
self.temp_asset.misload = True

has_next in Python iterators?

Have Python iterators got a has_next method?

There's an alternative to the StopIteration by using next(iterator, default_value).
For exapmle:
>>> a = iter('hi')
>>> print next(a, None)
h
>>> print next(a, None)
i
>>> print next(a, None)
None
So you can detect for None or other pre-specified value for end of the iterator if you don't want the exception way.

No, there is no such method. The end of iteration is indicated by an exception. See the documentation.

If you really need a has-next functionality, it's easy to obtain it with a little wrapper class. For example:
class hn_wrapper(object):
def __init__(self, it):
self.it = iter(it)
self._hasnext = None
def __iter__(self): return self
def next(self):
if self._hasnext:
result = self._thenext
else:
result = next(self.it)
self._hasnext = None
return result
def hasnext(self):
if self._hasnext is None:
try: self._thenext = next(self.it)
except StopIteration: self._hasnext = False
else: self._hasnext = True
return self._hasnext
now something like
x = hn_wrapper('ciao')
while x.hasnext(): print next(x)
emits
c
i
a
o
as required.
Note that the use of next(sel.it) as a built-in requires Python 2.6 or better; if you're using an older version of Python, use self.it.next() instead (and similarly for next(x) in the example usage). [[You might reasonably think this note is redundant, since Python 2.6 has been around for over a year now -- but more often than not when I use Python 2.6 features in a response, some commenter or other feels duty-bound to point out that they are 2.6 features, thus I'm trying to forestall such comments for once;-)]]
===
For Python3, you would make the following changes:
from collections.abc import Iterator # since python 3.3 Iterator is here
class hn_wrapper(Iterator): # need to subclass Iterator rather than object
def __init__(self, it):
self.it = iter(it)
self._hasnext = None
def __iter__(self):
return self
def __next__(self): # __next__ vs next in python 2
if self._hasnext:
result = self._thenext
else:
result = next(self.it)
self._hasnext = None
return result
def hasnext(self):
if self._hasnext is None:
try:
self._thenext = next(self.it)
except StopIteration:
self._hasnext = False
else: self._hasnext = True
return self._hasnext

In addition to all the mentions of StopIteration, the Python "for" loop simply does what you want:
>>> it = iter("hello")
>>> for i in it:
... print i
...
h
e
l
l
o

Try the __length_hint__() method from any iterator object:
iter(...).__length_hint__() > 0

You can tee the iterator using, itertools.tee, and check for StopIteration on the teed iterator.

hasNext somewhat translates to the StopIteration exception, e.g.:
>>> it = iter("hello")
>>> it.next()
'h'
>>> it.next()
'e'
>>> it.next()
'l'
>>> it.next()
'l'
>>> it.next()
'o'
>>> it.next()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
StopIteration docs: http://docs.python.org/library/exceptions.html#exceptions.StopIteration
Some article about iterators and generator in python: http://www.ibm.com/developerworks/library/l-pycon.html

No. The most similar concept is most likely a StopIteration exception.

I believe python just has next() and according to the doc, it throws an exception is there are no more elements.
http://docs.python.org/library/stdtypes.html#iterator-types

The use case that lead me to search for this is the following
def setfrom(self,f):
"""Set from iterable f"""
fi = iter(f)
for i in range(self.n):
try:
x = next(fi)
except StopIteration:
fi = iter(f)
x = next(fi)
self.a[i] = x
where hasnext() is available, one could do
def setfrom(self,f):
"""Set from iterable f"""
fi = iter(f)
for i in range(self.n):
if not hasnext(fi):
fi = iter(f) # restart
self.a[i] = next(fi)
which to me is cleaner. Obviously you can work around issues by defining utility classes, but what then happens is you have a proliferation of twenty-odd different almost-equivalent workarounds each with their quirks, and if you wish to reuse code that uses different workarounds, you have to either have multiple near-equivalent in your single application, or go around picking through and rewriting code to use the same approach. The 'do it once and do it well' maxim fails badly.
Furthermore, the iterator itself needs to have an internal 'hasnext' check to run to see if it needs to raise an exception. This internal check is then hidden so that it needs to be tested by trying to get an item, catching the exception and running the handler if thrown. This is unnecessary hiding IMO.

Maybe it's just me, but while I like https://stackoverflow.com/users/95810/alex-martelli 's answer, I find this a bit easier to read:
from collections.abc import Iterator # since python 3.3 Iterator is here
class MyIterator(Iterator): # need to subclass Iterator rather than object
def __init__(self, it):
self._iter = iter(it)
self._sentinel = object()
self._next = next(self._iter, self._sentinel)
def __iter__(self):
return self
def __next__(self): # __next__ vs next in python 2
if not self.has_next():
next(self._iter) # raises StopIteration
val = self._next
self._next = next(self._iter, self._sentinel)
return val
def has_next(self):
return self._next is not self._sentinel

No, there is no such method. The end of iteration is indicated by a StopIteration (more on that here).
This follows the python principle EAFP (easier to ask for forgiveness than permission). A has_next method would follow the principle of LBYL (look before you leap) and contradicts this core python principle.
This interesting article explains the two concepts in more detail.

Suggested way is StopIteration.
Please see Fibonacci example from tutorialspoint
#!usr/bin/python3
import sys
def fibonacci(n): #generator function
a, b, counter = 0, 1, 0
while True:
if (counter > n):
return
yield a
a, b = b, a + b
counter += 1
f = fibonacci(5) #f is iterator object
while True:
try:
print (next(f), end=" ")
except StopIteration:
sys.exit()

It is also possible to implement a helper generator that wraps any iterator and answers question if it has next value:
Try it online!
def has_next(it):
first = True
for e in it:
if not first:
yield True, prev
else:
first = False
prev = e
if not first:
yield False, prev
for has_next_, e in has_next(range(4)):
print(has_next_, e)
Which outputs:
True 0
True 1
True 2
False 3
The main and probably only drawback of this method is that it reads ahead one more element, for most of tasks it is totally alright, but for some tasks it may be disallowed, especially if user of has_next() is not aware of this read-ahead logic and may missuse it.
Code above works for infinite iterators too.
Actually for all cases that I ever programmed such kind of has_next() was totally enough and didn't cause any problems and in fact was very helpful. You just have to be aware of its read-ahead logic.

The way has solved it based on handling the "StopIteration" execption is pretty straightforward in order to read all iterations :
end_cursor = False
while not end_cursor:
try:
print(cursor.next())
except StopIteration:
print('end loop')
end_cursor = True
except:
print('other exceptions to manage')
end_cursor = True

I think there are valid use cases for when you may want some sort of has_next functionality, in which case you should decorate an iterator with a has_next defined.
Combining concepts from the answers to this question here is my implementation of that which feels like a nice concise solution to me (python 3.9):
_EMPTY_BUF = object()
class BufferedIterator(Iterator[_T]):
def __init__(self, real_it: Iterator[_T]):
self._real_it = real_it
self._buf = next(self._real_it, _EMPTY_BUF)
def has_next(self):
return self._buf is not _EMPTY_BUF
def __next__(self) -> _T_co:
v = self._buf
self._buf = next(self._real_it, _EMPTY_BUF)
if v is _EMPTY_BUF:
raise StopIteration()
return v
The main difference is that has_next is just a boolean expression, and also handles iterators with None values.
Added this to a gist here with tests and example usage.

With 'for' one can implement his own version of 'next' avoiding exception
def my_next(it):
for x in it:
return x
return None

very interesting question, but this "hasnext" design had been put into leetcode:
https://leetcode.com/problems/iterator-for-combination/
here is my implementation:
class CombinationIterator:
def __init__(self, characters: str, combinationLength: int):
from itertools import combinations
from collections import deque
self.iter = combinations(characters, combinationLength)
self.res = deque()
def next(self) -> str:
if len(self.res) == 0:
return ''.join(next(self.iter))
else:
return ''.join(self.res.pop())
def hasNext(self) -> bool:
try:
self.res.insert(0, next(self.iter))
return True
except:
return len(self.res) > 0

The way I solved my problem is to keep the count of the number of objects iterated over, so far. I wanted to iterate over a set using calls to an instance method. Since I knew the length of the set, and the number of items counted so far, I effectively had an hasNext method.
A simple version of my code:
class Iterator:
# s is a string, say
def __init__(self, s):
self.s = set(list(s))
self.done = False
self.iter = iter(s)
self.charCount = 0
def next(self):
if self.done:
return None
self.char = next(self.iter)
self.charCount += 1
self.done = (self.charCount < len(self.s))
return self.char
def hasMore(self):
return not self.done
Of course, the example is a toy one, but you get the idea. This won't work in cases where there is no way to get the length of the iterable, like a generator etc.