I was writing some Python 3.2 code and this question came to me:
I've got these variables:
# a list of xml.dom nodes (this is just an example!)
child_nodes = [node1, node2, node3]
# I want to add every item in child_node into this node (this also a xml.dom Node)
parent = xml_document.createElement('TheParentNode')
This is exactly what I want to do:
for node in child_nodes:
if node is not None:
parent.appendChild(node)
I wrote it in one line like this:
[parent.appendChild(c) for c in child_nodes if c is not None]
I'm not going to use the list result at all, I only need the appendChild to do its work.
I'm not a very experienced python programmer, so I wonder which one is better?
I like the single line solution, but I would like to know from experienced python programmers:
Which one is better, in a context of either code beauty/maintainability) and performance/memory use.
The former is preferable in this situation.
The latter is called a list comprehension and creates a new list object full of the results of each call to parent.appendChild(c). And then discards it.
However, if you want to make a list based on this kind of iteration, then you should certainly employ a list comprehension.
The question of code beauty/maintainability is a tricky one. It's really up to you and whoever you work with to decide.
For a long time I was uncomfortable with list comprehensions and so on and preferred writing it the first way because it was easier for me to read. Some people I work with, however, prefer the second method.
Related
In maya, I want to make an OpenMaya MSelectionList (api version 2.0) with more than one item in it... I've only been able to fill it with the add method like so:
import maya.api.OpenMaya as om
selList = om.MSelectionList()
selList.add('node1')
selList.add('node2')
selList.add('node3')
That's ok for populating it with just a few items, but it's tedious if you have more... I'm wondering if there's a way to do something more like this:
import maya.api.OpenMaya as om
selList = om.MSelectionList(['node1', 'node2', 'node3'])
I could write my own function to create an empty MSelectionList, loop through a list, and add them and then return it; I'm just wondering I have completely looked over something obvious? From what I can tell on the docs, you can only create an empty MSelectionList, or create one by passing in another MSelectionList (to basically duplicate it).
If this can't really be done inherently in the class, does anyone have an idea why it was implemented this way?
The MSelectionList is ultimately a wrapper around a C++ list of object pointers (the Maya api is unusual in that uses diffeent function sets to work on different aspects of an object, rather than the more familiar use of a classic inheritance tree).
Implementing variadic functions in C++ is not trivial (especially not back in the 90s when the Maya API was designed. I suspect nobody felt it was worth the time for what's essentially syntactic sugar.
sl = om.MSelectionList()
for node in nodelist:
sl.add(n0de)
or
sl = om.MSelectionList()
[sl.add(n0de) for node in nodelist]
although I would not recommend the shorter version, which produces an list of meaningless Nones as a side effect
Often times I find that, when working with Pythonic sets, the Pythonic way seems to be absent.
For example, doing something like a dijkstra or a*:
openSet, closedSet = set(nodes), set(nodes)
while openSet:
walkSet, openSet = openSet, set()
for node in walkSet:
for dest in node.destinations():
if dest.weight() < constraint:
if dest not in closedSet:
closedSet.add(dest)
openSet.add(dest)
This is a weakly contrived example, the focus is the last three lines:
if not value in someSet:
someSet.add(value)
doAdditionalThings()
Given the Python way of working with, for example, accessing/using values of a dict, I would have expected to be able to do:
try:
someSet.add(value)
except KeyError:
continue # well, that's ok then.
doAdditionalThings()
As a C++ programmer, my skin crawls a bit that I can't even do:
if someSet.add(value):
# add wasn't blocked by the value already being present
doAdditionalThings()
Is there a more Pythonic (and if possible more efficient) way to work with this sort of set-as-guard usage?
The add operation is not supposed to also tell you if the item was already in the set; it just makes sure it is in there after you add it. Or put another way, what you want is not "add an item and check if it worked"; you want to first check if the item is there, and if not, then do some special stuff. If all you wanted to do was add the item, you wouldn't do the check at all. There is nothing unpythonic about this pattern:
if item not in someSet:
someSet.add(item)
doStuff()
else:
doOtherStuff()
It is true that the API could have been designed so that .add returned whether the item was already in there, but in my experience that's not a particularly common use case. Part of the point of sets is that you can freely add items without worrying about whether they were already in there (since adding an already-included item has no effect). Also, having .add return None is consistent with the general convention for Python builtin types that methods that mutate their arguments return None. It is really things like dict.setdefault (which gets an item but first adds it if isn't there) that are the unusual case.
In a Python program I'm writing, I've built up a linked list using a dictionary which maps each node to its successor (with the last node mapped to None).
(Actually, the dictionary holds what Wikipedia tells me is called a spaghetti stack, which is a tree where each node is linked to its parent but not its children. This means there are many partially-overlapping paths from the leaf nodes to the root node. I only care about one of those paths, starting from a specific leaf node. This doesn't really matter for the question, other than to rule out any solution that involves iterating over all the elements in the dictionary.)
I need to pass this list to another function as an iterable. I know I can do this using a generator function (see code below), but it seems like there ought to be a built-in function to make the iterator I need in one line (or a perhaps a generator expression). I've done a bit of searching through the documentation, but there's nothing in the itertools or functools modules seems applicable, and I'm not sure where else to look.
Here's the generator function I have now. The outer function can be eliminated (inlined) but the inner generator seems to be the only simple way to make an iterable for the data:
def makeListGenerator(nextDict, start):
def gen(node):
while node:
yield node
node = nextDict[node]
return gen(start)
It seems like there should be a pattern for this sort of generator, but I'm not sure what it would be called. Here's a generic version:
def makeGenericGenerator(nextFunc, continueFunc, start):
def gen(value):
while continueFunc(value):
yield value
value = nextFunc(value)
return gen(start)
I could implement the specific version using this one using the call:
makeGenericGenerator(lambda v: nextDict[v], bool, start)
Does something like that already exist in the Python standard library?
The essential problem you face is that every time another value is taken from your iterable, your iterable has to remember that value, so that it knows how to generate the next value. In other words, your iterable needs to maintain its own state.
That means that, whether or not there's a good answer to your question, using a generator is probably the right solution -- because that's exactly what generators were made for! The whole point of generators is that they save their state between calls to their next method, and that's exactly what you need.
Generator expressions, on the other hand, are better for stateless transformations. A lot of people try to shoehorn state into them, and it's generally kind of ugly. I actually spent a little while trying to do it for your case, and couldn't get a generator expression to work. I finally found something that sort of works. This uses the little-known callable_iterator version of iter:
>>> d = {1:2, 2:3, 3:4, 4:5, 5:None}
>>> list(iter(lambda st=[1]: st.__setitem__(0, d[st[0]]) or st[0], None))
[2, 3, 4, 5]
I present this not as an answer, but as a demonstration of why this doesn't work very well.
nodes = {
'A':'B',
'B':'C',
'C':'D',
'D':None
}
print (lambda f: lambda g: f(f,g)) (lambda f,x: [x] + f(f, nodes[x]) if x else [])('A')
# ['A', 'B', 'C', 'D']
for i in vr_world.getNodeNames():
if i != "_error_":
World[i] = vr_world.getChild(i)
vr_world.getNodeNames() returns me a gigantic list, vr_world.getChild(i) returns a specific type of object.
This is taking a long time to run, is there anyway to make it more efficient? I have seen one-liners for loops before that are supposed to be faster. Ideas?
kaloyan suggests using a generator. Here's why that may help.
If getNodeNames() builds a list, then your loop is basically going over the list twice: once to build it, and once when you iterate over the list.
If getNodeNames() is a generator, then your loop doesn't ever build the list; instead of creating the item and adding it to the list, it creates the item and yields it to the caller.
Whether or not this helps is contingent on a couple of things. First, it has to be possible to implement getNodeNames() as a generator. We don't know anything about the implementation details of that function, so it's not possible to say if that's the case. Next, the number of items you're iterating over needs to be pretty big.
Of course, none of this will have any effect at all if it turns out that the time-consuming operation in all of this is vr_world.getChild(). That's why you need to profile your code.
I don't think you can make it faster than what you have there. Yes, you can put the whole thing on one line but that will not make it any faster. The bottleneck obviously is getNodeNames(). If you can make it a generator, you will start populating the World dict with results sooner (if that matters to you) and if you make it filter out the "_error_" values, you will not have the deal with that at a later stage.
World = dict((i, vr_world.getChild(i)) for i in vr_world.getNodeNames() if i != "_error_")
This is a one-liner, but not necessarily much faster than your solution...
Maybe you can use a filter and a map, however I don't know if this would be any faster:
valid = filter(lambda i: i != "_error_", vr_world.getNodeNames())
World = map(lambda i: vr_world.getChild(i), valid)
Also, as you'll see a lot around here, profile first, and then optimize, otherwise you may be wasting time. You have two functions there, maybe they are the slow parts, not the iteration.
i wanted to do something like this but this code return list of None (i think it's because list.reverse() is reversing the list in place):
map(lambda row: row.reverse(), figure)
i tried this one, but the reversed return an iterator :
map(reversed, figure)
finally i did something like this , which work for me , but i don't know if it's the right solution:
def reverse(row):
"""func that reverse a list not in place"""
row.reverse()
return row
map(reverse, figure)
if someone has a better solution that i'm not aware of please let me know
kind regards,
The mutator methods of Python's mutable containers (such as the .reverse method of lists) almost invariably return None -- a few return one useful value, e.g. the .pop method returns the popped element, but the key concept to retain is that none of those mutators returns the mutated container: rather, the container mutates in-place and the return value of the mutator method is not that container. (This is an application of the CQS principle of design -- not quite as fanatical as, say, in Eiffel, the language devised by Bertrand Meyer, who also invented CQS, but that's just because in Python "practicality beats purity, cfr import this;-).
Building a list is often costlier than just building an iterator, for the overwhelmingly common case where all you want to do is loop on the result; therefore, built-ins such as reversed (and all the wonderful building blocks in the itertools module) return iterators, not lists.
But what if you therefore have an iterator x but really truly need the equivalent list y? Piece of cake -- just do y = list(x). To make a new instance of type list, you call type list -- this is such a general Python idea that it's even more crucial to retain than the pretty-important stuff I pointed out in the first two paragraphs!-)
So, the code for your specific problem is really very easy to put together based on the crucial notions in the previous paragraphs:
[list(reversed(row)) for row in figure]
Note that I'm using a list comprehension, not map: as a rule of thumb, map should only be used as a last-ditch optimization when there is no need for a lambda to build it (if a lambda is involved then a listcomp, as well as being clearer as usual, also tends to be faster anyway!-).
Once you're a "past master of Python", if your profiling tells you that this code is a bottleneck, you can then know to try alternatives such as
[row[::-1] for row in figure]
applying a negative-step slicing (aka "Martian Smiley") to make reversed copies of the rows, knowing it's usually faster than the list(reversed(row)) approach. But -- unless your code is meant to be maintained only by yourself or somebody at least as skilled at Python -- it's a defensible position to use the simplest "code from first principles" approach except where profiling tells you to push down on the pedal. (Personally I think the "Martian Smiley" is important enough to avoid applying this good general philosophy to this specific use case, but, hey, reasonable people could differ on this very specific point!-).
You can also use a slice to get the reversal of a single list (not in place):
>>> a = [1,2,3,4]
>>> a[::-1]
[4, 3, 2, 1]
So something like:
all_reversed = [lst[::-1] for lst in figure]
...or...
all_reversed = map(lambda x: x[::-1], figure)
...will do what you want.
reversed_lists = [list(reversed(x)) for x in figure]
map(lambda row: list(reversed(row)), figure)
You can also simply do
for row in figure:
row.reverse()
to change each row in place.