Assuming that my list has only str and None, and I want to check assign lowered strings into another variable but if it's uppered or None, it should be rendered as None.
The code to checking for None and str.isupper() works:
for i in [None,'foo',None,'FOO',None,'bar']:
x = None if i is None or i.isupper() else i
print x
but the negated condition didn't work:
for i in [None,'foo',None,'FOO',None,'bar']:
x = i if i is not None or i.isupper() else None
print x
It returns AttributeError:
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
AttributeError: 'NoneType' object has no attribute 'isupper'
Why is that so?
Other than the ternary assignment in a loop, is there another way to perform the same operation?
You are testing if i is not None or i.isupper(). This doesn't work when i is None:
(i is not None) or (i.isupper())
is evaluated lazily; first the left is evaluated, and only if False is the second argument evaluated. The first is only False if i is None, so the right-hand expression is only ever evaluated as None.isupper().
Use and instead, and negate the isupper() test:
x = i if i is not None and not i.isupper() else None
The above expression is the proper logical inversion of i is None or i.isupper().
Alternatively use not (..) around your original expression:
x = i if not (i is None or i.isupper()) else None
Because you are using a regular for loop, there is no need to use a conditional expression here; the following, assigning to the existing loop variable i, suffices:
for i in [None,'foo',None,'FOO',None,'bar']:
if i and i.isupper():
i = None
print i
In the code that returns an AttributeError, it's because if i is none, it goes on to the second part of the or statement and checks i.isupper(), but None doesn't have an isupper() method, hence the error.
May I suggest a different approach?
items = [None,'foo',None,'FOO',None,'bar']
def splitItems(items, test):
trueItems = []
falseItems = []
for item in items:
if test(item):
trueItems.append(item)
else:
falseItems.append(item)
return trueItems, falseItems
trueItems, falseItems = splitItems(items, lambda i:i is None or i.isupper())
print "uppercase or None:", trueItems
print "lowercase:", falseItems
# uppercase or None: [None, None, 'FOO', None]
# lowercase: ['foo', 'bar']
I like #Brionius' edited answer, here is an even more general form of it, returning a dict of items from a list, keyed by the value returned by a discriminating function (could return more than just True or False).
from collections import defaultdict
def groupByEval(seq, fn):
ret = defaultdict(list)
for item in seq:
ret[fn(item)].append(item)
return dict(ret.iteritems())
test = [None,'foo',None,'FOO',None,'bar']
print groupByEval(test, lambda x: x is not None)
print groupByEval(test, lambda x: 0 if x is None else len(x))
print groupByEval(test, lambda x: x is not None and x.isupper())
print groupByEval(test, lambda x: x if x is None else sum(x.lower().count(c) for c in 'aeiou'))
Giving:
{False: [None, None, None], True: ['foo', 'FOO', 'bar']}
{0: [None, None, None], 3: ['foo', 'FOO', 'bar']}
{False: [None, 'foo', None, None, 'bar'], True: ['FOO']}
{1: ['bar'], 2: ['foo', 'FOO'], None: [None, None, None]}
Brionius' solution could then be implemented as:
def splitItems(seq, condition):
ret = groupByEval(seq, condition)
return ret.get(True,[]), ret.get(False,[])
Related
I have two separate lists (of lists) that can potentially have overlapping values. Each list has a date. I would like to modify mylist1 so that the date (item[3]) is the minimum of the date from mylist1 and the corressponding date in mylist2. The problem I am running into is that when there is no match, I get an error: IndexError: list index out of range. In these cases, I would just like to not take the minimum, but instead just use the date in mylist1. Is there some sort of if error clause that I can put around it?
My Code:
mylist1 = [[u'AAA', None, u'111', u'1/1/2015'],
[u'BBB', None, u'222', u'1/1/2012'],
[u'CCC', None, u'333', u'1/1/2012']]
mylist2 = [(u'111', u'1/1/2011'),
(u'333', u'2013-11-10'),
(u'444', u'1/1/2017')]
for key, item in enumerate(mylist1):
mylist1[key] = [item[0], item[1], item[2],
min(item[3], [x for x in mylist2 if x[0] == item[2]][0][1])]
Desired output:
[[u'AAA', None, u'111', u'1/1/2011'],
[u'BBB', None, u'222', u'1/1/2012'],
[u'CCC', None, u'333', u'1/1/2012']]
If I got this right, your mylist2 is kinda used as a dictionary. Why not just make it one:
mylist1 = [[u'AAA', None, u'111' ,u'1/1/2015'], [u'BBB', None, u'222' ,u'1/1/2012'], [u'CCC', None, u'333' ,u'1/1/2012']]
mylist2 = [(u'111', u'1/1/2011'), (u'333', u'2013-11-10'), (u'444', u'1/1/2017')]
# assuming, you are not responsible for the form of mylist2,
# we will change it into a dictionary here:
mydict = dict(mylist2) # easy :)
for elem_list in mylist1:
elem_list[3] = min(elem_list[3], mydict.get(elem_list[2], elem_list[3]))
elem_list is a reference to the list, so we don't need the index of it in mylist1 as a change in it will persist. This makes the rest of the last line easier, as we do not have to re-build the original list. The mydict.get prevents the error, if the desired element is not available.
mylist1 = [[u'AAA', None, u'111' ,u'1/1/2015'], [u'BBB', None, u'222' ,u'1/1/2012'], [u'CCC', None, u'333' ,u'1/1/2012']]
mylist2 = [(u'111', u'1/1/2011'), (u'333', u'2013-11-10'), (u'444', u'1/1/2017')]
for key, item in enumerate(mylist1):
try:
mylist1[key] = [item[0],item[1],item[2],min(item[3],[x for x in mylist2 if x[0] == item[2]][0][1])]
except IndexError:
#Put your codes that what you want to do if you got this error
#print ("An error happened but I dont care") <--- like this for example
pass #or just simply pass this error
Actually you can decide what will happen if you got IndexError, try this.
This is a cool way to do it if you have unique keys:
# produce dictionaries whose keys are the matching values (have to be uniue)
mylist1dict={x[2]: x for x in mylist1}
mylist2dict={x[0]: x for x in mylist2}
# replace where needed, only use common keys
for k in set(mylist2dict) & set(mylist1dict):
mylist1dict[k][2] = mylist2dict[k]
# this is the result
mylist1dict.values()
As in some case your min function return an empty list You can use an if statement for check the it then indexing :
>>> for key, item in enumerate(mylist1):
... m=min(item[3],[x for x in mylist2 if x[0] == item[2]])
... if m :
... mylist1[key] = [item[0],item[1],item[2],m[0][1]]
...
>>> mylist1
[[u'AAA', None, u'111', u'1/1/2011'],
[u'BBB', None, u'222', u'1/1/2012'],
[u'CCC', None, u'333', u'1/1/2012']]
I do have following problem -
dict1 = {'abc': {'x': 2, 'y': 3, 'z': 4}, 'mno': {'p': 3, 'q':4, 'r':5}}
def refine():
def calc(a_dict):
if some_condition :
return x ={'a':1}
else:
return None
for k, v in dict1:
return calc(v)
Now when I am iterating this function call inside a for loop , I get either None or a. The function call is such that I can get maximum of one a .
Sample Output-
None
None
{'a':1}
None
What I want calc function to return None in case all the values after the loop gives None or x in case any of the values after loop gives x .How can this be achieved ? Thanks in advance.
Edit -
What if I store the results in a list and want to check if the list has something else than None and if True return it.
If I did get the question, you may store results of conditions applied on a dictionary and then check with all quantifier function:
def calc(a_dict):
results = (some_condition(k,v) for k,v in a_dict.iteritems())
if all(res == None for res in results):
return None
elif all(res == <something> for res in results):
return {'a':1}
else:
return <something different>
u can use any() . any check for the condition if any one value is true. it return x . if all values fail it goes to else
if any(somecondition):
return x
else:
return None
Is there a way, lib, or something in python that I can set value in list at an index that does not exist?
Something like runtime index creation at list:
l = []
l[3] = 'foo'
# [None, None, None, 'foo']
And more further, with multi dimensional lists:
l = []
l[0][2] = 'bar'
# [[None, None, 'bar']]
Or with an existing one:
l = [['xx']]
l[0][1] = 'yy'
# [['xx', 'yy']]
There isn't a built-in, but it's easy enough to implement:
class FillList(list):
def __setitem__(self, index, value):
try:
super().__setitem__(index, value)
except IndexError:
for _ in range(index-len(self)+1):
self.append(None)
super().__setitem__(index, value)
Or, if you need to change existing vanilla lists:
def set_list(l, i, v):
try:
l[i] = v
except IndexError:
for _ in range(i-len(l)+1):
l.append(None)
l[i] = v
Not foolproof, but it seems like the easiest way to do this is to initialize a list much larger than you will need, i.e.
l = [None for i in some_large_number]
l[3] = 'foo'
# [None, None, None, 'foo', None, None None ... ]
If you really want the syntax in your question, defaultdict is probably the best way to get it:
from collections import defaultdict
def rec_dd():
return defaultdict(rec_dd)
l = rec_dd()
l[3] = 'foo'
print l
{3: 'foo'}
l = rec_dd()
l[0][2] = 'xx'
l[1][0] = 'yy'
print l
<long output because of defaultdict, but essentially)
{0: {2: 'xx'}, 1: {0: 'yy'}}
It isn't exactly a 'list of lists' but it works more or less like one.
You really need to specify the use case though... the above has some advantages (you can access indices without checking whether they exist first), and some disadvantages - for example, l[2] in a normal dict will return a KeyError, but in defaultdict it just creates a blank defaultdict, adds it, and then returns it.
Other possible implementations to support different syntactic sugars could involve custom classes etc, and will have other tradeoffs.
You cannot create a list with gaps. You could use a dict or this quick little guy:
def set_list(i,v):
l = []
x = 0
while x < i:
l.append(None)
x += 1
l.append(v)
return l
print set_list(3, 'foo')
>>> [None, None, None, 'foo']
Given a list, is there a way to get the first non-None value? And, if so, what would be the pythonic way to do so?
For example, I have:
a = objA.addreses.country.code
b = objB.country.code
c = None
d = 'CA'
In this case, if a is None, then I would like to get b. If a and b are both None, the I would like to get d.
Currently I am doing something along the lines of (((a or b) or c) or d), is there another way?
You can use next():
>>> a = [None, None, None, 1, 2, 3, 4, 5]
>>> next(item for item in a if item is not None)
1
If the list contains only Nones, it will throw StopIteration exception. If you want to have a default value in this case, do this:
>>> a = [None, None, None]
>>> next((item for item in a if item is not None), 'All are Nones')
All are Nones
I think this is the simplest way when dealing with a small set of values:
firstVal = a or b or c or d
Will always return the first non "Falsey" value which works in some cases (given you dont expect any values which could evaluate to false as #GrannyAching points out below)
first_true is an itertools recipe found in the Python 3 docs:
def first_true(iterable, default=False, pred=None):
"""Returns the first true value in the iterable.
If no true value is found, returns *default*
If *pred* is not None, returns the first item
for which pred(item) is true.
"""
# first_true([a,b,c], x) --> a or b or c or x
# first_true([a,b], x, f) --> a if f(a) else b if f(b) else x
return next(filter(pred, iterable), default)
One may choose to implement the latter recipe or import more_itertools, a library that ships with itertools recipes and more:
> pip install more_itertools
Use:
import more_itertools as mit
a = [None, None, None, 1, 2, 3, 4, 5]
mit.first_true(a, pred=lambda x: x is not None)
# 1
a = [None, None, None]
mit.first_true(a, default="All are None", pred=lambda x: x is not None)
# 'All are None'
Why use the predicate?
"First non-None" item is not the same as "first True" item, e.g. [None, None, 0] where 0 is the first non-None, but it is not the first True item. The predicate allows first_true to be useable, ensuring any first seen, non-None, falsey item in the iterable is still returned (e.g. 0, False) instead of the default.
a = [None, None, None, False]
mit.first_true(a, default="All are None", pred=lambda x: x is not None)
# 'False'
When the items in your list are expensive to calculate such as in
first_non_null = next((calculate(x) for x in my_list if calculate(x)), None)
# or, when receiving possibly None-values from a dictionary for each list item:
first_non_null = next((my_dict[x] for x in my_list if my_dict.get(x)), None)
then you might want to avoid the repetitive calculation and simplify to:
first_non_null = next(filter(bool, map(calculate, my_list)), None)
# or:
first_non_null = next(filter(bool, map(my_dict.get, my_list)), None)
Thanks to the usage of a generator expression, the calculations are only executed for the first items until a truthy value is generated.
Adapt from the following (you could one-liner it if you wanted):
values = (a, b, c, d)
not_None = (el for el in values if el is not None)
value = next(not_None, None)
This takes the first non None value, or returns None instead.
First of all want to mention that such function exists in SQL and is called coalesce. Found no such thing in Python so made up my own one, using the recipe of #alecxe.
def first_not_none(*values):
return next((v for v in values if v is not None), None)
Really helps in cases like this:
attr = 'title'
document[attr] = first_not_none(cli_args.get(attr), document_item.get(attr),
defaults_item.get(attr), '')
I need a list with the following behavior
>>> l = SparseList()
>>> l
[]
>>> l[2] = "hello"
>>> l
[ None, None, "hello"]
>>> l[5]
None
>>> l[4] = 22
>>> l
[ None, None, "hello", None, 22]
>>> len(l)
5
>>> for i in l: print i
None
None
"hello"
None
22
Although it can "emulated" via a dictionary, it's not exactly the same. numpy array can behave this way, but I don't want to import the whole numpy for something like this. Before coding it myself, I ask if something similar exists in the standard library.
Here's minimal code to pass your given examples (with indispensable adjustments: you expect weird spacing and quoting, 'None' to be printed out at the prompt without a print statement, etc etc):
class SparseList(list):
def __setitem__(self, index, value):
missing = index - len(self) + 1
if missing > 0:
self.extend([None] * missing)
list.__setitem__(self, index, value)
def __getitem__(self, index):
try: return list.__getitem__(self, index)
except IndexError: return None
__test__ = dict(allem='''
>>> l = SparseList()
>>> l
[]
>>> l[2] = "hello"
>>> l
[None, None, 'hello']
>>> print l[5]
None
>>> l[4] = 22
>>> l
[None, None, 'hello', None, 22]
>>> len(l)
5
>>> for i in l: print i
None
None
hello
None
22
''')
import doctest
doctest.testmod(verbose=1)
I imagine you'll want more (to support negative indices, slicing, and whatever else), but this is all your examples are implicitly specifying.
Dictionaries can be used as sparse lists.
Whilst they will not provide the characteristics you are after (as you are not actually after a sparse list, all the list elements are complete references to None in a dynamically-sized Array), they act as a textbook sparse array.
sparse_vars = [(0,"Hi"), (10000,"Bye"), (20000,"Try")]
sparse_list = {}
for var in sparse_vars:
sparse_list[var[0]] = var[1]
>>> print sparse_list
{0: 'Hi', 10000: 'Bye', 20000: 'Try'}
>>> print sparse_list[20000]
'Try'
The sparse_list package provides the behaviour OP asks for.