Given a list which contains both strings and None values, in which some of the strings have embedded newlines, I wish to split the strings with newlines into multiple strings and return a flattened list.
I've written code to do this using a generator function, but the code is rather bulky and I'm wondering if it's possible to do it more concisely using a list comprehension or a function from the itertools module. itertools.chain doesn't seem to be able to decline to iterate any non-iterable elements.
def expand_newlines(lines):
r"""Split strings with newlines into multiple strings.
>>> l = ["1\n2\n3", None, "4\n5\n6"]
>>> list(expand_newlines(l))
['1', '2', '3', None, '4', '5', '6']
"""
for line in lines:
if line is None:
yield line
else:
for l in line.split('\n'):
yield l
You can use yield from.
def expand(lines):
for line in lines:
if isinstance(line,str):
yield from line.split('\n')
elif line is None:
yield line
list(expand(l))
#['1', '2', '3', None, '4', '5', '6']
Here's a single line, but I think #Ch3steR's solution is more readable.
from itertools import chain
list(chain.from_iterable(i.splitlines() if i is not None and '\n' in i else [i]
for i in lines))
You could use itertools.chain if you did the following
import itertools
def expand_newlines(lines):
return itertools.chain.from_iterable(x.split("\n") if x else [None]
for x in lines)
Using more_itertools.collapse to flatten nested lists:
Given
import more_itertools as mit
lst = ["1\n2\n3", None, "7\n8\n9"]
Demo
list(mit.collapse([x.split("\n") if x else x for x in lst ]))
# ['1', '2', '3', None, '7', '8', '9']
more_itertools is a third-party package. Install via > pip install more_itertools.
If you might modify list inplace then you might do:
lst = ["1\n2\n3", None, "4\n5\n6"]
for i in range(len(lst))[::-1]:
if isinstance(lst[i], str):
lst[i:i+1] = lst[i].split("\n")
print(lst) # ['1', '2', '3', None, '4', '5', '6']
this solution utilize fact that you might not only get python's list slices, but also assign to them. It moves from right to left, as otherwise I would need to keep count of additional items, which would make it harder.
Similar to #blueteeth's answer but more concise by way of inverting the logic:
import itertools
chainfi = itertools.chain.from_iterable
def expand_newlines(lines):
r"""Split strings with newlines into multiple strings.
>>> l = ["1\n2\n3", None, "4\n5\n6"]
>>> list(expand_newlines(l))
['1', '2', '3', None, '4', '5', '6']
"""
return chainfi([None] if l is None else l.split('\n') for l in lines)
None is the special case so that's what we should be checking for.
This is concise enough that I wouldn't even bother writing a function for it—I just kept it in the function to confirm it works via doctest.
I have a list with (int, str)
x = ['2', '3/4', '+', '4', '3/5']
I have to parse int (here its 2and 4). After I should parse '3/4' and then '+'
Can you help me find a way how can I parse int from list like that?
I tried "try, except", but it didn't work.
I can't use index, because in another list may be like
x = ['3/4', '+', '1', '7/10']
This is the way to go:
x = ['2', '3/4', '+', '4', '3/5']
g = []
for i in x:
try:
g.append(int(i))
except ValueError:
pass
print(g) # [2, 4]
What you tried is not a try-except block. It is an if statement which failed because '4' is not an int instance (4 is, notice the quotes); it is a string.
There are of course other ways to do it too as mentioned in the comments. I only went for the try-except because you mentioned it in your comments.
I want to make a function that takes in a student's test answers and an answer key and return a list that states if they got the answer right or wrong using 1's and 0's.
Here is my code
answerkey='ABCABCDDD'
student11='BBCCBCDDD'
def check_answers(X='student',Y='answer key'):
result=[]
for i in range(len(X)):
for o in range(len(Y)):
if i==o:
result.append('1')
else:
result.append('0')
return result
print(check_answers(student11,answerkey))
My output should give me:
['0','1','1','0','1','1','1','1','1']
instead it gives me this:
['1', '0', '0', '0', '0', '0', '0', '0', '0']
What am i doing wrong?
This can all be simplified by using a comprehension and zip:
>>> ['1' if x == y else '0' for x, y in zip(answerkey, student11)]
['0', '1', '1', '0', '1', '1', '1', '1', '1']
However, to address your problem, you actually are doing an unnecessary extra step with that inner loop. It is an extra inefficiency that you don't need. Rely on the fact that your two lists are equal, so iterate over one, and just compare against the other list. Use enumerate, so that in each iteration you have access to the index and value.
Also, you are not using your default keyword arguments properly. You are assigning them as a string by default and not really doing anything with a default case for that kind of assignment, so remove it.
Observe:
answerkey='ABCABCDDD'
student11='BBCCBCDDD'
def check_answers(X, Y):
result=[]
for i, v in enumerate(answerkey):
if v == student11[i]:
result.append('1')
else:
result.append('0')
return result
print(check_answers(student11,answerkey))
Output:
['0', '1', '1', '0', '1', '1', '1', '1', '1']
This is happening because you o and i are counting through a range i.e. they are counting 0,1,2...len(X) and you are also iterating through every value of Y for each value of X instead you should compare variables in the list of the same index
answerkey='ABCABCDDD'
student11='BBCCBCDDD'
def check_answers(X,Y):
result=[]
for i in range(len(X)):
if X[i] == Y[i]:
result.append('1')
else:
result.append('0')
return result
print(check_answers(student11,answerkey))
In this algorithm it iterates through the length of X and compares each vallue of the list with the same indices e.g. it checks if X[0] = Y[0]
Also in your function you make X='student' and Y='answer key' this is fine because you only have two parameters and it overrides them but if you were to add more later this could cause a problem, if your intention was to just have these as comments I suggest just putting a #X = student and Y = answer key underneath it is cleaner
If you want to figure out what you're doing wrong, I've strategically put in 2 print statements that will show you what the issue is with your code.
def check_answers(X='student',Y='answer key'):
result=[]
print('X: %d, Y: %d' % (len(X), len(Y)))
for i in range(len(X)):
for o in range(len(Y)):
print('i: %d, o: %d' % (i, o))
if i==o:
result.append('1')
else:
result.append('0')
return result
Just in addition for other answers I'd recommend you use map function
answerkey='ABCABCDDD'
student11='BBCCBCDDD'
def check_answers(X='student',Y='answer key'):
return map(lambda a,b:a==b and '1' or '0',X,Y)
print(check_answers(answerkey,student11))
If i have a few lists
I am able to combine list 1 with list 2 however, I have not managed to succeed to combine the other lists.
def alternator():
iets = []
for i in range(len(list2)):
something += [list1[i]]
something +=[list2[i]]
result = something
result_weaver(result)
def result(x):
list31 = list3
if len(list3) < len(x) :
while len(list31) != len(x):
list31 += '-'
I decided to add '-' in order to make sure the lengths of both lists were equal, so the for loop could go to work.
Does anyone have better ideas on how to program this ?
Use itertools.zip_longest() here:
try:
from itertools import zip_longest
except ImportError:
# Python 2
from itertools import izip_longest as zip_longest
def alternate(list1, list2):
return [v for v in sum(zip_longest(list1, list2), ()) if v is not None]
The zip_longest() call adds None placeholders (similar to your own attempt to add - characters), which we need to remove again from the sum() output after zipping.
Demo:
>>> alternate(list1, list2)
['1', '5', '2', '6', '3', '7', '8']
>>> alternate(alternate(list1, list2), list3)
['1', '9', '5', '2', '6', '3', '7', '8']
I very often run into the need to split a sequence into the two subsequences of elements that satisfy and don't satisfy a given predicate (preserving the original relative ordering).
This hypothetical "splitter" function would look something like this in action:
>>> data = map(str, range(14))
>>> pred = lambda i: int(i) % 3 == 2
>>> splitter(data, pred)
[('2', '5', '8', '11'), ('0', '1', '3', '4', '6', '7', '9', '10', '12', '13')]
My question is:
does Python already have a standard/built-in way to do this?
This functionality is certainly not difficult to code (see Addendum below), but for a number of reasons, I'd prefer to use a standard/built-in method than a self-rolled one.
Thanks!
Addendum:
The best standard function I've found so far for handling this task in Python is itertools.groupby. To use it for this particular task however, it is necessary to call the predicate function twice for each list member, which I find annoyingly silly:
>>> import itertools as it
>>> [tuple(v[1]) for v in it.groupby(sorted(data, key=pred), key=pred)]
[('0', '1', '3', '4', '6', '7', '9', '10', '12', '13'), ('2', '5', '8', '11')]
(The last output above differs from the desired one shown earlier in that the subsequence of elements that satisfy the predicate comes last rather than first, but this is very minor, and very easy to fix if needed.)
One can avoid the redundant calls to the predicate (by doing, basically, an "inline memoization"), but my best stab at this gets pretty elaborate, a far cry from the simplicity of splitter(data, pred):
>>> first = lambda t: t[0]
>>> [zip(*i[1])[1] for i in it.groupby(sorted(((pred(x), x) for x in data),
... key=first), key=first)]
[('0', '1', '3', '4', '6', '7', '9', '10', '12', '13'), ('2', '5', '8', '11')]
BTW, if you don't care about preserving the original ordering, sorted's default sort order gets the job done (so the key parameter may be omitted from the sorted call):
>>> [zip(*i[1])[1] for i in it.groupby(sorted(((pred(x), x) for x in data)),
... key=first)]
[('0', '1', '3', '4', '6', '7', '9', '10', '12', '13'), ('2', '5', '8', '11')]
I know you said you didn't want to write your own function, but I can't imagine why. Your solutions involve writing your own code, you just aren't modularizing them into functions.
This does exactly what you want, is understandable, and only evaluates the predicate once per element:
def splitter(data, pred):
yes, no = [], []
for d in data:
if pred(d):
yes.append(d)
else:
no.append(d)
return [yes, no]
If you want it to be more compact (for some reason):
def splitter(data, pred):
yes, no = [], []
for d in data:
(yes if pred(d) else no).append(d)
return [yes, no]
Partitioning is one of those itertools recipes that does just that. It uses tee() to make sure it's iterating the collection in one pass despite the multiple iterators, the builtin filter() function to grab items that satisfies the predicate as well as filterfalse() to get the opposite effect of the filter. This is as close as you're going to get at a standard/builtin method.
def partition(pred, iterable):
'Use a predicate to partition entries into false entries and true entries'
# partition(is_odd, range(10)) --> 0 2 4 6 8 and 1 3 5 7 9
t1, t2 = tee(iterable)
return filterfalse(pred, t1), filter(pred, t2)
In more_itertools there is a function called partition, which does exactly what topicstarter asked for.
from more_itertools import partition
numbers = [1, 2, 3, 4, 5, 6, 7]
predicate = lambda x: x % 2 == 0
predicate_false, predicate_true = partition(predicate, numbers)
print(list(predicate_false), list(predicate_true))
The result is [1, 3, 5, 7] [2, 4, 6].
If you don't care about efficiency, I think groupby (or any "putting data into n bins" functions) has some nice correspondence,
by_bins_iter = itertools.groupby(sorted(data, key=pred), key=pred)
by_bins = dict((k, tuple(v)) for k, v in by_bins_iter)
You can then get to your solution by,
return by_bins.get(True, ()), by_bins.get(False, ())
A slight variation of one of the OP's implementations and another commenter's implementation above using groupby:
groups = defaultdict(list, { k : list(ks) for k, ks in groupby(items, f) })
groups[True] == the matching items, or [] if none returned True
groups[False] == the non-matching items, or [] if none returned False
Sadly, as you point out, groupby requires that the items be sorted by the predicate first, so if that's not guaranteed, you need this:
groups = defaultdict(list, { k : list(ks) for k, ks in groupby(sorted(items, key=f), f) })
Quite a mouthful, but it is a single expression that partitions a list by a predicate using only built-in functions.
I don't think you can just use sorted without the key parameter, because groupby creates a new group when it hits a new value from the key function. So sorted will only work if the items sort naturally by the predicate provided.
As a slightly more general solution to partitioning, consider grouping. Consider the following function, inspired by clojure's group-by function.
You give it a collection of items to group, and a function that will be used to group them. Here's the code:
def group_by(seq, f):
groupings = {}
for item in seq:
res = f(item)
if res in groupings:
groupings[res].append(item)
else:
groupings[res] = [item]
return groupings
For the OP's original case:
y = group_by(range(14), lambda i: int(i) % 3 == 2)
{False: [0, 1, 3, 4, 6, 7, 9, 10, 12, 13], True: [2, 5, 8, 11]}
A more general case of grouping elements in a sequence by string length:
x = group_by(["x","xx","yy","zzz","z","7654321"], len)
{1: ['x', 'z'], 2: ['xx', 'yy'], 3: ['zzz'], 7: ['7654321']}
This can be extended to many cases, and is a core functionality of functional languages. It works great with the dynamically typed python, as the keys in the resulting map can be any type. Enjoy!
Reducing the iterable into the 2 partitions using functools.reduce you can get rid of the key function:
import functools
functools.reduce(
lambda tf, x: (tf[0], [*tf[1], x]) if pred(x) else ([*tf[0], x], tf[1]),
data,
([], []),
)
>>> (['0', '1', '3', '4', '6', '7', '9', '10', '12', '13'], ['2', '5', '8', '11'])