I have two iterators, which consists of a "list" that looks something like this:
[[1, 2, 3, 4, 5, 6],
[2, 4, 6, 8, 10, 12],
[3, 5, 8, 6, 1, 19],
[5, 9, 1, 9, 4, 6]]
Or, that is what it will look like if I just ran a for loop over them.
The reason for the iterator and not a list per se is due to memory. The true lists/arrays are way larger, this is just an example.
What I need to do is take one list and sum the columns of each index inside the list for all "outside" indices and then add them together for both lists like sum(list1) + sum(list2).
So basically:
list1: list2:
[[1, 2, 3, 4, 5, 6], [[5, 4, 3, 2, 1, 9],
[2, 4, 6, 8, 10, 12], [6, 3, 8, 1, 1, 6],
[3, 5, 8, 6, 1, 19], [1, 3, 2, 8, 2, 3],
[5, 9, 1, 9, 4, 6]] [5, 2, 9, 4, 2, 5]]
=> =>
[11, 20, 18, 20, 43] [17, 12, 22, 15, 23]
=>
[28, 32, 40, 35, 66]
So I iterate over the two lists, and for each list I need to sum the columns, and then in the end at the columns of the final two lists into one combined list.
I know how to do this if it were just regular lists, but since this is iterators/generators (don't know the correct term) I am really not sure how it is done.
You can use this to sum each one without loading everything into memory:
def sumIter(iter):
result = [0, 0, 0, 0, 0, 0] #Assuming there are always 6 items in each sub-list
for list in iter:
result = [(result[i] + list[i]) for i in range(6)]
And then:
sum1 = sumIter(iter1)
sum2 = sumIter(iter2)
result = [(sum1[i] + sum2[i]) for i in range(6)]
Using zip
Ex:
l1 = [
[1, 2, 3, 4, 5, 6],
[2, 4, 6, 8, 10, 12],
[3, 5, 8, 6, 1, 19],
[5, 9, 1, 9, 4, 6]
]
l2 = [
[5, 4, 3, 2, 1, 9],
[6, 3, 8, 1, 1, 6],
[1, 3, 2, 8, 2, 3],
[5, 2, 9, 4, 2, 5]
]
l1 = (sum(i) for i in zip(*l1))
l2 = (sum(i) for i in zip(*l2))
print( [sum(i) for i in zip(l1, l2)] )
Output:
[28, 32, 40, 42, 26, 66]
Using reduce since row can be added in numpy array.
reduce is an build-in function in python2
import numpy as np
from functools import reduce # only in python3
def sumup(one_row, another_row):
return one_row + another_row
test_list = np.array([[1, 2, 3, 4, 5, 6],
[2, 4, 6, 8, 10, 12],
[3, 5, 8, 6, 1, 19],
[5, 9, 1, 9, 4, 6]])
reduce(sumup, test_list)
Output
array([11, 20, 18, 27, 20, 43])
using numpy.sum
import numpy as np
l1 = np.sum([[1, 2, 3, 4, 5, 6], [2, 4, 6, 8, 10, 12], [3, 5, 8, 6, 1, 19], [5, 9, 1, 9, 4, 6]], axis=0)
l2 = np.sum([[5, 4, 3, 2, 1, 9],[6, 3, 8, 1, 1, 6], [1, 3, 2, 8, 2, 3],[5, 2, 9, 4, 2, 5]], axis=0)
print(l1 + l2)
Output
[28 32 40 42 26 66]
Related
This is what I get by running train_test_split
In [1]:train_test_split([1,2,3,4,5,6,7,8,9,10],test_size = 0.2)
Out[1]: [[10, 3, 6, 5, 4, 2, 7, 9], [8, 1]]
However, what I want is a contiguous set, i.e.
[[1, 2, 3, 4, 5, 6, 7, 10], [8, 9]]
or
[[1, 2, 3, 4, 5, 8, 9, 10], [6, 7]]
or
[[1, 2, 5, 6, 7, 8, 9, 10], [3, 4]]
****** Please note that the following is also considered contiguous**
[[2, 3, 4, 5, 6, 7, 8, 9], [10, 1]]
How can I do this ?
Can you try the following:
import random
def custom_train_test_split(X, test_size=0.2):
temp = X.copy()
split_size = int(len(temp) * test_size)
start = random.randint(0, len(X) - split_size)
end = start + split_size
test = temp[start:end]
del temp[start:end]
return [temp, test]
X = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
print(custom_train_test_split(X, test_size=0.2))
print(custom_train_test_split(X, test_size=0.2))
print(custom_train_test_split(X, test_size=0.2))
Output:
[[1, 2, 5, 6, 7, 8, 9, 10], [3, 4]]
[[1, 2, 3, 4, 5, 8, 9, 10], [6, 7]]
[[3, 4, 5, 6, 7, 8, 9, 10], [1, 2]]
This question already has answers here:
How do I split a list into equally-sized chunks?
(66 answers)
Closed 1 year ago.
Example: I have a list:
[8, 3, 4, 1, 5, 9, 6, 7, 2]
And I need to make it look like this but without using numpy.array_split():
[[8, 3, 4], [1, 5, 9], [6, 7, 2]]
How can I do it? Not only for this one case, but when I have 4 elements, I want to have 2 and 2, (9 - 3,3,3 and 16 - 4,4,4,4) etc.
You can get the square root of the list's length then split it using a list comprehension. This will work for lists with the length of 4, 9, 16, ...:
lst = [8, 3, 4, 1, 5, 9, 6, 7, 2]
lst2 = [8, 3, 4, 1]
def split_equal(lst):
len_ = len(lst)
# returns emtpy list, if the list has no item.
if len_ == 0:
return []
n = int(len_ ** 0.5)
return [lst[i:i + n] for i in range(0, len_, n)]
output:
[[8, 3, 4], [1, 5, 9], [6, 7, 2]]
[[8, 3], [4, 1]]
You can use that:
def splitter(inlist):
n = len(inlist)
m = int(n ** 0.5)
if m*m != n:
raise Exception("")
return [[inlist[i+j] for j in range(m)] for i in range(m)]
print(splitter([8, 3, 4, 1]))
print(splitter([8, 3, 4, 1, 5, 9, 6, 7, 2]))
print(splitter([8, 3, 4, 1, 5, 9, 6, 7, 2, 8, 3, 4, 1, 5, 9, 6]))
Result:
[[8, 3], [3, 4]]
[[8, 3, 4], [3, 4, 1], [4, 1, 5]]
[[8, 3, 4, 1], [3, 4, 1, 5], [4, 1, 5, 9], [1, 5, 9, 6]]
Carefull, it will crash if the square of the len of input list is not integer.
def equal_array_split(arr, split_arr_len):
array_length = len(arr)
if array_length % split_arr_len == 0:
return [arr[i:i+split_arr_len] for i in range(0,array_length,split_arr_len)]
else:
return "Invalid split array length!!"
print(equal_array_split([1,2,3,4,5,6,7,8,9],3))
print(equal_array_split([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],4))
print(equal_array_split([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],8))
print(equal_array_split([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16],2))
Output:
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
[[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
[[1, 2, 3, 4, 5, 6, 7, 8], [9, 10, 11, 12, 13, 14, 15, 16]]
[[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12], [13, 14], [15, 16]]
You can slice the list with a list comprehension. Assuming the input is a square number length:
import numpy as np
arr = [8, 3, 4, 1, 5, 9, 6, 7, 2]
n = int(np.sqrt(len(arr)))
result = [arr[i*n:(i+1)*n] for i in range(int(n))]
the split value being a square
list = [8, 3, 4, 1, 5, 9, 6, 7, 2]
result = []
N = split_value #split_value being the value required to split the list
for i in range(0,len(list),N):
result.append(list[i:i+N])
print(result)
Whitout numpy....
a = [8, 3, 4, 1, 5, 9, 6, 7, 2]
splitedSize = 3
a_splited = [a[x:x+splitedSize] for x in range(0, len(a), splitedSize)]
print(a_splited)
I need to simulate a certain scenario.
So I'm defining a variable which generates a loop of a random number of integers.
I get for example:
list = [2, 35, 4, 8, 56, 10]
Then, I'm generating this random list 50 times through another loop and I store the data into a dictionary to visualize a Pandas Dataframe.
data_dict = {'random_numers': list}
data_dict_pd = pd.DataFrame(data_dict)
So I get for example this:
[1, 16, 6, 6, 1, 10]
[3, 8, 4, 4, 1, 20, 7, 25, 12]
[14, 8, 16, 4, 11, 18, 5, 15, 24, 2, 15, 5]
[7, 24, 1, 14]
[5, 14, 19, 24, 1]
... 50 times.
Now, I need to create another column enumerating each number in each list of elements, to get the following, based on the previous results:
[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6, 7, 8, 9]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
[1, 2, 3, 4]
[1, 2, 3, 4, 5]
...50 times.
Actually, came up with the following but it's wrong:
new_list = []
for index in enumerate(list)
new_list.append(index)
Any better idea?
I would strongly suggest changing the name of your list, as list is used for Python lists.
Assuming you change it to l, I would use:
l = [2, 35, 4, 8, 56, 10]
new_list = []
for i in range(1, len(l) + 1):
new_list.append(i)
print(new_list)
Output:
[1, 2, 3, 4, 5, 6]
If what you need is to iterate a list of lists, and incorporating #deceze suggestions (provided that you don't rename list):
lists = [
[1, 16, 6, 6, 1, 10],
[3, 8, 4, 4, 1, 20, 7, 25, 12],
[14, 8, 16, 4, 11, 18, 5, 15, 24, 2, 15, 5],
[7, 24, 1, 14],
[5, 14, 19, 24, 1]
]
new_lists = [list(range(1, len(lst) + 1)) for lst in lists]
print(new_lists)
Output:
[[1, 2, 3, 4, 5, 6], [1, 2, 3, 4, 5, 6, 7, 8, 9], [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], [1, 2, 3, 4], [1, 2, 3, 4, 5]]
new_list = []
for index in range(1,len(list)+1)
new_list.append(index)
should be used in place of enumerator,as it will going to return the pair.
Thank you,
There are 3D-array in my data. I just want to slice 3D-array 2 by 2 by 2 with overlapped interval in Python.
Here is an example for 2D.
a = [1, 2, 3, 4;
5, 6, 7, 8]
Also, this is what I expected after slicing the array in 2 by 2.
[1, 2; [2, 3; [3, 4;
5, 6] 6, 7] 7, 8]
In 3D,
[[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]],
[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]],
[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]]
Like this,(maybe not exactly..)
[1, 2 [2, 3
4, 5] 5, 6] ...
[1, 2 [2, 3
4, 5] 5, 6]
I think, by using np.split, I could slice the array, but without overlapped. Please give me some helpful tips.
You should have a look at numpy.ndarray.strides and numpy.lib.stride_tricks
Tuple of bytes to step in each dimension when traversing an array.
The byte offset of element (i[0], i[1], ..., i[n]) in an array a is:
offset = sum(np.array(i) * a.strides)
See also the numpy documentation
Following a 2D example using strides:
x = np.arange(20).reshape([4, 5])
>>> x
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]])
>>> from numpy.lib import stride_tricks
>>> stride_tricks.as_strided(x, shape=(3, 2, 5),
strides=(20, 20, 4))
...
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9]],
[[ 5, 6, 7, 8, 9],
[ 10, 11, 12, 13, 14]],
[[ 10, 11, 12, 13, 14],
[ 15, 16, 17, 18, 19]]])
Also see this question on Stackoverflow, where this example is from, to increase your understanding.
This is list of tile placements. Each integer stands for an id of a tile. Each time an integer is added to a new list it means that a new tile is placed. When a tile is removed, the last integer is removed from a new list. I want that every time a tile is placed the list to be unique. A list dont have to be unique, when a tile is removed. The code is placing these tiles in a for loop. So for this example, the last list of the lists is wrong, because it wasn't unique when a tile was placed. Is there a way to exclude numbers which will make the new list not unique. So for this example is there a way to exclude the id 18, before it adds to the list.
I know this is a very vague question, but I am new to python and can't make the code of this assignment easier. I hope someone could help me with this vague question
[[1, 2, 3, 13, 4, 5, 6, 7, 17],
[1, 2, 3, 13, 4, 5, 6, 7, 17, 8],
[1, 2, 3, 13, 4, 5, 6, 7, 17, 8, 15],
[1, 2, 3, 13, 4, 5, 6, 7, 17, 8, 15, 9],
[1, 2, 3, 13, 4, 5, 6, 7, 17, 8, 15, 9, 10],
[1, 2, 3, 13, 4, 5, 6, 7, 17, 8, 15, 9, 10, 18],
[1, 2, 3, 13, 4, 5, 6, 7, 17, 8, 15, 9, 11],
[1, 2, 3, 13, 4, 5, 6, 7, 17, 8, 15, 9, 11, 18],
[1, 2, 3, 13, 4, 5, 6, 7, 17, 8, 15, 9, 10],
[1, 2, 3, 13, 4, 5, 6, 7, 17, 8, 15, 9, 10, 18]]
The lists must be in this order. So for example i have had these lists:
[[1, 2, 3, 13, 4, 5, 6, 7, 17],
[1, 2, 3, 13, 4, 5, 6, 7],
[1, 2, 3, 13, 4, 5, 6, 7, 8],
[1, 2, 3, 13, 4, 5, 6, 7],
[1, 2, 3, 13, 4, 5, 6, 7, 19],
[1, 2, 3, 13, 4, 5, 6, 7]]
I want to exlude the ids 17,8,19
So for [1, 2, 3, 13, 4, 5, 6, 7] the output must look like this ( id ont care if the output is a list or integers)
[17,8,19]
But when i have this list [1, 2, 3, 13, 4, 5, 6] in lists
[[1, 2, 3, 13, 4, 5, 6, 7, 17],
[1, 2, 3, 13, 4, 5, 6],
[1, 2, 3, 13, 4, 5, 6, 7, 8],
[1, 2, 3, 13, 4, 5, 6, 7],
[1, 2, 3, 13, 4, 5, 6, 7, 19],
[1, 2, 3, 13, 4, 5, 6, 7]]
The output is this:
[7]
I hope this will make it more clear.
I tried with itertools and collections- pass a list, a list element index and to be added value to the adder function if uniquness is kept the adder will add that passed value otherwise return intact list.compare_func return TRUE if list is unique using all.
import collections,itertools
compare_func = lambda x, y: collections.Counter(x) != collections.Counter(y)
lst = [[1, 2, 3],[1, 2, 3,4]]
def adder(mylist,indx,val):
mylist[indx].append(val)
if all([compare_func(*i) for i in list(itertools.combinations(lst,2))]):
print "Added item"
else:
print "Did not add item"
mylist[indx].pop()
return mylist
Now run print adder(lst,0,4)
Output-
Did not add item
[[1, 2, 3], [1, 2, 3, 4]]
But if run
print adder(lst,1,4)
Output-
Added item
[[1, 2, 3], [1, 2, 3, 4, 4]]
EDIT
After OP cleared question i added this portion-
Try using set as below-
import collections,itertools
data = [[1, 2, 3, 13, 4, 5, 6, 7, 17],
[1, 2, 3, 13, 4, 5, 6, 7],
[1, 2, 3, 13, 4, 5, 6, 7, 8],
[1, 2, 3, 13, 4, 5, 6, 7],
[1, 2, 3, 13, 4, 5, 6, 7, 19],
[1, 2, 3, 13, 4, 5, 6, 7]]
interscntion = set.intersection(*map(set,data))
d = collections.Counter([i for j in data for i in j if i not in list(interscntion)])
if len(set(it[1] for it in d.most_common()))>1:
print [max(d.most_common(),key=lambda x:x[1])[0]]
else:
print [j[0] for j in d.most_common()]
Output-
[8, 17, 19]
Here you go:
def tiles(arrOfArrs):
final = []
for x in arrOfArrs:
final += x
return list(set(final))
Keep another list of lists and at each position add the tiles that appear at that index in any of the primary lists.