Related
I have two boolean Numpy arrays of boolean indicators:
v v v
A = np.array([0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1], dtype=bool)
B = np.array([1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1], dtype=bool)
^ ^ ^
Moving from left to right, I would like to isolate the first true A indicator, then the next true B indicator, then the next true A indicator, then the next true B indicator, etc. to end up with:
v v v
>>>> A_result = [0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]
B_result = [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1]
^ ^ ^
I have a feeling I could create a betweenAB array indicating all the places where A==1 is followed by B==1:
v v v
betweenAB = [0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1]
^ ^ ^
then take the start and end indices of each run, but I am still somewhat of a beginner when it comes to Numpy and am not sure how I might do that.
I'm looking for a fully vectorized approach as there are thousands of these arrays in my application each containing thousands of elements. Any help would be much appreciated.
This can barely be done efficiently with Numpy (probably not possible efficiently without loops), but easily and efficiently with the Numba's JIT. This is mainly due to the rather sequential nature of the applied operation.
Here is an example in Numba:
import numpy as np
import numba as nb
nb.jit('UniTuple(bool[::1],2)(bool[::1],bool[::1])')
def compute(A, B):
assert len(A) == len(B)
n = len(A)
i = 0
resA = np.zeros(n, dtype=bool)
resB = np.zeros(n, dtype=bool)
while i < n:
while i < n and A[i] == 0:
resA[i] = 0
i += 1
if i < n:
resA[i] = 1
if B[i] == 1:
resB[i] = 1
i += 1
continue
i += 1
while i < n and B[i] == 0:
resB[i] = 0
i += 1
if i < n:
resB[i] = 1
i += 1
return resA, resB
I have a 1D numpy array of 1's and 0's. I need to change it to an array according to these conditions.
If the number of 0's between two 1's is less than 3, all of those 0's should be set to 1.
In the resulting array, if the number of consecutive 1's are less than 4, all of those 1's should be set to 0.
i.e if my array is
[0,1,1,1,0,0,0,1,1,0,0,1,1,0,0,0,0,1,0]
it should be changed to
[0,1,1,1,0,0,0,1,1,1,1,1,1,0,0,0,0,1,0]
and then to
[0,0,0,0,0,0,0,1,1,1,1,1,1,0,0,0,0,0,0]
I have tried using moving average to no avail. How do I write a general function, preferably with n instead of 3 and m instead of 4?
If you use core Python?
l = [0,1,1,1,0,0,0,1,1,0,0,1,1,0,0,0,0,1,0]
def split(l):
res = []
subres = [l[0]]
for i in range(len(l) - 1):
if l[i] == l[i + 1]:
subres.append(l[i + 1])
else:
res.append(subres)
subres = [l[i + 1]]
res.append(subres)
return(res)
def setall(l, val):
for i in range(len(l)):
l[i] = val
return(l)
def recode(l, changeval, replaceval, lenlimit):
for i in range(len(l) - 1):
el = l[i + 1]
if (el[0] == changeval) & (len(el) < lenlimit) & (i < (len(l) - 2)):
el = setall(el, replaceval)
l[i + 1] = el
return(l)
def flatten(l):
res = []
for el in l:
res.extend(el)
return(res)
# starting list
print(l)
# step 1
want1 = split(l)
want1 = flatten(recode(want1, 0, 1, 3))
print(want1)
# step 2
want2 = split(want1)
want2 = flatten(recode(want2, 1, 0, 4))
print(want2)
#[0, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0]
#[0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0]
#[0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
Rule 30 is a one dimensional cellular automaton where only the cells in the previous generation are considered by the current generation. There are two states that a cell can be in: 1 or 0. The rules for creating the next generation are represented in the row below, and depend on the cell immediately above the current cell, as well as it's immediate neighbours.
The cellular automaton is applied by the following rule (using bitwise operators):
left_cell ^ (central_cell | right_cell)
This rule forms the table below:
Now I tried to implement these rules into Python, using numpy. I defined an initial state that accepts width as a parameter and produces an initial row of zeros with 1 in the middle.
def initial_state(width):
initial = np.zeros((1, width), dtype=int)
if width % 2 == 0:
initial = np.insert(initial, int(width / 2), values=0, axis=1)
initial[0, int(width / 2)] = 1
return initial
else:
initial[0, int(width / 2)] = 1
return initial
The function below just produces the second generation given an initial row. How do I create a for loop that keeps producing new generations until the first element of the last bottom row becomes 1?
def rule30(array):
row1 = np.pad(array,[(0,0), (1,1)], mode='constant')
next_row = array.copy()
for x in range(1, array.shape[0]+1):
for y in range(1, array.shape[1]+1):
if row1[x-1][y-1] == 1 ^ (row1[x-1][y] == 1 or row1[x-1][y+1] == 1):
next_row[x - 1, y - 1] = 1
else:
next_row[x - 1, y - 1] = 0
return np.concatenate((array, next_row))
For example, if the input is
A = [0, 0, 0, 1, 0, 0, 0]
The output should be
>>> print(rule30(A))
[[0, 0, 0, 1, 0, 0, 0],
[0, 0, 1, 1, 1, 0, 0],
[0, 1, 1, 0, 0, 1, 0],
[1, 1, 0, 1, 1, 1, 1]]
Here is the code based on string representations and lookup. It does use some of the ideas from the comments above. Besides I added padding for handling edge cells - the conditions were unclear about that. Also note that your proposed patterns table is not symmetric. Compare new states for '110' and '011'.
def rule30(a):
patterns = {'111': '0', '110': '0', '101': '0', '100': '1',
'011': '1', '010': '1', '001': '1', '000': '0', }
a = '0' + a + '0' # padding
return ''.join([patterns[a[i:i+3]] for i in range(len(a)-2)])
a = '0001000'
result = [list(map (int, a))]
while a[0] != '1':
a = rule30(a)
result.append (list(map (int, a)))
print (result) # list of lists
print (np.array(result)) # np.array
list of lists:
[[0, 0, 0, 1, 0, 0, 0], [0, 0, 1, 1, 1, 0, 0], [0, 1, 1, 0, 0, 1, 0], [1, 1, 0, 1, 1, 1, 1]]
np.array:
array([[0, 0, 0, 1, 0, 0, 0],
[0, 0, 1, 1, 1, 0, 0],
[0, 1, 1, 0, 0, 1, 0],
[1, 1, 0, 1, 1, 1, 1]])
Method 1 - Numpy
You could achieve this using the following slight modification to your current code - alter the return value of rule30 to return np.array(next_row). Then you can use the following function:
def apply_rule(n):
rv = initial_state(n)
while rv[-1][0] == 0:
rv = np.append(rv, rule30(rv[-1].reshape(1,-1)), axis=0)
return rv
Usage:
>>> apply_rule(7)
array([[0, 0, 0, 1, 0, 0, 0],
[0, 0, 1, 1, 1, 0, 0],
[0, 1, 1, 0, 0, 1, 0],
[1, 1, 0, 1, 1, 1, 1]])
Or plotted:
>>> plt.imshow(apply_rule(7), cmap='hot')
Method 2 - Lists
Alternatively, you could use the following solution without using numpy, which uses a few functions to apply the Rule 30 logic across each triple in each padded list, until the stop-condition is met.
Code:
def rule(t):
return t[0] ^ (t[1] or t[2])
def initial_state(width):
initial = [0]*width
if width%2:
initial[width // 2] = 1
else:
initial.insert(width//2, 1)
return initial
def get_triples(l):
return zip(l,l[1:],l[2:])
def rule30(l):
return [rule(t) for t in get_triples([0] + l + [0])]
def apply_rule(width):
rv = [initial_state(width)]
while not rv[-1][0]:
rv.append(rule30(rv[-1]))
return rv
Usage:
>>> apply_rule(7)
[[0, 0, 0, 1, 0, 0, 0],
[0, 0, 1, 1, 1, 0, 0],
[0, 1, 1, 1, 0, 1, 0],
[1, 1, 1, 0, 0, 1, 1]]
>>> [''.join(str(y) for y in x) for x in apply_rule(7)]
['0001000',
'0011100',
'0111010',
'1110011']
Matplotlib visualisation (using either method):
import matplotlib.pyplot as plt
plt.figure(figsize=(10,6))
plt.imshow(apply_rule(250), cmap='hot')
So I have a list that looks a bit like this:
my_list = [0,1,1,1,0,0,1,0,1,0,1,1,0,0,0,1,0,1,1,0,1 ... 0,1,0]
it contains thousands of 0's and 1's basically. Im looking for a way to find similar (repeating) combinations of elements in it (10 next elements to be specific). So (for example) if there is a :
... 0,1,1,1,0,0,1,1,0,1 ...
combination and it appears more than once I would like to know where it is in my list (index) and how many times it repeats.
I need to check all possible combinations here, that is 1024 possibilities...
Here is a solution using regex:
import random
from itertools import product
import re
testlist = [str(random.randint(0,1)) for i in range(1000)]
testlist_str = "".join(testlist)
for i in ["".join(seq) for seq in product("01", repeat=10)]:
print(f'pattern {i} has {len(re.findall(i, testlist_str))} matches')
outputs:
pattern 0000000000 has 0 matches
pattern 0000000001 has 0 matches
pattern 0000000010 has 1 matches
pattern 0000000011 has 2 matches
pattern 0000000100 has 2 matches
pattern 0000000101 has 2 matches
....
It looks like a homework problem, so I don't want to give the solution at once, just hints.
Don't look at it literally. It's 0s and 1s, so you can look at them like at binary numbers.
Some hints:
1024 "patterns" become just numbers from 0 to 1023.
Checking for a pattern is making a number from those 10 digits.
Think how you would do that then.
More hints, more technical:
If you have one number pattern, e.g. from 0th to 9th element, you can get 1st to 10th pattern by taking 9-digit (from 1st index to 9th index) value (aka %512), "move" them left (*2) and add the 10th digit.
Make a dictionary or list of lists where key/index is the pattern number (0 to 1023) and list contains indexes of the start pattern.
I'll edit this answer later to provide an example solution but I gotta take a short break first.
Edit:
Customisable base and length with defaults for your case.
def find_patterns(my_list, base=2, pattern_size=10):
modulo_value = base ** (pattern_size-1)
results = [[] for _ in range(base ** pattern_size)]
current_value = 0
for index, elem in enumerate(a):
if index < pattern_size:
current_value = base*current_value + elem
elif index == pattern_size:
results[current_value].append(0)
if index >= pattern_size:
current_value = base*(current_value % modulo_value) + elem
results[current_value].append(index+1-pattern_size) #index of the first element in the pattern
return results
IIUC, you could do:
my_list = [0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 0]
w = 10
occurrences = {}
for i in range(len(my_list) - w + 1):
key = tuple(my_list[i:i+w])
occurrences.setdefault(key, []).append(i)
for pattern, indices in occurrences.items():
print(pattern, indices)
Output
(0, 1, 1, 1, 0, 0, 1, 0, 1, 0) [0]
(1, 1, 1, 0, 0, 1, 0, 1, 0, 1) [1]
(1, 1, 0, 0, 1, 0, 1, 0, 1, 1) [2]
(1, 0, 0, 1, 0, 1, 0, 1, 1, 0) [3]
(0, 0, 1, 0, 1, 0, 1, 1, 0, 0) [4]
(0, 1, 0, 1, 0, 1, 1, 0, 0, 0) [5]
(1, 0, 1, 0, 1, 1, 0, 0, 0, 1) [6]
(0, 1, 0, 1, 1, 0, 0, 0, 1, 0) [7]
(1, 0, 1, 1, 0, 0, 0, 1, 0, 1) [8]
(0, 1, 1, 0, 0, 0, 1, 0, 1, 1) [9]
(1, 1, 0, 0, 0, 1, 0, 1, 1, 0) [10]
(1, 0, 0, 0, 1, 0, 1, 1, 0, 1) [11]
(0, 0, 0, 1, 0, 1, 1, 0, 1, 0) [12]
(0, 0, 1, 0, 1, 1, 0, 1, 0, 1) [13]
(0, 1, 0, 1, 1, 0, 1, 0, 1, 0) [14]
Treat the elements as bits that can be converted to integers. The solution below converts the input list to integers, find number of occurrence of each integer and what index they can be found on.
import collections
x = [0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1]
as_int = []
# given the input above there is no pattern longer than 6 that occure more than once...
pattern_length = 6
# convert input to a list of integers
# can this be done in a nicer way, like skipping the string-conversion?
for s in range(len(x) - pattern_length+1) :
bitstring = ''.join([str(b) for b in x[s:s+pattern_length]])
as_int.append(int(bitstring,2))
# create a dict with integer as key and occurence as value
count_dict = collections.Counter(as_int)
# empty dict to store index for each integer
index_dict = {}
# find index for each integer that occur more than once
for key in dict(count_dict):
if count_dict[key] > 1:
indexes = [i for i, x in enumerate(as_int) if x == key]
index_dict[key] = indexes
#print as binary together with its index
for key, value in index_dict.items():
print('{0:06b}'.format(key), 'appears', count_dict[key], 'times, on index:', value)
Output:
101011 appears 2 times, on index: [6, 18]
010110 appears 2 times, on index: [7, 14]
You are given an integer array with N elements: d[0], d[1], ... d[N - 1].
You can perform AT MOST one move on the array: choose any two integers [L, R], and flip all the elements between (and including) the L-th and R-th bits. L and R represent the left-most and right-most index of the bits marking the boundaries of the segment which you have decided to flip.
What is the maximum number of 1-bits (indicated by S) which you can obtain in the final bit-string?
'Flipping' a bit means, that a 0 is transformed to a 1 and a 1 is transformed to a 0 (0->1,1->0).
Input Format: An integer N, next line contains the N bits, separated by spaces: d[0] d[1] ... d[N - 1]
Output: S
Constraints:
1 <= N <= 100000,
d[i] can only be 0 or 1 ,
0 <= L <= R < n ,
Sample Input:
8
1 0 0 1 0 0 1 0
Sample Output: 6
Explanation:
We can get a maximum of 6 ones in the given binary array by performing either of the following operations:
Flip [1, 5] ==> 1 1 1 0 1 1 1 0
Cleaned up and made Pythonic
arr1 = [1, 0, 0, 1, 0, 0, 1, 0]
arr2 = [1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1]
arr3 = [0,0,0,1,1,0,1,0,1,1,0,0,1,1,1]
def maximum_ones(arr):
"""
Returns max possible number of ones after flipping a span of bit array
"""
total_one = 0
net = 0
maximum = 0
for bit in arr:
if bit:
total_one += 1
net -= 1
else:
net += 1
maximum = max(maximum, net)
if net < 0:
net = 0
return total_one + maximum
print(maximum_ones(arr1))
print(maximum_ones(arr2))
print(maximum_ones(arr3))
Output:
6
14
11
If we want the L and R indices
Not so sure about this one. It can probably be made cleaner.
arr1 = [1, 0, 0, 1, 0, 0, 1, 0]
arr2_0 = [1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1]
arr2_1 = [1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1]
arr2_2 = [1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1]
arr3 = [0,0,0,1,1,0,1,0,1,1,0,0,1,1,1]
def maximum_ones(arr):
"""
Returns max possible number of ones after flipping a span of bit array
and the (L,R) indices (inclusive) of such a flip
"""
total_one = 0
net = 0
maximum = 0
L = R = 0
started_flipping = False
for i, bit in enumerate(arr):
if bit:
total_one += 1
net -= 1
else:
net += 1
if not started_flipping:
started_flipping = True
L = i
if net > maximum:
maximum = net
R = i
if net < 0:
net = 0
if i < R:
L = i
return (total_one + maximum, (L,R))
print(maximum_ones(arr1))
print(maximum_ones(arr2_0))
print(maximum_ones(arr2_1))
print(maximum_ones(arr2_2))
print(maximum_ones(arr3))
Output:
(6, (1, 5))
(14, (1, 16))
(14, (2, 16))
(14, (3, 16))
(11, (0, 2))
First Iteration
Here is what I had originally, if you want to see the evolution of the thought processes. Here, I was essentially transliterating what I came up with on paper.
Essentially, we traverse the array and start flipping bits (ok, not really), keeping track of cumulative flipped zeros and cumulative flipped ones in two separate arrays along with the total flipped ones in an integer counter. If the difference between flipped ones and zeroes at a given index - the "net" - drops below zero, we 'reset' the cumulative counts back at zero at that index (but nothing else). Along the way, we also keep track of the maximum net we've achieved and the index at which that occurs. Thus, the total is simply the total 1's we've seen, plus the net at the maximum index.
arr = [1, 0, 0, 1, 0, 0, 1, 0]
total_one = 0
one_flip = [0 for _ in range(len(arr))]
zero_flip = [0 for _ in range(len(arr))]
# deal with first element of array
if arr[0]:
total_one += 1
else:
zero_flip[0] = 1
maximum = dict(index=0,value=0) #index, value
i = 1
# now deal with the rest
while i < len(arr):
# if element is 1 we definitely increment total_one, else, we definitely flip
if arr[i]:
total_one += 1
one_flip[i] = one_flip[i-1] + 1
zero_flip[i] = zero_flip[i-1]
else:
zero_flip[i] = zero_flip[i-1] + 1
one_flip[i] = one_flip[i-1]
net = zero_flip[i] - one_flip[i]
if net > 0:
if maximum['value'] < net:
maximum['value'] = net
maximum['index'] = i
else: # net == 0, we restart counting our "net"
one_flip[i] = 0
zero_flip[i] = 0
i += 1
maximum_flipped = total_one - one_flip[maximum['index']] + zero_flip[maximum['index']]
Results:
print(total_one, -one_flip[maximum['index']], zero_flip[maximum['index']] )
print(maximum_flipped)
print('________________________________________________')
print(zero_flip, arr, one_flip, sep='\n')
print('maximum index', maximum['index'])
Output:
3 -1 4
6
________________________________________________
[0, 1, 2, 2, 3, 4, 4, 5]
[1, 0, 0, 1, 0, 0, 1, 0]
[0, 0, 0, 1, 1, 1, 2, 2]
maximum index 5
if arr = [1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1]
6 -4 12
14
________________________________________________
[0, 1, 2, 3, 3, 4, 5, 5, 6, 6, 7, 8, 9, 10, 10, 11, 12, 12]
[1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1]
[0, 0, 0, 0, 1, 1, 1, 2, 2, 3, 3, 3, 3, 3, 4, 4, 4, 5]
maximum index 16
Finally, if arr = [0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1]
8 0 3
11
________________________________________________
[1, 2, 3, 3, 3, 4, 4, 5, 5, 0, 1, 2, 2, 0, 0]
[0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1]
[0, 0, 0, 1, 2, 2, 3, 3, 4, 0, 0, 0, 1, 0, 0]
maximum index 2
Great, now tear it apart, people!
Traverse the whole array. Keep a count in the following way:
Do +1 for every 0 bit encountered.
Do -1 for every 1.
If this count reaches -ve at any stage, reset it to 0. Keep track of max value of this count. Add this max_count to number of 1's in input array. This will be your answer.
Code:
arr = [1, 0, 0, 1, 0, 0, 1, 0]
# I'm taking your sample case. Take the input the way you want
count,count_max,ones = 0,0,0
for i in arr:
if i == 1:
ones += 1
count -= 1
if i == 0:
count += 1
if count_max < count:
count_max = count
if count < 0:
count = 0
print (ones + count_max)
Small and simple :)