Related
For editors: this is NOT stripping all strings in an array but stripping the array itself
So suppose i have an array like this:
[[0, 1, 8, 4, 0, 0],
[1, 2, 3, 0, 0, 0],
[3, 2, 3, 0, 5, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0]]
I want a function stripArray(0, array) where the first argument is the "empty" value. After applying this function i want the returned array to look like this:
[[0, 1, 8, 4, 0],
[1, 2, 3, 0, 0],
[3, 2, 3, 0, 5]]
Values that were marked as empty (in this case 0) were stripped from the right and bottom sides. How would I go about implementing such a function?
In the real case where I want to use it in the array instead of numbers there are dictionaries.
It is better to do this vectorized
import numpy as np
arr = np.array([[0, 1, 8, 4, 0, 0],
[1, 2, 3, 0, 0, 0],
[3, 2, 3, 0, 5, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0]])
def stripArray(e, arr):
return arr[(arr!=e).any(axis = 1), :][:, (arr!=e).any(axis = 0)]
stripArray(0, arr)
array([[0, 1, 8, 4, 0],
[1, 2, 3, 0, 0],
[3, 2, 3, 0, 5]])
Here is an answer which doesnt need numpy:
from typing import List, Any
def all_value(value: Any, arr: List[float]) -> bool:
return all(map(lambda x: x==value, arr))
def transpose_array(arr: List[List[float]]) -> List[List[float]]:
return list(map(list, zip(*arr)))
def strip_array(value: Any, arr: List[List[float]]) -> List[List[float]]:
# delete empty rows
arr = [row for row in arr if not all_value(value, row)]
#transpose and delete empty columns
arr = transpose_array(arr)
arr = [col for col in arr if not all_value(value, col)]
#transpose back
arr = transpose_array(arr)
return arr
test = [[0, 1, 8, 4, 0, 0],
[1, 2, 3, 0, 0, 0],
[3, 2, 3, 0, 5, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0]]
result = strip_array(0, test)
Output:
result
[[0, 1, 8, 4, 0],
[1, 2, 3, 0, 0],
[3, 2, 3, 0, 5]]
Code:
def strip_array(array, empty_val=0):
num_bad_columns = 0
while np.all(array[:, -(num_bad_columns+1)] == 0):
num_bad_columns += 1
array = array[:, :(-num_bad_columns)]
num_bad_rows = 0
while np.all(array[-(num_bad_rows+1), :] == 0):
num_bad_rows += 1
array = array[:(-num_bad_rows), :]
return array
array = np.array(
[[0, 1, 8, 4, 0, 0],
[1, 2, 3, 0, 0, 0],
[3, 2, 3, 0, 5, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0]]
)
print(array)
print(strip_array(array, 0))
Output:
[[0 1 8 4 0 0]
[1 2 3 0 0 0]
[3 2 3 0 5 0]
[0 0 0 0 0 0]
[0 0 0 0 0 0]]
[[0 1 8 4 0]
[1 2 3 0 0]
[3 2 3 0 5]]
try using np.delete to remove unwanted rows or columns
data=[[0, 1, 8, 4, 0, 0],
[1, 2, 3, 0, 0, 0],
[3, 2, 3, 0, 5, 0],
[0, 0, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0]]
def drop_row(data):
lstIdx=[]
for i in range(len(data)):
count=0
for j in range(len(data[i])):
if data[i][j] == 0:
count+=1
if count==len(data[i]):
print("row zero")
lstIdx.append(i)
#for i in lstIdx:
data=np.delete(data,lstIdx,axis=0)
return data
def drop_column(data):
lstIdx=[]
if len(data)==0:
return data
for j in range(len(data[0])):
count=0
for i in range(len(data)):
if data[i][j] == 0:
count+=1
if count==len(data):
print("column zero")
lstIdx.append(j)
data=np.delete(data,lstIdx,axis=1)
return data
data=drop_row(data)
data=drop_column(data)
print(data)
output:
[[0 1 8 4 0]
[1 2 3 0 0]
[3 2 3 0 5]]
I have a Pandas dataframe which looks like this:
col_1 col_2
a 4
a 3
b 2
c 2
d 1
b 4
c 1
I need to transform it into a NumPy array of 2D-arrays where each 2D-array corresponds to one of the letters. For example, if 'a' doesn't occur together with 1 and 2, and occurs with 3 and 4, 2D array corresponding to it should look like [0, 0, 1, 1]. So in this example I need:
[[0, 0, 1, 1], [0, 1, 0, 1], [0, 1, 0, 0], [1, 0, 0, 1]]
What is the best way to do this?
Here is one way crosstab
l = pd.crosstab(df.col_1,df.col_2).values.tolist()
Out[23]: [[0, 0, 1, 1], [0, 1, 0, 1], [1, 1, 0, 0], [1, 0, 0, 0]]
This question already has answers here:
How do I remove duplicates from a list, while preserving order?
(30 answers)
Closed 2 years ago.
How to make from this:
[[1, 0, 0], [1, 0, 0], [1, 0, 0], [1, 0, 1], [1, 0, 1], [1, 0, 1]]
to this?
array([[1, 0, 0],
[1, 0, 1]])
I tried this but it only works with 2 repeated elements. I want it to work for any number of repeated elements.
finalRes = []
for i in range(len(resList)-1):
if resList[i] == resList[i+1]:
finalRes.append(resList[i])
finalRes --> [[1, 0, 0], [1, 0, 0], [1, 0, 1], [1, 0, 1]]
Use itertools.groupby:
from itertools import groupby
lst = [[1, 0, 0], [1, 0, 0], [1, 0, 0], [1, 0, 1], [1, 0, 1], [1, 0, 1]]
result = [key for key, _ in groupby(lst)]
print(result)
Output
[[1, 0, 0], [1, 0, 1]]
You can use numpy.unique to find the unique elements in the array.
Use:
import numpy as np
arr = [[1, 0, 0], [1, 0, 0], [1, 0, 0], [1, 0, 1], [1, 0, 1], [1, 0, 1]]
unique_arr = np.unique(arr, axis=0)
print(unique_arr)
This prints:
[[1 0 0]
[1 0 1]]
I have a requirement where I want to convert a 2D matrix to 3D by separating 3 unique values across 3 dimensions.
For Example:
convert
A = [1 2 3 3
1 1 2 1
3 2 2 3
1 3 3 2]
to
A = [[1 0 0 0
1 1 0 1
0 0 0 0
1 0 0 0]
[0 1 0 0
0 0 1 0
0 1 1 0
0 0 0 1]
[0 0 1 1
0 0 0 0
1 0 0 1
0 1 1 0]]
Pardon me if the syntax of matrix representation is not correct.
Use broadcasting with outer-equality for a vectorized solution -
# Input array
In [8]: A
Out[8]:
array([[1, 2, 3, 3],
[1, 1, 2, 1],
[3, 2, 2, 3],
[1, 3, 3, 2]])
In [11]: np.equal.outer(np.unique(A),A).view('i1')
Out[11]:
array([[[1, 0, 0, 0],
[1, 1, 0, 1],
[0, 0, 0, 0],
[1, 0, 0, 0]],
[[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 1, 1, 0],
[0, 0, 0, 1]],
[[0, 0, 1, 1],
[0, 0, 0, 0],
[1, 0, 0, 1],
[0, 1, 1, 0]]], dtype=int8)
To use the explicit dimension-extension + comparison, it would be :
(A == np.unique(A)[:,None,None]).view('i1')
You can use np.unique and take advantage of boolean arrays and cast them to int using numpy.ndarray.astype.
import numpy as np
a=np.array([[1, 2, 3, 3], [1, 1, 2, 1], [3, 2, 2, 3], [1, 3, 3, 2]])
[a==i.astype(int) for i in np.unique(a)]
Output:
[array([[1, 0, 0, 0],
[1, 1, 0, 1],
[0, 0, 0, 0],
[1, 0, 0, 0]]),
array([[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 1, 1, 0],
[0, 0, 0, 1]]),
array([[0, 0, 1, 1],
[0, 0, 0, 0],
[1, 0, 0, 1],
[0, 1, 1, 0]])]
EDIT: Ch3steR's answer is better
A = np.array([[1,2,3,3], [1,1,2,1], [3,2,2,3], [1,3,3,2]])
unique_values = np.unique(A)
B = np.array([np.zeros_like(A) for i in range(len(unique_values))])
for idx, value in enumerate(unique_values):
B[idx][A == value] = 1
Could someone explain how to fix the below? I've read some explanations, but couldn't get my head around it...
Many thanks in advance!
k = 2 # number of possible values for each element, in this case 0 or 1
length = 3 # length of list
result = [0] * length # initialise list
results = []
# generate permutations of list
def permutations(i, k, length):
j = 0
while j < k:
result[i] = j
if i == length - 1:
print("Result: ", result)
results.append(result)
print("Results: ", results)
else:
permutations(i + 1, k, length)
j += 1
permutations(0, k, length)
Below the output. The problem is that all previous elements in the list are overwritten...
Result: [0, 0, 0]
Results: [[0, 0, 0]]
Result: [0, 0, 1]
Results: [[0, 0, 1], [0, 0, 1]]
Result: [0, 1, 0]
Results: [[0, 1, 0], [0, 1, 0], [0, 1, 0]]
Result: [0, 1, 1]
Results: [[0, 1, 1], [0, 1, 1], [0, 1, 1], [0, 1, 1]]
Result: [1, 0, 0]
Results: [[1, 0, 0], [1, 0, 0], [1, 0, 0], [1, 0, 0], [1, 0, 0]]
Result: [1, 0, 1]
Results: [[1, 0, 1], [1, 0, 1], [1, 0, 1], [1, 0, 1], [1, 0, 1], [1, 0, 1]]
Result: [1, 1, 0]
Results: [[1, 1, 0], [1, 1, 0], [1, 1, 0], [1, 1, 0], [1, 1, 0], [1, 1, 0], [1, 1, 0]]
Result: [1, 1, 1]
Results: [[1, 1, 1], [1, 1, 1], [1, 1, 1], [1, 1, 1], [1, 1, 1], [1, 1, 1], [1, 1, 1], [1, 1, 1]]
You are appending the same list everytime. Modifying the list via that reference will propagate changes to every where the list object lives; it is the same list.
You should append a shallow copy instead, so the reference result only modifies the current list:
...
results.append(result[:])
Otherwise, you could create a new list object at the start of the function so each recursive call gets its own list:
def permutations(i, k, length):
result = []
...
I believe changing results.append(result) to result.append(result[:]) should fix the problem. It is because of the mutability of lists
What you implement can be described as repeated permutations or cartesian product.
There are k ** length lists or tuples that can be generated this way.
As with any combination, permutation or product, itertools can help you :
from itertools import product
k = 2 # number of possible values for each element, in this case 0 or 1
length = 3 # length of list
print(list(product(range(k), repeat=length)))
#[(0, 0, 0), (0, 0, 1), (0, 1, 0), (0, 1, 1), (1, 0, 0), (1, 0, 1), (1, 1, 0), (1, 1, 1)]
Done!