I am trying to delete the last item in both the rows and columns in my numpy.ndarray (type = class numpy.ndarray). My array has 30 rows and 180 columns (i.e. 180 values per row). I have tried numpy.delete but this simply removes the whole row/column.
To illustrate what I want to achieve I created the following example in Python using and array and nested for loops:
a = np.array([[[1,2,3,4,5,6],[1,2,3,4],[1,2,3,4]],[[1,2,3,4,5,6],[1,2,3,4],[1,2,3,4]],[[1,2,3,4],[1,2,3,4],[1,2,3,4]],[[1,2,3,4],[1,2,3,4],[1,2,3,4]],[[1,2,3,4],[1,2,3,4],[1,2,3,4]],[[1,2,3,4],[1,2,3,4],[1,2,3,4]],[[1,2,3,4],[1,2,3,4]],[[1,2,3,4],[1,2,3,4]],[[1,2,3,4],[1,2,3,4]],[[1,2,3,4],[1,2,3,4]],[[1,2,3,4],[1,2,3,4]]])
for list in a:
for sublist in list:
del sublist[-1]
Using
print(a)
Gives the following array:
[[[1, 2, 3, 4, 5, 6], [1, 2, 3, 4], [1, 2, 3, 4]]
[[1, 2, 3, 4, 5, 6], [1, 2, 3, 4], [1, 2, 3, 4]]
[[1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4]]
[[1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4]]
[[1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4]]
[[1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4]] [[1, 2, 3, 4], [1, 2, 3, 4]]
[[1, 2, 3, 4], [1, 2, 3, 4]] [[1, 2, 3, 4], [1, 2, 3, 4]]
[[1, 2, 3, 4], [1, 2, 3, 4]] [[1, 2, 3, 4], [1, 2, 3, 4]]]
Using
print(list)
after the for loops gives:
[[1, 2, 3, 4, 5], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3, 4, 5], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3]]
Unfortunately using this on my array gives the following error:
TypeError: 'numpy.float64' object does not support item deletion
Thanks
Update:
I am extracting my information from a grid NetCDF file. I have changed the word list to l since list is a Python keyword. This didn't change it for me.
This provides a good example of my array:
c = np.arange(5400).reshape(30,180)
for l in c:
for i in l:
del i[-1]
When I run this code I get the following error:
Traceback (most recent call last): File "main.py", line 18, in <module>
del i[-1]
TypeError: 'numpy.int64' object does not support item deletion
del i[-1] is a list operation. np.array does not support that.
Count the occurrences of a specific value and remove them at the same time demonstrates the differences between lists and arrays when it comes to deletion.
Your example a is object dtype, containing lists
In [111]: a.shape
Out[111]: (11,)
In [112]: [len(i) for i in a]
Out[112]: [3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2]
In [113]: a[0]
Out[113]: [[1, 2, 3, 4, 5, 6], [1, 2, 3, 4], [1, 2, 3, 4]]
a[0] is a 3 element list, with sublists of different length.
It's not clear what you want to delete. Delete elements from a, or elements from each element of a, or elements from the sublists of those elements.
Furthermore, if the real data is from NetCDF it might actually a multidimensional array. Or if object dtype, the elements might themselves be (2d) arrays.
In case, slicing is the right way to remove rows/columns from an array:
In [114]: a = np.arange(12).reshape(3,4)
In [115]: a
Out[115]:
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
In [116]: a[:-1, :-1]
Out[116]:
array([[0, 1, 2],
[4, 5, 6]])
The result is a view; it does not change a itself. a = a[:-1, :-1].copy() is the cleanest way to creates a reduced size array without leaving the any of the original around.
Related
This is my code:
b = [6 * [1, 3, 4, 2],
4 * [2, 1, 4, 3],
3 * [3, 4, 2, 1],
4 * [4, 2, 1, 3],
4 * [4, 3, 2, 1],
]
Which returns an array which has 6X4=24 elements in the first line 4X4=16 in the second etc...
What i want to achieve is adding the exact same line multiple times like:
1, 3, 4, 2
1, 3, 4, 2
1, 3, 4, 2
1, 3, 4, 2
1, 3, 4, 2
1, 3, 4, 2 #6 tines the first line
2, 1, 4, 3
2, 1, 4, 3
2, 1, 4, 3
2, 1, 4, 3 # 4 times the second
..........
but of course by not copying the same line again and again
Try:
b = [
*[[1, 3, 4, 2] for _ in range(6)],
*[[2, 1, 4, 3] for _ in range(4)],
*[[3, 4, 2, 1] for _ in range(3)],
*[[4, 2, 1, 3] for _ in range(4)],
*[[4, 3, 2, 1] for _ in range(4)],
]
print(b)
Prints:
[
[1, 3, 4, 2],
[1, 3, 4, 2],
[1, 3, 4, 2],
[1, 3, 4, 2],
[1, 3, 4, 2],
[1, 3, 4, 2],
[2, 1, 4, 3],
[2, 1, 4, 3],
[2, 1, 4, 3],
[2, 1, 4, 3],
[3, 4, 2, 1],
[3, 4, 2, 1],
[3, 4, 2, 1],
[4, 2, 1, 3],
[4, 2, 1, 3],
[4, 2, 1, 3],
[4, 2, 1, 3],
[4, 3, 2, 1],
[4, 3, 2, 1],
[4, 3, 2, 1],
[4, 3, 2, 1],
]
You can also put it in one line with
b = 6*[[1, 3, 4, 2]] + 4*[[2, 1, 4, 3]] + 3*[[3, 4, 2, 1]] + 4*[[4, 2, 1, 3]] + 4* [[4, 3, 2, 1]])
print(b)
Only two steps if you don't want to change how b was created.
import numpy as np
B = np.concatenate(b).ravel()
b = np.reshape(B,(21,4))
You can use a list comprehension of a list comprehension to repeat y times your x lists (from bb and aa, respectively)
The notation is a bit strange, the "center" one is the outer element, then it expands on the right.
Advantage: numpy not required, and it will work for any size entries, unlike other answers "harcoding" the rows
aa=[[1, 3, 4, 2],
[2, 1, 4, 3],
[3, 4, 2, 1],
[4, 2, 1, 3],
[4, 3, 2, 1]]
bb=[6,4,3,4,4]
xx = [x for x,y in zip(aa,bb) for _ in range(y) ]
returns:
[[1, 3, 4, 2], [1, 3, 4, 2], [1, 3, 4, 2], [1, 3, 4, 2], [1, 3, 4, 2], [1, 3, 4, 2], [2, 1, 4, 3], [2, 1, 4, 3], [2, 1, 4, 3], [2, 1, 4, 3], [3, 4, 2, 1], [3, 4, 2, 1], [3, 4, 2, 1], [4, 2, 1, 3], [4, 2, 1, 3], [4, 2, 1, 3], [4, 2, 1, 3], [4, 3, 2, 1], [4, 3, 2, 1], [4, 3, 2, 1], [4, 3, 2, 1]]
It can be done systematically by pairing the factors and the sublists and multiplying them together. Then flat the list.
Here a hacky-way
b = [[1, 3, 4, 2],
[2, 1, 4, 3],
[3, 4, 2, 1],
[4, 2, 1, 3],
[4, 3, 2, 1],
]
factors = [6, 4, 3, 4, 4]
print(sum(f*[l] for f, l in zip(factors, b), []))
The flattening part (chaining of lists) is done with sum( , []) but is not performant (and designed for that)... but still useful, in my opinion, to test some small stuffs on the fly. The list chaining have to be done with i.e. itertools.chain. Here an example
import itertools as it
...
list(it.chain.from_iterable(f*[l] for f, l in zip(factors, b)))
# or (for many iterators)
list(it.chain(*(f*[l] for f, l in zip(factors, b))))
# or with repeat
list(it.chain.from_iterable(it.starmap(it.repeat, zip(b, factors))))
# or (another) with repeat
list(it.chain.from_iterable(map(it.repeat, b, factors))
# or ...
list(it.chain.from_iterable(map(list.__mul__, ([i] for i in b) , factors)))
# or with reduce
import functools as fc
list(fc.reduce(lambda i, j: i + j, zip(b, factors)))
Let's say I have a nested list:
list = [[10, 2, 8, 4], [12, 6, 4, 1], [8, 4, 3, 2], [9, 3, 4, 6]]
I want to rank the elements in the sublist against each other to create a new nested list with the rankings.
result = [[1, 4, 2, 3], [1, 2, 3, 4], [1, 2, 3, 4], [1, 4, 3, 2]]
in the first sublist 10 would be 1st, 8 2nd, etc.
There are already some good solutions. Here just another one - functional approach for reference:
No 3rd library used.
lst = # your lists - don't use builtin "list"
def ranking(nums):
ranks = {x:i for i, x in enumerate(sorted(nums, reverse=True),1)}
return [ranks[x] for x in nums] # quick mapping back: O(1)
Calling it:
result = list(map(ranking, lst))
As already mentioned in the comment, you can use numpy.argsort, using it twice gives you the rank for the values, which need to be subtracted from len of the sub list to rank from highest to lowest, you can use List-Comprehension to do it for all the sub lists.
>>> import numpy as np
>>> lst = [[10, 2, 8, 4], [12, 6, 4, 1], [8, 4, 3, 2], [9, 3, 4, 6]]
>>> [(len(sub)-np.argsort(sub).argsort()).tolist() for sub in lst]
[[1, 4, 2, 3], [1, 2, 3, 4], [1, 2, 3, 4], [1, 4, 3, 2]]
You can even use 2D numpy array and negate the values, then directly call argsort twice on the resulting array, and finally add 1:
>>> (-np.array(lst)).argsort().argsort()+1
array([[1, 4, 2, 3],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 4, 3, 2]], dtype=int64)
You can use scipy.stats.rankdata:
my_list = [[10, 2, 8, 4], [12, 6, 4, 1], [8, 4, 3, 2], [9, 3, 4, 6]]
from scipy.stats import rankdata
[list(len(l)+1-rankdata(l).astype(int)) for l in my_list]
output:
[[1, 4, 2, 3], [1, 2, 3, 4], [1, 2, 3, 4], [1, 4, 3, 2]]
Without numpy/scipy:
[[sorted(li, reverse=True).index(x)+1 for x in li] for li in data]
[[1, 4, 2, 3], [1, 2, 3, 4], [1, 2, 3, 4], [1, 4, 3, 2]]
Another solution with no external libraries, and with a better time complexity, just in case your sublists are a bit longer than 4 items (this has some overhead but I presume it is O(n log n) because of the call to sorted).
def rank_all(ls):
result = []
for subls in ls:
pairs = sorted([(subls[j],j) for j in range(len(subls))], reverse=True)
ranked = [0] * len(subls)
for j,p in enumerate(pairs):
ranked[p[1]]=j+1
result.append(ranked)
return result
If I have a list of lists
matrix = [[2, 3, 1, 2],[1, 2, 3, 2],[3, 3, 1, 2], [2, 2, 3, 3]]
how can I check with a for loop if for example element = 1 is present in each column
Using numpy:
np.any(a==1, 1).all()
>>> a = np.array([[2, 3, 1, 2],[1, 2, 3, 2],[3, 3, 1, 2], [2, 2, 3, 3]])
>>> np.any(a==1, 1).all()
False
>>> a = np.array([[2, 3, 1, 2],[1, 2, 3, 2],[3, 3, 1, 2], [2, 1, 3, 3]])
>>> np.any(a==1, 1).all()
True
Using all, in and a list comprehension:
matrix = [[2, 3, 1, 2], [1, 2, 3, 2], [3, 3, 1, 2], [2, 2, 3, 3]]
valid = all(1 in row for row in matrix)
Or, the verbose way:
matrix = [[2, 3, 1, 2], [1, 2, 3, 2], [3, 3, 1, 2], [2, 2, 3, 3]]
valid = True
for row in matrix:
if 1 not in row:
valid = False
break
I have an array:
X = [[2, 2, 2],
[3, 3, 3],
[4, 4, 4]]
I need to add extra column in numpy array and fill it with ones using hstack and reshape. Like that:
X = [[2, 2, 2, 1],
[3, 3, 3, 1],
[4, 4, 4, 1]]
What I do:
X = np.hstack(X, np.ones(X.reshape(X, (2,3))))
And a get an error:
TypeError: only length-1 arrays can be converted to Python scalars
What's a problem? What I've done wrong?
Here's a couple ways with numpy.append, numpy.hstack or numpy.column_stack:
# numpy is imported as np
>>> x
array([[2, 2, 2],
[3, 3, 3],
[4, 4, 4]])
>>> np.append(x, np.ones([x.shape[0], 1], dtype=np.int32), axis=1)
array([[2, 2, 2, 1],
[3, 3, 3, 1],
[4, 4, 4, 1]])
>>> np.hstack([x, np.ones([x.shape[0], 1], dtype=np.int32)])
array([[2, 2, 2, 1],
[3, 3, 3, 1],
[4, 4, 4, 1]])
>>> np.column_stack([x, np.ones([x.shape[0], 1], dtype=np.int32)])
array([[2, 2, 2, 1],
[3, 3, 3, 1],
[4, 4, 4, 1]])
You can use numpy.insert():
>>> X
array([[2, 2, 2],
[3, 3, 3],
[4, 4, 4]])
Ones at the begining of matrix:
>>> X=np.insert(X,0,1.0,axis=1)
>>> X
array([[1, 2, 2, 2],
[1, 3, 3, 3],
[1, 4, 4, 4]])
Ones at the end of matrix
>>> X=np.insert(X,3,1.0,axis=1)
>>> X
array([[2, 2, 2, 1],
[3, 3, 3, 1],
[4, 4, 4, 1]])
I wrote a function:
def expandList(aList):
"""expand a list"""
finalList = []
for j in aList:
tempList = []
if type(j) != type(list):
tempList.append(j)
finalList.extend(tempList)
else:
finalList.extend(expandList(j))
return finalList
to expand nested list within themselves like:
[[1, [2, 3], [3, 2]], [2, [1, 3], [3, 1]], [3, [1, 2], [2, 1]]]
into:
[[1,2,3], [1,3,2], [2,1,3], [2,3,1], [3,1,2], [3,2,1]]
or
[[1, [2, [3, 4], [4, 3]], [3, [2, 4], [4, 2]], [4, [2, 3], [3, 2]]],
[2, [1, [3, 4], [4, 3]], [3, [1, 4], [4, 1]], [4, [1, 3], [3, 1]]],
[3, [1, [2, 4], [4, 2]], [2, [1, 4], [4, 1]], [4, [1, 2], [2, 1]]],
[4, [1, [2, 3], [3, 2]], [2, [1, 3], [3, 1]], [3, [1, 2], [2, 1]]]]
into:
[[1, 2, 3, 4],[1, 2, 4, 3],[1, 3, 2, 4],
[1, 3, 4, 2],[1, 4, 3, 2],[1, 4, 2, 3],[2, 1, 3, 4],
[2, 1, 4, 3],[2, 3, 1, 4],[2, 3, 4, 1],[2, 4, 1, 3],
[2, 4, 3, 1],[3, 1, 2, 4],[3, 1, 4, 2],[3, 2, 1, 4],
[3, 2, 4, 1],[3, 4, 1, 2],[3, 4, 2, 1],[4, 1, 2, 3],
[4, 1, 3, 2],[4, 2, 1, 3],[4, 2, 3, 1],[4, 3, 1, 2],
[4, 3, 2, 1]]
and so forth. I wish to be able to do this in any size of nested lists.
My function doesn't seem to work right. What am I doing wrong? How can I fix/improve my function?
Thank you in advance
First of all using following command is a wrong way for checking the list type :
type(j) != type(list)
because type(list) returns <type 'type'> actually you are getting the type of a type object that is a type.
In edition you don't need to loop over your sub list and using extend method although you used it incorrect.Since your numbers are in the first index you can just convert it to list and append the rest to it.
You can use a simple list comprehension :
>>> [[[i[0]]+j for j in i[1:]] for i in l]
[[[1, 2, 3], [1, 3, 2]], [[2, 1, 3], [2, 3, 1]], [[3, 1, 2], [3, 2, 1]]]