I have a 2D array of Numpy data read from a .csv file. Each row represents a data point with the final column containing a a 'key' which corresponds uniquely to 'key' in another Numpy array - the 'lookup table' as it were.
What is the best (most Numpythonic) way to match up the lines in the first table with the values in the second?
Some example data:
import numpy as np
lookup = np.array([[ 1. , 3.14 , 4.14 ],
[ 2. , 2.71818, 3.7 ],
[ 3. , 42. , 43. ]])
a = np.array([[ 1, 11],
[ 1, 12],
[ 2, 21],
[ 3, 31]])
Build a dictionary from key to row number in the lookup table:
mapping = dict(zip(lookup[:,0], range(len(lookup))))
Then you can use the dictionary to match up lines. For instance, if you just want to join the tables:
>>> np.hstack((a, np.array([lookup[mapping[key],1:]
for key in a[:,0]])))
array([[ 1. , 11. , 3.14 , 4.14 ],
[ 1. , 12. , 3.14 , 4.14 ],
[ 2. , 21. , 2.71818, 3.7 ],
[ 3. , 31. , 42. , 43. ]])
In the special case when the index can be calculated from the keys, the dictionary can be avoided. It's an advantage when the key of the lookup table can be chosen.
For Vebjorn Ljosa's example:
lookup:
>>> lookup[a[:,0]-1, :]
array([[ 1. , 3.14 , 4.14 ],
[ 1. , 3.14 , 4.14 ],
[ 2. , 2.71818, 3.7 ],
[ 3. , 42. , 43. ]])
merge:
>>> np.hstack([a, lookup[a[:,0]-1, :]])
array([[ 1. , 11. , 1. , 3.14 , 4.14 ],
[ 1. , 12. , 1. , 3.14 , 4.14 ],
[ 2. , 21. , 2. , 2.71818, 3.7 ],
[ 3. , 31. , 3. , 42. , 43. ]])
Related
how to replace 4th and 5th column values in utl by new_values array and keep the remaining columns as it is
utl = np.array([[ 3. , 134.4 , 17. , 135.05 , 22. , 135.25 , 0.04 ],
[ 12. , 134.3 , 17. , 135.05 , 22. , 135.8 , 0.15 ]])
new_values=np.array([[ 27., 135.45],
[ 27., 136.55]])
i tried this but it does not work
# utl[:,[4,5]] = new_values
# utl[:,4] = new_values[:,0]
output must be
#values changed
[[ 3. , 134.4 , 17. , 135.05 , | 27. , 135.45 |, 0.04 ],
[ 12. , 134.3 , 17. , 135.05 , | 27. , 136.55 |, 0.15 ]])
this works fine, as expected:
utl[:, [4,5]] = new_values
output:
array([[ 3. , 134.4 , 17. , 135.05, 27. , 135.45, 0.04],
[ 12. , 134.3 , 17. , 135.05, 27. , 136.55, 0.15]])
Input:
array([[ 1. , 5. , 1. ],
[ 10. , 7. , 1.5],
[ 6.9, 5. , 1. ],
[ 19. , 9. , 100. ],
[ 11. , 11. , 11. ]])
Expected Output:
array([[ 19. , 9. , 100. ],
[ 11. , 11. , 11. ],
[ 10. , 7. , 1.5],
[ 6.9, 5. , 1. ],
[ 1. , 5. , 1. ]])
i tried doing the below:
for i in M:
ls = i.mean()
x = np.append(i,ls)
print(x) #found the mean
After this i am unable to arrange each column based on the mean value in each row. All i can do
is to arrange each row in descending order but that is not what i wanted.
You can do this:
In [405]: row_idxs = np.argsort(np.mean(a * -1, axis=1))
In [406]: a[row_idxs, :]
Out[406]:
array([[ 19. , 9. , 100. ],
[ 11. , 11. , 11. ],
[ 10. , 7. , 1.5],
[ 6.9, 5. , 1. ],
[ 1. , 5. , 1. ]])
Using argsort will sort the indices. Multiplying by -1 allows you to get descending order.
I have this numpy array:
array(
[[ 1. , 9. , 565.98653513],
[ 1. , 1. , 973.18466261],
[ 1. , 25. , 803.17747373],
[ 2. , 9. , 82.56336897],
[ 2. , 1. , 104.69517373],
[ 2. , 25. , 627.01127514],
[ 3. , 21. , 334.07622382],
[ 3. , 34. , 921.37623107],
[ 3. , 20. , 342.08177942],
... ... ... ...
[ 10. , 7. , 424.29338026],
[ 10. , 0. , 232.71475407],
[ 10. , 1. , 330.44846202]])
But I want to sort the matrix by the first column in a cycle: 1, 2, 3, ...,10. It should look like this:
array(
[[ 1. , 9. , 565.98653513],
[ 2. , 9. , 82.56336897],
[ 3. , 21. , 334.07622382],
... ... ... ...
[ 10. , 7. , 424.29338026],
[ 1. , 1. , 973.18466261],
[ 2. , 1. , 104.69517373],
[ 3. , 34. , 921.37623107],
... ... ... ...
[ 10. , 0. , 232.71475407],
[ 1. , 25. , 803.17747373],
[ 2. , 25. , 627.01127514],
[ 3. , 20. , 342.08177942],
... ... ... ...
[ 10. , 1. , 330.44846202]])
How can I do this?
I was thinking of converting it to a dataframe (i.e. pandas) for more sorting options, then covert back to an array...but I don't see a straight forward function to do this.
I appreciate any help or ideas.
Assuming your array is named x:
y = [x[i::10] for i in range(int(len(x)/10))]
y = np.array(y)
y.reshape(x.shape)
print(y)
The x[i:j:k] notation means x from i to j with step k. So x[i::10] mean x from i to the end with step 10.
See more here.
I want to sort the rows of a 2D array based on the elements of the first column, in Python 3. For example, if
x = array([[ 5. , 9. , 2. , 6. ],
[ 7. , 12. , 3.5, 8. ],
[ 2. , 6. , 7. , 9. ]])
then I need the sorted array to be
x = array([[ 2. , 6. , 7. , 9. ],
[ 5. , 9. , 2. , 6. ],
[ 7. , 12. , 3.5, 8. ]])
How can I do that? A similar question was asked and answered here, but it does not work for me.
The following should work:
import numpy as np
x = np.array([[ 5. , 9. , 2. , 6. ],
[ 7. , 12. , 3.5, 8. ],
[ 2. , 6. , 7. , 9. ]])
x[x[:, 0].argsort()]
Out[2]:
array([[ 2. , 6. , 7. , 9. ],
[ 5. , 9. , 2. , 6. ],
[ 7. , 12. , 3.5, 8. ]])
Documentation : numpy.argsort
#using sorted
x = ([[5.,9.,2.,6. ], [7.,12.,3.5,8.], [2.,6.,7.,9.]])
x = sorted(x, key=lambda i: i[0]) #1st col
print(x)
I though this would be super easy but I am struggling a little. I have a data structure as follows
array([[ 5. , 3.40166205],
[ 10. , 2.72778882],
[ 15. , 2.31881804],
[ 20. , 2.50643777],
[ 1. , 3.94076063],
[ 2. , 3.80598599],
[ 3. , 3.67121134],
[ 6. , 3.2668874 ],
[ 7. , 3.13211276],
[ 8. , 2.99733811],
[ 9. , 2.86256347],
[ 11. , 2.64599467],
[ 12. , 2.56420051],
[ 13. , 2.48240635],
[ 14. , 2.4006122 ],
[ 16. , 1.8280531 ],
[ 17. , 1.74625894],
[ 18. , 1.66446479],
[ 19. , 1.58267063],
[ 20. , 1.50087647]])
And I want to sort it ONLY on the first column ... so it is ordered as follows:
array([[1. , 3.9],
[2. , 3.8],
... ,
[20. , 1.5]])
np.sort doesn't seem to work as it moves array to a flat structure. I've also used itemgetter
from operator import itemgetter
sorted(data, key=itemgetter(1))
But this doesn't give me the output I'm looking for.
Help appreciated!
This is a common numpy idiom. You can use argsort (on the first column) + numpy indexing here -
x[x[:, 0].argsort()]
array([[ 1. , 3.94076063],
[ 2. , 3.80598599],
[ 3. , 3.67121134],
[ 5. , 3.40166205],
[ 6. , 3.2668874 ],
[ 7. , 3.13211276],
[ 8. , 2.99733811],
[ 9. , 2.86256347],
[ 10. , 2.72778882],
[ 11. , 2.64599467],
[ 12. , 2.56420051],
[ 13. , 2.48240635],
[ 14. , 2.4006122 ],
[ 15. , 2.31881804],
[ 16. , 1.8280531 ],
[ 17. , 1.74625894],
[ 18. , 1.66446479],
[ 19. , 1.58267063],
[ 20. , 2.50643777],
[ 20. , 1.50087647]])