When calling numpy.array in the following two ways:
>>> np.array((1,2,3,4))
array([1, 2, 3, 4])
>>> np.array([1,2,3,4])
array([1, 2, 3, 4])
I notice it returns two seemingly identical ndarrays.
Are both of these ndarrays identical? Why?
We can behave in a general way here. Suppose we don't know what X = np.array([1,2,3,4]) and Y = np.array((1,2,3,4)) are. If we print it, we can see an output which is the result of a secret built-in methods X.__repr__ and Y.__repr__. You can see here for sure that both X and Y has the same representations. It doesn't mean, however, that they are the same because they can be instances of different classes with the same representations. To make it sure, I usually use X.__class__ and Y.__class__. So both X and Y are instances of the same class np.ndarray.
Related
I'd like to create a numpy ndarray with entries of type ndarray itself. I was able to wrap ndarrays into another type to get it work but I want to do this without wrapping. With wrapping a ndarray x into e.g. the dictionary {1:x} I can do
F = np.vectorize(lambda x: {1:np.repeat(x,3)})
F(np.arange(9).reshape(3,3))
and get (3,3) ndarray with entries {1:[0,0,0]} ... {1:[8,8,8]} (with ndarrays). When change F to F = np.vectorize(lambda x: np.repeat(x,3)) numpy complains ValueError: setting an array element with a sequence. I guess it detects that the entries as arrays themselves and doesn't threat them as objects anymore.
How can I avoid this and do the same thing without wrapping the entries from ndarray into something different?
Thanks a lot in advance for hints :)
You can (ab-)use numpy.frompyfunc:
>>> F = np.arange(9).reshape(3, 3)
>>> np.frompyfunc(F.__getitem__, 1, 1)(range(3))
array([array([0, 1, 2]), array([3, 4, 5]), array([6, 7, 8])], dtype=object)
I am loading a dataset in my python code with contains two matrices. The name of those matrices are train_dataset_face and train_dataset_audio and the way that I am reading them is as list of np.arrays. Finally I convert them as np.arrrays of np.arrays. Initially my matrices during the debug look like that:
and
Then I convert them into np.arrays using the following code:
train_dataset_face = np.array(train_dataset_face)
train_dataset_audio = np.array(train_dataset_audio)
And in the end my matrices look like:
For some weird reason in the case of train_dataset_face I got this array indication before each vector of my array while in the case of train_dataset_audio i dont have it. Is it possible to remove it? This "array" indication cause me problems when I am trying to apply several algorithms to the train_dataset_face. Any idea what happened here?
You can only create a single array, if all arrays in your list has the same shape, which is true for train_dataset_audio but not for train_dataset_face.
>>> a = [numpy.array([1,2,3,4]), numpy.array([1,2,3,4])]
>>> numpy.array(a)
array([[1, 2, 3, 4],
[1, 2, 3, 4]])
>>> b = [numpy.array([1,2,3]), numpy.array([1,2,3,4])]
>>> numpy.array(b)
array([array([1, 2, 3]), array([1, 2, 3, 4])], dtype=object)
I have a function with one of the parameters as numpy.ndarray. It's mutable so it can not be cached by lru_cache.
Is there any existing solution for this?
Possibly the simplest way of doing so is to memoize a version taking only immutable objects.
Say your function takes an np.array, and let's assume it's a 1d array. Fortunately, it is easily translated to a tuple:
import numpy as np
a = np.array([1, 2, 3, 4])
>> tuple(a)
(1, 2, 3, 4)
and vice versa:
>> np.array(tuple(a))
array([1, 2, 3, 4])
So you get something like
# Function called by the rest of your program
array_foo(a) # `a` is an `np.array`
...
return tuple_foo(tuple(a))
then memoize instead this function:
# Internal implementation
#functools.lru_cache
tuple_foo(t) # `t` is a tuple
...
a = np.array(t)
In Python what is equivalent to Ruby's Array.each method? Does Python have a nice and short closure/lambda syntax for it?
[1,2,3].each do |x|
puts x
end
Does Python have a nice and short closure/lambda syntax for it?
Yes, but you don't want it in this case.
The closest equivalent to that Ruby code is:
new_values = map(print, [1, 2, 3])
That looks pretty nice when you already have a function lying around, like print. When you just have some arbitrary expression and you want to use it in map, you need to create a function out of it with a def or a lambda, like this:
new_values = map(lambda x: print(x), [1, 2, 3])
That's the ugliness you apparently want to avoid. And Python has a nice way to avoid it: comprehensions:
new_values = [print(x) for x in values]
However, in this case, you're just trying to execute some statement for each value, not accumulate the new values for each value. So, while this will work (you'll get back a list of None values), it's definitely not idiomatic.
In this case, the right thing to do is to write it explicitly—no closures, no functions, no comprehensions, just a loop:
for x in values:
print x
The most idiomatic:
for x in [1,2,3]:
print x
You can use numpy for vectorized arithmetic over an array:
>>> import numpy as np
>>> a = np.array([1, 2, 3])
>>> a * 3
array([3, 6, 9])
You can easily define a lambda that can be used over each element of an array:
>>> array_lambda=np.vectorize(lambda x: x * x)
>>> array_lambda([1, 2, 3])
array([1, 4, 9])
But as others have said, if you want to just print each, use a loop.
There are also libraries that wrap objects to expose all the usual functional programming stuff.
PyDash http://pydash.readthedocs.org/en/latest/
underscorepy (Google github underscore.py)
E.g. pydash allows you to do things like this:
>>> from pydash import py_
>>> from __future__ import print_function
>>> x = py_([1,2,3,4]).map(lambda x: x*2).each(print).value()
2
4
6
8
>>> x
[2, 4, 6, 8]
(Just always remember to "trigger" execution and/or to un-wrap the wrapped values with .value() at the end!)
without need of an assignment:
list(print(_) for _ in [1, 2, 3])
or just
[print(_) for _ in [1, 2, 3]]
As I only now noticed after commenting on this answer, slices in Python 3 return shallow copies of whatever they're slicing rather than views. Why is this still the case? Even leaving aside numpy's usage of views rather than copies for slicing, the fact that dict.keys, dict.values, and dict.items all return views in Python 3, and that there are many other aspects of Python 3 geared towards greater use of iterators, makes it seem that there would have been a movement towards slices becoming similar. itertools does have an islice function that makes iterative slices, but that's more limited than normal slicing and does not provide view functionality along the lines of dict.keys or dict.values.
As well, the fact that you can use assignment to slices to modify the original list, but slices are themselves copies and not views, is a contradictory aspect of the language and seems like it violates several of the principles illustrated in the Zen of Python.
That is, the fact you can do
>>> a = [1, 2, 3, 4, 5]
>>> a[::2] = [0, 0, 0]
>>> a
[0, 2, 0, 4, 0]
But not
>>> a = [1, 2, 3, 4, 5]
>>> a[::2][0] = 0
>>> a
[0, 2, 3, 4, 5]
or something like
>>> a = [1, 2, 3, 4, 5]
>>> b = a[::2]
>>> b
view(a[::2] -> [1, 3, 5]) # numpy doesn't explicitly state that its slices are views, but it would probably be a good idea to do it in some way for regular Python
>>> b[0] = 0
>>> b
view(a[::2] -> [0, 3, 5])
>>> a
[0, 2, 3, 4, 5]
Seems somewhat arbitrary/undesirable.
I'm aware of http://www.python.org/dev/peps/pep-3099/ and the part where it says "Slices and extended slices won't go away (even if the __getslice__ and __setslice__ APIs may be replaced) nor will they return views for the standard object types.", but the linked discussion provides no mention of why the decision about slicing with views was made; in fact, the majority of the comments on that specific suggestion out of the suggestions listed in the original post seemed to be positive.
What prevented something like this from being implemented in Python 3.0, which was specifically designed to not be strictly backwards-compatible with Python 2.x and thus would have been the best time to implement such a change in design, and is there anything that may prevent it in future versions of Python?
As well, the fact that you can use assignment to slices to modify the original list, but slices are themselves copies and not views.
Hmm.. that's not quite right; although I can see how you might think that. In other languages, a slice assignment, something like:
a[b:c] = d
is equivalent to
tmp = a.operator[](slice(b, c)) # which returns some sort of reference
tmp.operator=(d) # which has a special meaning for the reference type.
But in python, the first statement is actually converted to this:
a.__setitem__(slice(b, c), d)
Which is to say that an item assignment is actually specially recognized in python to have a special meaning, separate from item lookup and assignment; they may be unrelated. This is consistent with python as a whole, because python doesn't have concepts like the "lvalues" found in C/C++; There's no way to overload the assignment operator itself; only specific cases when the left side of the assignment is not a plain identifier.
Suppose lists did have views; And you tried to use it:
myView = myList[1:10]
yourList = [1, 2, 3, 4]
myView = yourList
In languages besides python, there might be a way to shove yourList into myList, but in python, since the name myView appears as a bare identifier, it can only mean a variable assignemnt; the view is lost.
Well it seems I found a lot of the reasoning behind the views decision, going by the thread starting with http://mail.python.org/pipermail/python-3000/2006-August/003224.html (it's primarily about slicing strings, but at least one e-mail in the thread mentions mutable objects like lists), and also some things from:
http://mail.python.org/pipermail/python-3000/2007-February/005739.html
http://mail.python.org/pipermail/python-dev/2008-May/079692.html and following e-mails in the thread
Looks like the advantages of switching to this style for base Python would be vastly outweighed by the induced complexity and various undesirable edge cases. Oh well.
...And as I then started wondering about the possibility of just replacing the current way slice objects are worked with with an iterable form a la itertools.islice, just as zip, map, etc. all return iterables instead of lists in Python 3, I started realizing all the unexpected behavior and possible problems that could come out of that. Looks like this might be a dead end for now.
On the plus side, numpy's arrays are fairly flexible, so in situations where this sort of thing might be necessary, it wouldn't be too hard to use one-dimensional ndarrays instead of lists. However, it seems ndarrays don't support using slicing to insert additional items within arrays, as happens with Python lists:
>>> a = [0, 0]
>>> a[:1] = [2, 3]
>>> a
[2, 3, 0]
I think the numpy equivalent would instead be something like this:
>>> a = np.array([0, 0]) # or a = np.zeros([2]), but that's not important here
>>> a = np.hstack(([2, 3], a[1:]))
>>> a
array([2, 3, 0])
A slightly more complicated case:
>>> a = [1, 2, 3, 4]
>>> a[1:3] = [0, 0, 0]
>>> a
[1, 0, 0, 0, 4]
versus
>>> a = np.array([1, 2, 3, 4])
>>> a = np.hstack((a[:1], [0, 0, 0], a[3:]))
>>> a
array([1, 0, 0, 0, 4])
And, of course, the above numpy examples don't store the result in the original array as happens with the regular Python list expansion.