We Keep Coding

How can I call values of a np.array rather than memory address? [duplicate]

This question already has an answer here:
Python The appended element in the list changes as its original variable changes
(1 answer)
Closed 4 years ago.
In the following code, I intended to make a list starting from an empty one
by appending (random) numpy arrays. For a temporary variable, I initialized a numpy array variable 'sample_pt' which worked as a temporary variable to save a (random) numpy array. While I expected to have a list of random numpy arrays, the output was a list filled with the same (final) numpy array. I suspect that calling a numpy array by its "variable name" returns its memory address. Am I on the right direction, or are there anything that would be good to know?
[Code]
import numpy as np
sample_pt=np.array([0.]) # initial point
sample_list=[]
number_iter=3
for _ in range(number_iter):
sample_pt[0]=np.random.randn()
sample_list.append(sample_pt)
print(sample_list)
[Output]
[array([-0.78614157])]
[array([0.7172035]), array([0.7172035])]
[array([0.47565398]), array([0.47565398]), array([0.47565398])]

I don't know what you mean by "call values", or "rather than memory address", or… most of the text of your question.
But the problem is pretty simple. You're appending the same array over and over, instead of creating new ones.
If you want to create a new array, you have to do that explicitly. Which is trivial to do; just move the np.array constructor into the loop, like this:
sample_list=[]
number_iter=3
for _ in range(number_iter):
sample_pt=np.array([0.]) # initial point
sample_pt[0]=np.random.randn()
sample_list.append(sample_pt)
print(sample_list)
But this can be dramatically simplified.
First, instead of creating an array of 1 zero and then replacing that zero, why not just create an array of the element you want?
sample_pt = np.array([np.random.randn()])
Or, even better, why not just let np.random build the array for you?
sample_pt = np.random.randn(1)
At which point you could replace the whole thing with a list comprehension:
number_iter = 3
sample_list = [np.random.randn(1) for _ in range(number_iter)]
Or, even better, why not make a 3x1 array instead of a list of 3 single-element arrays?
number_iter = 3
sample_array = np.random.randn((number_iter, 1))
If you really need to change that into a list of 3 arrays for some reason, you can always call list on it later:
sample_list = list(sample_array)
… or right at the start:
sample_list = list(np.random.randn((number_iter, 1)))
Meanwhile, I think you misunderstand how values and variables work in Python.
First, forget about "memory address" for a second:
An object is a value, with a type, somewhere in the heap. You don't care where.
Variables don't have memory addresses, or types; they're just names in some namespace (globals, locals, attributes of some instance, etc.) that refer to some value somewhere.
Notice that this is very different from, say, C++, where variables are typed memory locations, and objects live in those memory locations. This means there's no "copy constructor" or "assignment operator" or anything like that in Python. When you write a = b, all that means is that a is now another name for the same value as b. If you want a copy, you have to explicitly ask for a copy.
Now, if you look at how CPython implements things under the hood:
The CPython interpreter represents all objects as pointers to PyObject structs, which are always allocated on the heap.
Variables are just string keys in a dict, owned by the module (for globals), an instance (for attributes), or whatever. The values in the dict are just objects like any other. Which means that, under the covers, what's actually stored in the hash table is pointers to string objects for the variable names in the keys, and pointers to whatever value you've assigned in the values.
There is a special optimization for locals, involving an array of object pointers stored on the frame, but you usually don't have to worry about that.
There's another special trick for closure captures, involving pointers to cell objects that hold pointers to the actual objects, which you have to worry about even less often.
As you can see, thinking about the pointers is harder to understand, and potentially misleading, unless you really care about how CPython works under the covers.

Copy 2d array in Python: Confusion

My understanding is that:
def copy_2d(p):
return list(p)
Would make a full copy of p and return it as a result. list( p) seems to do this when I try it in the repl. However it seems like calling the above method like this:
b = [[1,2,3],[3,4,5]]
a = copy_2d(b)
a[0][0] = 0
if (b[0][0] == 0): print "Huh??"
It prints "Huh??", that is, it appears that b is just a reference to a. I double checked but I might be blind. Can someone clarify please?

Your current code for copy_2d returns a shallow copy of the list you pass as its argument. That is, you're creating a new outer list, but the inner values (which may be lists themselves) are not copied. The new list references the same inner lists, so when you mutate one of them, you'll see the same change in the shallow copy as you do in the original list.
You can fix the issue by copying the inner lists as well as creating a new outer list. Try:
def copy_2d(p):
return map(list, p) # warning: this only works as intended in Python 2
In Python 2, this works because map returns a list. In Python 3, the map function returns an iterator, so a list comprehension like [list(inner) for inner in p] would be better.
Of course, if you don't need to write your own code to solve this problem, you should just use copy.deepcopy from the standard library.

import copy
def copy_2d(p):
return copy.deepcopy(p)
or
def copy_2d(p):
return [list(p2) for p2 in p]
What you did copied the array with all the values inside. But inside you had objects with were arrays. Your function did not copied inside lists but their references.
The second solution still copies references, but one layer below.

Shallow copy made. Same logic as here?what-does-the-list-function-do-in-python
"list() converts the iterable passed to it to a list. If the iterable is already a list then a shallow copy is returned, i.e only the outermost container is new rest of the objects are still the same."

Why are integers immutable in Python?

I understand the differences between mutable and immutable objects in Python. I have read many posts discussing the differences. However, I have not read anything regarding WHY integers are immutable objects.
Does there exist a reason for this? Or is the answer "that's just how it is"?
Edit: I am getting prompted to 'differentiate' this question from other questions as it seems to be a previously asked question. However, I believe what I'm asking is more of a philosophical Python question rather than a technical Python question.
It appears that 'primitive' objects in Python (i.e. strings, booleans, numbers, etc.) are immutable. I've also noticed that derived data types that are made up of primitives (i.e. dicts, lists, classes) are mutable.
Is that where the line is drawn whether or not an object is mutable? Primitive vs derived?

Making integers mutable would be very counter-intuitive to the way we are used to working with them.
Consider this code fragment:
a = 1 # assign 1 to a
b = a+2 # assign 3 to b, leave a at 1
After these assignments are executed we expect a to have the value 1 and b to have the value 3. The addition operation is creating a new integer value from the integer stored in a and an instance of the integer 2.
If the addition operation just took the integer at a and just mutated it then both a and b would have the value 3.
So we expect arithmetic operations to create new values for their results - not to mutate their input parameters.
However, there are cases where mutating a data structure is more convenient and more efficient. Let's suppose for the moment that list.append(x) did not modify list but returned a new copy of list with x appended.
Then a function like this:
def foo():
nums = []
for x in range(0,10):
nums.append(x)
return nums
would just return the empty list. (Remember - here nums.append(x) doesn't alter nums - it returns a new list with x appended. But this new list isn't saved anywhere.)
We would have to write the foo routine like this:
def foo():
nums = []
for x in range(0,10):
nums = nums.append(x)
return nums
(This, in fact, is very similar to the situation with Python strings up until about 2.6 or perhaps 2.5.)
Moreover, every time we assign nums = nums.append(x) we would be copying a list that is increasing in size resulting in quadratic behavior.
For those reasons we make lists mutable objects.
A consequence to making lists mutable is that after these statements:
a = [1,2,3]
b = a
a.append(4)
the list b has changed to [1,2,3,4]. This is something that we live with even though it still trips us up now and then.

What are the design decisions to make numbers immutable in Python?
There are several reasons for immutability, let's see first what are the reasons for immutability?
1- Memory
Saves memory. If it's well known that an object is immutable, it can be easily copied creating a new reference to the same object.
Performance. Python can allocate space for an immutable object at creation time, and the storage requirements are fixed and unchanging.
2- Fast execution.
It doesn't have to copy each part of the object, only a simple reference.
Easy to be compared, comparing equality by reference is faster than comparing values.
3- Security:
In Multi-threading apps Different threads can interact with data contained inside the immutable objects, without to worry about data consistency.
The internal state of your program will be consistent even if you have exceptions.
Classes should be immutable unless there's a very good reason to make them mutable....If a class cannot be made immutable, limit its mutability as much as possible
4- Ease to use
Is easier to read, easier to maintain and less likely to fail in odd and unpredictable ways.
Immutable objects are easier to test, due not only to their easy mockability, but also the code patterns they tend to enforce.
5- Keys must be immutable. Which means you can use strings, numbers or tuples as dictionary key. This is something that you want to use.
The hash table implementation of dictionaries uses a hash value calculated from the key value to find the key. If the key were a mutable object, its value could change, and thus its hash could also change. But since whoever changes the key object can’t tell that it was being used as a dictionary key, it can’t move the entry around in the dictionary. Then, when you try to look up the same object in the dictionary it won’t be found because its hash value is different. If you tried to look up the old value it wouldn’t be found either, because the value of the object found in that hash bin would be different.
Going back to the integers:
Security (3), Easy to use (4) and capacity of using numbers as keys in dictionaries (5) are reasons for taken the decision of making numbers immutable.
Has fixed memory requirements since creation time (1).
All in Python is an object, the numbers (like strings) are "elemental" objects. No amount of activity will change the value 8 to anything else, and no amount of activity will change the string “eight” to anything else. This is because a decision in the design too.

Python: Create List of Object References

I want to clean up some code I've written, in order to scale the magnitude of what I'm trying to do. In order to do so, I'd like to ideally create a list of references to objects, so that I can systematically set the objects, using a loop, without actually have to put the objects in list. I've read about the way Python handles references and pass-by, but haven't quite found a way to do this effectively.
To better demonstrate what I'm trying to do:
I'm using bokeh, and would like to set up a large number of select boxes. Each box looks like this
select_one_name = Select(
title = 'test',
value = 'first_value',
options = ['first_value', 'second_value', 'etc']
)
Setting up each select is fine, when I only have a few, but when I have 20, my code gets very long and unwieldy. What I'd like to be able to do, is have a list of sample_list = [select_one_name, select_two_name, etc] that I can then loop through, to set the values of each select_one_name, select_two_name, etc. However, I want to have my reference select_one_name still point to the correct value, rather than necessarily refer to the value by calling sample_list[0].
I'm not sure if this is do-able--if there's a better way to do this, than creating a list of references, please let me know. I know that I could just create a list of objects, but I'm trying to avoid that.
For reference, I'm on Python 2.7, Anaconda distribution, Windows 7. Thanks!
To follow up on #Alex Martelli's post below:
The reason why I thought this might not work, is because when I tried a mini-test with a list of lists, I didn't get the results I wanted. To demonstrate
x = [1, 2, 3]
y = [4, 5, 6]
test = [x, y]
test[0].append(1)
Results in x = [1, 2, 3, 1] but if instead, I use test[0] = [1, 2], then x remains [1, 2, 3], although test itself reflects the change.
Drawing a parallel back to my original example, I thought that I would see the same results as from setting to equal. Is this not true?

Every Python list always is internally an array of references (in CPython, which is no doubt what you're using, at the C level it's an array of PyObject* -- "pointers to Python objects").
No copies of the objects get made implicitly: rather (again, in CPython) each object's reference count gets incremented when the you add "the object" (actually a reference to it) to the list. In fact when you do want an object's copy you need to specifically ask for one (with the copy module in general, or sometimes with type-specific copy methods).
Multiple references to the same object are internally pointers to exactly the same memory. If an object is mutable, then mutating it gets reflected through all the references to it. Of course, there are immutable objects (strings, numbers, tuples, ...) to which such mutation cannot apply.
So when you do, e.g,
sample_list = [select_one_name, select_two_name, etc]
each of the names (as long as it's in scope) still refers to exactly the same object as the corresponding item in sample_list.
In other words, using sample_list[0] and select_one_name is totally equivalent as long as both references to the same object exist.
IOW squared, your stated purpose is already accomplished by Python's most fundamental semantics. Now, please edit the Q to clarify which behavior you're observing that seems to contradict this, versus which behavior you think you should be observing (and desire), and we may be able to help further -- because to this point all the above observations amount to "you're getting exactly the semantics you ask for" so "steady as she goes" is all I can practically suggest!-)
Added (better here in the answer than just below in comments:-): note the focus on mutating operation. The OP tried test[0]= somelist followed by test[0].append and saw somelist mutated accordingly; then tried test[0] = [1, 2] and was surprised to see somelist not changed. But that's because assignment to a reference is not a mutating operation on the object that said reference used to indicate! It just re-seats the reference, decrement the previously-referred-to object's reference count, and that's it.
If you want to mutate an existing object (which needs to be a mutable one in the first place, but, a list satisfies that), you need to perform mutating operations on it (through whatever reference, doesn't matter). For example, besides append and many other named methods, one mutating operation on a list is assignment to a slice, including the whole-list slice denoted as [:]. So, test[0][:] = [1,2] would in fact mutate somelist -- very different from test[0] = [1,2] which assigns to a reference, not to a slice.

This is not recommended, but it works.
sample_list = ["select_one_name", "select_two_name", "select_three_name"]
for select in sample_list:
locals()[select] = Select(
title = 'test',value = 'first_value',
options = ['first_value', 'second_value', 'etc']
)
You can use select_one_name, select_two_name, etc directly because they're set in the local scope due the special locals() list.
A cleaner approach is to use a dictionary, e.g.
selects = {
'select_one_name': Select(...),
'select_two_name': Select(...),
'select_three_name': Select(...)
}
And reference selects['select_one_name'] in your code and you can iterate over selects.keys() or selects.items().

How to assign the same value to multiple variables in Python

I have a question about Python, which I am kinda new to. Let's assume I want to assign a 5x5 matrix to 10 different variables. I searched across the board, and what I found was this:
a, b, c, d, e = myMatrix
That is all good, but in Python, this means that when I change a, I also change the values of the other variables, because they all come down to the same memory adress if I got this correctly.
My question: Is there a fast way of assigning myMatrix to multiple Variables and giving each of them a unique memory adress? So that I can change myMatrix without changing a, b or c. I do explicitly search for some kind of multi-assignment.
Thanks in advance!

use the [copy] module
>>> import copy
>>> new_matrix = copy.deepcopy(myMatrix)

As Burhan Khalid and juanchopanza have pointed out, what happens in your example will be different in, for example,
the case where "myMatrix" is actually an array of 5 values (in which case "a" will get the first value and "e" will get the last value), and
the case where "myMatrix" is an instance of an Object (in which case "a" through "e" will each refer to the same object).
It sounds like you're thinking of case 2, and hoping for something like a macro which will automatically expand your single assignment statement (with a single Right Hand Side Value, whether Deep Copied or not) into 5 assignment statements, each with its own Left Hand Side, Right Hand Side, and Deep Copy.
I don't know of any way to do this, and I would point out that:
When most OO languages encounter an assignment operation like yours with an Object on the Right Hand Side, the compiler/interpreter looks for a "copy constructor" for the class of the RHS Object, and uses it (if found) to generate the value (an Object reference) which is actually assigned to the LHS. Can you even imagine what the syntax could look like for what you're describing, where the copy constructor is supposed to be called 5 times to yield 5 different Objects on the RHS, references to which are then assigned to five different variables on the LHS? What could you possibly write in a single assignment statement that would make this intent clear?
If you're writing code where Deep vs. Shallow copies will actually have an effect on behavior then IMHO you owe it to yourself and anyone else who has to read and maintain your code to make this obvious and explicit - like the answer from wong2, repeated 5 times (once for each of the 5 variables).