A rather simple question for most of you I suspect, although I can't seem to find the answer to it.
I have a function that takes in a 1D array and returns another array.
At the moment I am using a for loop to run this function many times, but I was suggested that I could increase my number of iterations if I was to create a function that would make a 1D array of a length equal to the number of iterations I want and then apply my function to this array.
For example:
inputs = np.array([23.,56,69])
def do_model(array):
... do magic ...
return new_array
What I have at the moment:
outputs= np.array([np.array([do_model(inputs) for x in xrange(iterations)])
What I would like:
def run_model(iterations, inputs):
# generate a 1D array of length iteration containing the input array
my_array = np.full((1,int(iterations)), inputs)
outputs = run_model(iterations, inputs)
# run my function on the array
return do_model(empty)
At the moment I get the following error when I try to create my_array for 5 iterations:
ValueError: could not broadcast input array from shape (3) into shape (1,5)
I tried to specify that the array should be filled by another array using the dtype keyword argument as such:
my_array = np.full((1,int(iterations)), inputs,dtype=np.ndarray
)
put that didn't work.
Any help welcome,
The reason I'm asking is I try to run the code on millions of iterations and at the moment I have a memory error.
Related
I started studying python and I have a question for the program. I am writing
as the first step I thought of creating a big matrix and a big vectors where I will put some smaller vectors and smaller matrices. I created at random.
I have created a function that returns the max eigen values of those small matrices and save them in a vector. Also I created a function that basically does the multiplication of the transpose of my small vectors with the small matrices that I had randomly calculated. However when I try to pass the value of the multiplication, I get the following error
"setting an array element with a sequence."
import numpy as np
from scipy import linalg as lg
import math
N=5;
n=3;
Qs=np.zeros((n,N*n))
xs=np.zeros(n*N)
qs=np.zeros(n*N)
grads=np.zeros(N*n)
Lfi=np.zeros(N)
pis=np.zeros(N*n)
pi=np.zeros(N)
phat=np.zeros(N)
j=0;
gamma=np.zeros(N)
def gradient(Q,x) :
return x.transpose().dot(Q)
def eigen(Q):
val, vec = lg.eig(Q)
return np.amax(Q)
for i in range(0,N):
A = np.random.randint(10, size=(n,n))
Qs[0:n,j:j+n] += A
x = np.random.randint(10, size=(n))
qs[j:j+n] += x
x = np.random.randint(10, size=(n))
xs[j:j+n] += x
j = j+n
Lfi[i] = eigen(A)
grads[i] = gradient(A,x)
Why do I get the error for the gradient and not for example at the Lfi?
Your gradient function returns something else than what is expected by grads. From the error message I can tell that grads is numpy array which is not as forgiving as default python list. It has a type associated with it and it requires every element to be of that type.
e.g.
a = np.array([0] * 10)
print(type(a[0]))
this will tell you that the type of elements in this array is numpy.int64 (obviously there are many types and this is just a one of them). If you try to assign some value of a different type, python will try to coerce it to numpy.int64. But obviously, there is no sensible (general) way to coerce another array or matrix to a single number.
If you try
a[0] = [1, 2, 3]
then you will receive the error mentioned by you
ValueError: setting an array element with a sequence.
And why don't you receive the error when calling eigen function? Well, because it just returns a single value, not an array. np.amax(Q) returns a single value if you don't specify axis parameter which is set to None by default.
I'm trying to write a function to compute the dot product of two 2D lists passed in as arguments, lets call them x and y.
My idea is to first create a 2D list of zeros of the proper dimensions for the result of the dot product. In order to do so, I need to find the column size of y when computing x * y
dim1 = len(x)
dim2 = len(y[0])
result = [0]*dim1*dim2
The above code was my idea for getting these dimensions, however it fails on the second line due to an error:
dim2 = len(y[0])
TypeError: object of type 'int' has no len()
My python interpreter seems to not like that I am assuming my arguments will be 2D lists. It seems to think it'll be a 1D list. How can I get the column length of the 2D list. I am assuming the 2D lists passed in will be of dimensions NxM, so it should be a clean rectangle shape list/matrix.
I am not able to use numpy for this case.
I defined a two-dimensional array as follows:
predict_result = np.zeros((69,1000))
In the loop, I was trying to inject a predicted one-dimensional array into it.
for ij in range(0,1000):
# # some code to generate Ypredict
predict_result[:,ij]=Ypredict
Ypredict is always the shape of (69,1). However, running the program gives the following error
predict_result[:,ij]=Ypredict ValueError: could not broadcast input
array from shape (69,1) into shape (69)
How can I correct this error?
Either change the (69,1) array to (69,), or make the receiving slot (69,1)
ravel is one on several ways of flattening the 2d array:
predict_result[:,ij]=Ypredict.ravel()
Index with a list rather than a scalar works the other way:
predict_result[:,[ij]]=Ypredict
You could also use broadcasting to put the smaller array into the larger without a loop - as noted in the other answer:
(69,1000) <= (69,1) works
The 2 basic rules of broadcasting:
a size 1 dimension can be added at the start to match the number of dimensions
all size 1 dimensions can be changed to match to same dimension of the other array(s).
(69,) cannot be changed to match (69,1). But (69,1) can be changed to match (69,1000).
You don't need for-loop:
predict_result[:] = Ypredict
Or you can create the result by repeat:
np.repeat(Ypredict, 1000, axis=1)
What is different between x.shape[0] and x.shape in numpy? I code without [0] then have a bug: "TypeError: arange: scalar arguments expected instead of a tuple.", but when I add [0] in, my code runs completely.
And why i can't type x.shape[1] or x.shape[1000]?
Looking forward to receiving answers from everyone, many thanks!!
From your error message:
"TypeError: arange: scalar arguments expected instead of a tuple."
It sounds to me like you are trying to use the shape of an existing array to define the shape of a new array using np.arange.
Your problem is that you don't understand what x.shape is giving you.
For example:
x = np.array([[1,2,3],[4,5,6]])
x.shape
produces (2,3), a tuple. If I try to use just x.shape to define a argument in np.arange like this:
np.arange(x.shape)
I get the following error:
"arange: scalar arguments expected instead of a tuple."
Reason being np.arange accepts either a scalar (which creates an array starting at 0 and increasing by 1 to the length provided) or 3 scalars which define where to start and end the array and the step size. You are giving it a tuple instead which it doesn't like.
So when you do:
np.arange(x.shape[0])
you are giving the arange function the first scalar in the tuple provided by x.shape and in my example producing an array like this [0,1] because the first index in the tuple is 2.
If I alternatively did
np.arange(x.shape[1])
I would get an array like [0,1,2] because the second index in the tuple is a 3.
If I did any of the following,
np.arange(x.shape[2])
np.arange(x.shape[1000])
np.arange(x.shape[300])
I would get an error because the tuple created by x.shape has only two dimensions and so can't be indexed any further than 0 or 1.
Hope that helps!
This is basically what I am trying to do:
array = np.array() #initialize the array. This is where the error code described below is thrown
for i in xrange(?): #in the full version of this code, this loop goes through the length of a file. I won't know the length until I go through it. The point of the question is to see if you can build the array without knowing its exact size beforehand
A = random.randint(0,10)
B = random.randint(0,10)
C = random.randint(0,10)
D = random.randint(0,10)
row = [A,B,C,D]
array[i:]= row # this is supposed to add a row to the array with A,C,B,D as column values
This code doesn't work. First of all it complains: TypeError: Required argument 'object' (pos 1) not found. But I don't know the final size of the array.
Second, I know that last line is incorrect but I am not sure how to call this in python/numpy. So how can I do this?
A numpy array must be created with a fixed size. You can create a small one (e.g., one row) and then append rows one at a time, but that will be inefficient. There is no way to efficiently grow a numpy array gradually to an undetermined size. You need to decide ahead of time what size you want it to be, or accept that your code will be inefficient. Depending on the format of your data, you can possibly use something like numpy.loadtxt or various functions in pandas to read it in.
Use a list of 1D numpy arrays, or a list of lists, and then convert it to a numpy 2D array (or use more nesting and get more dimensions if you need to).
import numpy as np
a = []
for i in range(5):
a.append(np.array([1,2,3])) # or a.append([1,2,3])
a = np.asarray(a) # a list of 1D arrays (or lists) becomes a 2D array
print(a.shape)
print(a)