I am trying to translate a project I have in MATLAB to Python+Numpy because MATLAB keeps running out of memory. The file I have is rather long, so I have tried to make a minimal example that shows the same error.
Basically I'm making a 2d histogram of a dataset, and want to save it after some processing. The problem is that the numpy.save function throws a "ValueError: setting an array element with a sequence" when I try to save the output of the histogram function. I can't find the problem when I look at the docs of Numpy.
My version of Python is 2.6.6, Numpy version 1.4.1 on a Debian distro.
import numpy as np
import random
n_samples = 5
rows = 5
out_file = file('dens.bin','wb')
x_bins = np.arange(-2.005,2.005,0.01)
y_bins = np.arange(-0.5,n_samples+0.5)
listy = [random.gauss(0,1) for r in range(n_samples*rows)]
dens = np.histogram2d( listy, \
range(n_samples)*rows, \
[y_bins, x_bins])
print 'Write data'
np.savez(out_file, dens)
out_file.close()
Full output:
$ python error.py
Write data
Traceback (most recent call last):
File "error.py", line 19, in <module>
np.savez(out_file, dens)
File "/usr/lib/pymodules/python2.6/numpy/lib/io.py", line 439, in savez
format.write_array(fid, np.asanyarray(val))
File "/usr/lib/pymodules/python2.6/numpy/core/numeric.py", line 312, in asanyarray
return array(a, dtype, copy=False, order=order, subok=True)
ValueError: setting an array element with a sequence.
Note that np.histogram2d actually returns a tuple of three arrays: (hist, x_bins, y_bins). If you want to save all three of these, you have to unpack them as #Francesco said.
dens = np.histogram2d(listy,
range(n_samples)*rows,
[y_bins, x_bins])
np.savez('dens.bin', *dens)
Alternatively, if you only need the histogram itself, you could save just that.
np.savez('dens.bin', dens[0])
If you want to keep track of which of these is which, use the **kwds instead of the *args
denskw = dict(zip(['hist','y_bins','x_bins'], dens))
np.savez('dens.bin', **denskw)
Then, you can load it like
dens = np.load('dens.bin')
hist = dens['hist']# etc
Related
I'm trying to generate and plot random numbers using:
from numpy import random
import matplotlib.pyplot as plt
z = 15 + 2*random.randn(200) #200 elements, normal dist with mean = 15, sd = 2
plt.plot(z)
plt.show(z)
The graph is plotted, but Python (2.7.5) freezes and I get the error
Traceback (most recent call last):
File "G:\Stage 2 expt\e298\q1.py", line 25, in <module>
plt.show(z)
File "C:\Python27\lib\site-packages\matplotlib\pyplot.py", line 145, in show
_show(*args, **kw)
File "C:\Python27\lib\site-packages\matplotlib\backend_bases.py", line 90, in __call__
if block:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
It's completely fine when I do a for loop like so:
from numpy import random
from pylab import plot,show
yvec = [] # set up an empty vector
for i in range(200): # want 200 numbers
yy = 25 + 3*random.randn() # normal dist with mean = 15, sd = 2
yvec.append(yy) # enter yy into vector
plot(yvec)
show(yvec)
Could someone please clarify?
The function pylab.show does not take a list or array, it takes an optional boolean (and certainly not your data array). The numpy array in the first example can't be implicitly converted to a boolean, thus throwing an error. The second one can however be converted to a boolean, and it will evaluate to True if non-empty.
To fix it, just call show without any arguments.
I get the following error while reshaping a numpy ndarray
DeprecationWarning: :func:`reshape` is deprecated, use :func:`numerix.reshape()<numpy.reshape>` instead!
return reshape(newshape, order=order)
Traceback (most recent call last):
File "./render2.py", line 374, in <module>
,u=np.reshape(voltage.grad[0], (ny, nx))
File "/home/jana/Builds/lib/python2.6/site-packages/numpy/core/fromnumeric.py", line 172, in reshape
return reshape(newshape, order=order)
File "/home/jana/Builds/lib/python2.6/site-packages/fipy/tools/decorators.py", line 151, in newfunc
return func(*args, **kwds)
TypeError: reshape() got an unexpected keyword argument 'order'
Below is the part of the code that gives this error. Note: plot.py is a user defined module.
plot.streamlinePlot(x = x
,y = y
,u=np.reshape(voltage.grad[0], (ny, nx))
,v=np.reshape(voltage.grad[1], (ny, nx))
,filename='Analysis/electricFieldStreamPlot_%s.png'
,show=False
,clear=True)
The output of
print "Voltage shape =", voltage.shape
print "Voltage.grad[0] shape =", voltage.grad[0].shape
print "ny times nx =", ny*nx
is
Voltage shape = (269700,)
Voltage.grad[0] shape = (269700,)
ny times nx = 269700
I am running FiPy 3.0 and NumPy 1.7.2.
Any clues? Thanks!
You should get the desired result by calling
from fipy import numerix as nx
nx.reshape(voltage.grad[0], (ny, nx))
FiPy overrides a number of NumPy routines for working with its own data structures in a self-consistent way. You should always use fipy.numerix instead of numpy when working with FiPy objects.
If you aren't aware, FiPY now includes a MatplotlibStreamViewer that may either serve your needs or at least show you the data manipulations you'll need to perform for your own display.
There's definitely something wrong in the interaction between numpy.reshape(), fipy.numerix.reshape(), and fipy.CellVariable.reshape(). I've filed a ticket to look into this. Thanks for raising the question.
I am trying to read float numbers from a Binary file using Struct module and then storing them in numpy 3D arrays.When I run it as an independent script, it works fine.But when I call it as a class's function from another script(after import) it gives value error
Here is my code.
import struct
from numpy import *
class DCD_read:
def read_cord(self,total_atoms,dcd_data):
cord_data=dcd_data[276:len(dcd_data)] ## binary data string
byte=0
count=0
total_frames=info_dict['nset']
coord=numpy.zeros((total_frames,total_atoms,3)) ## 3d array
for frames in range(0,total_frames):
for atoms in range(0,total_atoms):
x = list(struct.unpack('<f',cord_data[60+byte:64+byte])) ###reading float
byte+=4
y = list(struct.unpack('<f',cord_data[60+byte:64+byte]))
byte+=4
z = list(struct.unpack('<f',cord_data[60+byte:64+byte]))
byte+=4
ls=x
ls.extend(y)
ls.extend(z)
coord[frames][atoms]=ls
return coord
Error:
Traceback (most recent call last):
File "C:\Users\Hira\Documents\PROJECT\md.py", line 24, in <module>
coord=dcd.read_cord(total_atoms,dcd_data)
File "C:\Users\Hira\Documents\PROJECT\DCD_read.py", line 51, in read_cord
coord=numpy.zeros((total_frames,total_atoms,3))
File "C:\Python27\numpy\core\numeric.py", line 148, in ones
a = empty(shape, dtype, order)
ValueError: negative dimensions are not allowed
md.py is the main (calling script) while DCD_read.py is module. Here is code for md.py (main script)
from DCD_read import *
import numpy
dcd_file=open('frame3.dcd',"rb")
dcd_data=dcd_file.read()
dcd=read_dcd()
total_atoms=6141
coord=dcd.read_cord(total_atoms,dcd_data)
Please can any one help???? I hope I explained it completely and clearly.Thanx
I am trying to convert an image from cartesian to polar so that I can unravel the image, but I am getting a runtime error. If you are curious how this looks visually, see this example.
Code:
import scipy
import scipy.ndimage
import numpy as np
from math import *
import cv2
def logpolar(input):
# This takes a numpy array and returns it in Log-Polar coordinates.
coordinates = np.mgrid[0:max(input.shape[:])*2,0:360] # We create a cartesian array which will be used to compute log-polar coordinates.
log_r = 10**(coordinates[0,:]/(input.shape[0]*2.)*log10(input.shape[1])) # This contains a normalized logarithmic gradient
angle = 2.*pi*(coordinates[1,:]/360.) # This is a linear gradient going from 0 to 2*Pi
# Using scipy's map_coordinates(), we map the input array on the log-polar coordinate. Do not forget to center the coordinates!
lpinput = scipy.ndimage.interpolation.map_coordinates(input,(log_r*np.cos(angle)+input.shape[0]/2.,log_r*np.sin(angle)+input.shape[1]/2.),order=3,mode='constant')
# Returning log-normal...
return lpinput
# Load image
image = cv2.imread("test.jpg")
result = logpolar(image)
Error message in console:
Traceback (most recent call last):
File "test.py", line 23, in <module>
result = logpolar(image)
File "test.py", line 15, in logpolar
lpinput = scipy.ndimage.interpolation.map_coordinates(input,(log_r*np.cos(angle)+input.shape[0]/2.,log_r*np.sin(angle)+input.shape[1]/2.),order=3,mode='constant')
File "/Library/Python/2.7/site-packages/scipy-0.13.0.dev_c31f167_20130415-py2.7-macosx-10.8-intel.egg/scipy/ndimage/interpolation.py", line 295, in map_coordinates
raise RuntimeError('invalid shape for coordinate array')
RuntimeError: invalid shape for coordinate array
My first guess would be that you are passing in a colour image which is 3 dimensional. At first glance I don't think your code could handle that.
My guess was based off of the error you pasted, specifically
"invalid shape for coordinate array"
When using higher dimensional arrays like that usually you have to pass extra parameters around specifying which axis to repeat the operations over and even then sometimes it does not work. I didn't see a repeated extra integer at the end of your argument lists so I figured you weren't trying to handle that case explicitly and might have forgotten to check your array dimensions after reading in the image.
Glad it helped :)
For clarity I have isolated my problem and used a small but complete snippet to describe it.
I have a bunch of data but there is a lot of missing pieces. I want to ignore these (a break in the graph if it were a line graph). I have set "?" to be the symbol for missing data. Here is my snippet:
import math
import Gnuplot
gp = Gnuplot.Gnuplot(persist=1)
gp("set datafile missing '?'")
x = range(1000)
y = [math.sin(a) + math.cos(a) + math.tan(a) for a in x]
# Force a piece of missing data
y[4] = '?'
data = Gnuplot.Data(x, y, title='Plotting from Python')
gp.plot(data);
gp.hardcopy(filename="pyplot.png",terminal="png")
But it doesn't work:
> python missing_test.py
Traceback (most recent call last):
File "missing_test.py", line 8, in <module>
data = Gnuplot.Data(x, y, title='Plotting from Python')
File "/usr/lib/python2.6/dist-packages/Gnuplot/PlotItems.py", line 560, in Data
data = utils.float_array(data)
File "/usr/lib/python2.6/dist-packages/Gnuplot/utils.py", line 33, in float_array
return numpy.asarray(m, numpy.float32)
File "/usr/lib/python2.6/dist-packages/numpy/core/numeric.py", line 230, in asarray
return array(a, dtype, copy=False, order=order)
ValueError: setting an array element with a sequence.
What's going wrong?
Gnuplot is calling numpy.asarray to convert your Python list into a numpy array.
Unfortunately, this command (with dtype=numpy.float32) is incompatible with a Python list that contains strings.
You can reproduce the error like this:
In [36]: np.asarray(['?',1.0,2.0],np.float32)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/usr/lib/python2.6/dist-packages/numpy/core/numeric.pyc in asarray(a, dtype, order)
228
229 """
--> 230 return array(a, dtype, copy=False, order=order)
231
232 def asanyarray(a, dtype=None, order=None):
ValueError: setting an array element with a sequence.
Furthermore, the Gnuplot python module (version 1.7) docs say
There is no provision for missing data points in array data (which
gnuplot allows via the 'set missing' command).
I'm not sure if this has been fixed in version 1.8.
How married are you to gnuplot? Have you tried matplotlib?