from scipy import ndimage
import numpy as np
with open("Data.dat", "r+") as f:
content = f.readlines()
content = [s.strip() for s in content]
content = np.asarray(content)
weights = np.array([1, 2, 4, 2, 1])
final = np.empty(content)
ndimage.filters.convolve1d(content, weights, -1, final, 'reflect', 0.0, 0.0)
Here np is the numpy package. The length of content is 750, so tried to initialize the shape of the output array with np.empty() as I don't want any value in that.
But when I run I get this error:
Traceback (most recent call last):
File "C:\Users\Animesh\Desktop\Smoothing.py", line 18, in <module>
final = np.empty(content)
ValueError: sequence too large; must be smaller than 32
What should be done ?
To make final an empty array of the same shape and dtype as content, use:
final = np.empty_like(content)
Regarding the TypeError: integer argument expected, got float:
Although the docstring for convolve1d says
origin : scalar, optional
The `origin` parameter controls the placement of the filter.
Default 0.0.
the origin argument must be an integer, not a float.
Here is an example which runs without error:
import scipy.ndimage as ndimage
import numpy as np
# content = np.genfromtxt('Data.dat')
content = np.asarray(np.arange(750))
weights = np.array([1, 2, 4, 2, 1])
final = np.empty_like(content)
ndimage.convolve1d(content, weights, axis=-1, output=final, mode='reflect', cval=0.0,
origin=0
# origin=0.0 # This raises TypeError
)
print(final)
Uncommenting origin=0.0 raises the TypeError.
Regarding
with open("Data.dat", "r+") as f:
content = f.readlines()
content = [s.strip() for s in content]
content = np.asarray(content)
This makes content an array of strings. Since you are taking a convolution, you must want an array of numbers. So instead replace the above with
content = np.genfromtxt('Data.dat')
Related
Hi I am trying to work out why I am getting this Value Error. I am opening a table I created in Excel and assigning it to an array in Python. I have some satellite imagery that I want to find some statistics for. It is throwing up a value error, and I have checked that the image opens in Python beforehand and it does.
First I am opening the table in python
path = "D:\\Iceberg data\\RADARSAT\\"
Table_icebergs_all = pd.read_excel(path+'Image_index_SG_C.xlsx', '20160415')
# the last argument in the function selects the sheet inside the excel file
# HERE IT EXTRACTS THE ICEBERG POSITIONS IN THE TABLE
dim = Table_icebergs_all.shape
# GETS THE DIMENSION OF THE TABLES
dim_sat = iceberg_sat.shape[0]
# ALLOCATE THE COORDINATES TO NUMPY ARRAYS FOR SATELLITE POINTS
x_sat = np.zeros(dim_sat, dtype = 'int')
y_sat = np.zeros(dim_sat, dtype = 'int')
for i in range(0, dim_sat):
x_sat[i] = np.round(iceberg_sat['python x'][i])
y_sat[i] = np.round(iceberg_sat['python y'][i])
# HERE IT EXTRACTS THE CLUTTER POSITION IN THE TABLE
# cl1 is the upper left corner
# cl2 is the bottom right corner
############# FOR THE SATELLITE IMAGE
cl1_x_sat = np.zeros(dim_sat, dtype = 'int')
cl1_y_sat = np.zeros(dim_sat, dtype = 'int')
cl2_x_sat = np.zeros(dim_sat, dtype = 'int')
cl2_y_sat = np.zeros(dim_sat, dtype = 'int')
for i in range(0, dim_sat):
cl1_x_sat[i] = np.round(iceberg_sat['py x min'][i])
cl1_y_sat[i] = np.round(iceberg_sat['py y min'][i])
cl2_x_sat[i] = np.round(iceberg_sat['py x max'][i])
cl2_y_sat[i] = np.round(iceberg_sat['py y max'][i])
sat = T11_april15
gnd = gpri_april15
ice_sat_max = np.zeros(dim_sat)
cl_sat_max = np.zeros(dim_sat)
cl_sat_mean = np.zeros(dim_sat)
for i in range(dim_sat):
ice_sat_max[i] = sat[x_sat[i],y_sat[i]]
cl_sat_mean[i] = np.mean(sat[cl1_x_sat[i]:cl2_x_sat[i],cl1_y_sat[i]:cl2_y_sat[i]])
cl_sat_max[i] = np.max(sat[cl1_x_sat[i]:cl2_x_sat[i],cl1_y_sat[i]:cl2_y_sat[i]])
# we evalute the target to clutter ratio using the mean or the max of the clutter
TCR_sat_mean = ice_sat_max/cl_sat_mean
TCR_sat_max = ice_sat_max/cl_sat_max
Here is the error code:
Traceback (most recent call last):
File "<ipython-input-5-2483c16a03b9>", line 12, in <module>
cl_sat_max[i] = np.max(sat[cl1_x_sat[i]:cl2_x_sat[i],cl1_y_sat[i]:cl2_y_sat[i]])
File "<__array_function__ internals>", line 6, in amax
File "C:\Users\job2\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\fromnumeric.py", line 2668, in amax
keepdims=keepdims, initial=initial, where=where)
File "C:\Users\job2\AppData\Local\Continuum\anaconda3\lib\site-packages\numpy\core\fromnumeric.py", line 90, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation maximum which has no identity
Does anyone know how to fix this? It is using the numpy mean and numpy max.
I am using the code from https://github.com/pin3da/spectral-clustering/blob/master/spectral/utils.py to spectrally cluster data in https://cs.joensuu.fi/sipu/datasets/s1.txt
May i know how I can change the code such that it can take in txt file as input?
I have given the original code below for reference
Original code from GitHub
import numpy
import scipy.io
import h5py
def load_dot_mat(path, db_name):
try:
mat = scipy.io.loadmat(path)
except NotImplementedError:
mat = h5py.File(path)
return numpy.array(mat[db_name]).transpose()
I do not understand the purpose of the variable, db_name
The code you show here just opens a given mat or h5 file. The path to the file (path) and the name of the data set within the file (db_name) are provided as arguments to the load_dot_mat function.
To load your txt file, we can create our own little load function:
def load_txt(filename):
with open(filename, "r") as f:
data = [[int(x) for x in line.split(" ") if x != ""] for line in f]
return np.array(data)
This function takes the path to your "txt" file as an argument an returns a numpy array with the data from your file. The data array has shape (5000,2) for the file you provided. You may want to use float instead of int, if other files contain float values and not only integers.
The complete clustering step for your data could then look like this:
from itertools import cycle, islice
import matplotlib.pyplot as plt
import numpy as np
import seaborn
from spectral import affinity, clustering
seaborn.set()
def load_txt(filename):
with open(filename, "r") as f:
data = [[int(x) for x in line.split(" ") if x != ""] for line in f]
return np.array(data)
data = load_txt("s1.txt")
A = affinity.com_aff_local_scaling(data)
n_cls = 15 # found by looking at your data
Y = clustering.spectral_clustering(A, n_cls)
colors = np.array(list(islice(cycle(seaborn.color_palette()), int(max(Y) + 1))))
fig = plt.figure(1)
ax = fig.add_subplot(111)
ax.scatter(data[:, 0], data[:, 1], color=colors[Y], s=6, alpha=0.6)
plt.show()
i have a simple program that i have pasted below.I have a problem because when I run the program I get an error. Here are my errors:
Traceback (most recent call last):
File "C:\Users\Anaconda3\lib\site-packages\pywt\_multilevel.py", line 90, in wavedec
axes_shape = data.shape[axis]
IndexError: tuple index out of range
During handling of the above exception, another exception occurred:
python
Traceback (most recent call last):
File "C:/Users/Main.py", line 10, in <module>
tree = pywt.wavedec(data=record, wavelet='db2', level=5, mode='symmetric')
File "C:\Users\Anaconda3\lib\site-packages\pywt\_multilevel.py", line 92, in wavedec
raise ValueError("Axis greater than data dimensions")
ValueError: Axis greater than data dimensions
And this is my code:
import wfdb
import pywt
import matplotlib.pyplot as plt
record = wfdb.rdsamp('230', sampto = 2000)
annotation = wfdb.rdann('230', 'atr', sampto = 2000)
wfdb.plotrec(record, annotation = annotation, title='Output record', timeunits = 'seconds')
tree = pywt.wavedec(data=record, wavelet='db2', level=5, mode='symmetric')
newTree = [tree[0], tree[1], tree[2], tree[3]*0, tree[4]*0, tree[5]*0]
recSignal = pywt.waverec(newTree, 'db2')
plt.plot(recSignal[:2000])
What, in your opinion, could change in the code to make the program work?
This is this code to line 90
def wavedec(data, wavelet, mode='symmetric', level=None, axis=-1):
"""
Multilevel 1D Discrete Wavelet Transform of data.
Parameters
----------
data: array_like
Input data
wavelet : Wavelet object or name string
Wavelet to use
mode : str, optional
Signal extension mode, see Modes (default: 'symmetric')
level : int, optional
Decomposition level (must be >= 0). If level is None (default) then it
will be calculated using the ``dwt_max_level`` function.
axis: int, optional
Axis over which to compute the DWT. If not given, the
last axis is used.
Returns
-------
[cA_n, cD_n, cD_n-1, ..., cD2, cD1] : list
Ordered list of coefficients arrays
where `n` denotes the level of decomposition. The first element
(`cA_n`) of the result is approximation coefficients array and the
following elements (`cD_n` - `cD_1`) are details coefficients arrays.
Examples
--------
>>> from pywt import wavedec
>>> coeffs = wavedec([1,2,3,4,5,6,7,8], 'db1', level=2)
>>> cA2, cD2, cD1 = coeffs
>>> cD1
array([-0.70710678, -0.70710678, -0.70710678, -0.70710678])
>>> cD2
array([-2., -2.])
>>> cA2
array([ 5., 13.])
"""
data = np.asarray(data)
if not isinstance(wavelet, Wavelet):
wavelet = Wavelet(wavelet)
try:
axes_shape = data.shape[axis]
except IndexError:
raise ValueError("Axis greater than data dimensions")
level = _check_level(axes_shape, wavelet.dec_len, level)
coeffs_list = []
a = data
for i in range(level):
a, d = dwt(a, wavelet, mode, axis)
coeffs_list.append(d)
coeffs_list.append(a)
coeffs_list.reverse()
return coeffs_list
This is my code
import os
import sys
import numpy as np
import scipy
from scipy.optimize import leastsq
def peval (inp_mat,p):
m0,m1,m2,m3,m4,m5,m6,m7 = p
out_mat = np.array(np.zeros(inp_mat.shape,dtype=np.float32))
mid = inp_mat.shape[0]/2
for xy in range(0,inp_mat.shape[0]):
if (xy<(inp_mat.shape[0]/2)):
out_mat[xy] = ( ( (inp_mat[xy+mid]*m0)+(inp_mat[xy]*m1)+ m2 ) /( (inp_mat[xy+mid]*m6)+(inp_mat[xy]*m7)+1 ) )
else:
out_mat[xy] = ( ( (inp_mat[xy]*m3)+(inp_mat[xy-mid]*m4)+ m5 ) /( (inp_mat[xy]*m6)+(inp_mat[xy-mid]*m7)+1 ) )
return np.array(out_mat)
def residuals(p, out_mat, inp_mat):
m0,m1,m2,m3,m4,m5,m6,m7 = p
err=np.array(np.zeros(inp_mat.shape,dtype=np.float32))
if (out_mat.shape == inp_mat.shape):
for xy in range(0,inp_mat.shape[0]):
err[xy] = err[xy]+ (out_mat[xy] -inp_mat[xy])
return np.array(err)
f = open('/media/anilil/Data/Datasets/repo/txt_op/vid.txt','r')
x = np.loadtxt(f,dtype=np.int16,comments='#',delimiter='\t')
nof = x.shape[0]/72 # Find the number of frames
x1 = x.reshape(-1,60,40)
x1_1= x1[0,:,:].flatten()
x1_2= x1[1,:,:].flatten()
x= []
y= []
for xy in range(1,50,1):
y.append(x1[xy,:,:].flatten())
x.append(x1[xy-1,:,:].flatten())
x=np.array(x,dtype=np.float32)
y=np.array(y,dtype=np.float32)
length = x1_1.shape#initail guess
p0 = np.array([1,1,1,1,1,1,1,1],dtype=np.float32)
abc=leastsq(residuals, p0,args=(y,x))
print ('Size of first matrix is '+str(x1_1.shape))
print ('Size of first matrix is '+str(x1_2.shape))
print ("Done with program")
I have tried adding np.array in most places with no use.
Could someone please help me ?
Another question here is do I give the output of the residuals() as a single value by adding all errorsnp.sum(err,axis=1). or leave it the way it is ?
When I return np.sum(err,axis=1) in the function residuals(). There is no change in the initial guess. It just remains the same.
I.E error is for each item in the input output mapping. or a combined error overall ?
Example data.
Output
ValueError: object too deep for desired array
Traceback (most recent call last):
File "/media/anilil/Data/charm/mv_clean/.idea/nose_reduction_mpeg.py", line 49, in <module>
abc=leastsq(residuals, p0,args=(y,x))
File "/usr/lib/python2.7/dist-packages/scipy/optimize/minpack.py", line 378, in leastsq
gtol, maxfev, epsfcn, factor, diag)
minpack.error: Result from function call is not a proper array of floats.
leastsq requires a 1D array to be returned from your residuals function.
Currently you calculate the residuals for the whole image and return that as a 2D array.
The simple fix would be to flatten the array of residuals (turning your 2D array into a 1D one).
So instead of returning
return np.array(err)
Do this instead
return err.flatten()
Note that err is already a numpy array so doesn't need to be cast before the return (I guess that slipped in when you were trying to debug it!)
I cannot load an array from a binary file. What am I doing wrong?
pic = imread('headey-640.bmp')
save('test.in.npy', pic)
f = open('test.in.npy','r')
A = load(f)
---------------------------------------------------------------------------
ValueError: total size of new array must be unchanged
You have to open your file in binary mode:
import numpy as np
x = np.array([1,2,3])
np.save("test.npy", x)
with open("test.npy", "rb") as npy:
a = np.load(npy)