We Keep Coding

Problem with plotting multiple functions with a for loop in matplotlib

I am using a for loop to plot a curve for each parameter-value (k) - this works just fine for all the negative k-values, but when the loop reaches the k values = 0 or greater, the lambdify function seems to collapse and I get an error stating the dimensions of x and y are not equal.
This is my code:
import sympy as sym
import numpy as np
import matplotlib.pyplot as plt
eta = np.logspace(-1,2,21) #defines eta values, 21 decades from 0.1 to 100
relrho = np.logspace(-2,2,25) #defines values of rho2/rho1, 25 values from 0.01 to 100
k = (relrho-1)/(relrho+1) #defines the reflection coefficient
#parameter of type curve is k
#rhoa/rho1 is the y-axis
#eta is the x-axis
#R is assigned as the ratio of rho_a to rho_1
#x is assigned to eta
#y is assigned to k
x = sym.symbols('x', real = True)
y = sym.symbols('y')
for y in k: #for-loop assumes k value before while-loop is run, then plots the curve, then new k value is assumed
n=1; R=1;
while n<=500:
Rnew = 2*x**3*y**n/(((2*n)**2+x**2)**(3/2))
R = R + Rnew
n = n + 1
R = sym.lambdify(x,R)
plt.loglog(eta, R(eta))
plt.show()
What is going wrong? I am completely at a dead end right now... clueless...
If I plot the curves individually I can plot them for any k-value just fine, but in the loop it collapses after 12 iterations.
runfile('C:/Users/aslak/OneDrive/Desktop/Typecurves.py', wdir='C:/Users/aslak/OneDrive/Desktop')
Traceback (most recent call last):
File "C:\Users\aslak\OneDrive\Desktop\Typecurves.py", line 44, in <module>
plt.loglog(eta, R(eta))
File "C:\Users\aslak\anaconda3\lib\site-packages\matplotlib\pyplot.py", line 2750, in loglog
return gca().loglog(*args, **kwargs)
File "C:\Users\aslak\anaconda3\lib\site-packages\matplotlib\axes\_axes.py", line 1868, in loglog
return self.plot(
File "C:\Users\aslak\anaconda3\lib\site-packages\matplotlib\axes\_axes.py", line 1743, in plot
lines = [*self._get_lines(*args, data=data, **kwargs)]
File "C:\Users\aslak\anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 273, in __call__
yield from self._plot_args(this, kwargs)
File "C:\Users\aslak\anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 399, in _plot_args
raise ValueError(f"x and y must have same first dimension, but "
ValueError: x and y must have same first dimension, but have shapes (21,) and (1,)
runfile('C:/Users/aslak/OneDrive/Desktop/Typecurves.py', wdir='C:/Users/aslak/OneDrive/Desktop')
Traceback (most recent call last):
File "C:\Users\aslak\OneDrive\Desktop\Typecurves.py", line 34, in <module>
plt.loglog(eta, R(eta))
File "C:\Users\aslak\anaconda3\lib\site-packages\matplotlib\pyplot.py", line 2750, in loglog
return gca().loglog(*args, **kwargs)
File "C:\Users\aslak\anaconda3\lib\site-packages\matplotlib\axes\_axes.py", line 1868, in loglog
return self.plot(
File "C:\Users\aslak\anaconda3\lib\site-packages\matplotlib\axes\_axes.py", line 1743, in plot
lines = [*self._get_lines(*args, data=data, **kwargs)]
File "C:\Users\aslak\anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 273, in __call__
yield from self._plot_args(this, kwargs)
File "C:\Users\aslak\anaconda3\lib\site-packages\matplotlib\axes\_base.py", line 399, in _plot_args
raise ValueError(f"x and y must have same first dimension, but "
ValueError: x and y must have same first dimension, but have shapes (21,) and (1,)

The problem is that when y=0, then 2*x**3*y**n/(((2*n)**2+x**2)**(3/2)) will be 0, hence R will be zero. When you lambdify it and pass in a numpy array, it will return the scalar value 0. We need to take into account this fact. Note that in the following code block I also optimize for speed: only one symbolic addition will be executed for every iteration of the for loop.
import sympy as sym
import numpy as np
import matplotlib.pyplot as plt
eta = np.logspace(-1,2,21) #defines eta values, 21 decades from 0.1 to 100
relrho = np.logspace(-2,2,25) #defines values of rho2/rho1, 25 values from 0.01 to 100
k = (relrho-1)/(relrho+1) #defines the reflection coefficient
#parameter of type curve is k
#rhoa/rho1 is the y-axis
#eta is the x-axis
#R is assigned as the ratio of rho_a to rho_1
#x is assigned to eta
#y is assigned to k
x = sym.symbols('x', real = True)
y = sym.symbols('y')
for y in k: #for-loop assumes k value before while-loop is run, then plots the curve, then new k value is assumed
print("y", y)
n=1; R=1;
addends = []
while n<=500:
addends.append(2*x**3*y**n/(((2*n)**2+x**2)**(3/2)))
n = n + 1
R = Add(R, *addends)
R = sym.lambdify(x,R)
R_val = R(eta)
if not hasattr(R_val, "__iter__"):
R_val = R_val * np.ones_like(eta)
plt.loglog(eta, R_val)
plt.show()

Davide_sd pointed out why the code fails, but in this case it is faster to avoid sympy altogether:
import numpy as np
import matplotlib.pyplot as plt
eta = np.logspace(-1, 2, 21)
relrho = np.logspace(-2, 2, 25)
k = ((relrho - 1) / (relrho + 1))
n = np.r_[1:501]
y = k[None, :, None]
x = eta[:, None, None]
R = (2 * x**3 * y**n / (((2 * n)**2 + x**2)**(3 / 2))).sum(axis=2) + 1
plt.loglog(eta, R)
plt.show()

i keep getting this error in python (spyder) and i have no idea how to solve it , index 1 is out of bounds for axis 0 with size 1

im trying to estimate the gjr garch model using this code below and the dataset im using is the bitcoin returns (daily) total of 1600 observations. im getting this error ''index 1 is out of bounds for axis 0 with size 1''. and above it it says :
File "C:\Users\georgios\Downloads\untitled1.py", line 97, in
estimates = fmin_slsqp(gjr_garch_likelihood, startingVals,
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize\slsqp.py", line 207, in fmin_slsqp
res = _minimize_slsqp(func, x0, args, jac=fprime, bounds=bounds,
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize\slsqp.py", line 375, in _minimize_slsqp
sf = _prepare_scalar_function(func, x, jac=jac, args=args, epsilon=eps,
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize\optimize.py", line 261, in _prepare_scalar_function
sf = ScalarFunction(fun, x0, args, grad, hess,
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize_differentiable_functions.py", line 136, in init
self._update_fun()
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize_differentiable_functions.py", line 226, in _update_fun
self._update_fun_impl()
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize_differentiable_functions.py", line 133, in update_fun
self.f = fun_wrapped(self.x)
File "C:\Users\georgios\Nieuwe map\lib\site-packages\scipy\optimize_differentiable_functions.py", line 130, in fun_wrapped
return fun(x, *args)
File "C:\Users\georgios\Downloads\untitled1.py", line 21, in gjr_garch_likelihood
sigma2[t] = (omega + alpha * eps[t-1]**2
File "C:\Users\georgios\Nieuwe map\lib\site-packages\pandas\core\series.py", line 977, in setitem
values[key] = value
IndexError: index 1 is out of bounds for axis 0 with size 1
My dataset is fine from
What I see , I only have one excel file with only one column with the returns
And the garch model is Univariate
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from numpy import size, log, pi, sum, array, zeros, diag, mat, asarray, sqrt, \
copy
from numpy.linalg import inv
from scipy.optimize import fmin_slsqp
def gjr_garch_likelihood(parameters, data, sigma2, out=None):
''' Returns negative log-likelihood for GJR-GARCH(1,1,1) model.'''
mu = parameters[0]
omega = parameters[1]
alpha = parameters[2]
gamma = parameters[3]
beta = parameters[4]
T = size(data,0)
eps = data - mu
# Data and sigma2 are T by 1 vectors
for t in range(1,T):
sigma2[t] = (omega + alpha * eps[t-1]**2
+ gamma * eps[t-1]**2 * (eps[t-1]<0) + beta * sigma2[t-1])
logliks = 0.5*(log(2*pi) + log(sigma2) + eps**2/sigma2)
loglik = sum(logliks)
if out is None:
return loglik
else:
return loglik, logliks, copy(sigma2)
def gjr_constraint(parameters, data, sigma2, out=None):
''' Constraint that alpha+gamma/2+beta<=1'''
alpha = parameters[2]
gamma = parameters[3]
beta = parameters[4]
return array([1-alpha-gamma/2-beta])
def hessian_2sided(fun, theta, args):
f = fun(theta, *args)
h = 1e-5*np.abs(theta)
thetah = theta + h
h = thetah - theta
K = size(theta,0)
h = np.diag(h)
fp = zeros(K)
fm = zeros(K)
for i in range(K):
fp[i] = fun(theta+h[i], *args)
fm[i] = fun(theta-h[i], *args)
fpp = zeros((K,K))
fmm = zeros((K,K))
for i in range(K):
for j in range(i,K):
fpp[i,j] = fun(theta + h[i] + h[j], *args)
fpp[j,i] = fpp[i,j]
fmm[i,j] = fun(theta - h[i] - h[j], *args)
fmm[j,i] = fmm[i,j]
hh = (diag(h))
hh = hh.reshape((K,1))
hh = hh # hh.T
H = zeros((K,K))
for i in range(K):
for j in range(i,K):
H[i,j] = (fpp[i,j] - fp[i] - fp[j] + f
+ f - fm[i] - fm[j] + fmm[i,j])/hh[i,j]/2
H[j,i] = H[i,j]
return H
# Import data
FTSEreturn = pd.read_csv('1.csv')
# Starting values
startingVals = array([FTSEreturn.mean(),
FTSEreturn.var() * .01,
.03, .09, .90])
# Estimate parameters
finfo = np.finfo(np.float64)
bounds = [(-10*FTSEreturn.mean(), 10*FTSEreturn.mean()),
(finfo.eps, 2*FTSEreturn.var() ),
(0.0,1.0), (0.0,1.0), (0.0,1.0)]
T = FTSEreturn.shape[0]
sigma2 = T * FTSEreturn.var()
# Pass a NumPy array, not a pandas Series
args = (np.asarray(FTSEreturn), sigma2)
estimates = fmin_slsqp(gjr_garch_likelihood, startingVals,
f_ieqcons=gjr_constraint, bounds = bounds,
args = args)
loglik, logliks, sigma2final = gjr_garch_likelihood(estimates, FTSEreturn,
sigma2, out=True)
step = 1e-5 * estimates
scores = zeros((T,5))
for i in range(5):
h = step[i]
delta = np.zeros(5)
delta[i] = h
loglik, logliksplus, sigma2 = gjr_garch_likelihood(estimates + delta, \
np.asarray(FTSEreturn), sigma2, out=True)
loglik, logliksminus, sigma2 = gjr_garch_likelihood(estimates - delta, \
np.asarray(FTSEreturn), sigma2, out=True)
scores[:,i] = (logliksplus - logliksminus)/(2*h)
I = (scores.T # scores)/T
J = hessian_2sided(gjr_garch_likelihood, estimates, args)
J = J/T
Jinv = mat(inv(J))
vcv = Jinv*mat(I)*Jinv/T
vcv = asarray(vcv)
output = np.vstack((estimates,sqrt(diag(vcv)),estimates/sqrt(diag(vcv)))).T
print('Parameter Estimate Std. Err. T-stat')
param = ['mu','omega','alpha','gamma','beta']
for i in range(len(param)):
print('{0:<11} {1:>0.6f} {2:0.6f} {3: 0.5f}'.format(param[i],
output[i,0], output[i,1], output[i,2]))
any help would be appreciated because im stuck on this one month now and i cant solve it

The error says there's an indexing error during a setitem in
sigma2[t] = (omega + alpha * eps[t-1]**2
In other words, t is too large for the array sigma2.
That's in the gjr_garch_likelihood function. t is an iterator that starts at 1. sigma2 is an argument. So we need to look at how the function is called, and the corresponding argument is.
That's more complicated, since it's a function used by fmin_slsqp. So the next step is to review that function's docs, to understand how it calls the func, and especially what arguments it provides to your gjr.... I won't do that for you!
But it's a good idea when encountering errors like this to add some diagnostic prints to your gjr... function. You need to clearly understand what gets passed to it, paying particular attention to array shapes. Don't make assumptions. Verify.
You may need to change gjr ... to accomodate the arguments, or you may need to modify how you call fmin....
Question - how much of this did you already figure out in the past month of struggle? If you already figured this out, you should have included that information in the question.

scipy.optimize.curve_fit - TypeError('Improper input: N=%s must not exceed M=%s' % (n, m))

I am attempting to optimize parameters of a function I've written that rotates points and then transforms them from pixel location to mm from a particular point. I am encountering TypeError('Improper input: N=%s must not exceed M=%s' % (n, m)) which other questions on SO seem to indicate that I have more parameters than data points. The data I am feeding it has 26 sets of x,y coords. Can anybody see where I'm going wrong with the code below?
Thanks in advance
from scipy import optimize
import numpy as np
import math
def px_to_mm_v4(coords, cf_x, cf_y, nudge_x, nudge_y, center_x, center_y, rotate_degrees):
## set lower left loc
ll_x = center_x - (127.76/2/cf_x) ## lower left x location in pixels
ll_y = center_y + (85.47/2/cf_y) ## lower left y location in pixels
## unpack coordinates
x,y = coords
## rotate points around center
rotate_radians = math.radians(rotate_degrees)
x_rotated = center_x + math.cos(rotate_radians) * (x - center_x) - math.sin(rotate_radians) * (y - center_y)
y_rotated = center_y + math.cos(rotate_radians) * (x - center_x) + math.cos(rotate_radians) * (y - center_y)
## convert px to mm
x_converted = (x_rotated - ll_x) * cf_x + nudge_x
y_converted = (ll_y - y_rotated) * cf_y + nudge_y
ret_x = x_converted
ret_y = y_converted
ret = (ret_x,ret_y)
return ret
x_px = np.array([1723,1530,1334,1135,943,747,548,2520,2322,2120,1921,1726,1530,1331,1132,937,741,545,346,349,352,355,358,358,361,361,148])
y_px = np.array([596,791,986,1176,1373,1569,1769,1973,1967,1967,1964,1962,1964,1967,1967,1967,1962,1964,1967,1769,1569,1373,1178,986,791,602,2162])
x_mm = np.array([80,70,60,50,40,30,20,120,110,100,90,80,70,60,50,40,30,20,10,10,10,10,10,10,10,10,0])
y_mm = np.array([80,70,60,50,40,30,20,10,10,10,10,10,10,10,10,10,10,10,10,20,30,40,50,60,70,80,0])
test_coords_tup = (x_px,y_px)
points_to_fit_tup = (x_mm,y_mm)
cf_x_test = 0.05072
cf_y_test = 0.05076
nudge_x_test = -2.2
nudge_y_test = 2.1
center_x_test = 1374
center_y_test = 1290
rotate_degrees_test = 1.4
params0 = [cf_x_test,cf_y_test,nudge_x_test,nudge_y_test,center_x_test,center_y_test,rotate_degrees_test]
popt, pcov = optimize.curve_fit(px_to_mm_v4, test_coords_tup, points_to_fit_tup, p0=params0)
Edit: running the code gives back the error below
Traceback (most recent call last):
File "SO_example.py", line 48, in <module>
popt, pcov = optimize.curve_fit(px_to_mm_v4, test_coords_tup, points_to_fit_tup, p0=params0)
File "//anaconda3/lib/python3.7/site-packages/scipy/optimize/minpack.py", line 784, in curve_fit
res = leastsq(func, p0, Dfun=jac, full_output=1, **kwargs)
File "//anaconda3/lib/python3.7/site-packages/scipy/optimize/minpack.py", line 414, in leastsq
raise TypeError('Improper input: N=%s must not exceed M=%s' % (n, m))
TypeError: Improper input: N=7 must not exceed M=2

From leastsq docs
func callable
Should take at least one (possibly length N vector) argument and
returns M floating point numbers. It must not return NaNs or fitting
might fail. M must be greater than or equal to N.

scipy.minimize constrained optimization problem with multiple variables

I am currently trying to implement the following optimization problem in python (in order to resolve it with scipy.optimize.minimize).
Please note that alpha is given,T is the number of generated random values (i.e. via Monte Carlo simulation, also given), z is an array of artificial variables (ignore the last constraint). The function f(x,y) is equal to -y.T * x (y is an array of nT random values). Variable val is a pandas data frame with all the observed data. Variable r is the realization of the random events (randomly generated using MonteCarlo technique - fitting a distribution, pandas nT).
Unfortunately I am facing different problems while truing to solve it. Can anyone be so kind to help me to code it correctly?
EDIT: following the modified code with correct init and constraints. I am not able to figured out how to write correctly the bounds (for x[0] bounds should be (0, None), for x1 and x[2] bounds should be (None, None). Can anyone be so kind to suggest me the correct way?
def objective(x, alpha, t):
#
return x[1] + (1 / (1 - alpha) * t) * np.sum(x[2])
def problem(val, t = 10, alpha = 0.9):
#
y = []
for simbolo in val.columns:
loc, scale = sts.gumbel_l.fit(val[simbolo])
y.append(sts.gumbel_l.rvs(loc, scale, t))
init = np.array(([1 / len(val.columns)] * len(val.columns), [1] * t, [0] * t))
constraints = [
{"type": "ineq", "fun": lambda x: x[2]},
{"type": "ineq", "fun": lambda x: np.dot(x[0].T, np.asarray(y)) + x[1] + x[2]}
]
bounds = ((0, None),) * len(val.columns)
args = (alpha, t)
res = opt.minimize(
objective,
x0 = init,
args = args,
bounds = bounds,
constraints = constraints
)
return res['x']
I tried to write the bounds as follow:
bounds = (((0, None),) * len(var.columns), ((None, None),) * len(var.columns), ((None, None),) * len(var.columns))
by I get the following error:
File "main.py", line 250, in <module>
problem(r)
File "port.py", line 152, in vanilla_cvar
res = opt.minimize(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/scipy/optimize/_minimize.py", line 625, in minimize
return _minimize_slsqp(fun, x0, args, jac, bounds,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/scipy/optimize/slsqp.py", line 315, in _minimize_slsqp
new_bounds = old_bound_to_new(bounds)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/scipy/optimize/_constraints.py", line 316, in old_bound_to_new
lb, ub = zip(*bounds)
ValueError: too many values to unpack (expected 2)

From y(x) animation to y(t) at a given x

I am currently running code simulating a string's motion, and I wish to read its amplitude at a given point and plot this as a function of time, thus representing data that would be read from a guitar's pickup. As I am new to animation in python, the task of obtaining data while presenting this animation has proved a challenge. Here is the code:
import numpy as np
import matplotlib
matplotlib.use("TkAgg")
from matplotlib import pyplot as plt
from math import exp
#import matplotlib.animation as animation
#Power Spectrum
def PowerSpectrum(f):
return (f*f.conjugate()).real/len(f)**2
#triangular pulse
def triangular_pulse(x, xmean, sigma):
return np.where(x<xmean,x*sigma/xmean,sigma-(x-xmean)*(sigma/(200-xmean)))
N_masses = 200
T = 0.0669264714
mu = .03937
cSq = T/mu
c = np.sqrt(cSq)
dx = 1.0
dt = dx/c
print dt
#Initialize some arrays
x0 = np.arange(N_masses)*dx
y = np.zeros(N_masses)
vy = np.zeros(N_masses)
ay = np.zeros(N_masses)
#setup for animation
fig1 = plt.figure()
plt.ion()
wave, = plt.plot(x0,y)
plt.ylim(-30,30)
#Set Initial conditions (pluck)
# # half-pluck
# y = 30*gaussian_pulse(x0,x0[N_masses/2],2)
# quarter-pluck
y = triangular_pulse(x0,x0[N_masses/10],6)
yprev = y - vy*dt
y1 = []
t=0.0
i=0
while t<1000:
dydx = (y[1:] - y[:-1])/dx
ay[1:-1] = ( dydx[1:] - dydx[:-1] )/dx * cSq
# Notice we update both at the same time
yprev, y = y, exp(-.00001*t)*(2*y - yprev + ay * dt**2)
t = t + dt
y1.append(y[1])
i += 1
if i%1==0:
wave.set_data(x0,y)
plt.draw()
gauss_hat = np.fft.fft(y1)
freqs = np.fft.fftfreq(len(y1), d=1.0/100)
half_ps = PowerSpectrum(gauss_hat)
#half-pluck y[1] vs. t
plt.plot(range(1000), y1)
plt.xlabel('time')
plt.ylabel('y[1]')
plt.title('y[1] vs. t')
# power spectrum plots
#plt.plot(freqs,half_ps)
#plt.xlabel('frequency (Hz)')
#plt.ylabel('Intensity')
#plt.title('y[1] Power Spectrum (half pluck)')
#plt.title('y[1] Power Spectrum (quarter pluck)')
#plt.grid(True)
#plt.ioff()
plt.show()
The animation runs, but no plot is presented. I receive the error:
Traceback (most recent call last):
File "/Users/defaultuser/Downloads/compare.py", line 69, in <module>
plt.plot(range(1000), y1)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/pyplot.py", line 2467, in plot
ret = ax.plot(*args, **kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/axes.py", line 3893, in plot
for line in self._get_lines(*args, **kwargs):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/axes.py", line 322, in _grab_next_args
for seg in self._plot_args(remaining, kwargs):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/axes.py", line 300, in _plot_args
x, y = self._xy_from_xy(x, y)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/matplotlib/axes.py", line 240, in _xy_from_xy
raise ValueError("x and y must have same first dimension")
ValueError: x and y must have same first dimension
[Finished in 30.6s with exit code 1]
Update
That error no longer appears, as the code has been modified to plt.plot(y1) thus granting x and y the same dimension. After commenting out plotting code responsible for the animation, a good graph was obtained. Thus I learned the plt.plot function can't be called more than once and that subplot is required instead.

The first and last lines in the traceback:
File "/Users/defaultuser/Downloads/compare.py", line 69, in <module>
plt.plot(range(1000), y1)
...
raise ValueError("x and y must have same first dimension")
suggest that range(1000) and y1 do not have the same shape.
Indeed, if you paste the code in an interactive session, you'll find
In [17]: len(y1)
Out[17]: 1304
which makes sense since the loop increments t by an amount dt and dt < 1:
t = 0.0
while t<1000:
t = t + dt
y1.append(...)
In [18]: dt
Out[18]: 0.76697947735477701
You can fix the problem by simply using the 1-argument form of plt.plot:
plt.plot(y1)
which is equivalent to
plt.plot(range(len(y1)), y1)