We Keep Coding

drawing a vertical line in a sympy plot of a piecewise function

I have the following piecewise function defined in a jupyter notebook:
from sympy import *
Mt, F1, Fax, Fay, Faz, Fbz, Fby, l, x, alpha, a, b, c, d, e = symbols('Mt, F1, Fax, Fay, Faz, Fbz, Fby, l, x, alpha, a, b, c, d, e')
My1 = 0
My2 = sin(alpha)*F1*(x-a)
My3 = sin(alpha)*F1*(x-a) - Fay*(x-(a+b))
My4 = sin(alpha)*F1*(x-a) - Fay*(x-(a+b))- Fby*(x-(a+b+c))
My5 = 0
My = Piecewise((My1, x<a), (My2, And(a<x, x<b)), (My3, And(b<x, x<c)), (My4, And(c<x, x<d)), (My5, And(d<x, x<e)))
My
and try to draw it:
params = {alpha: 30, a:0.024, b:0.074, c:0.132, d:0.189, e:0.219, F1:11700, Fay:-8762.57, Fby:26973.41}
plot(My.subs(params), (x, 0, e.subs(params)), title= "Biegemoment $M_y(x)$")
with the following result:
This is basically fine, I would just like to have a vertical line at x=0.074. I tried with adaptiveand nb_of_points mentioned here but it did give an error
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/usr/lib/python3/dist-packages/sympy/plotting/experimental_lambdify.py in __call__(self, *args)
118 temp_args = (np.array(a, dtype=np.complex) for a in args)
--> 119 results = self.vector_func(*temp_args)
120 results = np.ma.masked_where(
...
/usr/lib/python3/dist-packages/numpy/lib/function_base.py in _vectorize_call(self, func, args)
2165 for a in args]
2166
-> 2167 outputs = ufunc(*inputs)
2168
2169 if ufunc.nout == 1:
FloatingPointError: invalid value encountered in ? (vectorized)

The current plotting module is unable to achieve what you would like to do with a single step. There are at least two approaches.
First:
Using the current plotting module, create a plot for each piece and combine them together
from sympy.plotting import plot
plots = []
for expr, cond in My.subs(params).args:
cond = cond.as_set().subs({-oo: 0, oo: e.subs(params)})
plots.append(splot(expr, (x, *cond.args[:-2]), show=False))
# combine all plots together
p = plots[0]
for i in range(1, len(plots)):
p.extend(plots[i])
# optionally set the same color for each piece
for series in p:
series.line_color = "b"
p.show()
Second:
Pretty much the same thing, but using lambdify and matplotlib to achieve full customization.
Third:
Install the Sympy Plotting Backends library. Here, you can either use the plot function activating the detect_poles algorithm (it detects vertical asymptotes, but it is also good for this case):
plot(My.subs(params), (x, 0, e.subs(params)), title= "Biegemoment $M_y(x)$", detect_poles=True, eps=1e-06)
Alternatively, you can use plot_piecewise: this is also going to highlight the extremities of the pieces. Filled circle means an extremity is included in the interval (<=, >=), empty circles it is excluded (<, >).
plot_piecewise(My.subs(params), (x, 0, e.subs(params)), title= "Biegemoment $M_y(x)$")

Interpolate_to_grid returns all nans

Practicing with MetPy Monday interpolate_to_grid for metar data and I successfully got the mslp grid to work.
Moving on to Potential temperature and the result has been all nan. When it "works". When it doesnt work, I get a set of errors that dont appear to help...
import numpy as np
import cartopy.crs as ccrs
import cartopy.feature as cfeature
import matplotlib.pyplot as plt
from siphon.catalog import TDSCatalog
from metpy.io import parse_metar_file
from metpy.interpolate import interpolate_to_grid, remove_nan_observations
from metpy.plots import add_metpy_logo, current_weather, sky_cover, StationPlot
from metpy.calc import wind_components, wet_bulb_temperature, altimeter_to_station_pressure,potential_temperature,gradient
from metpy.units import units
from datetime import datetime,timedelta
import pandas as pd
mapcrs = ccrs.LambertConformal(central_longitude=-100.,central_latitude=35.,standard_parallels=(30.,60.))
datacrs = ccrs.PlateCarree()
cat = TDSCatalog('https://thredds-test.unidata.ucar.edu/thredds/catalog/noaaport/text/metar/catalog.xml')
ds = cat.datasets[-4]
dattim = ds.name[6:14]+' '+ds.name[15:19]
ds.download()
df = parse_metar_file(ds.name)
#pandas dataframe
#df.head()
df.columns.values
extent = [-120,-72,24,50]
df = df.dropna(subset=['latitude','longitude','elevation','altimeter','air_temperature','eastward_wind','northward_wind','air_pressure_at_sea_level','dew_point_temperature'])
lon = df['longitude'].values
lat = df['latitude'].values
stn_ids = df['station_id'].values
elev = df['elevation'].values
altimeter = df['altimeter'].values
t2 = df['air_temperature'].values
mslp = df['air_pressure_at_sea_level'].values
#projected coords
xp, yp, _ = mapcrs.transform_points(datacrs,lon,lat).T # x,y returned
#mslp WORKS
x_masked, y_masked, mslp = remove_nan_observations(xp,yp,mslp)
#altgridx,altgridy,alt = interpolate_to_grid(x_masked,y_masked,alt, interp_type='cressman')
altgridx,altgridy,mslp = interpolate_to_grid(x_masked,y_masked,mslp, interp_type='barnes',gamma=.5,kappa_star=10, hres=25000)
#Potential Temperature doesnt work
pres = altimeter_to_station_pressure(altimeter * units('mbar'), elev * units('m'))*33.8639
print(pres)
# theta
x_masked, y_masked, temp = remove_nan_observations(xp,yp,t2*units('degC'))
x_masked, y_masked, pres = remove_nan_observations(xp,yp,pres)
print(np.size(temp))
potemp = potential_temperature(pres, temp)
print(np.size(potemp))
print(np.unique(np.array(potemp)))
grdx = 75000.
thgridx,thgridy,theta = interpolate_to_grid(x_masked,y_masked, potemp, interp_type='barnes',kappa_star=6, gamma=0.5,hres=grdx)
print(np.shape(thgridx))
print(np.unique(theta))
Here is what is returned from the last section:
[949.361081708803 993.4468013877739 987.2845093729651 ... 1029.0930108008558 1016.002484792407 930.3708063382303] millibar
5837
5837
[236.32885315 237.21299941 239.04372591 ... 368.37047837 369.20079652
370.76269267]
---------------------------------------------------------------------------
DimensionalityError Traceback (most recent call last)
~/miniconda3/lib/python3.7/site-packages/pint/quantity.py in __float__(self)
896 return float(self._convert_magnitude_not_inplace(UnitsContainer()))
--> 897 raise DimensionalityError(self._units, "dimensionless")
898
DimensionalityError: Cannot convert from 'kelvin' to 'dimensionless'
The above exception was the direct cause of the following exception:
ValueError Traceback (most recent call last)
/var/folders/5n/sg5k98bx6gg4flb4fskykh4m0000gn/T/ipykernel_41626/379842406.py in <module>
11
12 grdx = 75000.
---> 13 thgridx,thgridy,theta = interpolate_to_grid(x_masked,y_masked, potemp, interp_type='barnes',kappa_star=6, gamma=0.5,hres=grdx)
14 print(np.shape(thgridx))
15 print(np.unique(theta))
~/miniconda3/lib/python3.7/site-packages/metpy/pandas.py in wrapper(*args, **kwargs)
19 kwargs = {name: (v.values if isinstance(v, pd.Series) else v)
20 for name, v in kwargs.items()}
---> 21 return func(*args, **kwargs)
22 return wrapper
~/miniconda3/lib/python3.7/site-packages/metpy/interpolate/grid.py in interpolate_to_grid(x, y, z, interp_type, hres, minimum_neighbors, gamma, kappa_star, search_radius, rbf_func, rbf_smooth, boundary_coords)
301 minimum_neighbors=minimum_neighbors, gamma=gamma,
302 kappa_star=kappa_star, search_radius=search_radius,
--> 303 rbf_func=rbf_func, rbf_smooth=rbf_smooth)
304
305 return grid_x, grid_y, img.reshape(grid_x.shape)
~/miniconda3/lib/python3.7/site-packages/metpy/interpolate/points.py in interpolate_to_points(points, values, xi, interp_type, minimum_neighbors, gamma, kappa_star, search_radius, rbf_func, rbf_smooth)
365 return inverse_distance_to_points(points, values, xi, search_radius, gamma, kappa,
366 min_neighbors=minimum_neighbors,
--> 367 kind=interp_type)
368
369 # If this is radial basis function, make the interpolator and apply it
~/miniconda3/lib/python3.7/site-packages/metpy/interpolate/points.py in inverse_distance_to_points(points, values, xi, r, gamma, kappa, min_neighbors, kind)
268 img[idx] = cressman_point(dists, values_subset, r)
269 elif kind == 'barnes':
--> 270 img[idx] = barnes_point(dists, values_subset, kappa, gamma)
271
272 else:
ValueError: setting an array element with a sequence.
I struggled with Units, but I think the units are correct now. What could be causing this?
I tried cressman, I tried a larger Barnes grid, and I tried making sure search_radius was large. Still nan, when it worked.

The problem is caused by interpolate_to_grid choking on units when using Cressman or Barnes--which we definitely need to fix. For now the solution is to either use a different interpolation method (like interp_type='linear', the default), or to strip units before calling:
thgridx, thgridy, theta = interpolate_to_grid(x_masked, y_masked, potemp.magnitude,
interp_type='barnes', kappa_star=6, gamma=0.5, hres=grdx)
theta = units.Quantity(theta, 'K')
As far as your problems with NaNs is concerned, you may want to look at the search_radius parameter, which controls the maximum distance from a target point observations are considered. In some data-sparse areas, this could cause you to have some drop-outs. By default, it uses a guess of 5 times the average distance from one ob point to its nearest neighbor.

TypeError: Improper input: N=3 must not exceed M=1, not sure what's wrong with my dimensions?

I'm trying to fit a lorentzian to one of the peaks in my dataset.
We were given the fit for a gaussian, and aside from the actual fit equation, the code is very similar, so I'm not sure where I am going wrong. I don't see why there is an issue with the dimensions when I'm using curve_fit.
Here are the relevant pieces of my code for a better idea of what I'm talking about.
Reading the CSV file in and trimming it
import csv
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
from matplotlib.ticker import StrMethodFormatter
#reading in the csv file
with open("Data-Oscilloscope.csv") as csv_file:
csv_reader = csv.reader(csv_file, delimiter=",")
time =[]
voltage_raw = []
for row in csv_reader:
time.append(float(row[3]))
voltage_raw.append(float(row[4]))
print("voltage:", row[4])
#trimming the data
trim_lower_index = 980
trim_upper_index = 1170
time_trim = time[trim_lower_index:trim_upper_index]
voltage_trim = voltage_raw[trim_lower_index:trim_upper_index]
The Gaussian fit given
#fitting the gaussian function
def gauss_function(x, a, x0, sigma):
return a*np.exp(-(x-x0)**2/(2*sigma**2))
popt, pcov = curve_fit(gauss_function, time_trim, voltage_trim, p0=[1,.4,0.1])
perr = np.sqrt(np.diag(pcov))
#plot of the gaussian fit
plt.figure(2)
plt.plot(time_trim, gauss_function(time_trim, *popt), label = "fit")
plt.plot(time_trim, voltage_trim, "-b")
plt.show()
My attempted Lorentzian fit
#x is just the x values, a is the amplitude, x0 is the central value, and f is the full width at half max
def lorentz_function(x, a, x0,f):
w = f/2 #half width at half max
return a*w/ [(x-x0)**2+w**2]
popt, pcov = curve_fit(lorentz_function, time_trim, voltage_trim, p0=[1,.4,0.1])
I get an error running this that states:
in leastsq raise TypeError('Improper input: N=%s must not exceed M=%s' % (n, m))
TypeError: Improper input: N=3 must not exceed M=1
I'm probably missing something very obvious but just can't see it.
Thanks in advance!
EDIT: I have taken a look at other, similar questions and went through their explanations, but can't see how those fit in with my code because the number of parameters and dimensions of my inputs should be fine, given that they worked for the gaussian fit.

You didn't show the full traceback/error so I can only guess where it's happening. It's probably looking at the result returned by lorentz_function, and finding the dimensions to be wrong. So while the error is produced by your function, the testing is in its caller (in this case a level or two down).
def optimize.curve_fit(
f,
xdata,
ydata,
p0=None,... # p0=[1,.4,0.1]
...
res = leastsq(func, p0,
...
curve_fit passes the task to leastsq, which starts as:
def optimize.leastsq(
func,
x0,
args=(), ...
x0 = asarray(x0).flatten()
n = len(x0)
if not isinstance(args, tuple):
args = (args,)
shape, dtype = _check_func('leastsq', 'func', func, x0, args, n)
m = shape[0]
if n > m:
raise TypeError('Improper input: N=%s must not exceed M=%s' % (n, m))
I'm guessing _check_func does
res = func(x0, *args) # call your func with initial values
and returning the shape of res. The error says there's a mismatch between what it expects based on the shape of x0 and the result of your func.
I'm guessing that with a 3 element p0, it's complaining that your function returned a 1 element result (due to the []).
lorentz is your function. You don't test the output shape so it can't raise this error.

I had a similar problem yielding this error message.
In that case, the array of data passed to the optimize.leastsq was in the matrix form. the data seems to have to be a 1 row array.
For example, the sentence calling the leastsq was
result = optimize.leastsq(fit_func, param0, args=(xdata, ydata, zdata))
xdata, ydata, zdata was in the [ 1 x num ] matrix form. It was
[[200. .... 350.]]
It should be
[200. .... 350.]
So, I had to add the sentence for conversion
xdata = xdata[0, :]
So do ydata and zdata.
I hope this be a help for you.

Result from function not a proper array of floats using optimize.curve_fit

I have recorded experimental temperatures at five locations from the surface of a solid. At every time step, I want to fit these readings to a theoretical curve defined by my function: Temp_Function_JLT(X,h).
X is a multi-dimensional array that includes the x_coordinates as well as time, initial temperature and material properties (all independent variables). "h" is the heat transfer coefficient, which for the purpose of this exercise I'm trying to optimize (leaving the physics aside for a moment.)
This is the definition of my temperature function:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import pickle
import scipy.optimize as opt
from scipy.special import erfc
def Temp_Function_JLT(X,ht):
# Work around the fact that only one independent variable can be passed to optimize.curve_fit
x,t,T0,q,alpha,rho,c,k = X
term_a = q/ht
term_b = erfc(x/np.sqrt(4*alpha*t))
term_c = np.exp(((ht*x)/(np.sqrt(alpha)*np.sqrt(k*rho*c)))+((ht**2)/(k*rho*c)))
term_d = erfc((ht*np.sqrt(t))/(np.sqrt(k*rho*c)) + (x/np.sqrt(4*alpha*t)))
Temperature = (term_a * (term_b - term_c * term_d)) + T0 - 273
return Temperature
The function works. I can run it with some initial parameters and obtain sensible values. More importantly for this question, if I call it with the following data:
t = 1
x_test = np.linspace(0.004,0.02,5) # TC locations
time_test = range(1,180,30)
T0_test = 25 + 273
q_test = 20000
h_test = 10
I will obtain a numpy array as a solution of shape (1,) which gives an answer to np.ndim of 1 (This has been mentioned in the following previous questions:
Least Linear Squares: scipy.optimize.curve_fit() throws "Result from function call is not a proper array of floats."
Fitting a vector function with curve_fit in Scipy
Fitting a 2D Gaussian function using scipy.optimize.curve_fit - ValueError and minpack.error
The problem arises when I call opt.curve_fit(). indepth_temperatures is a list that contains each test as an array. I iterate over it (to iterate over each test) and then I perform the fit on each row (each time step), according to the following code:
for i,test in enumerate(indepth_temperatures):
# Iterate over every row
for j,row in enumerate(test):
# Define tuple that contains all independent variables
X = (TC_depth,
times[i][j],
T0_temperatures[i] + 273,
20000,
pmma_alpha,
pmma_rho,
pmma_c,
pmma_k)
print(Temp_Function_JLT(X,h0))
print(row)
print('---')
# Call function to optimize curve fit on h
popt, pcov = opt.curve_fit(Temp_Function_JLT,X,row,h0)
print(popt)
For the first iteration, I obtain the following result:
[23.2034 23.2034 23.2034 23.2034 23.2034] # comes from print(Temp_Function_JLT(X,h0))
[23.937 22.619 22.59 24.884 21.987000000000002] # comes from print(row)
Followed by this error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
TypeError: Cannot cast array data from dtype('O') to dtype('float64') according to the rule 'safe'
---------------------------------------------------------------------------
error Traceback (most recent call last)
<ipython-input-67-9c4545fd257b> in <module>()
22 print('---')
23 # Call function to optimize curve fit on h
---> 24 popt, pcov = opt.curve_fit(Temp_Function_JLT,X,row,h0)
25 print(popt)
~\AppData\Local\Continuum\anaconda2\envs\py36\lib\site-packages\scipy\optimize\minpack.py in curve_fit(f, xdata, ydata, p0, sigma, absolute_sigma, check_finite, bounds, method, jac, **kwargs)
749 # Remove full_output from kwargs, otherwise we're passing it in twice.
750 return_full = kwargs.pop('full_output', False)
--> 751 res = leastsq(func, p0, Dfun=jac, full_output=1, **kwargs)
752 popt, pcov, infodict, errmsg, ier = res
753 cost = np.sum(infodict['fvec'] ** 2)
~\AppData\Local\Continuum\anaconda2\envs\py36\lib\site-packages\scipy\optimize\minpack.py in leastsq(func, x0, args, Dfun, full_output, col_deriv, ftol, xtol, gtol, maxfev, epsfcn, factor, diag)
392 with _MINPACK_LOCK:
393 retval = _minpack._lmdif(func, x0, args, full_output, ftol, xtol,
--> 394 gtol, maxfev, epsfcn, factor, diag)
395 else:
396 if col_deriv:
error: Result from function call is not a proper array of floats.
I have tried returning from my function np.ravel(Temperature) or Temperature.flatten() with no luck. The error remains, and I can't figure out why it's there. As I mentioned, I have checked the dimensions of the return of my function and it is a 1D array.
Any help will be greatly appreciated!
UPDATE: I realized it was hard to replicate this code, so this is a simplified version:
Temp_Function_JLT(X,h0): stays the same.
pmma_rho = 1200 # kg/m3
pmma_c = 1500 # J/kgK
pmma_k = 0.16 # W/mK
pmma_alpha = pmma_k/(pmma_rho*pmma_c)
x_test = np.linspace(0.004,0.02,5) # TC locations
t = 1
T0_test = 25 + 273
q_test = 20000
h_test = 10
X = (x_test,t,T0_test,q_test,pmma_alpha,pmma_rho,pmma_c,pmma_k)
y_data = [23.937 22.619 22.59 24.884 21.987000000000002]
opt.curve_fit(Temp_Function_JLT, X, y_data, h_test)

I realized what was wrong with my code. Even though my y_data (row) was defined as a 1-D numpy array, its data type was object. I don't yet understand why this was the cause, but by forcing the data type with np.astype(np.float), opt.curve_fit worked.

python syntax for functions with multiple return values: using fsolve

I am new to python and trying to convert some matlab code as an exercise. One of the tasks involves finding the root, or minimum absolute value if no root exists, of a function. I can do that just fine, but I want to add some error checking to see that scipy.optimize.fsolve actually finds a solution. In particular, fsolve returns a parameter ier which is just a success flag and I want to read it.
My code is as follows:
import numpy as np
from scipy.optimize import fsolve, minimize
mu = -8
sigma = 4
mun = -2
ARL0 = 2000
hmin = 0.5
hmax = 100
f = lambda h: (np.exp(-2*mun*(h/sigma+1.166))-1+2*mun*(h/sigma+1.166))/(2*mun**2)-ARL0
if f(hmin)*f(hmax) < 0:
opth, ier = fsolve(f,hmax)
print ier
print opth[0]
else:
f = lambda h: np.abs((np.exp(-2*mun*(h/sigma+1.166))-1+2*mun*(h/sigma+1.166))/(2*mun**2)-ARL0)
opth = minimize(f,hmax,bounds=((hmin,hmax),))
print opth.success
print opth.x[0]
The "else" bit works fine, it prints the solution and a true/false if it found one. The first if block doesn't: I get the following error when it runs:
line 14, in <module>
opth, ier = fsolve(f,hmax)
ValueError: need more than 1 value to unpack
I'm guessing it's just a synatx error but I haven't been able to find an example using fsolve to do this. Can someone point me in the right direction?

You need to add the argument full_output=True and there are actually 4 return values, looking at the docs, so you need to unpack all of them.
Here's the source for fsolve in my version of SciPy (0.14.0), you can see the two options for return values:
def fsolve(func, x0, args=(), fprime=None, full_output=0,
col_deriv=0, xtol=1.49012e-8, maxfev=0, band=None,
epsfcn=None, factor=100, diag=None):
options = {'col_deriv': col_deriv,
'xtol': xtol,
'maxfev': maxfev,
'band': band,
'eps': epsfcn,
'factor': factor,
'diag': diag,
'full_output': full_output}
res = _root_hybr(func, x0, args, jac=fprime, **options)
if full_output:
x = res['x']
info = dict((k, res.get(k))
for k in ('nfev', 'njev', 'fjac', 'r', 'qtf') if k in res)
info['fvec'] = res['fun']
return x, info, res['status'], res['message']
else:
return res['x']