I am interesting in solving differential equation using fipy.
The following code is working correctly when I am using Grid2D.
from fipy import *
mesh = Grid2D(nx=3, ny=3)
#mesh = Grid3D(nx=3, ny=3, nz=3)
phi = CellVariable(name='solution variable', mesh=mesh, value=0.)
phi.constrain(0, mesh.facesLeft)
phi.constrain(10, mesh.facesRight)
coeff = CellVariable(mesh=mesh, value=1.)
eq = DiffusionTerm(coeff) == 0
eq.solve(var=phi)
When I am using Grid3D instead of Grid2D (commented line), I get following error:
Traceback (most recent call last):
File "/home/user/Programming/python/fdms/forSo.py", line 11, in <module>
eq.solve(var=phi)
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/terms/term.py", line 211, in solve
solver = self._prepareLinearSystem(var, solver, boundaryConditions, dt)
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/terms/term.py", line 169, in _prepareLinearSystem
diffusionGeomCoeff=self._getDiffusionGeomCoeff(var),
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/terms/abstractDiffusionTerm.py", line 458, in _getDiffusionGeomCoeff
return self._getGeomCoeff(var)
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/terms/term.py", line 465, in _getGeomCoeff
self.geomCoeff = self._calcGeomCoeff(var)
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/terms/abstractDiffusionTerm.py", line 177, in _calcGeomCoeff
tmpBop = (coeff * FaceVariable(mesh=mesh, value=mesh._faceAreas) / mesh._cellDistances)[numerix.newaxis, :]
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/variables/variable.py", line 1151, in __mul__
return self._BinaryOperatorVariable(lambda a,b: a*b, other)
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/variables/variable.py", line 1116, in _BinaryOperatorVariable
if not v.unit.isDimensionless() or len(v.shape) > 3:
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/variables/variable.py", line 255, in _getUnit
return self._extractUnit(self.value)
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/variables/variable.py", line 538, in _getValue
value = self._calcValue()
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/variables/cellToFaceVariable.py", line 48, in _calcValue
alpha = self.mesh._faceToCellDistanceRatio
File "/home/user/Programs/miniconda2/envs/FipyEnv2/lib/python3.6/site-packages/fipy/meshes/uniformGrid3D.py", line 269, in _faceToCellDistanceRatio
XZdis[..., 0,...] = 1
IndexError: an index can only have a single ellipsis ('...')
I installed fipy using «Recomended method» from https://www.ctcms.nist.gov/fipy/INSTALLATION.html. I tried to install using Miniconda for both Pthon 3.6 and Python 2.7 and got same errors.
How to solve equations using Grid3D?
This is because newer versions of numpy are less tolerant of our sloppy syntax. You can either checkout our develop source branch or make this change to your code.
Related
I am tackling a bayesian inference problem and am having trouble using a pymc3 sampler provided by pypesto on my windows laptop. To make sure I can run with the sampler I create a simple dummy objective to use.
I install create a conda (I tried both 3.7 & 3.8) environment and install the pymc3 and theano modules using pip3/pip. I've tried several different versions of both pymc3/theano and managed to import them succesfully. However, there is an error message I cannot figure out how to go around. I have tried looking online for a solution but was not able to find it either. I currently have the latest versions of pymc3 and theano installed (3.11.0 and 1.0.5 respectively). This is the final line of the message
theano.graph.fg.MissingInputError: Input 0 of the graph (indices start from 0), used to compute sigmoid(x2_interval__), was not provided and not given a value. Use the Theano flag exception_verbosity='high', for more information on this error.
Here is the full message:
Sampling 1 chain for 1_000 tune and 100 draw iterations (1_000 + 100 draws total) took 7 seconds.
Traceback (most recent call last):
File "samplingPymc3.py", line 70, in <module>
result2 = sample.sample(problem1, 100, sampler2, x0=np.array([0,0]))
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\pypesto\sample\sample.py", line 68, in sample
sampler.sample(n_samples=n_samples)
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\pypesto\sample\pymc3.py", line 102, in sample
**self.options)
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\pymc3\sampling.py", line 637, in sample
idata = arviz.from_pymc3(trace, **ikwargs)
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\arviz\data\io_pymc3.py", line 559, in from_pymc3
density_dist_obs=density_dist_obs,
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\arviz\data\io_pymc3.py", line 163, in __init__
self.observations, self.multi_observations = self.find_observations()
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\arviz\data\io_pymc3.py", line 176, in find_observations
multi_observations[key] = val.eval() if hasattr(val, "eval") else val
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\theano\graph\basic.py", line 554, in eval
self._fn_cache[inputs] = theano.function(inputs, self)
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\theano\compile\function\__init__.py", line 350, in function
output_keys=output_keys,
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\theano\compile\function\pfunc.py", line 532, in pfunc
output_keys=output_keys,
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\theano\compile\function\types.py", line 1978, in orig_function
name=name,
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\theano\compile\function\types.py", line 1584, in __init__
fgraph, additional_outputs = std_fgraph(inputs, outputs, accept_inplace)
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\theano\compile\function\types.py", line 188, in std_fgraph
fgraph = FunctionGraph(orig_inputs, orig_outputs, update_mapping=update_mapping)
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\theano\graph\fg.py", line 162, in __init__
self.import_var(output, reason="init")
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\theano\graph\fg.py", line 330, in import_var
self.import_node(var.owner, reason=reason)
File "C:\Users\germa\anaconda3\envs\sampling\lib\site-packages\theano\graph\fg.py", line 383, in import_node
raise MissingInputError(error_msg, variable=var)
theano.graph.fg.MissingInputError: Input 0 of the graph (indices start from 0), used to compute sigmoid(x2_interval__), was not provided and not given a value. Use the Theano flag exception_verbosity='high', for more information on this error.
I read somewhere that the issue may lie with the version of arviz used but that does not appear to be the issue in my case.
I wanted to include the script I am running. Here is the code for the script:
import numpy as np
import scipy as sp
import scipy.optimize as so
from scipy.stats import multivariate_normal
import pypesto
import pypesto.sample as sample
from pypesto import Objective
A = np.array([[2.0, 0.0], [0.0, 1.0]])
b = np.array([2.0, 1.0])
x_init = np.array([3.4302, 2.915])
x_true = np.array([1.0, 1.0])
temp = lambda x: A.dot(x) - b
f = lambda x: .5 * np.linalg.norm(temp(x))
A_t = A.transpose()
K = np.dot(A_t, A)
df = lambda x: K.dot(x) - A_t.dot(b)
def obj1(x):
# f_val = f(x)
# grad = df(x)
return (f(x), df(x))
objfun = lambda x: obj1(x)
dim_full = 2
lb = -10 * np.ones((dim_full, 1))
ub = 10 * np.ones((dim_full, 1))
x_names = ['x1', 'x2']
# step_fcn = pymc3.step_methods.hmc.hmc.HamiltonianMC
objective = pypesto.Objective(fun=objfun, grad=True, hess=False)
problem1 = pypesto.Problem(objective=objective, lb=lb, ub=ub, x_names=x_names)
sampler = sample.AdaptiveMetropolisSampler()
print('function val: ', objfun(x_init))
sampler2 = sample.Pymc3Sampler()
result2 = sample.sample(problem1, 100, sampler2, x0=np.array([0, 0]))
print('Done sampling!')
Thank you in advance for any help!
pymc3 support of pypesto is at the moment limited, as it was implemented at a time when theano was discontinued in favor of aesara in pymc3. Thus, pypesto only supports specific version of the involved tools, specifically
arviz >= 0.8.1, < 0.9.0
theano >= 1.0.4
packaging >= 20.0
pymc3 >= 3.8, < 3.9.2
(see https://github.com/ICB-DCM/pyPESTO/blob/main/setup.cfg#L111).
The switch to full aesara and later pymc3 version support is underway, but not out yet.
I want to solve the following optimization problem with scipy (python3):
where I[.,.] is the "incomplete gamma function" (scipy.special.gammainc)
I followed the scipy optimization guide and came up with this code:
from scipy.special import gammainc
from scipy.optimize import NonlinearConstraint
from scipy.optimize import BFGS
from scipy.optimize import minimize
import numpy as np
q = 200
k = 1.5625
alpha = 0.03125
def func(s):
return s
def constraints(s):
return q - k/alpha * (gammainc(k+1, s*alpha) + gammainc(k+1, (s+q)*alpha)) + s*gammainc(k, s*alpha) + (s+q)*gammainc(k, (s+q)*alpha)
nonlinear_constraint = NonlinearConstraint(constraints, -np.inf, 10.0, jac='2-point', hess=BFGS())
s0 = 1000
res = minimize(func, s0, method='trust-constr', jac='2-point', hess=BFGS(), constraints=[nonlinear_constraint], options={'verbose': 1})
print(res.x)
when I run the code, I get this (error) output:
/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/optimize/_hessian_update_strategy.py:187: UserWarning: delta_grad == 0.0. Check if the approximated function is linear. If the function is linear better results can be obtained by defining the Hessian as zero instead of using quasi-Newton approximations.
'approximations.', UserWarning)
Traceback (most recent call last):
File "s_opt.py", line 23, in <module>
res = minimize(func, s0, method='trust-constr', jac='2-point', hess=BFGS(), constraints=[nonlinear_constraint], options={'verbose': 1})
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/optimize/_minimize.py", line 622, in minimize
callback=callback, **options)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/optimize/_trustregion_constr/minimize_trustregion_constr.py", line 519, in _minimize_trustregion_constr
factorization_method)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/optimize/_trustregion_constr/tr_interior_point.py", line 329, in tr_interior_point
factorization_method, trust_lb, trust_ub, subprob.scaling)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/optimize/_trustregion_constr/equality_constrained_sqp.py", line 121, in equality_constrained_sqp
lb_t, ub_t)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/optimize/_trustregion_constr/qp_subproblem.py", line 499, in projected_cg
r = Z.dot(H.dot(x) + c)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/sparse/linalg/interface.py", line 415, in dot
return self.matvec(x)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/sparse/linalg/interface.py", line 229, in matvec
y = self._matvec(x)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/sparse/linalg/interface.py", line 527, in _matvec
return self.__matvec_impl(x)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/optimize/_trustregion_constr/projections.py", line 193, in null_space
aux2 = scipy.linalg.solve_triangular(R, aux1, lower=False)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/linalg/basic.py", line 336, in solve_triangular
b1 = _asarray_validated(b, check_finite=check_finite)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/scipy/_lib/_util.py", line 246, in _asarray_validated
a = toarray(a)
File "/home/andreasziegler/miniconda3/lib/python3.7/site-packages/numpy/lib/function_base.py", line 499, in asarray_chkfinite
"array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNs
which tells me that I did something wrong. It's the first time I use the optimizer in scipy and therefore chances are high, that I misunderstand something (essential). I'm glad for any hint.
I tried to use distplot to plot an array of double value but failed. Below is my source code:
>>> import seaborn as sns, numpy as np
>>> sns.set(); np.random.seed(0)
>>> x = np.random.randn(100)
>>> ax = sns.distplot(x)
Below is the error I got. I don't know what wrong with my code. Does anyone know the issue?
>>> ax = sns.distplot(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/anaconda/lib/python3.6/site-packages/seaborn/distributions.py", line 221, in distplot
kdeplot(a, vertical=vertical, ax=ax, color=kde_color, **kde_kws)
File "/anaconda/lib/python3.6/site-packages/seaborn/distributions.py", line 604, in kdeplot
cumulative=cumulative, **kwargs)
File "/anaconda/lib/python3.6/site-packages/seaborn/distributions.py", line 270, in _univariate_
kdeplot
cumulative=cumulative)
File "/anaconda/lib/python3.6/site-packages/seaborn/distributions.py", line 328, in _statsmodels
_univariate_kde
kde.fit(kernel, bw, fft, gridsize=gridsize, cut=cut, clip=clip)
File "/anaconda/lib/python3.6/site-packages/statsmodels/nonparametric/kde.py", line 146, in fit
clip=clip, cut=cut)
File "/anaconda/lib/python3.6/site-packages/statsmodels/nonparametric/kde.py", line 506, in kden
sityfft
f = revrt(zstar)
File "/anaconda/lib/python3.6/site-packages/statsmodels/nonparametric/kdetools.py", line 20, in
revrt
y = X[:m/2+1] + np.r_[0,X[m/2+1:],0]*1j
TypeError: slice indices must be integers or None or have an __index__ method
BTW, I am using python3.6.
This is caused by an old version of statsmodels and the problem is fixed in version 0.8.0. Upgrade it as described in https://github.com/mwaskom/seaborn/issues/1092
conda update statsmodels
I'm working in Python2.7 (Anaconda 4.0) on a Jupyter notebook on a EC2 instance with plenty of memory (60GB, 48GB free according to free). I've loaded a Pandas (v0.18) dataframe that is large (150K rows, ~30KB per row), but is nowhere near the memory capacity of the instance, even if many many copies are made. Certain Pandas and Scikit-learn (v0.17) calls will trigger a MemoryError instantly, e.g.:
#X is a subset of the original df with 60 columns instead of the 3000
#Y is a float column
X.add(Y)
#And for sklearn...
pca = decomposition.KernelPCA(n_components=5)
pca.fit(X,Y)
Meanwhile, these work fine:
Z = X.copy(deep=True)
pca = decomposition.PCA(n_components=5)
Most perplexingly, I can do this and it finishes in a few seconds:
huge = range(1000000000)
I've rebooted the notebook, the kernel, and the instance, but the same calls keep giving the MemoryError. I've also verified that I'm using 64-bit Python. Any suggestions?
Update: adding the traceback errors:
Traceback (most recent call last):
File "<ipython-input-9-ae71777140e2>", line 2, in <module>
Z = X.add(Y)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/pandas/core/ops.py", line 1057, in f
return self._combine_series(other, na_op, fill_value, axis, level)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/pandas/core/frame.py", line 3500, in _combine_series
fill_value=fill_value)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/pandas/core/frame.py", line 3528, in _combine_match_columns
copy=False)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/pandas/core/frame.py", line 2730, in align
broadcast_axis=broadcast_axis)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py", line 4152, in align
fill_axis=fill_axis)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/pandas/core/generic.py", line 4234, in _align_series
fdata = fdata.reindex_indexer(join_index, lidx, axis=0)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/pandas/core/internals.py", line 3528, in reindex_indexer
fill_tuple=(fill_value,))
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/pandas/core/internals.py", line 3591, in _slice_take_blocks_ax0
fill_value=fill_value))
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/pandas/core/internals.py", line 3621, in _make_na_block
block_values = np.empty(block_shape, dtype=dtype)
MemoryError
and
Traceback (most recent call last):
File "<ipython-input-13-d510bc16443e>", line 3, in <module>
pca.fit(X,Y)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/sklearn/decomposition/kernel_pca.py", line 202, in fit
K = self._get_kernel(X)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/sklearn/decomposition/kernel_pca.py", line 135, in _get_kernel
filter_params=True, **params)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/sklearn/metrics/pairwise.py", line 1347, in pairwise_kernels
return _parallel_pairwise(X, Y, func, n_jobs, **kwds)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/sklearn/metrics/pairwise.py", line 1054, in _parallel_pairwise
return func(X, Y, **kwds)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/sklearn/metrics/pairwise.py", line 716, in linear_kernel
return safe_sparse_dot(X, Y.T, dense_output=True)
File "/home/ec2-user/anaconda2/lib/python2.7/site-packages/sklearn/utils/extmath.py", line 184, in safe_sparse_dot
return fast_dot(a, b)
MemoryError
Figured out the Pandas side of the issue. I had a DF and a Series with matching indexes, X and Y. I figured I could add Y as another column like this:
X.add(Y)
But doing this tries to match Y on the columns, not on the index, thus creating a 150Kx150K array. I needed to supply the axis:
X.add(Y, axis='index')
I'm trying to code a multiple linear regression. Here's the line of code where my program raises an error:
least = optimize.minimize(residsq(xmat, ylist, coeff), coeff, constraints = ({'type': 'eq', 'fun': sum(resid(xmat, ylist, coeff))}), method = 'BFGS') # Choose the coefficients that minimize the sum of the residuals squared subject to keeping the sum of the residuals equal to 0.
xmat is a list of vectors: [[3,5,2],[3,1,6],[7,2,3], [9,-2,0]]. ylist is a list of the same length as xmat: [5,2,7,7]. coeff is the coefficient list, initially [mean(ylist), 0, 0, 0] ([constant, b_0, b_1, b_2]). resid is the list of residuals for each point, and residsq is the N2 norm of the residuals (sqrt of sum of squares).
Traceback (most recent call last):
File "<pyshell#3>", line 1, in <module>
import linregtest
File "C:\Python33\lib\site-packages\linregtest.py", line 4, in <module>
out = linreg.multilinreg(xmat, ylist, True)
File "C:\Python33\lib\site-packages\linreg.py", line 120, in multilinreg
least = optimize.minimize(residsq(xmat, ylist, coeff), coeff, constraints = ({'type': 'eq', 'fun': sum(resid(xmat, ylist, coeff))}), method = 'BFGS') # Choose the coefficients that minimize the sum of the residuals squared subject to keeping the sum of the residuals equal to 0.
File "C:\Python33\lib\site-packages\scipy\optimize\_minimize.py", line 302, in minimize
RuntimeWarning)
File "C:\Python33\lib\idlelib\PyShell.py", line 60, in idle_showwarning
file.write(warnings.formatwarning(message, category, filename,
AttributeError: 'NoneType' object has no attribute 'write'
Where does file come from, and how do I suppress this error?
EDIT: Solve one problem, find another. Maybe you can help me determine where SciPy is calling a float?
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import linregtest
File "C:\Python33\lib\site-packages\linregtest.py", line 4, in <module>
out = linreg.multilinreg(xmat, ylist, True)
File "C:\Python33\lib\site-packages\linreg.py", line 123, in multilinreg
least = optimize.minimize(residsq(xmat, ylist, coeff), coeff, constraints = ({'type': 'eq', 'fun': sumresid(xmat, ylist, coeff)}), method = 'SLSQP') # Choose the coefficients that minimize the sum of the residuals squared subject to keeping the sum of the residuals equal to 0.
File "C:\Python33\lib\site-packages\scipy\optimize\_minimize.py", line 364, in minimize
constraints, **options)
File "C:\Python33\lib\site-packages\scipy\optimize\slsqp.py", line 301, in _minimize_slsqp
meq = sum(map(len, [atleast_1d(c['fun'](x, *c['args'])) for c in cons['eq']]))
File "C:\Python33\lib\site-packages\scipy\optimize\slsqp.py", line 301, in <listcomp>
meq = sum(map(len, [atleast_1d(c['fun'](x, *c['args'])) for c in cons['eq']]))
TypeError: 'float' object is not callable
I just edited my python 3.2 IDLE, PyShell.py (fixing lines 59 and 62)
def idle_showwarning(message, category, filename, lineno,
file=None, line=None):
if file is None:
file = sys.stderr #warning_stream
try:
file.write(warnings.formatwarning(message, category, filename,
lineno, line=line))
use sys.stderr instead of the global warning_stream which uses sys.__stderr__. sys.__stderr__ is None in my case. I don't know why a global is used.
the call to warnings.formatwarning had an extra invalid file keyword.
Now, I get the warning printed, for example
>>> import numpy as np
>>> np.uint(1) - np.uint(2)
Warning (from warnings module):
File "C:\Programs\Python32\Lib\idlelib\idle.pyw", line 1
try:
RuntimeWarning: overflow encountered in ulong_scalars
>>> 4294967295
>>>
edit:
searching for python bug reports
http://bugs.python.org/issue12438 wrong file argument has been fixed
http://bugs.python.org/issue13582 problems with sys.__stderr__ is None is open