Related
My problem is about getting emissions results of my functional unit from a ecoinvent excel spreadsheet format.
I managed to get activities/process impacts thanks to ca.annotated_top_processes(lca) or lca.top_activities()but emissions/biosphere flows can't be displayed but through ca.hinton_matrix(lca, rows=10, cols=10). How can I get specific scores ?
Here's the situation:
import brightway2 as bw
from stats_arrays import *
import bw2analyzer as bwa
projects.set_current("excel_import_verif1")
bw.databases
db = bw.Database('IoTBOLLCA') #Excel spreadsheet
CC = [method for method in bw.methods if "('ReCiPe Midpoint (H) V1.13', 'climate change', 'GWP100')" in str(method)][0]
FU = [i for i in db if 'FU' in i['name']][0]
lca = bw.LCA({FU:1},CC)
lca.lci()
lca.lcia()
lca.score
ca = bwa.ContributionAnalysis()
lca.top_emissions()
and I get this error
TypeError Traceback (most recent call last)
File ~\Anaconda3\envs\bw2\lib\site-packages\scipy\sparse\_sputils.py:208, in isintlike(x)
207 try:
--> 208 operator.index(x)
209 except (TypeError, ValueError):
TypeError: 'numpy.float64' object cannot be interpreted as an integer
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
Input In [28], in <cell line: 1>()
----> 1 lca.top_emissions()
File ~\Anaconda3\envs\bw2\lib\site-packages\bw2calc\lca.py:575, in LCA.top_emissions(self, **kwargs)
573 except ImportError:
574 raise ImportError("`bw2analyzer` is not installed")
--> 575 return ContributionAnalysis().annotated_top_emissions(self, **kwargs)
File ~\Anaconda3\envs\bw2\lib\site-packages\bw2analyzer\contribution.py:152, in ContributionAnalysis.annotated_top_emissions(self, lca, names, **kwargs)
146 """Get list of most damaging biosphere flows in an LCA, sorted by ``abs(direct impact)``.
147
148 Returns a list of tuples: ``(lca score, inventory amount, activity)``. If ``names`` is False, they returns the process key as the last element.
149
150 """
151 ra, rp, rb = lca.reverse_dict()
--> 152 results = [
153 (score, lca.inventory[index, :].sum(), rb[index])
154 for score, index in self.top_emissions(
155 lca.characterized_inventory, **kwargs
156 )
157 ]
158 if names:
159 results = [(x[0], x[1], get_activity(x[2])) for x in results]
File ~\Anaconda3\envs\bw2\lib\site-packages\bw2analyzer\contribution.py:153, in <listcomp>(.0)
146 """Get list of most damaging biosphere flows in an LCA, sorted by ``abs(direct impact)``.
147
148 Returns a list of tuples: ``(lca score, inventory amount, activity)``. If ``names`` is False, they returns the process key as the last element.
149
150 """
151 ra, rp, rb = lca.reverse_dict()
152 results = [
--> 153 (score, lca.inventory[index, :].sum(), rb[index])
154 for score, index in self.top_emissions(
155 lca.characterized_inventory, **kwargs
156 )
157 ]
158 if names:
159 results = [(x[0], x[1], get_activity(x[2])) for x in results]
File ~\Anaconda3\envs\bw2\lib\site-packages\scipy\sparse\_index.py:47, in IndexMixin.__getitem__(self, key)
46 def __getitem__(self, key):
---> 47 row, col = self._validate_indices(key)
49 # Dispatch to specialized methods.
50 if isinstance(row, INT_TYPES):
File ~\Anaconda3\envs\bw2\lib\site-packages\scipy\sparse\_index.py:152, in IndexMixin._validate_indices(self, key)
149 M, N = self.shape
150 row, col = _unpack_index(key)
--> 152 if isintlike(row):
153 row = int(row)
154 if row < -M or row >= M:
File ~\Anaconda3\envs\bw2\lib\site-packages\scipy\sparse\_sputils.py:216, in isintlike(x)
214 if loose_int:
215 msg = "Inexact indices into sparse matrices are not allowed"
--> 216 raise ValueError(msg)
217 return loose_int
218 return True
ValueError: Inexact indices into sparse matrices are not allowed
This is an error as of Scipy version 1.9; for now, you can force a downgrade to Scipy 1.8.something.
This has been noted as an issue, but the focus for BW development is in other areas currently.
I've been studying time series forecasting, and I'm trying to learn how to use gluon-ts&python.
Here is the source code of gluon-ts:
https://github.com/awslabs/gluon-ts/
I tried to use LSTNetEstimator module, however, it turns out an error as follow. And I found that all the discussions of gluon-ts is about DeepAR.
Is there anyone that may come to help?
GluonTSDataError: Input for field "target" does not have the requireddimension (field: target, ndim observed: 1, expected ndim: 2)
I think it has something to do with my custom dataset, which includes a dataframe with the shape of [320000,3] and columns of ['time','Power_Cdp','Power_Dp'].
here is the trainset:
from gluonts.dataset.common import ListDataset
from gluonts.model.lstnet import LSTNetEstimator
from gluonts.mx.trainer import Trainer
training_data = ListDataset(
[{"start": LSTNet_df.index[0], "target": LSTNet_df['Power_Cdp'][:-10000}],
freq = "15min")
estimator = LSTNetEstimator(freq="15min", prediction_length=24*4, context_length=24*4,
num_series=48*4, skip_size=72*4, ar_window=24*4, channels=32,
trainer=Trainer(epochs=10))
predictor = estimator.train(training_data=training_data) # Error
The full report is bellow.
GluonTSDataError Traceback (most recent call last)
<ipython-input-311-ad48fdc20df5> in <module>
7 num_series=48*4, skip_size=72*4, ar_window=24*4, channels=32,
8 trainer=Trainer(epochs=10))
----> 9 predictor = estimator.train(training_data=training_data)
~/.local/lib/python3.8/site-packages/gluonts/mx/model/estimator.py in train(self, training_data, validation_data, num_workers, num_prefetch, shuffle_buffer_length, cache_data, **kwargs)
192 **kwargs,
193 ) -> Predictor:
--> 194 return self.train_model(
195 training_data=training_data,
196 validation_data=validation_data,
~/.local/lib/python3.8/site-packages/gluonts/mx/model/estimator.py in train_model(self, training_data, validation_data, num_workers, num_prefetch, shuffle_buffer_length, cache_data)
145 transformed_training_data = transformation.apply(training_data)
146
--> 147 training_data_loader = self.create_training_data_loader(
148 transformed_training_data
149 if not cache_data
~/.local/lib/python3.8/site-packages/gluonts/model/lstnet/_estimator.py in create_training_data_loader(self, data, **kwargs)
216 ) -> DataLoader:
217 input_names = get_hybrid_forward_input_names(LSTNetTrain)
--> 218 with env._let(max_idle_transforms=maybe_len(data) or 0):
219 instance_splitter = self._create_instance_splitter("training")
220 return TrainDataLoader(
~/.local/lib/python3.8/site-packages/gluonts/itertools.py in maybe_len(obj)
21 def maybe_len(obj) -> Optional[int]:
22 try:
---> 23 return len(obj)
24 except (NotImplementedError, AttributeError):
25 return None
~/.local/lib/python3.8/site-packages/gluonts/transform/_base.py in __len__(self)
99 # NOTE this is unsafe when transformations are run with is_train = True
100 # since some transformations may not be deterministic (instance splitter)
--> 101 return sum(1 for _ in self)
102
103 def __iter__(self) -> Iterator[DataEntry]:
~/.local/lib/python3.8/site-packages/gluonts/transform/_base.py in <genexpr>(.0)
99 # NOTE this is unsafe when transformations are run with is_train = True
100 # since some transformations may not be deterministic (instance splitter)
--> 101 return sum(1 for _ in self)
102
103 def __iter__(self) -> Iterator[DataEntry]:
~/.local/lib/python3.8/site-packages/gluonts/transform/_base.py in __iter__(self)
102
103 def __iter__(self) -> Iterator[DataEntry]:
--> 104 yield from self.transformation(
105 self.base_dataset, is_train=self.is_train
106 )
~/.local/lib/python3.8/site-packages/gluonts/transform/_base.py in __call__(self, data_it, is_train)
122 self, data_it: Iterable[DataEntry], is_train: bool
123 ) -> Iterator:
--> 124 for data_entry in data_it:
125 try:
126 yield self.map_transform(data_entry.copy(), is_train)
~/.local/lib/python3.8/site-packages/gluonts/transform/_base.py in __call__(self, data_it, is_train)
126 yield self.map_transform(data_entry.copy(), is_train)
127 except Exception as e:
--> 128 raise e
129
130 #abc.abstractmethod
~/.local/lib/python3.8/site-packages/gluonts/transform/_base.py in __call__(self, data_it, is_train)
124 for data_entry in data_it:
125 try:
--> 126 yield self.map_transform(data_entry.copy(), is_train)
127 except Exception as e:
128 raise e
~/.local/lib/python3.8/site-packages/gluonts/transform/_base.py in map_transform(self, data, is_train)
139
140 def map_transform(self, data: DataEntry, is_train: bool) -> DataEntry:
--> 141 return self.transform(data)
142
143 #abc.abstractmethod
~/.local/lib/python3.8/site-packages/gluonts/transform/convert.py in transform(self, data)
127 value = np.asarray(data[self.field], dtype=self.dtype)
128
--> 129 assert_data_error(
130 value.ndim == self.expected_ndim,
131 'Input for field "{self.field}" does not have the required'
~/.local/lib/python3.8/site-packages/gluonts/exceptions.py in assert_data_error(condition, message, *args, **kwargs)
114 exception message.
115 """
--> 116 assert_gluonts(GluonTSDataError, condition, message, *args, **kwargs)
~/.local/lib/python3.8/site-packages/gluonts/exceptions.py in assert_gluonts(exception_class, condition, message, *args, **kwargs)
93 """
94 if not condition:
---> 95 raise exception_class(message.format(*args, **kwargs))
96
97
GluonTSDataError: Input for field "target" does not have the requireddimension (field: target, ndim observed: 1, expected ndim: 2)
#darth baba I tried again lately, and this time I used DeepVAR.
Here is what got.
I reset "target": ... , stepped into my code and found
gluonts.exceptions.GluonTSDataError: Array 'target' has bad shape - expected 1 dimensions, got 2.
This is because I set "target": train_df[['Power_Cdp', 'Power_Active_Fan']], which might be reported as an error at ./gluonts/dataset/common.py +385.
I found self.req_ndim != value.ndim , which suggested that I could have input a wrong shape of target, thus I reset input like "target": train_df.index[:] and this problem solved.
But it reports an another error,
gluonts.exceptions.GluonTSDataError: Input for field "target" does not have the requireddimension (field: target, ndim observed: 1, expected ndim: 2)
To be sincerely, this one confused me a lot, for the gluonts's code seems so complicated.
I checked again and found it reports error at ./gluonts/transform/convert.py +129.
It seems like expected_ndim is not equals to value.ndim in this case.
However, after rewriting expected_ndim manually, the training progress worked eventually.
I have no idea whether this modification is right.
Although the avg_epoch_loss are decreasing, the final forecast is not as good as expected.
I am looking for a simple way (2 or 3 lines of code) to generate a Phi(k) correlation matrix in Python.
That should be possible since pandas_profiling is doing it, and it works fine.
But I want to be able to do it without pandas_profiling which is too heavy and computes things I don't need.
pandas_profiling is using phik library.
I tried phik library (didn't find anything else)
I don't understand the error I got :
TypeError: sequence item 0: expected str instance, int found
I have no int in my dataframe.
Seems like a bug in phik, but then how does pandas profiling do, since it's using it too ?
What's happening here ?
Many thanks
I have this code :
import numpy as np
import pandas as pd
import phik
NB_SAMPLES = 200
NB_VARIABLES = 3
rand_mat = np.random.uniform(low=0.5, high=15, size=(NB_SAMPLES,NB_VARIABLES))
df = pd.DataFrame(rand_mat)
df['cat_column'] = pd.cut(df[0], bins=5, labels=['F1','F2','F3','F4','F5'])
print(df)
df.phik_matrix()
Result :
0 1 2 cat_column
0 0.911098 8.549206 9.270484 F1
1 13.591250 9.161498 5.614470 F5
2 3.308305 1.589402 5.394675 F1
3 12.031064 9.968686 7.519628 F5
4 14.427813 1.533533 2.352659 F5
.. ... ... ... ...
195 10.556285 3.541869 4.804826 F4
196 5.721784 11.783908 13.104844 F2
197 7.336637 14.512256 14.993096 F3
198 4.375895 11.881784 1.129816 F2
199 0.519900 6.624423 9.239070 F1
[200 rows x 4 columns]
interval_cols not set, guessing: [0, 1, 2]
---------------------------------------------------------------------------
_RemoteTraceback Traceback (most recent call last)
_RemoteTraceback:
"""
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 418, in _process_worker
r = call_item()
File "/opt/conda/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 272, in __call__
return self.fn(*self.args, **self.kwargs)
File "/opt/conda/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 608, in __call__
return self.func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/joblib/parallel.py", line 256, in __call__
for func, args, kwargs in self.items]
File "/opt/conda/lib/python3.7/site-packages/joblib/parallel.py", line 256, in <listcomp>
for func, args, kwargs in self.items]
File "/opt/conda/lib/python3.7/site-packages/phik/phik.py", line 162, in _calc_phik
combi = ':'.join(comb)
TypeError: sequence item 0: expected str instance, int found
"""
The above exception was the direct cause of the following exception:
TypeError Traceback (most recent call last)
<ipython-input-31-398c72b34799> in <module>
11 df['cat_column'] = pd.cut(df[0], bins=5, labels=['F1','F2','F3','F4','F5'])
12 print(df)
---> 13 df.phik_matrix()
/opt/conda/lib/python3.7/site-packages/phik/phik.py in phik_matrix(df, interval_cols, bins, quantile, noise_correction, dropna, drop_underflow, drop_overflow)
215 data_binned, binning_dict = bin_data(df_clean, cols=interval_cols_clean, bins=bins, quantile=quantile, retbins=True)
216 return phik_from_rebinned_df(data_binned, noise_correction, dropna=dropna, drop_underflow=drop_underflow,
--> 217 drop_overflow=drop_overflow)
218
219
/opt/conda/lib/python3.7/site-packages/phik/phik.py in phik_from_rebinned_df(data_binned, noise_correction, dropna, drop_underflow, drop_overflow)
145
146 phik_list = Parallel(n_jobs=NCORES)(delayed(_calc_phik)(co, data_binned[list(co)], noise_correction)
--> 147 for co in itertools.combinations_with_replacement(data_binned.columns.values, 2))
148
149 phik_overview = create_correlation_overview_table(dict(phik_list))
/opt/conda/lib/python3.7/site-packages/joblib/parallel.py in __call__(self, iterable)
1015
1016 with self._backend.retrieval_context():
-> 1017 self.retrieve()
1018 # Make sure that we get a last message telling us we are done
1019 elapsed_time = time.time() - self._start_time
/opt/conda/lib/python3.7/site-packages/joblib/parallel.py in retrieve(self)
907 try:
908 if getattr(self._backend, 'supports_timeout', False):
--> 909 self._output.extend(job.get(timeout=self.timeout))
910 else:
911 self._output.extend(job.get())
/opt/conda/lib/python3.7/site-packages/joblib/_parallel_backends.py in wrap_future_result(future, timeout)
560 AsyncResults.get from multiprocessing."""
561 try:
--> 562 return future.result(timeout=timeout)
563 except LokyTimeoutError:
564 raise TimeoutError()
/opt/conda/lib/python3.7/concurrent/futures/_base.py in result(self, timeout)
433 raise CancelledError()
434 elif self._state == FINISHED:
--> 435 return self.__get_result()
436 else:
437 raise TimeoutError()
/opt/conda/lib/python3.7/concurrent/futures/_base.py in __get_result(self)
382 def __get_result(self):
383 if self._exception:
--> 384 raise self._exception
385 else:
386 return self._result
TypeError: sequence item 0: expected str instance, int found
Try to reinstall the phik module as the following:
pip install phik==0.10.0
Then, your code together with sns.heatmap results the following:
I'm trying to open the Sage's notebook, but it isn't working.
I have no idea where this error came from, because the notebook was working this week. I guess it just popped up out of nowhere.
The message's error is:
sage: notebook()
---------------------------------------------------------------------------
EnvironmentError Traceback (most recent call last)
<ipython-input-4-3728cb3d7c7d> in <module>()
----> 1 notebook()
/home/jerome/opt/SageMath/src/sage/misc/lazy_import.pyx in sage.misc.lazy_import.LazyImport.__call__ (/home/jerome/opt/SageMath/src/build/cythonized/sage/misc/lazy_import.c:3634)()
384 True
385 """
--> 386 return self._get_object()(*args, **kwds)
387
388 def __repr__(self):
/home/jerome/opt/SageMath/src/sage/misc/lazy_import.pyx in sage.misc.lazy_import.LazyImport._get_object (/home/jerome/opt/SageMath/src/build/cythonized/sage/misc/lazy_import.c:2241)()
244 elif self._at_startup and not startup_guard:
245 print('Option ``at_startup=True`` for lazy import {0} not needed anymore'.format(self._name))
--> 246 self._object = getattr(__import__(self._module, {}, {}, [self._name]), self._name)
247 alias = self._as_name or self._name
248 if self._deprecation is not None:
/home/jerome/opt/SageMath/local/lib/python2.7/site-packages/sagenb/notebook/notebook_object.py in <module>()
15 import time, os, shutil, signal, tempfile
16
---> 17 import notebook as _notebook
18
19 import run_notebook
/home/jerome/opt/SageMath/local/lib/python2.7/site-packages/sagenb/notebook/notebook.py in <module>()
33
34 # Sage libraries
---> 35 from sagenb.misc.misc import (pad_zeros, cputime, tmp_dir, load, save,
36 ignore_nonexistent_files, unicode_str)
37
/home/jerome/opt/SageMath/local/lib/python2.7/site-packages/sagenb/misc/misc.py in <module>()
379
380 try:
--> 381 from sage.misc.cython import cython
382 except ImportError:
383 #stub
/home/jerome/opt/SageMath/local/lib/python2.7/site-packages/sage/misc/cython.py in <module>()
28
29 # CBLAS can be one of multiple implementations
---> 30 cblas_pc = pkgconfig.parse('cblas')
31 cblas_libs = list(cblas_pc['libraries'])
32 cblas_library_dirs = list(cblas_pc['library_dirs'])
/home/jerome/opt/SageMath/local/lib/python2.7/site-packages/pkgconfig-1.1.0-py2.7.egg/pkgconfig/pkgconfig.py in parse(packages)
185
186 for package in packages.split():
--> 187 for k, v in parse_package(package).items():
188 result[k].update(v)
189
/home/jerome/opt/SageMath/local/lib/python2.7/site-packages/pkgconfig-1.1.0-py2.7.egg/pkgconfig/pkgconfig.py in parse_package(package)
158
159 # Execute the query to pkg-config and clean the result.
--> 160 out = _query(package, '--cflags --libs')
161 out = out.replace('\\"', '')
162
/home/jerome/opt/SageMath/local/lib/python2.7/site-packages/pkgconfig-1.1.0-py2.7.egg/pkgconfig/pkgconfig.py in _wrapper(*args, **kwargs)
56 return func(*args, **kwargs)
57 except OSError:
---> 58 raise EnvironmentError("pkg-config is not installed")
59
60 return _wrapper
EnvironmentError: pkg-config is not installed
If you guys can help me, I'll be very thankful!
I'm trying to reproduce coal mining example with deterministic function for switchpoint instead of using theano's switch function. Code:
%matplotlib inline
import matplotlib.pyplot as plt
import pymc3
import numpy as np
import theano.tensor as t
import theano
data = np.hstack((np.random.poisson(15,1000),np.random.poisson(2,100)))
plt.plot(data)
#theano.compile.ops.as_op(itypes=[t.lscalar, t.dscalar,t.dscalar],otypes=[t.dvector])
def rate1(sw,mu1,mu2):
n = len(data)
out = np.empty(n)
out[:sw] = mu1
out[sw:] = mu2
return out
with pymc3.Model() as dis:
switchpoint = pymc3.DiscreteUniform('switchpoint',lower=0, upper=len(data)-1)
mu1 = pymc3.Exponential('mu1', lam=1.)
mu2 = pymc3.Exponential('mu2',lam=1.)
disasters=pymc3.Poisson('disasters', mu=rate1, observed = data)
But this code rise an error:
--------------------------------------------------------------------------- KeyError Traceback (most recent call
last) c:\program files\git\theano\theano\tensor\type.py in
dtype_specs(self)
266 'complex64': (complex, 'theano_complex64', 'NPY_COMPLEX64')
--> 267 }[self.dtype]
268 except KeyError:
KeyError: 'object'
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call
last) c:\program files\git\theano\theano\tensor\basic.py in
constant_or_value(x, rtype, name, ndim, dtype)
407 rval = rtype(
--> 408 TensorType(dtype=x_.dtype, broadcastable=bcastable),
409 x_.copy(),
c:\program files\git\theano\theano\tensor\type.py in init(self,
dtype, broadcastable, name, sparse_grad)
49 self.broadcastable = tuple(bool(b) for b in broadcastable)
---> 50 self.dtype_specs() # error checking is done there
51 self.name = name
c:\program files\git\theano\theano\tensor\type.py in dtype_specs(self)
269 raise TypeError("Unsupported dtype for %s: %s"
--> 270 % (self.class.name, self.dtype))
271
TypeError: Unsupported dtype for TensorType: object
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call
last) c:\program files\git\theano\theano\tensor\basic.py in
as_tensor_variable(x, name, ndim)
201 try:
--> 202 return constant(x, name=name, ndim=ndim)
203 except TypeError:
c:\program files\git\theano\theano\tensor\basic.py in constant(x,
name, ndim, dtype)
421 ret = constant_or_value(x, rtype=TensorConstant, name=name, ndim=ndim,
--> 422 dtype=dtype)
423
c:\program files\git\theano\theano\tensor\basic.py in
constant_or_value(x, rtype, name, ndim, dtype)
416 except Exception:
--> 417 raise TypeError("Could not convert %s to TensorType" % x, type(x))
418
TypeError: ('Could not convert FromFunctionOp{rate1} to TensorType',
)
During handling of the above exception, another exception occurred:
AsTensorError Traceback (most recent call
last) in ()
14 mu2 = pymc3.Exponential('mu2',lam=1.)
15 #rate1 = pymc3.switch(switchpoint >= np.arange(len(data)), mu1,mu2)
---> 16 disasters=pymc3.Poisson('disasters', mu=rate1, observed = data)
C:\Users\User\Anaconda3\lib\site-packages\pymc3\distributions\distribution.py
in new(cls, name, *args, **kwargs)
19 if isinstance(name, str):
20 data = kwargs.pop('observed', None)
---> 21 dist = cls.dist(*args, **kwargs)
22 return model.Var(name, dist, data)
23 elif name is None:
C:\Users\User\Anaconda3\lib\site-packages\pymc3\distributions\distribution.py
in dist(cls, *args, **kwargs)
32 def dist(cls, *args, **kwargs):
33 dist = object.new(cls)
---> 34 dist.init(*args, **kwargs)
35 return dist
36
C:\Users\User\Anaconda3\lib\site-packages\pymc3\distributions\discrete.py
in init(self, mu, *args, **kwargs)
185 super(Poisson, self).init(*args, **kwargs)
186 self.mu = mu
--> 187 self.mode = floor(mu).astype('int32')
188
189 def random(self, point=None, size=None, repeat=None):
c:\program files\git\theano\theano\gof\op.py in call(self,
*inputs, **kwargs)
598 """
599 return_list = kwargs.pop('return_list', False)
--> 600 node = self.make_node(*inputs, **kwargs)
601
602 if config.compute_test_value != 'off':
c:\program files\git\theano\theano\tensor\elemwise.py in
make_node(self, *inputs)
540 using DimShuffle.
541 """
--> 542 inputs = list(map(as_tensor_variable, inputs))
543 shadow = self.scalar_op.make_node(
544 *[get_scalar_type(dtype=i.type.dtype).make_variable()
c:\program files\git\theano\theano\tensor\basic.py in
as_tensor_variable(x, name, ndim)
206 except Exception:
207 str_x = repr(x)
--> 208 raise AsTensorError("Cannot convert %s to TensorType" % str_x, type(x))
209
210 # this has a different name, because _as_tensor_variable is the
AsTensorError: ('Cannot convert FromFunctionOp{rate1} to TensorType',
)
How i handle this?
The second thing - when i'm using the pymc3.switch function like this:
with pymc3.Model() as dis:
switchpoint = pymc3.DiscreteUniform('switchpoint',lower=0, upper=len(data)-1)
mu1 = pymc3.Exponential('mu1', lam=1.)
mu2 = pymc3.Exponential('mu2',lam=1.)
rate1 = pymc3.switch(switchpoint >= np.arange(len(data)), mu1,mu2)
disasters=pymc3.Poisson('disasters', mu=rate1, observed = data)
And next try to sample:
with dis:
step1 = pymc3.NUTS([mu1, mu2])
step2 = pymc3.Metropolis([switchpoint])
trace = pymc3.sample(10000, step = [step1,step2])
I get an error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
c:\program files\git\theano\theano\compile\function_module.py in __call__(self, *args, **kwargs)
858 try:
--> 859 outputs = self.fn()
860 except Exception:
TypeError: expected type_num 9 (NPY_INT64) got 7
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-4-3247d908f897> in <module>()
2 step1 = pymc3.NUTS([mu1, mu2])
3 step2 = pymc3.Metropolis([switchpoint])
----> 4 trace = pymc3.sample(10000, step = [step1,step2])
C:\Users\User\Anaconda3\lib\site-packages\pymc3\sampling.py in sample(draws, step, start, trace, chain, njobs, tune, progressbar, model, random_seed)
153 sample_args = [draws, step, start, trace, chain,
154 tune, progressbar, model, random_seed]
--> 155 return sample_func(*sample_args)
156
157
C:\Users\User\Anaconda3\lib\site-packages\pymc3\sampling.py in _sample(draws, step, start, trace, chain, tune, progressbar, model, random_seed)
162 progress = progress_bar(draws)
163 try:
--> 164 for i, strace in enumerate(sampling):
165 if progressbar:
166 progress.update(i)
C:\Users\User\Anaconda3\lib\site-packages\pymc3\sampling.py in _iter_sample(draws, step, start, trace, chain, tune, model, random_seed)
244 if i == tune:
245 step = stop_tuning(step)
--> 246 point = step.step(point)
247 strace.record(point)
248 yield strace
C:\Users\User\Anaconda3\lib\site-packages\pymc3\step_methods\compound.py in step(self, point)
11 def step(self, point):
12 for method in self.methods:
---> 13 point = method.step(point)
14 return point
C:\Users\User\Anaconda3\lib\site-packages\pymc3\step_methods\arraystep.py in step(self, point)
116 bij = DictToArrayBijection(self.ordering, point)
117
--> 118 apoint = self.astep(bij.map(point))
119 return bij.rmap(apoint)
120
C:\Users\User\Anaconda3\lib\site-packages\pymc3\step_methods\metropolis.py in astep(self, q0)
123
124
--> 125 q_new = metrop_select(self.delta_logp(q,q0), q, q0)
126
127 if q_new is q:
c:\program files\git\theano\theano\compile\function_module.py in __call__(self, *args, **kwargs)
869 node=self.fn.nodes[self.fn.position_of_error],
870 thunk=thunk,
--> 871 storage_map=getattr(self.fn, 'storage_map', None))
872 else:
873 # old-style linkers raise their own exceptions
c:\program files\git\theano\theano\gof\link.py in raise_with_op(node, thunk, exc_info, storage_map)
312 # extra long error message in that case.
313 pass
--> 314 reraise(exc_type, exc_value, exc_trace)
315
316
C:\Users\User\Anaconda3\lib\site-packages\six.py in reraise(tp, value, tb)
656 value = tp()
657 if value.__traceback__ is not tb:
--> 658 raise value.with_traceback(tb)
659 raise value
660
c:\program files\git\theano\theano\compile\function_module.py in __call__(self, *args, **kwargs)
857 t0_fn = time.time()
858 try:
--> 859 outputs = self.fn()
860 except Exception:
861 if hasattr(self.fn, 'position_of_error'):
TypeError: expected type_num 9 (NPY_INT64) got 7
Apply node that caused the error: Elemwise{Composite{Switch(GE(i0, i1), i2, i3)}}(InplaceDimShuffle{x}.0, TensorConstant{[ 0 1..1098 1099]}, InplaceDimShuffle{x}.0, InplaceDimShuffle{x}.0)
Toposort index: 11
Inputs types: [TensorType(int64, (True,)), TensorType(int32, vector), TensorType(float64, (True,)), TensorType(float64, (True,))]
Inputs shapes: [(1,), (1100,), (1,), (1,)]
Inputs strides: [(4,), (4,), (8,), (8,)]
Inputs values: [array([549]), 'not shown', array([ 1.07762995]), array([ 1.01502801])]
Outputs clients: [[Elemwise{eq,no_inplace}(Elemwise{Composite{Switch(GE(i0, i1), i2, i3)}}.0, TensorConstant{(1,) of 0}), Elemwise{Composite{Switch(GE(i0, i1), ((Switch(i2, i3, (i4 * log(i0))) - i5) - i0), i3)}}[(0, 0)](Elemwise{Composite{Switch(GE(i0, i1), i2, i3)}}.0, TensorConstant{(1,) of 0}, InplaceDimShuffle{x}.0, TensorConstant{(1,) of -inf}, TensorConstant{[ 13. 13... 0. 1.]}, TensorConstant{[ 22.55216... ]})]]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
Being simple analyst, should i learn all this stuff about theano to being able to work with my statistical problems? Is a new mcmc sampler with gradient feature is only one thing that should motivates me to switch from pymc2 to pymc3?
For your first question, it looks like you're trying to pass a theano function as a variable. You need to call the function with the other variables as arguments, which will then return a theano variable. Try changing your line to
disasters=pymc3.Poisson('disasters', mu=rate1(switchpoint, mu1, mu2), observed = data)
I couldn't reproduce the error in your second part; the sampling worked just fine for me.