We Keep Coding

rpy2 python-r mixed function

I was trying to run an R function through python's rpy2:
def gph(input1):
frac = rpackages.importr('fracdiff')
a = robjects.r(f'''
fdGPH({input1})
''')
print(a)
it returns an error however:
Traceback (most recent call last):
File "C:\Users\************\mainpage.py", line 168, in update_output
analiza_statystyczna.gph()
File "C:\Users\************\analiza_statystyczna.py", line 151, in gph
a = robjects.r(f'''
File "C:\Users\************\lib\site-packages\rpy2\robjects\__init__.py", line 450, in __call__
p = rinterface.parse(string)
File "C:\Users\************\lib\site-packages\rpy2\rinterface_lib\conversion.py", line 45, in _
cdata = function(*args, **kwargs)
File "C:\Users\************\lib\site-packages\rpy2\rinterface.py", line 108, in parse
res = _rinterface._parse(robj.__sexp__._cdata, num, rmemory)
File "C:\Users\************\lib\site-packages\rpy2\rinterface_lib\_rinterface_capi.py", line 652, in _parse
raise RParsingError('Parsing status not OK',
rpy2.rinterface_lib._rinterface_capi.RParsingError: Parsing status not OK - PARSING_STATUS.PARSE_ERROR
What am I doing wrong?

Error executing FMU model with pyFMI: "pyfmi.fmi.FMUException: Failed to get the Boolean values"

I am using the code below to simulate a model.
def run_demo(with_plots=True):
traj = np.array([[start_time,2.25]])
input_object = ('input_1[1]', traj)
model = load_fmu('[pyfmimodel.fmu',log_level=7)
opts = model.simulate_options ()
opts['ncp']=266
# Simulate
res = model.simulate(options=opts, input=input_object,final_time=stop_time )
This is the error I am getting. I need help to resolve this error.
Traceback (most recent call last):
File "D:\Projects\Python\DOCKER\model_2.py", line 55, in <module>
run_demo()
File "D:\Projects\Python\DOCKER\model_2.py", line 38, in run_demo
res = model.simulate(options=opts, input=input_object,final_time=stop_time )
File "src\pyfmi\fmi.pyx", line 7519, in pyfmi.fmi.FMUModelCS2.simulate
File "src\pyfmi\fmi.pyx", line 378, in pyfmi.fmi.ModelBase._exec_simulate_algorithm
File "src\pyfmi\fmi.pyx", line 372, in pyfmi.fmi.ModelBase._exec_simulate_algorithm
File "C:\Users\tcto5k\Miniconda3\lib\site-packages\pyfmi\fmi_algorithm_drivers.py", line 984, in __init__
self.result_handler.simulation_start()
File "C:\Users\tcto5k\Miniconda3\lib\site-packages\pyfmi\common\io.py", line 2553, in simulation_start
[parameter_data, sorted_vars_real_vref, sorted_vars_int_vref, sorted_vars_bool_vref] = fmi_util.prepare_data_info(data_info, sorted_vars,
File "src\pyfmi\fmi_util.pyx", line 257, in pyfmi.fmi_util.prepare_data_info
File "src\pyfmi\fmi_util.pyx", line 337, in pyfmi.fmi_util.prepare_data_info
File "src\pyfmi\fmi.pyx", line 4377, in pyfmi.fmi.FMUModelBase2.get_boolean
pyfmi.fmi.FMUException: Failed to get the Boolean values.
This is the FMU model variable definition which accepts 1D array as input:
<ScalarVariable name="input_1[1]" valueReference="0" description="u" causality="input" variability="continuous">
<Real start="2.0"/>
</ScalarVariable>
<!-- 2 -->
<ScalarVariable name="dense_3[1]" valueReference="614" description="y (1st order)" causality="output" variability="continuous" initial="calculated">
<Real/>
</ScalarVariable>

PicklingError when getting the result from ray

I'm working on slowly converting my very serialized text analysis engine to use Modin and Ray. Feels like I'm nearly there, however, I seem to have hit a stumbling block. My code looks like this:
vectorizer = TfidfVectorizer(
analyzer=ngrams, encoding="ascii", stop_words="english", strip_accents="ascii"
)
tf_idf_matrix = vectorizer.fit_transform(r_strings["name"])
r_vectorizer = ray.put(vectorizer)
r_tf_idf_matrix = ray.put(tf_idf_matrix)
n = 2
match_results = []
for fn in files["c.file"]:
match_results.append(
match_name.remote(fn, r_vectorizer, r_tf_idf_matrix, r_strings, n)
)
match_returns = ray.get(match_results)
I'm following the guidance from the "anti-patterns" section in the Ray documentation, on what to avoid, and this is very similar to that of the "better" pattern.
Traceback (most recent call last):
File "alt.py", line 213, in <module>
match_returns = ray.get(match_results)
File "/home/myuser/.local/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 62, in wrapper
return func(*args, **kwargs)
File "/home/myuser/.local/lib/python3.7/site-packages/ray/worker.py", line 1501, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(PicklingError): ray::match_name() (pid=23393, ip=192.168.1.173)
File "python/ray/_raylet.pyx", line 564, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 565, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 1652, in ray._raylet.CoreWorker.store_task_outputs
File "/home/myuser/.local/lib/python3.7/site-packages/ray/serialization.py", line 327, in serialize
return self._serialize_to_msgpack(value)
File "/home/myuser/.local/lib/python3.7/site-packages/ray/serialization.py", line 307, in _serialize_to_msgpack
self._serialize_to_pickle5(metadata, python_objects)
File "/home/myuser/.local/lib/python3.7/site-packages/ray/serialization.py", line 267, in _serialize_to_pickle5
raise e
File "/home/myuser/.local/lib/python3.7/site-packages/ray/serialization.py", line 264, in _serialize_to_pickle5
value, protocol=5, buffer_callback=writer.buffer_callback)
File "/home/myuser/.local/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
cp.dump(obj)
File "/home/myuser/.local/lib/python3.7/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 580, in dump
return Pickler.dump(self, obj)
_pickle.PicklingError: args[0] from __newobj__ args has the wrong class
Definitely an unexpected result. I'm not sure where to go next with this and would appreciate help from folks who have more experience with Ray and Modin.

Problem with scapy summary function in python

I've imported the scapy module in my python code (arp-spoofer) and when i use the packet.show()/packet.summary() function the terminal return me this error:
Error:
Traceback (most recent call last):
File "arp-spoofer.py", line 10, in <module>
print(packet.show())
File "/home/baloo/.local/lib/python3.7/site-packages/scapy/packet.py", line 1261, in show
return self._show_or_dump(dump, indent, lvl, label_lvl)
File "/home/baloo/.local/lib/python3.7/site-packages/scapy/packet.py", line 1235, in _show_or_dump
reprval = f.i2repr(self, fvalue)
File "/home/baloo/.local/lib/python3.7/site-packages/scapy/fields.py", line 376, in i2repr
return fld.i2repr(pkt, val)
File "/home/baloo/.local/lib/python3.7/site-packages/scapy/fields.py", line 502, in i2repr
x = self.i2h(pkt, x)
File "/home/baloo/.local/lib/python3.7/site-packages/scapy/layers/l2.py", line 136, in i2h
iff = self.getif(pkt)
File "/home/baloo/.local/lib/python3.7/site-packages/scapy/layers/l2.py", line 132, in <lambda>
self.getif = (lambda pkt: pkt.route()[0]) if getif is None else getif
File "/home/baloo/.local/lib/python3.7/site-packages/scapy/layers/l2.py", line 400, in route
fld, dst = fld._find_fld_pkt_val(self, dst)
File "/home/baloo/.local/lib/python3.7/site-packages/scapy/fields.py", line 313, in _find_fld_pkt_val
if val == dflts_pkt[self.name] and self.name not in pkt.fields:
File "/home/baloo/.local/lib/python3.7/site-packages/scapy/base_classes.py", line 133, in __eq__
p2, nm2 = self._parse_net(other)
File "/home/baloo/.local/lib/python3.7/site-packages/scapy/base_classes.py", line 99, in _parse_net
tmp = net.split('/') + ["32"]
AttributeError: 'NoneType' object has no attribute 'split'
Code:
import scapy.all as scapy
victim_ip = ""
victim_mac_address = ""
router_ip = ""
packet = scapy.ARP(op=2, pdst=victim_ip, hwdst=victim_mac_address, psrc=router_ip)
print(packet.show())
print(packet.summary())

You need the IPs to be valid.
If you don't want to set them yourself, don't specify them and Scapy will take the default.

(Casting) errors using extract_(relevant_)features from tsfresh

Trying out Python package tsfresh I run into issues in the first steps. Given a series how to (automatically) make features for it? This snippet produces different errors based on which part I try.
import tsfresh
import pandas as pd
import numpy as np
#tfX, tfy = tsfresh.utilities.dataframe_functions.make_forecasting_frame(pd.Series(np.random.randn(1000)/50), kind='float64', max_timeshift=50, rolling_direction=1)
#rf = tsfresh.extract_relevant_features(tfX, y=tfy, n_jobs=1, column_id='id')
tfX, tfy = tsfresh.utilities.dataframe_functions.make_forecasting_frame(pd.Series(np.random.randn(1000)/50), kind=1, max_timeshift=50, rolling_direction=1)
rf = tsfresh.extract_relevant_features(tfX, y=tfy, n_jobs=1, column_id='id')
The errors are in the first case
""" Traceback (most recent call last): File "C:\Users\user\Anaconda3\envs\env1\lib\multiprocessing\pool.py", line
119, in worker
result = (True, func(*args, **kwds)) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\utilities\distribution.py",
line 38, in _function_with_partly_reduce
results = list(itertools.chain.from_iterable(results)) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\utilities\distribution.py",
line 37, in
results = (map_function(chunk, **kwargs) for chunk in chunk_list) File
"C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\feature_extraction\extraction.py",
line 358, in _do_extraction_on_chunk
return list(_f()) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\feature_extraction\extraction.py",
line 350, in _f
result = [("", func(data))] File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\feature_extraction\feature_calculators.py",
line 193, in variance_larger_than_standard_deviation
y = np.var(x) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\numpy\core\fromnumeric.py",
line 3157, in var
**kwargs) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\numpy\core_methods.py",
line 110, in _var
arrmean, rcount, out=arrmean, casting='unsafe', subok=False) TypeError: unsupported operand type(s) for /: 'str' and 'int' """
and in the second case
""" Traceback (most recent call last): File
"C:\Users\user\Anaconda3\envs\env1\lib\multiprocessing\pool.py", line
119, in worker
result = (True, func(*args, **kwds)) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\utilities\distribution.py",
line 38, in _function_with_partly_reduce
results = list(itertools.chain.from_iterable(results)) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\utilities\distribution.py",
line 37, in
results = (map_function(chunk, **kwargs) for chunk in chunk_list) File
"C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\feature_extraction\extraction.py",
line 358, in _do_extraction_on_chunk
return list(_f()) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\feature_extraction\extraction.py",
line 345, in _f
result = func(data, param=parameter_list) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\feature_extraction\feature_calculators.py",
line 1752, in friedrich_coefficients
coeff = _estimate_friedrich_coefficients(x, m, r) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\tsfresh\feature_extraction\feature_calculators.py",
line 145, in _estimate_friedrich_coefficients
result.dropna(inplace=True) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\frame.py",
line 4598, in dropna
result = self.loc(axis=axis)[mask] File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\indexing.py",
line 1500, in getitem
return self._getitem_axis(maybe_callable, axis=axis) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\indexing.py",
line 1859, in _getitem_axis
if is_iterator(key): File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\dtypes\inference.py",
line 157, in is_iterator
return hasattr(obj, 'next') File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\generic.py",
line 5065, in getattr
if self._info_axis._can_hold_identifiers_and_holds_name(name): File
"C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\indexes\base.py",
line 3984, in _can_hold_identifiers_and_holds_name
return name in self File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\indexes\category.py",
line 327, in contains
return contains(self, key, container=self._engine) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\arrays\categorical.py",
line 188, in contains
loc = cat.categories.get_loc(key) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\indexes\interval.py",
line 770, in get_loc
start, stop = self._find_non_overlapping_monotonic_bounds(key) File
"C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\indexes\interval.py",
line 717, in _find_non_overlapping_monotonic_bounds
start = self._searchsorted_monotonic(key, 'left') File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\indexes\interval.py",
line 681, in _searchsorted_monotonic
return sub_idx._searchsorted_monotonic(label, side) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\indexes\base.py",
line 4755, in _searchsorted_monotonic
return self.searchsorted(label, side=side) File "C:\Users\user\Anaconda3\envs\env1\lib\site-packages\pandas\core\base.py",
line 1501, in searchsorted
return self._values.searchsorted(value, side=side, sorter=sorter) TypeError: Cannot cast array data from dtype('float64') to
dtype('
np.version, tsfresh.version are ('1.15.4', 'unknown'). I installed tsfresh using conda, probably from conda-forge. I am on Windows 10. Using another kernel with np.version, tsfresh.version ('1.15.4', '0.11.2') lead to the same results.
Trying the first couple of cells from timeseries_forecasting_basic_example.ipynb yields the casting error as well.

Fixed it. Either the version on conda(-forge) or one of the dependencies was the issue. So using "conda uninstall tsfresh", "conda install patsy future six tqdm" and "pip install tsfresh" combined did the trick.