We Keep Coding

Multiprocessing array .get_lock works on one computer but not another

I am working a somewhat extensive python program that uses multiprocessing. Because I wanted the user to see some progress on the console when running the program, I read about using a shared counter on stackoverflow and after a while of playing around with my code, I got it to work. As I said it's too much code to post here, but the gist is that I instantiate a multiprocessing array after the name==main line,
if __name__ == "__main__":
total_progress_counter = Array('i',[0,0])
and then during the main portion of code I pass this array to a function in other module:
some_name.plot(<other variables>,
total_progress_counter=total_progress_counter)
Then within that other function, I used the .get_lock method that I found described here on stackoverflow:
with total_progress_counter.get_lock():
total_progress_counter[0] += self.total_panels_to_plot
I also update the other component, total_progress_counter[1], in the same function. This works fine for me on my work machine, where I wrote the code, and that machine has a Centos operating system.
But, when I run it on my personal MacBook it gives the following traceback:
Traceback (most recent call last):
File "./program.py", line 775, in <module>
program.run()
File "./program.py", line 177, in run
cases_plotted = pool.map(self.__plot__, all_cases)
File "/opt/anaconda3/lib/python3.8/multiprocessing/pool.py", line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/opt/anaconda3/lib/python3.8/multiprocessing/pool.py", line 771, in get
raise self._value
AttributeError: 'list' object has no attribute 'get_lock'
I have python3 version 3.8.3 on my personal machine and python3 version 3.7.4 on my work machine. Can anyone help me understand why I'm getting different behavior on these two environments? I'd be grateful, as this is meant to be software others might use on different machines.

cusolver library not found

I am trying to use skcuda in my python code but whenever i want to use it, it rises the following exception:
Traceback (most recent call last):
File "/home/rohola/Projects/Python/wordnetwork/s.py", line 6, in <module>
from skcuda import cusolver
File "/home/rohola/anaconda3/lib/python3.4/site- packages/skcuda/cusolver.py", line 51, in <module>
raise OSError('cusolver library not found')
OSError: cusolver library not found
i was just tried to use skcuda
from skcuda import cusolver
handle = cusolver.cusolverDnCreate()
i installed pycuda, NVIDIA CUDA Toolkit. What's wrong about my code or dependencies?

import ctypes
a = ctypes.cdll.LoadLibrary( "/usr/local/cuda-8.0/targets/x86_64-linux/lib/libcusolver.so" )
gets me
OSError: /usr/local/cuda-8.0/targets/x86_64-linux/lib/libcusolver.so: undefined symbol: GOMP_critical_end
which seems to be a yet unsolved issue according to this github thread
Though, a 7.5 version which
/usr/lib/x86_64-linux-gnu/libcusolver.so
seems to point to works well. And author's fix seems to work with it.

I finally found a solution to my problem. Firstly, i searched for libcusolver.so with
locate libcusolver.so
and then changed the following code in cusolver.py from:
_libcusolver = ctypes.cdll.LoadLibrary(_libcusolver_libname)
to:
_libcusolver = ctypes.cdll.LoadLibrary("/usr/local/cuda-7.5/targets/x86_64-linux/lib/"+_libcusolver_libname)
now the following code:
from skcuda import cusolver
handle = cusolver.cusolverDnCreate()
works without any error.
UPDATE:
If you have installed cuda8.0 or 9.0 the best way to solve the problem is to find this line in cusolver.py:
_libcusolver = ctypes.cdll.LoadLibrary(_libcusolver_libname)
and add this line before it:
ctypes.CDLL('libgomp.so.1', mode=ctypes.RTLD_GLOBAL)
You have to make sure that you already installed libgomp.

comtypes: in call_with_inout, ctypes TypeError: 'c_double' object is not iterable

Im working with Agilent IVI drivers in Python 2.7.9 and can't seem to get 'proven' code to work on a particular Windows 7 machine. It executes successfully on other machines.
While this issue seems rather limited to one instrument, it appears to be a broader Python issue, so I turn to Stack Overflow for help. Any help or insight would be great.
The following code
# Import the TLB
from comtypes.client import GetModule, CreateObject
GetModule('AgInfiniium.dll')
# Pull in the coefficients and classes, we'll need those
from comtypes.gen.AgilentInfiniiumLib import *
# Instantiate the driver
OScope = CreateObject('AgilentInfiniium.AgilentInfiniium')
# Make constants of our settings
Address = "TCPIP0::10.254.0.222::INSTR"
resetOScope = False
# Open a connection to the scope
OScope.Initialize(Address,False,resetOScope,'')
# Make a measurement
print OScope.Measurements.Item("Channel1").ReadWaveformMeasurement(
MeasFunction=AgilentInfiniiumMeasurementAmplitude, MaxTime=10)
yields the following error:
Traceback (most recent call last):
File "P:\Aperture\Validation\WMI_BGA_Board\TestMatrixCode\scopeTest2.py", line 29, in <module>
print OScope.Measurements.Item("Channel1").ReadWaveformMeasurement(MeasFunction=AgilentInfiniiumMeasurementAmplitude ,MaxTime=10)
File "C:\Python27\lib\site-packages\comtypes-1.1.0-py2.7.egg\comtypes\__init__.py", line 656, in call_with_inout
rescode = list(rescode)
TypeError: 'c_double' object is not iterable
In my limited debugging attempts, I have seen that this call_with_inout
function tries to convert my Python arguments into arguments for the following C++ function:
public void ReadWaveformMeasurement(
AgilentInfiniiumMeasurementEnum MeasFunction,
AgilentInfiniiumTimeOutEnum MaxTime,
ref double pMeasurement)
It's creating some kind of variable for the pMeasurement pointer that ends up being type c_double, and then complains that it's not iterable.
At this point, this seems like it's local to this machine. I've gone to the extent of uninstalling Python, reinstalling the Agilent driver, and trying two versions of comtypes (1.1.0 and 1.1.1). Yet the problem persists. Any ideas? Thanks.

gnuradio `ImportError undefined symbol`

I'm new to GNU Radio and python. I'm trying to write a correlation block,
somewhere in my code I use fft filter:
gr::filter::kernel::fft_filter_ccc *d_filter;
d_filter = new gr::filter::kernel::fft_filter_ccc(1, x_vector);
d_filter->filter(noutput_items,in_y,out);
I run
cmake ../
make
and it complies perfectly fine,
but when i try
make test
I'll get this error:
Traceback (most recent call last):
2: File "/home/mohammad/projects/FD/implementation_tests/oot_modules/gr-full_duplex/python/qa_fd_correlation_cc.py", line 25, in <module>
2: import full_duplex_swig as full_duplex
2: File "/home/mohammad/projects/FD/implementation_tests/oot_modules/gr-full_duplex/build/swig/full_duplex_swig.py", line 28, in <module>
2: _full_duplex_swig = swig_import_helper()
2: File "/home/mohammad/projects/FD/implementation_tests/oot_modules/gr-full_duplex/build/swig/full_duplex_swig.py", line 24, in swig_import_helper
2: _mod = imp.load_module('_full_duplex_swig', fp, pathname, description)
2: ImportError: /home/mohammad/projects/FD/implementation_tests/oot_modules/gr-full_duplex/build/lib/libgnuradio-full_duplex.so: undefined symbol: _ZN2gr6filter6kernel14fft_filter_cccC1EiRKSt6vectorISt7complexIfESaIS5_EEi
1/1 Test #2: qa_fd_correlation_cc .............***Failed 1.30 sec

This often happens when you have declared a method at the header file, but you do not implement it. For example a destructor or something else.
To find out which method it is you should demangle the undefined symbol _ZN2gr6filter6kernel14fft_filter_cccC1EiRKSt6vectorISt7complexIfESaIS5_EEi.
This can be done using the c++filt tool. For example,
c++filt \ _ZN2gr6filter6kernel14fft_filter_cccC1EiRKSt6vectorISt7complexIfESaIS5_EEi
In your case this symbol is an existing symbol of the GNU Radio, located in the gr-filter module. Each GNU Radio module creates a library, so in order to resolve the undefined symbol issue you have to link against the required library. To accomplish this you have to do the following steps:
At the CMakeLists.txt file of your module, specify on which components of GNU Radio you depend. In your case the FILTER component.
set(GR_REQUIRED_COMPONENTS RUNTIME FILTER)
find_package(Gnuradio "3.7.0" REQUIRED)
Further dependencies can be inserted, eg:
set(GR_REQUIRED_COMPONENTS RUNTIME FILTER DIGITAL)
After that you can use the ${GNURADIO_ALL_INCLUDE_DIRS} and ${GNURADIO_ALL_LIBRARIES} auto generated variables to properly include the proper header files and link against the appropriate libraries.
E.g:
include_directories(
${CMAKE_SOURCE_DIR}/lib
${CMAKE_SOURCE_DIR}/include
${CMAKE_BINARY_DIR}/lib
${CMAKE_BINARY_DIR}/include
${Boost_INCLUDE_DIRS}
${CPPUNIT_INCLUDE_DIRS}
${GNURADIO_RUNTIME_INCLUDE_DIRS}
${GNURADIO_ALL_INCLUDE_DIRS}
)
target_link_libraries(gnuradio-howto
${Boost_LIBRARIES}
${GNURADIO_RUNTIME_LIBRARIES}
${GNURADIO_ALL_LIBRARIES})
For more info refer here.

Update to answer by #manos:
As of GNU Radio 3.8+ the CMakeList.txt files have slightly changed.
The line one needs to change in the toplevel CMakeList.txt is:
find_package(Gnuradio "3.8" REQUIRED COMPONENTS blocks analog filter fft)
And in lib/CMakeList.txt:
target_link_libraries(gnuradio-tempest gnuradio::gnuradio-runtime gnuradio::gnuradio-blocks gnuradio::gnuradio-fft gnuradio::gnuradio-filter Volk::volk)
Source: https://wiki.gnuradio.org/index.php/GNU_Radio_3.8_OOT_Module_Porting_Guide

Wrapping c++ functions in python with ctypes on windows : function not found

I need to run a series of python scripts calculating various scripts, that are working fine, but one of them runs very slowly and has to be done in C++.
The C++ code is ready, but I need to find a way to call the C++ function from the python and get the return value.
I found information about SWIG, but didn't get it to work on Windows with Visual Studio (I have to do it in VS on Windows, because of other constraints). I found ctypes much easier for my very simple function with standart input and output values.
So I did many tests with ctypes, tried all the examples I could find online, and each and every time, what happens is that I can load the dll (built with visual studio 2012 compiler) but when I try calling the function in python, it just goes :
Traceback (most recent call last):
File "test.py", line 3, in <module>
print maLib.plop()
File "C:\Python27\lib\ctypes\__init__.py", line 378, in __getattr__
func = self.__getitem__(name)
File "C:\Python27\lib\ctypes\__init__.py", line 383, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'plop' not found
The python code is (test.py) :
from ctypes import cdll
lib = cdll.LoadLibrary('test')
print lib.plop()
The C++ code is (test.cpp, compiled as test.dll, Visual Studio set to build as dll) :
extern "C"
{
int plop();
}
int plop() { return 4; }
As you can see, I tried to make it as simple as possible, to avoid details making it fail. I have read the python ctypes help, and tutorials on how to use ctypes, trying exactly the same codes as them, but I had to adapt a bit because I am using Visual Studio / windows and most of the other users are using linux.
All of the files are in the same folder.
I have tried multiple ways of loading the library : LoadLibrary('./name.dll'), WinDLL('name.dll'), giving the function a different name, void return type, etc...
Do you think I should use SWIG ? Is my problem an obvious beginner mistake ? I am a student, and new to most of what I'm using, but I put a lot of effort in this, even if I just need it for a particular single task.
When using SWIG, I had to make a wrapper with function pointers, or references, so I thought this was the problem, which made me want to try a simpler solution.
I am using : Python 2.7.7, Visual Studio 2012, Windows 8.1
Many thanks in advance

The error reported by your Python is very clear.
function 'plop' not found
This means that the DLL does not export a function of that name. So, you need to export the function. Either with a .def file, or using __declspec(dllexport):
extern "C"
{
__declspec(dllexport) int plop();
}
To inspect your DLL to debug issues like this, use dumpbin or Dependency Walker.
Note that since you do not specify calling convention, the default of __cdecl is used. That means that cdll is correct on the ctypes side.

You need to give your function "C" linkage to avoid name mangling. Declare it like this:
extern "C"
{
int plop();
}
and it will then properly be called "plop" rather than being name-mangled.