We Keep Coding

Pyro4 [Errno -2]

I am trying out pyro4 connection between my PC and Raspberry Pi 4.
Code on my PC is:
# saved as server.py
import Pyro4, Pyro4.naming
import socket, threading
# Define an object that will be accessible over the network.
# This is where all your code should go...
#Pyro4.expose
class MessageServer(object):
def show_message(self, msg):
print("Message received: {}".format(msg))
# Start a Pyro nameserver and daemon (server process) that are accessible
# over the network. This has security risks; see
# https://pyro4.readthedocs.io/en/stable/security.html
hostname = socket.gethostname()
ns_thread = threading.Thread(
target=Pyro4.naming.startNSloop, kwargs={'host': hostname}
)
ns_thread.daemon = True # automatically exit when main program finishes
ns_thread.start()
main_daemon = Pyro4.Daemon(host=hostname)
# find the name server
ns = Pyro4.locateNS()
# register the message server as a Pyro object
main_daemon_uri = main_daemon.register(MessageServer)
# register a name for the object in the name server
ns.register("example.message", main_daemon_uri)
# start the event loop of the main_daemon to wait for calls
print("Message server ready.")
main_daemon.requestLoop()
And code on my Raspberry is:
import Pyro4
import sys
print("Message:")
msg=sys.stdin.readline().strip()
message_server = Pyro4.Proxy("PYRONAME:192.168.1.5")
message_server.show_message(msg)
Code on my PC doesn t show any errors, but when I try to send a message from raspberry i get this:
What s your message?
test
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 102, in getIpAddress
return getaddr(config.PREFER_IP_VERSION) if ipVersion is None else getaddr(ipVersion)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 94, in getaddr
ip = socket.getaddrinfo(hostname or socket.gethostname(), 80, family, socket.SOCK_STREAM, socket.SOL_TCP)[0][4][0]
File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 515, in connect_and_handshake
sslContext=sslContext)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 266, in createSocket
if getIpVersion(connect[0]) == 4:
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 68, in getIpVersion
address = getIpAddress(hostnameOrAddress)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 106, in getIpAddress
return getaddr(0)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/socketutil.py", line 94, in getaddr
ip = socket.getaddrinfo(hostname or socket.gethostname(), 80, family, socket.SOCK_STREAM, socket.SOL_TCP)[0][4][0]
File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/pi/Desktop/client.py", line 10, in <module>
message_server.show_message(msg)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 275, in __getattr__
self._pyroGetMetadata()
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 615, in _pyroGetMetadata
self.__pyroCreateConnection()
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 588, in __pyroCreateConnection
uri = _resolve(self._pyroUri, self._pyroHmacKey)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 1915, in _resolve
return nameserver.lookup(uri.object)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 275, in __getattr__
self._pyroGetMetadata()
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 615, in _pyroGetMetadata
self.__pyroCreateConnection()
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 596, in __pyroCreateConnection
connect_and_handshake(conn)
File "/home/pi/.local/lib/python3.7/site-packages/Pyro4/core.py", line 549, in connect_and_handshake
raise ce
Pyro4.errors.CommunicationError: cannot connect to ('JAKOB-PC', 9090): [Errno -2] Name or service not known
My PC has its firewall disabled, so there shouldn t be any problem with that. My local ip is 192.168.1.5.
I am using a headless Raspberry and write code on it with puTTY and VNC.
I have googled this error but couldn t find any answers. Any help would be appreciated.

I did this
#
# Server.py
#
from __future__ import print_function
import Pyro4
#Pyro4.expose
#Pyro4.behavior(instance_mode="single")
class Messenger(object):
# This is a constructor
def __init__(self):
pass
# This method will be called on the server
def send_message(self, name, message):
print("[{0}] {1}".format(name, message))
def main():
Pyro4.Daemon.serveSimple(
{
Messenger: "example.messenger"
},
ns=True)
if __name__ == "__main__":
main()
#
# Client.py
#
# This is the code that visits the warehouse.
import sys
import Pyro4
import Pyro4.util
sys.excepthook = Pyro4.util.excepthook
messenger = Pyro4.Proxy("PYRONAME:example.messenger#192.168.1.5")
messenger.send_message("Tim", "Hello!")
Then ran
python -m Pyro4.naming -n 192.168.1.5
python Server.py
python Client.py

In short I couldn't solve the problem with Pyro and (almost) no one helped so I decided to use 'websockets' instead.
You can read the documentation here but I'll explain it here anyway.
First of all you need two devices with network connection. You also have to run python 3.6.1 on both of them. After that you also need to install websockets if you don't have them already with pip install websockets or as I had to do it with pip3 install websockets.
Code below runs on the server and handles messages you send to it from client. Function 'hello' is a simple example of processing request and sending back a response. 'request' is the data server receives, that data must be bytes, string on iterable. Response is made by converting request to integer, squaring it and converting it back to string. This response is then sent back to client.
'start_server' defines the server, function that will define its behavior(hello), ip address on witch the server is running on(192.168.1.117) and port on witch it will receive requests(8765).
!/usr/bin/env python
import asyncio
import websockets
print("Running...")
async def hello(websocket, path):
request = await websocket.recv()
print("Request: " + request)
response = str(int(request)*int(request))
await websocket.send(response)
print("Response:" + response)
start_server = websockets.serve(hello, "192.168.1.117", 8765)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
Next bit is code on the client. 'uri' is ip and port of the server. Function 'tellServ' asks you to input some data('tell' variable) and sends it to the server. After that it waits for reply and once it gets it it prints it out. In this case if I would input number "6" server would reply with "36". Function loop is in a while loop so I can send multiple numbers without having to restart the script.
#!/usr/bin/env python
import asyncio
import websockets
uri = "ws://192.168.1.117:8765"
async def tellServ():
async with websockets.connect(uri) as websocket:
tell = input("Podatek ki ga posles: ")
await websocket.send(tell)
reply = await websocket.recv()
print("Odgovor:")
print(reply)
while 1:
asyncio.get_event_loop().run_until_complete(tellServ())

Custom exception handling when sending message to remote syslog server in Python

At the moment i am writing a syslog client that will send messages to a remote syslog server. So far this is working pretty ok but i am running into the following problem.
When the syslog server goes down for some reason i need to catch this so the program will stop sending syslog messages and we can investigate the problem.
Unfortunately, the program continues running and doesn't see that the TCP socket is closed and raise an exception.
I only receive a traceback in my terminal:
--- Logging error --- Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37\lib\logging\handlers.py", line 941, in emit
self.socket.sendall(msg) ConnectionAbortedError: [WinError 10053] Call stack:
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\ptvsd_launcher.py", line 45, in <module>
main(ptvsdArgs)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\__main__.py", line 265, in main
wait=args.wait)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\__main__.py", line 258, in handle_args
debug_main(addr, name, kind, *extra, **kwargs)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_local.py", line 45, in debug_main
run_file(address, name, *extra, **kwargs)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_local.py", line 79, in run_file
run(argv, addr, **kwargs)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_local.py", line 140, in _run
_pydevd.main()
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_vendored\pydevd\pydevd.py", line 1925, in main
debugger.connect(host, port)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_vendored\pydevd\pydevd.py", line 1283, in run
return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_vendored\pydevd\pydevd.py", line 1290, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "c:\Users\Administrator\.vscode\extensions\ms-python.python-2018.12.1\pythonFiles\lib\python\ptvsd\_vendored\pydevd\_pydev_imps\_pydev_execfile.py", line 25, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "c:\Users\Administrator\OneDrive\Documents\Python Scripts\testlogger.py", line 71, in <module>
my_logger.info(i) Message: 'test4' Arguments: ()
Relevant code:
my_logger = logging.getLogger('MyLogger')
my_logger.setLevel(logging.ERROR)
my_logger.setLevel(logging.INFO)
my_logger.setLevel(logging.DEBUG)
try:
handler = logging.handlers.SysLogHandler(('IP ADDRESS HOST', 514), socktype=socket.SOCK_STREAM)
my_logger.addHandler(handler)
except Exception as e:
print (e)
list1 = ['test','test2','test3','test4','test5','test6','test7','test8']
for i in list1:
try:
my_logger.info(i) #here i expected that an exception would be raised when the TCP socket is not alive anymore
except Exception as e:
print (e)
How can i make sure that the program stops and i can do the appropriate exception handling?
Thanks!

The default behavior of the SysLogHandler class (and all the ones who are using TCP) is to retry a connection, this is explained in the docs of the createSocket() method:
Tries to create a socket; on failure, uses an exponential back-off
algorithm. On initial failure, the handler will drop the message it
was trying to send. When subsequent messages are handled by the same
instance, it will not try connecting until some time has passed. The
default parameters are such that the initial delay is one second, and
if after that delay the connection still can’t be made, the handler
will double the delay each time up to a maximum of 30 seconds.
This behaviour is controlled by the following handler attributes:
retryStart (initial delay, defaulting to 1.0 seconds).
retryFactor (multiplier, defaulting to 2.0).
retryMax (maximum delay, defaulting to 30.0 seconds).
As it doesn't seem to have an option for the behavior of "not retrying" which you seem to want, so if you really want that you can create your own handler by subclassing it and overriding the createSocket() method with something like:
class MySysLogHandler(logging.handlers.SysLogHandler):
def createSocket(self):
try:
self.sock = self.makeSocket()
except OSError:
# do your own error handling here ...
You can dig a bit more by looking at the source code of createSocket() in CPython Github repo (beware, this is from master branch and might not be the exact version of Python you're using)

I switched to another solution and stopped using the SyslogHandler class.
I now use the following class were i wrote my own syslog sender through a socket.
class Syslog:
def __init__(self,host="localhost",port=514,facility=Facility.DAEMON):
self.host = host
self.port = port
self.facility = facility
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def connect(self):
try:
self.socket.connect((self.host, self.port))
except Exception as e:
print("failed setting up connection")
def send(self, message,level):
data = "<%d>%s" % (level + self.facility*8, message + "\n")
try:
self.socket.sendall(data.encode('utf-8'))
except Exception as e:
print("send failed")
#if __name__ == '__main__':
syslog1 = Syslog(host='HOSTIPADDRESS-NAME')
syslog1.connect()
messages = ["test1","test2","test3","test4","test5","test6","test7"]
for message in messages:
syslog1.send(message,Level.WARNING)
This is working quite well and runs into an exception when the syslog server goes down unexpectedly. The only problem now i discovered while debugging is the following:
When i shut down the syslog server it throws not immediatly an exception when i try to send a message.
Please see example below:
1.) the syslog server is started, i send the first message "test1" from the for loop, successfull.
2.) i shutdown the syslog server, now i send the second message "test2" from the for loop. Nothing happens, no exception!
3.) i send the third message "test3", now an exception is thrown.
How is this possible?

Tornado how to return error exception?

I want to run a method I know this method doesn't work and I want to get the error returned by the method.
This is my code :
def is_connect(s):
print("ok connection")
print(s)
ioloop.stop()
try:
current_job_ready = 0
print("ok1")
beanstalk = beanstalkt.Client(host='host', port=port)
print("ok1")
beanstalk.connect(callback=is_connect)
ioloop = tornado.ioloop.IOLoop.instance()
ioloop.start()
print("ok2")
except IOError as e:
print(e)
And this is the error I have when I run my program with wring port :
WARNING:tornado.general:Connect error on fd 7: ECONNREFUSED
ERROR:tornado.application:Exception in callback <functools.partial object at 0x7f5a0eac6f18>
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/ioloop.py", line 604, in _run_callback
ret = callback()
File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 275, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tornado/ioloop.py", line 619, in <lambda>
self.add_future(ret, lambda f: f.result())
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 237, in result
raise_exc_info(self._exc_info)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 270, in wrapper
result = func(*args, **kwargs)
TypeError: connect() takes exactly 1 argument (2 given)
I want to have e when I enter a false port or host.
How can I do this?
I tired to add raise IOError("connection error") after beanstalk = beanstalkt.Client(host='host', port=port) But this force the error, and I just want to have error when it exist.

Here's where reading the code helps. In beanstalkt 0.6's connect, it creates an IOStream to connect to the server:
https://github.com/nephics/beanstalkt/blob/v0.6.0/beanstalkt/beanstalkt.py#L108
It registers your callback to be executed on success, but if the connection fails it'll just call Client._reconnect once per second forever. I think you should open a feature request in their GitHub project asking for an error-notification system for connect. With the current beanstalkt implementation, you just have to decide how long you're willing to wait for success:
import sys
from datetime import timedelta
from tornado.ioloop import IOLoop
def is_connect(s):
print("ok connection")
print(s)
loop.remove_timeout(timeout)
# Do something with Beanstalkd....
def connection_failed():
print(sys.stderr, "Connection failed!")
# Could call IOLoop.stop() or just quit.
sys.exit(1)
loop = IOLoop.current()
timeout = loop.add_timeout(timedelta(seconds=1), connection_failed)
beanstalk.connect(callback=is_connect)
loop.start()

TypeError: context must be specified" in zmq unpickling

I am trying to create a very simple chat. Using the PyZMQ library. And since sockets are not threadsafe I am using two sockets and one thread running on each. One checking for incoming messages, and one to send messages.
But my program is giving me, error messages:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
self = load(from_parent)
File "C:\Python27\lib\pickle.py", line 1384, in load
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Python27\lib\multiprocessing\forking.py", line 381, in main
self = load(from_parent)
File "C:\Python27\lib\pickle.py", line 1384, in load
return Unpickler(file).load()
File "C:\Python27\lib\pickle.py", line 864, in load
return Unpickler(file).load()
File "C:\Python27\lib\pickle.py", line 864, in load
dispatch[key](self)
File "C:\Python27\lib\pickle.py", line 1089, in load_newobj
dispatch[key](self)
File "C:\Python27\lib\pickle.py", line 1089, in load_newobj
obj = cls.__new__(cls, *args)
File "zmq\backend\cython\socket.pyx", line 279, in zmq.backend.cython.socket.Socket.__cinit__ (zmq\backend\cython\socket.c:3456)
TypeError: context must be specified
obj = cls.__new__(cls, *args)
File "zmq\backend\cython\socket.pyx", line 279, in zmq.backend.cython.socket.Socket.__cinit__ (zmq\backend\cython\socket.c:3456)
TypeError: context must be specified
I can not figure out why I am getting them, or how to solve it.
Also is my logic wrong here?:
We start the server which creates a socket and binds it to a port. Then It listens to that sockets for messages/connections. Then we create another socket and binds it to the same port. We create a new thread that makes it so that we wait for messages to be recieved on the first socket to then be sent to the second one.
Then we have a client who connects to the first socket. We create a new thread for it so it can listen on the other socket for incoming messages, and thus it can use the first thread to send messages through the first socket.
server.py
from communication import Communication
from multiprocessing import Process
import zmq
import random
import sys
import time
if __name__ == '__main__':
if len(sys.argv) > 1:
port = sys.argv[1]
else:
port = "5556"
c = Communication(port)
c.bind()
recieverP = Process(target=c.reciever)
recieverP.start()
print("first process")
c2 = Communication(port)
c2.connect()
senderP = Process(target=c2.sender)
senderP.start()
print("second process")
client.py
from communication import Communication
from multiprocessing import Process
import zmq
import random
import sys
import time
if __name__ == '__main__':
if len(sys.argv) > 1:
port = sys.argv[1]
else:
port = "5556"
c = Communication(port)
c.connect()
recieverP = Process(target=c.reciever, args=())
senderP = Process(target=c.sender,args=())
recieverP.start()
senderP.start()
communications.py
import zmq
class Communication:
def __init__(self, port):
context = zmq.Context.instance()
self.port = port
self.socket = context.socket(zmq.PAIR)
def connect(self):
self.socket.connect("tcp://localhost:%s" % self.port)
def bind(self):
self.socket.bind("tcp://*:%s" % self.port)
def sender(self):
while True:
msg = raw_input("> ")
self.socket.send(msg)
def reciever(self):
while True:
msg = self.socket.recv()
print("> " + msg)

Fully working version below. I made a few changes:
I used threads instead of processes, as suggested by #Dunes.
I combined client.py and server.py into a single app.py with a "role" parameter. (This was just to avoid a lot of duplicate code.)
I handled KeyboardExceptions in the main thread so you can Ctrl+C to exit.
I dropped the "> " prefix on lines, since it led to confusing output. (When you type something in the other app, you'd get output like "> > hello".)
I took out the c2 that was in server.py. I'm not sure what the purpose of that was, and I don't know if that's expected to work. (Seems like you were connecting instead of binding?)
I fixed the spelling of "receiver" everywhere because it was bothering me. :-)
I made port an int instead of a str, because it is. :-)
app.py:
import argparse
from threading import Thread
import time
from communication import Communication
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument("role", help="either 'client' or 'server'", choices=['client', 'server'])
parser.add_argument("--port", "-p", type=int, help="port number", default=5556)
args = parser.parse_args()
c = Communication(args.port)
if args.role == 'client':
c.connect()
else:
c.bind()
receiverP = Thread(target=c.receiver)
senderP = Thread(target=c.sender)
receiverP.daemon = True
senderP.daemon = True
try:
receiverP.start()
senderP.start()
while True:
time.sleep(100)
except (KeyboardInterrupt, SystemExit):
pass
communication.py:
import zmq
class Communication:
def __init__(self, port):
self.port = port
context = zmq.Context.instance()
self.socket = context.socket(zmq.PAIR)
def connect(self):
self.socket.connect("tcp://localhost:%d" % self.port)
def bind(self):
self.socket.bind("tcp://*:%d" % self.port)
def sender(self):
while True:
self.socket.send(raw_input())
def receiver(self):
while True:
print(self.socket.recv())

These are processes not threads. The problem occurs because in the background Python has to send a copy of your Communication object to the child process. However, your object contains socket objects that cannot be serialised. Use threading and the Thread object in place of Process and you won't run into this problem. This is because threads run in the same process.

build a simple remote dispatcher using multiprocessing.Managers

Consider the following code :
Server :
import sys
from multiprocessing.managers import BaseManager, BaseProxy, Process
def baz(aa) :
l = []
for i in range(3) :
l.append(aa)
return l
class SolverManager(BaseManager): pass
class MyProxy(BaseProxy): pass
manager = SolverManager(address=('127.0.0.1', 50000), authkey='mpm')
manager.register('solver', callable=baz, proxytype=MyProxy)
def serve_forever(server):
try :
server.serve_forever()
except KeyboardInterrupt:
pass
def runpool(n):
server = manager.get_server()
workers = []
for i in range(int(n)):
Process(target=serve_forever, args=(server,)).start()
if __name__ == '__main__':
runpool(sys.argv[1])
Client :
import sys
from multiprocessing.managers import BaseManager, BaseProxy
import multiprocessing, logging
class SolverManager(BaseManager): pass
class MyProxy(BaseProxy): pass
def main(args) :
SolverManager.register('solver')
m = SolverManager(address=('127.0.0.1', 50000), authkey='mpm')
m.connect()
print m.solver(args[1])._getvalue()
if __name__ == '__main__':
sys.exit(main(sys.argv))
If I run the server using only one process as python server.py 1
then the client works as expected. But if I spawn two processes (python server.py 2) listening for connections, I get a nasty error :
$python client.py ping
Traceback (most recent call last):
File "client.py", line 24, in <module>
sys.exit(main(sys.argv))
File "client.py", line 21, in main
print m.solver(args[1])._getvalue()
File "/usr/lib/python2.6/multiprocessing/managers.py", line 637, in temp
authkey=self._authkey, exposed=exp
File "/usr/lib/python2.6/multiprocessing/managers.py", line 894, in AutoProxy
incref=incref)
File "/usr/lib/python2.6/multiprocessing/managers.py", line 700, in __init__
self._incref()
File "/usr/lib/python2.6/multiprocessing/managers.py", line 750, in _incref
dispatch(conn, None, 'incref', (self._id,))
File "/usr/lib/python2.6/multiprocessing/managers.py", line 79, in dispatch
raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.6/multiprocessing/managers.py", line 181, in handle_request
result = func(c, *args, **kwds)
File "/usr/lib/python2.6/multiprocessing/managers.py", line 402, in incref
self.id_to_refcount[ident] += 1
KeyError: '7fb51084c518'
---------------------------------------------------------------------------
My idea is pretty simple. I want to create a server that will spawn a number of workers that will share the same socket and handle requests independently. Maybe I'm using the wrong tool here ?
The goal is to build a 3-tier structure where all requests are handled via an http server and then dispatched to nodes sitting in a cluster and from nodes to workers via the multiprocessing managers...
There is one public server, one node per machine and x number of workers on each machine depending on the number of cores... I know I can use a more sophisticated library, but for such a simple task (I'm just prototyping here) I would just use the multiprocessing library... Is this possible or I should explore directly other solutions ? I feel I'm very close to have something working here ... thanks.

You're trying to invent a wheel, many have invented before.
It sounds to me that you're looking for task queue where your server dispatches tasks to, and your workers execute this tasks.
I would recommend you to have a look at Celery.