We Keep Coding

Async pymodbus application doesn't work after reading from "IllegalAddress"

Application: Python 2.7 + Pymodbus 1.4.0 (asynchronous client)
Problem: I want to handle a situation when user wants to read from the wrong address (Illegal Address) in asynchronous version, but my solution does not work.
Simplified code - asynchronous version:
#!/usr/bin/env python
from __future__ import print_function
from twisted.internet import reactor, protocol
from pymodbus.client.async import ModbusClientProtocol
from twisted.internet import serialport, reactor
def print_result(response, address):
try:
print("Response: ", response.registers[0])
except:
print("Response: ", response)
def start_measurements(client):
# 16408, 16409 addresses are ok
# 9015 is an Illegal Address
addresses = [16408, 9015, 16409]
for address in addresses:
rr = client.read_holding_registers(address, 1, unit=255)
# setting Callback/Errback function with an address argument
rr.addBoth(print_result, address)
reactor.callLater(10, client.transport.loseConnection)
reactor.callLater(11, reactor.stop)
def handler_tcp_connection_error(client):
print("Error while connecting")
reactor.callLater(0, reactor.stop)
if __name__ == "__main__":
defer = protocol.ClientCreator(reactor, ModbusClientProtocol).connectTCP("192.168.168.100", 502)
defer.addCallbacks(start_measurements, handler_tcp_connection_error)
reactor.run()
When I read from these registers in synchronous version I just don't receive a value from wrong address, but further registers are read anyway.
In asynchronous version (pasted above) when Illegal Address is read, I've got no response even from further registers and the output is:
Response: 15
Response: Exception Response(131, 3, IllegalAddress)
*waiting about 10s with no response*
Response: [Failure instance: Traceback (failure with no frames): <class 'pymodbus.exceptions.ConnectionException'>: Modbus Error: [Connection] Connection lost during request]
I'd like to get all correct values (not from Illegal Addresses) like in the synchronous version:
Response: 15
Exception Response(131, 3, IllegalAddress)
Response 7
How can I fix this, to skip IllegalAddreses and get values from all correct addreses?

Send commands to twisted server

I have a simple twisted TCP server that I am using to send and receive data from another application. I am able to communicate and send messages to this application once a connection has been established.
What I want to be able to do have an external module command when and what to send to the 3rd party application (instead of when connected). For example, here is my server code:
from twisted.internet import reactor, protocol
# Twisted
class Tcp(protocol.Protocol):
def dataReceived(self, data):
self.transport.write(data)
print 'data received: ', data
# do something with data
def connectionMade(self, msg):
# Want to only transport message when I command not immediately when connected
self.transport.write(msg)
class TcpFactory(protocol.Factory):
def buildProtocol(self, addr):
return Tcp()
reactor.listenTCP(8001, TcpFactory())
reactor.run()
I have another module (operated by a GUI interface) which defines what message to send and this message needs to be passed as 'msg' into connectionMade
What is the best way to do this?

Your 3rd party call will happen from within connectionMade and should not call it directly. The connectionMade is a Twisted callback and it's there to signal that a client is on the other side. It's up to you to dictate what happens after that. So you can do something like make a series of Deferreds/callbacks which take self.transport (this holds relevant client data). In those series of callbacks, you'll have your logic for
communicating with the 3rd party
validating client
sending a message to the client
This is the gist of it:
def connectionMade(self):
d = Deferred()
d.addCallback(self.thirdPartyComm).addErrBack(self.handleError)
d.addCallback(sendMsg)
d.callback(self.transport)
def thirdPartyComm(self, transport):
# logic for talking to 3rd party goes here
# 3rd party should validate the client then provide a msg
# or raise exception if client is invalid
return msg
def handleError(self, error):
# something went wrong, kill client connection
self.transport.loseConnection()
def sendMsg(self, msg):
self.transport.write(msg)

How can I transfer binary data over xmlrpc (python)?

As the name xmlrpc implies, this transfer protocol relies on XML to carry data, and cannot transfer binary data, or non-printable ASCII-characters (\n, \b, chr(2),...) [or can it?].
I would like to know if there is a way to transfer a character string safely from a client to a server with minimal impact on the coding (i.e. ONLY on the client side). I tried the xmlrpclib.Binary class but this only seem to work with files.
Testcode, server.py:
def output(text):
print "-".join([str(ord(x)) for x in text])
from SimpleXMLRPCServer import SimpleXMLRPCServer
server = SimpleXMLRPCServer(('localhost', 1234))
server.register_function(output)
server.serve_forever()
client.py:
import xmlrpclib
device = xmlrpclib.ServerProxy("http://localhost:1234/RPC2")
device.output(".\n."+chr(2))
Expected outcome:
46-10-46-2
Seen outcome (on server side):
xmlrpclib.Fault: <Fault 1: "<class 'xml.parsers.expat.ExpatError'>:not well-formed (invalid token): line 7, column 1">

I think the expected answer was using xml-rpc base64 type. In python, on client side, you have to manually specify that a string contains binary data, using the xmlrpclib.Binary type.
import xmlrpclib
device = xmlrpclib.ServerProxy("http://localhost:1234/RPC2")
device.output(xmlrpclib.Binary(".\n."+chr(2)))

You could try encoding your binary data in a text format in the client and decoding it back into binary in the server. One encoding you could use is base64.
In your client:
import xmlrpclib
import base64
device = xmlrpclib.ServerProxy("http://localhost:1234/RPC2")
device.output(base64.b64encode(".\n."+chr(2)))
In your server:
import base64
def output(text):
print "-".join([str(ord(x)) for x in base64.b64decode(text)])
from SimpleXMLRPCServer import SimpleXMLRPCServer
server = SimpleXMLRPCServer(('localhost', 1234))
server.register_function(output)
server.serve_forever()

Python twisted: A server that writes to the connection after an asynchronous operation?

I have a server where I have implemented a child of the NetstringReceiver protocol. I want it to perform an asynchronous operation (using txredisapi) based on the client's request and then respond with the results of the operation. A generalization of my code:
class MyProtocol(NetstringReceiver):
def stringReceived(self, request):
d = async_function_that_returns_deferred(request)
d.addCallback(self.respond)
# self.sendString(myString)
def respond(self, result_of_async_function):
self.sendString(result_of_async_function)
In the above code, the client connecting to my server does not get a response. However, it does get myString if I uncomment
# self.sendString(myString)
I also know that result_of_async_function is a non-empty string because I print it to stdout .
What can I do that will allow me to respond to the client with the result of the asynchronous function?
Update: Runnable source code
from twisted.internet import reactor, defer, protocol
from twisted.protocols.basic import NetstringReceiver
from twisted.internet.task import deferLater
def f():
return "RESPONSE"
class MyProtocol(NetstringReceiver):
def stringReceived(self, _):
d = deferLater(reactor, 5, f)
d.addCallback(self.reply)
# self.sendString(str(f())) # Note that this DOES send the string.
def reply(self, response):
self.sendString(str(response)) # Why does this not send the string and how to fix?
class MyFactory(protocol.ServerFactory):
protocol = MyProtocol
def main():
factory = MyFactory()
from twisted.internet import reactor
port = reactor.listenTCP(8888, factory, )
print 'Serving on %s' % port.getHost()
reactor.run()
if __name__ == "__main__":
main()

There's one specific feature about NetstringReceiver:
The connection is lost if an illegal message is received
Are you sure that your messages conform djb's Netstring protocol?
Obviously the client sends illegal string that could not be parsed, and connection is lost by protocol conditions. Everything else looks good in your code.
If you don't need that specific protcol, you'd better inherit LineReceiver instead of NetstringReceiver.

The reason you never get the response is because by the time it's sent the connection is closed. The reason the connection is closed is because the message you send with 'nc' is:
1:a,\n
Because you have to type a newline to get nc to send the message, but nc includes it as part of the message. That violates the NetString protocol...
I worked around it (with your code modified with some additional prints) by sending this message instead:
1:a,40:\n
blahblahblahDon't hit return here, just wait for the reply
8:RESPONSE,

Twisted client for a send only protocol that is tolerant of disconnects

I've decided to dip my toe into the world of asynchronous python with the help of twisted. I've implemented some of the examples from the documentation, but I'm having a difficult time finding an example of the, very simple, client I'm trying to write.
In short I'd like a client which establishes a tcp connection with a server and then sends simple "\n" terminated string messages off of a queue object to the server. The server doesn't ever respond with any messages so my client is fully unidirectional. I /think/ that what I want is some combination of this example and the twisted.internet.protocols.basic.LineReceiver convenience protocol. This feels like it should be just about the simplest thing one could do in twisted, but none of the documentation or examples I've seen online seem to fit quite right.

What I have done is not used a Queue but I am illustrating the code that sends a line, once a connection is made. There are bunch of print stuff that will help you understand on what is going on.
Usual import stuff:
from twisted.web import proxy
from twisted.internet import reactor
from twisted.internet import protocol
from twisted.internet.protocol import ReconnectingClientFactory
from twisted.protocols import basic
from twisted.python import log
import sys
log.startLogging(sys.stdout)
You create a protocol derived from line receiver, set the delimiter.
In this case, I simply write a string "www" once the connection is made.
The key thing is to look at protocol interface at twisted.internet.interface.py and understand the various methods of protocol and what they do and when they are called.
class MyProtocol(basic.LineReceiver):
#def makeConnection(self, transport):
# print transport
def connectionLost(self, reason):
print reason
self.sendData = False
def connectionMade(self):
print "connection made"
self.delimiter = "\n"
self.sendData = True
print self.transport
self.sendFromQueue()
def sendFromQueue(self):
while self.sendData:
msg = dataQueue.get()
self.sendLine(msg)
# you need to handle empty queue
# Have another function to resume
Finally, A protocol factory that will create a protocol instance for every connection.
Look at method : buildProtcol.
class myProtocolFactory():
protocol = MyProtocol
def doStart(self):
pass
def startedConnecting(self, connectorInstance):
print connectorInstance
def buildProtocol(self, address):
print address
return self.protocol()
def clientConnectionLost(self, connection, reason):
print reason
print connection
def clientConnectionFailed(self, connection, reason):
print connection
print reason
def doStop(self):
pass
Now you use a connector to make a connection:
reactor.connectTCP('localhost', 50000, myProtocolFactory())
reactor.run()
I ran this and connected it to an server that simply prints what it receives and hence send no ack back. Here is the output:
1286906080.08 82 INFO 140735087148064 __main__ conn_made: client_address=127.0.0.1:50277
1286906080.08 83 DEBUG 140735087148064 __main__ created handler; waiting for loop
1286906080.08 83 DEBUG 140735087148064 __main__ handle_read
1286906080.08 83 DEBUG 140735087148064 __main__ after recv
'www\n'
Recieved: 4
The above example if not fault tolerant. To reconnect , when a connection is lost, you can derive your protocol factory from an existing twisted class - ReconnectingClientFactory.
Twisted has almost all the tools that you would need :)
class myProtocolFactory(ReconnectingClientFactory):
protocol = MyProtocol
def buildProtocol(self, address):
print address
return self.protocol()
For further reference
I suggest that you read : http://krondo.com/?page_id=1327
[Edited: As per comment below]