We Keep Coding

python3.5: asyncio, How to wait for "transport.write(data)" to finish or to return an error?

I'm writing a tcp client in python3.5 using asyncio
After reading How to detect write failure in asyncio? that talk about the high-level streaming api, I've tried to implement using the low level protocol api.
class _ClientProtocol(asyncio.Protocol):
def connection_made(self, transport):
self.transport = transport
class Client:
def __init__(self, loop=None):
self.protocol = _ClientProtocol()
if loop is None:
loop = asyncio.get_event_loop()
self.loop = loop
loop.run_until_complete(self._connect())
async def _connect(self):
await self.loop.create_connection(
lambda: self.protocol,
'127.0.0.1',
8080,
)
# based on https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/#bug-3-closing-time
self.protocol.transport.set_write_buffer_limits(0)
def write(self, data):
self.protocol.transport.write(data)
def wait_all_data_have_been_written_or_throw():
pass
client = Client()
client.write(b"some bytes")
client.wait_all_data_have_been_written_or_throw()
As per the python documentation, I know write is non-blocking, and I would like the wait_all_data_have_been_written_or_throw to tell me if all data have been written or if something bad happened in the middle (like a connection lost, but I assume there's way more things that can go bad, and that the underlying socket already return exception about it?)
Does the standard library provide a way to do so ?

The question is mainly related to TCP sockets functionality, not asyncio implementation itself.
Let's look on the following code:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
s.send(b'data')
Successful send() call means the data was transferred into kernel space buffer for the socket, nothing more.
Data was not sent via wire, not received by peer and, obviously, not processed by received.
Actual sending is performed asynchronously by Operation System Kernel, user code has no control over it.
What's why wait_all_data_have_been_written_or_throw() make not much sense: writing without an error doesn't assume receiving these data by peer but only successful moving from user-space buffer to kernel-space one.

Cant receive data from socket

I'm making a client-server program, and there is problem with client part.
Problem is in infinite receiving data. I've tested this particular class, listed below, in a python interpreter. I've succesfuly(maybe not) connected to google, but then program stoped in function recvData() in data = self.socket.recv(1024)
class client():
def __init__(self, host, port):
self.host = host
self.port = port
self.socket = self.connect()
self.command = commands()
def connect(self):
'''
Connect to a remote host.
'''
try:
import socket
return socket.create_connection((self.host, self.port))
except socket.error:
print(":: Failed to connect to a remote port : ")
def sendCommand(self, comm):
'''
Send command to remote host
Returns server output
'''
comman = comm.encode()
# for case in switch(comman):
# if case(self.command.RETRV_FILES_LIST.encode()):
# self.socket.send(b'1')
# return self.recvData()
# if case():
# print(":: Got wrong command")
if (comman == b'1'):
self.socket.send(b'1')
return self.recvData()
def recvData(self):
'''
Receives all the data
'''
i = 0
total_data = []
while(True):
data = self.socket.recv(1024)
if not data: break
total_data.append(data)
i += 1
if i > 9:
break
return total_data
about commented part :
I thought problem in Case realization, so used just if-then statement. But it's not.

Your problem is that self.socket.recv(1024) only returns an empty string when the socket has been shut down on the server side and all data has been received. The way you coded your client, it has no idea that the full message has been received and waits for more. How you deal with the problem depends very much on the protocol used by the server.
Consider a web server. It sends a line-delimited header including a content-length parameter telling the client exactly how many bytes it should read. The client scans for newlines until the header is complete and then uses that value to do recv(exact_size) (if large, it can read chunks instead) so that the recv won't block when the last byte comes in.
Even then, there a decisions to make. The client knows how large the web page is but may want to send a partial data to the caller so it can start painting the page before all the data is received. Of course, the caller needs to know that is what happens - there is a protocol or set of rules for the API itself.
You need to define how the client knows a message is complete and what exactly it passes back to its caller. A great way to deal with the problem is to let some other protocol such as [zeromq](http://zeromq.org/ do the work for you. A simple python client / server can be implemented with xmlrpc. And there are many other ways.
You said you are implementing a client/server program then you mentioned "connected to google" and telnet... These are all very different things and a single client strategy won't work with all of them.

Twisted Python script on Raspberry Pi (Debian) to communicate with Arduino via USB

I have been working on an Arduino/Raspberry Pi project where I have found myself learning not just Python but Twisted Python as well; so I apologize in advance for my newbness. I am trying to keep it simple for now and just trying to send a char at any one time between the two devices.
So far I am able to send from the Raspberry Pi to the Arduino and effectively turn its LED off/on just as expected. However I cannot seem to generate Twisted code which will detect anything coming from the Arduino to the RPi on the serial port. I verified that the Arduino is sending chars every 2 seconds with a serial monitor application in the Arduino programmer running on the RPi.
The code below runs on the RPi, receives a GET request and passes some of that data through the serial port to the Arduino. I cannot seem to get this code to listen to that same serial port though. :/ I have been working on this for a bit over a month and seem to be stuck. I just cannot seem to find a good example for Twisted Python online to receive serial data; or at least an example that I understand. Anyway here is what I have so far:
import sys
from urlparse import urlparse
from twisted.web import server, resource
from twisted.internet import reactor
from twisted.internet.protocol import Factory, Protocol
from twisted.internet.serialport import SerialPort
serServ = None
class USBclient(Protocol):
def connectionMade(self):
global serServ
serServ = self
print 'Arduino device: ', serServ, ' is connected.'
def cmdReceived(self, cmd):
serServ.transport.write(cmd)
print cmd, ' - sent to Arduino.'
pass
def serialReadEvent(self): #maybe it should be: doRead()? Couldn't get either to work.
print 'data from arduino is at the serial port!'
class HTTPserver(resource.Resource):
isLeaf = True
def render_GET(self, request): #passes the data from the get request
print 'HTTP request received'
myArduino = USBclient()
stringit = str(request)
parse = stringit.split()
command, path, version = parse
myArduino.cmdReceived(path)
class cmdTransport(Protocol):
def __init__(self, factory):
self.factory = factory
class cmdTransportFactory(Factory):
protocol = cmdTransport
if __name__ == '__main__':
HTTPsetup = server.Site(HTTPserver())
reactor.listenTCP(5000, HTTPsetup)
SerialPort(USBclient(), '/dev/ttyACM0', reactor, baudrate='115200')
reactor.run()
As you can see the code is just looking for anything on the serial port but I can't seem to make that magic happen. Thanks in advance, any help is appreciated!

Judging from this: http://twistedmatrix.com/trac/browser/tags/releases/twisted-12.3.0/twisted/internet/_win32serialport.py#L84 you should be looking at dataReceived(self,...) method of your Protocol subclass
Thus:
class USBclient(Protocol):
def connectionMade(self):
global serServ
serServ = self
print 'Arduino device: ', serServ, ' is connected.'
def cmdReceived(self, cmd):
serServ.transport.write(cmd)
print cmd, ' - sent to Arduino.'
pass
def dataReceived(self,data):
print 'USBclient.dataReceived called with:'
print str(data)
try this to see if it works.

Network bridge using Scapy and Python

I am creating a network bridge that connects two ethernet cards on the same machine. One of the cards is connected to the LAN and the other is connected to a network device. It looks something like this,
I am sniffing packets on both the interfaces and then sending them to the other using sendp(x,iface='eth0') for a packet that I sniffed on eth1 and vice versa.
I verified the packets at both the interfaces and found them to be correct, but somehow I am unable to get an IP for the device.
Below is a piece of my code, I create two threads, one for each interface:
from scapy.all import*
**THREAD1:**
pkt=sniff(iface="eth0",store=1,count=1)
outbuff=[]
outbuff+=pkt[:]
for src in outbuff[:]
srcmac=src.sprintf(r"%Ether.src%")
if srcmac==deviceMAC:
pass
else:
sendp(self.outbuff[:],iface="eth1",verbose=0)
**THREAD2:**
pkt=sniff(iface="eth1",store=1,count=1)
outbuff=[]
outbuff+=pkt[:]
for src in outbuff[:]
srcmac=src.sprintf(r"%Ether.src%")
if srcmac==deviceMAC:
sendp(self.outbuff[:],iface="eth1",verbose=0)
else:
pass
Can some one help me with the problem or suggest me an alternative solution for this implementation?
SOLVED: Combining Python+IPTABLES and using the principles of TRIGGER solves this problem.

Posting a snippet of the bridging class
from threading import Thread
import threading
import socket
import thread
class iface0(threading.Thread):
def __init__(self, MAC):
Thread.__init__(self)
pass
def run(self):
self.a = socket.gethostbyname_ex(socket.gethostname())[2]
while 1:
self.sSock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
self.sSock.bind((self.a[1],23432))
self.iface0_sniff()
self.sSock.close()
def iface0_sniff(self):
self.sSock.sendto("THISISATESTWORLD",(self.a[1],78456))
data = ''
class iface1(threading.Thread):
def __init__(self,MAC):
Thread.__init__(self)
pass
def run(self):
self.a=socket.gethostbyname_ex(socket.gethostname())[2]
while 1:
self.sSock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
self.iface1_sniff()
self.sSock.close()
def iface1_sniff(self):
self.sSock.sendto("THISISATESTWORLD",(self.a[1],98658))
data = ''
if __name__ == '__main__':
MAC = ['XX:XX:XX:XX:XX:XX']
iface0 = iface0(MAC)
iface1 = iface1(MAC)
iface1.start()
iface0.start()

How to implement a hub in Python

Dear all, I need to implement a TCP server in Python which receives some data from a client and then sends this data to another client. I've tried many different implementations but no way to make it run. Any help would be really appreciated.
Below is my code:
import SocketServer
import sys
import threading
buffer_size = 8182
ports = {'toserver': int(sys.argv[1]), 'fromserver': int(sys.argv[2])}
class ConnectionHandler(SocketServer.BaseRequestHandler):
def handle(self):
# I need to send the data received from the client connected to port 'toserver'
# to the client connected to port 'fromserver' - see variable 'ports' above
class TwoWayConnectionServer(threading.Thread):
def __init__(self):
self.to_server = SocketServer.ThreadingTCPServer(("", ports['toserver']), ConnectionHandler)
self.from_server = SocketServer.ThreadingTCPServer(("", ports['fromserver']), ConnectionHandler)
threading.Thread.__init__(self)
def run(self):
while (1):
self.to_server.handle_request()
self.from_server.handle_request()
def serve_non_blocking():
server = TwoWayConnectionServer()
server.run()
if __name__ == '__main__':
serve_non_blocking()

See the Twisted tutorial, and also twisted.protocols.portforward. I think that portforward module does something slightly different from what you want, it opens an outgoing connection to the destination port rather than waiting for the second client to connect, but you should be able to work from there.

Can you be more specific about what you tried and what didn't work? There are lots of ways to do this. Probably the easiest would be to use the socket library - maybe looking at some examples will help:
http://docs.python.org/library/socket.html#example