This is my code
`
class TestProtocol(Protocol):
def connectionMade(self):
print 'Got connection from', self.transport.client
def dataReceived(self, data):
print(data)
class Test(Factory, HoneypotService):
NAME = 'test'
protocol = TestProtocol
def __init__(self, config=None, logger=None):
HoneypotService.__init__(self, config=config, logger=logger)
self.port = 8888
def getService(self):
return internet.TCPServer(self.port, self)
When I use the telnet 127.0.0.1 8888 command to connect to this service, and input abcdef,Method dataReceived is executed twice,execute result is as follows:
output picture:
enter image description here
Is there any way to execute dataReceived only once, And the output information is abcdef.
I am searching for a long time on net. But no use. Please help or try to give some ideas how to achieve this.
The output information is abcdef.
As explained in the Twisted FAQ:
TCP is a stream-oriented transport. This means that when you call transport.write, the data may be broken up into arbitrarily-sized chunks for transmission over the network. There is no way for Twisted to determine how large the data originally written to the transport was.
If you want to send a message and receive it whole on the other end of a connection, you must decide on a format for the message and parse it. For example, prefixing the message with a length or terminating it with a message boundary.
https://github.com/twisted/trac-wiki-archive/blob/trunk/FrequentlyAskedQuestions.mediawiki#why-is-protocoldatareceived-called-with-only-part-of-the-data-i-called-transportwrite-with
Related
I'm writing a tcp client in python3.5 using asyncio
After reading How to detect write failure in asyncio? that talk about the high-level streaming api, I've tried to implement using the low level protocol api.
class _ClientProtocol(asyncio.Protocol):
def connection_made(self, transport):
self.transport = transport
class Client:
def __init__(self, loop=None):
self.protocol = _ClientProtocol()
if loop is None:
loop = asyncio.get_event_loop()
self.loop = loop
loop.run_until_complete(self._connect())
async def _connect(self):
await self.loop.create_connection(
lambda: self.protocol,
'127.0.0.1',
8080,
)
# based on https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/#bug-3-closing-time
self.protocol.transport.set_write_buffer_limits(0)
def write(self, data):
self.protocol.transport.write(data)
def wait_all_data_have_been_written_or_throw():
pass
client = Client()
client.write(b"some bytes")
client.wait_all_data_have_been_written_or_throw()
As per the python documentation, I know write is non-blocking, and I would like the wait_all_data_have_been_written_or_throw to tell me if all data have been written or if something bad happened in the middle (like a connection lost, but I assume there's way more things that can go bad, and that the underlying socket already return exception about it?)
Does the standard library provide a way to do so ?
The question is mainly related to TCP sockets functionality, not asyncio implementation itself.
Let's look on the following code:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
s.send(b'data')
Successful send() call means the data was transferred into kernel space buffer for the socket, nothing more.
Data was not sent via wire, not received by peer and, obviously, not processed by received.
Actual sending is performed asynchronously by Operation System Kernel, user code has no control over it.
What's why wait_all_data_have_been_written_or_throw() make not much sense: writing without an error doesn't assume receiving these data by peer but only successful moving from user-space buffer to kernel-space one.
I'm making a client-server program, and there is problem with client part.
Problem is in infinite receiving data. I've tested this particular class, listed below, in a python interpreter. I've succesfuly(maybe not) connected to google, but then program stoped in function recvData() in data = self.socket.recv(1024)
class client():
def __init__(self, host, port):
self.host = host
self.port = port
self.socket = self.connect()
self.command = commands()
def connect(self):
'''
Connect to a remote host.
'''
try:
import socket
return socket.create_connection((self.host, self.port))
except socket.error:
print(":: Failed to connect to a remote port : ")
def sendCommand(self, comm):
'''
Send command to remote host
Returns server output
'''
comman = comm.encode()
# for case in switch(comman):
# if case(self.command.RETRV_FILES_LIST.encode()):
# self.socket.send(b'1')
# return self.recvData()
# if case():
# print(":: Got wrong command")
if (comman == b'1'):
self.socket.send(b'1')
return self.recvData()
def recvData(self):
'''
Receives all the data
'''
i = 0
total_data = []
while(True):
data = self.socket.recv(1024)
if not data: break
total_data.append(data)
i += 1
if i > 9:
break
return total_data
about commented part :
I thought problem in Case realization, so used just if-then statement. But it's not.
Your problem is that self.socket.recv(1024) only returns an empty string when the socket has been shut down on the server side and all data has been received. The way you coded your client, it has no idea that the full message has been received and waits for more. How you deal with the problem depends very much on the protocol used by the server.
Consider a web server. It sends a line-delimited header including a content-length parameter telling the client exactly how many bytes it should read. The client scans for newlines until the header is complete and then uses that value to do recv(exact_size) (if large, it can read chunks instead) so that the recv won't block when the last byte comes in.
Even then, there a decisions to make. The client knows how large the web page is but may want to send a partial data to the caller so it can start painting the page before all the data is received. Of course, the caller needs to know that is what happens - there is a protocol or set of rules for the API itself.
You need to define how the client knows a message is complete and what exactly it passes back to its caller. A great way to deal with the problem is to let some other protocol such as [zeromq](http://zeromq.org/ do the work for you. A simple python client / server can be implemented with xmlrpc. And there are many other ways.
You said you are implementing a client/server program then you mentioned "connected to google" and telnet... These are all very different things and a single client strategy won't work with all of them.
I have to pass a data from my test cases to a mock server.
What is the best way to do that ?
This is what I have so far
mock_server.py
class ThreadedUDPServer(SocketServer.ThreadingMixIn, SocketServer.UDPServer):
pass
class ThreadedUDPRequestHandler(SocketServer.BaseRequestHandler):
def __init__(self, request, client_address, server):
SocketServer.BaseRequestHandler.__init__(self,request,client_address,server)
def handle(self):
print server.data #this is where i need the data
class server_wrap:
def __init__(self):
self.server = ThreadedUDPServer( ("127.0.0.1",49555) , ThreadedUDPRequestHandler)
def set_data(self,data)
self.server.data = data
def start(self)
server_thread = threading.Thread(target=self.server.serve_forever())
def stop(self)
self.server.shutdown()
test_mock.py
server_inst = server_wrap()
server_inst.start()
#code which sets the data and expects the handle method to print the data set
server_inst.stop()
The problem which i have with this code is, the execution stops at server_inst.start(), where the server goes in to an infinite listening mode
Other Solutions that I have tried, but failed:
Using global variables
Using queues
starting mock_server.py
with its own main
Let me know about any other possible solutions. Thanks in advance
Update 1:
Using separate threads to send data to the socket:
Changes
test_mock.py
def test_set_data(data)
server_inst = server_wrap()
server_inst.set_data(data)
server_inst.start()
if __name__ == "__main__":
thread = Thread(target=test_set_data, args=("foo_data))
thread.setDaemon(True)
thread.start()
#test code which verifies if data set is same
#works so far, able to pass data
#problem starts now
thread = Thread(target=test_set_data, args=("bar_data))
thread.setDaemon(True)
thread.start()
#says address already in use error
#Tried calling server.shuddown() in handle , but error persists. Also there is no thread.shop in threading.Thread object
Thanks
The server should go to listening mode.
You don't need the server_inst.stop until all the data was sent, and the test finishes. Maybe in you test tear down, or when the the test suite is completed.
To send data to the server, and let the handle pick it, you should open a socket on anohter thread. Then send the data to the server via this socket.
This code should look something like this:
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(("127.0.0.1",49555))
sock.send(... the data ...)
received = sock.recv(1024) # the handle can send a response
sock.close()
Add a function in your django code, which does run on another thread. This function will open the socket, connect, send the data and get the response. You can call it from a view, a middleware etc.
I have to send data only to a connection, as I can do?
server:
import asyncore, socket, threading
class EchoHandler(asyncore.dispatcher_with_send):
def __init__(self,sock):
asyncore.dispatcher.__init__(self,sock=sock);
self.out_buffer = ''
def handle_read(self):
datos = self.recv(1024);
if datos:
print(datos);
self.sock[0].send("signal");
class Server(asyncore.dispatcher):
def __init__(self,host='',port=6666):
asyncore.dispatcher.__init__(self);
self.create_socket(socket.AF_INET, socket.SOCK_STREAM);
self.set_reuse_addr();
self.bind((host,port));
self.listen(1);
def handle_accept(self):
self.sock,self.addr = self.accept();
if self.addr:
print self.addr[0];
handler = EchoHandler(self.sock);
def handle_close(self):
self.close();
cliente = Server();
asyncore.loop()
this line is an example fails, but I want to send data to zero sock:
self.sock[0].send("probando");
for example, if I have 5 sockets choose who to send the data
Explanation
You tried to get sock from list and execute its send method. This causes error, because EchoHandler neither has sock attribute nor it's a list of sockets. The right method is to get instance of EchoHandler you want (based on, eg. IP address, or slots assigned by some user-defined protocol) and then use its send method - here (with dispatcher_with_send) its also better to use special buffer for that than send.
EchoHandler instantion is created on every accept of connection - from then it is an established channel for communication with the given host. Server listens for any non-established connection, while EchoHandlers use socks (given by Server in handle_accept) for established ones, so there are as many EchoHandler instances as connections.
Solution
You need to make some list of connections (EchoHandler instantions; we'll use buffer, not socket's send() directly) and give them opportunity to delete their entries on close:
class Server(asyncore.dispatcher):
def __init__(self, host='', port=6666):
...
self.connections = []
def handle_accept(self):
...
handler = EchoHandler(self.sock, self);
self.connections.append(self.sock)
...
def remove_channel(self, sock):
if sock in self.connections:
self.connections.remove(sock)
class EchoHandler(asyncore.dispatcher_with_send):
def __init__(self, sock, server):
...
self.server = server
def handle_read(self):
datos = self.recv(1024);
if datos:
print(datos);
self.out_buffer += 'I echo you: ' + datos
def handle_close(self):
self.server.remove_channel(self)
self.close()
EchoHandler is now aware of server instance and can remove its socket from list. This echo example is now fully functional, and with working socket list we can proceed to asynchronous sending.
But, at this point you can use this list as you wanted - cliente.connections[0].out_buffer += 'I am data' will do the work, but probably you'd want some better controlling of this. If yes, go ahead.
'For whom, by me'
In order to send data asynchronously, we need to separate asyncore from our control thread, in which we'll enter what to send and to whom.
class ServerThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True # if thread is a daemon, it'll be killed when main program exits
self.cliente = Server()
self.start()
def run(self):
print 'Starting server thread...'
asyncore.loop()
thread = ServerThread()
while True:
msg = raw_input('Enter IP and message divided by semicolon: ')
if msg == 'exit':
break
ip, data = msg.split('; ')
for sock in thread.cliente.connections:
if sock.addr[0] == ip:
sock.out_buffer += data
break
This will work and wait for destination IP and data. Remember to have client connected.
As I said, you can use anything to indicate which socket is which. It can be a class with fields for eg. IP and username, so you could send data only to peers whose usernames start with 'D'.
But...
This solution is a bit rough and needs better knowledge of asyncore module if you want to send data nicely (here it has some delay due to how select() works) and make good use of this socket wrapper.
Here and here are some resources.
Syntax note
Although your code will now work, your code has some not-nice things. Semicolons on instructions ends don't cause errors, but making nearly every variable of class attribute can lead to them. For example here:
def handle_accept(self):
self.sock,self.addr = self.accept();
if self.addr:
print self.addr[0];
handler = EchoHandler(self.sock);
self.sock and self.addr might be used in that class for something other (eg. socket-related thing; addresses) and overriding them could make trouble. Methods used for requests should never save state of previous actions.
I hope Python will be good enough for you to stay with it!
Edit: sock.addr[0] can be used instead of sock.socket.getpeername()[0] but it requires self.addr not to be modified, so handle_accept() should look like this:
def handle_accept(self):
sock, addr = self.accept()
if addr:
print addr[0]
handler = EchoHandler(sock, self)
self.connections.append(handler)
When I connect using Twisted conch I get Packet integrity error (6 bytes remaining) at serverloop.c:980 in the /var/log/secure.log
After that the connection gets dropped ("Disconnecting: Packet integrity error")
The server is a VMware system, no firewalls or other security is between the systems. I also see that the authentication runs successfully (Accepted password for from port ssh2).
My ssh.py is nearly the same as the example:
class SSHCommandChannel(channel.SSHChannel):
name = "session"
def __init__(self, eventparser, *args, **kwargs):
channel.SSHChannel.__init__(self, *args, **kwargs)
def _cbSendRequest(self, data):
print("%s" % data)
self.conn.sendEOF(self)
def channelOpen(self, data):
d = self.conn.sendRequest(self, 'exec', common.NS(data), wantReply=True)
d.addCallback(self._cbSendRequest)
return None
def extReceived(self, dataType, data):
self.dataRecieved(data)
def dataRecieved(self, data):
print("w00t: %s" % data)
def closed(self):
print("Channel closed =(")
I have no idea how to continue my quest on getting this example to work.
Any advice would be appreciated.
My guess is that you're passing strange data to the exec request. I don't recall offhand what the data passed to the client side of a channel is, but you probably don't want to be sending it back to the server. The server is expecting an NS-encoded command to execute, not random data. Unfortunately I can't reproduce the error your seeing here on my Mac, but hopefully this points you in the right direction!