My client sends a string "abcd" and half closes the socket (the write part). My server reads the data and appends it inside the list (collection) until end of file is recieved (half closed socket detected). Then it iterates through the list and sends the data.
My server code:
while True:
try:
sock,address = self.__mySocket.accept()
except:
print "Client is dead"
break
print "Client connect: " + str(address)
collection = []
while True:
data = sock.recv()
if len(data) == 0:
break
data = str(data[::-1])
collection.append(data)
for val in collection:
sock.send(val)
sock.close()
The client:
sslsock.sendall('abcd\n')
time.sleep(1)
sslsock.shutdown(socket.SHUT_WR)
data = ""
while True:
data = sslsock.recv()
if len(data) == 0:
sslsock.close()
sys.exit(1)
print data
Now when I print the data on the client it just print garbage. I've tried using pickle and that didn't work either. Now, when I comment out the shutdown on the client and work my server around it just works fine. It prints the reverse of the sent data.
In server Code. I put the for loop inside the if len(data) ==0 . And, It works. I 'm guessing that break statement was breaking out of even the outside While True. So, it never got to the point of sending.
Related
I am trying to do multiple while loops but somehow they don't work. I already searched the internet but none of the problems I found has the same issue.
So here is the code containing only the necessary information. I am basically opening a socket, giving an in input (i\n) and receiving the output in the first step. I want to continue receiving the output until I have some specific characters xxx in the output. Then I want to go to the elif statement in the next loop.
def netcat(h, p):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((h,p))
i = 0
a = True
while a == True:
socket_list=[sys.stdin, s]
r,w,x = select.select(socket_list, [], [])
if i==0:
time.sleep(1)
message = s.recv(1024)
print(message)
s.send("i\n")
sys.stdout.flush()
while "xxx" not in message:
message = s.recv(1024)
print(message)
i+=1
elif i==1:
print("+++++++++++++++++++++++++++")
i+=1
print("hello")
server.close()
What I would expect the code to do is to print the message from the if statement, then print hello, then the message from the elif statement and then hello over and over again because the while loop is still active. So in summary this is the expected output:
message
hello
+++++++++++++++++++++++++++
hello
hello
hello
hello...
What it really prints is
message
hello
and then it finishes.
What I found out is that if I comment out the following lines:
while "xxx" not in message:
message = s.recv(1024)
print(message)
it works as expected. The hello at the end of the code gets printed to the screen over and over again. I just don't get it why this second while loop has anything to do with it. I would really appreciate help here.
Since the working code was requested, here is also the full code. The hostname and port are from a CTF which is still working so you will be interacting with the CTF-server:
#!/usr/bin/env python
import socket
import time
import select
import sys
base64="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/ ="
hostname = "18.188.70.152"
port = 36150
def netcat(h, p):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((h,p))
i = 0
a = True
b = True
while a == True:
socket_list=[sys.stdin, s]
r,w,x = select.select(socket_list, [], [])
if i==0:
time.sleep(1)
message = s.recv(1024)
print(message)
s.send("i\n")
sys.stdout.flush()
while "flag" not in message:
message = s.recv(1024)
print(message)
txtfile = message[9:38]
print(txtfile)
i+=1
elif i==1:
print("+++++++++++++++++++++++++++")
i+=1
print("hello")
server.close()
netcat(hostname, port)
You're mixing event-based code (select.select()) with blocking synchronous code (your small while loop with the s.recv()).
If you want your code not to block, every recv() needs to be paired up with a preceding select().
Not only that, but you must also check the returned values from select(). Only s.recv() if s was in the first returned list. If you s.recv() in any other case, the code will also block on the receive call.
Update:
Try something along the lines of:
not_done = True
while not_done:
read_sockets, _, _ = select.select([sys.stdin, s], [], [])
if s in read_sockets:
message = s.recv(1024)
print(message)
... more code ...
if 'flag' in message:
... react on flag ...
if 'quit' in message:
not_done = False
... processing of other sockets or file descriptors ...
The important point being that there is only this one s.recv() in the if branch that checks for whether select detected something was received.
The outer while will just come back to the same if branch later when additional data was received.
Note that processing stdin alongside socket code is tricky and will likely also block at some point. You will likely have to put the terminal into raw mode or something first and then be ready to process partial lines yourself as well as maybe also echoing the input back to the user.
Update:
If you want to do something while no message was received, you can give a timeout to select() and then do other processing if there was nothing received on the socket. Something like this:
say_hello_from_now_on = False
not_done = True
while not_done:
read_sockets, _, _ = select.select([s], [], [], 1)
if s in read_sockets:
message = s.recv(1024)
print(message)
say_hello_from_now_on = True
elif say_hello_from_now_on:
print("hello")
I'd check your indentation, try installing and running autopep8 on your code and see if that fixes any of your issues.
[edit] user has updated their question and it's clear that this isn't the answer.
I'm working on a CTF challenge and I need to code a bot that does math problems. The way I'm doing it is by using sockets in python. My script seems to work once, but then it doesn't send or receive any data and it keeps listening. I tried putting print statements in the code to see where it was stuck, but none of the prints seem to be printing out after they are printed out 2 to 3 times.
Could someone help me figure out how to make the loop run forever so it can receive and send data?
I'm using python 2.7 and I'm using debian if it matters at all.
#!/usr/bin/python
import socket, re, time
FLAG = 'FLAG'
BOT = ('IP to server', port)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(BOT)
while 1:
string = ""
while True:
chunk = s.recv(4096)
string += chunk
if len(chunk) < 4096:
break
if len(string) <= 4096:
if string != "":
break
# print "loop 1\n"
# time.sleep(.5)
for line in string.split("\n"):
# print "loop 2\n"
if FLAG in line:
print line
if re.search("([0-9])+", line):
result = eval(line)
print result
s.send(str(result))
# raw_input("enter:")
s.close()
I have this following code which is in my server.py file. It is waiting to receive data from the client. Additionally I cannot use any of the http libraries. Only the socket Library:
def handle_client(conn, addr):
print ('New client from', addr)
x = []
try:
while True:
data = conn.recv(1024)
decoded_data = data.decode('utf-8')
# if "GET / " in data.decode('utf-8'):
# handle_client_get(conn)
# else:
if data:
print(data)
x.append(decoded_data)
else:
print(x)
break
finally:
print("in close now")
conn.close()
The problem I am having is that I can only reach the print(x) statement once I manually CTRL + Cclose the client. Otherwise it doesn't print.
Why is that the case.
ANSWER
You need to send an acknowledgment to the client such that the data sent has been received correctly.
that will terminate the connection and not wait for a timeout.
This is because the client sends: Expect: 100-continue
And you need to send an acknowledgment back to the client
You need to implement the protocol which will have a means to tell you how much data there is to read. In the case of HTTP, the request starts with a CRLF delimited header, and we can read that to get the information we want.
w3.org has a good description of the http request protocol. Its more complicated than I want to implement here but I've included an example that pulls in the request header by reading the socket one character at a time and looking for an empty \n terminated line. By reading one character at a time, I don't have to implement my own line buffer.
The first line is the request method and the remaining lines are other parameters included with the request. For a POST, for instance, there would be more data still to read from the socket.
import re
def handle_client(conn, addr):
print ('New client from', addr)
header = []
line = []
try:
# read ascii http client header.
while True:
c = conn.recv(1)
# check for early termination
if not c:
return None
# check for end of request line
elif c == b"\n":
# make line a string to add to header
line = ''.join(line).decode('ascii').strip()
# are we at the empty line signalling end-of-header?
if not line:
break
header.append(line)
line = []
# filter out \r
elif c == b"\r":
continue
# request is first line of header
request_line = header.pop(0)
method, uri, http_version = re.split(r" +" request_line)
if method.upper() == "GET":
# this function needs more parameters... the uri to get and the protocol
# version to use.
handle_client_get(...)
finally:
print("in close now")
conn.close()
Edit:
Now that you mention this is a HTTP server and is to be done using only socket library, I suggest you remove certain complex elements from the program at this stage like removing the while block until this problem is solved.
server.py
def handle_client(conn, addr):
print ('New client from', addr)
x = []
try:
data = conn.recv(1024)
decoded_data = data.decode('utf-8')
if data:
print(data)
x.append(decoded_data)
else:
print(x)
break
finally:
print("in close now")
conn.close()
Original Answer:
Try sending an additional \n at the sender end and use the flush function later to send out any remaining data on the buffer something like:
sender.py
def send_data(data,sock_obj,addr):
sock_obj.send(data+"\n")
sock_obj.flush()
this is question is really focused on my problem and not relative to any of the other question I could find on this topic.
PSA: When I say "packet" I mean a full string received in a single socket.recv(maxsize)
I developed similar code for the same result on Java (my pref language) and it is ok, now I have to do in python.
I have two processes that run in parallel:
1-Normal client socket connected to a specific IP
2-A "client" Datagram socket binded to "ALL" IPs.
The normal socket is working correctly as I expect, while the datagram not.
I continuosly receive packets from a server (not mine and not opensource) at a rate of more than 5 per second, but I want to process only one of them every 3 seconds. In java I did just a "sleep" and it was ok, I was getting only the last live packet, while in Python with a "time.sleep(3)" the packets are queued (I don't know how and where) and not dropped.
I HAVE to drop them because those are not need and I have to do an HTTP call between one and the other so I can't fire an HTTP post for every set of data received at that rate!
here it is my "code" for the listening socket, some comments are for private code:
def listenPositions():
lsSocket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
lsSocket.bind(("0.0.0.0", 8787))
lsSocket.setblocking(0)
try:
while True:
ready = select.select([lsSocket], [], [], 1)
if ready[0]:
lsSocket.settimeout(1)
recvData = lsSocket.recv(16384)
if len(recvData) != 0:
recv = recvData[0:len(recvData)].decode("utf-8")
#print("LS: Received: " + recv)
strings = filter(None, str(recv).split('\n'))
print("Strings count=" + str(len(strings))+ ": " + str(strings))
for item in strings:
#parse the received strings as json and get the items
jsonPosition = json.loads(item)
strId = jsonPosition["id"]
coordinates = jsonPosition.get("coordinates")
if coordinates is None:
continue
print("coordinates not null:" + str(coordinates))
#DO THE HTTP POST REQUEST
time.sleep(3) #Pause the system for X seconds, but other packets are queued!
else:
print("LS: Received empty")
else:
print("LS: No data, timeout")
except Exception as e:
print(e)
#handle exceptions...
print("Exception, close everything")
When you have an open socket, all correctly addressed packets should be delivered to the application. We want to have our network connections as realiable as possible, don't we? Dropping a packet is an action of last resort.
If you want to get a packet only from time to time, you could create a listening socket, get a packet and close the socket.
However there is nothing easier than ignoring a packet. Just skip its processing and move on. The code below is incomplete, but hopefully expresses what I mean.
TIMEOUT = 1.0
INT = 3.0 # interval in seconds
# create udp_socket
last = time.time() - INT
udp_socket.settimeout(TIMEOUT)
while True:
try:
packet = udp_socket.recv(MAXSIZE)
except socket.timeout:
# handle recv timeout
continue # or break, or return
except OSError:
# handle recv error (Python 3.3+)
break # or continue, or return
now = time.time()
if now - last >= INT:
# process the packet
last = now
Please note that the select is not needed if you read only from one source.
I'm new to Python programming and I'm trying to create a server and a client. I still want to be able to type something from the keyboard so i can close the server from the server by typing 'exit'. I've taken samples codes from various sites to get to where I'm at in socket programming and this code.
However, whenever I run the code I get the following error message:
The host name of this machine is 127.0.0.1
The IP address of the host is 127.0.0.1
Server now awaiting client connection on port 2836
im right before the select
Traceback (most recent call last):
File "/root/Server_2.py", line 42, in <module>
inputready, outputready, exceptready = select.select(input, [], [])
TypeError: argument must be an int, or have a fileno() method.
>>>
I was reading around that to get passed this (in Windows) is to remove the sys.stdin because Windows only accepts sockets. I'm trying to write this code in Linux. I've tried all sorts of things to try to get it to work and I'm all out of resources and ideas to try. Below is the server code:
import socket #import socket module
import select
import sys
host = "127.0.0.1"
print ("The host name of this machine is " + host)
hostIP = socket.gethostbyname(host) # get host IP address
print ("The IP address of the host is %s" % (hostIP))
port = 2836 # Reserve the port for the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((hostIP, port)) # This server to a port
s.listen(4) # Now wait for client connection
print("Server now awaiting client connection on port %s" % (port))
#WINDOWS ONLY ACCEPTS SOCKETS FOR SELECT(), no standard in
input = [s, sys.stdin]
running = 1
while running:
print("im right before the select")
# when there's something in input, then we move forward
# ignore what's in output and except because there's nothing
# when it comes to sockets
inputready, outputready, exceptready = select.select(input, [], [])
print("i'm here na")
# check who made a response
for x in inputready:
if x == s:
print(s)
#handle the server socket
client, address = s.accept()
print("connection comming in")
input.append(client)
elif x == sys.stdin:
# handle standard input
stuff = sys.stdin.readline()
if stuff == "exit":
running = 0
else:
print("you typed %s" % (stuff))
else:
#handle all other sockets
data = x.recv(1024)
print("i received " + data)
if data:
if data == "exit":
x.close()
input.remove(x)
running = 0
else:
x.send(data)
print("I am sending %s" % (data))
else:
x.close()
input.remove(x)
s.close()
Any help or ideas would be greatly appreciated. Thanks!!
Well I know you asked this 7 years ago, but I had similar questions so I would figure I answer you. I'm still working and bugfixing a program that has the same functionality, but one thing I do know is that the lists that are the arguments in select.select() need to be file descriptors (ints).
So if you have this block
input = [s, sys.stdin]
running = 1
while running:
print("im right before the select")
# when there's something in input, then we move forward
# ignore what's in output and except because there's nothing
# when it comes to sockets
inputready, outputready, exceptready = select.select(input, [], [])
The first thing I'd say is change your read list to not be input. You'll likely get some clashing with the input() function, which may cause confusing bugs. After that, you want the values to be file descriptors. So that first line should be
inputSockets = [s.fileno(), sys.stdin.fileno()]
Then when checking which socket is ready to ready, you would want to do it like this
for x in inputready:
if x == s.fileno():
# Read from your s socket
elif x == sys.stdin().fileno():
# Read from stdin
else:
'''
Here you would want to read from any other sockets you have.
The only problem is your inputSockets array has ints, not socket
objects. What I did was store an array of actual socket objects
alongside the array of file descriptors. Then I looped through the
list of sockets and found which socket's .fileno() matched x. You could
probably be clever and use a dict() with the filenos as key and socket as
value
'''
I just came across this while writing a unix domain socket (UDS) interface. The server socket id is used to accept incoming client connections. That is pretty much all it does. Once the client is accepted, reading uses its own file descriptor. Something like this works:
conn = None
inputReady, Null, Null = select.select(inputSockets, [], [])
for x in inputReady:
if x == s.fileno():
# accept incoming connect and add to poll list
conn, addr = s.accept()
inputReady.append(conn.fileno())
elif x = sys.stdin.fileno():
# read whole line and remove newline
cmd = sys.stdin.readline()[:-1]
...
elif conn and x == conn.fileno():
data = conn.recv(msglen)
if data:
....
else:
# connection has ended, remove from poll list and close
if conn.fileno() in inputReady:
inputReady.remove(conn.fileno())
conn.close()