So, suppose in one thread the UDP socket is hanging on the recvfrom() call, waiting for incoming messages.
However, in another thread you would like to write through the socket to another address.
Is this possible? If not, what are my alternatives?
Related
I have a script that creates raw PACKET sockets to capture all incoming traffic, and I want to make sure that when the script finishes, the sockets are closed.
From python socket documentation, I understand that shutdown() and close() methods should be used to close a socket in a timely fashion.
However, I guess that for this type of socket, SHUT_RD, SHUT_WR and SHUT_RDWR modes can not be used, which renders shutdown() unusable.
On occasions, when I used close() only, the script hanged and I had to wait for it a lot, for the socket to actually close.
My question is: On Linux, how can I close a raw socket immediately, if I no longer need it?
Socket.close() does not stop any blocking socket.accept() calls that are already running on that socket.
I have several threads in my python program that only run a blocking socket.accept() call on a unix domain socket that has been closed already.
I want to kill these threads by making the socket.accept() calls stop or
raise an exception.
I am trying to do this by loading new code in the program, without stopping the program.
Therefore, changing the code that spawned these threads or that closed the sockets is not an option.
Is there any way to do this?
This is similar to https://stackoverflow.com/a/10090348/3084431, but these solutions wont work for my code:
This point is not true, closing won't raise an exception on the accept. shutdown does, but that can not be called anymore when the thread is closed.
I can not connect to this socket anymore. The socket is closed.
The threads with the accept calls are already running, I can't change them.
Same as 3
For clarification, I have written some example code that has this problem.
This code works in both python 2 and python 3.
import socket
import threading
import time
address = "./socket.sock"
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
sock.bind(address)
sock.listen(5)
def accept():
print(sock.accept())
t = threading.Thread(target=accept, name="acceptorthread")
t.start()
sock.close()
time.sleep(0.5) # give the thread some time to register the closing
print(threading.enumerate()) # the acceptorthread will still be running
What I need is something that I can run after this code has finished that can stop the acceptor thread somehow.
There is no mechanism in kernel to notify every listener that a socket is closed. You have to write something yourself. A simple solution is to use timeout on socket:
sock.settimeout(1)
def accept():
while True:
try:
print(sock.accept())
except socket.timeout:
continue
break
Now when you close the socket the next call (after a timeout) to .accept() will throw a "bad descriptor" exception.
Also remember that sockets api in Python is not thread safe. Wrapping every socket call with a lock (or other synchronization methods) is advised in multi-threaded environment.
More advanced (and efficient) would be to use wrap your socket with a select call. Note that the socket does not have to be in non-blocking mode in order to use it.
Therefore, changing the code that spawned these threads or that closed the sockets is not an option.
If that's the case, then you are doomed. Without changing the code running in threads it is impossible to achieve. It's like asking "how can I fix my broken car without modifying the car". Won't happen, mate.
You should only call .accept() on a socket that has given the "readable" result from some selectors. Then, accept doesn't need to be interrupted.
But in case of spurious wakeup, you should have the listening socket in O_NONBLOCK mode anyway.
I have a python tcp server, there is thread for every connection to listen.
When I call close on connection object exception "bad file descriptor" is thrown.
By googling I've found some solutions, each using loop in order to receive client data and breaking that loop, when they decide to disconnect client. My client is written in C# and does not "get", that it's "disconnected" from server, python simply ignores incomming data from C# client.
What's the legit, best practice way to disconnect tcp connection from server side in python ?
Thanks in advance
A bad file descriptor, most likely, means that the socket was already closed by another thread.
Here are some thoughts on general practices. For the client, one way to know that it is disconnected is to check if the recv() value is 0. If it is, then that means the remote side has closed the connection. Basically, you should use select (or poll) and pass fds of all the clients and teh server to select. If you get a read event on any of the fds, then depending upon the fd type, here is what happens. If the fd is server type, then a read event means that there is a pending connection and you should issue an accept() to get the new connection. On the other hand, if hte fd is a non-server type (meaning a regular tcp connection), then a read event means that there is some data and you should issue a recv() event to read data.
You would use a loop for the select. Basically, start the loop using a select() call and once you get an event, do something with that event, and then reenter the loop and issue the next select().
You might find these links helpful: http://ilab.cs.byu.edu/python/select/echoserver.html and http://docs.python.org/2/library/socket.html
From the docs:
Note: close() releases the resource associated with a connection but
does not necessarily close the connection immediately. If you want to
close the connection in a timely fashion, call shutdown() before
close().
So you should call shutdown() before calling close(). Also you should pass SHUT_RDWR flag to completely shutdown the connection:
from socket import SHUT_RDWR
...
try:
s.shutdown(SHUT_RDWR)
s.close()
except Exception:
pass
The "bad file description" error means (most likely) that the socket is already closed (at least from Python side).
I'd like to create a python socket (or SocketServer) that, once connected to a single device, maintains an open connection in order for regular checks to be made to see if any data has been sent. The socket will only listen for one connection.
E.g.:
def get_data(conn):
response='back atcha'
data = conn.recv(1024)
print 'get_data:',data
if data:
conn.send(response)
s = open_socket()
conn, addr = s.accept()
while True:
print 'running'
time.sleep(1)
get_data(conn)
#do other stuff
Once the server socket is bound and the connection has been accepted, the socket blocks when running a .recv until either the connecting client sends some data or closes its socket. As I am waiting for irregular data (could be seconds, could be a day), and the program needs to perform other tasks in the meantime, this blocking is a problem.
I don't want the client to close its socket, as it may need to send (or receive) data at any time to (from) the server. Is the only solution to run this in a separate thread, or is there a simple way to setup the client/server sockets to maintain the connection forever (and is this safe? It'll be running on a VLAN) while not blocking when no data has been received?
You're looking for non-blocking I/O, also called asynchronous I/O. Using a separate thread which blocks on this is very inefficient but it's pretty straightforward.
For a Python asynchronous I/O framework I highly recommend Twisted. Also check out asyncore which comes with the standard library.
I want to create a python network application that can run on multiple ports (ex: TCP:1234, TCP:5678, etc).
So I have lets say n number of Sockets, each listening for a client connection. I programmed a simple network application that listens to a range of ports but when I run the application it gets stuck at the listening phase of the first socket process!
How can I make my single python program when run to listen to N number of ports and each waiting for a client to connect to it. All sockets are running and listening at the same time.
Socket/Process #1: Listening on TCP Port 5000
Socket/Process #2: Listening on TCP Port 5001
Socket/Process #3: Listening on TCP Port 5002
...
Socket/Process #N: Listening on TCP Port 6000
Appreciate any ideas.
#!/usr/bin/env python
import socket
def getPortList():
ports=[]
nPort=int(raw_input("# how many ports you want? "))
j = 0
for i in range(0,nPort):
ports.append(int(raw_input("Enter port number: ")))
j+=1
return ports
def myTCPSocket(port=5000):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR,1)
s.bind(("", int(port)))
print ("\nWaiting for connections!\n")
s.listen(5)
(clientsock, clientaddr) = s.accept()
print(clientaddr)
data = "start"
while len(data):
clientsock.send("\nWelcome to Echo Server\n")
data = clientsock.recv(1024)
print ("Data sent is: ", data)
clientsock.send(data)
if data == "exit\r\n":
clientsock.close()
plst = getPortList()
for item in plst:
myTCPSocket(item)
Listening on multiple sockets is really no different from listening on a single socket.
You already need to handle the listener socket and all client connection sockets somehow. You can do this by:
Writing a loop around select.select (or poll, kqueue, epoll, etc.).
Using the standard-library reactor asyncore.
Using a third-party reactor or proactor like Twisted.
Using OS-specific functionality (e.g., using a Cocoa runloop and server via PyObjC).
Creating a thread for each new connection.
Creating a subprocess for each new connection.
Almost all of these schemes will also work for dealing with multiple listeners. The simplest thing to do is to combine the two into one (e.g., a single select loop that handles all of their listeners and all of their client sockets, or a separate thread for each listener and client socket).
For performance or debugging reasons, you might want to instead use a two-tier hybrid approach (e.g., a thread for each listener, each with a select loop for all of its client sockets, or a process for each listener, each with a thread for each client socket). But if you don't have any good reason to do that, don't add the complexity.
http://pastebin.com/QebZMKz3 shows a simple single-select implementation. Here's the output:
$ ./multiserve.py 22222 22223 &
(('127.0.0.1', 22222), ' listening')
(('127.0.0.1', 22223), ' listening')
$ echo 'abc' | nc localhost 22222
(('127.0.0.1', 22222), ' <- ', ('127.0.0.1', 64633))
(('127.0.0.1', 64633), ' <- ', 'abc\n')
(('127.0.0.1', 64633), ' EOF')
If you think you'll never actually need to handle two simultaneous clients… well, you're probably wrong, but… You can use most of the above techniques, and it may be marginally simpler. For example, you can select on the listeners, and then do the accept and client-socket communication synchronously before returning to the loop. Or you can create a process or thread for each listener but handle the accept and client-socket communication synchronously within each. And so on.
http://pastebin.com/wLVLT49i shows a simple example that seems to be what you were trying to do. Since it uses a process for each socket (via os.fork), it does allow simultaneous connections on different ports; since it doesn't do anything asynchronously within each process, it doesn't allow simultaneous connections to the same port. (And of course it's POSIX-specific because it uses fork.)
If you're looking to learn how to write asynchronous network servers in the first place, I'd suggest you do two different implementations: select and threads. They conceptually fundamental, and relatively easy to code.
First, for select, you have to get your head around the idea of an event loop—the events are each new incoming connection, each incoming network packet on an existing connection, even each time a pipe you were writing to gets unclogged. The tricky bit here is that, as with any event loop, you need to handle each event and return without blocking, and without spending too much CPU time. For example, for an echo server, you can't just do a write on the other sockets, because some of them might be busy. So instead, you have to stick the output in a write buffer for each socket, and they'll get it in some future run through the event loop, when thye're ready.
Meanwhile, for threads, a separate thread for each connection seems like it makes everything trivial, but what happens when you need to echo a message from one thread to another? You either need some form of inter-thread communication, or shared data with inter-thread synchronization. So, you might have a Queue for writes on each socket, so any other socket's thread can just push a message onto the queue.
Neither of these will be as good as what a well-turned reactor or proactor can do, but it'd worth learning the basics—especially since you're going to face both the blocking issue (from select) and the communication issue (from threads) with any solution, and they'll be much more mysterious and harder to debug when you're working at a higher level.