We Keep Coding

Can I use the same socket for multiple connections?

I'm trying to make a python function that scans a range of addresses. I started a socket and pass the socket as an argument to the function that connects to it:
def scan(socket, address, port):
c = socket.connect_ex((address, port))
print(c)
then I call scan for each address, each in its own thread. I'm getting Error 114: Operation already in progress..
Do I need to start a new socket for each connection? I'm trying to read about socket reusage, and I found that there exists flags like SO_ADDREUSE or something like that. I tried to insert but it didn't work.
I'm trying to think how a socket works. I think the moment I create one, it choses a tcp source port, and then when I create a connection, it sends to a destination port. I think I can't reuse the same socket because the source port would be the same for all destination ports, so the clients would answer to the same port and would cause confusion.
So do I need to create a new socket for each connection?

You can not connect stream socket multiple times.
One of the connect possible errors is EISCONN.
The socket is already connected.
This goes for stream sockets.
man bind also has this:
[EINVAL] The socket is already bound to an address, and the
protocol does not support binding to a new address; or
the socket has been shut down.
Again, this goes for stream sockets.
From the man connect:
Generally, stream sockets may successfully connect() only once; datagram sockets may use connect() multiple times to change their association.
I made emphasis on the important line.
stream sockets can not be connected multiple times. datagram sockets can be connected multiple times. Generally speaking, BSD sockets have multiple protocols, types, domains avaible. You shall read documentation for your particular case.
P.S Get yourself familiar with the readings that were suggested in the comment to your question. That will explain enough to manipulate socket family of functions.

Do I need to start a new socket for each connection?
Yes.
I'm trying to read about socket reusage
There is no such thing as 'socket reusage'. There is port reuse. Not the same thing. You cannot reconnect an existing socket once you've tried to connect it, even if the connect attempt failed.
I found that there exists flags like SO_ADDREUSE or something like that
SO_REUSEADDR means to reuse the port. Not the socket.

I'm trying to think how a socket works. I think the moment I create one, it choses a tcp source port,
Between creating a socket using the socket() system call and using it to create an outgoing connection with the connect() system call, there is an opportunity to optionally use the bind() system call to set source IP address and/or port if you want to. If you don't use bind(), the operating system will automatically bind the socket to the first available port in the appropriate range when you use the connect() system call. In this case, the source IP address is normally selected to match the network interface that provides the shortest route to the specified destination according to the routing table.
At least, that's how it works at the system call level. Some programming languages or libraries may choose to combine some of these operations into one.
To your actual question, man 7 ip says:
A TCP local socket address that has been bound is unavailable for some
time after closing, unless the SO_REUSEADDR flag has been set. Care
should be taken when using this flag as it makes TCP less reliable.
The idea is to delay the re-use of a port until any possible re-sent copies of packages that belonged to the closed connection have for sure expired on the network.
According to the bind() man page, trying to re-bind a socket that is already bound to an address will result in an EINVAL error. So "recycling" a socket using bind(socket, INADDR_ANY, 0) (after ending a connection that used SO_REUSEADDR) does not seem to be possible.
And even if that would be possible, when you're using multiple threads on a modern multi-core system, you end up (very probably) doing multiple things in parallel. A socket can be used for just one outgoing connection at a time. Each of your scan threads will need its own socket.

(Python/TFTP-Server) how to listen to an non (yet) existing IP-address (RNDIS)?

Hy folks,
my problem: I want to start a (tftp) server for an non-existing IP-address.
the server is meant for USB/RNDIS where its IP-address by nature only is existing when there is actual network-traffic going on -- but I want to start the server 'early' (e.g. when Windows starts).
idea was to bind() the socket to 0.0.0.0 - and then to check each request for "valid" addresses.
problem with that approach: recfrom() only returns the source-address (client), but not the destination (server)!
how do I get the IP-address this client has talked to?
(I could of course check for the same subnet at least, but I need the real server-address this client was talking to!)
or, are there by chance any options to let bind() use a non-existing IP-address?
cheers.
p.s.
this is for the Python tftpy server...
-- at the moment I need to ping from client side when starting the server, which is quite meh...

There's no way to get the local address directly but there's a trick that will usually work for you.
Let's say that you just obtained a buffer and client address from recvfrom. Now you create a new auxiliary UDP socket, connect it to the client's address, and then use getsockname to obtain the local address on this new socket. (With UDP, connect doesn't actually send anything to the peer, it just does address resolution.)
So in this way, you can discover the IP address that the server system would use as source were it to send a datagram back to the client system. That's usually the same address that the client used to target the server.
>>> cbytes, caddr = server_sock.recvfrom(2048)
>>> print(caddr) # Client IP and port
('192.168.0.11', 38869)
>>> aux_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
>>> aux_socket.connect((caddr[0], 9999)) # Connect back to client (port doesn't matter)
>>> saddr = aux_socket.getsockname() # Get local IP and port (ignore port here too)
>>> print(saddr)
('192.168.0.12', 39753)
This works on linux. I'm not 100% sure it would work the same way on Windows but it seems reasonable to expect that it would.

Unable to use python logger remotely using socket, but works on localhost

I am using the logging utility on my local network to get log data from one python script to another. The server and client scripts works on the same machine, but does not work from a different machine.
The client script snippet, running on IP "192.168.1.9" is-
import logging, logging.handlers
rootLogger = logging.getLogger('')
rootLogger.setLevel(logging.DEBUG)
socketHandler = logging.handlers.SocketHandler('192.168.1.10', 50005)
The server script snippet, running on IP "192.168.1.10" locally is -
def __init__(self, host='localhost', port=50005, handler=LogRecordStreamHandler):
socketserver.ThreadingTCPServer.__init__(self, (host, port), handler)
When I run this, both the client and server are unresponsive, as if no message was ever sent.
My iptables is empty (default), and there is nothing except a simple switch between the two machines on the network. I can remotely use MySQL just fine. Even a basic TCP socket connection at a random open port worked fine. So what could be going wrong here? Is it something to do with the logger code above, or could be an entirely different networking reason?

When you construct a socketserver.TCPServer, the (host, port) ultimately gets passed to socket.socket.bind.
The Socket Programming HOWTO explains a bit about what this means, but the short version is that the point of specifying a host is to tell the listener socket which interface(s) to listen to. It resolves the name to an address, asks your OS which interface owns that address, and listens only to that interface.
I'll use my Macintosh as an example here, but the details will be pretty much the same anywhere, except some slight differences in the names.
'localhost' resolves to '127.0.0.1', which belongs to an interface named 'lo0', the "loopback" interface, which listens to nothing but connections from the same machine on the localhost address.
'192.168.1.10' belongs to an interface named 'en0', an Ethernet adapter that listens to everything coming over my Ethernet port. So, that's why it works for you. But it's still not what you (probably) want.
'0.0.0.0' is a special address that belongs to every interface. So this may be what you want.
But notice that specifying an IPv4 address—even '0.0.0.0'—means, at least on some platforms, that you'll only listen for IPv4 connections. If you want to handle IPv6 as well, how do you do that? Well, you can't do it on all platforms, but on those you can, it's just ''.
(On some platforms, that still won't work for IPv6; you need to actually create IPv6 and IPv4 sockets, and bind them to the specific IPv6 and IPv4 "any" addresses separately. But on such platforms, Python still lets you use '' for both "any" addresses, and the default socket will be IPv4, so work-case scenario, this works just as well as '0.0.0.0'.)
So, most likely, you just want:
def __init__(self, host='', port=50005, handler=LogRecordStreamHandler):

Python - Twisted - Simple UDP forwarder. Preserve source IP?

I have this basic UDP forward script in Python 3.
from twisted.internet.protocol import DatagramProtocol
from twisted.internet import reactor
class Forward(DatagramProtocol):
def __init__(self, targetTuples):
print ('in init, targetTuples are ', targetTuples)
self._targetTuples = targetTuples
def datagramReceived(self, data, hostAndPort):
print ('self._targetTuples is ', self._targetTuples)
for (targetHost, targetPort) in self._targetTuples:
self.transport.write(data, (targetHost, targetPort))
reactor.listenUDP(5005, Forward([('10.35.203.24', 5000), ('10.35.200.251', 5005)]))
reactor.run()
So I'm listening on port 5005 UDP, and forwarding those packets to the two IP addresses and different ports.
My question is this -
How do I preserve the original IP address that twisted gets while listening on port 5005?
Source IP (10.1.1.1) --> Twisted (10.30.1.1) --> Multiple Destinations
How can I get Multiple Destinations to see the packet source preserved from the Source IP of (10.1.1.1) ?

When sending UDP datagrams using the BSD socket API (around which, as a first approximation, Twisted is a wrapper), the source address is set to the address the socket is bound to. You can specify the IP of the bind address for a UDP socket in Twisted by passing a value for the interface argument to reactor.listenTCP. However, you are typically restricted in what addresses you are allowed to bind to. Typically the only values allowed are addresses which are assigned to a local network interface. If you are forwarding traffic for 10.1.1.1 but you are on host 10.30.1.1 then you probably cannot set the source address of the UDP packets you send to 10.1.1.1 because that address isn't assigned to a network interface on the host doing the forwarding. If you assigned it to one, routing on your network would probably break in weird ways because then two different hosts would have the same IP address.
This doesn't mean it's not possible to do what you want - but it does mean you probably cannot do it using Twisted's basic UDP support. There are a number of other approaches you could take. For example, you can rewrite source addresses using iptables on Linux. Or you can operate on the IP level and parse and generate full UDP datagrams yourself letting you specify any source address you want (you can do this with Twisted on Linux, too, using twisted.pair.tuntap). There are probably a number of other ways. The best solution for you may depend on which platforms you're targeting and the particular reasons you want to do this kind of forwarding.

Creating a socket restricted to localhost connections only

I have a python program with many threads. I was thinking of creating a socket, bind it to localhost, and have the threads read/write to this central location. However I do not want this socket open to the rest of the network, just connections from 127.0.0.1 should be accepted. How would I do this (in Python)? And is this a suitable design? Or is there something a little more elegant?

Given a socket created with socket.socket(), you can use bind() before listening:
socket.bind(('127.0.0.1', 80))
Using the address 127.0.0.1 indicates that the socket should bind to the local interface only.

http://www.amk.ca/python/howto/sockets/
Shows some socket example. This tidbit is interesting to you I think
we used socket.gethostname() so that the socket would be visible to the outside world. If we had used s.bind(('', 80)) or s.bind(('localhost', 80)) or s.bind(('127.0.0.1', 80)) we would still have a "server" socket, but one that was only visible within the same machine.
I guess there is your answer (see below for correction)
As to the validity of using this method for thread communications. I'm not sure how well this handles multiple threads and reading/writing
EDIT
There seems to be a python recipe linked below that does some inter-thread communication
http://code.activestate.com/recipes/491281/
Have fun!
EDIT
The article is incorrect and as pointed out "s.bind(('', 80)) will bind to INADDR_ANY"

If you are running on a UNIX-based system, you might want to consider using UNIX Domain Sockets instead of Internet sockets. I think something like the following should work:
>>> # in one window/shell
>>> import socket
>>> sd = socket.socket(socket.AF_UNIX)
>>> sd.bind('/path/to/my/socket')
>>> sd.listen(5)
>>> (client,addr) = sd.accept()
>>> client.recv(1024)
'hello'
>>>
>>> # in a different shell
>>> import socket
>>> sd = socket.socket(socket.AF_UNIX)
>>> sd.connect('/path/to/my/socket')
>>> sd.send('hello')

You might want to use the queue module from the standard library instead. It's designed specifically to facilitate communication between threads. A quote from the docs:
The Queue module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads. The Queue class in this module implements all the required locking semantics. It depends on the availability of thread support in Python; see the threading module.

notionOn TCP/IP networks 127.0.0.0/8 is a non-routeable network, so you should not be able to send an IP datagram destined to 127.0.0.1 across a routed infrastructure. The router will just discard the datagram. However, it is possible to construct and send datagrams with a destination address of 127.0.0.1, so a host on the same network (IP sense of network) as your host could possibly get the datagram to your host's TCP/IP stack. This is where your local firewal comes into play. Your local (host) firewall should have a rule that discards IP datagrams destined for 127.0.0.0/8 coming into any interface other than lo0 (or the equivalent loopback interface). If your host either 1) has such firewall rules in place or 2) exists on its own network (or shared with only completely trusted hosts) and behind a well configured router, you can safely just bind to 127.0.0.1 and be fairly certain any datagrams you receive on the socket came from the local machine. The prior answers address how to open and bind to 127.0.0.1.

If you do sock.bind((port,'127.0.0.1')) it will only listen on localhost, and not on other interfaces, so that's all you need.