Firstly i know that you can use the socket constructor with port = 0 to find an unused port, but this isn't my question.
I'm using a python script as an intermediate code between two applications, i want the python script to locate a free port for both applications so that they can communicate, i haven't found a graceful way to locate unused ports in python without binding them to a socket, is it possible without binding? or am i forced to do so?
This is a code that does so, by binding the port and closing it asap but i still don't feel that this is good practice
import socket
s = socket.socket()
s.bind(('localhost',0))
addr,port = s.getsockname()
s.close()
print(addr)
print(port)
the connection is going to be temporary between the apps and many instances could occur
I'm literally looking for pointers on good practice of doing so, and i wanna know if what I'm doing is correct or I'm heading the wrong way
Related
The context related to my question is that I work for a company in the networking area. This company has several stores around the country where DVRs are accessible through port 2781 and a domain for people to access security cameras, the problem is that in order for these people to successfully access DVRs through the domain and port you must have a DMZ configured in the modem of the stores. To corroborate the DMZ I'm trying to use Python with the sockets module but I don't understand the best way to do it yet.
import socket
s = socket.socket()
s.connect((domain, port))
s.close()
Once I make the proper connection which is the best way to check if there is a communication? Work it with an exception or just use socket.recv and detect if it is empty?
In order for connect to succeed there already has to be some kind of connection be done. Otherwise the TCP handshake would fail. Thus the first step would be to check if connect succeeds or throws an exception.
It can still be possible that there is some deep packet inspection firewall in place which does not block the initial connection but only blocks the later data exchange. To find out if this is the case you have to do actual bidirectional communication. But how this communication should look like depends on the specific application protocol which is unknown in your case. Still you need to check that a) sending and receiving works (catching exceptions) and b) that a response returns the expected data.
I'm trying to make a python function that scans a range of addresses. I started a socket and pass the socket as an argument to the function that connects to it:
def scan(socket, address, port):
c = socket.connect_ex((address, port))
print(c)
then I call scan for each address, each in its own thread. I'm getting Error 114: Operation already in progress..
Do I need to start a new socket for each connection? I'm trying to read about socket reusage, and I found that there exists flags like SO_ADDREUSE or something like that. I tried to insert but it didn't work.
I'm trying to think how a socket works. I think the moment I create one, it choses a tcp source port, and then when I create a connection, it sends to a destination port. I think I can't reuse the same socket because the source port would be the same for all destination ports, so the clients would answer to the same port and would cause confusion.
So do I need to create a new socket for each connection?
You can not connect stream socket multiple times.
One of the connect possible errors is EISCONN.
The socket is already connected.
This goes for stream sockets.
man bind also has this:
[EINVAL] The socket is already bound to an address, and the
protocol does not support binding to a new address; or
the socket has been shut down.
Again, this goes for stream sockets.
From the man connect:
Generally, stream sockets may successfully connect() only once; datagram sockets may use connect() multiple times to change their association.
I made emphasis on the important line.
stream sockets can not be connected multiple times. datagram sockets can be connected multiple times. Generally speaking, BSD sockets have multiple protocols, types, domains avaible. You shall read documentation for your particular case.
P.S Get yourself familiar with the readings that were suggested in the comment to your question. That will explain enough to manipulate socket family of functions.
Do I need to start a new socket for each connection?
Yes.
I'm trying to read about socket reusage
There is no such thing as 'socket reusage'. There is port reuse. Not the same thing. You cannot reconnect an existing socket once you've tried to connect it, even if the connect attempt failed.
I found that there exists flags like SO_ADDREUSE or something like that
SO_REUSEADDR means to reuse the port. Not the socket.
I'm trying to think how a socket works. I think the moment I create one, it choses a tcp source port,
Between creating a socket using the socket() system call and using it to create an outgoing connection with the connect() system call, there is an opportunity to optionally use the bind() system call to set source IP address and/or port if you want to. If you don't use bind(), the operating system will automatically bind the socket to the first available port in the appropriate range when you use the connect() system call. In this case, the source IP address is normally selected to match the network interface that provides the shortest route to the specified destination according to the routing table.
At least, that's how it works at the system call level. Some programming languages or libraries may choose to combine some of these operations into one.
To your actual question, man 7 ip says:
A TCP local socket address that has been bound is unavailable for some
time after closing, unless the SO_REUSEADDR flag has been set. Care
should be taken when using this flag as it makes TCP less reliable.
The idea is to delay the re-use of a port until any possible re-sent copies of packages that belonged to the closed connection have for sure expired on the network.
According to the bind() man page, trying to re-bind a socket that is already bound to an address will result in an EINVAL error. So "recycling" a socket using bind(socket, INADDR_ANY, 0) (after ending a connection that used SO_REUSEADDR) does not seem to be possible.
And even if that would be possible, when you're using multiple threads on a modern multi-core system, you end up (very probably) doing multiple things in parallel. A socket can be used for just one outgoing connection at a time. Each of your scan threads will need its own socket.
I start a server using sockets and want to allow clients to connect to it.
self.sock.bind(('0.0.0.0',0)) # 0.0.0.0 will allow all connections and port 0 -> os chooses a open port.
stroke_port=self.sock.getsockname()[1]
self.sock.listen(75)
self.open_port_popup(stroke_port)
Now, for other clients to connect I have port forward it manually and it works fine.
I want to do this in automated fashion. -> I try upnp.
import miniupnpc
def open_port(port_no):
'''this function opens a port using upnp'''
upnp = miniupnpc.UPnP()
upnp.discoverdelay = 10
upnp.discover()
upnp.selectigd()
# addportmapping(external-port, protocol, internal-host, internal-port, description, remote-host)
result=upnp.addportmapping(port_no, 'TCP', upnp.lanaddr, port_no, 'testing', '')
return result
It opens a port shown in the image below. But the port-forwarding list shown in the first image is empty. It doesn't work and clients can't connect. How can I fix this? What am I missing?
I think you made a mistake using upnp.lanaddr as internal-host address. upnp.lanaddr is the address of the upnp device which is your router, you want to use the local address of your server.
If needed take a look at Finding local IP addresses using Python's stdlib if you want to get your server local IP dynamically.
I think that we are missing lot of related info to know what's the main problem here. I see so many people guessing.
By the way, just editing that line
result=upnp.addportmapping(port_no, 'TCP', upnp.lanaddr, port_no, 'testing', '') to
result=upnp.addportmapping('7777', 'TCP', '192.168.1.8', '7777', 'testing', '') would tell you if it works at all.
Doing port testing from localhost it's dummy, you're not under the router at all.
Also, remember to use Try/Except blocks to tell you what's wrong on your code.
try:
print "1" + 1
except Exception as e:
print str(e)
Another way, not fashioned is to use html/web automation, even cURL to make those requests instead using uPnp, this way you don't really need to handle it.
Most of the time ISP don't allow port forwarding, and you spend hours on this trying to forward port.
I went for ngrok - it's a lightweight free of cost (for basic usage) program that tunnels the port and give its own tunneled domain which can be accessed everywhere.
this is interesting question.
from what I could summon I think
GUI shows that UPNP port forwarding rules are added.
so Most likely there is issue in UPNPC configuration.
I doubt you are doing this on Router or similar platform with X-WRT or OpenWRT
the issue I think is you can't use upnp for this or it doesn't work for some strange reason.
I suggest you try this library pytables.
I know you wanted to know why and I am working on figuring out the reason.
this is just for you to get going on this project
and for quick solution
Try this
import subprocess
p = subprocess.Popen(["iptables", "-A", "INPUT", "-p", "tcp", "-m", "tcp", "--dport", "22" , "-j", "ACCEPT"], stdout=subprocess.PIPE)
output , err = p.communicate()
print output
For a class project I'm trying to do some socket programming Python but running into a very basic issue. I can't create a TCP connection from my laptop to a lab machine. (Which I'm hoping to use as the "server") Without even getting into the scripts I have written, I've been simply trying interpreter line commands with no success. On the lab machine (kh4250-39.cselabs.umn.edu) I type the following into Python:
from socket import *
sock = socket()
sock.bind(('', 8353))
sock.listen(5)
sock.accept()
And then on my laptop I type:
from socket import *
sock = socket()
sock.connect(('kh4250-39.cselabs.umn.edu', 8353))
At which point both machines block and don't do anything until the client times out or I send a SIGINT. This code is pretty much exactly copied from examples I've found online and from Mark Lutz's book Programming Python (using '' for the server host name apparently uses the OS default and is fairly common). If I run both ends in my computer and use 'localhost' for the hostname it works fine, so I suspect it's some problem with the hostnames I'm using on one or both ends. I'm really not sure what could be going wrong on such a simple example. Does anyone have an idea?
A good way to confirm whether it's a firewall issue or not is to perform a telnet from the command-line to the destination host in question:
% telnet kh4250-39.cselabs.umn.edu 8353
Trying 128.101.38.44...
And then sometime later:
telnet: connect to address 128.101.38.44: Connection timed out
If it just hangs there at Trying and then eventually times out, chances are the connection to the remote host on that specific port is being blocked by a firewall. It could either be at the network layer (e.g. a real firewall or a router access-list) or at the host, such as iptables or other host-based filtering mechanisms.
Access to this lab host might only be available from within the lab or the campus network. Talk with your professor or a network administrator or someone "in the know" on the network to find out for sure.
Try to bind the server to 'kh4250-39.cselabs.umn.edu' instead of '':
sock.bind(('kh4250-39.cselabs.umn.edu', 8353))
If this does not work: Another reason could be a firewall blocking the port 8353....
In Hyperterminal I am able to connect to a serial port called "X64-CL_iPro_1_Serial_0" where I am able to send/receive ASCII text to a camera. However when I try to connect to the same port with pySerial, it throws an exception:
SerialException: could not open port X64-CL_iPro_1_Serial_0: [Error 2] The system cannot find the file specified.
I don't understand why Hyperterminal can detect the port and communicate with it, but Python can't. I downloaded this script from the pySerial website that displays a list of serial ports, and the only ports it came up with was COM1 and COM2, neither of which I can connect to.
My code is very simple, and looks like this:
import serial
port = "X64-CL_iPro_1_Serial_0"
ser = serial.Serial(port)
Am I doing anything wrong? Is there a way to work around this? Thanks ahead of time.
Edit:
It should also be noted that this port does not show up in the device manager, and neither does COM1 or COM2.
The problem lies in the enumeration code you linked. It is wrong in two regards:
It uses a fixed GUID_CLASS_COMPORT to enumerate. It should instead ask the GUID through SetupDiClassGuidsFromName, passing "Ports" as description of the class for which it is asking for names.
The code insists of asking for the friendly name of the port. But if the only goal is to open the device (instead of displaying to an user), it should directly access the DevicePath element, which is a weird-looking-but-perfectly-valid port name to pass to pySerial. The friendly name might even be totally missing.
aside
I'm not clear the question is about non-serial-port use through pyserial, or serial port that is not a COMX port in enumeration.
This may be a bit OT or too hard code for your use, but here goes first, using some other file in a pyserial object:
foo = serial.Serial()
peer = serial.Serial()
foo.fd, peer.fd = posix.openpty()
try: foo._isOpen = peer._isOpen = True # depending on pyserial version
except: pass
foo._reconfigurePort()
peer.setTimeout(timeout=0.1)
peer._reconfigurePort()
Regarding second, remember that ports beyond COM9 use weird windows notation \\.\COM10, perhaps that's what Hyperterminal does for you. pyserial doesn't do it, so perhaps you need to open the port something like this:
s = serial.Serial("\\\\.\\X64-CL_iPro_1_Serial_0") # also remember to escape backslash