I am running a script that telnets to a terminal server. Occasionally the script is launched while one instance is already running, which causes the already running script to fail with
EOFError: telnet connection closed
Is there a quick and easy and pythonic way to check if the required socket is already in use on the client computer before I try to open a connection with telnetlib?
SOLUTION:
I wanted to avoid making a subprocess call but since I do not control software on the client computers, and other programs may be using the same socket, the file lock suggestion below (a good idea) wouldn't work for me. I ended up using SSutrave's suggestion. Here is my working code that uses netstat in Windows 7:
# make sure the socket is not already in use
try:
netstat = subprocess.Popen(['netstat','-nao'],stdout=subprocess.PIPE)
except:
raise ValueError("couldn't launch netstat to check sockets. exiting")
ports = netstat.communicate()[0]
if (ip + ':' + port) in ports:
print 'socket ' + ip + ':' + port + ' in use on this computer already. exiting'
return
You can check for open ports by running the following linux command netstat | grep 'port number' | wc -l by importing subprocess library in python.
There is not a standard way to know if a server has other opened connections before you attempt to connect to it. You must ask it either connecting to another service in the server that checks it, or by asking the other clients, if you know all of them.
That said, telnet servers should be able to handle more than one connection at a time, so it should not matter if there are more clients connected.
Related
I'm trying to listen on a certain ip address so I use a socket but when I specify the udp_IP, I get "address already in use".
Is there a way I can reuse the ip address for both my application and my listener?
I don't have a very good networking knowledge, any help would be great.
Usually, you cannot use the same port or socket with different programs. If what you want is stopping the listener and start the application in that port, open a terminal and type:
sudo netstat -nlp | grep:yourSocketNumberGoesHere
This is to get the process ID of the program running. Then just use kill followed by the process ID you get to kill the program. Then, try starting your application.
If what you want is using 2 different programs in the same socket or port, you can't, without maybe doing something more complex (which I do not know).
You could use 2 ports that communicate with each other if you want the 2 programs to interact.
I have a python client that needs to talk to a remote server I manage. They communicate using zeromq. When I tested the client/server locally everything worked. But now I have the client and server deployed on the cloud, each using a different provider. My question is, what's the simplest way (that is safe) to make the connection? I'm assuming I can't pass the password over, and even if I could I'm guessing there are safer alternatives.
I know how to set an ssh connection without a password using ssh-keygen. Would that work? Would the client need to make an ssh connection with the server before sending the tcp req? If there's a python library that helps with this it'd be a big help.
Thanks!
Update:
So more than 24 hours passed and no one replied/answered. I think I'm getting closer to solve this, but not quite there yet. I added my client's key to .ssh/authorized_key on the server, and now I can ssh from the client to the server without a password. Next, I followed this post about "Tunneling PyZMQ Connections with SSH". Here's what I have in my client code:
1 context = zmq.Context()
2 socket = context.socket(zmq.REQ)
3 socket.connect("tcp://localhost:5555")
4 ssh.tunnel_connection(socket, "tcp://locahost:5555", "myuser#remote-server-ip:5555")
5 socket.send_string(some_string)
6 reply = socket.recv()
This doesn't work. I don't really understand lines 3 & 4 and I assume I do something wrong there. Also, my server (hosted on linode) has a "Default Gateway" IP and a "Public IP" -- in the tunnel connection I only specify the public ip, which is also the ip I use to ssh to the machine.
Indeed, ZMQ way is - tunnelling connection with the SSH. Your example is exactly what needs to be done, except that one should either use connect or tunnel_connection, not both.
Also, when specifying server to connect to, make sure to define the SSH port, not the ZMQ REP socket port. That is, instead of myuser#remote-server-ip:5555 you might try myuser#remote-server-ip or myuser#remote-server-ip:22.
import zmq
import zmq.ssh
context = zmq.Context()
socket = context.socket(zmq.REQ)
zmq.ssh.tunnel_connection(socket, "tcp://locahost:5555", "myuser#remote-server-ip")
socket.send(b"Hello")
reply = socket.recv()
Finally, make sure you've installed either pexpect or paramiko - they will do the tunnelling actually. Note that if you're using Windows, paramiko is the only solution which will work - pexpect openssh tunnelling won't work on Windows.
If you use paramiko instead of pexpect, make sure to set paramiko=True in the tunnel_connection arguments.
I have found ssh in Python to be iffy at best, even with paramiko and fabric libraries, so to debug, you might try setting up a tunnel separately, just to see if that's the issue with the broken connection.
For example:
ssh myuser#remote-server-ip -L 5050:localhost:5555 -N
This says: connect to myuser#remote-server-ip, and whenever I request a connection to localhost:5050 on my machine, forward it across the ssh connection so that the server at remote-server-ip thinks it's receiving a connection from localhost:5555.
-L constructs the tunnel, and -N means don't do anything else on the connection.
With that running in another shell, e.g., a different Terminal window, on your local development machine, try to connect to a zeromq server at localhost:5050, which will actually be the zeromq running on the remote server.
You could use 5555:localhost:5555 in the ssh command above, but I find that can be confusing and often conflicts with a local copy of the same service.
I'm working with a bit of Python that looks like this:
HOST = '127.0.0.1'
PORT = 43434
single = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
single.bind((HOST, PORT))
except socket.error as e:
# Print an error, exit.
While it's been functioning well in the past, we now get the error [Errno 98] Address already in use. The SIGINT handler closes the socket connection, so I'm not sure how it got in that state, but for now I'm just trying to fix it.
Both lsof and netstat say there's nothing using that port:
[$]> sudo netstat -an | grep 43434
[$]> sudo lsof -i :43434
TIME_WAIT is set to 60 seconds, according to /proc/sys/net/ipv4/tcp_fin_timeout, but the error occurs even hours after last run successfully.
I've tried (temporarily) setting REUSEADDR (via single.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)), but that appears to have no effect.
What in tarnation is going on? Will I ever be able to use this port again without having to reboot the machine?
Try this:
tcpkill -i eth0 port 43434
I wanted to add a comment but It will not let me do so. so I have to answer this way. In my experience I have had to do the following to get sockets to work with any success. I've had to explicitly set the timeout myself.
In your code before you open the socket, you should set this if you are using python version 2.3 or later. I usually start with a timeout of 30 seconds and then tune it to what works for a particular website. When I don't use this parameter i get flakey issues.
socket.setdefaulttimeout(timeout)
Other issues I've done after the open are catch some of the following exceptions:
except socket.gaierror, err:
# you might want to handle this
except socket.sslerror, err2:
# you might want to handle this if doing SSL
You could try to set SO_REUSEPORT socket options. HOwever, it only works if the conflicting socket also had the SO_REUSEPORT options set before it was bound, so you should try after a clean reboot.
However, you said your python code looks like the code you posted. Have you actually tried a very simplified version of your program to test only the binding ?
You can also try to strace your program.
For a class project I'm trying to do some socket programming Python but running into a very basic issue. I can't create a TCP connection from my laptop to a lab machine. (Which I'm hoping to use as the "server") Without even getting into the scripts I have written, I've been simply trying interpreter line commands with no success. On the lab machine (kh4250-39.cselabs.umn.edu) I type the following into Python:
from socket import *
sock = socket()
sock.bind(('', 8353))
sock.listen(5)
sock.accept()
And then on my laptop I type:
from socket import *
sock = socket()
sock.connect(('kh4250-39.cselabs.umn.edu', 8353))
At which point both machines block and don't do anything until the client times out or I send a SIGINT. This code is pretty much exactly copied from examples I've found online and from Mark Lutz's book Programming Python (using '' for the server host name apparently uses the OS default and is fairly common). If I run both ends in my computer and use 'localhost' for the hostname it works fine, so I suspect it's some problem with the hostnames I'm using on one or both ends. I'm really not sure what could be going wrong on such a simple example. Does anyone have an idea?
A good way to confirm whether it's a firewall issue or not is to perform a telnet from the command-line to the destination host in question:
% telnet kh4250-39.cselabs.umn.edu 8353
Trying 128.101.38.44...
And then sometime later:
telnet: connect to address 128.101.38.44: Connection timed out
If it just hangs there at Trying and then eventually times out, chances are the connection to the remote host on that specific port is being blocked by a firewall. It could either be at the network layer (e.g. a real firewall or a router access-list) or at the host, such as iptables or other host-based filtering mechanisms.
Access to this lab host might only be available from within the lab or the campus network. Talk with your professor or a network administrator or someone "in the know" on the network to find out for sure.
Try to bind the server to 'kh4250-39.cselabs.umn.edu' instead of '':
sock.bind(('kh4250-39.cselabs.umn.edu', 8353))
If this does not work: Another reason could be a firewall blocking the port 8353....
I'm about to do a simple testimplementation to use PagSeguro (brazilian "PayPal"), and for this I downloaded their Python server to do tests on my localhost. I work on a Mac, have a XAMPP server (Apache and MySQL parts on during my process).
My problem should be very simple for someone who knows Python and sockets etc, and I did found a lot of clues in my information searches. However - with my own poor knowledge in this area, I wasn't able to put two and two together to fix it.
Short question: How do I free a socket (from Terminal) whos program quit before it had closed the socket. Alt - how do I make a Python function for me to call when I want to close the socket and stop/restart the server.
Scenario:
I start the server (In terminal with #: sudo python ./PagSeguroServer.py) and it works fine, I did some of the tests I wanted to. Then, I needed to change some settings for the server, and to make it work i need to restart the server. I resolved by closing the terminal window, but then when I reopen and type the same command to start the server again I get the "socket.error: [Errno 48] Address already in use". Ok I can see why but not how to fix, so I Google that, and find the tip to add
socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
in the code. Looked at the Python class and tried to put it there to my best knowledge (as below). However, didn't solve my problem. Since my "search and puzzle" doesnt seem to be able to help me - I'm now giving it up and posting this customized question instead!
This is part of the server code:
class SecureHTTPServer(HTTPServer):
'''Servidor HTTPS com OpenSSL.'''
def __init__(self, server_address, HandlerClass,fpem):
BaseServer.__init__(self, server_address, HandlerClass)
ctx = SSL.Context(SSL.SSLv23_METHOD)
ctx.use_privatekey_file (fpem)
ctx.use_certificate_file(fpem)
self.socket = SSL.Connection(ctx, socket.socket(self.address_family,
self.socket_type))
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.server_bind()
self.server_activate()
....
def run(HandlerClass = HTTPSHandler, ServerClass = SecureHTTPServer):
'''Roda o servidor'''
server_address = ('', 443) # (address, port)
httpd = ServerClass(server_address, HandlerClass, fpem)
httpd.serve_forever()
if __name__ == '__main__':
run()
Note: One time I actually did managed to reopen it - it was before I added the setsockopt call, and I got the feeling that the socket had closed after a timeout - read about that somewhere. This however doesn't seem to happen again - have waited and tried several times.
Edit: In the end this is how I resolved: Needed to kill the python process that was holding the socket (see comment on jd's answer), then added the KeyboardInterrupt catch to be able to close the socket correctly. Thanks all!
If you're using Ctrl+C to close the process, you could try catching the KeyboardInterrupt exception (or even better, handle the SIGINT signal) in your code and cleanly close the socket before exiting.
Of course, this won't prevent you from killing your application in some other manner and having a stale socket, but you can try to handle those cases as well.
You could create a function that closes the socket properly and register it to be run at program termination using atexit.register(). See http://docs.python.org/library/atexit.html
Use the following command in a terminal window to kill the TCP port 28355:
npx kill-port 28355
In my python socket server script I used the following lines:
import os
os.system("npx kill-port 28355")
This command solves the "Address already in use" error.
It was the only solution that solved the error out of all the other solutions I found, like enabling the SO_REUSEADDR option.
AFAIR SO_REUSEADDR works a bit different.
But what you should start with - when you kill your working server type
netstat -tapdn |grep PORT
and replace PORT with PORT of yoiur application
Than you will get info about state of the socket....
On OSX you can kill the port(e.g. 8000) with:
kill `lsof -i :8000 | grep Python | cut -c8-11`
hth