I'm about to do a simple testimplementation to use PagSeguro (brazilian "PayPal"), and for this I downloaded their Python server to do tests on my localhost. I work on a Mac, have a XAMPP server (Apache and MySQL parts on during my process).
My problem should be very simple for someone who knows Python and sockets etc, and I did found a lot of clues in my information searches. However - with my own poor knowledge in this area, I wasn't able to put two and two together to fix it.
Short question: How do I free a socket (from Terminal) whos program quit before it had closed the socket. Alt - how do I make a Python function for me to call when I want to close the socket and stop/restart the server.
Scenario:
I start the server (In terminal with #: sudo python ./PagSeguroServer.py) and it works fine, I did some of the tests I wanted to. Then, I needed to change some settings for the server, and to make it work i need to restart the server. I resolved by closing the terminal window, but then when I reopen and type the same command to start the server again I get the "socket.error: [Errno 48] Address already in use". Ok I can see why but not how to fix, so I Google that, and find the tip to add
socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
in the code. Looked at the Python class and tried to put it there to my best knowledge (as below). However, didn't solve my problem. Since my "search and puzzle" doesnt seem to be able to help me - I'm now giving it up and posting this customized question instead!
This is part of the server code:
class SecureHTTPServer(HTTPServer):
'''Servidor HTTPS com OpenSSL.'''
def __init__(self, server_address, HandlerClass,fpem):
BaseServer.__init__(self, server_address, HandlerClass)
ctx = SSL.Context(SSL.SSLv23_METHOD)
ctx.use_privatekey_file (fpem)
ctx.use_certificate_file(fpem)
self.socket = SSL.Connection(ctx, socket.socket(self.address_family,
self.socket_type))
self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.server_bind()
self.server_activate()
....
def run(HandlerClass = HTTPSHandler, ServerClass = SecureHTTPServer):
'''Roda o servidor'''
server_address = ('', 443) # (address, port)
httpd = ServerClass(server_address, HandlerClass, fpem)
httpd.serve_forever()
if __name__ == '__main__':
run()
Note: One time I actually did managed to reopen it - it was before I added the setsockopt call, and I got the feeling that the socket had closed after a timeout - read about that somewhere. This however doesn't seem to happen again - have waited and tried several times.
Edit: In the end this is how I resolved: Needed to kill the python process that was holding the socket (see comment on jd's answer), then added the KeyboardInterrupt catch to be able to close the socket correctly. Thanks all!
If you're using Ctrl+C to close the process, you could try catching the KeyboardInterrupt exception (or even better, handle the SIGINT signal) in your code and cleanly close the socket before exiting.
Of course, this won't prevent you from killing your application in some other manner and having a stale socket, but you can try to handle those cases as well.
You could create a function that closes the socket properly and register it to be run at program termination using atexit.register(). See http://docs.python.org/library/atexit.html
Use the following command in a terminal window to kill the TCP port 28355:
npx kill-port 28355
In my python socket server script I used the following lines:
import os
os.system("npx kill-port 28355")
This command solves the "Address already in use" error.
It was the only solution that solved the error out of all the other solutions I found, like enabling the SO_REUSEADDR option.
AFAIR SO_REUSEADDR works a bit different.
But what you should start with - when you kill your working server type
netstat -tapdn |grep PORT
and replace PORT with PORT of yoiur application
Than you will get info about state of the socket....
On OSX you can kill the port(e.g. 8000) with:
kill `lsof -i :8000 | grep Python | cut -c8-11`
hth
Related
I have a basic HTML page with some javascript that I want to use to display the results of a python script at repeated steps during its execution, as well as waiting for some user trigger to continue. The webpage consists of an HTML file, a main javascript file, a secondary javascript file generated by the script containing the custom data, and references to external javascript dependencies (e.g. "https://unpkg.com/d3-graphviz#3.0.5/build/d3-graphviz.js".
To complicate things, the python script is running in a Docker container (Linux subsystem) via Visual Studio Code, so some operations like automatically opening an HTML file on the host system are difficult to accomplish (and there also seems to be a problem, with firefox at least, preventing opening local HTML files that reference other local files - see).
My initial plan was to, at each display step in the script, start a simple python http server, serve the page on localhost by piping it through a forwarded port see, wait for the user to open the localhost in their host browser, then once the page has been loaded close the server and continue with the script. I'm sure there is some fancier way of keeping the server up the entire time using threading and passing the updated data but web dev isn't really my field and this isn't exactly production code either so I felt this approach made more sense.
So assuming that this is a valid approach (and if there is a better one, I am more than happy to hear it), here's my best attempt at it (following this guide):
import http.server # Our http server handler for http requests
import socketserver # Establish the TCP Socket connections
from time import sleep # To sleep before shutting down server
PORT = 8000
count = 0
class requestHandler(http.server.SimpleHTTPRequestHandler):
def do_GET(self):
global count
if self.path == "/":
self.path == ".server/"
count += 1
return http.server.SimpleHTTPRequestHandler.do_GET(self)
# Prevent errors when restarting server and address (port) is still in use
socketserver.TCPServer.allow_reuse_address = True
handler = requestHandler
# Start server
with socketserver.TCPServer(("", PORT), handler) as httpd:
# Checking for request count, 3 expected:
# "GET /.server/ HTTP/1.1"
# "GET /.server/dots.js HTTP/1.1"
# "GET /.server/script.js HTTP/1.1"
# so accept 3 requests and end
while count < 3:
httpd.handle_request()
This is called from the main script with exec(open().read()).
Currently I get the error "NameError: name 'http' is not defined" in line 23. Weirdly, I'm sure a couple of days ago this was working fine, and I've just come back to it so I wouldn't have thought anything had changed. I've also tried a bunch of simple http.server examples with and without custom handlers, with and without the with...as block, with various errors from "Loading failed for the with source "http://localhost:8000/.server/dots.js" (in the browser F12 console) to failures to load the entire ./server directory and others. So I'm pretty much out of ideas at this point and am hoping one of you nice people can help me out.
P.S. Sorry for the long post, had to get it all out.
Okay so I seem to have found a rather inelegant but at least functional solution for the time being. For anyone reading in the future here is the server.py file:
import http.server
import socketserver
PORT = 8000
socketserver.TCPServer.allow_reuse_address = True
httpd = socketserver.TCPServer(("", PORT), http.server.SimpleHTTPRequestHandler)
for i in range(3):
httpd.handle_request()
I'll leave the question up for a little longer in case something better comes along, otherwise I'll mark this as a solution.
can you try to change:
with socketserver.TCPServer(("", PORT), handler) as httpd:
to:
with socketserver.TCPServer(("0.0.0.0", PORT), handler) as httpd:
so your application may need to listen on all interfaces?
EDIT:
as I wrote in the comments, using simple webserver such as flask might be easier for you, only to serve static files.
Here is the first post I found: https://code-maven.com/flask-serve-static-files
I am running a script that telnets to a terminal server. Occasionally the script is launched while one instance is already running, which causes the already running script to fail with
EOFError: telnet connection closed
Is there a quick and easy and pythonic way to check if the required socket is already in use on the client computer before I try to open a connection with telnetlib?
SOLUTION:
I wanted to avoid making a subprocess call but since I do not control software on the client computers, and other programs may be using the same socket, the file lock suggestion below (a good idea) wouldn't work for me. I ended up using SSutrave's suggestion. Here is my working code that uses netstat in Windows 7:
# make sure the socket is not already in use
try:
netstat = subprocess.Popen(['netstat','-nao'],stdout=subprocess.PIPE)
except:
raise ValueError("couldn't launch netstat to check sockets. exiting")
ports = netstat.communicate()[0]
if (ip + ':' + port) in ports:
print 'socket ' + ip + ':' + port + ' in use on this computer already. exiting'
return
You can check for open ports by running the following linux command netstat | grep 'port number' | wc -l by importing subprocess library in python.
There is not a standard way to know if a server has other opened connections before you attempt to connect to it. You must ask it either connecting to another service in the server that checks it, or by asking the other clients, if you know all of them.
That said, telnet servers should be able to handle more than one connection at a time, so it should not matter if there are more clients connected.
I'm trying to run Tor through python. My goal is to be able to switch exits or otherwise alter my IP from time-to-time at a time of my choosing. I've followed several tutorials and encountered several different errors.
This code prints my IP address
import requests
r = requests.get('http://icanhazip.com/')
r.content
# returns my regular IP
I've downloaded and 'installed' the Tor browser (although it seems to run from an .exe). When it is running, I can use the following code to return my Tor IP, which seems to change every 10 minutes or so, so everything seems to be working so far
import socks
import socket
socks.set_default_proxy(socks.SOCKS5, "127.0.0.1", 9150)
socket.socket = socks.socksocket
r = requests.get('http://icanhazip.com/')
print r.content
# Returns a different IP
Now, I've found instructions here on how to request a new identity programmatically, but this is where things begin to go wrong. I run the following code
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9151) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)
and I get the error "SOCKS5Error: 0x01: General SOCKS server failure" at the line that creates the controller. I thought this might be a configuration problem - the instructions say that there should be a config file called 'torrc' that contains the port numbers, apart from other things.
Eventually, in the directory C:\Users\..myname..\Desktop\Tor Browser\Browser\TorBrowser\Data\Tor I found three files, a 'torrc', a 'torrc-defaults', and a 'torrc.orig.1'. My 'torrc-defaults' looks similar to the 'torrc' shown in the documentation, my 'torrc.orig.1' is empty, and my 'torrc' has two lines of comments that look as follows with some more settings afterwards but no port settings
# This file was generated by Tor; if you edit it, comments will not be preserved
# The old torrc file was renamed to torrc.orig.1 or similar, and Tor will ignore it
...
I've tried to make sure that the two ports listed in the 'torrc-defaults' match the ports in the socks statement at the beginning and the controller statment further on. When these are 9150 and 9151 respectively I get the general SOCKS server failure listed above.
When I try and run the Controller with the wrong port (in this case, 9051 which was recommended in other posts but for me leads to Tor browser failing to load when I adjust the 'torrc-defaults') then I instead get the error message "[Errno 10061] No connection could be made because the target machine actively refused it"
Finally, when I run the Controller with Tor browser running in the background but without first running the SOCKS statement, I instead get a lot of warning text and finally an error, as shown in part below:
...
...
250 __OwningControllerProcess
DEBUG:stem:GETCONF __owningcontrollerprocess (runtime: 0.0010)
TRACE:stem:Sent to tor:
SIGNAL NEWNYM
TRACE:stem:Received from tor:
250 OK
TRACE:stem:Received from tor:
650 SIGNAL NEWNYM
TRACE:stem:Sent to tor:
QUIT
TRACE:stem:Received from tor:
250 closing connection
INFO:stem:Error while receiving a control message (SocketClosed): empty socket content
At first I thought this looked quite promising - I put in a quick check line like this both before and after the new user request
print requests.get('http://icanhazip.com/').content
It prints the IP, but unfortunately it's the same both before and afterwards.
I'm pretty lost here - any support would be appreciated!
Thanks.
Get your hashed password :
tor --hash-password my_password
Modify the configuration file :
sudo gedit /etc/tor/torrc
Add these lines
ControlPort 9051
HashedControlPassword 16:E600ADC1B52C80BB6022A0E999A7734571A451EB6AE50FED489B72E3DF
CookieAuthentication 1
Note :- The HashedControlPassword is generated from first command
Refer this link
I haven't tried it on my own because I am using mac, but I hope this helps with configuration
You can use torify to achieve your goal. for example run the python interpreter as this command "sudo torify python" and every connection will be routed through TOR.
Reference on torify: https://trac.torproject.org/projects/tor/wiki/doc/TorifyHOWTO
I'm working with a bit of Python that looks like this:
HOST = '127.0.0.1'
PORT = 43434
single = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
single.bind((HOST, PORT))
except socket.error as e:
# Print an error, exit.
While it's been functioning well in the past, we now get the error [Errno 98] Address already in use. The SIGINT handler closes the socket connection, so I'm not sure how it got in that state, but for now I'm just trying to fix it.
Both lsof and netstat say there's nothing using that port:
[$]> sudo netstat -an | grep 43434
[$]> sudo lsof -i :43434
TIME_WAIT is set to 60 seconds, according to /proc/sys/net/ipv4/tcp_fin_timeout, but the error occurs even hours after last run successfully.
I've tried (temporarily) setting REUSEADDR (via single.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)), but that appears to have no effect.
What in tarnation is going on? Will I ever be able to use this port again without having to reboot the machine?
Try this:
tcpkill -i eth0 port 43434
I wanted to add a comment but It will not let me do so. so I have to answer this way. In my experience I have had to do the following to get sockets to work with any success. I've had to explicitly set the timeout myself.
In your code before you open the socket, you should set this if you are using python version 2.3 or later. I usually start with a timeout of 30 seconds and then tune it to what works for a particular website. When I don't use this parameter i get flakey issues.
socket.setdefaulttimeout(timeout)
Other issues I've done after the open are catch some of the following exceptions:
except socket.gaierror, err:
# you might want to handle this
except socket.sslerror, err2:
# you might want to handle this if doing SSL
You could try to set SO_REUSEPORT socket options. HOwever, it only works if the conflicting socket also had the SO_REUSEPORT options set before it was bound, so you should try after a clean reboot.
However, you said your python code looks like the code you posted. Have you actually tried a very simplified version of your program to test only the binding ?
You can also try to strace your program.
For a class project I'm trying to do some socket programming Python but running into a very basic issue. I can't create a TCP connection from my laptop to a lab machine. (Which I'm hoping to use as the "server") Without even getting into the scripts I have written, I've been simply trying interpreter line commands with no success. On the lab machine (kh4250-39.cselabs.umn.edu) I type the following into Python:
from socket import *
sock = socket()
sock.bind(('', 8353))
sock.listen(5)
sock.accept()
And then on my laptop I type:
from socket import *
sock = socket()
sock.connect(('kh4250-39.cselabs.umn.edu', 8353))
At which point both machines block and don't do anything until the client times out or I send a SIGINT. This code is pretty much exactly copied from examples I've found online and from Mark Lutz's book Programming Python (using '' for the server host name apparently uses the OS default and is fairly common). If I run both ends in my computer and use 'localhost' for the hostname it works fine, so I suspect it's some problem with the hostnames I'm using on one or both ends. I'm really not sure what could be going wrong on such a simple example. Does anyone have an idea?
A good way to confirm whether it's a firewall issue or not is to perform a telnet from the command-line to the destination host in question:
% telnet kh4250-39.cselabs.umn.edu 8353
Trying 128.101.38.44...
And then sometime later:
telnet: connect to address 128.101.38.44: Connection timed out
If it just hangs there at Trying and then eventually times out, chances are the connection to the remote host on that specific port is being blocked by a firewall. It could either be at the network layer (e.g. a real firewall or a router access-list) or at the host, such as iptables or other host-based filtering mechanisms.
Access to this lab host might only be available from within the lab or the campus network. Talk with your professor or a network administrator or someone "in the know" on the network to find out for sure.
Try to bind the server to 'kh4250-39.cselabs.umn.edu' instead of '':
sock.bind(('kh4250-39.cselabs.umn.edu', 8353))
If this does not work: Another reason could be a firewall blocking the port 8353....