I have a rest server implemented by python with flask. And implement an api to restart ntpd. The code test_flask.py:
from flask import Flask
import subprocess
import logging
import sys
app = Flask(__name__)
def run_shell_cmd(cmd):
logging.info("run cmd: %s", cmd)
try:
rc = subprocess.call(cmd, shell=True)
if rc != 0:
logging.error("Fail to run %s , rc: %s" % (cmd, rc))
except OSError as e:
logging.error("Fail to run cmd: %s" % e)
return rc
#app.route("/restart_ntpd")
def restart():
run_shell_cmd("service ntpd restart")
return "Success!"
if __name__ == "__main__":
LOG_FORMAT = '%(asctime)s, %(levelname)s, %(filename)s:%(lineno)d, %(message)s'
logging.basicConfig(
format=LOG_FORMAT,
level=logging.INFO,
stream=sys.stdout,
)
app.run()
Then I operated as follow:
start flask server: python test_flask.py
curl "http://localhost:5000/restart_ntpd. Then ntpd restart & return "success"
stop flask server: just use Ctrl+c to stop
start flask server again, it will raise a exception:
socket.error: [Errno 98] Address already in use.
use sh $ netstat -ntlp | grep 5000, the port was deforced by ntpd
I think the ntpd will use port 123 in default. In my scene, why the port 5000 is deforced by ntpd? Is it the problem of flask?
ntpd is not listening on TCP port 5000 itself, it's the environment where it is running - the process.
And that process is a child of your Flask server process, which opens a socket listening on TCP port 5000.
This socket is inherited in the child process, and since the ntpd process is a long-running one, it continues running with the socket it inherits from you, occupying the port 5000.
Check how to deal with Python BaseHTTPServer killed,but the port is still be occupied? on how to prevent children processes from inheriting the socket.
Of course, first you have to find a way to customize the way Flask start the server.
Related
I have a CherryPy script that I frequently run to start a server. Today I was having to start and stop it a few times to fix some bugs in a config file, and I guess the socket didn't close all the way because when I tried to start it up again I got this issue:
[23/Mar/2015:14:08:00] ENGINE Listening for SIGHUP.
[23/Mar/2015:14:08:00] ENGINE Listening for SIGTERM.
[23/Mar/2015:14:08:00] ENGINE Listening for SIGUSR1.
[23/Mar/2015:14:08:00] ENGINE Bus STARTING
CherryPy Checker:
The Application mounted at '' has an empty config.
[23/Mar/2015:14:08:00] ENGINE Started monitor thread 'Autoreloader'.
[23/Mar/2015:14:08:00] ENGINE Started monitor thread '_TimeoutMonitor'.
[23/Mar/2015:14:08:00] ENGINE Error in HTTP server: shutting down
Traceback (most recent call last):
File "/home/andrew/virtualenvs/mikernels/lib/python2.7/site-packages/cherrypy/process/servers.py", line 188, in _start_http_thread
self.httpserver.start()
File "/home/andrew/virtualenvs/mikernels/lib/python2.7/site-packages/cherrypy/wsgiserver/wsgiserver2.py", line 1848, in start
raise socket.error(msg)
error: No socket could be created
I edited CherryPy's wsgiserver2.py to see the details of the socket.error and error.strerror was
98 (98, 'Address already in use') Address already in use
Meanwhile my socket is constructed as:
af = 2
socktype = 1
proto = 6
canonname = ''
sa = ('0.0.0.0', 2112)
self.bind(af, socktype, proto)
(that's not exact code but that's what the values are when the error is fired)
I checked netstat and didn't see anything listening on port 2112, what could be causing the problem and how can I go about diagnosing it?
Thanks!
You can try the following
from socket import *
sock=socket()
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
# then bind
From the docs:
The SO_REUSEADDR flag tells the kernel to reuse a local socket in TIME_WAIT state, without waiting for its natural timeout to expire.
Here's the complete explanation:
Running an example several times with too small delay between executions, could lead to this error:
socket.error: [Errno 98] Address already in use
This is because the previous execution has left the socket in a TIME_WAIT state, and can’t be immediately reused.
There is a socket flag to set, in order to prevent this, socket.SO_REUSEADDR:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((HOST, PORT))
You could find the process and kill it by doing:
ps aux | grep python
, finding the process ID, and stopping it manually by doing:
sudo kill -9 PID
replacing PID with your PID.
I often have to do this while testing with Flask/CherryPy. Would be interested to see if there's an easier way (for e.g. to prevent it in the first place)
Much more easier to do it by:
Check the PID(:5000 is the host since I've been running on 127.0.0.1:5000):$ lsof -i :5000Then kill it:$ sudo kill -9 PID
I wrote this small piece of code:
from flask import Flask
from flask_restful import Resource, Api
import subprocess
app = Flask(__name__)
api = Api(app)
class SlackReport(Resource):
def get(self):
subprocess.Popen(['python', 'ingestion_ice3_check.py'])
return 'Request submitted'
api.add_resource(SlackReport, '/slackReport')
if __name__ == '__main__':
app.run(debug=True)
If I execute:
curl --ipv4 http://127.0.0.1:5000/slackReport
the server sends the response but if I execute the same command from another machine in the same LAN I receive a 'connection refused' message
$ curl -X GET http://10.113.12.20:5000/slackReport
curl: (7) Failed to connect to 10.113.12.20 port 5000: Connection refused
If I request the resource from localhost I can see the request from the debug console of the server
127.0.0.1 - - [28/Sep/2017 18:15:02] "GET /slackReport HTTP/1.1" 200 -
However if I request the same resource from a remote machine I can see the packet on the port but the flask server doesn't receive anything.
$ sudo tcpdump -i ens192 port 5000
18:17:53.132514 IP 10.113.12.25.37096 > tl020dash.commplex-main: Flags [S], seq 3147843104, win 14600, options [mss 1460,sackOK,TS val 822032144 ecr 0,nop,wscale 7], length 0
This is the output of netstat command
sudo netstat -tnlp | grep :5000
tcp 0 0 127.0.0.1:5000 0.0.0.0:* LISTEN 21872/python
Your are running your service on localhost (127.0.0.1) so only your computer has access to it. If you want to run that Flask service so other stuff on your network can access it you can set some keyword args when you run your app. You might need to run this as an elevated user so it can bind to that network port (10.113.12.20:5000).
if __name__ == '__main__':
app.run(debug=True, host='10.113.12.20', port=5000)
I'm trying to implement a Python server supporting both HTTP and HTTPS based in BaseHTTPServer. This is my code:
server_class = BaseHTTPServer.HTTPServer
# Configure servers
httpd = server_class(("0.0.0.0", 1044), MyHandler)
httpsd = server_class(("0.0.0.0", 11044), MyHandler)
httpsd.socket = ssl.wrap_socket(httpsd.socket, keyfile="/tmp/localhost.key", certfile="/tmp/localhost.crt", server_side=True)
# Run the servers
try:
httpd.serve_forever()
httpsd.serve_forever()
except KeyboardInterrupt:
print("Closing the server...")
httpd.server_close()
httpsd.server_close()
So, HTTP runs in port 1044 and HTTPS runs in 11044. The MyHandler class is omitted for the sake of briefness.
Using that code, when I send requests to HTTP port (e.g. curl http://localhost:1044/path) it works. However, when I send requests to the HTTPS port (e.g. curl -k https://localhost:11104/path) the server never responses, i.e. the curl terminal gets hanged.
I have observed that if I comment the line starting the HTTP server (i.e. httpd.server_forever()) then the HTTPS server works,.i.e. curl -k https://localhost:11104/path works. Thus, I guess that I'm doing something wrong which is precluding not being able to set both servers at the same time.
Any help is appreciated!
Following feedback comments, I have refactored the code in a multithread way and now it works as expected.
def init_server(http):
server_class = BaseHTTPServer.HTTPServer
if http:
httpd = server_class(("0.0.0.0", 1044), MyHandler)
else: # https
httpd = server_class(("0.0.0.0", 11044), MyHandler)
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile="/tmp/localhost.key", certfile="/tmp/localhost.crt", server_side=True)
httpd.serve_forever()
httpd.server_close()
VERBOSE = "True"
thread.start_new_thread(init_server, (True, ))
thread.start_new_thread(init_server, (False, ))
while 1:
time.sleep(10)
I am running my application with Windows Azure on a Virtual Machine with ubuntu 14.04 lts. I am running my Django application through WSGI on Apache.
Previously i ran django locally with the command "python manage.py runserver", and every thing worked fine when my website connected to my sockets.py file from website.html.
I am running Django through Apache on public ip port 80
I am running the sockets.py separately in a terminal through Putty
I am reading the error through the Google Chrome console
Suddenly this error occurs: Failed to execute 'send' on 'WebSocket': Still in CONNECTING state. whenever i try to connect to the socket.
After a while my page response with: failed: Error in connection establishment: net::ERR_CONNECTION_TIMED_OUT
website.html js:
ws = new WebSocket("ws://10.77.22.74:1339/ws");
function load_all() {
target = "load_all"
ws.send(target)
}
ws.onmessage = function(evt) {
console.log(evt.data)
}
The ip is my internal ip on my virtual machine.
sockets.py:
import tornado.httpserver
import tornado.websocket
import tornado.ioloop
import tornado.web
class WSHandler(tornado.websocket.WebSocketHandler):
def check_origin(self, origin):
return True
def open(self):
print 'new connection'
def on_message(self, message):
self.write_message(message)
def on_close(self):
print 'connection closed'
application = tornado.web.Application([
(r'/ws', WSHandler),
])
if __name__ == "__main__":
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(1339)
tornado.ioloop.IOLoop.instance().start()
I have tried to change the http_server.listen(1339) to http_server.listen(1339, adress='10.77.22.74') (sockets.py)
I have tried using my public ip and opening a port through tcp (endpoints) and adjusting the scripts after that (sockets.py & website.html)
I have tried running with the localhost & 127.0.0.1 (sockets.py & website.html)
I have tried with ws & wss
I still get the error for some reason, do i need to give Apache (www-data) some permissions to connect to the sockets.py?
Which IP should i use, both on socket.py and website.htm?
Solved
Turn out Tornado is listening to all IP:s if you not specify the adress in the socket server. I opened a port on the public IP and used that IP for my javascript socket connection.
I have a vpn connection and when I'm running python -m SimpleHTTPServer, it serves on 0.0.0.0:8000, which means it can be accessed via localhost and via my real ip.
I don't want robots to scan me and interested that the server will be accessed only via localhost.
Is it possible?
python -m SimpleHTTPServer 127.0.0.1:8000 # doesn't work.
Any other simple http server which can be executed instantly using the command line is also welcome.
In Python versions 3.4 and higher, the http.server module accepts a bind parameter.
According to the docs:
python -m http.server 8000
By default, server binds itself to all interfaces. The option
-b/--bind specifies a specific address to which it should bind. For example, the following command causes the server to bind to localhost
only:
python -m http.server 8000 --bind 127.0.0.1
New in version 3.4: --bind argument was introduced.
As #sberry explained, simply doing it by using the nice python -m ... method won't be possible, because the IP address is hardcoded in the implementation of the BaseHttpServer.test function.
A way of doing it from the command line without writing code to a file first would be
python -c 'import BaseHTTPServer as bhs, SimpleHTTPServer as shs; bhs.HTTPServer(("127.0.0.1", 8888), shs.SimpleHTTPRequestHandler).serve_forever()'
If that still counts as a one liner depends on your terminal width ;-) It's certainly not very easy to remember.
If you read the source you will see that only the port can be overridden on the command line. If you want to change the host it is served on, you will need to implement the test() method of the SimpleHTTPServer and BaseHTTPServer yourself. But that should be really easy.
Here is how you can do it, pretty easily:
import sys
from SimpleHTTPServer import SimpleHTTPRequestHandler
import BaseHTTPServer
def test(HandlerClass=SimpleHTTPRequestHandler,
ServerClass=BaseHTTPServer.HTTPServer):
protocol = "HTTP/1.0"
host = ''
port = 8000
if len(sys.argv) > 1:
arg = sys.argv[1]
if ':' in arg:
host, port = arg.split(':')
port = int(port)
else:
try:
port = int(sys.argv[1])
except:
host = sys.argv[1]
server_address = (host, port)
HandlerClass.protocol_version = protocol
httpd = ServerClass(server_address, HandlerClass)
sa = httpd.socket.getsockname()
print "Serving HTTP on", sa[0], "port", sa[1], "..."
httpd.serve_forever()
if __name__ == "__main__":
test()
And to use it:
> python server.py 127.0.0.1
Serving HTTP on 127.0.0.1 port 8000 ...
> python server.py 127.0.0.1:9000
Serving HTTP on 127.0.0.1 port 9000 ...
> python server.py 8080
Serving HTTP on 0.0.0.0 port 8080 ...