We Keep Coding

http.server not serving pages as per example from python.org

I'm trying to start a simple dir server on a local network But I am getting this error
Error response
Error code: 501
Message: Unsupported method ('GET').
Error code explanation: HTTPStatus.NOT_IMPLEMENTED - Server does not
support this operation.
This is the example given at https://docs.python.org/3/library/http.server.html If I run it from the command line it works python3 -m http.server. I need to control this server over time so I need to turn it on for a while and turn it off automatically
from http.server import BaseHTTPRequestHandler, HTTPServer
def run(server_class=HTTPServer, handler_class=BaseHTTPRequestHandler):
server_address = ('0.0.0.0', 8000)
httpd = server_class(server_address, handler_class)
httpd.serve_forever()

The answer is in the documentation that you linked to:
The HTTPServer must be given a RequestHandlerClass on
instantiation, of which this module provides three different variants:
class http.server.BaseHTTPRequestHandler(request, client_address, server)
This class is used to handle the HTTP requests that arrive at the
server. By itself, it cannot respond to any actual HTTP requests; it
must be subclassed to handle each request method (e.g. GET or
POST). ...
For your case you should use http.server.SimpleHTTPRequestHandler instead:
class http.server.SimpleHTTPRequestHandler(request, client_address, server)
This class serves files from the current directory and below, directly
mapping the directory structure to HTTP requests.

Attempting to use LetsEncrypt to run SSL-wrapped BaseHTTPServer in Python fails

I had a working HTTP server using BaseHTTPServer in Python, so I attempted to add an SSL cert to allow for https using LetsEncrypt, and now it won't serve any files or respond. No exceptions or errors thrown, nor will it serve any content.
server_address = ('0.0.0.0', 80)
httpd = HTTPServer(server_address, MyHandler)
# I can comment out the following line and it'll work
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile=ssl_key, certfile=ssl_cert, server_side=True)
httpd.serve_forever()
#ssl_key = '/etc/letsencrypt/live/example.com/privkey.pem'
#ssl_cert = '/etc/letsencrypt/live/example.com/fullchain.pem'
Where MyHandler is the following:
class MyHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(204)
self.send_header("Content-Type", "text/html")
self.end_headers()
return
def do_POST(self):
self.send_response(204)
self.send_header("Content-Type", "text/html")
self.end_headers()
return
Attempting to access the site via web browser from https://example.com returns a standard no-response "Server not found".
I followed the following instructions to generate a certificate using LetsEncrypt: https://certbot.eff.org/#ubuntuxenial-other
sudo apt-get install letsencrypt
Followed by:
letsencrypt certonly --standalone -d example.com
Is there any way I can easily figure out what the problem is here? Using Python 3.5. Happy to provide additional info if needed.

server_address = ('0.0.0.0', 80)
Attempting to access the site via web browser from https://example.com returns a standard no-response "Server not found".
https://host without explicit port specification means that the server is accessed on the default port for the https protocol, which is 443. But, you have setup your server to use port 80 in server_address.
There are two ways to fix this: either explicitly specify a non-standard port for https in the URL, i.e. https://host:80 or change the port in server_address from 80 to 443. The last option is probably the better one.

How to redirect HTTP requests on a Python App on Bluemix to HTTPS only?

I have python app on Bluemix and would like to make it accessible over https only. By default I can connect both via http and https. Want to restrict access via https only. So what is the best way to disable http access, or redirect request to https only?

As ralphearle mentioned in his answer, Bluemix proxy server terminates the SSL, so you can look into the X-Forwarded-Proto header to find out if request comes from http or https.
See below a simple example based on the Python starter code from Bluemix. I added the RedirectHandler class to check for the X-Forwarded-Proto header value and redirects the request to https if it not https.
import os
try:
from SimpleHTTPServer import SimpleHTTPRequestHandler as Handler
from SocketServer import TCPServer as Server
except ImportError:
from http.server import SimpleHTTPRequestHandler as Handler
from http.server import HTTPServer as Server
class RedirectHandler(Handler):
def do_HEAD(self):
if ('X-Forwarded-Proto' in self.headers and
self.headers['X-Forwarded-Proto'] != 'https'):
self.send_response(301)
self.send_header("Location", 'https://' + self.headers['Host'] + self.path)
self.end_headers()
def do_GET(self):
self.do_HEAD()
Handler.do_GET(self)
# Read port selected by the cloud for our application
PORT = int(os.getenv('PORT', 8000))
# Change current directory to avoid exposure of control files
os.chdir('static')
httpd = Server(("", PORT), RedirectHandler)
try:
print("Start serving at port %i" % PORT)
httpd.serve_forever()
except KeyboardInterrupt:
pass
httpd.server_close()

The Bluemix proxy server terminates the SSL, so that all traffic looks like HTTP to your app. However, the proxy also adds a special HTTP header named $WSSC with a value that can be either http or https. Check this header and, if the value is set to http, then change it to https.
As Adam pointed out in his comment, the IBM forum has further discussion of this approach: https://developer.ibm.com/answers/questions/16016/how-do-i-enforce-ssl-for-my-bluemix-application.html

How to communicate with a HTTP/HTTPS server via Python sockets?

I'm trying to create a TCP socket server in Python that after receiving a string of bytes from a client passes the received data(without knowing what it's actually inside, assuming it's a valid HTTP request) to a HTTP or HTTPS proxy and waits for results, my code looks like this:
import socket
def test(host, port):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((socket.gethostbyname(host), int(port))
msg = """GET / HTTP/1.1
Host: www.bing.com
User-Agent: Firefox
"""
sent_count = sock.send(msg)
recv_value = sock.recv(2048)
print('recvieved:',)
print str(recv_value)
pass
if __name__ == '__main__':
test(host='x.x.x.xx', port='80') # a https proxy server
test(host='yy.yy.yy.yy', port='80') # a http proxy server
But when i connect to the HTTP proxy server it returns something like:
HTTP/1.1 404 Not Found
And when i connect to the HTTPS proxy server it shows something like:
HTTP/1.0 400 Bad Request
So wanted to ask if anybody know how could i send HTTP requests to HTTP/HTTPS servers via sockets in Python? or how can i redirect arbitrary strings of data toward HTTP/HTTPS proxy servers in general in Python using sockets?, any suggestions are very much appreciated, thanks in advance.

Convert HTTP Proxy to HTTPS Proxy in Twisted

Recently I have been playing around with the HTTP Proxy in twisted. After much trial and error I think I finally I have something working. What I want to know though, is how, if it is possible, do I expand this proxy to also be able to handle HTTPS pages? Here is what I've got so far:
from twisted.internet import reactor
from twisted.web import http
from twisted.web.proxy import Proxy, ProxyRequest, ProxyClientFactory, ProxyClient
class HTTPProxyClient(ProxyClient):
def handleHeader(self, key, value):
print "%s : %s" % (key, value)
ProxyClient.handleHeader(self, key, value)
def handleResponsePart(self, buffer):
print buffer
ProxyClient.handleResponsePart(self, buffer)
class HTTPProxyFactory(ProxyClientFactory):
protocol = HTTPProxyClient
class HTTPProxyRequest(ProxyRequest):
protocols = {'http' : HTTPProxyFactory}
def process(self):
print self.method
for k,v in self.requestHeaders.getAllRawHeaders():
print "%s : %s" % (k,v)
print "\n \n"
ProxyRequest.process(self)
class HTTPProxy(Proxy):
requestFactory = HTTPProxyRequest
factory = http.HTTPFactory()
factory.protocol = HTTPProxy
reactor.listenSSL(8001, factory)
reactor.run()
As this code demonstrates, for the sake of example for now I am just printing out whatever is going through the connection. Is it possible to handle HTTPS with the same classes? If not, how should I go about implementing such a thing?

If you want to connect to an HTTPS website via an HTTP proxy, you need to use the CONNECT HTTP verb (because that's how a proxy works for HTTPS). In this case, the proxy server simply connects to the target server and relays whatever is sent by the server back to the client's socket (and vice versa). There's no caching involved in this case (but you might be able to log the hosts you're connecting to).
The exchange will look like this (client to proxy):
C->P: CONNECT target.host:443 HTTP/1.0
C->P:
P->C: 200 OK
P->C:
After this, the proxy simply opens a plain socket to the target server (no HTTP or SSL/TLS yet) and relays everything between the initial client and the target server (including the TLS handshake that the client initiates). The client upgrades the existing socket it has to the proxy to use TLS/SSL (by starting the SSL/TLS handshake). Once the client has read the '200' status line, as far as the client is concerned, it's as if it had made the connection to the target server directly.

I'm not sure about twisted, but I want to warn you that if you implement a HTTPS proxy, a web browser will expect the server's SSL certificate to match the domain name in the URL (address bar). The web browser will issue security warnings otherwise.
There are ways around this, such as generating certificates on the fly, but you'd need the root certificate to be trusted on the browser.