I have a python file running a server:
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
import json
class S(BaseHTTPRequestHandler):
def _set_headers(self):
self.send_response(200)
self.send_header('Content-type', 'application/json')
self.end_headers()
def do_GET(self):
self._set_headers()
with open('files.json') as myfile:
self.wfile.write(myfile.read())
....
def run(server_class=HTTPServer, handler_class=S, port=8080):
server_address = ('', port)
httpd = server_class(server_address, handler_class)
print 'Starting httpd...'
httpd.serve_forever()
And another doing a get request:
import requests
SESSION = requests.Session()
SESSION.trust_env = False
url = "http://localhost:8080"
response = SESSION.get(url)
data = response.json()
This works fine when I run the two normally. However when I containerise the app (not the server), requests throws this error at me:
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ec03e5ed0>: Failed to establish a new connection: [Errno 111] Connection refused',))
I assume this is because docker somehow prevents me from connecting to my server on localhost:8080?
Thanks for you help ☺
A container is meant to be an isolated environment. It's a "jail" for running processes protecting the host os. Think of containers as a virtual machine with its own ip address. localhost would be the container trying to connect to itself.
I'm guessing you could contact port 8080 on the host using its public ip address.
It would make more sense to run the server in docker. Then you can map a port in the server container to a port on your host. Then your client would work as expected.
The two most common ways a container can interact with the host os is:
When mapping network ports from the container to to host
When mapping files as volumes from the host into the container
It's fairly restrictive for very good reasons.
Related
I have a autobahn twisted websocket running in python which is working in a dev vm correctly but I have been unable to get working when the server is running in openshift.
Here is the shortened code which works for me in a vm.
from autobahn.twisted.websocket import WebSocketServerProtocol, WebSocketServerFactory, listenWS
from autobahn.twisted.resource import WebSocketResource
class MyServerProtocol(WebSocketServerProtocol):
def onConnect(self, request):
stuff...
def onOpen(self):
stuff...
def onMessage(self,payload):
stuff...
factory = WebSocketServerFactory(u"ws://0.0.0.0:8080")
factory.protocol = MyServerProtocol
resource = WebSocketResource(factory)
root = File(".")
root.putChild(b"ws", resource)
site = Site(root)
reactor.listenTCP(8080, site)
reactor.run()
The connection part of the client is as follows:
var wsuri;
var hostname = window.document.location.hostname;
wsuri = "ws://" + hostname + ":8080/ws";
if ("WebSocket" in window) {
sock = new WebSocket(wsuri);
} else if ("MozWebSocket" in window) {
sock = new MozWebSocket(wsuri);
} else {
log("Browser does not support WebSocket!");
window.location = "http://autobahn.ws/unsupportedbrowser";
}
The openshift configuration is as follows:
1 pod running with app.py listening on port 8080
tls not enabled
I have a non-tls route 8080 > 8080.
Firefox gives the following message in the console:
Firefox can’t establish a connection to the server at ws://openshiftprovidedurl.net:8080/ws.
when I use wscat to connect to the websocket.
wscat -c ws://openshiftprovidedurl.net/ws
I get the following error:
error: Error: unexpected server response (400)
and the application log in openshift shows the following:
2018-04-03 01:14:24+0000 [-] failing WebSocket opening handshake ('missing port in HTTP Host header 'openshiftprovidedurl.net' and server runs on non-standard port 8080 (wss = False)')
2018-04-03 01:14:24+0000 [-] dropping connection to peer tcp4:173.21.2.1:38940 with abort=False: missing port in HTTP Host header 'openshiftprovidedurl.net' and server runs on non-standard port 8080 (wss = False)
2018-04-03 01:14:24+0000 [-] WebSocket connection closed: connection was closed uncleanly (missing port in HTTP Host header 'openshiftprovidedurl.net' and server runs on non-standard port 8080 (wss = False))
Any assistance would be appreciated!
Graham Dumpleton hit the nail on the head, I modified the code from
factory = WebSocketServerFactory(u"ws://0.0.0.0:8080")
to
factory = WebSocketServerFactory(u"ws://0.0.0.0:8080", externalPort=80)
and it corrected the issue. I had to modify my index to point to the correct websocket but I am now able to connect.
Thanks!
Based on the source code of autobahn-python, you can get that message only in 2 cases.
Here is the implementation:
if not ((self.factory.isSecure and self.factory.externalPort == 443) or (not self.factory.isSecure and self.factory.externalPort == 80)):
return self.failHandshake("missing port in HTTP Host header '%s' and server runs on non-standard port %d (wss = %s)" % (str(self.http_request_host), self.factory.externalPort, self.factory.isSecure))
Because I think you are using Deployment + Service (and maybe Ingress on top of them) for your server, you can bind your server to port 80 instead of 8080 and set that port in Service and in Ingress, if you are using them.
I had a working HTTP server using BaseHTTPServer in Python, so I attempted to add an SSL cert to allow for https using LetsEncrypt, and now it won't serve any files or respond. No exceptions or errors thrown, nor will it serve any content.
server_address = ('0.0.0.0', 80)
httpd = HTTPServer(server_address, MyHandler)
# I can comment out the following line and it'll work
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile=ssl_key, certfile=ssl_cert, server_side=True)
httpd.serve_forever()
#ssl_key = '/etc/letsencrypt/live/example.com/privkey.pem'
#ssl_cert = '/etc/letsencrypt/live/example.com/fullchain.pem'
Where MyHandler is the following:
class MyHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(204)
self.send_header("Content-Type", "text/html")
self.end_headers()
return
def do_POST(self):
self.send_response(204)
self.send_header("Content-Type", "text/html")
self.end_headers()
return
Attempting to access the site via web browser from https://example.com returns a standard no-response "Server not found".
I followed the following instructions to generate a certificate using LetsEncrypt: https://certbot.eff.org/#ubuntuxenial-other
sudo apt-get install letsencrypt
Followed by:
letsencrypt certonly --standalone -d example.com
Is there any way I can easily figure out what the problem is here? Using Python 3.5. Happy to provide additional info if needed.
server_address = ('0.0.0.0', 80)
Attempting to access the site via web browser from https://example.com returns a standard no-response "Server not found".
https://host without explicit port specification means that the server is accessed on the default port for the https protocol, which is 443. But, you have setup your server to use port 80 in server_address.
There are two ways to fix this: either explicitly specify a non-standard port for https in the URL, i.e. https://host:80 or change the port in server_address from 80 to 443. The last option is probably the better one.
To begin with, I understand there are other modules such as Requests that would be better suited and simpler to use, but I want to use the socket module to better understand HTTP.
I have a simple script that does the following:
Client ---> HTTP Proxy ---> External Resource (GET Google.com)
I am able to connect to the HTTP proxy alright, but when I send the GET request headers for google.com to the proxy, it doesn't serve me any response at all.
#!/usr/bin/python
import socket
import sys
headers = """GET / HTTP/1.1\r\n
Host: google.com\r\n\r\n"""
socket = socket
host = "165.139.179.225" #proxy server IP
port = 8080 #proxy server port
try:
s = socket.socket()
s.connect((host,port))
s.send(("CONNECT {0}:{1} HTTP/1.1\r\n" + "Host: {2}: {3}\r\n\r\n").format(socket.gethostbyname(socket.gethostname()),1000,port,host))
print s.recv(1096)
s.send(headers)
response = s.recv(1096)
print response
s.close()
except socket.error,m:
print str(m)
s.close()
sys.exit(1)
To make a HTTP request to a proxy open a connection to the proxy server and then send a HTTP-proxy request. This request is mostly the same as the normal HTTP request, but contains the absolute URL instead of the relative URL, e.g.
> GET http://www.google.com HTTP/1.1
> Host: www.google.com
> ...
< HTTP response
To make a HTTPS request open a tunnel using the CONNECT method and then proceed inside this tunnel normally, that is do the SSL handshake and then a normal non-proxy request inside the tunnel, e.g.
> CONNECT www.google.com:443 HTTP/1.1
>
< .. read response to CONNECT request, must be 200 ...
.. establish the TLS connection inside the tunnel
> GET / HTTP/1.1
> Host: www.google.com
Python 3 requires the request to be encoded. Thus, expanding on David's original code, combined with Steffens answer, here is the solution written for Python 3:
def connectThroughProxy():
headers = """GET http://www.example.org HTTP/1.1
Host: www.example.org\r\n\r\n"""
host = "192.97.215.348" #proxy server IP
port = 8080 #proxy server port
try:
s = socket.socket()
s.connect((host,port))
s.send(headers.encode('utf-8'))
response = s.recv(3000)
print (response)
s.close()
except socket.error as m:
print (str(m))
s.close()
sys.exit(1)
This allows me to connect to the example.org host through my corporate proxy (at least for non SSL/TLS connections).
I'm trying to create a TCP socket server in Python that after receiving a string of bytes from a client passes the received data(without knowing what it's actually inside, assuming it's a valid HTTP request) to a HTTP or HTTPS proxy and waits for results, my code looks like this:
import socket
def test(host, port):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((socket.gethostbyname(host), int(port))
msg = """GET / HTTP/1.1
Host: www.bing.com
User-Agent: Firefox
"""
sent_count = sock.send(msg)
recv_value = sock.recv(2048)
print('recvieved:',)
print str(recv_value)
pass
if __name__ == '__main__':
test(host='x.x.x.xx', port='80') # a https proxy server
test(host='yy.yy.yy.yy', port='80') # a http proxy server
But when i connect to the HTTP proxy server it returns something like:
HTTP/1.1 404 Not Found
And when i connect to the HTTPS proxy server it shows something like:
HTTP/1.0 400 Bad Request
So wanted to ask if anybody know how could i send HTTP requests to HTTP/HTTPS servers via sockets in Python? or how can i redirect arbitrary strings of data toward HTTP/HTTPS proxy servers in general in Python using sockets?, any suggestions are very much appreciated, thanks in advance.
I have a problem with my ssl server (in Python).
I set the SSL proxy connection in my browser, and try to connect to my ssl server.
This is the server:
import BaseHTTPServer, SimpleHTTPServer
import ssl
httpd = BaseHTTPServer.HTTPServer(('0.0.0.0', 443), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.socket = ssl.wrap_socket (httpd.socket, server_side=True, certfile='server.crt', keyfile='server.key', do_handshake_on_connect=False)
httpd.serve_forever()
This is the error:
SSLError: [SSL: HTTPS_PROXY_REQUEST] https proxy request (_ssl.c:1750)
I try to connect to the server in the browser.
its work if I went to address "https://127.0.0.1:443".
But, if I use in the server to proxy, I get the error...
How can I fix this?
I don't think you understand how a proxy server for HTTPS works.
What you are doing is to create a plain HTTPS server. What you should do is to create a HTTP server which handles the CONNECT request and creates a tunnel to the requested target. See http://en.wikipedia.org/wiki/HTTP_tunnel#HTTP_CONNECT_tunneling