We Keep Coding

Gunicorn Two-way SSL Error "SSL_ERROR_UNKNOWN_CA_ALERT"

I am running a Python3 app via Gunicorn with two-way SSL configured. This requires a local cert/key to verify the app as well as a ca_certs file to verify the client. Gunicorn relies on the standard ssl module in Python, particularly the wrap_socket method.
The service starts and responds to curl requests fine when I use self-signed certificates for both server and client authentication. However when I use certificates signed by another CA, I get an error SSL_ERROR_UNKNOWN_CA_ALERT.
A working setup, with self-signed certs:
# Server cert
openssl req \
-newkey rsa:2048 -nodes -keyout domain.key \
-x509 -days 365 -out domain.crt
# Client (CA) cert
openssl req \
-newkey rsa:2048 -nodes -keyout twoway.key \
-x509 -days 365 -out twoway.crt
With Gunicorn configured as follows:
keyfile = domain.key
certfile = domain.crt
cert_reqs = ssl.CERT_REQUIRED
ca_certs=twoway.crt
And curling as follows:
curl -vk --key twoway.key --cert twoway.crt https://my.service
Produces a successful response:
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 5000 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-SHA
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
* start date: Dec 7 18:35:54 2018 GMT
* expire date: Dec 7 18:35:54 2019 GMT
* issuer: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET /manage/info HTTP/1.1
> Host: localhost:5000
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: gunicorn/19.9.0
< Date: Tue, 11 Dec 2018 18:26:19 GMT
< Connection: keep-alive
< Content-Type: application/json
< Content-Length: 73
A failing setup, with a different series of certs:
With Gunicorn configured as follows:
keyfile = my_service_key.key
certfile = my_service_cert.crt
cert_reqs = ssl.CERT_REQUIRED
ca_certs = my_trusted_clients.crt
And curling as follows:
curl -vk --key my_trusted_key.key --cert my_trusted_clients.crt https://my.service
Produces an error:
About to connect() to localhost port 5000 (#0)
Initializing NSS with certpath: sql/etc/pki/nssdb
warning: ignoring value of ssl.verifyhost
skipping SSL peer certificate verification
NSS: client certificate from file
subject: CN=mycn,OU=abc,O=def,...
NSS error -12195
Closing connection #0
SSL connect error
curl: (35) SSL connect error
Any thoughts on whether I am configuring this the wrong way? And why self-signed certs are working but other certs are not?
Note this config worked previously when using Stunnel, where I set the verify level to 4 ("Ignore the chain and only verify the peer certificate."). If there is something similar in Python I believe that would get me in the right direction.

I don't think that this is be possible as of Gunicorn 19.9.
As well as having the complete certificate chain on the server, in order to validate the client/peer certificate, I believe you will need to be able to configure the SSLContext, and especially be able to set ssl.CERT_REQUIRED when in server-mode.
Gunicorn 19.9 (and master at the time of writing) does not currently use the SSLContext-based wrapper on the connection, so this is not possible, see https://github.com/benoitc/gunicorn/issues/1140 .

How exactly should I properly interrupt HTTP upload requests?

I developed a server app which, among others, handles uploading of large chunks of data. When the upload request starts, before even receiving the chunk, the server app performs a few checks in order for the client to abort the operation if something goes wrong, instead of finding that there is an issue only after he sends gigabytes of data.
When playing with the server app using curl, I discovered a strange behavior.
curl starts the request, being ready to stream the data.
The server responds immediately with a HTTP 403 to signal a problem and provides a JSON response with the details of the problem.
curl fails with exit code 18 and the following output:
curl: (18) transfer closed with 30 bytes remaining to read
When enabling verbose output, here's what I see:
$ curl -X PUT --limit-rate 2M http://127.0.0.1/blob -F files[]=#/tmp/tmp75hw30vc -v
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
> PUT /blob HTTP/1.1
> Host: 127.0.0.1
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Length: 2439352842
> Content-Type: multipart/form-data; boundary=------------------------32c442f4cf8abe0c
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 403 FORBIDDEN
< Server: nginx/1.10.3
< Date: Sat, 29 Sep 2018 22:03:16 GMT
< Content-Type: application/json
< Content-Length: 30
< Connection: keep-alive
* HTTP error before end of send, stop sending
<
* transfer closed with 30 bytes remaining to read
* stopped the pause stream!
* Closing connection 0
curl: (18) transfer closed with 30 bytes remaining to read
On server side, the code (using Flask) is the following:
def receive_blob():
if _can_upload():
return flask.jsonify({"error": "already-uploading"}), 403
...
I'm not sure I understand if the problem is related to my way to use Flask, or to curl options I'm using.
What should I do to avoid this situation, i.e. to make curl display the JSON error message returned by the server?
The question is not a duplicate of How to handle "100 continue" HTTP message? since mine explicitly asks how to make curl display the JSON error message. The linked question invites to add --fail which would lead instead to the output “curl: (22) The requested URL returned error: 403 FORBIDDEN”

curl: (56) Recv failure: Connection reset by peer on flask_restful on AWS

* Trying x.x.x.x...
* TCP_NODELAY set
* Connected to x.x.x.x (x.x.x.x) port 443 (#0)
> GET /sents HTTP/1.1
> Host: x.x.x.x:443
> User-Agent: curl/7.51.0
> Accept: */*
>
* Recv failure: Connection reset by peer
* Curl_http_done: called premature == 1
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer
^^ Port 80 works fine, 443 and any other port fails. I've tried using flask_sslify on the app, but it fails anyways (the redirect comes out for port 80 though)
ufw status shows 443 as open / lsof shows the app as listening
Not sure if it's a AWS-specific problem, everything works fine on my local machine.
Any help would be greatly appreciated.

Requests Library Force Use of HTTP/1.1 On HTTPS Proxy CONNECT

I am having a problem with a misbehaving HTTP Proxy server. I have no control over the proxy server, unfortunately -- it's an 'enterprise' product from IBM. The proxy server is part of a service virtualization solution being leveraged for software testing.
The fundamental issue (I think*) is that the proxy server sends back HTTP/1.0 responses. I can get it to work fine from SOAP UI ( A Java application) and curl from the command line, but Python refuses to connect. From what I can tell, Python is behaving correctly, and the other two are not, as the server expects HTTP/1.1 responses (it wants Host headers, at the very least, to route the service request to a given stub).
Is there a way to get Requests, or the underlying urllib3, or the even farther down http lib to always use http1.1, even if the other end appears to be using 1.0?
Here is a sample program (unfortunately, it requires you to have an IBM Ration Integration Tester installation with RTCP to really replicate) to reproduce the problem:
import http.client as http_client
http_client.HTTPConnection.debuglevel = 1
import logging
import requests
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True
requests.post("https://host:8443/axl",
headers={"soapAction": '"CUCM:DB ver=9.1 updateSipTrunk"'},
data='<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tns="http://www.cisco.com/AXL/API/9.1"><soapenv:Header/><soapenv:Body><tns:updateSipTrunk><name>PLACEHOLDER</name><newName>PLACEHOLDER</newName><destinations><destination><addressIpv4>10.10.1.5</addressIpv4><sortOrder>1</sortOrder></destination></destinations></tns:updateSipTrunk></soapenv:Body></soapenv:Envelope>',
verify=False)
(Proxy is configured via HTTPS_PROXY environment variable)
Debug output before the error, note the HTTP/1.0:
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): host.com
send: b'CONNECT host.com:8443 HTTP/1.0\r\n'
send: b'\r\n'
header: Host: host.com:8443
header: Proxy-agent: Green Hat HTTPS Proxy/1.0
The exact error text that occurs in RHEL 6 is:
requests.exceptions.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:646)
Even though the Host header is shown here, it does NOT show up on the wire. I confirmed this with a tcpdump:
14:03:14.315049 IP sourcehost.53214 > desthost.com: Flags [P.], seq 0:32, ack 1, win 115, options [nop,nop,TS val 2743933964 ecr 4116114841], length 32
0x0000: 0000 0c07 ac00 0050 56b5 4044 0800 4500 .......PV.#D..E.
0x0010: 0054 3404 4000 4006 2ca0 0af8 3f15 0afb .T4.#.#.,...?...
0x0020: 84f8 cfde 0c7f a4f8 280a 4ebd b425 8018 ........(.N..%..
0x0030: 0073 da46 0000 0101 080a a38d 1c0c f556 .s.F...........V
0x0040: XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX ..CONNECT.host
0x0050: XXXX XXXX XXXX XXXX XXXX XXXX XXXX XXXX xx:8443.HTTP/1.0
0x0060: 0d0a
When I curl it with verbose, this is what the output looks like:
* About to connect() to proxy proxy-host.com port 3199 (#0)
* Trying 10.**.**.** ... connected
* Connected to proxy-host.com (10.**.**.**) port 3199 (#0)
* Establish HTTP proxy tunnel to host.com:8443
> CONNECT host.com:8443 HTTP/1.1
> Host: host.com:8443
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Proxy-Connection: Keep-Alive
> soapAction: "CUCM:DB ver=9.1 updateSipTrunk"
>
< HTTP/1.0 200 OK
< Host: host.com:8443
< Proxy-agent: Green Hat HTTPS Proxy/1.0
<
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /path/to/store/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Truncated after this point. You can see the HTTP/1.0 response from the proxy after connecting. The curl's tcpdump also clearly shows the host header, as well as HTTP 1.1.
*I can't be entirely sure this is the fundamental issue, as I can't test it. I do see HTTP/1.0 responses, and can tell that my non-working Python code sends CONNECT HTTP/1.0 messages, while the working Java sends HTTP/1.1 messages, as does Curl. It's possible the problem is unrelated (although I find that unlikely) or that Python is misbehaving, and not Java/curl. I simply don't know enough to know for sure.
So, is there a way to force urllib3/requests to use HTTP v1.1 at all times?

httplib (which requests relies upon for HTTP(S) heavy lifting) always uses HTTP/1.0 with CONNECT:
Lib/httplib.py:788:
def _tunnel(self):
self.send("CONNECT %s:%d HTTP/1.0\r\n" % (self._tunnel_host,
self._tunnel_port))
for header, value in self._tunnel_headers.iteritems():
self.send("%s: %s\r\n" % (header, value))
self.send("\r\n")
<...>
So you can't "force" it to use "HTTP/1.1" here other than by editing the subroutine.
This MAY be the problem if the proxy doesn't support HTTP/1.0 - in particular, 1.0 does not require a Host: header, and indeed, as you can see by comparing your log output with the code above, httplib does not send it. While, in verity, a proxy may expect it regardless. But if this is the case, you should've gotten an error from the proxy or something in response to CONNECT -- unless the proxy is so borken that it substitutes some default (or garbage) for Host:, returns 200 anyway and tries to connect God-knows-where, at which point you're getting timeouts.
You can make httplib add the Host: header to CONNECT by adding it to _tunnel_headers (indirectly):
s=requests.Session()
proxy_url=os.environ['HTTPS_PROXY']
s.proxies["https"]=proxy_url
# have to specify proxy here because env variable is only detected by httplib code
#while we need to trigger requests' proxy logic that acts earlier
# "https" means any https host. Since a Session persists cookies,
#it's meaningless to make requests to multiple hosts through it anyway.
pm=s.get_adapter("https://").proxy_manager_for(proxy_url)
pm.proxy_headers['Host']="host.com"
del pm,proxy_url
<...>
s.get('https://host.com')

If you do not depend on the requests library you may find the following snippet useful:
import http.client
conn = http.client.HTTPSConnection("proxy.domain.lu", 8080)
conn.set_tunnel("www.domain.org", 443, headers={'User-Agent': 'curl/7.56.0'})
conn.request("GET", "/api")
response = conn.getresponse()
print( response.read() )

Python: Tor not connecting with a certain website

I am trying to access a specific website with Python and Tor, but somehow it just loads forever and no response comes.
import requests
import socks
import socket
url = 'http://www.ryanair.com'
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', 9150)
socket.socket = socks.socksocket
response = requests.get(url)
#no response comes
Some weeks ago it was working. In addition, the same code works still fine for other sites.
My guess would be that it is not a problem on my side, but on the website's side, but I have no clue what can be wrong.
Any ideas of what can be and how it could be solved?
Edit:
If I run the command:
curl --proxy socks5h://127.0.0.1:9150 -vvv http://www.ryanair.com
I get:
* Rebuilt URL to: http://www.ryanair.com/
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 9150 (#0)
> GET / HTTP/1.1
> Host: www.ryanair.com
> User-Agent: curl/7.49.1
> Accept: */*
>
< HTTP/1.1 302 Found : Moved Temporarily
< Location: https://www.ryanair.com/
< Connection: close
< Cache-Control: no-cache
< Pragma: no-cache
<
* Closing connection 0
Trying with https:
$ curl --proxy socks5h://127.0.0.1:9150 -vvv https://www.ryanair.com
* Rebuilt URL to: https://www.ryanair.com/
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 9150 (#0)
* TLS 1.2 connection using TLS_RSA_WITH_3DES_EDE_CBC_SHA
* Server certificate: www.ryanair.com
* Server certificate: GeoTrust SSL CA - G3
* Server certificate: GeoTrust Global CA
> GET / HTTP/1.1
> Host: www.ryanair.com
> User-Agent: curl/7.49.1
> Accept: */*
>
#...and it remains here
Edit 2:
I just tried to load this website directly at the Tor browser and it will also keep loading without end.

It will be difficult to run diagnostics without access to your proxy server, but in general, if you're debugging a request, curl is your most invaluable tool.
This should help:
curl --proxy socks5h://127.0.0.1:9150 -vvv http://www.ryanair.com
The -vvv will show you information about the request and the response. That will show you whether your request is well structured or whether the server response is at fault.