Our client wants a client script that will be installed on their customers' computers to be as trivial to install as possible. This means no extra-install packages, in this case PyCurl.
We need to be able to connect to a website using SSL and expecting a client certificate. Currently this is done calling Curl with os.system() but to get the http return code doing this it looks like we'll have to use the '-v' option to Curl and comb through this output. Not difficult, just a bit icky.
Is there some other way to do this using the standard library that comes with Python 2.6?
I read everything I could find on this and I couldn't see a non-Curl way of doing it.
Thanks in advance for any guidance on this subject whatsoever!
this will do the trick. Note that Verisign don't require a client certificate, it's just a randomly taken HTTPS site.
import httplib
conn = httplib.HTTPSConnection('verisign.com', key_file='./my-key.pem', cert_file='./my-cert.pem')
conn.connect()
conn.request('GET', '/')
conn.set_debuglevel(20)
response = conn.getresponse()
print('HTTP status', response.status)
EDIT: Just for the posterity, Bruno's comment below is a valid one and here's an article how to roll it using the stdlib's socket ssl and socket modules in case it's needed.
EDIT2: Seems I cannot post links - just do a web search for 'Validating SSL server certificate with Python 2.x another day'
Related
I'm about to use Python.requests to get data from my own online api to my local pc. My api requires authentication which for now is done trough simply posting user/pass:
params = {'user': 'username', 'pass':'password'}
requests.post(url, params=params)
Are this requests safe or is it going to allow a middle-man to capture that user/pass?
P.S My api is using a letsencrypt ssl certificate. Python version 3.7.0
this has nothing to do with the python-requests package, but with the HTTP (and HTTPS) protocols. HTTP is plain-text so anyone that manages to sniff your packets can read the content (hence the username/password pair in clear text). HTTPS uses strong encryption, so even someone sniffing your traffic will have a hard-time deciphering it - no encryption scheme is 100% safe of course but decrypting SSL traffic is currently way too costly even for the NSA.
IOW, what will make your requests "safe" is the use of the HTTPS protocol, not which python (or not python) package you use to write your client code.
Use the HTTPS protocol and it's safe provided you have a valid SSL certificate on your api. If you still feel paranoid/insecure, you can implement end-to-end encryption using an existing algorithm or create your custom algorithm either.
My code :-
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("www.python.org" , 80))
s.sendall(b"GET https://www.python.org HTTP/1.0\n\n")
print(s.recv(4096))
s.close()
Why the output shows me this:-
b'HTTP/1.1 500 Domain Not Found\r\nServer: Varnish\r\nRetry-After: 0\r\ncontent-type: text/html\r\nCache-Control: private, no-cache\r\nconnection: keep-alive\r\nContent-Length: 179\r\nAccept-Ranges: bytes\r\nDate: Tue, 11 Jul 2017 15:23:55 GMT\r\nVia: 1.1 varnish\r\nConnection: close\r\n\r\n\n\n\nFastly error: unknown domain \n\n\nFastly error: unknown domain: . Please check that this domain has been added to a service.'
How can I fix it?
This is wrong on multiple levels:
to access a HTTPS resource you need to create a TLS connection (i.e. ssl_wrap on top of an existing TCP connection, with proper certificate checking etc) and then send the HTTP request. Of course the TCP connection in this case should go to port 443(https) not 80 (http).
the HTTP request should only contain the path, not the full URL
the line end must be \r\n not \n
you better send a Host header too since many severs require it
And that's only the request. Properly handling the response is a different topic.
I really really recommend to use an existing library like requests. HTTP(S) is considerably more complex as most think who only had a look at a few traffic captures.
import requests
x = requests.get('https://www.python.org')
print x.text
With the requests library, HTTPS requests are very simple! If you're doing this with raw sockets, you have to do a lot more work to negotiate a cipher and etc. Try the above code (python 2.7).
I would also note that, in my experience, Python is excellent for doing things quickly. If you are learning about networking and cryptography, try writing a HTTPS client on your own using sockets. If you want to automate something quickly, use the tools that are available to you. I almost always use requests for this type of task. As an additional note, if you're interested in parsing HTML content, check out the PyQuery library. I've used it to automate interaction with many web services.
Requests
PyQuery
I am trying to connect to a website with requests that requires using a client certificate.
import requests
r = requests.get(url, cert='path to cert')
print(r.status_code)
This works for one site that uses the same client cert. That server is using TLS_RSA_WITH_AES_128_CBC_SHA, TLS 1.0. However my target site uses TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS 1.1. So basically the difference is TLS 1 works and TLS 1.1 doesn't. Everything works fine in browser so it must have something to do with Python's SSL.
I am using requests version 2.7.0 and I have requests[security] installed as well. pip freeze:
cffi==0.9.2
cryptography==0.8.1
ndg-httpsclient==0.3.3
pyasn1==0.1.7
pycparser==2.10
pyOpenSSL==0.15.1
requests==2.7.0
six==1.9.0
The specific error I am getting is requests.exceptions.SSLError: [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:600). This is on Windows 7 with Python 3.4.3. Unfortunately this is on an internal machine so I am stuck with Windows and our internal mirror of PyPi does not have the latest versions of everything. It seems to me like this has something to do with ssl failing and not necessarily requests.
Google does not give back promising results. There is this StackOverflow post that describes the same problem, but the solution provided (using a custom adapter) does not work for me.
Hopefully someone else has run into this before and can give me some tips on how to fix it. Please and thanks.
EDIT: I did a wireshark capture of the interaction. The SSL alert sent back is "Level: Fatal (2) Description: Internal Error (80)". After the TCP connection start, my machine sends a client hello.
Content Type: Handshake (22)
Version: TLS 1.0 (0x0301)
Length: 512
Then the handshake protocol segment of that packet is
Handshake Type: Client Hello (1)
Length: 508
Version: TLS 1.2 (0x0301)
followed by a list of the supported cipher suites, etc. I looked in the list of cipher suites sent by my client and TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA is listed. The server ACKs this message then sends the Alert packet.
I got rid of an identical SSLError by removing the first entry ECDH+AESGCM from requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS, with which the server seemed to have problems. The line
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:!eNULL:!MD5'
solved the problem for me.
For me, request.request('GET'... instead of request.get(... works.
And I got rid of the above SSLError by removing almost all the first entry:
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS = 'RSA+AESGCM:RSA+AES:RSA+HIGH:RSA+3DES:!aNULL:!eNULL:!MD5'
I'm trying to set up a server and client in python where the server authenticates clients using SSL with certificates. There are a lot of examples of SSL certificates online, but everything I've found has the server providing a certificate to the client and the client checking it. I need the server to ensure that the client has the authority to connect to the server. I understand how to generate and send certificates and the basics of how they work. I would type out my code, but my client/server without SSL is working fine and I've been referencing this for SSL. The client/server example at the bottom of that page summarizes my understanding of SSL certs in python.
I realize this isn't much to go on, but if someone could explain the basic modifications to that example to have the server authenticate the client instead of the other way around, that would be awesome. Alternatively, a link to an example or even just some socket methods to investigate would be very helpful. Let me know if more information is needed. I don't mean to be vague and promise I've spent all morning looking for info myself :).
Edit: I'm trying to stick to the basic ssl library. Aka "import ssl".
You would use SSLSocket.getpeercert to get the certificate. The client would need to specify a key and certificate when wrapping the socket just like the server side. On the server side, you will also need to pass ca_certs="path_to_ca_cert_file" and probably also want to specify cert_reqs=CERT_REQUIRED (see. args for ssl.wrap_socket.
In addition to this, it sounds like you might be looking to do certificate based client authentication/authorization? This is a simple matter of using getpeercert to get the client certificate and accessing fields within the certificate to use in a normal authentication path (i.e. Common Name == User Id)
Not really sure what your question refers, however you can see SSL in Python, other resource for SSL in Python, Validating SSL, get SSL Certificate information and you probably found other good links.
i want to know if there is a way to generate dns queries via python and get the dns responses as well as their headers so as to see things in the dns response as NXDomain, connection refused, server failure etc..
thanks in advance
Sure, there is a library called dnspython that does exactly that.