Trying to send email with a certificate file using the following script:
import smtplib
client = smtplib.SMTP(myhost, myport)
client.ehlo()
client.starttls(certfile=mycertfile)
client.ehlo()
client.login(myusername, mypassword)
client.sendmail(sender, receiver, Message)
client.quit()
I get the following error:
SSLError: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib
I think documentations (smtplib.html and ssl.html) say I need to provide a private key. I only have the certificate file (base64 PEM format). My devops says that a private key is not required in this case because I do not need to identify the local side of the connection.
Question
Is there a way to send the email without providing the private key? If a private key is required, why?
There are two ways to use SSL/TLS: client authenticated and "basic" where the client is unauthenticated. In client authenticated connections, the both the server and the client send a certificate to the other. In "basic" only the server does.
If you pass neither a certificate nor a keyfile, smtplib will use a basic connection, where the client is authenticated.
If you use a certificate, it will be used for a client authenticated connection. In that case, the server will demand that the client shows it owns the certificate by signing a handshake message. For the client to be able to do that it also needs the private key, which can be either in the certificate file or as a separate keyfile.
Long story short, if you want to use a client certificate, you must also use a key. If not, you can just leave both empty.
OTOH, maybe you have a server certificate file or CA list you want to use with the connection?
In that case you need to pass it to ssl.wrap_socket in the ca_certs parameter. Since you use Python 2.6 there's no easy way to do that with smtplib (Python 3.3+ has a context argument to starttls).
How to solve this depends on your application. For example, if you do not need ssl for anything else, a hackish solution would be to monkey-patch ssl.wrap_socket with one that provides your ca_cert (as well as cert_reqs=CERT_REQUIRED, likely).
A more full blown solution would be to extend smtplib.SMTP with your own variant that does allow passing in those parameters.
Here's a monkey-patch taken from this page:
class SMTPExt(smtplib.SMTP):
"""
This class extends smtplib.SMTP and overrides the starttls method
allowing extra parameters and forwarding them to ssl.wrap_socket.
"""
def starttls(self, keyfile=None, certfile=None, **kwargs):
self.ehlo_or_helo_if_needed()
if not self.has_extn("starttls"):
raise SMTPException("STARTTLS extension not supported by server.")
(resp, reply) = self.docmd("STARTTLS")
if resp == 220:
self.sock = ssl.wrap_socket(self.sock, keyfile, certfile, **kwargs)
self.file = SSLFakeFile(self.sock)
# RFC 3207:
# The client MUST discard any knowledge obtained from
# the server, such as the list of SMTP service extensions,
# which was not obtained from the TLS negotiation itself.
self.helo_resp = None
self.ehlo_resp = None
self.esmtp_features = {}
self.does_esmtp = 0
return (resp, reply)
Using the root cert from requests
Related
I'm implementing an IMAP proxy which securely communicates with a client.However, I have a problem when handshaking.
The code of my proxy is:
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.bind((host, port))
ssock, addr = self.sock.accept()
context = ssl.SSLContext(ssl.PROTOCOL_TLS)
self.conn_client = context.wrap_socket(ssock)
And I receive the error:
ssl.SSLError: [SSL: UNEXPECTED_MESSAGE] unexpected message (_ssl.c:833)
The code of my tests is:
M = imaplib.IMAP4_SSL(IP_PROXY)
And I receive the error:
ssl.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:777)
However, when the code of the proxy is:
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.bind((host, port))
ssock, addr = self.sock.accept()
self.conn_client = ssl.wrap_socket(ssock, certfile=CERT, server_side= True)
It correctly works but I don't want to use certificate.
Thank you
It correctly works but I don't want to use certificate.
SSL/TLS is almost everywhere used with a certificate to make sure that the client is talking to the expected server and not to some man in the middle. If you don't want to use a certificate you need to either use a different kind of authentication (like PSK) or use no authentication at all ("anonymous authentication" - very bad idea).
In any way you would need to set the relevant ciphers to enable this alternative authentication on both client and server. This can be done with the ciphers attribute to wrap_socket on the server side and in your client code it could probably be done by constructed a SSLContext with the necessary ciphers and using the ssl_context argument to specific the context to be used in IMAP4_SSL.
But this is only for your specific Python based IMAP client. Don't expect that you will be able to configure commonly used IMAP clients like Thunderbird or Outlook to be usable with a server without certificates. And like I said, it is a bad idea in the first place.
I have a python function to send an email notification. I have included the login call to authenticate with my local SMTP server, however emails are being returned stating '553-mail rejected because your IP is in the PBL'. Further reading on https://www.spamhaus.org/pbl reveals that apparently I am not being prevented from sending email, I am simply required to authenticate first.
I have tried base64 encoding to avoid sending the password as plain text without success.
I have also looked at the Pysasl library, but I am a little unsure how I might use this for authenticating with my SMTP server.
Would anyone have any guidance as to the correct use of either base64 encoding, the pysasl library or if a better method exists for satisfying the correct authentication requirements?
My code is below.
def emailNotify(self,userid):
SMTPserver = 'localhost'
sender = '***' # blanked
username = '***' # blanked
receiver = '***' # blanked
pwd = '***' # blanked
text_subtype = 'plain'
msg = 'requested data was posted to dashboard for '+userid
subject = 'confirmation of POST\'d data for '+userid
try:
msg = MIMEText(msg, text_subtype)
msg['Subject'] = subject
msg['From'] = sender
conn = smtplib.SMTP(SMTPserver)
conn.login(username, pwd)
try:
conn.sendmail(sender, receiver, msg.as_string())
finally:
conn.quit()
except:
print('message sending failed')
Thanks in advance.
You have 2 ways of building a program for sending mail:
build a bullet proof solution:
analyze what server announces as its capabilities
choose the one that better meets your requirement
use that authentication method
actually send the mail
simply that means that you have to implement code for reading and decoding what the server returns (which can be weird) and also all various authentication methods, not speaking of TLS
use a custom connection method adapted to that server
read the documentation of the server to know what it declares to support
test it manually first through a mere telnet connection then in an interactive Python session (idle is enough here, but you can choose your prefered Python shell)
carefully program the way you have just tested - but leave relevant error messages in cases the server capabilities change...
IMHO much simpler...
I am trying to connect to an ActiveMQ message broker which uses SSL. I am getting the error:
invalid uri: ssl://myserver.com:61613 [invalid broker(s): 'NoneType' object has no attribute 'groupdict']
Example code taken from stompest documentation: I only changed server, user and pass:
import time
from stompest.config import StompConfig
from stompest.sync import Stomp
while True:
try:
client = Stomp(StompConfig("ssl://myserver.com:61613", login = 'me', passcode = 'me', version = "1.2" ))
client.connect(versions = ["1.2"], host = vhost, heartBeats = (0, 60000)) #CONNECT
subscription = client.subscribe(destination, {"ack": "client", "id": "0"}) #SUBSCRIBE
while True:
frame = client.receiveFrame()
try:
print frame.body
client.ack(frame) #ACK
except:
print "Error: Can't handle message received, NACKing"
client.nack(frame) #NACK
except Exception, e:
# Reconnect on exception
print "Exception handled, reconnecting...\nDetail:\n%s" % e
try:
client.disconnect()
except:
pass
time.sleep(5)
I believe Stompest can handle SSL, but I can't find any reference in the documentation.
Thanks
Your StompConfig has to be provided with an SSLContext with a configuration of the SSL connection. Exactly how this one should look will depend on the specifics of your setup, so let us have a look at some of the options.
For more information on how to set up ActiveMQ to use SSL in general, see this Apache guide, and this StackOverflow question which addresses the same issue but for stomp.py.
No validation
If all you need is a quick and dirty connection with no validation of broker or client -- that is, a connection which would be susceptible to MITM attacks from an active attacker -- then you can supply an SSLContext with validation disabled:
import ssl
sslContext = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
# Let's at least disable some of the older SSL protocols
sslContext.options |= ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_SSLv3
sslContext.check_hostname = False
sslContext.verify_mode = ssl.CERT_NONE
With this, all you need to do is to provide sslContext to your StompConfig:
client = Stomp(StompConfig('ssl://myserver.com:61613', login='me', passcode='me', version='1.2', sslContext=sslContext))
Broker certificate validation
The first improvement over this would be to verify the certificate provided by the server. Assuming that you are using a self-signed certificate (as opposed to one signed by a certificate authority), you can simply provide your SSLContext with the certificate directly:
First, on the broker, export the certificate from its keystore, which we will assume is stored in a file called broker.ks by using the Java keytool through
keytool -exportcert -rfc -alias broker -keystore broker.ks -file broker.pem
Now, move broker.pem to the Python client and change the sslContext configuration as follows:
sslContext = ssl.create_default_context(cafile='broker.pem')
sslContext.check_hostname = True
sslContext.verify_mode = ssl.CERT_REQUIRED
Client certificate validation
ActiveMQ servers can also be set up to only allow connections from clients with predefined certificates. Assuming that you have a PEM encoded client certificate in client.pem, have stored the private key in client.key (cf. e.g. the first part of this answer), and set up ApacheMQ to validate that certificate (cf. the third part of the same answer), then all you need to do to have stompest use that pair for the connection is to call
sslContext.load_cert_chain('client.pem', 'client.key')
as part of the configuration.
I'm writing some software that is supposed to acquire information from SSL-secured web page. Below there's piece of code I use to connect to server.
s = socket.socket (socket.AF_INET, socket.SOCK_STREAM)
ssl_sock = ssl.wrap_socket (
s,
ca_certs = '/home/stilz/Desktop/Certyfikaty/GLOWNE_CA.cer',
cert_reqs = ssl.CERT_REQUIRED,
ssl_version = ssl.PROTOCOL_SSLv3,
)
ssl_sock.connect ((HOST, PORT))
However, this doesn't work. It throws exception with message "Handshake alert failed". I captured some TCP packets that come out of my script and also corresponding packets from Internet Explorer and figured out that my script doesn't send certificate at all (server returns something like "Fatal: no certificate supplied") while IE sends it normally. As far as I know file ca.cer is x509 certificate (beginning with "-----BEGIN CERTIFICATE-----").
Please help me and tell what I'm doing wrong. If I've supplied to few information, please let me know.
Regards
First of all, you need to determine if you need to authenticate yourself on the server (certificate is sent to the server only in this case) or you need to validate server's authenticity (in which case the certificate is not sent to the server).
Now about issues in your code:
Case 1: you don't need to authenticate on the server. In this case you don't need to specify your certificate and nothing is sent to the server.
Case 2: you need to authenticate yourself on the server. In that case you need to provide your certificate and your private key file in keyfile and certfile parameters. Only then your certificate is sent to the server (and the private key is used in the handshake).
I guess that your have case 1 in fact. So first of all you need to check if the server provides a valid certificate by connecting to it with a web browser and inspecting site's certificate chain. It can happen that the site sends its certificate but omits intermediate CA certificates.
In any case I'd like to remind about the discussion about certificates in Python docs which I suggest you re-read.
If the server requests (and requires) a certificate to be sent by the client, you need to supply ssl.wrap_socket with the path to your certificate (certfile) and its matching private key (keyfile). The ca_certs parameter is only used for the client to verify the server's certificate against known CA certificates, it has nothing to do with sending a client-certificate.
You may need to export your client-certificate and private key from IE (and then convert them to PEM format).
Recently I have been playing around with the HTTP Proxy in twisted. After much trial and error I think I finally I have something working. What I want to know though, is how, if it is possible, do I expand this proxy to also be able to handle HTTPS pages? Here is what I've got so far:
from twisted.internet import reactor
from twisted.web import http
from twisted.web.proxy import Proxy, ProxyRequest, ProxyClientFactory, ProxyClient
class HTTPProxyClient(ProxyClient):
def handleHeader(self, key, value):
print "%s : %s" % (key, value)
ProxyClient.handleHeader(self, key, value)
def handleResponsePart(self, buffer):
print buffer
ProxyClient.handleResponsePart(self, buffer)
class HTTPProxyFactory(ProxyClientFactory):
protocol = HTTPProxyClient
class HTTPProxyRequest(ProxyRequest):
protocols = {'http' : HTTPProxyFactory}
def process(self):
print self.method
for k,v in self.requestHeaders.getAllRawHeaders():
print "%s : %s" % (k,v)
print "\n \n"
ProxyRequest.process(self)
class HTTPProxy(Proxy):
requestFactory = HTTPProxyRequest
factory = http.HTTPFactory()
factory.protocol = HTTPProxy
reactor.listenSSL(8001, factory)
reactor.run()
As this code demonstrates, for the sake of example for now I am just printing out whatever is going through the connection. Is it possible to handle HTTPS with the same classes? If not, how should I go about implementing such a thing?
If you want to connect to an HTTPS website via an HTTP proxy, you need to use the CONNECT HTTP verb (because that's how a proxy works for HTTPS). In this case, the proxy server simply connects to the target server and relays whatever is sent by the server back to the client's socket (and vice versa). There's no caching involved in this case (but you might be able to log the hosts you're connecting to).
The exchange will look like this (client to proxy):
C->P: CONNECT target.host:443 HTTP/1.0
C->P:
P->C: 200 OK
P->C:
After this, the proxy simply opens a plain socket to the target server (no HTTP or SSL/TLS yet) and relays everything between the initial client and the target server (including the TLS handshake that the client initiates). The client upgrades the existing socket it has to the proxy to use TLS/SSL (by starting the SSL/TLS handshake). Once the client has read the '200' status line, as far as the client is concerned, it's as if it had made the connection to the target server directly.
I'm not sure about twisted, but I want to warn you that if you implement a HTTPS proxy, a web browser will expect the server's SSL certificate to match the domain name in the URL (address bar). The web browser will issue security warnings otherwise.
There are ways around this, such as generating certificates on the fly, but you'd need the root certificate to be trusted on the browser.