I'm wondering if I can recv/send data on a raw socket before wrapping it - I've looked through the documentation and searched for it but couldn't find anything specific. What I basically want to do:
client, addr = listeningSocket.accept()
client.recv(32)
client.send(b'hello')
client.setblocking(0)
sslSocket = ssl.wrap_socket(client, keyfile='key.pem', certfile='cert.pem', server_side=True, ssl_version=ssl.PROTOCOL_TLSv1)
sslSocket.write(b'hello')
The problem is I get an error that I'm pretty sure is related to the client.recv() before wrapping (or at least I think that's it since I do not get it before adding the recv?)
sslSocket = ssl.wrap_socket(client, keyfile='key.pem', certfile='cert.pem', server_side=True, ssl_version=ssl.PROTOCOL_TLSv1)
File "/usr/lib/python3.1/ssl.py", line 381, in wrap_socket
suppress_ragged_eofs=suppress_ragged_eofs)
File "/usr/lib/python3.1/ssl.py", line 135, in __init__
raise x
File "/usr/lib/python3.1/ssl.py", line 131, in __init__
self.do_handshake()
File "/usr/lib/python3.1/ssl.py", line 327, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 8] _ssl.c:488: EOF occurred in violation of protocol
Is this legal? Is there anyway to do this (I really need to send before wrapping since the client expects a raw string before the SSL data starts flooding.) Appreciate any guidance possible.
Note: I need to respond to a policy request from flash. The connection with flash is going to be secure, but the policy request isn't
I notice in the documentation for do_handshake ( http://docs.python.org/library/ssl.html#ssl.SSLSocket.do_handshake ), which is called by wrap_socket (as is apparent from your callstack), that it may raise an ssl.SSLError on nonblocking sockets, and you have to wrap it in a try/except to try until it works. So in your case, if you need a nonblocking socket, you would either need to do the handshake yourself first and pass do_handshake_on_connect=False to wrap_socket, or just wait until after the handshake to set nonblocking:
client, addr = listeningSocket.accept()
client.recv(32)
client.send(b'hello')
sslSocket = ssl.wrap_socket(client, keyfile='key.pem', certfile='cert.pem', server_side=True, ssl_version=ssl.PROTOCOL_TLSv1)
client.setblocking(0)
sslSocket.write(b'hello')
or maybe:
client, addr = listeningSocket.accept()
client.recv(32)
client.send(b'hello')
client.setblocking(0)
while True:
try:
client.do_handshake()
break
except ssl.SSLError, err:
if err.args[0] == ssl.SSL_ERROR_WANT_READ:
select.select([client], [], [])
elif err.args[0] == ssl.SSL_ERROR_WANT_WRITE:
select.select([], [client], [])
else:
raise
sslSocket = ssl.wrap_socket(client, do_handshake_on_connect=False, keyfile='key.pem', certfile='cert.pem', server_side=True, ssl_version=ssl.PROTOCOL_TLSv1)
sslSocket.write(b'hello')
Yet another alternative is just to run a dedicated Flash policy server on port 843.
I believe if you close the socket after the Flash policy request, Flash will handle this correctly and reconnect after validating the policy response.
Related
We are relatively new to in Python therefore may be the question is too simple.
We are using Python version 2.7.15.
We are trying to use Python over TLS without success.
This is our code:
import ssl,socket
import urllib2
context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = False
context.load_verify_locations("/py-test/python/bin/certificate.pem")
url = "https://10.0.0.12"
request = urllib2.Request(url)
websocket = urllib2.urlopen(request,None,None,None,None,None,context)
pages=websocket.readlines()
print pages
As you see, we have configured context.check_hostname = False
Unfortunately, it fails with the following exception
Traceback (most recent call last):
File "./test.py", line 11, in <module>
websocket = urllib2.urlopen(request,None,None,None,None,None,context)
File "/py-test/python/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/py-test/python/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/py-test/python/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/py-test/python/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/py-test/python/lib/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
File "/py-test/python/lib/python2.7/urllib2.py", line 1198, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)>
It is definitely the hostname verification.
If we use the correct certificate and correct hostname the request successful.
If we use the wrong certificate it fails with the following exception.
File "./test.py", line 8, in <module>
context.load_verify_locations("/py-test/python/bin/certificate_bad.pem")
ssl.SSLError: [X509] PEM lib (_ssl.c:3027)
Therefore, we need help to understand how to configure Python to ignore the hostname verification.
One more question (can be asked in the separate thread).
Do we have in Python a trustore file that include all known CA? Like cacerts.jks in Java.
Where can we find the trustore?
Added
We “want to verify that the certificate was signed by a valid CA”, but we “don't care whether it identifies the site you're actually connecting to”.
We need help to configure Python to ignore the hostname verification?
What is mistake in our code?
We have tried to create the code according to the documentation https://docs.python.org/2/library/ssl.html
Added 2
We have invested a lot of time but unfortunately we do not have the progress.
Is anyone has the working example in Python 2.7?
I mean is the code works if you access with other URL then appears in a certificate.
May be Python 2.7 cannot be configured to ignore the hostname verification?
What can be our problem?
We use it on CentOS 6.
May be it is related to OpenSSL? We use the latest version openssl-1.0.1e-57.el6.x86_64.
May be we should upgrade to Python 3.x?
As you discovered, you can accomplish this by customizing the SSLContext object used for verifying the connection.
However, to hook this into urllib2.urlopen you'll need to build a custom opener and install that.
Here's an example:
import httplib
import socket
import ssl
import urllib2
import certifi
class InterceptedHttpsConnection(httplib.HTTPSConnection):
def connect(self):
# Open an unencrypted TCP socket first
sock = socket.create_connection((self.host, self.port), self.timeout)
# Build up a custom SSLContext. (Might be better to do this once rather
# than on every request.)
context = ssl.SSLContext(ssl.PROTOCOL_TLS)
# We require the SSL context to verify the certificate.
context.verify_mode = ssl.CERT_REQUIRED
# But we instruct the SSL context to *not* validate the hostname.
context.check_hostname = False
# Load CA certificates from the certifi bundle.
context.load_verify_locations(cafile=certifi.where())
# Use our SSLContext object to wrap the bare socket into an SSL socket.
self.sock = context.wrap_socket(sock, server_hostname=self.host)
class InterceptedHttpsHandler(urllib2.HTTPSHandler):
def https_open(self, req):
return self.do_open(InterceptedHttpsConnection, req)
def main():
opener = urllib2.build_opener(InterceptedHttpsHandler)
urllib2.install_opener(opener)
contents = urllib2.urlopen('https://example.com/').read()
print contents
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
Forces python to skip the validation of certificates by default. Hope it helps.
Hello I would ask if i could do connection to extract some data from Tool Server.
so i making this code but i need to modify it to open this tool with username and password and extract data from My tool server.
import socket
import ssl
HOST, PORT = '10.74.159.82', 31039
def handle(conn):
conn.write(b'GET / HTTP/1.1\n')
print(conn.recv() . decode())
def main():
sock = socket.socket(socket.AF_INET)
context = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
conn = context.wrap_socket(sock, server_hostname=HOST)
try:
conn.connect((HOST, PORT))
handle(conn)
finally:
conn.close()
if __name__ == '__main__':
main()
RESULT
!! C:\Users\Admin\.PyCharmCE2018.1\config\venv\Scripts\python.exe!! C:/Users/Admin/.PyCharmCE2018.1/config/codestyles/Under_Building.py
Traceback (most recent call last):
File "C:/Users/Admin/.PyCharmCE2018.1/config/codestyles/Under_Building.py", line 22, in <module>
main()
File "C:/Users/Admin/.PyCharmCE2018.1/config/codestyles/Under_Building.py", line 16, in main
conn.connect((HOST, PORT))
File "C:\Users\Admin\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 1141, in connect
self._real_connect(addr, False)
File "C:\Users\Admin\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 1132, in _real_connect
self.do_handshake()
File "C:\Users\Admin\AppData\Local\Programs\Python\Python37-32\lib\ssl.py", line 1108, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1045)
Process finished with exit code 1
I'm Beginner so i need to learn and supporting
It's hard to see exactly, but it seems the server you're connecting to (on https://10.74.159.82:31039) is using a self-signed certificate (or its CA is, perhaps).
It's perhaps better to create or buy a proper (non-self-signed) certificate, but assuming this is a test server this is fine.
So with what you have, your best bet is to allow self-signed certs in your SSL Context (and also, not to check the hostname - that will probably fail too as you're using an IP address):
context = ssl.create_default_context()
context.check_hostname=False
context.verify_mode=ssl.CERT_NONE
# etc...
I have already been through these set of solutions for connecting IMAP under proxy
Information to add:
I am trying to write a python code that can fetch mails from gmails IMAP server using imapclient under http,https and socks proxy server of my academic insitute
When tried without any proxy handling, it used to give error
socket.error [101] network is unreachable
import imapclient
import pyzmail
imapObj = imapclient.IMAPClient('imap.gmail.com',ssl=True)
imapObj.login('***********#gmail.com','*********')
imapObj.select_folder('INBOX', readonly=True)
UIDs = imapObj.search(['SINCE 07-Jul-2016'])
for item in UIDs:
rawMessages = imapObj.fetch(item, ['BODY[]', 'FLAGS'])
message = pyzmail.PyzMessage.factory(rawMessages[item]['BODY[]'])
message.get_subject()
message.get_addresses('from')
message.get_addresses('to')
message.get_addresses('cc')
message.get_addresses('bcc')
message.text_part != None
message.text_part.get_payload().decode(message.text_part.charset)
message.html_part != None
message.html_part.get_payload().decode(message.html_part.charset)
imapObj.logout()
However, the process gives error as
File "mailtotext.py", line 16, in <module>
imapObj = imapclient.IMAPClient('imap.gmail.com',ssl=True)
File "/usr/local/lib/python2.7/dist-packages/imapclient/imapclient.py", line 152, in __init__
self._imap = self._create_IMAP4()
File "/usr/local/lib/python2.7/dist-packages/imapclient/imapclient.py",line 164, in _create_IMAP4
self._timeout)
File "/usr/local/lib/python2.7/dist-packages/imapclient/tls.py", line 153, in __init__
imaplib.IMAP4.__init__(self, host, port)
File "/usr/lib/python2.7/imaplib.py", line 172, in __init__
self.open(host, port)
File "/usr/local/lib/python2.7/dist-packages/imapclient/tls.py", line 158, in open
sock = socket.create_connection((host, port), self._timeout)
File "/usr/lib/python2.7/socket.py", line 571, in create_connection
raise err
socket.error: [Errno 101] Network is unreachable
I then followed the above mentioned link procedures owing to my institute proxy as http,https,socks
I have already set my system proxy settings as
http_proxy="http://10.3.100.207:8080/"
https_proxy="https://10.3.100.207:8080/"
ftp_proxy="ftp://10.3.100.207:8080/"
socks_proxy="socks://10.3.100.207:8080/"
and edited the code as
import imapclient
import pyzmail
import socks
import socket
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS4,"10.3.100.207",8080,True)
socket.socket = socks.socksocket
imapObj = imapclient.IMAPClient('imap.gmail.com',ssl=True)
imapObj.login('***********#gmail.com','*********')
imapObj.select_folder('INBOX', readonly=True)
UIDs = imapObj.search(['SINCE 07-Jul-2016'])
for item in UIDs:
rawMessages = imapObj.fetch(item, ['BODY[]', 'FLAGS'])
message = pyzmail.PyzMessage.factory(rawMessages[item]['BODY[]'])
message.get_subject()
message.get_addresses('from')
message.get_addresses('to')
message.get_addresses('cc')
message.get_addresses('bcc')
message.text_part != None
message.text_part.get_payload().decode(message.text_part.charset)
message.html_part != None
message.html_part.get_payload().decode(message.html_part.charset)
imapObj.logout()
But this process seems to freeze for very long time and finally I always need to make a keyboard interrupt. Its sure that it was freezed somewhere in socket files as per keyboard interrupt output.
Please help me through this, there are hardly solution to such problems over web. I have even tried tunneling but it isn't solving my problem or say making it worse (I might have not implemented it well :P)I would provide any other information and output if needed here
I have the following pretty simple Python code, derived from here:
import ssl
import base64
from socket import *
cc = socket(AF_INET, SOCK_STREAM)
cc.connect(("smtp.gmail.com", 587))
cc.send('helo tester.com\r\n')
cc.send('starttls\r\n')
scc = ssl.wrap_socket(cc, ssl_version=ssl.PROTOCOL_SSLv23)
scc.send('AUTH LOGIN\r\n')
scc.send(base64.b64encode('myBase64EncodedUsernameAndPassword==')+'\r\n')
scc.send
However upon running it I get:
Traceback (most recent call last):
File "Untitled 2.py", line 16, in <module>
scc = ssl.wrap_socket(cc, ssl_version=ssl.PROTOCOL_SSLv23)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 387, in wrap_socket
ciphers=ciphers)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 143, in __init__
self.do_handshake()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 305, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:507: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
What am I doing wrong?
You blindly send data to the server without making any attempts to read and check the answer from the server. But this is required, because only after a successful response to the STARTTLS command you can upgrade the socket to SSL.
Since you don't read the responses from the server and try to upgrade the socket blindly to SSL it will start the SSL handshake by sending the ClientHello but then croak with unknown protocol because instead of reading the ServerHello it will read the plain text data from the server which you were supposed to read earlier.
For details on how to correctly handle SMTP see the standard, that is RFC 5321 and RFC 3207.
I'm really new to networking in general and I'm trying to set up a simple exchange between a python server and client.
This is the code for the server
import socket, ssl
def do_something(connstream, data):
print "HALLO"
def deal_with_client(connstream):
data = connstream.read()
# null data means the client is finished with us
while data:
if not do_something(connstream, data):
# we'll assume do_something returns False
# when we're finished with client
break
data = connstream.read()
# finished with client
bindsocket = socket.socket()
bindsocket.bind(('127.0.0.1', 10024))
bindsocket.listen(5)
while True:
newsocket, fromaddr = bindsocket.accept()
print "Setting up connection"
connstream = ssl.wrap_socket(newsocket,
server_side=True,
ca_certs=None,
certfile="cert.pem",
keyfile="privatekey.pem",
ssl_version=ssl.PROTOCOL_TLSv1)
try:
deal_with_client(connstream)
finally:
connstream.shutdown(socket.SHUT_RDWR)
connstream.close()
Here is the code for the client.py
import socket, ssl
clientsocket = socket.socket()
ssl_sock = ssl.wrap_socket(clientsocket,
certfile="cert.pem",
cert_reqs=ssl.CERT_REQUIRED)
ssl_sock.connect(('127.0.0.1', 10024))
print ssl_sock.getpeername()
print ssl_sock.getpeercert()
data = ssl_sock.recv(1024)
ssl_sock.close()
print 'Received', repr(data)
I generated the "cert.pem" and the "privatekey.pem" using openssl.
Traceback (most recent call last):
File "server.py", line 30, in <module>
ssl_version=ssl.PROTOCOL_TLSv1)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 344, in wrap_socket
ciphers=ciphers)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 121, in __init__
self.do_handshake()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 283, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 8] _ssl.c:499: EOF occurred in violation of protocol
I was wondering if somehow who knows more could point me in the right direction. I really want to do this using SSL btw, but I would be willing to switch to TLS if that is the better approach.
It could be that the sockets are not running with a compatible ssl version, you should put a "ssl.PROTOCOL_TLSv1" compatible version in your client too (or removing it from server and use the default value too).
Googling you can find many examples of socket comunication