We Keep Coding

Twisted http client using client certificate

I'm trying to write a twisted https client, that uses a client certificate for authentication. This is the example from the twisted docs for making a simple https request:
from __future__ import print_function
from twisted.internet import reactor
from twisted.web.client import Agent
from twisted.web.http_headers import Headers
agent = Agent(reactor)
d = agent.request(
b'GET',
b'https://127.0.0.1:8880/test',
Headers({'User-Agent': ['Twisted Web Client Example']}),
None)
def cbResponse(ignored):
print('Response received')
d.addCallback(cbResponse)
def cbShutdown(ignored):
reactor.stop()
d.addBoth(cbShutdown)
reactor.run()
I've found this explanation on how to use a client with ssl certificates. I have no idea though, how I can tell the agent to use a certificate for authentication. Any ideas?

Construct Agent with an IPolicyForHTTPS which can create a contextFactory to provide a client certificate to the TLS connection.
certData = getModule(__name__).filePath.sibling('public.pem').getContent()
authData = getModule(__name__).filePath.sibling('server.pem').getContent()
clientCertificate = ssl.PrivateCertificate.loadPEM(authData)
authority = ssl.Certificate.loadPEM(certData)
options = ssl.optionsForClientTLS(u'example.com', authority,
clientCertificate)
class SinglePolicy(object):
def creatorForNetloc(self, hostname, port):
return options
agent = Agent(reactor, SinglePolicy())

one strange behavior of xmlrpc

Here is the code:
server part and called from machine 10.42.0.1:
from xmlrpc.server import SimpleXMLRPCServer
from xmlrpc.server import SimpleXMLRPCRequestHandler
class RequestHandler(SimpleXMLRPCRequestHandler):
rpc_paths = ()
server = SimpleXMLRPCServer(('10.42.0.1', 8000),
requestHandler=RequestHandler)
server.register_function(adder, 'add')
print('initialize finish')
server.serve_forever()
client part, and call from machine 10.42.0.2:
import xmlrpc.client
s = xmlrpc.client.ServerProxy('http://10.42.0.1:8000')
print(s.add(2,3))
However, I've got the error message from machine 10.42.0.2:
ConnectionRefusedError: [Errno 111]. And telnet 10.42.0.1 8000 also failed.
Then, I change this sentence:
server = SimpleXMLRPCServer(('10.42.0.1', 8000),
requestHandler=RequestHandler)
to:
server = SimpleXMLRPCServer(('', 8000),
requestHandler=RequestHandler)
And restart xmlrpc server, the xmlrpc client works this time. Then I change this sentence to:
server = SimpleXMLRPCServer(('10.42.0.1', 8001),
requestHandler=RequestHandler)
And open a new xmlrpc server, and change the client code to:
import xmlrpc.client
s = xmlrpc.client.ServerProxy('http://10.42.0.1:8001')
print(s.add(2,3))
And start a new xmlrpc client, the client now also works.
Anyone help me explain this strange phenomenon?
A bit like arp table not built.

How to write a web server using twisted?

How do i write a simple http web server using twisted framework?
I want a web server that can receive http request and return a response to the client.
Am going through the twisted documentation and am kinda confused (maybe am just lazy), but it does not look very direct on how to do this, especially how does the twisted server receive the request parameters?
This is my effort...
from twisted.web import proxy, http
from twisted.internet import reactor
from twisted.python import log
from twisted.protocols import basic
import sys
log.startLogging(sys.stdout)
class InformixProtocol(basic.LineReceiver):
def lineReceived(self, user):
self.transport.write("Hello this is twisted web server!")
self.transport.loseConnection()
class ProxyFactory(http.HTTPFactory):
#protocol = proxy.Proxy
protocol = InformixProtocol
reactor.listenTCP(8080, ProxyFactory())
reactor.run()
Thanks
Gath

Basic SSL Server Using Twisted Not Responding

I am currently trying to pull together a basic SSL server in twisted. I pulled the following example right off their website:
from twisted.internet import ssl, reactor
from twisted.internet.protocol import Factory, Protocol
class Echo(Protocol):
def dataReceived(self, data):
"""As soon as any data is received, write it back."""
print "dataReceived: %s" % data
self.transport.write(data)
if __name__ == '__main__':
factory = Factory()
factory.protocol = Echo
print "running reactor"
reactor.listenSSL(8080, factory,
ssl.DefaultOpenSSLContextFactory(
"./test/privatekey.pem", "./test/cacert.pem"))
reactor.run()
I then tried to hit this server using firefox by setting the url to https://localhost:8080 yet I receive no response. I do, however, see the data arriving at the server. Any ideas why I'm not getting a response?

You're not sending an http header back to the browser, and you're not closing the connection

You've implemented an SSL echo server here, not an HTTPS server. Use the openssl s_client command to test it interactively, not firefox (or any other HTTP client, for that matter).

Convert HTTP Proxy to HTTPS Proxy in Twisted

Recently I have been playing around with the HTTP Proxy in twisted. After much trial and error I think I finally I have something working. What I want to know though, is how, if it is possible, do I expand this proxy to also be able to handle HTTPS pages? Here is what I've got so far:
from twisted.internet import reactor
from twisted.web import http
from twisted.web.proxy import Proxy, ProxyRequest, ProxyClientFactory, ProxyClient
class HTTPProxyClient(ProxyClient):
def handleHeader(self, key, value):
print "%s : %s" % (key, value)
ProxyClient.handleHeader(self, key, value)
def handleResponsePart(self, buffer):
print buffer
ProxyClient.handleResponsePart(self, buffer)
class HTTPProxyFactory(ProxyClientFactory):
protocol = HTTPProxyClient
class HTTPProxyRequest(ProxyRequest):
protocols = {'http' : HTTPProxyFactory}
def process(self):
print self.method
for k,v in self.requestHeaders.getAllRawHeaders():
print "%s : %s" % (k,v)
print "\n \n"
ProxyRequest.process(self)
class HTTPProxy(Proxy):
requestFactory = HTTPProxyRequest
factory = http.HTTPFactory()
factory.protocol = HTTPProxy
reactor.listenSSL(8001, factory)
reactor.run()
As this code demonstrates, for the sake of example for now I am just printing out whatever is going through the connection. Is it possible to handle HTTPS with the same classes? If not, how should I go about implementing such a thing?

If you want to connect to an HTTPS website via an HTTP proxy, you need to use the CONNECT HTTP verb (because that's how a proxy works for HTTPS). In this case, the proxy server simply connects to the target server and relays whatever is sent by the server back to the client's socket (and vice versa). There's no caching involved in this case (but you might be able to log the hosts you're connecting to).
The exchange will look like this (client to proxy):
C->P: CONNECT target.host:443 HTTP/1.0
C->P:
P->C: 200 OK
P->C:
After this, the proxy simply opens a plain socket to the target server (no HTTP or SSL/TLS yet) and relays everything between the initial client and the target server (including the TLS handshake that the client initiates). The client upgrades the existing socket it has to the proxy to use TLS/SSL (by starting the SSL/TLS handshake). Once the client has read the '200' status line, as far as the client is concerned, it's as if it had made the connection to the target server directly.

I'm not sure about twisted, but I want to warn you that if you implement a HTTPS proxy, a web browser will expect the server's SSL certificate to match the domain name in the URL (address bar). The web browser will issue security warnings otherwise.
There are ways around this, such as generating certificates on the fly, but you'd need the root certificate to be trusted on the browser.