I need to HTTP-GET a document over TLS without having the process let the OS resolve the domain name.
The reason for this is that I know that the IP address I want to use will be correct, but fetching for https://123.123.123.123 won't give me a valid SSL certificate. Yet I know that https://example.com will have the correct certificate and be reachable at the address 123.123.123.123. The OS's resolver will very likely yield an outdated IP address in my use case.
How can I make a request to a given IP address by explicitly specifying which domain name to use for the SSL certificate?
A possible solution would be to update the /etc/hosts file and add an entry like 123.123.123.123 example.com, but it requires root and I don't want to be editing that file only for this purpose.
As shown in my answer below, monkey-patching can also be an approach, though I'm not sure if it is really reliable in the long term.
Monkey-patching socket.getaddrinfo can be a solution when using the resuests package.
import socket
import requests
getaddrinfo_original = socket.getaddrinfo
def getaddrinfo_patched(*args, **kwargs):
if args[0] in ['example.com']:
return [(
socket.AF_INET, # family
socket.SOCK_STREAM, # type
socket.IPPROTO_TCP, # proto
'', # canonname
('123.123.123.123', 443) # sockaddr
)]
return getaddrinfo_original(*args, **kwargs)
socket.getaddrinfo = getaddrinfo_patched
r = requests.get('https://example.com/')
print(r.status_code, r.content)
But in this case maybe socket.gethostbyname and other functions would need to get patched as well.
Related
Is it possible to get the IP of the default interface with Sanic?
Here is how I do it with Socket. The idea is to do the same thing with Sanic.
import socket
hostname = socket.gethostname()
IP_address = socket.gethostbyname(hostname)
print(IP_address) # 192.168.1.239
It depends upon what information you want and how the app is being served (reverse proxy, etc).
Check out these values:
request.ip (connected interface)
request.remote_addr (likely what you want https://sanic.readthedocs.io/en/stable/sanic/api/core.html#sanic.request.Request.remote_addr)
request.conn_info (object with a bunch of details you may want)
I use dsnPython in a project.
I use many resolvers same as explained at Set specific DNS server using dns.resolver (pythondns).
In order to send several requests I need to dispatch my request on many IPs.
I have some IPs on my interface eth0.
Do you know a way to send a request through an specific IP ?
It's possible by using resolvers and source attribute :
import dns.resolver
my_resolver.nameservers = ['8.8.8.8']
answer = my_resolver.query(
qname = fqdn_port,
source = '1.2.3.4',
)
8.8.8.8 is the resolver IP
1.2.3.4 is an IP of server
Trying to send email with a certificate file using the following script:
import smtplib
client = smtplib.SMTP(myhost, myport)
client.ehlo()
client.starttls(certfile=mycertfile)
client.ehlo()
client.login(myusername, mypassword)
client.sendmail(sender, receiver, Message)
client.quit()
I get the following error:
SSLError: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib
I think documentations (smtplib.html and ssl.html) say I need to provide a private key. I only have the certificate file (base64 PEM format). My devops says that a private key is not required in this case because I do not need to identify the local side of the connection.
Question
Is there a way to send the email without providing the private key? If a private key is required, why?
There are two ways to use SSL/TLS: client authenticated and "basic" where the client is unauthenticated. In client authenticated connections, the both the server and the client send a certificate to the other. In "basic" only the server does.
If you pass neither a certificate nor a keyfile, smtplib will use a basic connection, where the client is authenticated.
If you use a certificate, it will be used for a client authenticated connection. In that case, the server will demand that the client shows it owns the certificate by signing a handshake message. For the client to be able to do that it also needs the private key, which can be either in the certificate file or as a separate keyfile.
Long story short, if you want to use a client certificate, you must also use a key. If not, you can just leave both empty.
OTOH, maybe you have a server certificate file or CA list you want to use with the connection?
In that case you need to pass it to ssl.wrap_socket in the ca_certs parameter. Since you use Python 2.6 there's no easy way to do that with smtplib (Python 3.3+ has a context argument to starttls).
How to solve this depends on your application. For example, if you do not need ssl for anything else, a hackish solution would be to monkey-patch ssl.wrap_socket with one that provides your ca_cert (as well as cert_reqs=CERT_REQUIRED, likely).
A more full blown solution would be to extend smtplib.SMTP with your own variant that does allow passing in those parameters.
Here's a monkey-patch taken from this page:
class SMTPExt(smtplib.SMTP):
"""
This class extends smtplib.SMTP and overrides the starttls method
allowing extra parameters and forwarding them to ssl.wrap_socket.
"""
def starttls(self, keyfile=None, certfile=None, **kwargs):
self.ehlo_or_helo_if_needed()
if not self.has_extn("starttls"):
raise SMTPException("STARTTLS extension not supported by server.")
(resp, reply) = self.docmd("STARTTLS")
if resp == 220:
self.sock = ssl.wrap_socket(self.sock, keyfile, certfile, **kwargs)
self.file = SSLFakeFile(self.sock)
# RFC 3207:
# The client MUST discard any knowledge obtained from
# the server, such as the list of SMTP service extensions,
# which was not obtained from the TLS negotiation itself.
self.helo_resp = None
self.ehlo_resp = None
self.esmtp_features = {}
self.does_esmtp = 0
return (resp, reply)
Using the root cert from requests
I'm using TOR to proxy connections but am having difficulty proxying DNS lookups via socket.gethostbyname("www.yahoo.com") -- I learned that it was not sending DNS traffic via proxy by sniffing traffic with wireshark. Here's a copy of the code I'm using
import StringIO
import socket
import socks # SocksiPy module
import stem.process
from stem.util import term
SOCKS_PORT = 7000
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', SOCKS_PORT)
socket.socket = socks.socksocket
def getaddrinfo(*args):
return [(socket.AF_INET, socket.SOCK_STREAM, 6, '', (args[0], args[1]))]
socket.getaddrinfo = getaddrinfo
socket.gethostbyname("www.yahoo.com") <--- This line is not sending traffic via proxy
Any help is greatly appreciated!
You're calling gethostbyname in the socket module. It doesn't know anything about your SOCKS socket; it is simply interacting with your operating system's name resolution mechanisms. Setting socket.socket = socks.socksocket may affect network connections made through the socket module, but the module does not make direct connections to DNS servers to perform name resolution so replacing socket.socket has no impact on this behavior.
If you simply call the connect(...) method on a socks.socksocket object using a hostname, the proxy will perform name resolution via SOCKS:
s = socks.socksocket()
s.connect(('www.yahoo.com', 80))
If you actually want to perform raw DNS queries over your SOCKS connection, you'll need to find a Python DNS module to which you can provide your socksocket object.
If you resolve the DNS yourself with Socks5 you may leak information about your own computer. Instead try tunneling with Proxifier, then to Tor. Alternatively you can use SocksiPy's Socks4A extension. This will make sure information is not leaked.
I need client IP address using python. I have tried below code but its not working in server:
from socket import gethostname, gethostbyname
ip = gethostbyname(gethostname())
print ip
On the server, I get '127.0.0.1' every time. Is there any way to find IP address of the client?
You're getting the IP address of your server, not of your server's clients.
You want to look at the request's REMOTE_ADDR, like this:
from bottle import Bottle, request
app = Bottle()
#app.route('/hello')
def hello():
client_ip = request.environ.get('REMOTE_ADDR')
return ['Your IP is: {}\n'.format(client_ip)]
app.run(host='0.0.0.0', port=8080)
EDIT #1: Some folks have observed that, for them, the value of REMOTE_ADDR is always the same IP address (usually 127.0.0.1). This is because they're behind a proxy (or load balancer). In this case, the client's original IP address is typically stored in header HTTP_X_FORWARDED_FOR. The following code will work in either case:
#app.route('/hello')
def hello():
client_ip = request.environ.get('HTTP_X_FORWARDED_FOR') or request.environ.get('REMOTE_ADDR')
return ['Your IP is: {}\n'.format(client_ip)]
EDIT #2: Thanks to #ArtOfWarfare's comment, I learned that REMOTE_ADDR is not required per PEP-333. Couple of observations:
The CGI spec does require REMOTE_ADDR:
The REMOTE_ADDR variable MUST be set to the network address of the client sending the request to the server.
However, PEP-333 does not explicitly require HTTP_REMOTE_ADDR, only going as far as this (emphasis mine):
A server or gateway SHOULD attempt to provide as many other CGI variables as are applicable.
All of the (admittedly few) web frameworks that I'm familiar with set HTTP_REMOTE_ADDR. AFAICT, it's a de facto "standard." But technically, YMMV.
Server might be behind a proxy. Use this for proxy and forward support:
request.environ.get('HTTP_X_FORWARDED_FOR') or request.environ.get('REMOTE_ADDR')
If you're trying to get the external IP, you will need to get it from an external source, i.e whatismyip.com or somewhere that offers an api. If that's what you're looking for, take a look at the Requests module http://docs.python-requests.org/