I'm making a small tool that tests CDN performance and would like to check where the response comes from. I thought of getting the host's IP and then using one of the geolocation API's on github to check the country.
I've tried doing so with
import socket
...
raw._fp.fp._sock.getpeername()
...however that only works when i use stream=True for the request and that in turn breaks the tool's functionality.
Is there any other option to get the server ip with requests or in a completely different way?
The socket.gethostbyname() function from Python's socket library should solve your problem. You can check it out in the Python docs here.
Here is an example of how to use it:
import socket
url="cdnjs.cloudflare.com"
print("IP:",socket.gethostbyname(url))
All you need to do is pass the url to socket.gethostbyname() and it will do the rest. Just make sure to remove the http:// before the URL because that will trip it up.
I could not get Akilan's solution to give the IP address of a different host that I was using. socket.gethostbyname() and getpeername() were not working for me. They are not even available. His solution did open the door.
However, navigating the socket object, I did find this:
socket.getaddrinfo('host name', 443)[0][4][0]
I wrapped this in a try/except block.
Maybe there is a prettier way.
While trying to perform a tcp TWH in scapy I encountered a problem.
When my host receives the syn/ack, the kernel ip/tcp stack aborts the handshake by sending rst.
It happens because the first packet sent (the syn) is transparent to the os due to the raw socket that scapy is using.
In some other questions regarding the same issue the solution was to set up iptables to drop packets with the rst flag.
Does anyone have a solution for windows?
(I prefer not to use the FW unless there is no choice)
Thanks!
That's a common problem, and it is not specific to Windows (or Scapy, for that matters - you can face similar issues with Masscan, for example).
You have totally understood and describe it: the problem lies in the IP address being "shared" between Scapy and your host's IP stack.
You have two options:
Use a firewall to prevent your host from getting the SYN/ACK packet (Scapy will get it anyway) or to prevent your host from sending a RST packet.
Use a different IP address with Scapy. This is a bit more difficult because it requires some kind of ARP proxy (or a Scapy script) to answer ARP request for the IP address you will use with Scapy.
Hope this helps, happy hacking!
I read many articles and found how to send custom packet based on IP using socket(AF_INET, SOCK_RAW, IPPROTO_RAW). But I want to send completely custom packet starting from Ethernet header. I can't send ARP packet if I can't form Ethernet header cause ARP don't based IP. Please, help!
P.S. I am on Windows 7, not Linux :(
In python, the easiest way is to use the cross-platform scapy library. It’s well known for that
Scapy
You can sniff, send.... lots of packets, add your own protocols, use existing ones... and it works on nearly all platforms. (On windows, it uses Npcap/Winpcap)
You can then build an ARP packet using
from scapy.all import *
pkt = ARP()
pkt.show()
sendp(Ether(dst=..., src=...)/pkt)
Which will create such packets
###[ ARP ]###
hwtype= 0x1
ptype= 0x800
hwlen= 6
plen= 4
op= who-has
hwsrc= 00:50:56:00:1e:3d
psrc= 212.83.148.19
hwdst= 00:00:00:00:00:00
pdst= 0.0.0.0
To build the packet, use the / operator
ether = Ether()
ether.src = “00:00:00:00:00:00”
ether.dst = “ff:ff:ff:ff:ff:ff”
arp = ARP()
[edit arp.psrc, arp.pdst, arp.hwsrc, arp.hwdst]
packet = ether/arp
sendp(packet) # sens packet on layer 2
Have a look at its Scapy documentation
There's no cross-platform way to do what you want, of course.
Python is just passing these values through to the underlying C API. So, on a platform with a complete BSD sockets API including the packet interface, you can just use AF_PACKET and the other appropriate flags. (I think you'd want ETH_P_ALL or ETH_P_802_2 rather than IPPROTO_RAW, or you might want SOCK_DGRAM… anyway, read your platform's man packet and figure it out based on what you actually need to do.) On Linux, at least most of these flags should be available on the SOCKET module; on other Unixes, they often don't get picked up, so you have to manually look them up in the system headers and use hardcoded constant ints in your code.
Unfortunately, if you're on Windows, this doesn't do any good. While WinSock has a feature they call TCP/IP Raw Sockets, accessed via SOCK_RAW, and recent versions of Python do expose this, it's just an emulation of a small subset of what actual BSD sockets implementations can do, and doesn't offer any way to go below the IP level (hence the name of the feature).
Microsoft's solution to this used to be that you'd write a TDI provider with the DDK, which would implement whatever protocol you wanted to expose as another WinSock protocol, and then your application-level code could just use that protocol the same way it would use, e.g., TCP. From the linked document above, it looks like this is obsolete, but the replacement seems like the same idea but with different acronyms (and, presumably, different APIs).
On the other hand, I'm pretty sure Windows already comes with protocols for ARP, ICMP, and anything other protocols needed for its usermode tools (because they obviously can't be written around raw packets). I'm just not sure how to access them.
As far as I know, the usual alternative is to use WinPcap.
While this was originally designed to be a packet capture library, it also implements a complete link-level socket interface that you can use for sending and receiving raw frames.
And there are Python wrappers for it, like WinPcapy.
So, as long as you can require that the WinPcap driver be installed, you can write ARP code, etc., on Windows in Python. It's just different from doing it on Unix.
In fact, one of the examples on the front page of WinPcapY, "Easy Packet sending", should get you started:
from winpcapy import WinPcapUtils
# Build a packet buffer
# This example-code is built for tutorial purposes, for actual packet crafting use modules like dpkt
arp_request_hex_template = "%(dst_mac)s%(src_mac)s08060001080006040001" \
"%(sender_mac)s%(sender_ip)s%(target_mac)s%(target_ip)s" + "00" * 18
packet = arp_request_hex_template % {
"dst_mac": "aa"*6,
"src_mac": "bb"*6,
"sender_mac": "bb"*6,
"target_mac": "cc"*6,
# 192.168.0.1
"sender_ip": "c0a80001",
# 192.168.0.2
"target_ip": "c0a80002"
}
# Send the packet (ethernet frame with an arp request) on the interface
WinPcapUtils.send_packet("*Ethernet*", packet.decode("hex"))
I'm trying to intercept all the requests received by a website, to get a certain file. For example, when you use Firefox's network monitor. Can I do that in Python? Sorry for being so vague. I'd like to get all the URLs that the website requests, like you can see in the picture. Example: the favicon, js files, xml files, etc.
Example:
So you probably need a packet sniffer like tcpdump. The best python sniffer I know is scapy. Here is in example of how HTTP may be sniffed with it:
http://www.r00tsec.com/2013/12/simple-sniffer-http-request-and-http.html
Note that you couldn't do that trick with HTTPS. Also packet sniffing usually requires root privileges on a host system.
I;m a completely new in network programming and a starter in python.
I want so set a socket in Python to just accept certain packages.
I have the following values at my disposal:
destination address and port
source MAC address
ethtype (own)
how can I set a filter to my socket class to only accept packets for me (at dest address,port)
and/or the right ethtype, and/or send by the device with the known mac address ?
Any help would be greatly appreciated, I tried to look via Google, but the amount of hits is incredible, and I haven't been able to find an answer that solves my question so far.
Thanx,
Arthur
I would recommend you to use scapy. It is a great tool for crafting custom packets and to do lot of other stuff.
You can add filters in the scapy's sniff() to capture the packets you desire. You can also use scapy with your own python programm.
More over you'll find ton of tutorials on the internet on how to use scapy.