I am writing an script to analyse the countries of a list of domain names(e.g. third.second.first). The data set is pretty old and many of the fully qualified domain names cannot be found via socket.gethostbyname(domain_str) in python. Here are some of the alternatives I come up with:
Retrieving the ip of second.first if the ip of third.second.first
cannot be found and then find the country of that ip
This seems not to be a good idea since a dns A-record can map a subdomain to an ip different from its primary domain.
detect the country code of the domain name. e.g. if it is ..jp, it is from Japan
My questions are:
Is the first method acceptable ?
are there other methods to retrieve the country information of a domain name ?
Thank you.
I would recommend using the geolite2 module:
https://pypi.python.org/pypi/maxminddb-geolite2
So you could do something like this:
#!/usr/bin/python
import socket
from geolite2 import geolite2
def origin(ip, domain_str, result):
print("{0} [{1}]: {2}".format(domain_str.strip(), ip, result))
def getip(domain_str):
ip = socket.gethostbyname(domain_str.strip())
reader = geolite2.reader()
output = reader.get(ip)
result = output['country']['iso_code']
origin(ip, domain_str, result)
with open("/path/to/hostnames.txt", "r") as ins:
for domain_str in ins:
try:
getip(domain_str)
except socket.error as msg:
print("{0} [could not resolve]".format(domain_str.strip()))
if len(domain_str) > 2:
subdomain = domain_str.split('.', 1)[1]
try:
getip(subdomain)
except:
continue
geolite2.close()
Output:
bing.com [204.79.197.200]: US
dd15-028.compuserve.com [could not resolve]
compuserve.com [149.174.98.149]: US
google.com [172.217.11.78]: US
Related
I'm using AWS Cloudformation to create a stack and would like to get the value of the 'PublicIP' field out of the dictionary returned from describe_stacks().
The following schematic code does the work, but it is not resilient to changes in the dictionary structure:
#!/usr/bin/python
import sys
import boto3
import rest_client
if len(sys.argv) < 2:
print "Bad usage: missing stack name"
exit(1)
session = boto3.Session(profile_name='profile name')
client = session.client('cloudformation')
response = client.describe_stacks(StackName=sys.argv[1])
try:
ip = response['Stacks'][0]['Outputs'][1]['OutputValue']
print "Extracted instance IP address ({0})".format(ip)
except IndexError:
print "IP address not found"
exit(1)
Is there a more specific API I can use to get this field directly?
Unfortunately, AWS doesn't support filtering outputs by name. But it's pretty easy to do a filter that will:
#!/usr/bin/python
import sys
import boto3
import rest_client
OUTPUT_KEY = 'InstanceIp' # <-- Use the proper output name here
if len(sys.argv) < 2:
print "Bad usage: missing stack name"
exit(1)
stack_name = sys.argv[1]
session = boto3.Session(profile_name='profile name')
cf_resource = session.resource('cloudformation')
stack = cf_resource.Stack(stack_name)
try:
ip = filter(lambda x: x['OutputKey'] == OUTPUT_KEY, stack.outputs)[0]['OutputValue']
print "Extracted instance IP address ({0})".format(ip)
except IndexError:
print OUTPUT_KEY + " not found in " + stack_name
exit(1)
Also, I can assure you that it is future-proof as they never (to my knowledge) update the syntax of their response payloads once the API is officially released.
Currently I have a text file with mutiple IP's I am currently attempting to pull only the domain name from the set of information given using nslookup (code below)
with open('test.txt','r') as f:
for line in f:
print os.system('nslookup' + " " + line)
This works in so far that it pulls all the information from the first IP's. I can't get it passed the first IP but I'm currently attempting to clean up the information recived to only the Domain name of the IP. Is there any way to do that or do I need to use a diffrent module
Like IgorN, I wouldn't make a system call to use nslookup; I would also use socket. However, the answer shared by IgorN provides the hostname. The requestor asked for the domain name. See below:
import socket
with open('test.txt', 'r') as f:
for ip in f:
fqdn = socket.gethostbyaddr(ip) # Generates a tuple in the form of: ('server.example.com', [], ['127.0.0.1'])
domain = '.'.join(fqdn[0].split('.')[1:])
print(domain)
Assuming that test.txt contains the following line, which resolves to a FQDN of server.example.com:
127.0.0.1
this will generate the following output:
example.com
which is what (I believe) the OP desires.
import socket
name = socket.gethostbyaddr(‘127.0.0.1’)
print(name) #to get the triple
print(name[0]) #to just get the hostname
I finally have to throw in the towel after working with this for quite some time today. I am trying to retrieve all the IP addresses from a output that looks like this:
My Address: 10.10.10.1
Explicit Route: 192.168.238.90 192.168.252.209 192.168.252.241 192.168.192.209
192.168.192.223
Record Route:
I need to pull all the IP addresses between from 'Explicit Route' and 'Record Route'. I am using textfsm and I seem not to be able to get everything I need.
Use regex and string operations:
import re
s = '''My Address: 10.10.10.1
Explicit Route: 192.168.238.90 192.168.252.209 192.168.252.241 192.168.192.209
192.168.192.223
Record Route:'''
ips = re.findall(r'\d+\.\d+\.\d+\.\d+', s[s.find('Explicit Route'):s.find('Record Route')])
import re
with open('file.txt', 'r') as file:
f = file.read().splitlines()
for line in f:
found = re.findall(r'(?:[\d]{1,3})\.(?:[\d]{1,3})\.(?:[\d]{1,3})\.(?:[\d]{1,3})', line)
for f in found:
print(f)
Edit:
We open the txt and read by line, then for each line using regular exp. to find the ips ( can have 1-3 numbers, then . and repeat 4 times)
Does anyone know of a Python module or a solution for how I could lookup company info (Name preferably) via the ASN (autonomous system number) number?
There are lots of IP to ASN tools but that is not what I require.
ASN needs to be the input - company name output.
This website has the sort of info I need:
http://bgp.potaroo.net/cgi-bin/as-report?as=AS5607&view=2.0
Any ideas are appreciated!
Try this, it's maybe what you need
from cymruwhois import Client
import ipresolved
domain='facebook.com'
ips=ipresolved.getipresolvedfromdomain(domain)
c=Client()
for i in ips.json()['resolutions']:
ip=i['ip_address']
print('ip : '+ip)
r=c.lookup(ip)
print('asn number: ',r.asn)
print('asn owener : ',r.owner)
print('==============')
That information is available publicly on the CIDR-Report website.
This url has all the info you need and is updated daily. Big file, it might take a while to load :
http://www.cidr-report.org/as2.0/autnums.html
A slightly updated version of #Al-Pha answer:
Multi lookup:
from cymruwhois import Client
import socket
c = Client()
ip = socket.gethostbyname('globalresearch.ca')
for r in c.lookupmany([ip, "213.73.91.35"]):
print(r.__dict__)
# print(r.asn)
Single lookup:
c = Client()
r = c.lookup("213.73.91.35")
print(r.asn)
I've searched all over the web for the answer to my question and I've used aspects of what I've learned to help me get to this point. However, I've been unable to find the solution to get me where I need to be.
In the shortest way possible, I need to create a dictionary containing a dictionary and a list of values as read in from a file, and print the output.
I was able to do this using a statically created dictionary, but I seem to be unable to create the dictionary in the same format while reading in from a file.
Here is the code I was able to get working:
routers = {'fre-agg1': {'interface Te0/1/0/0': ["rate-limit input 135", "rate-limit input 136"],
'interface Te0/2/0/0': ["rate-limit input 135", "rate-limit input 136"]},
'fre-agg2': {'interface Te0/3/0/0': ["rate-limit input 135", "rate-limit input 136", "rate-limit input 137"]}}
for rname in routers:
print rname
for iname in routers[rname]:
print iname
for int_config in routers[rname][iname]:
print int_config
The output of this prints exactly in the format I need it to be:
fre-agg2
interface Te0/3/0/0
rate-limit input 135
rate-limit input 136
rate-limit input 137
fre-agg1
interface Te0/1/0/0
rate-limit input 135
rate-limit input 136
interface Te0/2/0/0
rate-limit input 135
rate-limit input 136
The file I am trying to read in is in a different format:
ama-coe:interface Loopback0
ama-coe: ip address 10.1.1.1 255.255.255.255
ama-coe:interface GigabitEthernet0/0/0
ama-coe: description EGM to xyz Gi2/0/1
ama-coe: ip address 10.2.1.1 255.255.255.254
ama-coe:interface GigabitEthernet0/0/1
ama-coe: description EGM to abc Gi0/0/1
ama-coe: ip address 10.3.1.1 255.255.255.254
For this file, I'd like the output of the file as the same output shown above, with the interface configuration listed under the interface name, listed under the device name
ama-coe
interface Loopback0
ip address 10.1.1.1 255.255.255.255
interface GigabitEthernet0/0/0
etc etc etc
So far, here is the code I have:
routers = {}
with open('cpe-interfaces-ipaddress.txt') as inputFile:
inputData = inputFile.read().splitlines()
for rname in inputData:
device, stuff = rname.split(':')
if not device in routers:
routers[device] = None
elif stuff == "interface":
routers[device][None] = stuff
I know this code is extremely incomplete but I can't for the life of me figure out the dictionary and list structure as I did when statically creating the dict.
Any help that can be provided would be greatly appreciated.
Thank you.
routers = {}
with open('cpe-interfaces-ipaddress.txt') as inputFile:
cur_interface = None
for rname in inputFile:
device, stuff = rname.strip().split(':')
print device, stuff, cur_interface
if not device in routers:
routers[device] = {}
if stuff.startswith("interface"):
key_word, interface = stuff.split(' ')
routers[device].setdefault(interface, [])
cur_interface = interface
else: # ip address
routers[device][cur_interface].append(stuff)
Not knowing if this meet your need. I made an assumption that each ip address is belong to previous interface.
The way using a dictionary to organize stuff is common. You should learn some built-in methods, such as setdefault and append.