We Keep Coding

sort list in python by descending IP address and string order

I have a list of strings like the below:
ip route vrf t141 0.0.0.0/0 10.76.102.101 name edge
ip route vrf t141 10.0.0.0/8 10.76.101.101
ip route vrf t141 172.16.0.0/12 10.76.101.101
ip route vrf t141 192.168.0.0/16 10.76.101.101
ip route vrf t141 29.0.0.0/8 10.76.101.101
ip route vrf t20 0.0.0.0/0 10.76.102.27
ip route vrf t20 10.0.0.0/8 10.76.101.27
ip route vrf t20 172.16.0.0/12 10.76.101.27
ip route vrf t20 192.168.0.0/16 10.76.101.27
ip route vrf t20 29.0.0.0/8 10.76.101.27
I need to sort the list by descending vrf name (t141, t20) as well as from descending IP address. sort and sorted are working ok for sorting the string based on vrf name but is not working ok for the IP addresses: the line ip route vrf t141 29.0.0.0/8 10.76.101.101 should be after ip route vrf t141 10.0.0.0/8 10.76.101.101 as well as ip route vrf t20 29.0.0.0/8 10.76.101.27 should be after ip route vrf t20 10.0.0.0/8 10.76.101.27 and not at the bottom.
Here the diff for better reading
ip route vrf t10 0.0.0.0/0 10.76.102.25
ip route vrf t10 10.0.0.0/8 10.76.101.25
+ip route vrf t10 29.0.0.0/8 10.76.101.25
ip route vrf t10 172.16.0.0/12 10.76.101.25
ip route vrf t10 192.168.0.0/16 10.76.101.25
-ip route vrf t10 29.0.0.0/8 10.76.101.25
ip route vrf t141 0.0.0.0/0 10.76.102.101 name edge
ip route vrf t141 10.0.0.0/8 10.76.101.101
+ip route vrf t141 29.0.0.0/8 10.76.101.101
ip route vrf t141 172.16.0.0/12 10.76.101.101
ip route vrf t141 192.168.0.0/16 10.76.101.101
-ip route vrf t141 29.0.0.0/8 10.76.101.101
Any suggestion is greatly appreciated

You can use regex to accomplish this result within the sorted method.
import re
ls = ['ip route vrf t141 0.0.0.0/0 10.76.102.101 name edge',
'ip route vrf t141 10.0.0.0/8 10.76.101.101',
'ip route vrf t141 172.16.0.0/12 10.76.101.101',
'ip route vrf t141 192.168.0.0/16 10.76.101.101',
'ip route vrf t141 29.0.0.0/8 10.76.101.101',
'ip route vrf t20 0.0.0.0/0 10.76.102.27',
'ip route vrf t20 10.0.0.0/8 10.76.101.27',
'ip route vrf t20 172.16.0.0/12 10.76.101.27',
'ip route vrf t20 192.168.0.0/16 10.76.101.27',
'ip route vrf t20 29.0.0.0/8 10.76.101.27']
print(*sorted(ls, key=lambda x: (int(re.search(r"(?:vrf t)(\w*)", x).group(1)), " ".join([i for i in x.split(" ") if not re.sub("[0-9.\/]", "", i)]))), sep="\n")
Output
ip route vrf t20 0.0.0.0/0 10.76.102.27
ip route vrf t20 10.0.0.0/8 10.76.101.27
ip route vrf t20 172.16.0.0/12 10.76.101.27
ip route vrf t20 192.168.0.0/16 10.76.101.27
ip route vrf t20 29.0.0.0/8 10.76.101.27
ip route vrf t141 0.0.0.0/0 10.76.102.101 name edge
ip route vrf t141 10.0.0.0/8 10.76.101.101
ip route vrf t141 172.16.0.0/12 10.76.101.101
ip route vrf t141 192.168.0.0/16 10.76.101.101
ip route vrf t141 29.0.0.0/8 10.76.101.101

How to replace words/digits in a file

I'm trying to replace the global IP inside a file with a private one and create a mapping of them, so i can revert it back even if part of the new string is different.
I'm stuck at the point of replace the global IP with the bogus one and write it to a file.
Starting file example:
ip route 192.168.1.0 255.255.0.0 10.10.10.2
ip route 192.168.1.0 255.255.0.0 1.1.1.2
ip route 1.1.1.1 255.255.0.0 1.1.1.3
interface FastEthernet1
ip address 1.1.1.1
duplex auto
speed auto
Wanted end result, some wording may change before revert back:
ip route ipv4 192.168.1.0 255.255.0.0 10.10.10.2
ip route ipv4 192.168.1.0 255.255.0.0 10.1.1.11
ip route ipv4 10.1.1.10 255.255.0.0 10.1.1.12
interface FastEthernet1
ip address 10.1.1.10
duplex auto
speed auto
The mapping I though is a dictionary like this:
mapping = {
'1.1.1.2': "10.1.1.10",
'1.1.1.1': "10.1.1.10",
'1.1.1.3': "10.1.1.30
}
I came out with this script until now, but it not doing what I want:
import re
import ipaddress
def load_file(file) -> str:
with open(file, 'r') as f:
return f.read()
def find_ips(config) -> set:
ip_regex = '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}'
match = set(re.findall(ip_regex, config))
return match
def mapping_ip(ips) -> dict:
counter = 0
ip_table = {}
for ip in ips:
ip4_type = ipaddress.IPv4Address(ip)
if ip4_type.is_global:
counter += 1
private = ipaddress.IPv4Address('10.1.1.10') + counter
ip_table.update({
ip: str(private),
})
return ip_table
def replace(mapping, s_file, d_file):
with open(s_file, 'r') as reader, open(d_file, 'w') as writer:
for line in reader:
for orig, temp in mapping.items():
if orig in line:
x = line.replace(orig, temp)
writer.write(x)
Any suggestion on how should I do the replace funcion?
Only the IP can be change, the rest of the string need to stay as it's(revert back process).

You can simply use string replace on the lines of your source file:
Create source file:
t = """ip route 192.168.1.0 255.255.0.0 10.10.10.2
ip route 192.168.1.0 255.255.0.0 1.1.1.2
ip route 1.1.1.1 255.255.0.0 1.1.1.3
interface FastEthernet1
ip address 1.1.1.1
duplex auto
speed auto"""
with open("t.txt","w") as f:
f.write(t)
Replace stuff and write to "mod.txt":
mapping = {
'1.1.1.2': "10.1.1.10",
'1.1.1.1': "10.1.1.10",
'1.1.1.3': "10.1.1.30"
}
with open("mod.txt","w") as m, open("t.txt") as data:
for line in data:
for key,replacewith in mapping.items():
line = line.replace(key,replacewith)
m.write(line)
with open("mod.txt") as f:
print(f.read())
Output:
ip route 192.168.1.0 255.255.0.0 10.10.10.2
ip route 192.168.1.0 255.255.0.0 10.1.1.10
ip route 10.1.1.10 255.255.0.0 10.1.1.30
interface FastEthernet1
ip address 10.1.1.10
duplex auto
speed auto
This will try to replace each line m times (m == len(mapping)) and is not very speedy due to creating lots of intermediate strings (if something got replaced) - it is more a hacky solution to your problem.

You could harness re.sub in this case, following way:
import re
txt = 'ip route 192.168.1.0 255.255.0.0 10.10.10.2\nip route 192.168.1.0 255.255.0.0 1.1.1.2\nip route 1.1.1.1 255.255.0.0 1.1.1.3\ninterface FastEthernet1\nip address 1.1.1.1\nduplex auto\nspeed auto'
out = re.sub(r'1\.1\.1\.([1-3])','10.1.1.\g<1>0',txt)
print(out)
Output:
ip route 192.168.1.0 255.255.0.0 10.10.10.2
ip route 192.168.1.0 255.255.0.0 10.1.1.20
ip route 10.1.1.10 255.255.0.0 10.1.1.30
interface FastEthernet1
ip address 10.1.1.10
duplex auto
speed auto
For simplicity I hardcoded txt, most important line is that of re.sub:
out = re.sub(r'1\.1\.1\.([1-3])','10.1.1.\g<1>0',txt)
It replaces substrings of txt which match first argument with second argument, first argument contain one group ([1-3]) which is later referenced in second argument (\g<1>) thus it in fact of executing following relacement:
1.1.1.1 to 10.1.1.10
1.1.1.2 to 10.1.1.20
1.1.1.3 to 10.1.1.30
However keep in mind that re.sub is working in single-pass fashion, unlike repeating usage of .replace method of str.

Python UDP Sockets with Multiple Interfaces

I'm Writing a script in python2.7 on a windows XP machine. The machine is connected to multiple networks using different network cards.
I'm running into an issue where I've bound a UDP Socket to a specific interface(I understand that you can accomplish this in windows by just providing the network cards existing IP address)
self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self.sock.bind(('10.31.9.0', 6466)) #<<< 10.31.9.0 is address of desired card
I then set the timeout to 5s
self.sock.settimeout(5)
Then I try to send a message out to a server that I can prove exists and works. then wait for a response.
self.destintation = ('10.42.40.34', 62434)
# Send the msg
self.sock.sendto(msg, self.destintation)
# receive data
reply, addr = self.sock.recvfrom(1024)
However a socket.timeout is always thrown. so I open up wire shark to see what is going wrong, and it turns out that my initial message never gets sent on the desired interface.
What I do see is an arp broadcast on a different interface(10.10.10.12
) from my machine asking who is attached to my desired destination IP:
1 0.000000 IntelCor_8c:6d:97 Broadcast ARP 42 Who has 10.42.40.34? Tell 10.10.10.12
Of course there is no response to the broadcast because the 10.42.40.34 Address/machine is not reachable from the 10.10.10.12 interface
How do I tell Python to send the ARP broadcast out on '10.31.9.0'? What have I done Wrong?
EDIT:
Additional Information>
The network for the interface I am using is a Class B
(netmask is 255.255.0.0)
The interface IP is : 10.31.9.0
The target IP is: 10.42.40.34.
I am wondering if the issue is a result of my target sitting on a separate subnet. However, as described in a related issue here. there is traffic from the server to me... =/
UPDATE:
Results of "route PRINT 10*"
Active Routes:
Network Destination Netmask Gateway Interface Metric
10.0.0.0 255.0.0.0 10.10.10.12 10.10.10.12 10
10.10.10.12 255.255.255.255 127.0.0.1 127.0.0.1 10
10.31.0.0 255.255.0.0 10.31.9.0 10.31.9.0 10
10.31.9.0 255.255.255.255 127.0.0.1 127.0.0.1 10
10.255.255.255 255.255.255.255 10.10.10.12 10.10.10.12 10
10.255.255.255 255.255.255.255 10.31.9.0 10.31.9.0 10
Default Gateway: 153.4.84.1
===========================================================================
Persistent Routes:
None
UPDATE #2
Full route PRINT
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 153.4.84.1 153.4.85.81 10
10.10.0.0 255.255.0.0 10.10.10.12 10.10.10.12 10
10.10.10.12 255.255.255.255 127.0.0.1 127.0.0.1 10
10.31.0.0 255.255.0.0 10.31.9.0 10.31.9.0 10
10.31.9.0 255.255.255.255 127.0.0.1 127.0.0.1 10
10.255.255.255 255.255.255.255 10.10.10.12 10.10.10.12 10
10.255.255.255 255.255.255.255 10.31.9.0 10.31.9.0 10
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1
153.4.84.0 255.255.252.0 153.4.85.81 153.4.85.81 10
153.4.85.81 255.255.255.255 127.0.0.1 127.0.0.1 10
153.4.255.255 255.255.255.255 153.4.85.81 153.4.85.81 10
192.168.56.0 255.255.255.0 192.168.56.1 192.168.56.1 20
192.168.56.1 255.255.255.255 127.0.0.1 127.0.0.1 20
192.168.56.255 255.255.255.255 192.168.56.1 192.168.56.1 20
224.0.0.0 240.0.0.0 10.10.10.12 10.10.10.12 10
224.0.0.0 240.0.0.0 10.31.9.0 10.31.9.0 10
224.0.0.0 240.0.0.0 153.4.85.81 153.4.85.81 10
224.0.0.0 240.0.0.0 192.168.56.1 192.168.56.1 20
255.255.255.255 255.255.255.255 10.10.10.12 10.10.10.12 1
255.255.255.255 255.255.255.255 10.31.9.0 10.31.9.0 1
255.255.255.255 255.255.255.255 153.4.85.81 153.4.85.81 1
255.255.255.255 255.255.255.255 192.168.56.1 192.168.56.1 1
255.255.255.255 255.255.255.255 192.168.56.1 5 1
Default Gateway: 153.4.84.1
===========================================================================
Persistent Routes:
None

Given the output from "route", it looks like you're 10.10.10.12 and 10.31.9.0 interfaces have been configured with overlapping subnets. The OS is choosing to use 10.10.10.12 for all 10.x.x.x addresses as it's the first rule that applies.
Having overlapping subnets is normally a network configuration error: it's probably intended that 10.10.x.x and 10.31.x.x are the valid subnets and both should use a netmask of 255.255.0.0, and so the current 255.0.0.0 netmask used by the 10.10.10.12 interface is incorrect.
(It may be possible to 'fudge' a fix, if the intention is to make all 10.x.x.x requests use the 10.10.10.12 interface except for those in 10.31.x.x which should use the 10.31.9.0 address, by changing the 'metric' of the 10.31.0.0 routing rule so that anything for 10.31.x.x addresses matches that rule before the 10.x.x.x rule is checked. You can use the route command to make that change, but it's definitely not recommended! Fixing the overlapping subnets is the proper solution.)

Turns out, the Packets that my "server" was sending where not IP kosher. so they where getting rejected at the network and transport layers. Solution was to not use python socket class, but instead communicate directly to OSI-L2 using winpcap and ctypes

How do I use "not in" correctly to identify an item is not in a list of strings in Python?

I have a list of tuples comprised of an interface name and access list name. Like this:
exempt_int_acl_tuple=[('(app)', 'access-list nonat'), ('(app2)', 'access-list nonat')]
Sample config to search through is something like this:
config=['access-list nonat extended permit ip 10.0.0.0 255.0.0.0 10.0.0.0 255.0.0.0','access-list nonat extended permit ip 10.0.0.0 255.0.0.0 192.168.15.0 255.255.255.0','access-list nonat extended permit ip 10.0.0.0 255.0.0.0 1.1.1.1 255.255.255.240','blah','blah blah','some more blah']
I have a list strings in which I look for a specific pattern. If the list matches the pattern, I add it to a new list called exempt_acl.
So my code looks like this:
exempt_acl=[]
for interface,acl_name in exempt_int_acl_tuple:
for someline in config:
acl_statement=acl_name+' extended permit ip '
if (acl_statement in someline) and (someline not in exempt_acl):
exempt_acl.append(someline)
In this case, the access list name in the tuple is repeated so the config file is searched for twice. So exempt_acl looks like this:
['access-list nonat extended permit ip 10.0.0.0 255.0.0.0 10.0.0.0 255.0.0.0 ', 'access-list nonat extended permit ip 10.0.0.0 255.0.0.0 192.168.15.0 255.255.255.0 ', 'access-list nonat extended permit ip 10.0.0.0 255.0.0.0 1.1.1.1 255.255.255.240 ']
['access-list nonat extended permit ip 10.0.0.0 255.0.0.0 10.0.0.0 255.0.0.0 ', 'access-list nonat extended permit ip 10.0.0.0 255.0.0.0 192.168.15.0 255.255.255.0 ', 'access-list nonat extended permit ip 10.0.0.0 255.0.0.0 1.1.1.1 255.255.255.240 ']
However, the same lines are added twice, and it is creating a list of lists, instead of a list of strings. I thought the boolean evaluation for (someline not in exempt_acl) would prevent the line being added a second time, but it is. What am I doing wrong? I am using file.read().splitlines() to read the config file in case that makes a difference.

To start with, delistify someline by saying someline = someline[0].
If lines still appear to be appended twice, it probably comes down to the ambiguity of the condition if (acl_statement in someline) . If two lines are slightly different (e.g. different amounts of whitespace) but nonetheless contain the same acl_statement, both will be appended.

Assuming you have a file named config.txt with these contents:
access-list nonat extended permit ip 10.0.0.0 255.0.0.0 10.0.0.0 255.0.0.0
access-list nonat extended permit ip 10.0.0.0 255.0.0.0 192.168.15.0 255.255.255.0
access-list nonat extended permit ip 10.0.0.0 255.0.0.0 1.1.1.1 255.255.255.240
And test.py containing this:
exempt_acl = [('(app)', 'access-list nonat'),
('(app2)', 'access-list nonat'),
]
results = []
with open('config.txt') as config:
for line in config:
line = line.strip()
for _,acl_name in exempt_acl:
acl_statement = acl_name + ' extended permit ip '
if line.startswith(acl_statement) and line not in results:
results.append(line)
for result in results:
print(result)
That should do what you're looking for, if I understand your question correctly.
I'm not sure exactly what you were doing in your original code base - the code in your question was definitely not an MCVE. This code also works as expected:
config = [line.strip() for line in open('config.txt')]
results = []
for _, acl_name in exempt_acl:
for line in config:
acl_statement = acl_name + ' extended permit ip '
if acl_statement in line and line not in results:
results.append(line)
for result in results:
print(result)

I found the answer to why the configuration was being printed twice.
Main was calling the function twice. Once incorrectly (not mapping the function output to a value, but performing the printing involved in the debugging), and once correctly, hence not affecting output.
Thanks to all who assisted. I'm very grateful for your help, and I learned a few things too! The most important lesson for me was to formulate an MCVE before posting in the future.

Regex Match on IP Address Before Known String

I'm using nmap to search for hostnames and related IPs on my local (home) network. I can pull a string that looks something like this:
Starting Nmap 6.40 ( http://nmap.org ) at 2014-02-15 22:20 PST
Nmap scan report for 192.168.1.1
Host is up (0.00025s latency).
MAC Address: ZZ:ZZ:11:ZZ:ZZ:ZZ (Cisco-Linksys)
Nmap scan report for 192.168.1.2
Host is up (0.0084s latency).
MAC Address: ZZ:ZZ:A1:2E:ZZ:ZZ (Apple)
Nmap scan report for 192.168.1.9
Host is up (0.012s latency).
MAC Address: A4:ZZ:57:17:ZZ:ZZ (Seiko Epson)
Nmap scan report for 192.168.1.103
Host is up (0.036s latency).
MAC Address: ZZ:ZZ:6D:05:ZZ:ZZ (Apple)
I know that I can put together a regular expression to give me the IP address directly above the "Seiko Epson" line, but I cannot figure out how to do it.
I'm specifically looking for a way to find the IP address of the host that I'm searching for, I'm currently using:
(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)
to find IP addresses, but I do not know how to augment this to find the IP address above a given string.

If you get the whole input as a single string, then
You firstly search for a fixed string Nmap scan report for,
nextly remember a sequence 0-9 or . (which should be there) as the output IP address,
then skip until the MAC addr part (containing :),
skip until the next opening paren,
and finally check if the string inside parens is Seiko Epson.
Example:
>>> inp='''Starting Nmap 6.40 ( http://nmap.org ) at 2014-02-15 22:20 PST
... Nmap scan report for 192.168.1.1
... Host is up (0.00025s latency).
... MAC Address: ZZ:ZZ:11:ZZ:ZZ:ZZ (Cisco-Linksys)
... Nmap scan report for 192.168.1.2
... Host is up (0.0084s latency).
... MAC Address: ZZ:ZZ:A1:2E:ZZ:ZZ (Apple)
... Nmap scan report for 192.168.1.9
... Host is up (0.012s latency).
... MAC Address: A4:ZZ:57:17:ZZ:ZZ (Seiko Epson)
... Nmap scan report for 192.168.1.103
... Host is up (0.036s latency).
... MAC Address: ZZ:ZZ:6D:05:ZZ:ZZ (Apple)'''
>>> import re
>>> r1 = re.compile(r'Nmap scan report for ([0-9.]*)[^:]*[^(]*\(Seiko Epson\)')
>>> r1.search(inp).group(1)
'192.168.1.9'
The idea behind [^...]'s is finite state machine.