i'm trying to save output of a python code that actually it has written with os.system
import os
os.system("sudo nmap -p5433 -P0 -oG - -sS 127.0.0.1 | \
sed -n 's/.* \([0-9\.]\{7,\}\).*\/open\/.*/\1/p' > result.txt")
As you can see at the end of a line,output should be save in "result.txt" and i'm sure the output should be an ip (127.0.0.1) but the output is something like this:
the output is a symbol or something like that,is there any way that i can save the output of this code correctly?
If I understood you correctly, you want to save only hosts, that are found in your nmap scan and have status up?
If so you could use:
sudo nmap -p5433 -P0 -oG - -sS 127.0.0.1 | grep 'Up' | grep -oP '\d*\.\d*\.\d*\.\d* > result.txt'
You use nmap, then you grep all lines, containing the Up status, then you only grep the ip-addresses and put them in the text file
If you just want all ip, addresses that return in the scan (without status Up check) you can simplify:
sudo nmap -p5433 -P0 -oG - -sS 127.0.0.1 | grep -oP '\d*\.\d*\.\d*\.\d*' > result.txt
Although there will be duplicates if nmap prints something like:
# Nmap 7.80 scan initiated Fri Dec 25 14:05:05 2020 as: nmap -p5433 -P0 -oG - -sS 127.0.0.1
Host: 127.0.0.1 (localhost) Status: Up
Host: 127.0.0.1 (localhost) Ports: 5433/closed/tcp//pyrrho///
As there are two lines with the same ip address
Related
this is my first question here, I am so excited :)
It's maybe a noob question but I don't understand it...
I'am trying to upgrade an Edgerouter firmware (https://www.ui.com/edgemax/edgerouter-pro/) with Gitlab-CI and Ansible. The stages are absolutely identical but the stdout of the same task, with the same ansible.cfg, with the same gitlab-runner, in the same pipeline etc., differs:
STAGE1
CI Deployment Docker Image:
ansible-playbook 2.8.3
python version = 3.7.4
Edgerouter:
USER1#HOSTNAME1:~$ python --version
Python 2.7.13
USER1#HOSTNAME1:~$ show system image
The system currently has the following image(s) installed:
v2.0.8.5247496.191120.1124 (running image) (default boot)
v2.0.8.5247496.191120.1124-1
OUTPUT
...identical verbose output, but:
ok: [HOSTNAME1] => changed=false
invocation:
module_args:
commands:
- show version | grep "Build ID" | cut -d ':' -f 2 | tr -d ' '
interval: 1
match: all
retries: 10
wait_for: null
stdout:
- '5247496'
stdout_lines: <omitted>
Works like a charm!
BUT:
STAGE2
CI Deployment Image:
ansible-playbook 2.8.3
python version = 3.7.4
Edgerouter
USER2#HOSTNAME2:~$ python --version
Python 2.7.13
USER2#HOSTNAME2:~$ show system image
The system currently has the following image(s) installed:
v2.0.8.5247496.191120.1124 (running image) (default boot)
v2.0.8.5247496.191120.1124-1
OUTPUT
...identical verbose output, but:
ok: [HOSTNAME2] => changed=false
invocation:
module_args:
commands:
- show version | grep "Build ID" | cut -d ':' -f 2 | tr -d ' '
interval: 1
match: all
retries: 10
wait_for: null
stdout:
- |-
show version | grep "Build ID" | cut -d ':' -f 2 |
tr -d ' '
5247496
stdout_lines: <omitted>
DOES NOT...
This is the Ansible task:
- name: get installed firmware build ID to compare with config
edgeos_command:
commands: show version | grep "Build ID" | cut -d ':' -f 2 | tr -d ' '
register: installed_firmware_build_id
tags: router-upgrade
What am I missing here?
I ended up like this:
- set fact:
edgeos_command:
commands: show version
register: installed_firmware_version_raw
tags: router-upgrade
- set_fact:
installed_firmware_version: "{{ (installed_firmware_version_raw.stdout[0] | regex_findall('Version:\\s+(v.+)'))[0] }}"
tags: router-upgrade
- set_fact:
installed_firmware_build_id: "{{ (installed_firmware_version_raw.stdout[0] | regex_findall('Build ID:\\s+(\\d+)'))[0] }}"
tags: router-upgrade
I have a text file like below with host, site and port to telnet
Hostname site port
appwlsqa02.comp.xxx.com fgh-fst13-scan.comp.xxx.com 1521
appwlsqa03.comp.xxx.com fgh-fst23-scan.comp.xxx.com 1521
cappwlsqa01.comp.xxx.com fgh-fst13-scan.comp.xxx.com 1521
My goal is to ssh each hostname and execute telnet command
for example for the first row in the file
ssh appwlsqa02.comp.xxx.com
then,
telnet fgh-fst13-scan.comp.xxx.com 1521
I have tried below:
while read HOST site port ; do ssh $HOST "echo exit | telnet $site $port " < /dev/null; done < text.txt`
I have to read hostnames as one variable $HOST and host to check as $site and port as$port
Update:
Here is the script i have developed based on sugegstions:
# /bin/bash
while read HOST site port ;
do
echo $HOST $site $port
ssh -n $HOST "telnet $site $port | grep "Connected" " >> t.txt &
PID=$!
sleep 2
kill "$PID"
if [ -s t.txt ]
then
echo from $HOST to $site $port is open >> succ.txt
else
echo from $HOST to $site $port is closed >> fail.txt
fi
rm t.txt
done < text.txt
When trying to use telnet, you need to run something like expect. Telnet grabs the terminal io. Expect replaces the terminal with a pseudo terminal and allows you to script the input and output.
I have the following TcpDump command written in Python but it doesn't give me any output file with the requested packets although I have TcpDump installed and tested on my Ubuntu VM :
command = 'sudo /usr/sbin/tcpdump -i eth1 {} -c {} -s 0 -w {}'\
.format( 'tcp host 10.0.2.15','30000',
'/home/results/xyz.pcap')
cat test.py
import os
command = '/usr/sbin/tcpdump -i eth1 {} -c {} -s 0 -w {}'.format( 'host 192.168.254.74','30000','res.pcap')
print(command)
os.system(command)
sudo python test.py
/usr/sbin/tcpdump -i eth1 host 192.168.1.10 -c 30000 -s 0 -w res.pcap
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 10 bytes
^C0 packets captured
6 packets received by filter
0 packets dropped by kernel
ls -l | grep test
-rw------- 1 admin admin 155 Dec 2 23:05 test.py
Seems to work just fine for me.
The test file is 'test.py'. I run it under sudo and exit after some time. I can see that 6 packets were captured and the file size is > 0.
Make sure the command itself runs properly outside of python.
I try to start the Python SimpleHTTPServer on port 7054 :
$ sudo python -m SimpleHTTPServer 7054
...
socket.error: [Errno 98] Address already in use
So, I ran the following commands :
$ sudo netstat -ntpu | grep 7054
$ sudo lsof -i -n -P | grep 7054
But I have no results.
From the netstat manpage:
netstat [address_family_options] [--tcp|-t] [--udp|-u] [--raw|-w] [--listening|-l] [--all|-a] [--numeric|-n] [--numeric-hosts] [--numeric-ports]
[--numeric-users] [--symbolic|-N] [--extend|-e[--extend|-e]] [--timers|-o] [--program|-p] [--verbose|-v] [--continuous|-c]
I use the following options:
sudo netstat -tanl | grep 7054
Which is --numeric, --tcp, --all, --listening
I think the minimal netstat options you need to show the pid of the process listening on a particular port are -nlp.
The lsof options you specify work for me. Using the example code at https://wiki.python.org/moin/UdpCommunication#Receiving and python -m SimpleHTTPServer 7054:
$ netstat -nlp | grep 7054
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 0.0.0.0:7054 0.0.0.0:* LISTEN 20458/python
udp 0 0 0.0.0.0:7054 0.0.0.0:* 20498/python
$ lsof -i -n -P | grep 7054
python 20458 michael 3u IPv4 143736 0t0 TCP *:7054 (LISTEN)
python 20498 michael 3u IPv4 173739 0t0 UDP *:7054
Extra credit: stick it in an alias:
listening() {
netstat -nlp | grep $1
}
And use it:
$ listening 7054
An address can be in use, but not shown by lsof, ss or netstat once bind has been used on a SOCK_STREAM socket, but before the named socket has been set the LISTEN state.
This was found with a test performed using AlmaLinux 8.6 with a 4.18.0-372.19.1.el8_6.x86_64 Kernel.
The source for the test program is in bind_local.c
Start the test program, specifying an IPv6 link-local address and port number (10000) to bind to:
[mr_halfword#haswell-alma ibv_message_passing]$ ibv_message_passing_c_project/bin/debug/bind_local/bind_local -6 fe80::207:43ff:fe15:2298%4 -p 10000 -l
fd 3 bound to fe80::207:43ff:fe15:2298 scope-id 4 port 10000
Press enter to listen on port
At the above port a SOCK_STREAM socket has been created, bind called and getsockname used to get the socket name which is displayed (i.e. the address the socket has been bound to).
The socket file-descriptor the test program has bound is shown as socket 398999:
[mr_halfword#haswell-alma ~]$ ls -l /proc/`pgrep bind_local`/fd
total 0
lrwx------. 1 mr_halfword mr_halfword 64 Sep 10 17:08 0 -> /dev/pts/0
lrwx------. 1 mr_halfword mr_halfword 64 Sep 10 17:08 1 -> /dev/pts/0
lrwx------. 1 mr_halfword mr_halfword 64 Sep 10 17:08 2 -> /dev/pts/0
lrwx------. 1 mr_halfword mr_halfword 64 Sep 10 17:08 3 -> 'socket:[398999]'
In this state attempting to use nc to listen on port 10000 fails with Address already in use, but neither lsof not ss show the address:
[mr_halfword#haswell-alma ~]$ nc -l 10000
Ncat: bind to :::10000: Address already in use. QUITTING.
[mr_halfword#haswell-alma ~]$ sudo lsof -i -n -P | grep 10000
[mr_halfword#haswell-alma ~]$ sudo ss -nlp | grep 10000
[mr_halfword#haswell-alma ~]$
Cause the test program to call listen on the bound socket, by pressing return:
Press return to exit
Now that the bound socket is in the LISTEN state attempting to use nc to listen on port 10000 fails with Address already in use, but now lsof and ss are showing the address and which program is using the address:
[mr_halfword#haswell-alma ~]$ sudo lsof -i -n -P | grep 10000
bind_loca 16929 mr_halfword 3u IPv6 398999 0t0 TCP [fe80::207:43ff:fe15:2298]:10000 (LISTEN)
[mr_halfword#haswell-alma ~]$ sudo ss -nlp | grep 10000
tcp LISTEN 0 1 [fe80::207:43ff:fe15:2298]%enp1s0f4d1:10000 [::]:* users:(("bind_local",pid=16929,fd=3))
I haven't yet tried looking at the Linux Kernel source code to determine if a SOCK_STREAM socket which has been bound to an address by being named, but left in that state, has any user space method which can locate the program using the address.
The reason the above was investigating how the iwpmd iWARP Port Mapper Daemon was claiming TCP ports, for which was unable to find a way to list the claimed TCP ports.
Is there any way to use the Standford Tagger in a more performant fashion?
Each call to NLTK's wrapper starts a new java instance per analyzed string which is very very slow especially when a larger foreign language model is used...
http://www.nltk.org/api/nltk.tag.html#module-nltk.tag.stanford
Found the solution. It is possible to run the POS Tagger in servlet mode and then connect to it via HTTP. Perfect.
http://nlp.stanford.edu/software/pos-tagger-faq.shtml#d
example
start server in background
nohup java -mx1000m -cp /var/stanford-postagger-full-2014-01-04/stanford-postagger.jar edu.stanford.nlp.tagger.maxent.MaxentTaggerServer -model /var/stanford-postagger-full-2014-01-04/models/german-dewac.tagger -port 2020 >& /dev/null &
adjust firewall to limit access to port 2020 from localhost only
iptables -A INPUT -p tcp -s localhost --dport 2020 -j ACCEPT
iptables -A INPUT -p tcp --dport 2020 -j DROP
test it with wget
wget http://localhost:2020/?die welt ist schön
shutdown server
pkill -f stanford
restore iptable settings
iptables -D INPUT -p tcp -s localhost --dport 2020 -j ACCEPT
iptables -D INPUT -p tcp --dport 2020 -j DROP
Using nltk.tag.stanford.POSTagger.tag_sents() for tagging multiple sentences.
The tag_sents has replaced the old batch_tag function, see https://github.com/nltk/nltk/blob/develop/nltk/tag/stanford.py#L61
DEPRECATED:
Tag the sentences using batch_tag instead of tag, see http://www.nltk.org/_modules/nltk/tag/stanford.html#StanfordTagger.batch_tag