We Keep Coding

ping constantly an IP and write into txt

I have this code
def ping_google(command):
with open('google.txt', 'a') as f:
f.write(subprocess.check_output(command))
t1 = threading.Thread(target=ping_anel, args=("ping -t 8.8.8.8",))
And i would like to save the infinite pinging to google in a txt file. Is it possible?

command = "ping -i20 8.8.8.8 > ping.txt"
Ask subprocess to execute that with , shell=True,
and you're done.
It will append three status reports each minute, forever,
or until you hit CTRL/C.
(I expect ping
to require a time-to-live value after a -t switch,
so I'm not sure what the intent of your command was.)

Pyserial send and receive multiple commands

I'm working on a script to send a list of commands to a device and return the output.
When the device first boots up, it has a few prompts. I am able to get through the prompts.
However, after completing the prompts, when I try to send a command the command isn't sent.
Commands
The commands.txt is set up like this:
200,
2,no
2,
The first line (200) is to let the device boot up.
The 2nd and 3rd lines answer 2 different prompts.
Issues
The issues come after these 3 inputs. The code runs and completes. Python prints out each of the commands. So the list is processed by Python. However, I don't think the device is receiving the commands.
In the log, the \n and no are written out, but none of the commands after it are. The commands do show when I use ser.inWaiting()
When I access the device through putty and run the commands through the console, everything works as expected.
Why aren't the commands going through?
Small update:
I read somewhere that python may be sending the commands to quickly, so I tried sending the commands 1 char at a time with a .01 delay.
It still didn't work:
for i in lines[1]:
cmd = i
encoded_cmd = cmd.encode("utf-8")
ser.write(encoded_cmd)
sleep(0.1)
print(cmd)
Code
import serial
import time
from time import sleep
from datetime import datetime
# create list of commands
with open('commands.txt') as commands:
list_of_commands = [tuple(map(str, i.split(','))) for i in commands]
# open and name log file
date = datetime.now().strftime("%Y-%m-%d")
log = open(f'{date}.txt', 'w+')
# serial configuration
info = open('info.txt', 'r')
lines = info.readlines()
port = lines[0].strip('\n')
baud = int(lines[1].strip('\n'))
try:
# open port
ser = serial.Serial(port=port, baudrate=baud, timeout=5, parity=serial.PARITY_NONE, stopbits=serial.STOPBITS_ONE, write_timeout=0)
except ConnectionError:
log.write(''.join('There was a connection error'))
else:
# run commands
x = 0
for lines in list_of_commands:
ser.close()
ser.open()
sleep(2)
cmd = lines[1]
encoded_cmd = cmd.encode("utf-8")
sleep_time = int(lines[0])
ser.write(encoded_cmd)
time.perf_counter()
# log output
while 1:
test = ser.readline()
text = test.decode('utf-8')
print(text)
log.write(''.join(text))
print(time.perf_counter())
print(time.perf_counter() - x)
if time.perf_counter() - x > sleep_time:
x = time.perf_counter()
ser.flushInput()
ser.flushOutput()
break
print(cmd)
# close port
ser.close()
# close files
log.close()

From the question it's obvious that multiple issues are intermingled. The same observation comes when reading the code. So I tried to list some of those I struggled with.
Issues
Try-except-else
What is the intention behind try .. except .. else ?
Not sure, its used correctly on purpose here. See try-except-else explained:
The else clause is executed if and only if no exception is raised. This is different from the finally clause that’s always executed.
The serial connection
Why opening and closing inside the loop:
ser.close()
ser.open()
Why the misleading comment:
# close server
ser.close()
Usage of sleep_time
What is the purpose of using the first column sleep_time of your CSV commands.txt inside a conditional break inside you read-loop?
sleep_time = int(lines[0])
Instead the sleep is fix 2 seconds before sending the command:
sleep(2)
How to debug
I would recommend adding some print (or log) statements to
verify the list_of_commands has been read correctly
verify which commands (cmd or even encoded_cmd) have been sent to the serial output

Run multiple commands in a single ssh session using popen and save the output in separate files

I am trying to run several commands in a single ssh session and save the output of each command in a different file.
The code that works (but it saves all the output in a single file)
conn = Popen(['ssh',host, "ls;uname -a;pwd"], stdin=PIPE, stdout = open ('/output.txt','w'))
mypassword = conn.communicate('password')
Codes that I am trying to work but not working...
cmd = ['ls', 'pwd', 'uname']
conn = Popen(['ssh',host, "{};{};{}".format(cmd[0],cmd[1],cmd[2])], stdin=PIPE, stdout = output.append('a'))
mypassword = conn.communicate('password')
print (output)
length = range(len(output))
print length
for i in output:
open("$i",'w')
and
cmd = ['ls', 'pwd', 'uname']
conn = Popen(['ssh',host, "{};{};{}".format(cmd[0],cmd[1],cmd[2])], stdin=PIPE, stdout = output())
mypassword = conn.communicate('password')
def output():
for i in cmd:
open(i,'w')
return
Not sure what is the best way of doing it. Should I save it in an array and then save each item in a separate file or should I call a function that will do it?
NOTE that the commands I want to run do not have small output like given in examples here (uname, pwd); it is big as tcpdump, lsof etc.

A single ssh session runs a single command e.g., /bin/bash on the remote host -- you can pass input to that command to emulate running multiple commands in a single ssh session.
Your code won't run even a single command. There are multiple issues in it:
ssh may read the password directly from the terminal (not its stdin stream). conn.communicate('password') in your code writes to ssh's stdin therefore ssh won't get the password.
There are multiple ways to authenticate via ssh e.g., use ssh keys (passwordless login).
stdout = output.append('a') doesn't redirect ssh's stdout because .append list method returns None.
It won't help you to save output of several commands to different files. You could redirect the output to remote files and copy them back later: ls >ls.out; uname -a >uname.out; pwd >pwd.out.
A (hacky) alternative is to use inside stream markers (echo <GUID>) to differentiate the output from different commands. If the output can be unlimited; learn how to read subprocess' output incrementally (without calling .communicate() method).
for i in cmd: open(i,'w') is pointless. It opens (and immediately closes on CPython) multiple files without using them.
To avoid such basic mistakes, write several Python scripts that operate on local files.

SYS_STATS={"Number of CPU Cores":"cat /proc/cpuinfo|grep -c 'processor'\n",
"CPU MHz":"cat /proc/cpuinfo|grep 'cpu MHz'|head -1|awk -F':' '{print $2}'\n",
"Memory Total":"cat /proc/meminfo|grep 'MemTotal'|awk -F':' '{print $2}'|sed 's/ //g'|grep -o '[0-9]*'\n",
"Swap Total":"cat /proc/meminfo|grep 'SwapTotal'|awk -F':' '{print $2}'|sed 's/ //g'|grep -o '[0-9]*'\n"}
def get_system_details(self,ipaddress,user,key):
_OutPut={}
values=[]
sshProcess = subprocess.Popen(['ssh','-T','-o StrictHostKeyChecking=no','-i','%s' % key,'%s#%s'%(user,ipaddress),"sudo","su"],
stdin=subprocess.PIPE, stdout = subprocess.PIPE, universal_newlines=True,bufsize=0)
for element in self.SYS_STATS1:
sshProcess.stdin.write("echo END\n")
sshProcess.stdin.write(element)
sshProcess.stdin.close()
for element in sshProcess.stdout:
if element.rstrip('\n')!="END":
values.append(element.rstrip('\n'))
mapObj={k: v for k, v in zip(self.SYS_STATS_KEYS, values)}
return mapObj

How to get the actual shell prompt string in Python?

I have a Python routine which invokes some kind of CLI (e.g telnet) and then executes commands in it. The problem is that sometimes the CLI refuses connection and commands are executed in the host shell resulting in various errors. My idea is to check whether the shell prompt alters or not after invoking the CLI.
The question is: how can I get the shell prompt string in Python?
Echoing PS1 is not a solution, because some CLIs cannot run it and it returns a notation-like string instead of the actual prompt:
SC-2-1:~ # echo $PS1
\[\]\h:\w # \[\]
EDIT
My routine:
def run_cli_command(self, ssh, cli, commands, timeout = 10):
''' Sends one or more commands to some cli and returns answer. '''
try:
channel = ssh.invoke_shell()
channel.settimeout(timeout)
channel.send('%s\n' % (cli))
if 'telnet' in cli:
time.sleep(1)
time.sleep(1)
# I need to check the prompt here
w = 0
while (channel.recv_ready() == False) and (w < timeout):
w += 1
time.sleep(1)
channel.recv(9999)
if type(commands) is not list:
commands = [commands]
ret = ''
for command in commands:
channel.send("%s\r\n" % (command))
w = 0
while (channel.recv_ready() == False) and (w < timeout):
w += 1
time.sleep(1)
ret += channel.recv(9999) ### The size of read buffer can be a bottleneck...
except Exception, e:
#print str(e) ### for debugging
return None
channel.close()
return ret
Some explanation needs here: the ssh parameter is a paramiko.SSHClient() instance. I use this code to login to a server and from there I call another CLI which can be SSH, telnet, etc.

I’d suggest sending commands that alter PS1 to a known string. I’ve done so when I used Oracle sqlplus from a Korn shell script, as coprocess, to know when to end reading data / output from the last statement I issued. So basically, you’d send:
PS1='end1>'; command1
Then you’d read lines until you see "end1>" (for extra easiness, add a newline at the end of PS1).

Unix `at` scheduling with python script: Permission denied

I'm trying to create a scheduled task using the Unix at command. I wanted to run a python script, but quickly realized that at is configured to use run whatever file I give it with sh. In an attempt to circumvent this, I created a file that contained the command python mypythonscript.py and passed that to at instead.
I have set the permissions on the python file to executable by everyone (chmod a+x), but when the at job runs, I am told python: can't open file 'mypythonscript.py': [Errno 13] Permission denied.
If I run source myshwrapperscript.sh, the shell script invokes the python script fine. Is there some obvious reason why I'm having permissions problems with at?
Edit: I got frustrated with the python script, so I went ahead and made a sh script version of the thing I wanted to run. I am now finding that the sh script returns to me saying rm: cannot remove <filename>: Permission denied (this was a temporary file I was creating to store intermediate data). Is there anyway I can authorize these operations with my own credentials, despite not having sudo access? All of this works perfectly when I run it myself, but everything seems to go to shit when I have at do it.

Start the script using python not the actual script name, ex : python path/to/script.py.
at tries to run everything as a sh script.

EDIT: The at command tries running everything as a list of shell commands. So you should start your script like this:
at now + 1 minute < python mypythonscript.py
In this case, the #! line at the beginning of the script is not necessary.

I have been working on task scheduling between servers and clients recently. I just abstracted out my scheduling code and put it up on Github. It was meant to schedule several simulations across multiple machines that have all simulations in their filesystems. The idea is that since each machine had a different processor, it would compute each simulation, scp the results back into the server and request the server for the next simulation. The server responds by scheduling a task on the client to run the next unrun simulation
Hope this will help you.
NOTE: Since I only abstracted and uploaded the files about 5 minutes ago, I haven't had the chance to test the abstractions. However, if you come across any bugs, please let me know and I'll debug then as soon as I can.
Github seems to be down now. So here are the files that you'll need:
On the server:
serverside
#!/bin/bash
projectDir=~/
minute=`atq | sort -t" " -k1 -nr | head -n1 | cut -d' ' -f4 | cut -d":" -f1,2`
curr=`date | cut -d' ' -f4 | cut -d':' -f1,2`
time=`python -c "import sys; hour,minute=map(int,max(sys.argv[1:]).split(':')); minute += 2; hour, minute = [(hour,minute), ((hour+1)%24,minute%60)][minute>=60]; print '%d:%02d'%(hour, minute)" "$minute" "$curr"`
cat <<EOF | at "$time"
python $projectDir/serverside.py $1
EOF
serverside.py
import sys
import time
import smtplib
import subprocess
import os
import itertools
IP = sys.argv[1].strip()
PROJECT_DIR = "" # relative path (relative to the home directory) to the root directory of the project, which contains all subdirs containing simulation files
USERS = { # keys are IPs of the clients, values are user names on those clients
}
HOMES = { # keys are the IPs of clients, values are the absolute paths to the home directories on these clients for the usernames on these clients identified in USERS
}
HOME = None # absolute path to the home directory on the server
SMTP_SERVER = ""
SMTP_PORT = None
FROM_ADDR = None # the email address from which notification emails will be sent
TO_ADDR = None # the email address to which notification emails will be sent
def get_next_simulation():
""" This function returns a list.
The list contains N>0 elements.
Each of the first N-1 elements are names of directories (not paths), which when joined together form a relative path (relative from PROJECT_DIR).
The Nth element is the name of the file - the simulation to be run.
Before the end user implements this function, it is assumed that N=3.
Once this function has been implemented, if N!=3, change the code in the lines annotated with "Change code for N in this line"
Also look for this annotation in clientside.py and clientsideexec """
pass
done = False
DIR1, DIR2, FILENAME = get_next_simulation() # Change code for N in this line
while not done:
try:
subprocess.check_call("""ssh %(user)s#%(host)s 'sh %(home)s/%(project)/clientside %(dir1)s %(dir2)s %(filename)s %(host)s' """ %{'user':USER, 'host':IP, 'home':HOME[IP], 'project':PRJECT_DIR, 'dir1':DIR1, 'dir2':DIR2, 'filename':FILENAME}, shell=True) # Change code for N in this line
done = True
os.remove("%(home)s/%(project)/%(dir1)s/%(dir2)s/%(filename)s" %{'home':HOME, 'project':PROJECT_DIR, 'dir1':DIR1, 'dir2':DIR2, 'filename':FILENAME}) # Change code for N in this line
sm = smtplib.SMTP(SMTP_SERVER, SMTP_PORT)
sm.sendmail(FROM_ADDR, TO_ADDR, "running %(project)s/%(dir1)s/%(dir2)s/%(filename)s on %(host)s" %{'project':PROJECT_DIR, 'dir1':DIR1, 'dir2':DIR2, 'filename':FILENAME, 'host':IP}) # Change code for N in this line
except:
pass
On the client:
clientside
#!/bin/bash
projectpath=~/
python $projectpath/clientside.py "$#"
clientside.py
import subprocess
import sys
import datetime
import os
DIR1, DIR2, FILENAME, IP = sys.argv[1:]
try:
subprocess.check_call("sh ~/cisdagp/clientsideexec %(dir1)s %(dir2)s %(filename)s %(ip)s" %{'dir1':, 'dir2':, 'filename':, ip':IP}, shell=True, executable='/bin/bash') # Change code for N in this line
except:
pass
clientsideexec
#!/bin/bash
projectpath=~/
user=''
serverIP=''
SMTP_SERVER=''
SMTP_PORT=''
FROM_ADDR=''
TO_ADDR=''
MESSAGE=''
cat <<EOF | at now + 2 minutes
cd $projectpath/$1/$2 # Change code for N in this line
sh $3
# copy the logfile back to the server
scp logfile$3 $user#$serverIP:$projectpath/$1/$2/
cd $projectpath
python -c "import smtplib; sm = smtplib.SMTP('$SMTP_SERVER', $SMTP_PORT); sm.sendmail('$FROM_ADDR', '$TO_ADDR', '$MESSAGE')"
python clientsiderequest.py
EOF

Could you try: echo 'python mypythonscript.py' | at ...