We Keep Coding

Pexect in Windows Batch File > Cygwin > Python > SSH

I have a Linux box that runs Cisco IOS and need to SSH into it sometimes to reboot it. I've written a batch file that calls on Cygwin. Cygwin then calls on Python+PythonScript.
Batch File:
cd c:\cygwin64\bin
bash --login -i -c "python3 /home/Owner/uccxtesting.py"
Python Script
import pexpect
import time
import sys
server_ip = "10.0.81.104"
server_user = "administrator"
server_pass = "secretpassword"
sshuccx1 = pexpect.spawn('ssh %s#%s' % (server_user, server_ip))
sshuccx1.logfile_read = sys.stdout.buffer
sshuccx1.timeout = 180
sshuccx1.expect('.*password:')
sshuccx1.sendline(server_pass)
sshuccx1.expect('.admin:')
sshuccx1.sendline('utils system restart')
sshuccx1.expect('Enter (yes/no)?')
sshuccx1.sendline('yes')
time.sleep(30)
When I run this, it stops at Enter yes/no. This is what I'm getting:
I've seen plenty of examples of pexpect with expect, but there is some white space out beside the question mark. I just don't know how to tell python to expect it.

There may be a bug:
utils system restart prompts for force restart (https://bst.cisco.com/bugsearch/bug/CSCvw22828)
Replace time.sleep(30) with the following code to answer a possible force restart prompt. If it works, you can get rid of the try...except and print commands that I added for debugging:
try:
index = -1
while index != 0:
sshuccx1.expect_exact(['succeeded', 'force', ], timeout=300)
if index == 2:
print('Forcing restart...')
sshuccx1.sendline('yes')
print('Operation succeeded')
print(str(child.before))
except pexpect.ExceptionPexpect:
e_type, e_value, _ = sys.exc_info()
print('Error: ' + pexpect.ExceptionPexpect(e_type).get_trace())
print(e_type, e_value)
Also, change sshuccx1.expect('Enter (yes/no)?') to sshuccx1.expect_exact('Enter (yes/no)?'). The expect method tries to match a regex pattern, and it may get caught on the parentheses (see https://pexpect.readthedocs.io/en/stable/api/pexpect.html#pexpect.spawn.expect_exact)

Run multiple bash lines in Python and separetely check their status and output

I am trying to execute several lines of bash in Python 3 and check the status of each line separately.
I first tried to use gestatusoutput from subprocess, but each line is run in a separated process that does not communicate with the others (for the sake of simplicity, the given MWE consists of setting a variable, but what I intend to do in my actual code is more complex than that — and I know about os.environ for this very specific example):
from subprocess import getstatusoutput as cmd
stat, out = cmd("export TEST=1")
stat, out = cmd("echo $TEST")
will therefore returns:
>>> print(out)
(0, "")
I then tried the following:
cmdline = """export TEST=1
echo $TEST"""
stat, out = cmd(cmdline)
That works but forces me to parse the output, specially if I want to check the status of the first command (if echo works, the status returns by cmd is 0 whatever happens before), that is not very robust.
I saw some things using Popen (still from subprocess) but was unable to use it efficiently.
Any help would be appreciated!

To me, you are trying to share the environment variable between two process, which is not possible.
It looks like this:
Process 1 python main.py #TEST = ""
|Process 2-->"export TEST=1" #Change Process2 env variable TEST to '1'
|Process 3-->"echo $TEST" #Print Process3 env variable TEST (get from process 1)
You can use os.environ[] to change the current environment first (Process 1 variable),Later on use the variable after fork.
Something like this
import os
import subprocess
import sys
os.environ['TEST'] = '1'
out = subprocess.check_call('echo $TEST',shell = True)

I resulted doing the following:
create a launch command wrapping subprocess.Popen to launch my bash commands, that in addition allows me either to retrieve the current environment or to pass a custom environment
create a get_env to parse the return from the previous command and get a dict of the environment
launch wrapper
import os
import subprocess as sp
def launch(cmd_, env=os.environ, get_env=False):
if get_env: cmd_ += " && printenv"
load = sp.Popen(cmd_, shell=True, stdout=sp.PIPE, stderr=sp.PIPE, env=env)
out = load.communicate()
err = load.returncode
return(err, out)
Retrieve the environment
def get_env(out, encoding='utf-8'):
lout = str(out[0], encoding).split('\n')
new_env = {}
for line in lout:
if len(line.split('=')) <= 1:
pass
else:
k = line.split("=")[0]
v = "=".join(line.split("=")[1:])
new_env[k] = v
return new_env
(This is a simple version, it may be more complicated if you have things like functions in your environment — it happens.)
Results:
I can use it as follow:
err, out = launch("export TEST=1", get_env=True)
if not err: new_env = get_env(out)
err, out = launch("echo $TEST", env=new_env)
and therefore:
>>> print(str(out[0], encoding='utf-8'))
1

How to check whether a shell command returned nothing or something

I am writing a script to extract something from a specified path. I am returning those values into a variable. How can i check whether the shell command has returned something or nothing.
My Code:
def any_HE():
global config, logger, status, file_size
config = ConfigParser.RawConfigParser()
config.read('config2.cfg')
for section in sorted(config.sections(), key=str.lower):
components = dict() #start with empty dictionary for each section
#Retrieving the username and password from config for each section
if not config.has_option(section, 'server.user_name'):
continue
env.user = config.get(section, 'server.user_name')
env.password = config.get(section, 'server.password')
host = config.get(section, 'server.ip')
print "Trying to connect to {} server.....".format(section)
with settings(hide('warnings', 'running', 'stdout', 'stderr'),warn_only=True, host_string=host):
try:
files = run('ls -ltr /opt/nds')
if files!=0:
print '{}--Something'.format(section)
else:
print '{} --Nothing'.format(section)
except Exception as e:
print e
I tried checking 1 or 0 and True or false but nothing seems to be working. In some servers, the path '/opt/nds/' does not exist. So in that case, nothing will be there on files. I wanted to differentiate between something returned to files and nothing returned to files.

First, you're hiding stdout.
If you get rid of that you'll get a string with the outcome of the command on the remote host. You can then split it by os.linesep (assuming same platform), but you should also take care of other things like SSH banners and colours from the retrieved outcome.

As perror commented already, the python subprocess module offers the right tools.
https://docs.python.org/2/library/subprocess.html
For your specific problem you can use the check_output function.
The documentation gives the following example:
import subprocess
subprocess.check_output(["echo", "Hello World!"])
gives "Hello World"

plumbum is a great library for running shell commands from a python script. E.g.:
from plumbum.local import ls
from plumbum import ProcessExecutionError
cmd = ls['-ltr']['/opt/nds'] # construct the command
try:
files = cmd().splitlines() # run the command
if ...:
print ...:
except ProcessExecutionError:
# command exited with a non-zero status code
...
On top of this basic usage (and unlike the subprocess module), it also supports things like output redirection and command pipelining, and more, with easy, intuitive syntax (by overloading python operators, such as '|' for piping).

In order to get more control of the process you run, you need to use the subprocess module.
Here is an example of code:
import subprocess
task = subprocess.Popen(['ls', '-ltr', '/opt/nds'], stdout=subprocess.PIPE)
print task.communicate()

How do you get the process ID of a program in Unix or Linux using Python?

I'm writing some monitoring scripts in Python and I'm trying to find the cleanest way to get the process ID of any random running program given the name of that program
something like
ps -ef | grep MyProgram
I could parse the output of that however I thought there might be a better way in python

From the standard library:
os.getpid()

If you are not limiting yourself to the standard library, I like psutil for this.
For instance to find all "python" processes:
>>> import psutil
>>> [p.info for p in psutil.process_iter(attrs=['pid', 'name']) if 'python' in p.info['name']]
[{'name': 'python3', 'pid': 21947},
{'name': 'python', 'pid': 23835}]

Try pgrep. Its output format is much simpler and therefore easier to parse.

Also:
Python: How to get PID by process name?
Adaptation to previous posted answers.
def getpid(process_name):
import os
return [item.split()[1] for item in os.popen('tasklist').read().splitlines()[4:] if process_name in item.split()]
getpid('cmd.exe')
['6560', '3244', '9024', '4828']

With psutil:
(can be installed with [sudo] pip install psutil)
import psutil
# Get current process pid
current_process_pid = psutil.Process().pid
print(current_process_pid) # e.g 12971
# Get pids by program name
program_name = 'chrome'
process_pids = [process.pid for process in psutil.process_iter() if process.name == program_name]
print(process_pids) # e.g [1059, 2343, ..., ..., 9645]

For Windows
A Way to get all the pids of programs on your computer without downloading any modules:
import os
pids = []
a = os.popen("tasklist").readlines()
for x in a:
try:
pids.append(int(x[29:34]))
except:
pass
for each in pids:
print(each)
If you just wanted one program or all programs with the same name and you wanted to kill the process or something:
import os, sys, win32api
tasklistrl = os.popen("tasklist").readlines()
tasklistr = os.popen("tasklist").read()
print(tasklistr)
def kill(process):
process_exists_forsure = False
gotpid = False
for examine in tasklistrl:
if process == examine[0:len(process)]:
process_exists_forsure = True
if process_exists_forsure:
print("That process exists.")
else:
print("That process does not exist.")
raw_input()
sys.exit()
for getpid in tasklistrl:
if process == getpid[0:len(process)]:
pid = int(getpid[29:34])
gotpid = True
try:
handle = win32api.OpenProcess(1, False, pid)
win32api.TerminateProcess(handle, 0)
win32api.CloseHandle(handle)
print("Successfully killed process %s on pid %d." % (getpid[0:len(prompt)], pid))
except win32api.error as err:
print(err)
raw_input()
sys.exit()
if not gotpid:
print("Could not get process pid.")
raw_input()
sys.exit()
raw_input()
sys.exit()
prompt = raw_input("Which process would you like to kill? ")
kill(prompt)
That was just a paste of my process kill program I could make it a whole lot better but it is okay.

For posix (Linux, BSD, etc... only need /proc directory to be mounted) it's easier to work with os files in /proc
Works on python 2 and 3 ( The only difference is the Exception tree, therefore the "except Exception", which i dislike but kept to maintain compatibility. Also could've created custom exception.)
#!/usr/bin/env python
import os
import sys
for dirname in os.listdir('/proc'):
if dirname == 'curproc':
continue
try:
with open('/proc/{}/cmdline'.format(dirname), mode='rb') as fd:
content = fd.read().decode().split('\x00')
except Exception:
continue
for i in sys.argv[1:]:
if i in content[0]:
# dirname is also the number of PID
print('{0:<12} : {1}'.format(dirname, ' '.join(content)))
Sample Output (it works like pgrep):
phoemur ~/python $ ./pgrep.py bash
1487 : -bash
1779 : /bin/bash

This is a simplified variation of Fernando's answer. This is for Linux and either Python 2 or 3. No external library is needed, and no external process is run.
import glob
def get_command_pid(command):
for path in glob.glob('/proc/*/comm'):
if open(path).read().rstrip() == command:
return path.split('/')[2]
Only the first matching process found will be returned, which works well for some purposes. To get the PIDs of multiple matching processes, you could just replace the return with yield, and then get a list with pids = list(get_command_pid(command)).
Alternatively, as a single expression:
For one process:
next(path.split('/')[2] for path in glob.glob('/proc/*/comm') if open(path).read().rstrip() == command)
For multiple processes:
[path.split('/')[2] for path in glob.glob('/proc/*/comm') if open(path).read().rstrip() == command]

The task can be solved using the following piece of code, [0:28] being interval where the name is being held, while [29:34] contains the actual pid.
import os
program_pid = 0
program_name = "notepad.exe"
task_manager_lines = os.popen("tasklist").readlines()
for line in task_manager_lines:
try:
if str(line[0:28]) == program_name + (28 - len(program_name) * ' ': #so it includes the whitespaces
program_pid = int(line[29:34])
break
except:
pass
print(program_pid)

Python script running in linux

I am having trouble trying to get this script to work. When I debug this code it will not read into the class or functions. The code will not execute properly. Has anyone know the problem here, Thanks
#!/home/build/test/Python-2.6.4
import os, subprocess
class mks_function:
sandbox="new_sandbox"
def mks_create_sandbox():
try:
retcode=call("si createsandbox" + "--no --hostname=bel --port=70 --user=user --password=1234 --populate --project=e:/project.pj --lineTerminator=lf new_sandbox", shell=True)
if retcode < 0:
print >>sys.stderr, "Child was terminated by signal", -retcode
else:
print >>sys.stderr, "Child returned", retcode
except OSError, e:
print >>sys.stderr, "Execution failed:", e
print "sandbox retVal="+retcode
print "Creating a new sandbox called "+sandbox+" "
###############################################################

Few things to check your code
call should be subprocess.call
better use full path when you call for example, /usr/bin/si createsandbox, you can check with which si in shell
instead of concatenating the commands "si createsandbox" + "--no ...", please use list ["/usr/bin/si","createsandbox --no ..."]
you didn't import sys, but using it
sandbox should be self.sandbox and def mks_create_sandbox(): should be def mks_create_sandbox(self):
Use an IDE for example Ulipad.

Try put as the first line:
#!/usr/bin/env python
If you really need specific version of Python, setup your environment before running.
Possible problems:
your code is never executed (it's like you define the class only). Use it in the file (names are misleading):
if __name__ == '__main__':
myObject = mks_function()
show us how are you executing the code? Have you changed the permissions to be able to run the script?
chmod +x filename.py
or are you trying to start it as:
python filename.py