I am using Paramiko invoke_shell to pull results of the top command from a remote system. But I am getting truncated lines when looking at the results.
Code as follows :
channel = token.invoke_shell()
channel.send ('terminal length 0\n')
time.sleep(1)
resp = channel.recv(9999)
output = resp.decode('ascii')
channel.send('top -n 1\n')
time.sleep(1)
resp = channel.recv(9999)
output = resp.decode('ascii')
result = (''.join(output))
return (result)
The result is as follows (note cn_node+ is not the complete name, it is longer):
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
28639 root 20 0 17.0g 38912 28336 S 111.1 0.1 1101:49 cn-node+
29889 root 20 0 3379668 16532 13428 S 94.4 0.1 991:39.71 Flare
If directly ssh'ed into the system and running the command, the result is :
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
28639 root 20 0 17.0g 38912 28336 S 116.7 0.1 1186:17 cn-node-cnfp
29889 root 20 0 3379668 16796 13428 S 94.4 0.1 1067:53 Flare
Wondering how to get the whole line and not the truncated line (get cn-node-cnfp instead of cn-node+).
Thanks!
Set width parameter of SSHClient.invoke_shell to more than the default value of 80:
channel = token.invoke_shell(width=160)
Though better is probably to use COLUMN environment variable.
See Linux: How to not limit output from top based on screen width
In any case, you in general should not use SSHClient.invoke_shell for automating command execution. Use `SSHClient.exec_command. See What is the difference between exec_command and send with invoke_shell() on Paramiko?
Related
I use sendevent to simulate complex gestures (clicks, swipes, long clicks with a swipe at the end). the problem is that sendevent is quite slow, 0.1 second per sendevent command in cmd, and click needs 7 sendevent commands. I so understood that sendevent opens the file, writes, closes and so 7 times. how to optimize it?
i know input tap, but input slower than sendevent
my code on python:
import time
from ppadb.client import Client
adb = Client(host='127.0.0.1', port=5037)
devices = adb.devices()
if len(devices) == 0:
print('Devices not found')
quit()
device = devices[0]
def sendevent(_type, _code, _value, _deviceName='/dev/input/event4'):
last_time = time.time()
device.shell(f"su -c 'sendevent {_deviceName} {_type} {_code} {_value}'")
print(time.time()-last_time)
def TapScreen(x, y):
sendevent(EV_ABS, ABS_MT_ID, 0)
sendevent(EV_ABS, ABS_MT_TRACKING_ID, 0)
sendevent(1, 330, 1)
sendevent(1, 325, 1)
sendevent(EV_ABS, ABS_MT_PRESSURE, 5)
sendevent(EV_ABS, ABS_MT_POSITION_X, x)
sendevent(EV_ABS, ABS_MT_POSITION_Y, y)
sendevent(EV_SYN, SYN_REPORT, 0)
sendevent(EV_ABS, ABS_MT_TRACKING_ID, -1)
sendevent(1, 330, 0)
sendevent(1, 325, 0)
sendevent(EV_SYN, SYN_REPORT, 0)
EV_ABS = 3
EV_SYN = 0
SYN_REPORT = 0
ABS_MT_ID = 47
ABS_MT_TOUCH_MAJOR = 48
ABS_MT_POSITION_X = 53
ABS_MT_POSITION_Y = 54
ABS_MT_TRACKING_ID = 57
ABS_MT_PRESSURE = 58
TapScreen(1000, 500)
Is it possible to write all actions into one file and then send them together as one sendevent?
pardon my english, i use google translate :)
You’re calling adb shell sendevent … for each one of the sendevent commands. If you send all of with just one call to adb shell (Add all send events commands to the same string, separated by ;) then it would run faster (but it is still a bit slow).
E.g.: adb shell “sendevent1 ; sendevent2 ; sendevent3 ; …”
I am working on my own project. In which these steps have to be performed:
Connect to remote server.
Get pid, process name, cpu usage, swap memory usage by each running process on remote server daily on some specific time(say at 4'0 clock).
I have to compare every day's result with previous day's result (e.g. day1-pid with day2 pid and day1 process name with day2 process name etc.)
So far I have done up to step-2. Now I want to know that how to extract the pid, process name, cpu usage, swap memory usage from remote server and store it in some iterable variable. So that I can compare it for checking memory spike?
Any other way apart from my idea will be appreciable.
My code sample is like this:
import paramiko
import re
import psutil
class ShellHandler:
def __init__(self, host, user, psw):
self.ssh = paramiko.SSHClient()
self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
self.ssh.connect(host, username=user, password=psw, port=22)
channel = self.ssh.invoke_shell()
self.stdin = channel.makefile('wb')
self.stdout = channel.makefile('r')
def __del__(self):
self.ssh.close()
#staticmethod
def _print_exec_out(cmd, out_buf, err_buf, exit_status):
print('command executed: {}'.format(cmd))
print('STDOUT:')
for line in out_buf:
print(line, end="")
print('end of STDOUT')
print('STDERR:')
for line in err_buf:
print(line, end="")
print('end of STDERR')
print('finished with exit status: {}'.format(exit_status))
print('------------------------------------')
#print(psutil.pids())
pass
def execute(self, cmd):
"""
:param cmd: the command to be executed on the remote computer
:examples: execute('ls')
execute('finger')
execute('cd folder_name')
"""
cmd = cmd.strip('\n')
self.stdin.write(cmd + '\n')
finish = 'end of stdOUT buffer. finished with exit status'
echo_cmd = 'echo {} $?'.format(finish)
self.stdin.write(echo_cmd + '\n')
shin = self.stdin
self.stdin.flush()
shout = []
sherr = []
exit_status = 0
for line in self.stdout:
if str(line).startswith(cmd) or str(line).startswith(echo_cmd):
# up for now filled with shell junk from stdin
shout = []
elif str(line).startswith(finish):
# our finish command ends with the exit status
exit_status = int(str(line).rsplit(maxsplit=1)[1])
if exit_status:
# stderr is combined with stdout.
# thus, swap sherr with shout in a case of failure.
sherr = shout
shout = []
break
else:
# get rid of 'coloring and formatting' special characters
shout.append(re.compile(r'(\x9B|\x1B\[)[0-?]*[ -/]*[#-~]').sub('', line).replace('\b', '').replace('\r', ''))
# first and last lines of shout/sherr contain a prompt
if shout and echo_cmd in shout[-1]:
shout.pop()
if shout and cmd in shout[0]:
shout.pop(0)
if sherr and echo_cmd in sherr[-1]:
sherr.pop()
if sherr and cmd in sherr[0]:
sherr.pop(0)
self._print_exec_out(cmd=cmd, out_buf=shout, err_buf=sherr, exit_status=exit_status)
return shin, shout, sherr
obj=ShellHandler('Servername','username','password')
pID=[]
## I want this(pid, cmd, swap memory) to store in a varible which would be iterable.
pID=ShellHandler.execute(obj,"ps -eo pid,cmd,lstart,%mem,%cpu|awk '{print $1}'")
print(pID[0])##---------------------------------Problem not giving any output.
Your ShellHandler's execute method returns three items, the first of which is the input you sent to it.
You should probably call it directly like this, anyway:
obj = ShellHandler('Servername','username','password')
in, out, err = obj.execute("ps -eo pid,lstart,%mem,%cpu,cmd")
for line in out.split('\n'):
pid, lstartwd, lstartmo, lstartdd, lstartm, lstartyy, mem, cpu, cmd = line.split(None, 8)
I moved cmd last because it might contain spaces. The lstart value also contains multiple space-separated fields. Here's what the output looks like in Debian:
19626 Tue Jan 15 15:03:57 2019 0.0 0.0 less filename
There are many questions about how to parse ps output in more detail; I'll refer you to them for figuring out how to handle the results from split exactly.
Splitting out the output of ps using Python
Is there any way to get ps output programmatically?
ps aux command should have all the info you need (pid, process name, cpu, memory)
I'm working on python script. My script want to do the same things like ps aux | grep qemu, but for some reason, I have to use python. I want to get this string and get some configure in this string:
root 46358 0.7 0.1 3398200 137232 ? Sl Aug31 8:10
/usr/local/bin/qemu-system-x86_64_2.6.0 -enable-kvm -cpu
qemu64,+vmx,+ssse3,+sse4.1,+sse4.2,+x2apic,+aes,+avx,+vme,+pat,+ss,+pclmulqdq,+xsave,level=13 -machine pc,accel=kvm -chardev socket,id=hmqmondev,port=55902,host=127.0.0.1,nodelay,server,nowait
-mon chardev=hmqmondev,id=hmqmon,mode=readline -rtc base=utc,clock=host,driftfix=none -usb -device usb-tablet -daemonize
-nodefaults -nodefconfig -no-kvm-pit-reinjection -global kvm-pit.lost_tick_policy=discard -vga std -k en-us -smp 8 -name
gangyewei-3 -m 2048 -boot order=cdn -vnc :2,password -drive
file=/opt/cloud/workspace/disks/d955706a-a091-4c80-8d0c-3b9267953ff2,if=none,id=drive_0,format=qcow2,cache=none,aio=native
-device virtio-blk-pci,id=dev_drive_0,drive=drive_0,bus=pci.0,addr=0x5 -drive file=/opt/cloud/workspace/disks/c0b5bb2b-38e6-4c54-8953-fa90962abae9,if=none,id=drive_1,format=qcow2,cache=none,aio=native
-device virtio-blk-pci,id=dev_drive_1,drive=drive_1,bus=pci.0,addr=0x6 -drive file=/opt/cloud/workspace/disks/d60c0a8b-14ce-4adf-92cb-e6a136228825,if=none,id=drive_2,format=qcow2,cache=none,aio=native
-device virtio-blk-pci,id=dev_drive_2,drive=drive_2,bus=pci.0,addr=0x7 -device ide-cd,drive=ide0-cd0,bus=ide.1,unit=1 -drive id=ide0-cd0,media=cdrom,if=none -chardev
socket,id=char-n-2a6106e0,path=/usr/local/var/run/openvswitch/n-2a6106e0,server -netdev type=vhost-user,id=n-2a6106e0,chardev=char-n-2a6106e0,vhostforce=on
-device virtio-net-pci,netdev=n-2a6106e0,mac=00:22:2a:61:06:e0,id=netdev-n-2a6106e0,addr=0xf
-object memory-backend-file,id=mem,size=2048M,mem-path=/mnt/huge,share=on
-numa node,memdev=mem -mem-prealloc -pidfile /opt/cloud/workspace/servers/1925a92d-f003-4d77-871c-bce8f85229aa/pid
-chardev socket,path=/opt/cloud/workspace/servers/1925a92d-f003-4d77-871c-bce8f85229aa/qga.sock,server,nowait,id=qga0
-device virtio-serial -device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0
My python code like this:
def process_system_cmd(command):
response = {}
# use subprocess to process cmd
child = subprocess.Popen(command, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
ret, err = child.communicate()
if err:
response["err"] = -1
response["ret"] = err.partition(":")[2]
else:
response["err"] = 0
response["ret"] = ret
return response
# vm_socks = {"qga_socks": ["qga1.sock", ...],
# "monitor_socks":["127.0.0.1:55632", ...]}
def process_vm_socks():
response = process_system_cmd("ps aux".split())
if (response["err"]):
return
vm_socks = {"qga_socks":[], "monitor_socks":[]}
res_list = response["ret"].splitlines()
for res in res_list:
patt = '.*/usr/local/bin/qemu-system-x86_64.*'
if (res == None or res == '' or
re.match(patt, res) == None):
continue
# parse qga socket
socks = re.split(r'.*socket,path=(.*),server,nowait,id=.*', res)
if (len(socks) == 3 and is_qga_socket(socks[1])):
vm_socks["qga_socks"].append(socks[1])
But when I debug, I found in the codes below '# parse qga socket', the value in 'res' is:
root 46358 0.7 0.1 3398200 137232 ? Sl Aug31 8:10
/usr/local/bin/qemu-system-x86_64_2.6.0 -enable-kv
This is not whole string as my expect. For shell, I use ps aux | grep qemu, I will get whole string I expect as above said. I guess python add '\n' at the end of line, but I don't know why and how to fix this bug?
So what should I do? Thank you~
import subprocess
subprocess.check_output('ps aux | grep qemu', shell=True)
this will give you an output\n after each process listed and you can replace like below,
a = str(a)
a.replace('\\n','')
a.replace('\n','')
I'm creating a simple program in python that should save my current processes (using linux and pycharm).
my class code:
class pidSaver:
__pidDictionary={}
def __init__(self):
pids = [pid for pid in os.listdir('/proc') if pid.isdigit()]
for pid in pids:
try:
os.kill(int(pid), 0)
except OSError as e:
if e.errno != errno.EPERM: #no premission error
continue
try:
self.__pidDictionary[pid]=open(os.path.join('/proc', pid, 'cmdline'), 'rb').read()
except IOError: # proc has already terminated
continue
def getDic(self):
return self.__pidDictionary
and my main code:
pidsTry = pidSaver()
printList= pidsTry.getDic()
keyList= list(printList.keys())
IntegerKeyList=[]
for key in keyList:
IntegerKeyList.append(int(key))
IntegerKeyList.sort()
for key in IntegerKeyList:
print "%d : %s" %(key ,printList[str(key)])
the output:
1 : /sbin/init
2 :
3 :
5 :
...
7543 : less
...
so from some reason for some of the process I can't get a names and I got a blank out put.
when I run on my computer the command ps -aux | less I got this result:
root 1 0.0 0.0 33776 4256 ? Ss אפר24 0:01 /sbin/init
root 2 0.0 0.0 0 0 ? S אפר24 0:00 [kthreadd]
root 3 0.0 0.0 0 0 ? S אפר24 0:00 [ksoftirqd/0]
myUser 7543 0.0 0.0 13752 1548 pts/9 T אפר24 0:00 less
so basically, the process that I cannot see in my python are the process that surrounded by "[]".
I don't understand why is this. Also, I want to get them too. how can I do it and why this is happening?
thank you!
These processes you can't see are kernel threads. As the name says they are running in kernel space and are therefore no childs of PID 1, i.e. the init system. Their cmdline is empty because they don't have any corresponding executable that gets called and no arguments to be passed, and this empty cmdline is a pretty safe way to identify them. If you still want to get their name it's in the file /proc/"pid"/status under the name field.
In order to work around a bug in Apple's lldb (rdar://13702081) I very frequently need to type two commands in sequence, like this:
(lldb) p todo.matA
(vMAT_Array *) $2 = 0x000000010400b5a0
(lldb) po $2.dump
$3 = 0x0000000100503ce0 <vMAT_Int8Array: 0x10400b5a0; size: [9 1]> =
1
1
1
1
1
1
1
1
1
Is it possible to write a new lldb command using the Python library (or something) that could combine those steps for me? Ideally to something like:
(lldb) pmat todo
$3 = 0x0000000100503ce0 <vMAT_Int8Array: 0x10400b5a0; size: [9 1]> =
1
1
1
1
1
1
1
1
1
Solution
Thanks to Jason Molenda here is output from a working lldb command script:
(lldb) pmat Z
$0 = 0x0000000100112920 <vMAT_DoubleArray: 0x101880c20; size: [9 3]> =
7 9 0.848715
3 5 0.993378
0 1 1.11738
4 12 1.2013
11 13 1.20193
6 10 1.29206
14 15 1.53283
8 16 1.53602
2 17 1.68116
I did have to tweak the script provided in the answer below very slightly, using Jason's suggestions for working around the lldb bug with overly-complex expressions. Here is my final script:
# import this into lldb with a command like
# command script import pmat.py
import lldb
import shlex
import optparse
def pmat(debugger, command, result, dict):
# Use the Shell Lexer to properly parse up command options just like a
# shell would
command_args = shlex.split(command)
parser = create_pmat_options()
try:
(options, args) = parser.parse_args(command_args)
except:
return
target = debugger.GetSelectedTarget()
if target:
process = target.GetProcess()
if process:
frame = process.GetSelectedThread().GetSelectedFrame()
if frame:
var = frame.FindVariable(args[0])
if var:
array = var.GetChildMemberWithName("matA")
if array:
id = array.GetValueAsUnsigned (lldb.LLDB_INVALID_ADDRESS)
if id != lldb.LLDB_INVALID_ADDRESS:
debugger.HandleCommand ('po [0x%x dump]' % id)
def create_pmat_options():
usage = "usage: %prog"
description='''Print a dump of a vMAT_Array instance.'''
parser = optparse.OptionParser(description=description, prog='pmat',usage=usage)
return parser
#
# code that runs when this script is imported into LLDB
#
def __lldb_init_module (debugger, dict):
# This initializer is being run from LLDB in the embedded command interpreter
# Make the options so we can generate the help text for the new LLDB
# command line command prior to registering it with LLDB below
# add pmat
parser = create_pmat_options()
pmat.__doc__ = parser.format_help()
# Add any commands contained in this module to LLDB
debugger.HandleCommand('command script add -f %s.pmat pmat' % __name__)
You can do this either with a regex command or by creating your own python command and loading it in to lldb. In this specific instance the regex command won't help you because you'll hit the same crasher you're hitting. But just for fun, I'll show both solutions.
First, python. This python code gets the currently selected frame on the currently selected thread. It looks for a variable whose name is provided on the command argument. It finds a child of that variable called matA and it runs GetObjectDescription() on that SBValue object.
# import this into lldb with a command like
# command script import pmat.py
import lldb
import shlex
import optparse
def pmat(debugger, command, result, dict):
# Use the Shell Lexer to properly parse up command options just like a
# shell would
command_args = shlex.split(command)
parser = create_pmat_options()
try:
(options, args) = parser.parse_args(command_args)
except:
return
target = debugger.GetSelectedTarget()
if target:
process = target.GetProcess()
if process:
frame = process.GetSelectedThread().GetSelectedFrame()
if frame:
var = frame.FindVariable(args[0])
if var:
child = var.GetChildMemberWithName("matA")
if child:
print child.GetObjectDescription()
def create_pmat_options():
usage = "usage: %prog"
description='''Call po on the child called "matA"'''
parser = optparse.OptionParser(description=description, prog='pmat',usage=usage)
return parser
#
# code that runs when this script is imported into LLDB
#
def __lldb_init_module (debugger, dict):
# This initializer is being run from LLDB in the embedded command interpreter
# Make the options so we can generate the help text for the new LLDB
# command line command prior to registering it with LLDB below
# add pmat
parser = create_pmat_options()
pmat.__doc__ = parser.format_help()
# Add any commands contained in this module to LLDB
debugger.HandleCommand('command script add -f %s.pmat pmat' % __name__)
In use,
(lldb) br s -p break
Breakpoint 2: where = a.out`main + 31 at a.m:8, address = 0x0000000100000eaf
(lldb) r
Process 18223 launched: '/private/tmp/a.out' (x86_64)
Process 18223 stopped
* thread #1: tid = 0x1f03, 0x0000000100000eaf a.out`main + 31 at a.m:8, stop reason = breakpoint 2.1
#0: 0x0000000100000eaf a.out`main + 31 at a.m:8
5 #autoreleasepool {
6 struct var myobj;
7 myobj.matA = #"hello there";
-> 8 printf ("%s\n", [(id)myobj.matA UTF8String]); // break here
9 }
10 }
(lldb) p myobj
(var) $0 = {
(void *) matA = 0x0000000100001070
}
(lldb) comm scri imp ~/lldb/pmat.py
(lldb) pmat myobj
hello there
(lldb)
You can put the command script import line in your ~/.lldbinit file if you want to use this.
It's easy to use the Python APIs once you have a general idea of how the debugger is structured. I knew that I would find the variable based on the frame, so I looked at the help for the SBFrame object with
(lldb) script help (lldb.SBFrame)
The method FindVariable returns an SBValue so then I looked at the lldb.SBValue help page, etc. There's a lot of boilerplate in my example python above - you're really looking at 4 lines of python that do all the work.
If this is still triggering the code path that is crashing your lldb process, you can do the last little bit of the script in two parts - get the address of the object and run po on that raw address. e.g.
child = var.GetChildMemberWithName("matA")
if child:
id = child.GetValueAsUnsigned (lldb.LLDB_INVALID_ADDRESS)
if id != lldb.LLDB_INVALID_ADDRESS:
debugger.HandleCommand ('po 0x%x' % id)
Second, using a command regex:
(lldb) br s -p break
Breakpoint 1: where = a.out`main + 31 at a.m:8, address = 0x0000000100000eaf
(lldb) r
Process 18277 launched: '/private/tmp/a.out' (x86_64)
Process 18277 stopped
* thread #1: tid = 0x1f03, 0x0000000100000eaf a.out`main + 31 at a.m:8, stop reason = breakpoint 1.1
#0: 0x0000000100000eaf a.out`main + 31 at a.m:8
5 #autoreleasepool {
6 struct var myobj;
7 myobj.matA = #"hello there";
-> 8 printf ("%s\n", [(id)myobj.matA UTF8String]); // break here
9 }
10 }
(lldb) command regex pmat 's/(.*)/po %1.matA/'
(lldb) pmat myobj
$0 = 0x0000000100001070 hello there
(lldb)
You can't use the simpler command alias in this instance - you have to use a regex alias - because you're calling a command which takes raw input. Specifically, po is really an alias to expression and you need to use regex command aliases to substitute values into those.