We Keep Coding

Python3 / subprocess.check_output / Timeout does not work for every command

I'm running Debian 10.8 with Python 3.7.3 and running a subprocess inside a script which I would like to interrupt after some seconds. This works for cmd_1 but not for cmd_2 in the example below (timeout is never triggered):
import subprocess
import os, sys
# collect proxies and verify
try:
# cmd_1 = "while true; do sleep 1; done"
cmd_2 = "proxybroker find --types HTTPS -l 1000 --outfile proxybroker.txt"
timeout_sec = 3
subprocess.check_output(cmd_2, shell=True, timeout=timeout_sec)
except Exception as e:
print(e)
If I run cmd_2 in a bash, it works fine.
I did install the proxybroker package with pip3 install proxybroker.
On another system running Ubuntu 21.04 the timeout works for both commands. Thanks if someone can provide a hint.

I can't precisely explain why this happens, but getting rid of the gratuitous shell fixes it.
tripleee#debian-buster-docker$ cat >subn.py
import subprocess
# import os, sys # unused import
# collect proxies and verify
try:
subprocess.check_output(['proxybroker', 'find', '--types', 'HTTPS', '-l', '1000', '--outfile', 'proxies.txt'], timeout=3)
except Exception as e:
print(e)
^D
tripleee#debian-buster-docker$ python3 subn.py
Command '['proxybroker', 'find', '--types', 'HTTPS', '-l', '1000', '--outfile', 'proxies.txt']' timed out after 3 seconds
I notice that your original attempt does print the timeout message as part of the traceback when I interrupt it with a KeyboardInterrupt.
tripleee#debian-buster-docker$ python3 sub.py
^CTraceback (most recent call last):
File "/usr/lib/python3.7/subprocess.py", line 474, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "/usr/lib/python3.7/subprocess.py", line 939, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/lib/python3.7/subprocess.py", line 1682, in _communicate
self._check_timeout(endtime, orig_timeout)
File "/usr/lib/python3.7/subprocess.py", line 982, in _check_timeout
raise TimeoutExpired(self.args, orig_timeout)
subprocess.TimeoutExpired: Command 'proxybroker find --types HTTPS -l 1000 --outfile proxybroker.txt' timed out after 3 seconds
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "sub.py", line 9, in <module>
subprocess.check_output(cmd_2, shell=True, timeout=timeout_sec)
File "/usr/lib/python3.7/subprocess.py", line 395, in check_output
**kwargs).stdout
File "/usr/lib/python3.7/subprocess.py", line 477, in run
stdout, stderr = process.communicate()
File "/usr/lib/python3.7/subprocess.py", line 939, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/lib/python3.7/subprocess.py", line 1681, in _communicate
ready = selector.select(timeout)
File "/usr/lib/python3.7/selectors.py", line 415, in select
fd_event_list = self._selector.poll(timeout)
KeyboardInterrupt

As a workaround, timeout as shell command can be used:
cmd_2 = "timeout 3s proxybroker find --types HTTPS -l 1000 --outfile proxybroker.txt"

Paramiko "TypeError: 'NoneType' object is not callable" at the end of a script that execute command on the server

I'm writing a script to execute commands in an EC2 instance, and come back. For that, I'm using Paramiko. My script is like this:
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('<hostname>', username='<username>', password='<password>')
print('connected')
stdin, stdout, stderr = ssh.exec_command("ls -lah")
ssh.close()
print('done')
The trace I'm getting is:
connected
done
Exception ignored in: <function BufferedFile.__del__ at 0x7fa5de814950>
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/paramiko/file.py", line 66, in __del__
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/paramiko/channel.py", line 1392, in close
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/paramiko/channel.py", line 991, in shutdown_write
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/paramiko/channel.py", line 963, in shutdown
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/paramiko/channel.py", line 1246, in _send_eof
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/paramiko/message.py", line 232, in add_int
TypeError: 'NoneType' object is not callable
The command which I use to login is:
ssh -o PubkeyAuthentication=no -l <username> <hostname>
Can someone tell me what I'm doing wrong here?

You are not waiting for the command to complete before you close the SSH connection.
When Python garbage-collects the stdin, stdout and stderr, Paramiko crashes, as it does not expect the connection to be closed.
For a correct code, see Wait to finish command executed with Python Paramiko.

if ssh is not None:
ssh.close()
del client, stdin, stdout, stderr

paramiko not executing basic command on machine

I am trying to execute a basic command on a device using paramiko ("show clock", which displays the time):
#!/usr/bin/python
import paramiko
import time
import re
hostname = 'HIDDEN1'
port = '22'
username = 'admin'
password = 'HIDDEN2'
if __name__ == "__main__":
s = paramiko.SSHClient()
s.load_system_host_keys()
s.set_missing_host_key_policy(paramiko.AutoAddPolicy())
s.connect(hostname, port, username, password, timeout=3)
command = 'show clock'
print("Starting...")
stdin, stdout, stderr = s.exec_command(command)
s.close()
It doesn't run the command; I'm sure it connects though, since if I purposely make the password incorrect it hangs instead of returning an error. I ensured I can connect manually to the device and run the "show clock" command, but the paramiko snippet doesn't work. This is the error it returns:
Starting...
Traceback (most recent call last):
File "./para2.py", line 21, in <module>
stdin, stdout, stderr = s.exec_command('show clock')
File "/usr/lib/python2.6/site-packages/paramiko/client.py", line 350, in exec_command
chan.exec_command(command)
File "/usr/lib/python2.6/site-packages/paramiko/channel.py", line 213, in exec_command
self._wait_for_event()
File "/usr/lib/python2.6/site-packages/paramiko/channel.py", line 1084, in _wait_for_event
raise e
EOFError

The server might not be allowing exec_command()
Try using the interactive shell
ssh_client = paramiko.SSHClient()
shh_client.connect(#creds)
shell = ssh_client.invoke_shell()
You can now use shell.send() and shell.recv() to execute commands and get back their outputs
http://docs.paramiko.org/en/2.7/api/channel.html#paramiko.channel.Channel.send
http://docs.paramiko.org/en/2.7/api/channel.html#paramiko.channel.Channel.recv
Example: https://www.semicolonworld.com/question/56794/implement-an-interactive-shell-over-ssh-in-python-using-paramiko

Python SSH Paramiko EOFError

I have just simple script like that to connect via SSH on Nokia router and execute command "show time":
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('adres ip', port=22, username='username', password='password')
ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command('show time')
output = stdout.readlines()
print '\n'.join(output)
ssh.close()
Login to the node is successful. I see myself on the router, but executing command are not gonna work. I get error like that:
Traceback (most recent call last): File "C:\Users\pkudalsk\Desktop\pyt.py", line 6, in <module>
ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command('show time') File "C:\Users\pkudalsk\Desktop\paramiko\client.py", line 479, in exec_command
chan.exec_command(command) File "C:\Users\pkudalsk\Desktop\paramiko\channel.py", line 63, in _check
return func(self, *args, **kwds) File "C:\Users\pkudalsk\Desktop\paramiko\channel.py", line 241, in exec_comman d
self._wait_for_event() File "C:\Users\pkudalsk\Desktop\paramiko\channel.py", line 1198, in
_wait_for_ event
raise e EOFError
Does anyone know what can cause this problem? I tried on python 3.6 and 2.7. The same result.
Thanks

Python paramiko is not giving complete output and I believe session is closing unexpectedly

Environment:
Python: 2.7.6
Paramiko: 1.13.0
Target SSH Server: Cisco MDS NX-OS 6.2(3)
Code:
import paramiko
import StringIO
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect("switchname", username="username", password="password")
stdin, stdout, stderr = ssh.exec_command("term length 0")
stdin, stdout, stderr = ssh.exec_command("show zoneset active")
t = stdout.read()
ssh.close()
for line in StringIO.StringIO(t):
line = line.replace("\n","")
print line
It is only producing few lines of output instead of the entire output. If I modify above code and add one more ssh.exec_command like shown below.
Code:
import paramiko
import StringIO
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect("switchname", username="username", password="password")
stdin, stdout, stderr = ssh.exec_command("term length 0")
stdin, stdout, stderr = ssh.exec_command("show zoneset active")
stdin2, stdout2, stderr2 = ssh.exec_command("show version ")
t = stdout.read()
ssh.close()
for line in StringIO.StringIO(t):
line = line.replace("\n","")
print line
It produce following errors:
Traceback (most recent call last):
File "paramiko_issue.py", line 15, in <module>
stdin, stdout, stderr = ssh.exec_command("show zoneset active")
File "build/bdist.macosx-10.9-intel/egg/paramiko/client.py", line 307, in exec_command
File "build/bdist.macosx-10.9-intel/egg/paramiko/transport.py", line 543, in open_session
File "build/bdist.macosx-10.9-intel/egg/paramiko/transport.py", line 607, in open_channel
paramiko.ssh_exception.SSHException: SSH session not active
I would really appreciate any help in resolving this issue. Thank you in advance.