We Keep Coding

How to run rabbitmqctl commands using python subprocess?

Running rabbitmqctl from a Python package using subprocess returns "command not found".
proc = subprocess.Popen(['/path/to/rabbitmqctl', 'arguments'], stdout=subprocess.PIPE)
output = proc.communicate()[0]
rt = proc.returncode
The above code is part of a python project that will be packaged to a wheel distribution. After installing the wheel through pip, the above code returns an exit code 127 which is "command not found".
I tried with the full path to rabbitmqctl, used sudo with the command, used preexec_fn in subprocess and set the uid to rabbitmq user but everything returns returncode 127.
The command executes fine in the python interpreter. Issue is only when the code is installed as a package.
This code is part of a flask app which is controlled by gunicorn. I've even tried to start gunicorn with sudo, but ended up getting the same error.

The issue was due to the python virtual environment.
I installed the package that has the rabbitmqctl command in a python virtual environment. So even though the module had root privileges, it is not able to find rabbitmqctl command because the path to that binary was not part of the PATH environment variable of the virtual environment. I fixed it by adding the env parameter in subprocess.
rabbit_env = os.environ.copy()
rabbit_env['PATH'] = '/path/where/rabbitmqctl/is/located/:' + rabbit_env['PATH']
proc = subprocess.Popen(['/path/to/rabbitmqctl', 'arguments'], env=rabbit_env, stdout=subprocess.PIPE)
output = proc.communicate()[0]
rt = proc.returncode
The reason why I got exit code 127 even when I specified the full path of rabbitmqctl is because rabbitmqctl is a script that runs some other commands and rabbitmqctl was not able to find those dependent commands in the PATH because those commands' locations are not part of the virtual environment PATH. So make sure you add the locations of all the rabbitmqctl dependent commands in the rabbit_env['PATH'] above.

How to activate virtualenv using python subprocess module?

I am trying to invoke a shell script using python's subprocess module.
The shell script activates a virtualenv using virtualenvwrapper and in turn invokes a python script.
The last invoked python script needs libraries installed in virtualenv and it is crashing.
tried activating virtualenv again in python script but of no use
Parent Python code-
command = "/home/arman/analysis_server/new_analysis/run"
output = subprocess.Popen([command], shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
/run script -
#!/bin/bash
export WORKON_HOME=~/Envs
source /usr/local/bin/virtualenvwrapper.sh
workon analytics
python /home/arman/analysis_server/new_analysis/AnalysisWrapper.py
AnalysisWrapper.py -
cmd = "python /home/arman/analysis_server/new_analysis/DataHandlerWrapper.py " + instrument + " &"
subprocess.Popen(cmd, shell=True, executable='/bin/bash', stdout=out, stderr=out)
The DataHandlerWrapper.py script needs virtualenv but it is crashing

I think your issue is that Popen spawns a subshell, so you activating the virtualenv in one subprocess and trying to use it in another is never going to work.
If there's nothing happening in between you could perhaps try chaining your commands into the same process:
command = "/home/arman/analysis_server/new_analysis/run && python /home/arman/analysis_server/new_analysis/DataHandlerWrapper.py " + instrument + " &"

mkdir command not found with Python 3 on Debian

I'm sure this is something simple, but I'm trying several settings and I just can't seem to get this to work.
I have the following code:
import subprocess
p = subprocess.Popen('mkdir -p /backups/my_folder', stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
This is running in a flask application on nginx and python 3
When this executes I'm getting the following error:
/bin/sh: 1: mkdir: not found
I've tried with shell=False, I've tried with Popen(['mkdir', ...]), and I've tried subprocess.run like this question/answer
If I run with shell=False, I get the following error:
Error: [Errno 2] No such file or directory: 'mkdir -p
/backups/my_folder': 'mkdir -p /backups/my_folder'
When I do /bin/mkdir, it works. But, there are other commands which call sub commands that fail (tar calling gz for instance)
What am I missing to get this to work?
Running:
Debian 9.8, Nginx 1.14.0, Python 3.6.8
EDIT
I need this to work for other commands as well. I know I can use os.makedirs, but I have several different commands I will be executing (rsync, ssh, tar, and more)

For these simple commands, try to use python instead of invoking the shell - it makes you more independent of the environment:
os.makedirs('/backups/my_folder', exist_ok=True)

I found the problem.
I realized that my /etc/systemd/system/site.service uWSGI settings had a hard coded path:
Environment = /usr/local/bin
Once, I changed this to include /bin, all my subprocess commands executed just fine.

import subprocess
p = subprocess.Popen('mkdir -p my_folder', stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
(result, error) = p.communicate()
print(result)
this is for only windows 10.

psexec commands works perfectly from powershell but not recognized by python

disclaimer - I know there is a package pypsexec for this i'm asking why this happens and how to solve it.
the command
psexec -s -i -d \\<PC-NAME> -u <UserName> -p <Password> <Command>
works perfectly when typed manually at powershell
however when I tried mimicking this with python as
from subprocess import Popen,PIPE
p = Popen("""psexec -s -i -d \\<PC-NAME> -u <UserName> -p <Password>
<Command>""", stdin=PIPE, stdout=PIPE, shell= True )
stdout, stderr = p.communicate()
print(stdout, stderr)
I get the following:
'psexec' is not recognized as an internal or external command,
operable program or batch file.
b'' None
any idea why? psexec is configured at variable path and as i said works from cmd/powershell same error for pskill etc.
Solved - read the comments

Move psexec.exe to C:\windows\SysWOW64. 32-bit python reads from there
If you are interested, I made a package for PsExec:
You can perform a lot of fun operations with it.
Project here
Please check it out :)

How to execute psexec command from python without specifying psexec exe location?

I have a python script from which i am executing a process on remote machine as follows:
sample= sp.Popen( ['c:/psexec/PsExec.exe','-i','-s','\\\\' + 'xyz','-u', 'sample','-p', 'xyz','C:/sample.bat'],stdin=sp.PIPE, stdout = sp.PIPE, stderr=sp.PIPE)
It executes well but what i want not to provide complete exe path as follows:
sample= sp.Popen( ['psexec','-i','-s','\\\\' + 'xyz','-u', 'sample','-p', 'xyz','C:/sample.bat'],stdin=sp.PIPE, stdout = sp.PIPE, stderr=sp.PIPE)
It does not work when i remove the complete psexec exe path. So, suggest what am i not doing right and how shall i execute python script using only psexec keyword.

I know this question is fairly old, but it appears that psexec is not installed in the standard location, so it is likely that the location of the psexec binary is not in your systems's PATH environment variable. Add c:\psexec to the PATH env var on the local machine, start a new command prompt, and this should work.