Related
I have a command /usr/bin/virsh dumpxml <UUID> that I need to run inside of a python program, the thing is that the command needs to run like sudo -u <user> sudo /usr/bin/virsh dumpxml <UUID>, I try to call it with the following code:
cmd = "sudo -u <user> sudo /usr/bin/virsh dumpxml %s" % uuid
data = os.popen(cmd).read()
...
But I'm getting an error message indicating that the domain is not found, but, if I run the same command from the bash, it works fine.
The /usr/bin/virsh dumpxml command is inside the /etc/sudoers.d/
Also, I need to run with the second 'sudo' because if not, it won't work
Any ideas?
Through Fabric, I am trying to start a celerycam process using the below nohup command. Unfortunately, nothing is happening. Manually using the same command, I could start the process but not through Fabric. Any advice on how can I solve this?
def start_celerycam():
'''Start celerycam daemon'''
with cd(env.project_dir):
virtualenv('nohup bash -c "python manage.py celerycam --logfile=%scelerycam.log --pidfile=%scelerycam.pid &> %scelerycam.nohup &> %scelerycam.err" &' % (env.celery_log_dir,env.celery_log_dir,env.celery_log_dir,env.celery_log_dir))
I'm using Erich Heine's suggestion to use 'dtach' and it's working pretty well for me:
def runbg(cmd, sockname="dtach"):
return run('dtach -n `mktemp -u /tmp/%s.XXXX` %s' % (sockname, cmd))
This was found here.
As I have experimented, the solution is a combination of two factors:
run process as a daemon: nohup ./command &> /dev/null &
use pty=False for fabric run
So, your function should look like this:
def background_run(command):
command = 'nohup %s &> /dev/null &' % command
run(command, pty=False)
And you can launch it with:
execute(background_run, your_command)
This is an instance of this issue. Background processes will be killed when the command ends. Unfortunately on CentOS 6 doesn't support pty-less sudo commands.
The final entry in the issue mentions using sudo('set -m; service servicename start'). This turns on Job Control and therefore background processes are put in their own process group. As a result they are not terminated when the command ends.
For even more information see this link.
you just need to run
run("(nohup yourcommand >& /dev/null < /dev/null &) && sleep 1")
DTACH is the way to go. It's a software you need to install like a lite version of screen.
This is a better version of the "dtach"-method found above, it will install dtach if necessary. It's to be found here where you can also learn how to get the output of the process which is running in the background:
from fabric.api import run
from fabric.api import sudo
from fabric.contrib.files import exists
def run_bg(cmd, before=None, sockname="dtach", use_sudo=False):
"""Run a command in the background using dtach
:param cmd: The command to run
:param output_file: The file to send all of the output to.
:param before: The command to run before the dtach. E.g. exporting
environment variable
:param sockname: The socket name to use for the temp file
:param use_sudo: Whether or not to use sudo
"""
if not exists("/usr/bin/dtach"):
sudo("apt-get install dtach")
if before:
cmd = "{}; dtach -n `mktemp -u /tmp/{}.XXXX` {}".format(
before, sockname, cmd)
else:
cmd = "dtach -n `mktemp -u /tmp/{}.XXXX` {}".format(sockname, cmd)
if use_sudo:
return sudo(cmd)
else:
return run(cmd)
May this help you, like it helped me to run omxplayer via fabric on a remote rasberry pi!
You can use :
run('nohup /home/ubuntu/spider/bin/python3 /home/ubuntu/spider/Desktop/baidu_index/baidu_index.py > /home/ubuntu/spider/Desktop/baidu_index/baidu_index.py.log 2>&1 &', pty=False)
nohup did not work for me and I did not have tmux or dtach installed on all the boxes I wanted to use this on so I ended up using screen like so:
run("screen -d -m bash -c '{}'".format(command), pty=False)
This tells screen to start a bash shell in a detached terminal that runs your command
You could be running into this issue
Try adding 'pty=False' to the sudo command (I assume virtualenv is calling sudo or run somewhere?)
This worked for me:
sudo('python %s/manage.py celerycam --detach --pidfile=celerycam.pid' % siteDir)
Edit: I had to make sure the pid file was removed first so this was the full code:
# Create new celerycam
sudo('rm celerycam.pid', warn_only=True)
sudo('python %s/manage.py celerycam --detach --pidfile=celerycam.pid' % siteDir)
I was able to circumvent this issue by running nohup ... & over ssh in a separate local shell script. In fabfile.py:
#task
def startup():
local('./do-stuff-in-background.sh {0}'.format(env.host))
and in do-stuff-in-background.sh:
#!/bin/sh
set -e
set -o nounset
HOST=$1
ssh $HOST -T << HERE
nohup df -h 1>>~/df.log 2>>~/df.err &
HERE
Of course, you could also pass in the command and standard output / error log files as arguments to make this script more generally useful.
(In my case, I didn't have admin rights to install dtach, and neither screen -d -m nor pty=False / sleep 1 worked properly for me. YMMV, especially as I have no idea why this works...)
I'm trying to write a small script to mount a VirtualBox shared folder each time I execute the script. I want to do it with Python, because I'm trying to learn it for scripting.
The problem is that I need privileges to launch mount command. I could run the script as sudo, but I prefer it to make sudo by its own.
I already know that it is not safe to write your password into a .py file, but we are talking about a virtual machine that is not critical at all: I just want to click the .py script and get it working.
This is my attempt:
#!/usr/bin/env python
import subprocess
sudoPassword = 'mypass'
command = 'mount -t vboxsf myfolder /home/myuser/myfolder'
subprocess.Popen('sudo -S' , shell=True,stdout=subprocess.PIPE)
subprocess.Popen(sudoPassword , shell=True,stdout=subprocess.PIPE)
subprocess.Popen(command , shell=True,stdout=subprocess.PIPE)
My python version is 2.6
Many answers focus on how to make your solution work, while very few suggest that your solution is a very bad approach. If you really want to "practice to learn", why not practice using good solutions? Hardcoding your password is learning the wrong approach!
If what you really want is a password-less mount for that volume, maybe sudo isn't needed at all! So may I suggest other approaches?
Use /etc/fstab as mensi suggested. Use options user and noauto to let regular users mount that volume.
Use Polkit for passwordless actions: Configure a .policy file for your script with <allow_any>yes</allow_any> and drop at /usr/share/polkit-1/actions
Edit /etc/sudoers to allow your user to use sudo without typing your password. As #Anders suggested, you can restrict such usage to specific commands, thus avoiding unlimited passwordless root priviledges in your account. See this answer for more details on /etc/sudoers.
All the above allow passwordless root privilege, none require you to hardcode your password. Choose any approach and I can explain it in more detail.
As for why it is a very bad idea to hardcode passwords, here are a few good links for further reading:
Why You Shouldn’t Hard Code Your Passwords When Programming
How to keep secrets secret
(Alternatives to Hardcoding Passwords)
What's more secure? Hard coding credentials or storing them in a database?
Use of hard-coded credentials, a dangerous programming error: CWE
Hard-coded passwords remain a key security flaw
sudoPassword = 'mypass'
command = 'mount -t vboxsf myfolder /home/myuser/myfolder'
p = os.system('echo %s|sudo -S %s' % (sudoPassword, command))
Try this and let me know if it works. :-)
And this one:
os.popen("sudo -S %s"%(command), 'w').write('mypass')
To pass the password to sudo's stdin:
#!/usr/bin/env python
from subprocess import Popen, PIPE
sudo_password = 'mypass'
command = 'mount -t vboxsf myfolder /home/myuser/myfolder'.split()
p = Popen(['sudo', '-S'] + command, stdin=PIPE, stderr=PIPE,
universal_newlines=True)
sudo_prompt = p.communicate(sudo_password + '\n')[1]
Note: you could probably configure passwordless sudo or SUDO_ASKPASS command instead of hardcoding your password in the source code.
Use -S option in the sudo command which tells to read the password from 'stdin' instead of the terminal device.
Tell Popen to read stdin from PIPE.
Send the Password to the stdin PIPE of the process by using it as an argument to communicate method. Do not forget to add a new line character, '\n', at the end of the password.
sp = Popen(cmd , shell=True, stdin=PIPE)
out, err = sp.communicate(_user_pass+'\n')
subprocess.Popen creates a process and opens pipes and stuff. What you are doing is:
Start a process sudo -S
Start a process mypass
Start a process mount -t vboxsf myfolder /home/myuser/myfolder
which is obviously not going to work. You need to pass the arguments to Popen. If you look at its documentation, you will notice that the first argument is actually a list of the arguments.
I used this for python 3.5. I did it using subprocess module.Using the password like this is very insecure.
The subprocess module takes command as a list of strings so either create a list beforehand using split() or pass the whole list later. Read the documentation for moreinformation.
#!/usr/bin/env python
import subprocess
sudoPassword = 'mypass'
command = 'mount -t vboxsf myfolder /home/myuser/myfolder'.split()
cmd1 = subprocess.Popen(['echo',sudoPassword], stdout=subprocess.PIPE)
cmd2 = subprocess.Popen(['sudo','-S'] + command, stdin=cmd1.stdout, stdout=subprocess.PIPE)
output = cmd2.stdout.read.decode()
sometimes require a carriage return:
os.popen("sudo -S %s"%(command), 'w').write('mypass\n')
Please try module pexpect. Here is my code:
import pexpect
remove = pexpect.spawn('sudo dpkg --purge mytool.deb')
remove.logfile = open('log/expect-uninstall-deb.log', 'w')
remove.logfile.write('try to dpkg --purge mytool\n')
if remove.expect(['(?i)password.*']) == 0:
# print "successfull"
remove.sendline('mypassword')
time.sleep(2)
remove.expect(pexpect.EOF,5)
else:
raise AssertionError("Fail to Uninstall deb package !")
To limit what you run as sudo, you could run
python non_sudo_stuff.py
sudo -E python -c "import os; os.system('sudo echo 1')"
without needing to store the password. The -E parameter passes your current user's env to the process. Note that your shell will have sudo priveleges after the second command, so use with caution!
I know it is always preferred not to hardcode the sudo password in the script. However, for some reason, if you have no permission to modify /etc/sudoers or change file owner, Pexpect is a feasible alternative.
Here is a Python function sudo_exec for your reference:
import platform, os, logging
import subprocess, pexpect
log = logging.getLogger(__name__)
def sudo_exec(cmdline, passwd):
osname = platform.system()
if osname == 'Linux':
prompt = r'\[sudo\] password for %s: ' % os.environ['USER']
elif osname == 'Darwin':
prompt = 'Password:'
else:
assert False, osname
child = pexpect.spawn(cmdline)
idx = child.expect([prompt, pexpect.EOF], 3)
if idx == 0: # if prompted for the sudo password
log.debug('sudo password was asked.')
child.sendline(passwd)
child.expect(pexpect.EOF)
return child.before
It works in python 2.7 and 3.8:
from subprocess import Popen, PIPE
from shlex import split
proc = Popen(split('sudo -S %s' % command), bufsize=0, stdout=PIPE, stdin=PIPE, stderr=PIPE)
proc.stdin.write((password +'\n').encode()) # write as bytes
proc.stdin.flush() # need if not bufsize=0 (unbuffered stdin)
without .flush() password will not reach sudo if stdin buffered.
In python 2.7 Popen by default used bufsize=0 and stdin.flush() was not needed.
For secure using, create password file in protected directory:
mkdir --mode=700 ~/.prot_dir
nano ~/.prot_dir/passwd.txt
chmod 600 ~/.prot_dir/passwd.txt
at start your py-script read password from ~/.prot_dir/passwd.txt
with open(os.environ['HOME'] +'/.prot_dir/passwd.txt') as f:
password = f.readline().rstrip()
import os
os.system("echo TYPE_YOUR_PASSWORD_HERE | sudo -S TYPE_YOUR_LINUX_COMMAND")
Open your ide and run the above code. Please change TYPE_YOUR_PASSWORD_HERE and TYPE_YOUR_LINUX_COMMAND to your linux admin password and your desired linux command after that run your python script. Your output will show on terminal. Happy Coding :)
You can use SSHScript . Below are example codes:
## filename: example.spy
sudoPassword = 'mypass'
command = 'mount -t vboxsf myfolder /home/myuser/myfolder'
$$echo #{sudoPassword} | sudo -S #{command}
or, simply one line (almost the same as running on console)
## filename: example.spy
$$echo mypass | sudo -S mount -t vboxsf myfolder /home/myuser/myfolder
Then, run it on console
sshscript example.spy
Where "sshscript" is the CLI of SSHScript (installed by pip).
solution im going with,because password in plain txt in an env file on dev pc is ok, and variable in the repo and gitlab runner is masked.
use .dotenv put pass in .env on local machine, DONT COMMIT .env to git.
add same var in gitlab variable
.env file has:
PASSWORD=superpass
from dotenv import load_dotenv
load_dotenv()
subprocess.run(f'echo {os.getenv("PASSWORD")} | sudo -S rm /home//folder/filetodelete_created_as_root.txt', shell=True, check=True)
this works locally and in gitlab. no plain password is committed to repo.
yes, you can argue running a sudo command w shell true is kind of crazy, but if you have files written to host from a docker w root, and you need to pro-grammatically delete them, this is functional.
I am struggling to launch my fabfile within my Python script. I have looked at similar posts on Stack Overflow regarding this but they don't solve my problem... Or maybe they do but I am not understanding them...Not sure.
My script writes to the fab file depending on what the user wants to run on the remote host. Here is an example of the fabfile:
[root#ip-50-50-50-50 bakery]# cat fabfile.py
from fabric.api import run
def deploy():
run('wget -P /tmp https://s3.amazonaws.com/MyBucket/httpd-2.2.26-1.1.amzn1.x86_64.rpm')
run('sudo yum localinstall /tmp/httpd-2.2.26-1.1.amzn1.x86_64.rpm')
I then need to run the fabfile from my script. If I run the following manually form the Command Line, then it works fine:
fab -f fabfile.py -u ec2-user -i id_rsa -H 10.10.15.150 deploy
1) How do I run that from inside my script with all of the options?
2) The IP address is a variable called "bakery_internalip". How do I call that variable as part of the fab line?
Try with subprocess :
import subprocess
subprocess.call(['fab', '-f', 'fabfile.py', '-u ec2-user', '-i', 'id_rsa', '-H', bakery_internalip, 'deploy'])
should do the trick
You can call fabric enabled commands directly from code. Usually you will have to set the env dictionary first to specify keys and host but is is very straightforward:
# the settings for the env dict
from fabric.api import env, execute
env.hosts = ["10.10.15.150", ]
env.user = "ec2-user"
env.key = "id_rsa"
# and call the function itself
from fabfile import deplot
execute(deploy, hosts=env.hosts)
You can find more inspiration in the fabric documentation:
http://docs.fabfile.org/en/1.11/usage/execution.html#using-execute-with-dynamically-set-host-lists
I'm trying to write a small script to mount a VirtualBox shared folder each time I execute the script. I want to do it with Python, because I'm trying to learn it for scripting.
The problem is that I need privileges to launch mount command. I could run the script as sudo, but I prefer it to make sudo by its own.
I already know that it is not safe to write your password into a .py file, but we are talking about a virtual machine that is not critical at all: I just want to click the .py script and get it working.
This is my attempt:
#!/usr/bin/env python
import subprocess
sudoPassword = 'mypass'
command = 'mount -t vboxsf myfolder /home/myuser/myfolder'
subprocess.Popen('sudo -S' , shell=True,stdout=subprocess.PIPE)
subprocess.Popen(sudoPassword , shell=True,stdout=subprocess.PIPE)
subprocess.Popen(command , shell=True,stdout=subprocess.PIPE)
My python version is 2.6
Many answers focus on how to make your solution work, while very few suggest that your solution is a very bad approach. If you really want to "practice to learn", why not practice using good solutions? Hardcoding your password is learning the wrong approach!
If what you really want is a password-less mount for that volume, maybe sudo isn't needed at all! So may I suggest other approaches?
Use /etc/fstab as mensi suggested. Use options user and noauto to let regular users mount that volume.
Use Polkit for passwordless actions: Configure a .policy file for your script with <allow_any>yes</allow_any> and drop at /usr/share/polkit-1/actions
Edit /etc/sudoers to allow your user to use sudo without typing your password. As #Anders suggested, you can restrict such usage to specific commands, thus avoiding unlimited passwordless root priviledges in your account. See this answer for more details on /etc/sudoers.
All the above allow passwordless root privilege, none require you to hardcode your password. Choose any approach and I can explain it in more detail.
As for why it is a very bad idea to hardcode passwords, here are a few good links for further reading:
Why You Shouldn’t Hard Code Your Passwords When Programming
How to keep secrets secret
(Alternatives to Hardcoding Passwords)
What's more secure? Hard coding credentials or storing them in a database?
Use of hard-coded credentials, a dangerous programming error: CWE
Hard-coded passwords remain a key security flaw
sudoPassword = 'mypass'
command = 'mount -t vboxsf myfolder /home/myuser/myfolder'
p = os.system('echo %s|sudo -S %s' % (sudoPassword, command))
Try this and let me know if it works. :-)
And this one:
os.popen("sudo -S %s"%(command), 'w').write('mypass')
To pass the password to sudo's stdin:
#!/usr/bin/env python
from subprocess import Popen, PIPE
sudo_password = 'mypass'
command = 'mount -t vboxsf myfolder /home/myuser/myfolder'.split()
p = Popen(['sudo', '-S'] + command, stdin=PIPE, stderr=PIPE,
universal_newlines=True)
sudo_prompt = p.communicate(sudo_password + '\n')[1]
Note: you could probably configure passwordless sudo or SUDO_ASKPASS command instead of hardcoding your password in the source code.
Use -S option in the sudo command which tells to read the password from 'stdin' instead of the terminal device.
Tell Popen to read stdin from PIPE.
Send the Password to the stdin PIPE of the process by using it as an argument to communicate method. Do not forget to add a new line character, '\n', at the end of the password.
sp = Popen(cmd , shell=True, stdin=PIPE)
out, err = sp.communicate(_user_pass+'\n')
subprocess.Popen creates a process and opens pipes and stuff. What you are doing is:
Start a process sudo -S
Start a process mypass
Start a process mount -t vboxsf myfolder /home/myuser/myfolder
which is obviously not going to work. You need to pass the arguments to Popen. If you look at its documentation, you will notice that the first argument is actually a list of the arguments.
I used this for python 3.5. I did it using subprocess module.Using the password like this is very insecure.
The subprocess module takes command as a list of strings so either create a list beforehand using split() or pass the whole list later. Read the documentation for moreinformation.
#!/usr/bin/env python
import subprocess
sudoPassword = 'mypass'
command = 'mount -t vboxsf myfolder /home/myuser/myfolder'.split()
cmd1 = subprocess.Popen(['echo',sudoPassword], stdout=subprocess.PIPE)
cmd2 = subprocess.Popen(['sudo','-S'] + command, stdin=cmd1.stdout, stdout=subprocess.PIPE)
output = cmd2.stdout.read.decode()
sometimes require a carriage return:
os.popen("sudo -S %s"%(command), 'w').write('mypass\n')
Please try module pexpect. Here is my code:
import pexpect
remove = pexpect.spawn('sudo dpkg --purge mytool.deb')
remove.logfile = open('log/expect-uninstall-deb.log', 'w')
remove.logfile.write('try to dpkg --purge mytool\n')
if remove.expect(['(?i)password.*']) == 0:
# print "successfull"
remove.sendline('mypassword')
time.sleep(2)
remove.expect(pexpect.EOF,5)
else:
raise AssertionError("Fail to Uninstall deb package !")
To limit what you run as sudo, you could run
python non_sudo_stuff.py
sudo -E python -c "import os; os.system('sudo echo 1')"
without needing to store the password. The -E parameter passes your current user's env to the process. Note that your shell will have sudo priveleges after the second command, so use with caution!
I know it is always preferred not to hardcode the sudo password in the script. However, for some reason, if you have no permission to modify /etc/sudoers or change file owner, Pexpect is a feasible alternative.
Here is a Python function sudo_exec for your reference:
import platform, os, logging
import subprocess, pexpect
log = logging.getLogger(__name__)
def sudo_exec(cmdline, passwd):
osname = platform.system()
if osname == 'Linux':
prompt = r'\[sudo\] password for %s: ' % os.environ['USER']
elif osname == 'Darwin':
prompt = 'Password:'
else:
assert False, osname
child = pexpect.spawn(cmdline)
idx = child.expect([prompt, pexpect.EOF], 3)
if idx == 0: # if prompted for the sudo password
log.debug('sudo password was asked.')
child.sendline(passwd)
child.expect(pexpect.EOF)
return child.before
It works in python 2.7 and 3.8:
from subprocess import Popen, PIPE
from shlex import split
proc = Popen(split('sudo -S %s' % command), bufsize=0, stdout=PIPE, stdin=PIPE, stderr=PIPE)
proc.stdin.write((password +'\n').encode()) # write as bytes
proc.stdin.flush() # need if not bufsize=0 (unbuffered stdin)
without .flush() password will not reach sudo if stdin buffered.
In python 2.7 Popen by default used bufsize=0 and stdin.flush() was not needed.
For secure using, create password file in protected directory:
mkdir --mode=700 ~/.prot_dir
nano ~/.prot_dir/passwd.txt
chmod 600 ~/.prot_dir/passwd.txt
at start your py-script read password from ~/.prot_dir/passwd.txt
with open(os.environ['HOME'] +'/.prot_dir/passwd.txt') as f:
password = f.readline().rstrip()
import os
os.system("echo TYPE_YOUR_PASSWORD_HERE | sudo -S TYPE_YOUR_LINUX_COMMAND")
Open your ide and run the above code. Please change TYPE_YOUR_PASSWORD_HERE and TYPE_YOUR_LINUX_COMMAND to your linux admin password and your desired linux command after that run your python script. Your output will show on terminal. Happy Coding :)
You can use SSHScript . Below are example codes:
## filename: example.spy
sudoPassword = 'mypass'
command = 'mount -t vboxsf myfolder /home/myuser/myfolder'
$$echo #{sudoPassword} | sudo -S #{command}
or, simply one line (almost the same as running on console)
## filename: example.spy
$$echo mypass | sudo -S mount -t vboxsf myfolder /home/myuser/myfolder
Then, run it on console
sshscript example.spy
Where "sshscript" is the CLI of SSHScript (installed by pip).
solution im going with,because password in plain txt in an env file on dev pc is ok, and variable in the repo and gitlab runner is masked.
use .dotenv put pass in .env on local machine, DONT COMMIT .env to git.
add same var in gitlab variable
.env file has:
PASSWORD=superpass
from dotenv import load_dotenv
load_dotenv()
subprocess.run(f'echo {os.getenv("PASSWORD")} | sudo -S rm /home//folder/filetodelete_created_as_root.txt', shell=True, check=True)
this works locally and in gitlab. no plain password is committed to repo.
yes, you can argue running a sudo command w shell true is kind of crazy, but if you have files written to host from a docker w root, and you need to pro-grammatically delete them, this is functional.