I use the following bash code to send email:
echo abc | mail -s Subject abc#def.com
It works fine. Now I want to use the same command in python, so I write the following:
call(["echo abc" "|" "mail", "-s Subject", "abc#def.com"], shell=True)
This python code does not send me the email and when I call the python script using
$python email.py
I get the following information from shell
No mail for rex
How does this happen?
There's no need for echo and piping between commands if you're using Python.
You can start a process and use the communicate method:
import subprocess
def send_message(recipient, subject, body):
process = subprocess.Popen(['mail', '-s', subject, recipient],
stdin=subprocess.PIPE)
process.communicate(body)
Using subprocess.communicate as suggested elsewhere is a much better approach. You should avoid using shell=True for the reasons described here. But to answer your question why your call doesn't work, the problem is that you've broken the shell string up into rather arbitrary strings. If you pass the whole command as one string, it should work on Unix-y platforms at least.
call(["echo abc|mail -s Subject abc#def.com"], shell=True)
See the details here. But such usage is not recommended.
Related
I'm trying to write a script to log in to 1 of 30+ accounts on a single ftp site which I use for work.
The behavior that I'd like to see:
$ ftp ftp.someplace.com
connected blah blah
220 blah blah ready...
Name (something): username
Password: <enter password>
couple lines saying successful login
ftp> <manually enter commands such as ls, cd, get, put, etc>
I've tried this in Python with ftplib using FTP.connect()/.login(), but nothing that I found in the ftplib relinquishes control and makes the ftp session interactive.
My next step was to try curl:
curl ftp://user:password#ftp.someplace.com
This executes and exits with return code 0.
Lastly, I tried ftp from the CLI in a method I would call it from a script:
ftp ftp.someplace.com <<END_SCRIPT
> quote USER username
> quote PASS password
> END_SCRIPT
This executes and exits with return code 0.
Is there a way to write a script to do this? I'd prefer this in python, but I'm okay with making a bash subprocess call. At this point I'm looking at some complex .netrc entry but I feel like I'm going down a rabbit hole.
Thanks for your help!
**************EDITED AFTER ANSWERED WITH PYTHON IMPLEMENTATION****************
The following 2 lines in python work:
process = subprocess.Popen("lftp -e ls -u {0},{1} ftp.someplace.com".format(username,password), shell=True)
output, error = process.communicate()
Would installing lftp be feasible? It has a flag, -e, that may suit your needs:
-e cmd execute the command just after selecting the server
After it executes the command it stays interactive. You could use the following approach where you have one file per account that has the following pattern (name this one, say, user1.lftp:
open ftp://user1:password#ftp.someplace.com
And when you want to connect as user1:
lftp -e 'source user1.lftp'
This will drop you to a prompt from which you can execute ftp commands. You could make it more convenient by defining function like the following in your .bashrc:
lftp-open() {
lftp -e "source $1.lftp"
}
And then it would be as simple as:
$ lftp-open user1
Obviously the downside of this, or probably of any approach to automating ftp login, is the requirement to store your password in plaintext, so you'd want to ensure that you have the proper read permissions set on the *.lftp files.
All the previous posts on this topic deal with specific challenges for their use case. I thought it would be useful to have a post only dealing with the cleanest way to run PowerShell scripts from Python and ask if anyone has an better solution than what I found.
What seems to be the generally accepted solution to get around PowerShell trying to interpret different control characters in your command differently to what's intended is to feed your Powershell command in using a file:
ps = 'powershell.exe -noprofile'
pscommand = 'Invoke-Command -ComputerName serverx -ScriptBlock {cmd.exe \
/c "dir /b C:\}'
psfile = open(pscmdfile.ps1, 'w')
psfile.write(pscommand)
psfile.close()
full_command_string = ps + ' pscmdfile.ps1'
process = subprocess.Popen(full_command_string , shell=True, \
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
When your python code needs to change the parameters for the Powershell command each time you invoke it you end up writing and deleting a lot of temporary files for subprocess.Popen to run. It works perfectly but it's unnecessary and not very clean. It's really nice to be able to tidy up and wanted to get suggestions on any improvements I could make to the solution I found.
Instead of writing a file to disk containing the PS command create a virtual file using the io module. Assuming that the "date" and "server" strings are being fed in as part of a loop or function that contains this code, not including the imports of course:
import subprocess
import io
from string import Template
raw_shellcmd = 'powershell.exe -noprofile '
--start of loop with server and date variables populated--
raw_pslistcmd = r'Invoke-Command -ComputerName $server -ScriptBlock ' \
r'{cmd.exe /c "dir /b C:\folder\$date"}'
pslistcmd_template = Template(raw_pslistcmd)
pslistcmd = pslistcmd_template.substitute(server=server, date=date)
virtualfilepslistcommand = io.BytesIO(pslistcmd)
shellcmd = raw_shellcmd + virtualfilepslistcommand.read()
process = subprocess.Popen(shellcmd, shell=True, stdout=subprocess.PIPE, \
stderr=subprocess.PIPE)
--end of loop--
Arguably the best approach is to use powershell.exe -Command rather than writing the PowerShell command to a file:
pscommand = 'Invoke-Command ...'
process = subprocess.Popen(['powershell.exe', '-NoProfile', '-Command', '"&{' + pscommand + '}"'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Make sure double quotes in the pscommand string are properly escaped.
Note that shell=True is required only in certain edge cases, and should not be used in your scenario. From the documentation:
On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.
After spending a fair amount of time on this.
I think that running powershell commands from python may not make sense to a lot of people, especially people who work exclusively in windows environments. There are numerous clear advantages to python over powershell however so the ability to do all your business logic in python and then selectively execute powershell on remote servers is truly a great thing.
I've now been through several improvements of my "winrmcntl" module which I can't share due to company policy unfortunately but here is my advice to anyone who would like to do something similar. The module should take as input an unmodified PS command or scriptblock as you'd run it if you were typing directly in PS on the destination box. A few tricks:
To avoid permission difficulties, ensure the user running your python script and hence the one running powershell.exe via process.Popen is the user that has the correct permissions on the windows box you're invoke-command is pointing at. We use an enterprise scheduler which has windows vms as agents on which the python code lives which takes care of that.
You will sometimes rarely but still get the odd esoteric exception from powershell land, if they're anything like the one in particular I saw the odd time, microsoft scratch their heads at a little and get you to do time consuming application stack tracing. This is not only time consuming but very difficult to get right because it's resource intensive and you don't know when the exception will next occur. In my opinion, it's much better and easier to parse the output of the exception and retry up to x number of times if a certain text appears in those exceptions. I keep a list of strings in my winrmcntl module which currently contains a single string.
If you want to not have to "massage" the powershell commands as they traverse the python -> windows -> powershell -> powershell stack to make them work as expected on destination boxes, the most consistent method I've found is to write your one liners and scriptblocks alike into a ps_buffer.ps1 file which you then feed to powershell on the source box so that every process.popen looks exactly the same but the content of ps_buffer.ps1 changes with each execution.
powershell.exe ps_buffer.ps1
To keep your python code nice and clean, it's great having your list of powershell one liners in a json file or similar as well as pointers to scriptblocks you want to run saved into static files. You load up your json file as an ordered dict and cycle through issuing commands based on what you're doing.
Can't be overstated, as far as is possible try to be on the latest stable version of PS but more than that, it's imperative to be on the same version on client and server.
"scriptblock" and "server" are the values fed to this module or function
import subprocess
from string import Template
scriptblock = 'Get-ChildItem' #or a PS scriptblock as elaborate as you need
server = 'serverx'
psbufferfile = os.path.join(tempdir, 'pscmdbufferfile_{}.ps1'.format(server))
fullshellcmd = 'powershell.exe {}'.format(psbufferfile)
raw_pscommad = 'Invoke-Command -ComputerName $server -ScriptBlock {$scriptblock}'
pscmd_template = Template(raw_pscommand)
pscmd = pscmd_template.substitute(server=server, scriptblock=scriptblock)
try:
with open(psbufferfile, 'w') as psbf:
psbf.writelines(pscmd)
....
try:
process = subprocess.Popen(fullshellcmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output, error = process.communicate()
....
I am using Python 2.7.5, since this version is installed on the machine which I want to run script.
I have created a simple GUI in Tkinter, with button and text input.
Now in one input I provide the ip, or hostname of server, in next step I read the value of input fields and send it to linux bash terminal, and here I have a problem.
Reading the value from input field(works good)
nazwa_ip = self.Input_IP_hostname.get("1.0", 'end-1c')
and next:
os.system('gnome-terminal --window-with-profile=MY_PROFILE -e "ssh -t user_name#nazwa_ip"')
and here is the problem, because it wont change "nazwa_ip" to the read value. That comand send to terminal:
ssh -t user_name#nazwa_ip
but i want to send:
ssh -t user_name#ip_adres_from_input_field
Can somebody help me to resolve the issue?
according to the Python docs, it is recommended that os.system be replaced with the subprocess module .
status = os.system("mycmd" + " myarg")
# becomes
status = subprocess.call("mycmd" + " myarg", shell=True)
String formatting will work here:
os.system('gnome-terminal --window-with-profile=MY_PROFILE -e "ssh -t user_name#%s"' % nazwa_ip)
Using the subprocess method might be better to do this.
import subprocess
nazwa_ip = self.Input_IP_hostname.get("1.0", 'end-1c')
ssh_param = "ssh -t user_name#{}".format(nazwa_ip)
subprocess.call(['gnome-terminal', '--window-with-profile=MY_PROFILE', '-e', ssh_param])
Whilst running a subprocess is easy, starting one in a graphical terminal that behaves exactly like one the user launched is a little tricker. You could use my program interminal (link), which basically does what Stephen Rauch's answer does with gnome-terminal, but via a shell so that user environment variables and aliases etc are all available, which could be useful on the offchance that they affect how ssh runs.
You would use it from Python like this:
import subprocess
subprocess.Popen(['interminal', 'ssh', '-t', 'username#{}'.format(ip_address)])
I am using Python script to invoke a Java virtual machine. The following command works:
subprocess.call(["./rvm"], shell=False) # works
subprocess.call(["./rvm xyz"], shell=True) # works
But,
subprocess.call(["./rvm xyz"], shell=False) # not working
does not work. Python documentation advices to avoid shell=True.
You need to split the commands into separate strings:
subprocess.call(["./rvm", "xyz"], shell=False)
A string will work when shell=True but you need a list of args when shell=False
The shlex module is useful more so for more complicated commands and dealing with input but good to learn about:
import shlex
cmd = "python foo.py"
subprocess.call(shlex.split(cmd), shell=False)
shlex tut
If you want to use shell=True, this is legit, otherwise it would have been removed from the standard library. The documentation doesn't say to avoid it, it says:
Executing shell commands that incorporate unsanitized input from an untrusted source makes a program vulnerable to shell injection, a serious security flaw which can result in arbitrary command execution. For this reason, the use of shell=True is strongly discouraged in cases where the command string is constructed from external input.
But in your case you are not constructing the command from user input, your command is constant, so your code doesn't present the shell injection issue. You are in control of what the shell will execute, and if your code is not malicious per se, you are safe.
Example of shell injection
To explain why the shell injection is so bad, this is the example used in the documentation:
>>> from subprocess import call
>>> filename = input("What file would you like to display?\n")
What file would you like to display?
non_existent; rm -rf / #
>>> call("cat " + filename, shell=True) # Uh-oh. This will end badly...
Edit
With the additional information you have provided editing the question, stick to Padraic's answer. You should use shell=True only when necessary.
In addition to Enrico.bacis' answer, there are two ways to call programs. With shell=True, give it a full command string. With shell=False, give it a list.
If you do shell tricks like *.jpg or 2> /dev/null, use shell=True; but in general I suggest shell=False -- it's more durable as Enrico said.
source
import subprocess
subprocess.check_call(['/bin/echo', 'beer'], shell=False)
subprocess.check_call('/bin/echo beer', shell=True)
output
beer
beer
Instead of using the filename directory, add the word python in front of it, provided that you've added python path to your environmental variables. If you're not sure, you can always rerun the python installer, once again, provided that you have a new version of python.
Here's what I mean:
import subprocess
subprocess.Popen('python "C:/Path/To/File/Here.py"')
I am trying to execute a command inside a Python script:
import subprocess
output_process =
subprocess.Popen("javac -cp C:\Users\MyUsername\Desktop\htmlcleaner-2.2.jar Scrapping_lastfm.java",
shell=True, stdout=subprocess.PIPE)
But I am getting an error package org.htmlcleaner does not exist.
If I run the javac command independently, it executes fine..
My current working directry is C:\Users\MyUsername.
The error is not raised by python but by the java subprocess. Most likely the java machine is not finding some libraries, and that refines the problem to a PATH configuration problem, most likely
the variable CLASSPATH has not been set in the environment. to solve :
import shlex
JAVA_COMMAND=r"javac -cp C:\\Users\\MyUsername\\Desktop\\htmlcleaner-2.2.jar Scrapping_lastfm.java"
cmdline = shlex.split(JAVA_COMMAND)
output_process = subprocess.Popen(cmdline,shell=True, stdout=subprocess.PIPE, env={'CLASSPATH':'/path/to/java/packages'})
Try
output_process = subprocess.Popen(["javac", "-cp",
"C:\Users\MyUsername\Desktop\htmlcleaner-2.2.jar", "Scrapping_lastfm.java"],
shell=True, stdout=subprocess.PIPE, env={'ENVIRONMENTAL': '/variables/here'})
with whatever java-related environmental variables you have when you run javac normally as items in the env dictionary. asgs suggests you need CLASSPATH.
You don't have to split the command up into a list I just did that to make it easier to see the whole thing.
Be aware, that you have to escape the backslash (\) in the string. Your example is fine, however if your username is not actually MyUsername but maybe „nerd“ or any other string forming a valid escape-sequence, the command will fail.
Also make sure that you don't have spaces in the filename (or use the split syntax in the other example).
So you might want to do:
output_process = subprocess.Popen(["javac", "-cp",
"C:\\Users\\MyUsername\\Desktop\\htmlcleaner-2.2.jar", "Scrapping_lastfm.java"],
shell=True, stdout=subprocess.PIPE)