We Keep Coding

How to run perl script with multiple args from python script

I made a python code that must sequentially execute a series of perl commands on the PC shell, the problem is that I did not realize that to send these scripts I have to add parameters (i have n_params), advice?
example command to send
perl [file_name.pl] [params]
To run these commands on the windows CMD I am using os and subprocess
python code example
# command = perl [file_name.pl] [params]
# path = location/of/where/the/pl/file/is/saved
perl_script = subprocess.Popen(["C:\\Perl64\\bin\\perl.exe",path + command], stdout=sys.stdout)
perl_script.communicate()
But running the script like this, the code gets me wrong because it says it can't find the filename in the specific directory

This argument to Popen()
["C:\\Perl64\\bin\\perl.exe", path + command]
does not look correct since you wrote that command is perl [file_name.pl] [params]. Instead try:
p = subprocess.Popen(["C:\\Perl64\\bin\\perl.exe", path+"file_name.pl", "param1", "param2", ...])

Using WSL bash from within python

I'm using windows 10 and while working on a new project, I need to interact with WSL(Ubuntu on windows) bash from within python (windows python interpreter).
I tried using subprocess python library to execute commands.. what I did looks like this:
import subprocess
print(subprocess.check_call(['cmd','ubuntu1804', 'BashCmdHere(eg: ls)']))#not working
print(subprocess.check_output("ubuntu1804", shell=True).decode())#also not working
The expected behavior is to execute ubuntu1804 command which starts a wsl linux bash on which I want to execute my 'BashCmdHere' and retrieve its results to python but it just freezes. What am I doing wrong ? or how to do this ?
Thank you so much

Found 2 ways to achieve this:
A correct version of my code looks like this
#e.g: To execute "ls -l"
import subprocess
print(subprocess.check_call(['wsl', 'ls','-l','MaybeOtherParamHere']))
I should have used wsl to invoke linux shell from windows aka bash then my command and parameters in separated arguments for the subprocess command.
The other way which I think is cleaner but may be heavier is using PowerShell Scripts:
#script.ps1
param([String]$folderpath, [String]$otherparam)
Write-Output $folderpath
Write-Output $otherparam
wsl ls -l $folderpath $otherparam
Then to execute it in python and get the results:
import subprocess
def callps1():
powerShellPath = r'C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe'
powerShellCmd = "./script.ps1"
#call script with argument '/mnt/c/Users/aaa/'
p = subprocess.Popen([powerShellPath, '-ExecutionPolicy', 'Unrestricted', powerShellCmd, '/mnt/c/Users/aaa/', 'SecondArgValue']
, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output, error = p.communicate()
rc = p.returncode
print("Return code given to Python script is: " + str(rc))
print("\n\nstdout:\n\n" + str(output))
print("\n\nstderr: " + str(error))
# Test
callps1()
Thank you for helping out

What about:
print(subprocess.check_call(['ubuntu1804', 'run', 'BashCmdHere(eg: ls)'])) #also try without "run" or change ubuntu1804 to wsl
Or
print(subprocess.check_call(['cmd', '/c', 'ubuntu1804', 'run', 'BashCmdHere(eg: ls)']))#also try without "run" or change "ubuntu1804" to "wsl"
# I think you need to play with quotes here to produce: cmd /c 'ubuntu1804 run BashCmdHere(eg: ls)'
First, try to call your command from cmd.exe to see the right format and then translate it to Python.

os.system('bash')
I figured this out by accident.

What's the best way to execute PowerShell scripts from Python

All the previous posts on this topic deal with specific challenges for their use case. I thought it would be useful to have a post only dealing with the cleanest way to run PowerShell scripts from Python and ask if anyone has an better solution than what I found.
What seems to be the generally accepted solution to get around PowerShell trying to interpret different control characters in your command differently to what's intended is to feed your Powershell command in using a file:
ps = 'powershell.exe -noprofile'
pscommand = 'Invoke-Command -ComputerName serverx -ScriptBlock {cmd.exe \
/c "dir /b C:\}'
psfile = open(pscmdfile.ps1, 'w')
psfile.write(pscommand)
psfile.close()
full_command_string = ps + ' pscmdfile.ps1'
process = subprocess.Popen(full_command_string , shell=True, \
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
When your python code needs to change the parameters for the Powershell command each time you invoke it you end up writing and deleting a lot of temporary files for subprocess.Popen to run. It works perfectly but it's unnecessary and not very clean. It's really nice to be able to tidy up and wanted to get suggestions on any improvements I could make to the solution I found.
Instead of writing a file to disk containing the PS command create a virtual file using the io module. Assuming that the "date" and "server" strings are being fed in as part of a loop or function that contains this code, not including the imports of course:
import subprocess
import io
from string import Template
raw_shellcmd = 'powershell.exe -noprofile '
--start of loop with server and date variables populated--
raw_pslistcmd = r'Invoke-Command -ComputerName $server -ScriptBlock ' \
r'{cmd.exe /c "dir /b C:\folder\$date"}'
pslistcmd_template = Template(raw_pslistcmd)
pslistcmd = pslistcmd_template.substitute(server=server, date=date)
virtualfilepslistcommand = io.BytesIO(pslistcmd)
shellcmd = raw_shellcmd + virtualfilepslistcommand.read()
process = subprocess.Popen(shellcmd, shell=True, stdout=subprocess.PIPE, \
stderr=subprocess.PIPE)
--end of loop--

Arguably the best approach is to use powershell.exe -Command rather than writing the PowerShell command to a file:
pscommand = 'Invoke-Command ...'
process = subprocess.Popen(['powershell.exe', '-NoProfile', '-Command', '"&{' + pscommand + '}"'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Make sure double quotes in the pscommand string are properly escaped.
Note that shell=True is required only in certain edge cases, and should not be used in your scenario. From the documentation:
On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.

After spending a fair amount of time on this.
I think that running powershell commands from python may not make sense to a lot of people, especially people who work exclusively in windows environments. There are numerous clear advantages to python over powershell however so the ability to do all your business logic in python and then selectively execute powershell on remote servers is truly a great thing.
I've now been through several improvements of my "winrmcntl" module which I can't share due to company policy unfortunately but here is my advice to anyone who would like to do something similar. The module should take as input an unmodified PS command or scriptblock as you'd run it if you were typing directly in PS on the destination box. A few tricks:
To avoid permission difficulties, ensure the user running your python script and hence the one running powershell.exe via process.Popen is the user that has the correct permissions on the windows box you're invoke-command is pointing at. We use an enterprise scheduler which has windows vms as agents on which the python code lives which takes care of that.
You will sometimes rarely but still get the odd esoteric exception from powershell land, if they're anything like the one in particular I saw the odd time, microsoft scratch their heads at a little and get you to do time consuming application stack tracing. This is not only time consuming but very difficult to get right because it's resource intensive and you don't know when the exception will next occur. In my opinion, it's much better and easier to parse the output of the exception and retry up to x number of times if a certain text appears in those exceptions. I keep a list of strings in my winrmcntl module which currently contains a single string.
If you want to not have to "massage" the powershell commands as they traverse the python -> windows -> powershell -> powershell stack to make them work as expected on destination boxes, the most consistent method I've found is to write your one liners and scriptblocks alike into a ps_buffer.ps1 file which you then feed to powershell on the source box so that every process.popen looks exactly the same but the content of ps_buffer.ps1 changes with each execution.
powershell.exe ps_buffer.ps1
To keep your python code nice and clean, it's great having your list of powershell one liners in a json file or similar as well as pointers to scriptblocks you want to run saved into static files. You load up your json file as an ordered dict and cycle through issuing commands based on what you're doing.
Can't be overstated, as far as is possible try to be on the latest stable version of PS but more than that, it's imperative to be on the same version on client and server.
"scriptblock" and "server" are the values fed to this module or function
import subprocess
from string import Template
scriptblock = 'Get-ChildItem' #or a PS scriptblock as elaborate as you need
server = 'serverx'
psbufferfile = os.path.join(tempdir, 'pscmdbufferfile_{}.ps1'.format(server))
fullshellcmd = 'powershell.exe {}'.format(psbufferfile)
raw_pscommad = 'Invoke-Command -ComputerName $server -ScriptBlock {$scriptblock}'
pscmd_template = Template(raw_pscommand)
pscmd = pscmd_template.substitute(server=server, scriptblock=scriptblock)
try:
with open(psbufferfile, 'w') as psbf:
psbf.writelines(pscmd)
....
try:
process = subprocess.Popen(fullshellcmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output, error = process.communicate()
....

Execute multiple shell/command prompt commands in same instance

I am looking for a way to execute multiple commands in the same shell instance using a separate function for each, something that I can define when the shell process opens/closes and can pass commands to.
so far all the answers I have found have only been in a single function
ie:
#!/usr/bin/env python
from subprocess import check_call
check_call(r"""set -e
ls -l
<some command> # This will change the present working directory
launchMyApp""", shell=True)
I need the same effect but with each command in a different function like
shell.open()
shell.exec("dir")
shell.exec("cd C:/Users/" + User + "/Desktop)
shell.close()
if you are wondering whyyy it has to be separate the command to run is coming from user input. yes I realize that is a security risk, but security isn't a problem in this case, as its purely an educational venture and not going to be used for anything

you could use subprocess.check_call(cmds_str, shell=True) in conjunction with multiple commands in the same line: How to run two commands in one line in Windows CMD?
You could build each command individually and add them to a list, and then use ' & '.join(cmd_list) to get cmds_str.

I don't use Windows but it works on Linux.
You can try pexpect with cmd.exe
import pexpect
child = pexpect.spawn("cmd.exe")
child.expect_exact("> ")
#print(child.before.decode('utf-8'))
print(child.before)
child.sendline("dir")
child.expect_exact("> ")
print(child.before)
child.sendline("cd C:/Users/" + User + "/Desktop")
child.expect_exact("> ")
print(child.before)
It runs cmd.exe, sends command in child.sendline() and looks for prompt child.expect_exact("> ") to get all text generated by command child.before.

FreeBSD rc script with output to file

I have this script below where I start a python program.
The python program outputs to stdout/terminal. But I want the program to be started via rc script silently.
I can start the and stop the program perfectly. And it also creates the log file, but dosent fill anything to it. I tried a lot of different ways. Even with using daemon as starter.
Where is my problem?
#!/bin/sh
# REQUIRE: DAEMON
# KEYWORD: shutdown
. /etc/rc.subr
location="/rpiVent"
name="rpiVentService"
rcvar=`set_rcvar`
command="$location/$name"
#command_args="> $location/$name.log" // Removed
command_interpreter="/usr/bin/python"
load_rc_config $name
run_rc_command "$1"

Piping with > is a feature of the shell and not an actual part of the command line. When commands are programmatically involved, the arguments given them cannot contain shell directives (unless the parent process has special support for shell, like with Python subprocess.Popen(shell=True) (doc).
What in this case you can do is that you can wrap your command (/rpiVent/rpiVentService) to a shell script then invoke this shell script in FreeBSD rc script::
Creat /rpiVent/run.sh:
#!/bin/sh
/rpiVent/rpiVentservice > /rpiVent/rpiVentService.log
and then use this is a command (no args needed).

The correct way to do this is probably by "overriding" the start command using start_cmd variable, like this:
#!/bin/sh
# REQUIRE: DAEMON
# KEYWORD: shutdown
. /etc/rc.subr
location="/rpiVent"
name="rpiVentService"
rcvar=`set_rcvar`
load_rc_config $name
command="$location/$name"
command_interpreter="/usr/bin/python"
start_cmd=rpivent_cmd
rpivent_cmd()
{
$command_interpreter $command >$location/$name.log
}
run_rc_command "$1"