We Keep Coding

How to run a python program from powershell based on python hashbang? - python

I have several python3 scripts using the hashbang for identifying it as a Python3 script. The scripts can not be identified as python based on the file extension, as they don't have any (just like in any other *nix system).
I've already looked at related issues, but they don't address this particular need, and seem to rely upon the *.py extension, for recognition.
Set up Python on Windows to not type "python" in cmd
How to make python scripts executable on Windows?
So my script is named: myscript, and the first line of the file is:
#!/usr/bin/env python3
...
How can I make Windows powershell recognize this and run it with the python interpreter located in C:\Python3.7?
UPDATE-1
To clarify, I want to run it from powershell CLI and not by clicking on it.
In addition, I just discovered (to my horror) that when you use pip install with a native Windows Python3, the first line hashbang is autotragically replaced with:
#!c:\python37\python.exe
Ouch!
UPDATE-2
Thanks to #eryksun's comments, I managed to get a PowerShell script to do some basic checking for me. However, it will need to be fixed to support more than Python.
test4script.ps1:
Param([parameter(Mandatory=$true, HelpMessage="Need a valid filename")] $fileName)
$firstLine = Get-Content -Path $fileName -TotalCount 1
$SHEBANG="^#!"
$shes=#("python3","python2","python","bash","sh","perl","pwsh")
If ($firstLine -match $SHEBANG) {
Write-Host "DEBUG: checking for common shebangs" -ForegroundColor Green
foreach ($i in $shes) {
If ($firstLine -match $i) {
Write-Host "DEBUG: found shebang for: $i" -ForegroundColor Green
C:\python37\python.exe $fileName
break
}
}
} else {
Write-Host "File is not a known script. No shebang fund in file!" -ForegroundColor Red
return
}
Write-Host "DEBUG: Done" -ForegroundColor Green
The result is:
$ Get-Content -Path nonscript -TotalCount 3
#This aint right
echo "hello"
$ Get-Content -Path pip-describe -TotalCount 3
#!c:\python37\python.exe
# pip-describe - Show full text package description from PyPI
# -*- coding: utf-8 -*-
$ .\test4script.ps1 nonscript
File is not a known script. No shebang fund in file!
$ .\test4script.ps1 pip-describe
DEBUG: checking for common shebangs
DEBUG: found shebang for: python3
Usage: pip-describe <package-name>
This will return the full-text package description (usually the README)
as found on PyPI, for any given <package-name>.
...
Now we should be able to associate this script with ., using CMD.exe with:
cmd /c assoc .=unknown
cmd /c "ftype unknown=""C:\mybin\test4script.ps1"" %1"
But it would have been nicer to do this independently with PowerShell and not having to hop through CMD.

eryksun provides great pointers in comments on the question, and your edits based on them show the way toward installing a generic, machine-wide shebang-line-aware launcher for extension-less scripts made executable by adding . to $env:PATHEXT.
Things to note about this approach:
PowerShell currently (as of PowerShell Core 6.2.0) invariably executes extension-less files in a new console window, which makes this configuration useless in PowerShell - it does work as expected from cmd.exe, though.
PowerShell's behavior should be considered a bug and was reported in this GitHub issue.
The mechanism presents a potential security risk, because any plain-text file without an extension that has a shebang line effectively becomes executable, potentially bypassing security features that focus on files that have extensions known to be executable.
Implementing a file-type definition's default operation via a [PowerShell] script invariably requires creating a child process with the script file's interpreter, which in the case at hand means invoking powershell.exe with its -File parameter. powershell.exe's startup performance cost is nontrivial, which delays execution.
If you do want to implement this generic mechanism after all, see the Install-ShebangSupport.ps1 script at the bottom.
Given the above, here's a more lightweight, Python-specific approach based on automatically creating individual *.ps1 wrapper scripts for extension-less shebang-line Python scripts:
This takes advantage of the fact that PowerShell allows execution of its own *.ps1 script files by filename only.
Limitations:
You need to run the wrapper-script-generation script (printed below) at least once, and every time you add new extension-less Python scripts.
Conceivably, a file-system watcher could be used to trigger the generation script, but setting that up is a nontrivial effort.
On the plus side, the wrapper scripts execute more quickly than a generic file-type-based solution, because no extra PowerShell instance (child process) is involved.
Run the following script from the directory in which the extension-less Python scripts are located[1]:
Get-ChildItem -File | Where-Object Extension -eq '' | % {
if ((Get-Content -LiteralPath $_.fullname -First 1) -match '^#!.*\bpython') {
#'
py.exe ($PSCommandPath -replace '\.ps1$') $Args; exit $LASTEXITCODE
'# > ($_.FullName + '.ps1')
}
}
For every extension-less Python script somescript, a companion somescript.ps1 file is created that passes somescript to the Python launcher py.exe, along with any command-line arguments; exit $LASTEXTICODE ensures that py.exe's exit code is passed through.
As eryksun notes, py.exe should be capable of interpreting the shebang line to invoke the appropriate Python executable.
If you don't want to clutter your system with wrapper files, auto-generate functions as an alternative, but note that you'll have to load them into every session to be available, typically via your $PROFILE file:
Get-ChildItem -File | Where-Object Extension -eq '' | % {
if ((Get-Content -LiteralPath $_.FullName -First 1) -match '^#!.*\bpython') {
Invoke-Expression #"
Function global:$($_.Name) {
py.exe "$($_.FullName)" `$Args
}
"#
}
}
Note:
This will make the current directory's extension-less Python scripts available as if they were located in a directory listed in $env:PATH - whether or not the current directory is listed there.
Each target Python script is hard-coded into a function of the same name and will invariably target that script.
By contrast, the *.ps1 wrapper script-file approach permits targeted invocation in a given directory, with something like .\foo.
This particular use of Invoke-Expression is safe - to define the functions based on expandable strings - but Invoke-Expression should generally be avoided.
Script Install-ShebangSupport.ps1 for installing generic support for direct execution of extension-less shebang-line-based scripts on Windows:
The script supports installation at the current-user level (by default or with -Scope CurrentUser) or at the all-users level (with -Scope AllUsers, requires running as admin).
Assuming presence in the current dir, run Get-Help .\Install-ShebangSupport for basic help.
Invoking the script without arguments prints a confirmation prompt with details about the required modifications to the system; Ctrl-C can be used to abort without installation; passing -Force performs installation without prompting for confirmation.
To uninstall later, pass -Uninstall; note that you must match the (implied) -Scope value used during installation.
Implementation note: Defining the no-extension file type via the cmd.exe-internal commands assoc and ftype invariably takes effect for all users, because the definitions are stored in the registry in HKEY_LOCAL_MACHINE\Software\Classes; also, the invocation therefore invariably requires elevation (administrative privileges).
However, it is possible to create user-level definitions by direct manipulation of the registry, which is what this script uses, also for the machine-level definitions.
Note: Syntax highlighting is broken in the code below, but it does work.
<#
.SYNOPSIS
Support for direct execution of extension-less script files with shebang lines
on Windows.
.DESCRIPTION
For details, invoke this script without arguments: the confirmation prompt
will show the required modifications to your system. Submit "N" to opt out
of the installation.
Note that extension-less files that do not have a shebang line will open in
the default text editor.
.PARAMETER Scope
Whether to install support for the current user only (the default) or
for all users (requires invocation as admin).
.PARAMETER Uninstall
Uninstalls previously installed support.
Note that the (implied) -Scope value must match the one that was used during
installation.
.PARAMETER Force
Bypasses the confirmation prompt that is shown by default.
.EXAMPLE
Install-ShebangSupport
Installation for the current user that requires answering a confirmation prompt.
.EXAMPLE
Install-ShebangSupport -Scope AllUsers -Force
Installation for all users without confirmation prompt. Requires invocation
as admin.
.EXAMPLE
Install-ShebangSupport -Uninstall
Uninstallation for the current user with confirmation prompt.
#>
[CmdletBinding(PositionalBinding=$false)]
param(
[ValidateSet('CurrentUser', 'AllUsers')]
[string] $Scope = 'CurrentUser'
,
[switch] $Force
,
[switch] $Uninstall
)
$ErrorActionPreference = 'Stop'; Set-StrictMode -Version 1
if ($env:OS -ne 'Windows_NT') { Throw ("This script can only run on Windows.")}
# ---------------------- BEGIN: Internal helper functions
# === INSTALL
function install {
Write-Verbose ('Installing shebang-script support for {0}:' -f ('the current user', 'ALL users')[$forAllUsers])
# NOTE:
# * assoc and ftype only ever operate on HKEY_LOCAL_MACHINE\Software\Classes, not HKEY_CURRENT_USER\Software\Classes - both on reading and writing.
# * *HKEY_CURRENT_USER*-level definitions DO work, but *neither assoc nor ftype report them or can update them*.
# Therefore, we perform direct registry manipulation below.
Write-Verbose 'Creating file type for extension-less file names via the registry...'
# Map the "extension-less extension", "." to the name of the new file type to be created below.
# Caveat: Sadly, New-Item -Force blindly recreates the registry key if it already exists, discarding
# all existing content in the process.
$key = New-Item -Force -Path $regkeyExtensionToFileType
$null = New-ItemProperty -LiteralPath $key.PSPath -Name '(default)' -Value $fileTypeName
# Define the new file type:
$key = New-Item -Force -Path "$regkeyFileType\Shell\Open\Command"
$null = New-ItemProperty -LiteralPath $key.PSPath -Name '(default)' -Value ('powershell.exe -noprofile -file "{0}" "%1" %*' -f $helperScriptFullName)
# Get the current $env:PATHEXT definition from the registry.
$currPathExt = [Environment]::GetEnvironmentVariable('PATHEXT', ('User', 'Machine')[$forAllUsers])
if (-not $forAllUsers -and -not $currPathExt) {
Write-Verbose "Creating a static user-level copy of the machine-level PATHEXT environment variable..."
$currPathExt = [Environment]::GetEnvironmentVariable('PATHEXT', 'Machine')
}
# Add "." as an executable extension to $env:PATHEXT so as to support
# direct execution of extension-less files.
if ($currPathExt -split ';' -notcontains '.') {
Write-Verbose "Appending '.' to PATHEXT..."
[Environment]::SetEnvironmentVariable('PATHEXT', $currPathExt + ';.', ('User', 'Machine')[$forAllUsers])
# Also define it for the running session
$env:PATHEXT += ';.'
} else {
Write-Verbose "'.' is already contained in PATHEXT."
}
# The following here-string is the source code for the
# $helperScriptFullName script to create.
# To debug and/or modify it:
# * Edit and/or debug $helperScriptFullName
# * After applying fixes / enhancements, replace the here-string
# below with the updated source code.
#'
# When invoked by direct execution of a script file via the file-type definition, the arguments are:
# * The full path of the script file being invoked.
# * Arguments passed to the script file on invocation, if any.
# CAVEAT: PowerShell's own parsing of command-line arguments into $args
# breaks unquoted tokens such as >> -true:blue << and >> -true.blue << into *2* arguments
# ('-true:', 'blue' and '-true', '.blue', respectively).
# The only way to avoid that is to pass the argument *quoted*: '-true:blue' and '-true.blue'
# See https://github.com/PowerShell/PowerShell/issues/6360
# Parse the arguments into the script
param(
[Parameter(Mandatory=$true)] [string] $LiteralPath,
[Parameter(ValueFromRemainingArguments=$true)] [array] $passThruArgs
)
$ErrorActionPreference = 'Stop'; Set-StrictMode -Version 1
# Note: When invoked via the file-type definition, $LiteralPath is guaranteed to be a full path.
# To also support interactive use of this script (for debugging), we resolve the script
# argument to a full path.
# Note that if you pass just a script filename (<script>), it'll be interpreted relative
# to the current directory rather than based on an $env:PATH search; to do the latter,
# pass (Get-Command <script>).Source
if ($LiteralPath -notmatch '^(?:[a-z]:)?[\\/]') { $LiteralPath = Convert-Path -LiteralPath $LiteralPath }
# Check the script's first line for a shebang.
$shebangLine = ''
switch -Regex -File $LiteralPath {
'^#!\s*(.*)\s*$' { # Matches a shebang line.
# Save the shebang line and its embedded command.
$shebangLine = $_
$cmdLine = $Matches[1]
Write-Verbose "Shebang line found in '$LiteralPath': $shebangLine"
break # We're done now that we have the shebang line.
}
default { # no shebang line found -> open with default text editor
# Note: We cannot use Invoke-Item or Start-Process, as that would
# reinvoke this handler, resulting in an infinite loop.
# The best we can do is to open the file in the default text editor.
Write-Verbose "No shebang line, opening with default text editor: $LiteralPath"
# Obtain the command line for the default text editor directly from the registry
# at HKEY_CLASSES_ROOT\txtfile\shell\Open\command rather than via `cmd /c ftype`,
# because assoc and ftype only ever report on and update the *machine-level* definitions at
# HKEY_LOCAL_MACHINE\Software\Classes
$cmdLine = [environment]::ExpandEnvironmentVariables((((Get-ItemProperty -EA Ignore registry::HKEY_CLASSES_ROOT\txtfile\shell\Open\command).'(default)') -split '=')[-1])
if (-not $cmdLine) { $cmdLine = 'NOTEPAD.EXE %1' } # Fall back to Notepad.
break # We're done now that we know this file doesn't have a shebang line.
}
}
# Parse the shebang line's embedded command line or the default-text-editor's command line into arguments.
# Note: We use Invoke-Expression and Write-Output so as to support *quoted*
# arguments as well - though presumably rare in practice.
# If supporting quoted tokens isn't necessary, the next line can be replaced
# with a strictly-by-whitespace splitting command:
# $cmdArgs = -split $cmdLine
[array] $cmdArgs = (Invoke-Expression "Write-Output -- $($cmdLine -replace '\$', "`0")") -replace "`0", '$'
if ($shebangLine) {
# Extract the target executable name or path.
# If the first argument is '/usr/bin/env', we skip it, as env (on Unix-like platforms) is merely used
# to locate the true target executable in the Path.
$exeTokenIndex = 0 + ($cmdArgs[0] -eq '/usr/bin/env')
$exeNameOrPath = $cmdArgs[$exeTokenIndex]
$exeFullPath = ''
# Note: We do NOT pass any remaining arguments from the shebang line through.
# (Such arguments are rare anyway.)
# The rationale is that an interpreter that understands shebang lines will
# also respect such arguments when reading the file - this is true of at
# least py.exe, the Python launcher, and ruby.exe
# Python is a special case: the Python launcher, py.exe, is itself
# capable of interpreting shebang lines, so we defer to it.
if ($exeNameOrPath -match '\bpython\d?') {
# Ensure that 'py.exe' is available; if not, we fall back to the same approach
# as for all other executables.
$exeFullPath = (Get-Command -CommandType Application py.exe -ErrorAction Ignore).Source
}
if (-not $exeFullPath) {
# Try the executable spec. as-is first, should it actually contain a *Windows* path name.
$exeFullPath = (Get-Command -CommandType Application $exeNameOrPath -ErrorAction Ignore).Source
if (-not $exeFullPath) {
# If not found, assume it is a Unix path that doesn't apply, and try to locate the hopefully
# appropriate executable by its filename only, in the Path.
$exeFullPath = (Get-Command -CommandType Application (Split-Path -Leaf -LiteralPath $exeNameOrPath) -ErrorAction Ignore).Source
}
}
# Abort, if we can't find a suitable executable.
if (-not $exeFullPath) { Throw "Could not find a suitable executable to run '$LiteralPath' based on its shebang line: $shebangLine" }
# Synthesize the complete list of arguments to pass to the target exectuable.
$passThruArgs = , $LiteralPath + $passThruArgs
} else { # NON-shebang-line files: invocation of default text editor
$exeFullPath, [array] $editorArgs = $cmdArgs -replace '%1', ($LiteralPath -replace '\$', '$$')
# Synthesize the complete list of arguments to pass to the target exectuable.
# Replace the '%1' placeholder with the script's path.
# Note that we don't really expect additional arguments to have been passed in this scenario,
# and such arguments may be interpreted as additional file arguments by the editor.
$passThruArgs = ($editorArgs -replace '"?%1"?', ($LiteralPath -replace '\$', '$$$$')) + $passThruArgs
# If the editor is a GUI application, $LASTEXITCODE won't be set by PowerShell.
# We set it to 0 here, as it has no value by default, and referencing it below with exit
# would cause an error due to Set-StrictMode -Version 1.
$LASTEXITCODE = 0
}
Write-Verbose "Executing: $exeFullPath $passThruArgs"
# Invoke the target executable with all arguments.
# Important:
# * We need to manually \-escape embeded " chars. in arguments
# because PowerShell, regrettably, doesn't do that automatically.
# However, even that may fail in edge cases in Windows PowerShell (fixed in PS Core),
# namely when an unbalanced " char. is part of the first word - see https://stackoverflow.com/a/55604316/45375
& $exeFullPath ($passThruArgs -replace '"', '\"')
# Pass the target executable's exit code through.
# (In the case of invoking the default editor for non-shebang-line files, it
# won't have been set, if the editor is a GUI application.)
exit $LASTEXITCODE
'# |
Set-Content -Encoding Utf8 -LiteralPath $helperScriptFullName
}
# === UNINSTALL
function uninstall {
Write-Verbose ('Uninstalling shebang-script support for {0}:' -f ('the current user', 'ALL users')[$forAllUsers])
Write-Verbose 'Removing file type information from the registry...'
foreach ($regKey in $regkeyExtensionToFileType, $regkeyFileType) {
if (Test-Path -LiteralPath $regKey) {
Remove-Item -Force -Recurse -LiteralPath $regkey
}
}
# Get the current $env:PATHEXT definition from the registry.
$currPathExt = [Environment]::GetEnvironmentVariable('PATHEXT', ('User', 'Machine')[$forAllUsers])
# Remove the "." entry from $env:PATHEXT
$newPathExt = ($currPathExt -split ';' -ne '.') -join ';'
if ($newPathExt -eq $currPathExt) {
Write-Verbose "'.' is not contained in PATHEXT; nothing to do."
} else {
# For user-level uninstallations: as a courtesy, we compare the new PATHEXT value
# to the machine-level one, and, if they're now the same, simply REMOVE the user-level definition.
Write-Verbose "Removing '.' from PATHEXT..."
if (-not $forAllUsers) {
$machineLevelPathExt = [Environment]::GetEnvironmentVariable('PATHEXT', 'Machine')
if ($newPathExt -eq $machineLevelPathExt) { $newPathExt = $null }
Write-Verbose "User-level PATHEXT no longer needed, removing..."
}
[Environment]::SetEnvironmentVariable('PATHEXT', $newPathExt, ('User', 'Machine')[$forAllUsers])
# Also update for the running session
$env:PATHEXT = if ($newPathExt) { $newPathExt } else { $machineLevelPathExt }
}
Write-Verbose "Removing helper PowerShell script..."
if (Test-Path -LiteralPath $helperScriptFullName) {
Remove-Item -Force -LiteralPath $helperScriptFullName
}
}
# ---------------------- END: Internal helper functions
$forAllUsers = $Scope -eq 'AllUsers'
$verb = ('install', 'uninstall')[$Uninstall.IsPresent]
$operation = $verb + 'ation'
# If -Scope AllUsers was passed, ensure that the session is elevated.
$mustElevate = $forAllUsers -and -not ([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole('BUILTIN\Administrators')
if ($mustElevate) {
Throw "In order to $verb for ALL users, you must run this script WITH ELEVATED PRIVILEGES (Run As Administrator)."
}
# --- Define names, registry and file locations.
# The path of the generic shebang runner script that we'll create below.
$helperScriptFullName = Join-Path ($HOME, $env:ALLUSERSPROFILE)[$forAllUsers] 'Invoke-ShebangScript.ps1'
# The name of the file type to create for extension-less files.
$fileTypeName = 'ShebangScript'
# Registry keys that need to be modified.
# "." represents extension-less files
$regkeyExtensionToFileType = 'registry::{0}\SOFTWARE\Classes\.' -f ('HKEY_CURRENT_USER', 'HKEY_LOCAL_MACHINE')[$forAllUsers]
$regkeyFileType = 'registry::{0}\SOFTWARE\Classes\{1}' -f ('HKEY_CURRENT_USER', 'HKEY_LOCAL_MACHINE')[$forAllUsers], $fileTypeName
# ---
# Prompt for confirmation, unless -Force was passsed.
if ($Uninstall) { # UNINSTALL
if (-not $Force -and -not $PSCmdlet.ShouldContinue(#"
You are about to UNINSTALL support for direct execution of extension-less
script files that have shebang lines.
Uninstallation will be performed for $(("the CURRENT USER only`n(invoke as admin with -Scope AllUsers to change that)", 'ALL USERS')[$forAllUsers]).
IMPORTANT: Uninstallation will only be effective if it is performed in the same
(implied) -Scope as the original installation.
The following modifications to your system will be performed:
* "." will be persistently REMOVED from your `$env:PATHEXT variable.
* The following registry keys will be REMOVED:
$($regkeyExtensionToFileType -replace '^registry::')
$($regkeyFileType -replace '^registry::')
* The following helper PowerShell script will be REMOVED:
$helperScriptFullName
Press ENTER to proceed, or Ctrl-C to abort.
"#, "Shebang-Script Direct-Execution Support - Uninstallation")) { # , $true, [ref] $null, [ref] $null)) {
exit 1
}
# Call the uninstallation helper function
uninstall
} else { # INSTALL
if (-not $Force -and -not $PSCmdlet.ShouldContinue(#"
You are about to install support for direct execution of Unix-style scripts
that do not have a filename extension and instead define the interpreter to run
them with via shebangline ("#!/path/to/interpreter").
Support will be installed for $(("the CURRENT USER only`n(invoke as admin with -Scope AllUsers to change that)", 'ALL USERS')[$forAllUsers]).
Once installed, you will be able to run such scripts by direct invocation,
via a helper PowerShell script that analyzes the shebang line and calls the
appropriate interpreter.
CAVEATS:
* ENABLING THIS INVOCATION MECHANISM IS A SECURITY RISK, because any
plain-text file without an extension that has a shebang line
effectively becomes executable, potentially bypassing security features
that focus on files that have extensions known to be executable.
* AS OF POWERSHELL CORE 6.2.0, direct execution of such extension-less files
from PowerShell INVARIABLY RUNS IN A NEW CONSOLE WINDOW, WHICH MAKES USE
FROM POWERSHELL VIRTUALLY USELESS.
However, this is a BUG that should be fixed; see:
https://github.com/PowerShell/PowerShell/issues/7769
The following modifications to your system will be performed:
* "." will be added persistently to your `$env:PATHEXT variable, to enable
direct execution of filenames without extension.
NOTE: If you install with -Scope CurrentUser (the default), a static
user-level copy of the machine-level PATHEXT environment variable is
created, unless already present.
* The following registry locations will be created or replaced to define a
new file type for extension-less filenames:
$($regkeyExtensionToFileType -replace '^registry::')
$($regkeyFileType -replace '^registry::')
* The helper PowerShell script will be created in:
$helperScriptFullName
NOTE: Any existing registry definitions or helper script will be REPLACED.
Press ENTER to proceed, or CTRL-C to abort.
"#, "Shebang-Script Direct-Execution Support - Installation")) {
# !! The prompt defaults to *Yes* (!)
# !! Sadly, if we wanted the prompt to be default to *No*, we'de be forced
# !! to also present pointless 'Yes/No to *All*' prompts, which would be confusing.
# !! See https://github.com/PowerShell/PowerShell/issues/9428
exit 1
}
# Call the installation helper function
install
}
Write-Verbose "Shebang-support ${operation} completed."
if (-not $Force) {
Write-Host "Shebang-support ${operation} completed."
}
exit 0
[1] In Windows PowerShell, you can use Get-ChildItem -File -Filter *. to find extension-less files more conveniently and efficiently, but this feature is broken in PowerShell Core as of v6.2.0 - see this GitHub issue.

Related

In pwsh call the following:
Write-Host '{"drop_attr": "name"}'
Result ok:
{"drop_attr": "name"}
Now do the same via pwsh:
pwsh -Command Write-Host '{"drop_attr": "name"}'
Result is missing quotation marks and square brackets?
drop_attr: name

Update:
PowerShell 7.3.0 mostly fixed the problem, with selective exceptions on Windows, and it seems that in some version after 7.3.1 the fix will require opt-in - see this answer for details.
For cross-version, cross-edition code, the Native module discussed at the bottom may still be of interest.
Unfortunately, PowerShell's handling of passing arguments with embedded " chars. to external programs - which includes PowerShell's own CLI (pwsh) - is fundamentally broken (and always has been), up to at least PowerShell 7.2.x:
You need to manually \-escape " instances embedded in your arguments in order for them to be correctly passed through to external programs (which happens to be PowerShell in this case as well):
# Note: The embedded '' sequences are the normal and expected
# way to escape ' chars. inside a PowerShell '...' string.
# What is *unexpected* is the need to escape " as \"
# even though " can normally be used *as-is* inside a '...' string.
pwsh -Command ' ''{\"drop_attr\": \"name\"}'' '
Note that I'm assuming your intent is to pass a JSON string, hence the inner '' ... '' quoting (escaped single quotes), which ensures that pwsh ultimately sees a single-quoted string ('...'). (No need for an explicit output command; PowerShell implicitly prints command and expression output).
Another way to demonstrate this on Windows is via the standard choice.exe utility, repurposed to simply print its /m (message) argument (followed by verbatim [Y,N]?Y):
# This *should* preserve the ", but doesn't as of v7.2
PS> choice /d Y /t 0 /m '{"drop_attr": "name"}'
{drop_attr: name} [Y,N]?Y # !! " were REMOVED
# Only the extra \-escaping preserves the "
PS> choice /d Y /t 0 /m '{\"drop_attr\": \"name\"}'
{"drop_attr": "name"} [Y,N]?Y # OK
Note that from inside PowerShell, you can avoid the need for \-escaping, if you call pwsh with a script block ({ ... }) - but that only works when calling PowerShell itself, not other external programs:
# NOTE: Works from PowerShell only.
pwsh -Command { '{"drop_attr": "name"}' }
Background info on PowerShell's broken handling of arguments with embedded " in external-program calls, as of PowerShell 7.2.1:
This GitHub docs issue contains background information.
GitHub issue #1995 discusses the problem and the details of the broken behavior as well as manual workarounds are summarized in this comment; the state of the discussion as of PowerShell [Core] 7 seems to be:
A fix is being considered as an experimental feature, which may become an official feature, in v7.3 at the earliest. Whether it will become a regular feature - i.e whether the default behavior will be fixed or whether the fix will require opt-in or even if the feature will become official at all - remains to be seen.
Fixing the default behavior would substantially break backward compatibility; as of this writing, this has never been allowed, but a discussion as to whether to allow breaking changes in the future and how to manage them has begun: see GitHub issue #13129.
See GitHub PR #14692 for the relevant experimental feature, which, however, as of this writing is missing vital accommodations for batch files and msiexec-style executables on Windows - see GitHub issue #15143.
In the meantime, you can use the PSv3+ ie helper function from the Native module (in PSv5+, install with Install-Module Native from the PowerShell Gallery), which internally compensates for all broken behavior and allows passing arguments as expected; e.g.,
ie pwsh -Command ' ''{"drop_attr": "name"}'' ' would then work properly.

Another way. Are you in Windows or Unix?
pwsh -c "[pscustomobject]#{drop_attr='name'} | convertto-json -compress"
{"drop_attr":"name"}

Another way is to use "encoded commands".
> $cmd1 = "Write-Host '{ ""description"": ""Test program"" }'"
> pwsh -encoded ([Convert]::ToBase64String([Text.Encoding]::Unicode.GetBytes($cmd1)))
{ "description": "Test program" }

I want to execute a Python script from AutoIt using ShellExecuteWait(). My Attempt:
$x = ShellExecuteWait("E:/Automation/Python/Scripts/ReadLog.py", '-f "file.log" -k "key" -e "errMsg" ')
MsgBox(0,"x=",String($x))
If #error Then
MsgBox(0,"Error=",String(#error))
EndIf
I can see some process id in $x, and #error also gets set to 0 (means AutoIt executed the script). But my Python script is not producing results (it writes to a txt file when executed independently). Seems the problem is with passing command line arguments like:
ShellExecuteWait("E:/Automation/Python/Scripts/ReadLog.py", '-f "file.log" -k "key" -e "errMsg" ')
How can I pass command line arguments using ShellExecuteWait()? Syntax:
ShellExecuteWait ( "filename" [, "parameters" [, "workingdir" [,"verb" [, showflag]]]] )
Parameters:
filename :- The name of the file to run (EXE, .txt, .lnk, etc).
parameters :- [optional] Any parameters for the program. Blank ("") uses none.
This misses examples for use of parameters. There are no problems with the Python script (it requires 3 command line arguments, strings with options -f, -k and -e).
Related: How to run or execute python file from autoit.

Check the path to your Python binary (e.g. Python.exe, wherever your Python program/binary is located) in your Window's system environment/path.
Execute Python script from AutoIt If path is there then your code must work. In $x you will receive return exit code of the Python script.
Also you can try:
RunWait('full_path\Python.exe ReadLog.py -f "file.log" -k "key" -e "errMsg"', 'full_path_of_working_directory')

AutoIt does not execute external programs/scripts until you pass the working directory (optional parameter to all the execute and run commands). So pass the working directory as a separate parameter and it will work:
RunWait('full_path\Python.exe ReadLog.py -f "file.log" -k "key" -e "errMsg"', 'full_path_of_working_directory')

I have this script below where I start a python program.
The python program outputs to stdout/terminal. But I want the program to be started via rc script silently.
I can start the and stop the program perfectly. And it also creates the log file, but dosent fill anything to it. I tried a lot of different ways. Even with using daemon as starter.
Where is my problem?
#!/bin/sh
# REQUIRE: DAEMON
# KEYWORD: shutdown
. /etc/rc.subr
location="/rpiVent"
name="rpiVentService"
rcvar=`set_rcvar`
command="$location/$name"
#command_args="> $location/$name.log" // Removed
command_interpreter="/usr/bin/python"
load_rc_config $name
run_rc_command "$1"

Piping with > is a feature of the shell and not an actual part of the command line. When commands are programmatically involved, the arguments given them cannot contain shell directives (unless the parent process has special support for shell, like with Python subprocess.Popen(shell=True) (doc).
What in this case you can do is that you can wrap your command (/rpiVent/rpiVentService) to a shell script then invoke this shell script in FreeBSD rc script::
Creat /rpiVent/run.sh:
#!/bin/sh
/rpiVent/rpiVentservice > /rpiVent/rpiVentService.log
and then use this is a command (no args needed).

The correct way to do this is probably by "overriding" the start command using start_cmd variable, like this:
#!/bin/sh
# REQUIRE: DAEMON
# KEYWORD: shutdown
. /etc/rc.subr
location="/rpiVent"
name="rpiVentService"
rcvar=`set_rcvar`
load_rc_config $name
command="$location/$name"
command_interpreter="/usr/bin/python"
start_cmd=rpivent_cmd
rpivent_cmd()
{
$command_interpreter $command >$location/$name.log
}
run_rc_command "$1"

I need to start a python script with bash using nohup passing an arg that aids in defining a constant in a script I import. There are lots of questions about passing args but I haven't found a successful way using nohup.
a simplified version of my bash script:
#!/bin/bash
BUCKET=$1
echo $BUCKET
script='/home/path/to/script/script.py'
echo "starting $script with nohup"
nohup /usr/bin/python $script $BUCKET &
the relevant part of my config script i'm importing:
FLAG = sys.argv[0]
if FLAG == "b1":
AWS_ACCESS_KEY_ID = "key"
BUCKET = "bucket1"
AWS_SECRET_ACCESS_KEY = "secret"
elif FLAG == "b2":
AWS_ACCESS_KEY_ID = "key"
BUCKET = "bucket2"
AWS_SECRET_ACCESS_KEY = "secret"
else:
AWS_ACCESS_KEY_ID = "key"
BUCKET = "bucket3"
AWS_SECRET_ACCESS_KEY = "secret"
the script thats using it:
from config import BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
#do stuff with the values.
Frankly, since I'm passing the args to script.py, I'm not confident that they'll be in scope for the import script. That said, when I take a similar approach without using nohup, it works.

In general, the argument vector for any program starts with the program itself, and then all of its arguments and options. Depending on the language, the program may be sys.argv[0], argv[0], $0, or something else, but it's basically always argument #0.
Each program whose job is to run another program—like nohup, and like the Python interpreter itself—generally drops itself and all of its own options, and gives the target program the rest of the command line.
So, nohup takes a COMMAND and zero or more ARGS. Inside that COMMAND, argv[0] will be COMMAND itself (in this case, '/usr/bin/python'), and argv[1] and later will be the additional arguments ('/home/path/to/script/script.py' and whatever $BUCKET resolves to).
Next, Python takes zero or more options, a script, and zero or more args to that script, and exposes the script and its args as sys.argv. So, in your script, sys.argv[0] will be '/home/path/to/script/script.py', and sys.argv[1] will be whatever $BUCKET resolves to.
And bash works similarly to Python; $1 will be the first argument to the bash wrapper script ($0 will be the script itself), and so on. So, sys.argv[1] in the inner Python script will end up getting the first argument passed to the bash wrapper script.
Importing doesn't affect sys.argv at all. So, in both your config module and your top-level script, if you import sys, sys.argv[1] will hold the $1 passed to the bash wrapper script.
(On some platforms, in some circumstances argv[0] may not have the complete path, or may even be empty. But that isn't relevant here. What you care about is the eventual sys.argv[1], and bash, nohup, and python are all guaranteed to pass that through untouched.)

nohup python3 -u ./train.py --dataset dataset_directory/ --model model_output_directory > output.log &
Here Im executing train.py file with python3, Then -u is used to ignore buffering and show the logs on the go without storing, specifying my dataset_directory with argument style and model_output_directory then Greater than symbol(>)
then the logs is stored in output.log and them atlast and(&) symbol is used
To terminate this process
ps ax | grep train
then note the process_ID
sudo kill -9 Process_ID

I updated my python interpreter, but I think the old one is still called. When I check for the version I get:
$ python -V
Python 3.0.1
But I believe the old interpreter is still being called. When I run the command:
python myProg.py
The script runs properly. But when I invoke it with the command
./myProg.py
I get the error message:
AttributeError: 'str' object has no attribute 'format'
Which apparently is due to the old interpreter being called. How can I fix this? I run Mac OS X 10.5. Has it something to do with the first line:
#!/usr/bin/python
I just started out with python and am not very familiar with interpreted languages, so I am not too sure what is going on.

According to the first line of the script, #!/usr/bin/python, you are calling the Python interpreter at /usr/bin/python (which is most likely the one that ships with Mac OS X). You have to change that path to the path where you installed your Python 3 interpreter (likely /usr/local/bin/python or /opt/local/bin/python); or you can just change that line to read #!/usr/bin/env python, which will call the python listed first in your PATH variable (which seems to be the newer version you installed).

Firstly, the recommended shebang line is:
#!/usr/bin/env python
This will make sure the python interpreter that is invoked when you ./foo.py is the same interpreter that is invoked when you invoke python from the command line.
From your description, I suspect that if you did:
which python
It would not give you /usr/bin/python. It would give you something else, which is where the python 3 interpreter lives. You can either modify your shebang line to the above, or replace the path to the python interpreter with the path returned by which.

Try which python. I will tell you which python interpreter is used in your environment.
If it is not /usr/bin/python like in the script, then your suspicion is confirmed.

It's very possibly what you suspect, that the shebang line is calling the older version. Two things you might want to check:
1) what version is the interpreter at /usr/bin/python:
/usr/bin/python -V
2) where is the python 3 interpreter you installed:
which python
If you get the correct one from the command line, then replace your shebang line with this:
#!/usr/bin/env python
Addendum: You could also replace the older version of python with a symlink to python 3, but beware that any major OS X updates (ie: 10.5.6 to 10.5.7) will likely break this:
sudo mv /usr/bin/python /usr/bin/python25
sudo ln -s /path/to/python/3/python /usr/bin/python

run 'which python' - if this gives a different answer than /usr/bin/python, change #!/usr/bin/python to have that path instead.

It may be a bit odd providing a Perl script to answer a Python question, but it works for Python just as well as it does for Perl. This is a script called 'fixin', meaning 'fix interpreter'. It changes the shebang line to the correct string for your current PATH.
#!/Users/jleffler/perl/v5.10.0/bin/perl
#
# #(#)$Id: fixin.pl,v 1.3 2003/03/11 21:20:08 jleffler Exp $
#
# FIXIN: from Programming Perl
# Usage: fixin [-s] [file ...]
# Configuration
$does_hashbang = 1; # Kernel recognises #!
$verbose = 1; # Verbose by default
# Construct list of directories to search.
#absdirs = reverse grep(m!^/!, split(/:/, $ENV{'PATH'}, 999));
# Process command line arguments
if ($ARGV[0] eq '-s')
{
shift;
$verbose = 0;
}
die "Usage: $0 [-s] [file ...]\n" unless #ARGV || !-t;
#ARGV = '-' unless #ARGV;
# Process each file.
FILE: foreach $filename (#ARGV)
{
open(IN, $filename) || ((warn "Can't process $filename: $!\n"), next);
$_ = <IN>;
next FILE unless /^#!/; # Not a hash/bang file
chop($cmd = $_);
$cmd =~ s/^#! *//;
($cmd, $arg) = split(' ', $cmd, 2);
$cmd =~ s!^.*/!!;
# Now look (in reverse) for interpreter in absolute path
$found = '';
foreach $dir (#absdirs)
{
if (-x "$dir/$cmd")
{
warn "Ignoring $found\n" if $verbose && $found;
$found = "$dir/$cmd";
}
}
# Figure out how to invoke interpreter on this machine
if ($found)
{
warn "Changing $filename to $found\n" if $verbose;
if ($does_hashbang)
{
$_ = "#!$found";
$_ .= ' ' . $arg if $arg ne '';
$_ .= "\n";
}
else
{
$_ = <<EOF;
:
eval 'exec $found $arg -S \$0 \${1+"\$#"}'
if \$running_under_some_shell;
EOF
}
}
else
{
warn "Can't find $cmd in PATH, $filename unchanged\n" if $verbose;
next FILE;
}
# Make new file if necessary
if ($filename eq '-') { select(STDOUT); }
else
{
rename($filename, "$filename.bak") ||
((warn "Can't modify $filename"), next FILE);
open(OUT, ">$filename") ||
die "Can't create new $filename: $!\n";
($def, $ino, $mode) = stat IN;
$mode = 0755 unless $dev;
chmod $mode, $filename;
select(OUT);
}
# Print the new #! line (or the equivalent) and copy the rest of the file.
print;
while (<IN>)
{
print;
}
close IN;
close OUT;
}
The code is derived from a script of the same name in the original Camel Book ('Programming Perl', first edition). This copy has been hacked a bit since then - and should be hacked some more. But I use it routinely -- indeed, I just copied it from one Mac to another, and since I've not installed Perl 5.10.0 on the second, I ran:
$ perl fixin fixin
Changing fixin to /usr/bin/perl
$
Thereby changing from the private install Perl to the standard one.
Exercise for the reader - rewrite the script in Python.