We Keep Coding

Python code to run bash commands in Windows [duplicate]

I'm on Windows using PowerShell and WSL 'Ubuntu 20.04 LTS'. I have no native Linux Distro, and I cant use virtualisation because of nested device reasons.
My purpose is to use a Windows Python script in PowerShell to call WSL to decrypt some avd-snapshots into raw-images. I already tried os.popen, subprocess.Popen/run/call, win32com.client, multiprocessing, etc.
I can boot the WSL shell, but no further commands are getting passed to it. Does somebody know how to get the shell into focus and prepared for more instructions?
Code Example:
from multiprocessing import Process
import win32com.client
import time, os, subprocess
def wsl_shell():
shell = win32com.client.Dispatch("wscript.shell")
shell.SendKeys("Start-Process -FilePath C:\\Programme\\WindowsApps\\CanonicalGroupLimited.Ubuntu20.04onWindows_2004.2021.825.0_x64__79rhkp1fndgsc\\ubuntu2004.exe {ENTER}")
time.sleep(5)
os.popen("ls -l")
if __name__ == '__main__':
ps = Process(target = wsl_shell)
ps.start()

There are a few ways of running WSL scripts/commands from Windows Python, but a SendKeys-based approach is usually the last resort, IMHO, since it's:
Often non-deterministic
Lacks any control logic
Also, avoid the ubuntu2004.exe (or, for other users who find this, the deprecated bash.exe command). The much more capable wsl.exe command is what you are looking for. It has a lot of options for running commands that the <distroname>.exe versions lack.
With that in mind, here are a few simplified examples:
Using os.system
import os
os.system('wsl ~ -e sh -c "ls -l > filelist.txt"')
After running this code in Windows Python, go into your Ubuntu WSL instance and you should find filelist.txt in your home directory.
This works because:
os.system can be used to launch the wsl command
The ~ tells WSL to start in the user's home directory (more deterministic, while being able to avoid specifying each path in this case)
wsl -e sh runs the POSIX shell in WSL (you could also use bash for this)
Passing -c "<command(s)>" to the shell runs those commands in the WSL shell
Given that, you can pretty much run any Linux command(s) from Windows Python. For multiple commands:
Either separate them with a semicolon. E.g.:
os.system('wsl ~ -e sh -c "ls -l > filelist.txt; gzip filelist.txt')
Or better, just put them all in a script in WSL (with a shebang line), set it executable, and run the script via:
wsl -e /path/to/script.sh
That could even by a Linux Python script (assuming the correct shebang line in the script):
wsl -e /path/to/script.py
So if needed, you can even call Linux Python from Windows Python this way.
Using subprocess.run
The os.system syntax is great for "fire and forget" scripts where you don't need to process the results in Python, but often you'll want to capture the output of the WSL/Linux commands for processing in Python.
For that, use subprocess.run:
import subprocess
cp = subprocess.run(["wsl", "~", "-e", "ls", "-l"], capture_output=True)
print(cp.stdout)
As before, the -e argument can be any type of Linux script you want.
Note that subprocess.run also gives you the exit status of the command.

Python, the relationship between the bash/python/subprocess processes (shells)?

When trying to write script with python, I have a fundamental hole of knowledge.
Update: Thanks to the answers I corrected the word shell to process/subprocess
Nomenclature
Starting with a Bash prompt, lets call this BASH_PROCESS
Then within BASH_PROCESS I run python3 foo.py, the python script runs in say PYTHON_SUBPROCESS
Within foo.py is a call to subprocess.run(...), this subprocess command runs in say `SUBPROCESS_SUBPROCESS
Within foo.py is subprocess.run(..., shell=True), this subprocess command runs in say SUBPROCESS_SUBPROCESS=True
Test for if a process/subprocess is equal
Say SUBPROCESS_A starts SUBPROCESS_B. In the below questions, when I say is SUBPROCESS_A == SUBPROCESS_B, what I means is if SUBPROCESS_B sets an env variable, when it runs to completion, will they env variable be set in SUBPROCESS_A? If one runs eval "$(ssh-agent -s)" in SUBPROCESS_B, will SUBPROCESS_A now have an ssh agent too?
Question
Using the above nomenclature and equality tests
Is BASH_PROCESS == PYTHON_SUBPROCESS?
Is PYTHON_SUBPROCESS == SUBPROCESS_SUBPROCESS?
Is PYTHON_SUBPROCESS == SUBPROCESS_SUBPROCESS=True?
If SUBPROCESS_SUBPROCESS=True is not equal to BASH_PROCESS, then how does one alter the executing environment (e.g. eval "$(ssh-agent -s)") so that a python script can set up the env for the calller?

You seem to be confusing several concepts here.
TLDR No, there is no way for a subprocess to change its parent's environment. See also Global environment variables in a shell script
You really don't seem to be asking about "shells".
Instead, these are subprocesses; if you run python foo.py in a shell, the Python process is a subprocess of the shell process. (Many shells let you exec python foo.py which replaces the shell process with a Python process; this process is now a subprocess of whichever process started the shell. On Unix-like systems, ultimately all processes are descendants of process 1, the init process.)
subprocess runs a subprocess, simply. If shell=True then the immediate subprocess of Python is the shell, and the command(s) you run are subprocesses of that shell. The shell will be the default shell (cmd on Windows, /bin/sh on Unix-like systems) though you can explicitly override this with e.g. executable="/bin/bash"
Examples:
subprocess.Popen(['printf', '%s\n', 'foo', 'bar'])
Python is the parent process, printf is a subprocess whose parent is the Python process.
subprocess.Popen(r"printf '%s\n' foo bar", shell=True)
Python is the parent process of /bin/sh, which in turn is the parent process of printf. When printf terminates, so does sh, as it has reached the end of its script.
Perhaps notice that the shell takes care of parsing the command line and splitting it up into the four tokens we ended up explicitly passing directly to Popen in the previous example.
The commands you run have access to shell features like wildcard expansion, pipes, redirection, quoting, variable expansion, background processing, etc.
In this isolated example, none of those are used, so you are basically adding an unnecessary process. (Maybe use shlex.split() if you want to avoid the minor burden of splitting up the command into tokens.) See also Actual meaning of 'shell=True' in subprocess
subprocess.Popen(r"printf '%s\n' foo bar", shell=True, executable="/bin/bash")
Python is the parent process of Bash, which in turn is the parent process of printf. Except for the name of the shell, this is identical to the previous example.
There are situations where you really need the slower and more memory-hungry Bash shell, when the commands you want to execute require features which are available in Bash, but not in the Bourne shell. In general, a better solution is nearly always to run as little code as possible in a subprocess, and instead replace those Bash commands with native Python constructs; but if you know what you are doing (or really don't know what you are doing, but need to get the job done rather than solve the problem properly), the facility can be useful.
(Separately, you should probably avoid bare Popen when you can, as explained in the subprocess documentation.)
Subprocesses inherit the environment of their parent when they are started. On Unix-like systems, there is no way for a process to change its parent's environment (though the parent may participate in making that possible, as in your eval example).
To perhaps accomplish what you may ultimately be asking about, you can set up an environment within Python and then start your other command as a subprocess, perhaps then with an explicit env= keyword argument to point to the environment you want it to use:
import os
...
env = os.environ.copy()
env["PATH"] = "/opt/foo:" + env["PATH"]
del env["PAGER"]
env["secret_cookie"] = "xyzzy"
subprocess.Popen(["otherprogram"], env=env)
or have Python print out values in a form which can safely be passed to eval in the Bourne shell. (Caution: this requires you to understand the perils of eval in general and the target shell's quoting conventions in particular; also, you will perhaps need to support the syntax of more than one shell, unless you are only targeting a very limited audience.)
... Though in many situations, the simplest solution by far is to set up the environment in the shell, then run Python as a subprocess of that shell instance (or exec python if you want to get rid of the shell instance after it has performed its part; see also What are the uses of the exec command in shell scripts?)
Python without an argument starts the Python REPL, which could be regarded as a "shell", though we would commonly not use that term (perhaps instead call it "interactive interpreter" - see also below); but python foo.py simply runs the script foo.py and exits, so there is no shell there.
The definition of "shell" is slightly context-dependent, but you don't really seem to be asking about shells here. (Some GUIs have a concept of "graphical shell" etc but we are already out of the scope of what you were trying to ask about.) Some programs are command interpreters (the Python executable interprets and executes commands in the Python language; the Bourne shell interprets and executes shell scripts) but generally only those whose primary purposes include running other programs are called "shells".

None of those equalities are true, and half of those "shells" aren't actually shells.
Your bash shell is a shell. When you launch your Python script from that shell, the Python process that runs the script is a child process of the bash shell process. When you launch a subprocess from the Python script, that subprocess is a child process of the Python process. If you launch the subprocess with shell=True, Python invokes a shell to parse and run the command, but otherwise, no shell is involved in running the subprocess.
Child processes inherit environment variables from their parent on startup (unless you take specific steps to avoid that), but they cannot set environment variables for their parent. You cannot run a Python script to set environment variables in your shell, or run a subprocess from Python to set your Python script's environment variables.

set environment in tcsh opened form python

I work in tc shell and have environment set script with commands like setenv foo bar. I use to run this script by command source set_env.sh and then, when environment is set, run compilation command make my_ target...
I would like to automate the whole process by executing it from Python by calling subprocess.Popen or even os.system. The troubles are:
Shell opened form Python do not recognize command source. My assumption that /bin/bash shell is opened. How do I force Python to open /bin/tcsh?
How do I preserve changed environment setting for second command (make my_ target...)?

If you use the subprocess package in Python then it won't run the shell by default, it will just launch a process (without using the shell). You can add shell=True, but this won't do you much good since environment variables are set for the current process only, and are not sent back to the parent process.
Python starts a tcsh child process, and child processes can't affect parent processes.
The solution would be to make your script print out the environment variables, and then parse the output of that in Python.
So if your script prints something like:
XX=one
YY=two
Then in Python do something along the likes of:
for line in output.split('\n'):
if line == '':
continue
k, v = line.split('=')
os.environ[k] = v

How to run python code in the currently activated virtualenv with subprocess?

I have a virtualenv named 'venv' and it is activate:
(venv)>
and I wrote codes that I'll run it in the virtualenv (main.py):
import subprocess
result = subprocess.run('python other.py', stdout=subprocess.PIPE)
but when I run main.py file:
(venv)> python main.py
subprocess does not execute the command (python other.py) in the virtualenv i.e venv
How to run subprocess command in the current virtualenv session?

A child process can't run commands in its parent process without that process's involvement.
This is why ssh-agent requires usage as eval "$(ssh-agent -s)" to invoke the shell commands it emits on output, for example. Thus, the literal thing you're asking for here is impossible.
Fortunately, it's also unnecessary.
virtualenvs use environment variables inherited by child processes.
This means that you don't actually need to use the same shell that has a virtualenv activated to start a new Python interpreter intended to use the interpreter/libraries/etc. from that virtualenv.
subprocess.run must be passed a list, or shell=True must be used.
Either do this (which is better!)
import subprocess
result = subprocess.run(['python', 'other.py'], stdout=subprocess.PIPE)
Or this (which is worse!)
import subprocess
result = subprocess.run('python other.py', stdout=subprocess.PIPE, shell=True)

If you want to run a script with the same Python executable being used to run the current script, don't use python and rely on the path being set up properly, just use sys.executable:
A string giving the absolute path of the executable binary for the Python interpreter, on systems where this makes sense.
This works if you executed the script with python myscript.py relying on the active virtualenv's PATH. It also works if you executed the script with /usr/local/bin/python3.6 to ignore the PATH and test your script with a specific interpreter. Or if you executed the script with myscript.py, relying on a shbang line created at installation time by setuptools. Or if the script was run as a CGI depending on your Apache configuration. Or if you sudod the executable, or did something else that scraped down your environment. Or almost anything else imaginable.1
As explained in Charles Duffy's answer, you still need to use a list of arguments instead of a string (or use shell=True, but you rarely want to do that). So:
result = subprocess.run([sys.executable, 'other.py'], stdout=subprocess.PIPE)
1. Well, not quite… Examples of where it doesn't work include custom C programs that embed a CPython interpreter, some smartphone mini-Python environments, old-school Amiga Python, … The one most likely to affect you—and it's a pretty big stretch—is that on some *nix platforms, if you write a program that execs Python by passing incompatible names for the process and arg0, sys.executable can end up wrong.

create unix alias using a python3 script

Well, as we all known, creating an alias in a terminal shell is quite easy:
ZZ:~ zhangzhao$ alias c='uname'
ZZ:~ zhangzhao$ c
Darwin
ZZ:~ zhangzhao$
But now I want to do the same thing through a Python3 script. I've checked the ref manual and found these sort of command work can be solved using subprocess module.
Then I write the script below:
import subprocess
subprocess.call(["alias", "c=\'uname\'"])
But note that this operation will not take effect to the shell you are currently using, instead, it will use a subshell and then leave. So what this script has done is totally in vain.
So my problem is: how to create an alias in the currently using shell by executing a python script?

In general, you can't
All alias you set in only works in current shell and new aliaes can be added only by shell it self rather than sub-shell or sub-process.
In hack way, you can use gdb to attach you parent shell and change its alias table. But in modern Unix, child process is not allowed to attach parent process. You need to low down the system security level