We Keep Coding

Issues with the "subprocess" module in Python 3.9.1 (or 3.9.2) - "subprocess.Popen" doesn't work

When my script has the .pyw extension, the function subprocess.Popen doesn't work, but if I use the .py extension, it works. Actually the extension is not so important, the main point is if I use the terminal or not to run the script, if I don't use it I have the issue, otherwise no issues.
This wird behaviout happens only in my PC with Python 3.9.1 or 3.9.2 (the last version currently available). However I have another PC with Python 3.9.0, and there the issue doesn't exist.
I'll give you an example. here I have two scripts, below is the first one named main_script.pyw:
from tkinter import *
from tkinter import ttk
import sys, subprocess
def MyFunction():
subprocess.Popen(["start", sys.executable, "script.py"], shell=True)
parent=Tk()
parent.geometry("300x250+370+100")
parent.title("Test")
TestButton=ttk.Button(parent, text="start", width=16, command=MyFunction)
TestButton.place(x=10, y=10)
parent.mainloop()
Here is the second one named script.py:
a=input("give me a number: ")
Both of them are placed in the same directory. The user can start the second script using the function subprocess.Popen only if the extension of the main script is .py, otherwise he can't. How can I solve the issue? Is it a BUG?
I attached a GIF too:
UPDATE! 1
I tried to install Python 3.9.0 in a virtual machine, and there I found the same issue described before! so, I can't understand why in my PC, with the same version, the script works. if I type python --version in my terminal the output is Python 3.9.0. Here is another GIF to show you that it works:
In the installation path (C:\Users\USER_NAME\AppData\Local\Programs\Python), I saw two folders, Python39 and Python38-32. Probably the second one is a refuse from an old installation, I don't know, but maybe it helps to make the script work. What do you think?
I just want to run CLI scripts via Tkinter as you saw in the last GIF (without using the terminal of course). How can I reach my goal?
UPDATE! 2
I'm not sure but, maybe it's a Windows issue. I tried to run Google Chrome using the subprocess.Popen(["start", sys.executable, "script.py"], shell=True) instruction (before you have to open the terminal in the Chrome installation folder C:\Program Files\Google\Chrome\Application) and it worked only in my PC where I never have issues, but with the other one Chrome didn't start, and in the terminal I got this output (I use ConEmu as terminal):
>>> subprocess.Popen(["start", sys.executable, "chrome.exe"], shell=True)
<Popen: returncode: None args: ['start', 'C:\\Users\\aquer\\AppData\\Local\\...>
SyntaxError: Non-UTF-8 code starting with '\x83' in file C:\Program Files\Google\Chrome\Application\chrome.exe on line 2, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
Current directory:
C:\Program Files\Google\Chrome\Application
Command to be executed:
"C:\Users\USER_NAME\AppData\Local\Programs\Python\Python39\python.exe" chrome.exe
ConEmuC: Root process was alive less than 10 sec, ExitCode=1.
Press Enter or Esc to close console...
I tried to open the terminal in Admin mode but, also in this case, Google Chrome didn't start. Where is the issue from your point of viewes?

I think that it is because for .py files Windows uses python.exe and for .pyw files Windows uses pythonw.exe. The problem is that sys.executable has the file location of the program used to start the main python file (python.exe when the extension is .py and pythonw.exe when the extension is .pyw). So the problem is that you are trying to start the .py program using the python executable that is for .pyw file extensions.
In cmd:
>>> import sys
>>> sys.executable
'C:\\Program Files\\Python37\\python.exe'
in IDLE (doesn't have the terminal window):
>>> import sys
>>> sys.executable
'C:\\Program Files\\Python37\\pythonw.exe'
Therefore, you have to decide which program (python.exe or pythonw.exe) you want to use. You can do it based on the file extension.

A simple solution would be to open a command prompt from your main_script.py manually, and execute script.py in there, if you really have to execute it as an extra process. If you do not have to, just add import script at the place you want the content of script.py to be executed, or, even better, wrap the code in script.py into a function, import script.py at the top of main_script.py and call the funcction at the approproate place. For the first solution, there is an example on DataToFish that has almost the same code as you (using tkinter GUI a.o., you have to scroll down a bit to find the example with a GUI). In short, you just have execute cmd /c {sys.executable} script.pyw from main_script.py.

How to close console when my tkinter GUI is running after converting it in exe format [duplicate]

I am writing an IRC bot in Python.
I wish to make stand-alone binaries for Linux and Windows of it. And mainly I wish that when the bot initiates, the console window should hide and the user should not be able to see the window.
What can I do for that?

Simply save it with a .pyw extension. This will prevent the console window from opening.
On Windows systems, there is no notion of an “executable mode”. The Python installer automatically associates .py files with python.exe so that a double-click on a Python file will run it as a script. The extension can also be .pyw, in that case, the console window that normally appears is suppressed.
Explanation at the bottom of section 2.2.2

In linux, just run it, no problem. In Windows, you want to use the pythonw executable.
Update
Okay, if I understand the question in the comments, you're asking how to make the command window in which you've started the bot from the command line go away afterwards?
UNIX (Linux)
$ nohup mypythonprog &
Windows
C:/> start pythonw mypythonprog
I think that's right. In any case, now you can close the terminal.

On Unix Systems (including GNU/Linux, macOS, and BSD)
Use nohup mypythonprog &, and you can close the terminal window without disrupting the process. You can also run exit if you are running in the cloud and don't want to leave a hanging shell process.
On Windows Systems
Save the program with a .pyw extension and now it will open with pythonw.exe. No shell window.
For example, if you have foo.py, you need to rename it to foo.pyw.

This will hide your console. Implement these lines in your code first to start hiding your console at first.
import win32gui, win32con
the_program_to_hide = win32gui.GetForegroundWindow()
win32gui.ShowWindow(the_program_to_hide , win32con.SW_HIDE)
Update May 2020 :
If you've got trouble on pip install win32con on Command Prompt, you can simply pip install pywin32.Then on your python script, execute import win32.lib.win32con as win32con instead of import win32con.
To show back your program again win32con.SW_SHOW works fine:
win32gui.ShowWindow(the_program_to_hide , win32con.SW_SHOW)

If all you want to do is run your Python Script on a windows computer that has the Python Interpreter installed, converting the extension of your saved script from '.py' to '.pyw' should do the trick.
But if you're using py2exe to convert your script into a standalone application that would run on any windows machine, you will need to make the following changes to your 'setup.py' file.
The following example is of a simple python-GUI made using Tkinter:
from distutils.core import setup
import py2exe
setup (console = ['tkinter_example.pyw'],
options = { 'py2exe' : {'packages':['Tkinter']}})
Change "console" in the code above to "windows"..
from distutils.core import setup
import py2exe
setup (windows = ['tkinter_example.pyw'],
options = { 'py2exe' : {'packages':['Tkinter']}})
This will only open the Tkinter generated GUI and no console window.

Some additional info. for situations that'll need the win32gui solution posted by Mohsen Haddadi earlier in this thread:
As of python 361, win32gui & win32con are not part of the python std library.
To use them, pywin32 package will need to be installed; now possible via pip.
More background info on pywin32 package is at: How to use the win32gui module with Python?.
Also, to apply discretion while closing a window so as to not inadvertently close any window in the foreground, the resolution could be extended along the lines of the following:
try :
import win32gui, win32con;
frgrnd_wndw = win32gui.GetForegroundWindow();
wndw_title = win32gui.GetWindowText(frgrnd_wndw);
if wndw_title.endswith("python.exe"):
win32gui.ShowWindow(frgrnd_wndw, win32con.SW_HIDE);
#endif
except :
pass

After writing the code you want to convert the file from .py to .exe, so possibly you will use pyinstaller and it is good to make exe file. So you can hide the console in this way:
pyinstaller --onefile main.py --windowed
I used to this way and it works.

just change the file extension from .py to .pyw

As another answer for all upcoming readers:
If you are using Visual Studio as IDE, you can set "Window Application" in the Project settings with a single checkmark. Which is working with py-extension as well.

a decorator factory for this (windows version, unix version should be easier via os.fork)
def deco_factory_daemon_subprocess(*, flag_env_var_name='__this_daemon_subprocess__', **kwargs_for_subprocess):
def deco(target):
#functools.wraps(target)
def tgt(*args, **kwargs):
if os.environ.get(flag_env_var_name) == __file__:
target(*args, **kwargs)
else:
os.environ[flag_env_var_name] = __file__
real_argv = psutil.Process(os.getpid()).cmdline()
exec_dir, exec_basename = path_split(real_argv[0])
if exec_basename.lower() == 'python.exe':
real_argv[0] = shutil.which('pythonw.exe')
kwargs = dict(env=os.environ, stdout=subprocess.PIPE, stderr=subprocess.PIPE, )
kwargs.update(kwargs_for_subprocess)
subprocess.Popen(real_argv, **kwargs)
return tgt
return deco
use it like this:
#deco_factory_daemon_subprocess()
def run():
...
def main():
run()

Python shutil.which not working with wsl.exe

I am trying to use shutil.which to check if the Linux Subsystem is installed on Windows 10.
Using the Windows where command in Command Prompt, I can see the location of the wsl.exe executable.
C:\Users\spike>where wsl
C:\Windows\System32\wsl.exe
The above shows that WSL does exist, and is in my system PATH.
When I use the which function in Python, it says that the executable was not found.
print(which("wsl")) # Returns None
Just to make sure that which works, I test it on cmd.exe.
print(which("cmd")) # Returns "C:\Windows\System32\cmd.exe"
That works. Well, what if I make a system shell call with the command that did work?
print(system("where wsl")) # Returns 1
Exit code 1, the command wsl was not found.
So I test it on cmd.exe again.
print(system("where cmd")) # Returns 0
Okay, so that does work. What is the problem?
For each Python 3 example assume these imports.
from shutil import which
from os import system
Why can Python not find wsl.exe even though it is proven to exist?
Thanks.

Credit to #eryksun, who helped solve this in the comments.
The issue is that I am using 32 bit Python, and wsl.exe is only in C:/Windows/System32. The problem with this is that Python is looking in C:/Windows/SysWOW64 for the executable instead.
wsl.exe is only 64-bit, and you're looking in SysWOW64 instead of the real System32 because you're using 32-bit Python. – eryksun
Because WSL only supports 64-bit systems, I ended up just running my code with 64-bit Python. However, and alternate solution if you only use Py32 would be to access SysWOW64 directly, using the system root environment variable and os.path.join.
In Windows 7+, the real System32 directory is accessible in a 32-bit process as "SysNative". Unfortunately this virtual directory isn't available in a native 64-bit process, so you need to first check whether it exists. For example: sysnative = os.path.join(os.environ['SystemRoot'], 'SysNative'); if os.path.exists(sysnative): .... – eryksun

Now you can install python in 64 bit and everything will work just fine. (Be careful that when the python installer runs, there should be written 64 bit packages and other things as 64 bits [right where the loading bar is])

Can't Open Python IDLE

Getting the following code opening in the console in 9 separate screens when I try to open Python IDLE. As far as I'm aware, there isn't anything that I have done to the original install. Any ideas? Tried uninstalling and reinstalling but no luck. The code shown when I open the program is...
try:
import idlelib.PyShell
except ImportError:
# IDLE is not installed, but maybe PyShell is on sys.path:
try:
from . import PyShell
except ImportError:
raise
else:
import os
idledir = os.path.dirname(os.path.abspath(PyShell.__file__))
if idledir != os.getcwd():
# We're not in the IDLE directory, help the subprocess find run.py
pypath = os.environ.get('PYTHONPATH', '')
if pypath:
os.environ['PYTHONPATH'] = pypath + ':' + idledir
else:
os.environ['PYTHONPATH'] = idledir
PyShell.main()
else:
idlelib.PyShell.main()
I only have the option to close this window and not to create a new Python file.

I am an Idle maintainer and it is very unclear how you tried to open Idle, what you mean by '9 screens', and what you 're-installed', and on what system and version of Python.
The easiest way to start Idle with installed Python is to use the Start menu icon (on Windows) or the equivalent on other systems. On a command line, use python -m idlelib, where python starts some version of installed 3.x. For uninstalled 3.x or 2.x, use idlelib.idle. From within Python, import idlelib.idle starts Idle.
If you have python 2.7 or 3.4 properly installed and running, so that python -m test.regrtest runs and passes (.regrtest is only needed for 2.x), I would be very surprised if Idle did not run with the methods above.
The code you posted is idlelib/idle.pyw. It is used by idlelib/idle.bat and may have other specialized uses, such as running Idle with a subprocess call. I suspect that it is partially obsolete. In any case, I doubt it will be helpful to you.

Eclipse running 64-bit Python, rather than 32-bit

I am running OS X Lion, Python 2.7, and I am trying to setup Pygame to work with PyDev in Eclipse. I set up PyDev to use a custom-installed Python (not the default one). I forced this install to use 32-bit, which works fine in the Terminal - I can import Pygame, and other modules. However, when I use it in PyDev, it gives me a no matching architecture error. It also appears to be running in 64-bit mode.
The paths to the interpreter are the same.
import sys
print ("%x" % sys.maxsize, sys.maxsize > 2**32)
prints out ('7fffffff', False) while using Terminal, but in Eclipse/PyDev it prints out ('7fffffffffffffff', True)
The two paths (using sys.executable) are:
In Terminal it is:
/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
And in Eclipse it is
/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
The path to the interpreter I used is: /Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
I also tried manually specifying the interpreter in Terminal - Using the above path. It worked.
The default python that comes with the system is /usr/bin/python
I am using a 32-bit version of Eclipse classic as shown by an answer to this question.
Does anyone have any idea what is wrong?

You need to instruct eclipse to use 32-bit python.
Right-click on your project -> properties -> PyDen/Interpreter grammar and select "Click here to configure an interpreter not listed"
After you add your new python binary (e.g. C:\Python27\python.exe), you go back to the interpreter menu in the properties window and you select this interpreter from the corresponding drop-down menu.

I solved this by using a method describe in an answer to another question.
The answer said to go to your plugins/org.python.pydev/pysrc and open interpreterInfo.py. Then you replace all the instances of sys.executable with the path to the interpreter you want.
In my case, this meant changing them to /Library/Frameworks/Python.framework/Version/2.7/bin/python2.7
After that, open up eclipse and create a new interpreter with the same path, and it should work.