I'm running on Windows Server 2012 R2 standard machine with 32Bit python.
I'm trying to run a powershell script for a 64Bit system which check if a key exists in the registry:
$path = "Registry::HKLM\Software\<folder>"
if (!(Test-Path $path))
{
write "$path does not exist!"
}
When running through powershell, script works perfect.
When I'm running it from python, it doesn't find the key:
from gevent.subprocess import call
call("powershell <script.ps1>, shell=True")
After some research, I found that the 32bit python process is calling the 32bit version of powershell.
I verified it with this simple script that checks if the process running is 32bit or 64bit:
powershell [System.Environment]::Is64BitProcess
Will return True for 64Bit process and False for a 32Bit process.
Manually checking this command works:
C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe [System.Environment]::Is64BitProcess
Returns False as this is the 32Bit version of powershell (yes, quite confusing due to the WOW64 folder).
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe [System.Environment]::Is64BitProcess
Returns True.
But running:
from gevent.subprocess import call
call("C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe [System.Environment]::Is64BitProcess", shell=True)
Returns False
What am I missing?
What you're missing is that Windows transparently redirects file system calls from 32-bit processes to C:\Windows\System32 to C:\Windows\SysWOW64 :)
There a simple solution to this - if a 32-bit process tries to fetch file system resources from C:\Windows\sysnative instead, Windows will not redirect to SysWOW64.
From PowerShell it's fairly easy to figure out whether your process will be affected by all of this SysWOW64 redirection, basically:
$IsSysWOW64Process = [Environment]::Is64BitOperatingSystem -and -not [Environment]::Is64BitProcess
In python however, I haven't been able to find a surefire way of doing this, so I suspect your best bet is to call kernel32!IsWow64Process2() via ctypes:
from ctypes import windll,c_ushort,byref
import platform
def is_syswow64_process():
if platform.architecture()[0] != '64bit':
# 32-bit OS, no syswow64 handling
return False
# Ok, 64-bit OS, let's see if the process is 32-bit
# Obtain process handle to self
this_process = windll.kernel32.GetCurrentProcess()
# Declare ref arguments
proc_type, platform_type = c_ushort(), c_ushort()
# Query Windows for the information
wow64_call = windll.kernel32.IsWow64Process2(this_process, byref(proc_type), byref(platform_type))
if wow64_call == 0:
# you'd probably want to call kernel32!GetLastError here
raise Exception("Problem querying kernel32!IsWow64Process2")
return proc_type.value == 0
And now you can conditionally replace the path when needed:
powershell = "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe"
if is_syswow64_process():
powershell = re.sub("(?i)syswow64|system32", "sysnative", powershell)
call("%s .\path\to\script.ps1" % powershell)
Related
I am trying to create a python setup script that will be converted to an executable file in windows using pyinstaller. Currently, I am trying to figure out how this script would check if python is installed on the system, if not, then it will go ahead & download the installation exe file from the python website.
My current code is as follows:
import subprocess, requests, platform
architecture = platform.uname()[4]
if architecture == 'AMD64':
executable_file = "https://www.python.org/ftp/python/3.11.1/python-3.11.1-amd64.exe"
elif architecture == 'ARM64':
executable_file = "https://www.python.org/ftp/python/3.11.1/python-3.11.1-arm64.exe"
r = requests.get(executable_file)
with open("python-3.11.1-amd64.exe",'wb') as f:
f.write(r.content)
cmd = "python-3.11.1-amd64.exe"
returned_value = subprocess.call(cmd, shell=True)
if returned_value == 1602:
print("Installation Cancelled. Exitting.")
quit()
print('returned value:', returned_value)
I currently have no idea how to check if python is installed so the exe will not be downloaded & ran by the script. So I would like some ideas on how I could go with this.
Directly execute the python command to see the output, or use the winreg module to obtain the installed applications of the local machine
Created an executable using pyinstaller on ubuntu 16.04 and trying to run it on SuSe 12 SP4 gives error at a certain portion of the code.
The code works like this:
Its a flask app that receives the input from user over web
Process those inputs and creates a .sh script and run that shell script
Reads the output from the shell script and present it to the web as a return render
The executable was successfully created on the ubuntu machine and works successfully and no issues seens but when I use this executable on SuSe12 SP4, it starts but when it reaches the code where it runs the bash script, it throws the following error:
sh: /tmp/_MEI369vhy/libreadline.so.6: no version information available (required by sh)
I am really tired of looking for solutions and have done the following so far:
Tried both --onefile and --onedir, no difference
Tried creating the executable on SuSe12 sp4 itself but it throws a different error regarding subprocess not being found
Tried finding the libreadline.so on Suse with no luck
Tried creating an env on ubuntu 14 but too many dependencies errors
I'm finally out of suggestions and could use some help here. If you can please assist.
Environment
Python 2.7.12
Ubuntu 16.04
SuSe12 SP4
Pyinstaller 3.6
P.S. The code as a raw python code works flawlessly on SuSe 12 SP4 if i create proper build environment
So, I finally solved it with some assistance from Rokm. The warning message above was not causing any issues but it was due to the environment variable not being passed to subprocess.
In order to solve this issue, I simply did the following:
###Add the following code to your existing code
env = dict(os.environ) # make a copy of the environment
lp_key = 'LD_LIBRARY_PATH' # for GNU/Linux and *BSD.
lp_orig = env.get(lp_key + '_ORIG')
if lp_orig is not None:
env[lp_key] = lp_orig # restore the original, unmodified value
else:
# This happens when LD_LIBRARY_PATH was not set.
# Remove the env var as a last resort:
env.pop(lp_key, None)
Next, add the env variable to subprocess Popen command. Here is the full code for reference. This code will give you the output of the command and also return code aka exit code of the command. Also, you don't have to use any shelix or any other things to run it, simple .strip() command will do it for you. Hope you guys find it useful, enjoy. !!
from subprocess import Popen,PIPE,STDOUT
env = dict(os.environ) # make a copy of the environment
lp_key = 'LD_LIBRARY_PATH' # for GNU/Linux and *BSD.
lp_orig = env.get(lp_key + '_ORIG')
if lp_orig is not None:
env[lp_key] = lp_orig # restore the original, unmodified value
else:
# This happens when LD_LIBRARY_PATH was not set.
# Remove the env var as a last resort:
env.pop(lp_key, None)
cmd = raw_input('Enter your command:')
out = Popen(cmd.split(),stderr=STDOUT,stdout=PIPE, env=env)
t, y = out.communicate()[0],out.returncode
print('output : ' + str(t))
print ('Return Code : ' + str(y))
P.S. Unfortunately, cmd.split() will fail in some cases, ie when an argument has spaces, like cmd='/usr/bin/ls "/home/user/my directory"' will fail with cmd.split(). In that case, cmd = shlex.split(cmd, posix=True) will work better. But shlex.split() will fail when stdout is captured, hence there is no allrounder solution IMHO
I have 2 python interpreters installed on a windows 10 computer. Python 3.6 from ArcGIS pro 2.2 and Python 2.7 from ArcMap 10.6, using the cmd on Windows:
"C:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy" -c "import arcpy;print(arcpy.__path__)"
The output is:
'C:\\Program Files\\ArcGIS\\Pro\\Resources\\ArcPy\\arcpy'
This is the intended result, but I need to do the same using subprocess.Popen from Python2.7 and also need to use shell=True
cmd = """ "C:/Program Files/ArcGIS/Pro/bin/Python/Scripts/propy" -c "import arcpy;print(arcpy.__path__)" """
subprocess.Popen(cmd, shell=True,stdout = subprocess.PIPE)
process.stdout.read()
This will raise the error:
Fatal Python error: Py_Initialize: unable to load the file system codec
File "C:\Python27\ArcGIS10.5\lib\encodings\__init__.py", line 123
raise CodecRegistryError,\
Basically the interpreter being used is Python2.7, I have tried to set %PYYHONPATH% to Python3.6 but still the same problem
Just a note, propy is a .bat that activates Python's virtualenv and therefore should be setting everything properly
How could I have the proper output using subprocess.Popen ?
We had a similar problem when trying to run python scripts within ArcGIS Workflow Manager within ArcGIS Pro. The problem is ArcGIS Workflow Manager loads up Python 2.X but we needed Python 3.X
To resolve the problem we added the following in our __init__.py
import sys
if sys.version_info[0] >= 3:
# When run through Workflow Manager 10.5, the PYTHONPATH environment
# variable is being set to the Python 2 environment, and that needs to be removed
# when running in the Python 3 environment.
sys.path = list(filter(lambda p: 'arcgis\\desktop' not in p, sys.path))
A script is developed both on OS X and Windows using a virtualenv. The so-called developer has already installed all required packages using a requirements.txt file, but one problem remains:
If the script is running on OS X, the beginning of each Python file must start like this:
#!/Users/os-x-username/.virtualenvs/some_env/bin/python
#!C:\Users\windows-username\Envs\some_env\Scripts\python.exe
But if developing on Windows, the line order must be switched:
#!C:\Users\windows-username\Envs\some_env\Scripts\python.exe
#!/Users/os-x-username/.virtualenvs/some_env/bin/python
How can the so-called developer avoid this tediousness?
If you don't mind adding extra steps, ou can create a launcher script launcher.py like:
#!/usr/bin/env python
import subprocess
import sys
if __name__ != "__main__":
print("This is a launcher. Please run only as a main script.")
exit(-1)
INTERPRETERS = {
"win": r"C:\Users\windows-username\Envs\some_env\Scripts\python.exe", # Windows
"darwin": "/Users/os-x-username/.virtualenvs/some_env/bin/python", # OSX
"linux": "/home/linux-user/.virtualenvs/some_env/bin/python" # Linux
# etc.
}
TARGET_SCRIPT = "original_script_name.py"
interpreter = None
for i in INTERPRETERS: # let's find a suitable interpreter for the current platform
if sys.platform.startswith(i):
interpreter = i
break
if not interpreter:
print("No suitable interpreter found for platform:", sys.platform)
exit(-1)
main_proc = subprocess.Popen([interpreter, TARGET_SCRIPT] + sys.argv[1:]) # call virtualenv
main_proc.communicate() # wait for it to finish
exit(main_proc.poll()) # redirect the return code
Since this script is there only to run the original_script_name.py in the desired interpreter for the current platform, it doesn't matter what its shebang is - as long as it picks any Python interpreter it will be fine.
It would act as a drop-in replacement for your original script (original_script_name.py) so just call launcher.py instead and it will even redirect the CLI arguments if needed.
I have this simple script..that does not work
import subprocess
subprocess.call(["C:\Program Files\Oracle\VirtualBox\VBoxManage.exe", "VBoxManage startvm WIN7"])
I have the same thing in a bat file...which works perfectly.
cd C:\Program Files\Oracle\VirtualBox
VBoxManage startvm "WIN7"
I have the VBoxManage.exe in the PATH of Windows 8.1 (My host OS).
The python script understands the VBoxManage executable and spits out it's manual and then this ..
Syntax error: Invalid command 'VBoxManage startvm WIN7'
Could you give me a way to start a VM from inside a python script, either by invoking the .exe directly or by running the .bat file ?
Note: I have searched for the vboxshell.py file but not found it anywhere...:[
subprocess.call() expects a list of arguments, like so:
subprocess.call(['C:\Program Files\Oracle\VirtualBox\VBoxManage.exe',
'startvm',
'WIN7'])
Your code passes 'VBoxManage startvm WIN7' as a single argument to VBoxManage.exe, which expects to find only a command (e.g. 'startvm') there. The subsequent arguments ('WIN7' in this case) need to be passed separately.
In addition, there is no need to repeat the executable name when using subprocess.call(). The example from the Python docs invokes the UNIX command "ls -l" as follows:
subprocess.call(['ls', '-l'])
In other words, you don't need to repeat the 'VBoxManage' part.
The trick is to pass the command as separate arguments
import subprocess
subprocess.call(["C:\Program Files\Oracle\VirtualBox\VBoxManage.exe", "startvm", "WIN7"])