Best way to call subprocess scripts in a Python exe - python

I'm currently trying to make a cross-platform Python exe file that relies on calling other Python and R scripts. One issue I was facing was that my exe file expected my script files to be in the root directory as opposed to the directory where my exe file is. I've managed to fix this by doing the following
if getattr(sys, 'frozen', False):
PROJECT_ROOT = sys.executable
else:
PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
pwd = os.path.dirname(PROJECT_ROOT)
# messagebox is for the tkinter based GUI
messagebox.showinfo('Info', 'Please wait a moment')
subprocess.call(['python', pwd + '/Customs.py'], shell = False)
subprocess.call(['Rscript', pwd + '/r_script.R'], shell=False)
subprocess.call(['python', pwd + '/by_month.py'], shell = False)
messagebox.showinfo('Info', 'Processing completed)
I'm wondering if there is a cleaner/ more reliable way of doing this to decrease the potential of an error occurring that might break the software.
I should also mention that I've read something about turning the other scripts into an exe file first and I would like to hear your opinion on this.
Thanks

First I suggest you use a function that returns the current path of your file when it is running as a frozen script or running normally to make testing of your code easy.
Second, if you want to bundle your script files you need to first bundle your files with executable as a DATA file. Next prepare the path for that file when using them with the said function.
def app_path():
if getattr(sys, 'frozen', False):
app_path = os.path.dirname(sys.executable)
elif __file__:
app_path = os.path.dirname(__file__)
return app_path
def resource_path(relative_path):
if hasattr(sys, '_MEIPASS'):
return os.path.join(sys._MEIPASS, relative_path)
return os.path.join(os.path.abspath("."), relative_path)
# messagebox is for the tkinter based GUI
messagebox.showinfo('Info', 'Please wait a moment')
subprocess.call(['python', resource_path('Customs.py')], shell=False)
subprocess.call(['Rscript', resource_path('r_script.R')], shell=False)
subprocess.call(['python', resource_path('by_month.py')], shell=False)
messagebox.showinfo('Info', 'Processing completed)
And your command to build your app would be something like this:
pyinstaller -F --add-data "Customs.py;." --add-data "r_script.R;." --add-data "by_month.py;." myscript.py
When your app runs, the data files Customs.py, r_script.R, etc would be extracted to a temp folder and resource_path would return the exact path for each file. But remember that if you want to load some files from your current directory (Where the exe file located) you can use app_path function.

Related

How can I get a PyInstaller executable to create a file in the same directory?

I found the answer
So it looks like PyInstaller actually runs in a temp directory, not your own, which explains my issue. This is an explanation for that. I guess I will keep this up incase people in the future have problems.
Original question
I am trying to use PyInstaller to create an executable of a simple python script called test.py that just creates a text file and adds numbers to it, as a test of PyInstaller. This file works correctly when run normally.
from os.path import dirname, abspath
def main():
txtfile = dirname(abspath(__file__)) + '/text.txt'
with open(txtfile, 'w') as f:
for num in range(101):
f.write(str(num) + '\n')
if __name__ == '__main__':
main()
print('script executed')
When I use:
pyinstaller test.py --onefile in the same directory as test.py it successfully creates the dist file with the binary file test inside of it.
when I cd dist and do ./test to execute the file inside dist it successfully prints out main called and script executed but it doesn't actually create the file. So, main is being called and the script is being executed, but the file isn't created at all, and I am quite confused about what I'm doing wrong..I must be getting file paths messed up? But I have specified the exact full path with os.path, so it doesn't make sense to me.
The system exit code is 0, and there are no errors raised when I call ./test
I found this that shows that PyInstaller will save to a temp file. I created this script below to check if the script is being executed directly or via PyInstaller.
import os
import sys
def create_file(path):
with open(path + '/test.txt', 'w') as f:
for num in range(101):
f.write(str(num) + '\n')
def check_using_pyinstaller():
if getattr(sys, 'frozen', False):
application_path = os.path.dirname(sys.executable)
return application_path
return os.path.dirname(os.path.abspath(__file__))
def main():
path = check_using_pyinstaller()
os.chdir(path)
create_file(path)
if __name__ == '__main__':
main()

Pyinstaller Question about icons and writing to bundled files

I am using Python 3.6.8 and Linux Ubuntu
I have an icon.ico image, and i want to embed it into the exe file in Pyinstaller.
I have tried
pyinstaller -F -i= 'icon.ico' main.py and
pyinstaller -F --icon= 'icon.ico:.' main.py
i tried all sorts of ways and i cant get it to work...
I basically have a tkinter app that i want to make an exe from...and i have created a png file turned it into icon and try to load it, but i am not sure it can be done,....is there a way to assign image.ico to apps? or change them, i know there is a way from the .desktop ...how do software do it that you download?
This is the internal code i am using:
import os, sys
from con import * # this is just a configuration file that has g='play' in it.
import subprocess
def resource_path(relative_path):
if getattr(sys, 'frozen', False):
bundle_dir = sys._MEIPASS # for --onefile
# bundle_dir = path.dirname(path.abspath(sys.executable)) # for --onedir
else:
bundle_dir = os.path.dirname(os.path.abspath(__file__))
return os.path.join(bundle_dir, relative_path)
resource_path('data/bitbud.ico')
basedir = getattr(sys, "_MEIPASS", os.path.realpath(os.path.dirname(__file__)))
file=os.path.join(basedir, 'data/bell.mp3')
file2=os.path.join(basedir, 'data/testfile2.txt')
file3=os.path.join(basedir, 'data/bitbud.ico')
f=open('testfile1','w')
f.write('This has worked')
f.close()
f=open(file2,'a')
f.write('append also works')
f.close()
#if 'play' == g:
try:
subprocess.call(['/usr/bin/cvlc',file],timeout=6)
except:
pass
f1=open(file2,'r')
f=open('testfile3','w')
readit='The app has inside it:' +f1.read()
f.write(readit)
f.close()
f1.close()
Some line for you :D
Get file in bundled
def resource_path(relative_path):
if getattr(sys, 'frozen', False):
bundle_dir = sys._MEIPASS # for --onefile
# bundle_dir = path.dirname(path.abspath(sys.executable)) # for --onedir
else:
bundle_dir = path.dirname(path.abspath(__file__))
return path.join(bundle_dir, relative_path)
---
resource_path('assets/file.jpg')
Edit some.spec file after run Pyinstaller first time, add params like below:
a = Analysis(
...
datas=[('path/to/file.jpg', 'assets')]
...
...
exe = EXE(
...
icon='path/to/icon.ico')

Pyinstaller single exe does not work properly and apparently, there are many people having problems that go unsolved

I have two files, one bell.mp3 one main.py file, that plays back bell.mp3 via subprocess.
If I do:
pyinstaller main.py
the Dist file ends up correctly, and everything works fine, The program runs in a directory.
This is the code for my file which i call pyinst_tester.py
it creates a text file, and plays a bell.mp3 file
#
from con import * # this is just a configuration file that has g='play' in it.
import subprocess
f=open(r'/home/godzilla/Desktop/Pyinstaller testing/testfile1','w')
f.write('This has worked')
f.close()
file='/home/godzilla/Desktop/Pyinstaller testing/data/bell.mp3'
if 'play' == g:
subprocess.call(['/usr/bin/cvlc',file])
a single file is created, but if I delete the bell.mp3 file it doesn't work. In a single file isn't the bell.mp3 zipped inside the main.exe ? therefore, redundant as a separate file ?
What Is the point having a single file exe, if you need an adjacent file with all the mp3s inside?
Pyinstaller has many features and if you want to include non python files (for example mp3 files) you have to do so explicitly with the --add-binary switch.
In one file mode the executable will be unpacked into a temporary directory prior to execution of the python code.
So how to write your code to access these data files.
You might want to look at the pyinstaller documention at following sections:
https://pyinstaller.readthedocs.io/en/stable/runtime-information.html#run-time-information
https://pyinstaller.readthedocs.io/en/stable/runtime-information.html#using-sys-executable-and-sys-argv-0
I personally place all my files in a separate directory. e.g. data.
If you place the file bell.mp3 in the directory data, then you had to call pyinstaller with the option --add-binary data:data
in the one file mode the executable is extracted into a temporary directory
whose path you get get from the variable sys._MEIPASS
Your data directory will bi in the sub directory data of sys._MEIPASS
In my example I create a function, that will be able to locate the data files in normal python mode and in pyinstaller one file or one directory mode.
Just try it out it should be self explaining
simple example:
minitst.py
import os, sys
import time
is_frozen = getattr(sys, "frozen", False)
MYDIR = os.path.realpath(os.path.dirname(__file__))
def data_fname(fname):
if is_frozen:
return os.path.join(sys._MEIPASS, "data", fname)
else:
return os.path.join(MYDIR, "data", fname)
def main():
print("This application is %s frozen" %
("" if is_frozen else "not"))
print("executable =", sys.executable,
"File =", __file__,
"mydir =", MYDIR)
if is_frozen:
print("MEIPASS", sys._MEIPASS)
fname = data_fname("tst.txt")
print("will open", fname)
with open(fname) as fin:
print(fin.read())
time.sleep(5) # this shall allow to view the console e.g. on windows if clicking on the executable.
if __name__ == "__main__":
main()
now create a directory data and place a file "tst.txt"
data/tst.txt
Hello world
Now call
pyinstaller -F minitst.py --add-binary data:data -c
and call dist/minitst from a console.
The output should look like:
This application is frozen
executable = /home/gelonida/so/pyinst/dist/minitst File = minitst.py mydir = /home/gelonida/so/pyinst
MEIPASS /tmp/_MEIKGqah9
will open /tmp/_MEIKGqah9/data/tst.txt
Hello
Now concerning your code.
I compacted the code to determine the datadir a little, but it is the same logic as in the upper example
import os, sys
from con import * # this is just a configuration file that has g='play' in it.
import subprocess
basedir = getattr(sys, "_MEIPASS", os.path.realpath(os.path.dirname(__file__)))
f=open('testfile1','w')
f.write('This has worked')
f.close()
file=os.path.join(basedir, 'data/bell.mp3')
if 'play' == g:
subprocess.call(['/usr/bin/cvlc',file])

Pyinstaller app is accessing txt files, but not writing to them (works before app compilation)

settings.txt is stored and accessed within the compiled single file app, but it's not being written to. This works prior to Pyinstaller compilation when the file is in the same directory as the script.
The app is compiled from the terminal:
pyinstaller script.spec script.py --windowed --onefile
a.datas is set in the spec file as:
a.datas += [(‘settings.txt’,’/path/to/settings.txt’, "DATA”)]
and the file is read properly within the app:
with open(resource_path('settings.txt'), 'r') as f2
However, the file isn’t updated when attempting to overwrite the file:
def OnExit(self, event):
with open(resource_path('settings.txt'), 'w') as f2:
f2.write('update')
self.Destroy()
resource_path is defined as:
def resource_path(relative_path):
""" Get absolute path to resource, works for dev and for PyInstaller """
try:
# PyInstaller creates a temp folder and stores path in _MEIPASS
base_path = sys._MEIPASS
except Exception:
base_path = os.environ.get("_MEIPASS2", os.path.abspath("."))
return os.path.join(base_path, relative_path)
If you are on Windows, _MEIPASS returns the "short" name for the path in case that any component of it is more than 8 characters long. So, to test that this is the issue, try to make it a one-folder frozen app and then move it in a simple and short path: e.g., C:/test.
If this is the issue, you can workaround the problem by retrieving the long path using something like:
if hasattr(sys, '_MEIPASS'):
import win32api
sys_meipass = win32api.GetLongPathName(sys._MEIPASS)
I wanted to share my solution, which simultaneously addresses many issues with relative paths in general (see function __doc__string).
I have a module named top_level_locator.py, with a modified function module_path as seen in other answers which takes a relative_path.
usage in other .py files:
from top_level_locator import module_path
resource_location = module_path(relative_path = 'resource.ext')
import sys
from pathlib import Path
from inspect import getsourcefile
def module_path(relative_path):
"""
Combine top level path location, in this project app.py folder because it serves as main/entry_point, with user relative path.
NOTE: top_level_locator.py should be in same folder as entry_point.py(/main.py) script
- TEST this with executable
- TEST this without executable
NOTE: care with use of __file__ as it comes with unwarranted side effects when:
- running from IDLE (Python shell), no __file__ attribute
- freezers, e.g. py2exe & pyinstaller do not have __file__ attribute!
NOTE: care with use of sys.argv[0]
- unexpected result when you want current module path and get path where script/executable was run from!
NOTE: care with use of sys.executable
- if non-frozen application/module/script: python/path/python.exe
- else : standalone_application_executable_name.exe
"""
# 0 if this module next to your_entry_point.py (main.py) else += 1 for every directory deeper
n_deep = 1
print('sys.executable:', sys.executable)
print(' sys.argv[0]:', Path(sys.argv[0]).parents[n_deep].absolute() / sys.argv[0])
print(' __file__:', __file__)
print(' getsourcefile:', Path(getsourcefile(lambda:0)).parents[n_deep].absolute())
if hasattr(sys, "frozen"):
# retreive possible longpath if needed from _MEIPASS: import win32api;
# sys_meipass = win32api.GetLongPathName(sys._MEIPASS)
base_path = getattr(sys, '_MEIPASS', Path(sys.executable).parent)
print(' _MEIPASS:', base_path)
return Path(base_path).joinpath(relative_path)
return Path(getsourcefile(lambda:0)).parents[n_deep].absolute().joinpath(relative_path)
if __name__ == '__main__':
module_path()
In non-frozen applications the output will (should) be as such:
sys.executable: C:\Users\<usr_name>\AppData\Local\Programs\Python\Python37\python.exe
sys.argv[0]: c:\Users\<usr_name>\Desktop\<project_name>\<project_code_folder>\app.py
__file__: c:\Users\<usr_name>\Desktop\<project_name>\<project_code_folder>\utils\top_level_locator.py
getsourcefile: c:\Users\<usr_name>\Desktop\<project_name>\<project_code_folder>
In frozen applications:
sys.executable: C:\Users\<usr_name>\Desktop\<project_name>\dist\app.exe
sys.argv[0]: C:\Users\<usr_name>\Desktop\<project_name>\dist\app.exe
__file__: C:\Users\<usr_name>\AppData\Local\Temp\_MEI155562\utils\top_level_locator.pyc
getsourcefile: C:\Users\<usr_name>\Desktop\<project_name>
_MEIPASS: C:\Users\<usr_name>\AppData\Local\Temp\_MEI155562

PyInstaller: problems on update a json file from an EXE program file (-in the next relunch, json file it's not been update)

I'm testing an EXE made by pyinstaller.
In the project folder there is folder named config, which contains a json file where the user stores all the information he want about -for the GUI im using tkinter-
But finally after I restart this application and reopen the json file, it's appearing the original file.
I read about to create a new folder in execution time, where I put the origina json file. But i'm not properly satisfied with this solution.
Please any help would be appreciated
Update:
Here is the project structure:
/config
|----config.json
/modules
|----admin
|----core
|----graphwo
init.py
The code execute well, except that I want user save their info inside the config.json file in other word, in execution time. But because the PyInstaller I've used is --onefile that's not permiting to update the config.json file
Update II:
Also I have this code which gets the current path at execution time of each file -images, data and json files- the application needs:
def getPathFileAtExecution(relative):
try:
base_path = sys._MEIPASS
except Exception:
base_path = os.path.abspath(".")
return os.path.join(base_path, relative)
I trace any steps of the program when it's calling the json file for read and write. But after it finishes and restart againg, all changes made previously are not reflected.
May is more clear now?
First check that you pointing into a right path. Some operating systems responds differently for some system variables and function calls.
import sys
import os
if getattr(sys,'frozen',False):
current_path = os.path.dirname(sys.executable)
else:
current_path = os.path.dirname(os.path.realpath(__file__))
config_json_file_path = os.path.join(current_path, 'config', 'config.json')
print config_json_file_path
import os
import sys
if getattr(sys, 'frozen', False):
# we are running in a |PyInstaller| bundle
base_path = sys._MEIPASS
extDataDir = os.getcwd()
print base_path
print extDataDir
else:
# we are running in a normal Python environment
base_path = os.getcwd()
extDataDir = os.getcwd()
The sys._MEIPASS variable is where your app's bundled files are run while your program is running. This is different from where your application lives. In order for your program to find and manipulate that non bundled .json file I have used os.getcwd() to get the folder where your application lives.
The os.getcwd() gets the current working directory that your executable is in. Then if your .json file is in a folder called config and that folder is in the current working directory of where your exe is run from, you would use
ext_config = os.path.join(extDataDir, 'config', 'your.json')

Categories