I am writing a program that has a modular approach to its components. That is, I hope to be able to add modules after the main program has been distributed. I have been using 'importlib' to get everything working, and it works fine in my development environment.
However, when it comes to packaging the project up, maybe using PyInstaller, I am not sure how to package the additional modules so that they can be added to an existing installation (at a later date), but without them being .py files. Is there some way round this? Maybe add the models as .pyc?
This is the first time i have tried anything like this.
thanks
Related
I've made this question because I had to go through the whole process of creating my own application using Apple's somewhat lacking documentation, and without the use of py2app. I wanted to create the whole application structure so I know exactly what was inside, as well as create an installer for it. The latter of these is still a mystery, so any additional answers with information on making a custom installer would be appreciated. As far as the actual "bundle" structure goes, however, I think I've managed to get the basics down. See the answer below.
Edit: A tutorial has been linked at the end of this answer on using PyInstaller; I don't know how much it helps as I haven't used it yet, but I have yet to figure out how to make a standalone Python application without the use of a tool like this and it may just be what you're looking for if you wish to distribute your application without relying on users knowing how to navigate their Python installations.
A generic application is really just a directory with a .app extension. So, in order to build your application, just make the folder without the extension first. You can rename it later when you're finished putting it all together. Inside this main folder will be a Contents folder, which will hold everything your application needs. Finally, inside Contents, you will place a few things:
Info.plist
MacOS
Resources
Frameworks
Here you can find some information on how to write your Info.plist file. Basically, this is where you detail information about your application.
Inside the MacOS you want to place your main executable. I'm not sure that it matters how you write it; at first, I just had a shell script that called python3 ./../Resources/MyApp.py. I didn't think this was very neat though, so eventually I called the GUI from a Python script which became my executable (I used Tkinter to build my application's GUI, and I wrote several modules which I will get to later). So now, my executable was a Python script with a shebang pointing to the Python framework in my application's Frameworks folder, and this script just created an instance of my custom Tk() subclass and ran the mainloop. Both methods worked, though, so unless someone points out a reason to choose one method over the other, feel free to pick. The one thing that I believe is necessary, is that you name your executable the SAME as your application (before adding the .app). That, I believe, is the only way that MacOS knows to use that file as your application's executable. Here is a source that describes the bundle structure in more detail; it's not a necessary read unless you really want to get into it.
In order to make your executable run smoothly, you want to make sure you know where your Python installation is. If you're like me, the first thing you tried doing on your new Mac was open up Terminal and type in python3. If this is the case, this prompted you to install the Xcode Command Line tools, which include an installation of Python 3.8.2 (most recent on Xcode 12). Then, this Python installation would be located at /usr/bin/python3, although it's actually using the Python framework located at
/Applications/Xcode.app/Developer/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3
I believe, but am NOT CERTAIN, that you could simply make a copy of this framework and add it to your Frameworks folder in order to make the app portable. Make a copy of the Python3.framework folder, and add it to your app's Frameworks folder. A quick side note to be wary of; Xcode comes packaged with a lot of useful tools. In my current progress, the tool I am most hurting for is the Fortran compiler (that I believe comes as a part of GCC), which comes with Xcode. I need this to build SciPy with pip install scipy. I'm sure this is not the only package that would require tools that Xcode provides, but SciPy is a pretty popular package and I am currently facing this limitation. I think by copying the Python framework you still lose some of the symlinks that point to Xcode tools, so any additional input on this would be great.
In any case, locate the Python framework that you use to develop your programs, and copy it into the Frameworks folder.
Finally, the Resources folder. Here, place any modules that you wrote for your Python app. You also want to put your application's icon file here. Just make sure you indicate the name of the icon file, with extension, in the Info.plist file. Also, make sure that your executable knows how to access any modules you place in here. You can achieve this with
import os
os.chdir('./../Resources')
import MyModules
Finally, make sure that any dependencies your application requires are located in the Python framework site-packages. These will be located in Frameworks/Python3.framework/Versions/3.X.Y/lib/python3.x.y/site-packages/. If you call this specific installation of Python from the command line, you can use path/to/application/python3 -m pip install package and it should place the packages in the correct folder.
P.S. As far as building the installer for this application, there are a few more steps needed before your application is readily downloaded. For instance, I believe you need to use the codesign tool in order to approve your application for MacOS Gatekeeper. This requires having a developer license and manipulating certificates, which I'm not familiar with. You can still distribute the app, but anyone who downloads it will have to bypass the security features manually and it will seem a bit sketchy. If you're ready to build the installer (.pkg) file, take a look at the docs for productbuild; I used it and it works, but I don't yet know how to create custom steps and descriptions in the installer.
Additional resources:
A somewhat more detailed guide to the anatomy of a macOS app
A guide I found, but didn't use, on using codesign to get your app past Gatekeeper
A RealPython tutorial I found on using PyInstaller to build Python-based applications for all platforms
I have tried many times to use a compiler like cx_freeze and other programs, but for some reason nothing seems to be working. I made a little game which I want to send to a friend, but he needs python installed.
Can't I just put python.exe and pygame into the folder that I will send my friend and won't python be installed then, and all he needs to do is run the program .py and it will work? Sorry if I'm not being clear.. I'm just trying to find a simple way to compile my code to let users not waste time on downloading pygame and python.
py2exe allows you to package python applications for Windows. Right now it supports everything from 2.4 - 3.1 of python. You do however need to be able to redistribute MSVCR90.dll.
There are a range of distribution tools and you can find a list here.
Since you've had difficultly with several tools now updating your question with error codes and speific problems will yeild better responses.
From my knowledge, just putting a bunch of your stuff in one folder and sending doesn't work. It would be easier to make a .exe
That way your stuff will be protected, and users can easily start it. Otherwise idk. Try using pyinstaller again. It should work if you have a python.x
CX_Freeze is known for having many bugs and problems, Pyg.exe is new to me also. Your best bet is just keep trying until you find a solution.
Putting python.exe and your script together in a folder will not work for distribution. You require all Python dependencies - at best your would need to include all of your Python folder, and it still might not work. The best method would be compilation or packaging with programs such as py2exe, cx_freeze, Cython, pyg.exe, etc.
I would like to be able to use the services that the Blockcypher module provides for my programme, however i have (at least i think) downloaded the correct module package but cant get it to integrate with my Python on my Computer. I am fairly new to python so I have no idea on where to even start tackling this problem.
Modules, regardless of where you've got hold of them, will be searched for in the sys.path. If nothing is found there, they will be looked up in the current directory.
When you download some code directly it will be a good first guess to place it in the directory of the script from where you are using the download. If it's just a .py-file, place it there. If it's an archive with a directory, then place the directory there (not the files).
Generally, you should prefer installing modules via a package manager such as pip or conda. Such package managers take care of placing modules properly for usage with your Python installation from wherever you will write your script. They also provide support for updating these modules to newer versions later.
Update: If you cannot make anything from this remarks, you should first read the section on modules in the Python tutorial, or even work thru the full tutorial or thru a good book (or any other ;) to get a smooth entry into the friendly world of Python programming.
Update (2023): The Dive Into Python link above is outdated, so here is the updated link to this great resource:
https://diveintopython3.problemsolving.io
I think it's still the best beginner's resource, but, well, here are many more:
https://wiki.python.org/moin/IntroductoryBooks
I'm trying to use pyinstaller to build an exe from my python code. One of the modules I'm using is pubsub (pypubsub really. It used to be a part of wxpython). I'm getting errors when I try to run the exe. It complains "ImportError: No module named listenerimpl".
I've seen some articles about getting wx.lib.pubsub to work (it has known issues with pyinstaller). I've tried the solutions presented in those articles (slightly modified to account for it not being a part of wx anymore) but no luck.
I can get past the initial "ImportError: No module named listenerimpl" error by adding the path to the right listenerimpl (the kwargs one) to the list of files for Analysis in my spec file but then I hit further errors on importing "publisher". That error isn't fixed by adding its path in the spec file.
I think the solution shouldn't involve adding the path to listenerimpl.py in my spec file... but I'm not sure how to get this working happily.
extra info
using pubsub version: 3.1.1b1.201005.r243
using pyinstaller version: 2.0
platform: win7
pubsub problems solved (although exe still not running).
So if you look at comments here (especially comment #15 by sebastian.hilbert) it mostly solves the problems with some tweaking.
The necessary tweaks.
Change the names on the hook files to hook-pubsub.core.py and hook-pubsub.setuparg1.py.
Internal to those files you should get rid of references to wx.
Enable the hook files. You can do this one of two ways. The easy way is to drop these new hook files into 'pyinstaller-2.0\PyInstaller\hooks' which is where pyinstaller looks for hooks by default. The clean/nice way to do this is to put these hooks into their own folder and add that folder as hookspath in your spec file.
NB: It was not clear to me how to add a hookspath. In your specfile, in the call to Analyze, there is a hookspath arg. It wants a list not a string. So you want to do something like hookspath=['path1', 'path2', etc].
NB2: Additionally if you ask for "path.dirname(path.abspath(__file__))" you will get the directory for pyinstaller not the location where your spec file lives.
For those coming in late to this like myself, there's a new recommended procedure.
I recommend using oliver's PyPubSub. For Python 2.7, use his 3.4.2 release. Make sure you're only using this PubSub. This means if you have any other PubSub's in your env/lib/site-packages - not counting the one bundled with wxPython - you'll want to get rid of them. They WILL cause problems.
Despite what you may have read, do not use from pubsub import setupkwargs.
Don't add any new hooks to PyInstaller. They've already got it covered.
That's it. You're done.
Run PyInstaller. Check for any bugs in the build. Hopefully, PubSub won't be one of them.
My Delphi program uses Python4Delphi, and the Python script uses some standard libs. When deploying my program I don't want an entire Python installation, and it must work with python27.dll. What are the minimal set of necessary files? The document in Python4Delphi is dated and it's not clear to me...
Thanks for your help.
When I did this, I made the list myself, of what I needed for my embedded python application to work.
I remember this worked with python15.dll:
PythonXX.dll should work, without any other external files other than the Visual C++ Runtime DLLs, which require a side-by-side manifest (see the p4d wiki page) to work.
If you want to IMPORT something, then you need to ship it and anything it depends on. That means, either you pick part of the python standard libraries you want, or you pick all of it. There is no way you need all of Python's standard libraries. But I wouldn't want to live without OS and a a few other key ones. BUt the decision is yours.