Looking to use a include a redis server for storing application specific data with my pyinstaller bundled application.
Before getting into it hands-on, need some guidance.
Are these the steps to follow for it?
(1) Bundle redis-server executable. And run it as a standalone application via some script in my bundled package.
(2) Use redis client packages in python to connect to the redis-server
I guess (2) should surely work. But is there any easy way of doing (1).
You can bundle arbitrary binaries with the --add-binary option on the command line or the binaries argument to the Analysis call in your .spec file. Check out the manual for details, but one example:
pyinstaller -F main.py --add-binary=`which redis-server`:bin
I don't know of a way to run arbitrary executables, but you could have some python code in your app to detect when you're bundled, find the redis binary, and start it up. Again, you can check out the documentation for details on how to go about this but, again, a example of how this could look (optional contextmanager elegance stolen from another answer):
import sys
import os
import subprocess
from contextlib import contextmanager
#contextmanager
def bundledredis():
proc = subprocess.Popen(
[os.path.join(sys._MEIPASS, 'bin', 'redis-server')])
yield
proc.terminate()
#contextmanager
def optional(condition, context_manager):
if condition:
with context_manager:
yield
else:
yield
def realmain():
print('doing stuff')
def main():
with optional(getattr(sys, 'frozen', False), bundledredis()):
realmain()
if __name__ == '__main__':
main()
I use PyCharm/IntelliJ community editions from a wile to write and debug Python scripts, but now I'm trying to debug a Python module, and PyCharm does a wrong command line instruction parsing, causing an execution error, or maybe I'm making a bad configuration.
This is my run/debug configuration:
And this is executed when I run the module (no problems here):
/usr/bin/python3.4 -m histraw
But when I debug, this is the output in the IntelliJ console:
/usr/bin/python3.4 -m /opt/apps/pycharm/helpers/pydev/pydevd.py --multiproc --client 127.0.0.1 --port 57851 --file histraw
/usr/bin/python3.4: Error while finding spec for '/opt/apps/pycharm/helpers/pydev/pydevd.py' (<class 'ImportError'>: No module named '/opt/apps/pycharm/helpers/pydev/pydevd')
Process finished with exit code 1
As you can see, the parameters are wrong parsed, and after -m option a IntelliJ debug script is passed before the module name.
I also tried just put -m histraw in the Script field, but doesn't work, that field is only to put Python script paths, not modules.
Any ideas?
There is another way to make it work.You can write a python script to run your module.Then just configure PyCharm to run this script.
import sys
import os
import runpy
path = os.path.dirname(sys.modules[__name__].__file__)
path = os.path.join(path, '..')
sys.path.insert(0, path)
runpy.run_module('<your module name>', run_name="__main__",alter_sys=True)
Then the debugger works.
In PyCharm 2019.1 (professional), I'm able to select run as module option under configurations, as below
I found it easiest to create a bootstrap file (debuglaunch.py) with the following contents.
from {package} import {file with __main__}
if __name__ == '__main__':
{file with __main__}.main()
For example, to launch locustio in the pycharm debugger, I created debuglaunch.py like this:
from locust import main
if __name__ == '__main__':
main.main()
And configured pycharm as follows.
NOTE: I found I was not able to break into the debugger unless I added a breakpoint on main.main() . That may be specific to locustio, however.
The problem is already fixed since PyCharm 4.5.2. See corresponding issue in PyCharm tracker:
https://youtrack.jetbrains.com/issue/PY-15230
I use nunjucks for templating the frontend in a python project. Nunjucks templates must be precompiled in production. I don't use extensions or asynchronous filters in the nunjucks templates. Rather than use grunt-task to listen for changes to my templates, I prefer to use the nunjucks-precompile command (offered via npm) to sweep the entire templates directory into templates.js.
The idea is to have the nunjucks-precompile --include ["\\.tmpl$"] path/to/templates > templates.js command execute within setup.py so I can simply piggyback our deployer scripts' regular execution.
I found a setuptools override and a distutils scripts argument might serve the right purpose, but I'm not so sure either is the simplest approach to execution.
Another approach is to use subprocess to execute the command directly within setup.py, but I've been cautioned against this (rather preemptively IMHO). I don't really deeply understand why not.
Any ideas? Affirmations? Confirmations?
Update (04/2015): - If you don't have the nunjucks-precompile command available, simply use Node Package Manager to install nunjucks like so:
$ npm install nunjucks
Pardon the quick self-answer. I hope this helps someone out there in the ether. I want to share this now that I've worked out a solution I'm satisfied with.
Here's a solution that's safe and based on Peter Lamut's write-up. Note that this does not use shell=True in the subprocess invocation. You may bypass grunt-task requirements on your python deployment system and also use this for obfuscation and JS packaging all the same.
from setuptools import setup
from setuptools.command.install import install
import subprocess
import os
class CustomInstallCommand(install):
"""Custom install setup to help run shell commands (outside shell) before installation"""
def run(self):
dir_path = os.path.dirname(os.path.realpath(__file__))
template_path = os.path.join(dir_path, 'src/path/to/templates')
templatejs_path = os.path.join(dir_path, 'src/path/to/templates.js')
templatejs = subprocess.check_output([
'nunjucks-precompile',
'--include',
'["\\.tmpl$"]',
template_path
])
f = open(templatejs_path, 'w')
f.write(templatejs)
f.close()
install.run(self)
setup(cmdclass={'install': CustomInstallCommand},
...
)
I think that the link here encapsulates what you are trying to achieve.
I'm quite new to python and just created my first opensource python project - a daemon to create wifi hotspots on linux.
I've used distutils to build the package. To start the daemon once installation is done, I've registered the below script that simply starts the daemon by calling the corresponding python module:
#!/bin/bash
python -m hotspotd $*
And this is the setup.py that registers it:
#INSTALL IT
from distutils.core import setup
s = setup(name='hotspotd',
version='0.1',
description='Small daemon to create a wifi hotspot on linux',
license='MIT',
author='Prahlad Yeri',
author_email='prahladyeri#yahoo.com',
url='https://github.com/prahladyeri/hotspotd',
#py_modules=['hotspotd','cli'],
packages=['hotspotd'],
package_dir={'hotspotd': ''},
package_data={'hotspotd': ['run.dat']},
scripts=['hotspotd']
#data_files=[('config',['run.dat'])],
)
Now, this works fine on my machine and some other machines I've tested. However, as indicated by the open issue on the github, some users are unable to run that script. It gives the error:
No module named hotspotd.main; 'hotspotd' is a package and cannot be directly executed
Apparently it expects the entire package.module syntax which is hotspotd.hotspotd on their setups. However, on my machine the full syntax doesn't work, and only hotspotd works. Whats going on here?
I had to change my script, so that instead of directly passing the module as argument, I import this module in the script itself and call the hotspotd.main function from there:
#!/usr/bin/env python
##author: Prahlad Yeri
##description: Small daemon to create a wifi hotspot on linux
##license: MIT
#python -m hotspotd $*
import hotspotd.main
import sys
import os
import argparse
if __name__ == "__main__":
#check root or not
if os.getenv('USER') != 'root':
print "You need root permissions to do this, sloth!"
sys.exit(1)
parser = argparse.ArgumentParser(description='A small daemon to create a wifi hotspot on linux')
parser.add_argument('-v', '--verbose', required=False, action='store_true')
parser.add_argument('command', choices=['start', 'stop', 'configure'])
args = parser.parse_args()
hotspotd.main.main(args)
I am writing a GUI application using python and Qt. When I launch my application on Mac, the first menu item in the Mac menu bar at the top of the screen is "Python". I would prefer the application name there to be the name of my application. How can I get my program name up there?
The following demo program creates a window with two menus: "Python", and "Foo". I don't like that, because it makes no difference to my users whether I wrote the app in python or COBOL. Instead I want menus "MyApp" and "Foo".
#!/usr/bin/python
# This example demonstrates unwanted "Python"
# application menu name on Mac.
# Makes no difference whether we use PySide or PyQt4
from PySide.QtGui import *
# from PyQt4.QtGui import *
import sys
app = QApplication(sys.argv)
# Mac menubar application menu is always "Python".
# I want "DesiredAppTitle" instead.
# setApplicationName() does not affect Mac menu bar.
app.setApplicationName("DesiredAppTitle")
win = QMainWindow()
# need None parent for menubar on Mac to get custom menus at all
mbar = QMenuBar()
# Add a custom menu to menubar.
fooMenu = QMenu(mbar)
fooMenu.setTitle("Foo")
mbar.addAction(fooMenu.menuAction())
win.setMenuBar(mbar)
win.show()
sys.exit(app.exec_())
How can I change that application menu name on Mac? EDIT: I would prefer to continue to use the system python (or whatever python is on the user PATH) if possible.
You seem to need an OSX .app for this to work, as the Info.plist file in there contains the user-visible name for the application that is put there. This defaults to Python, which is the title you see for the program menu. This blog post outlines the steps you need to take, while the OSX Developer Library has the docs on the property list you need to fill.
I have found the kernel of an answer to this question. Because I want to award the bounty to someone other than myself (I am the OP), please, anyone, take this kernel and elaborate it into a more complete answer of your own.
I can get the application menu to be "MyApp" as follows:
ln -s /System/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python MyApp
./MyApp MyApp.py
There are two elements required to get this to work:
The symbolic link must be named "MyApp" (or whatever you want to appear in the Application Menu)
The symbolic link must point to the Python executable inside the system python app bundle. It does not work if you link to /usr/bin/python, for example.
There must be a clever way to create an app bundle or shell script that exploits this mechanism in a robust way...
If you intend to distribute the app, then symlinking the python binary is not guaranteed to work, considering that typically on development machines the system python is not the default python, and regular users most likely won't have Qt and PyQt installed.
A more reliable approach is to have a native OSX bootstrapping binary that would take care of starting your PyQt app. Both py2app and PyInstaller can generate native wrappers. While py2app worked well for creating an aliased app, i.e. a wrapper that symlinks to existing files on your system, getting it to bundle only the required PyQt dependencies proved to be non-trivial. Moreover the 'aliased app' generated by py2app with the --aliased flag will stop working if you move it to another folder since the symlinks are relative to the folder where you originally ran the build script.
PyInstaller
PyInstaller worked out-of-the-box and I ended up with an OSX bundle that included the dependencies all at around 16MB.
Bundle a python script to a standalone OSX app:
pyinstaller -w --noconfirm -i=myappicon.icns --clean -F myscript.py
This generates a standalone bundle that will display whatever you have in your .setWindowTitle() as the app name in the OSX title bar. The -w switch is important as it will create a MacOS app bundle with a proper .plist file.
NB: For some reason, the version of pyinstaller installed via pip did not work for me. So I removed the original pip version (pip uninstall pyinstaller), and installed the latest develop branch from github with:
pip install -v -e git+https://github.com/pyinstaller/pyinstaller.git#develop#egg=pyinstaller-github
After that it worked like a charm.
Mac Automator
Yet another option to create an app wrapper is to use Automator (type the name in Spotlight), go to File > New > Application > search for and drag Run shell script into the editor and under Shell select bash, python. You can also use the Run Applescript and bootstrap your app using apple script - can be useful if you need, for example, to ask the user's password in order to run with elevated privileges.
A better alternative to Automator is Platypus - a free and open-source app that wraps various types of scripts into OSX apps, and also provides some extra features like gui text output windows, running with admin privileges, setting custom icons, etc. The code is available on github.
There is also the option to create a barebones OSX app in Xcode that will launch your PyQt script (some examples here), and do other custom tasks required by your app.
NOTE: Keep in mind that an app bundle that calls your python code with a predefined interpreter is a 'shallow' app bundle, and will depend on whatever python dependencies you have install locally. Most likely will not work OOTB on someone else's Mac.
Nuitka
You might also want to try Nuitka (pip3 install nuitka) which aims to be a project to build truly native executables by converting your python code to C++ and then compiling it.
An example of building a native app with nuitka:
nuitka3 --follow-imports --python-flag=no_site --verbose --standalone --show-progress --show-modules --output-dir=my_build_dir myscript.py
This will create the mac executable with libraries in the my_build_dir folder. You will need to wrap them yourself in a MacOS app bundle, for example with Platypus or with Automator.
I know this isn't what the OP wants exactly, but figured it could add some value to searching for a solution to this issue.
You can make the file menus appear in your window UI instead of in the native Mac OS X system menubar:
menubar.setNativeMenuBar(False)
This will cause the menubar to appear as it would appear in e.g. Windows.
I would rather end users have access to my Python code so they and see and maybe improve on the code, so I am not interested in options that hide the code. I have found that I can get wx applications (have not tried with Qt) to run with a specified application name using different standard Python interpreters by wrapping them into a bundle as I install them (for example, as part of the conda package install process). Here is short script that creates an .app bundle with a specified name using the Python executing the script. Running the script using the created soft link (see pyAlias, below) as in
./MyApplication.app/Contents/MacOS/MyApplication /path/MyApplication.py
does the job.
from __future__ import division, print_function
import os,sys
appName = 'MyApplication' # name of app
scriptdir = '.'
iconfile = os.path.join(scriptdir,'MyApplication.icns') # optional icon file
if __name__ == '__main__':
wrapApp = os.path.join(scriptdir,appName+'.app','Contents')
os.makedirs(wrapApp)
wrapPy = os.path.join(wrapApp,'MacOS')
os.makedirs(wrapPy)
fp = open(os.path.join(wrapApp,'PkgInfo'),'w')
fp.write('APPL????\n')
fp.close()
fp = open(os.path.join(wrapApp,'Info.plist'),'w')
fp.write('''<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE plist SYSTEM "file://localhost/System/Library/DTDs/PropertyList.dtd">\n<plist version="0.9">\n<dict>\n <key>CFBundleIconFile</key>\n <string></string>\n <key>CFBundlePackageType</key>\n <string>APPL</string>\n <key>CFBundleGetInfoString</key>\n <string>Created in makeApp.py (Brian Toby/QMake</string>\n <key>CFBundleSignature</key>\n <string>????</string>\n <key>CFBundleExecutable</key>\n <string>{:}</string>\n <key>CFBundleIdentifier</key>\n <string>com.continuum.python</string>\n <key>NSPrincipalClass</key>\n <string>NSApplication</string>\n</dict>\n</plist>\n'''.format(appName))
fp.close()
pyAlias = os.path.join(wrapPy,appName)
pythonpath = os.path.realpath(sys.executable)
os.symlink(pythonpath,pyAlias)
if os.path.exists(iconfile):
shutil.copyfile(iconfile,oldicon)
print('Use',pyAlias,'to run wxPython scripts')
Below is an older answer, which I am leaving for any historical value. In my application now, I use a hybrid between these where I create a drag-and-drop AppleScript bundle (as below) where I place a soft link to Python (named to match my app) as above.
Building on Christopher Bruns answer, as well as the script from "How to create Mac application bundle for Python script via Python", here is a Python script that creates a bundle (app) for an user Python script, which will show the app name rather than "Python" in the menus. To do this, it tries to locate a bundled version of Python, and symlinks to that with the name of the app. I tested it out with a wxpython script, but it should work for Qt as well.
The user script is run from its original location rather than placing it in the app. If you want to place your script(s) into a bundle (along with python) for redistribution, see py2app instead.
#!/usr/bin/env python
'''This creates an app to launch a python script. The app is
created in the directory where python is called. A version of Python
is created via a softlink, named to match the app, which means that
the name of the app rather than Python shows up as the name in the
menu bar, etc, but this requires locating an app version of Python
(expected name .../Resources/Python.app/Contents/MacOS/Python in
directory tree of calling python interpreter).
Run this script with one or two arguments:
<python script>
<project name>
The script path may be specified relative to the current path or given
an absolute path, but will be accessed via an absolute path. If the
project name is not specified, it will be taken from the root name of
the script.
'''
import sys, os, os.path, stat
def Usage():
print("\n\tUsage: python "+sys.argv[0]+" <python script> [<project name>]\n")
sys.exit()
version = "1.0.0"
bundleIdentifier = "org.test.test"
if not 2 <= len(sys.argv) <= 3:
Usage()
script = os.path.abspath(sys.argv[1])
if not os.path.exists(script):
print("\nFile "+script+" not found")
Usage()
if os.path.splitext(script)[1].lower() != '.py':
print("\nScript "+script+" does not have extension .py")
Usage()
if len(sys.argv) == 3:
project = sys.argv[2]
else:
project = os.path.splitext(os.path.split(script)[1])[0]
# find the python application; must be an OS X app
pythonpath,top = os.path.split(os.path.realpath(sys.executable))
while top:
if 'Resources' in pythonpath:
pass
elif os.path.exists(os.path.join(pythonpath,'Resources')):
break
pythonpath,top = os.path.split(pythonpath)
else:
print("\nSorry, failed to find a Resources directory associated with "+str(sys.executable))
sys.exit()
pythonapp = os.path.join(pythonpath,'Resources','Python.app','Contents','MacOS','Python')
if not os.path.exists(pythonapp):
print("\nSorry, failed to find a Python app in "+str(pythonapp))
sys.exit()
apppath = os.path.abspath(os.path.join('.',project+".app"))
newpython = os.path.join(apppath,"Contents","MacOS",project)
projectversion = project + " " + version
if os.path.exists(apppath):
print("\nSorry, an app named "+project+" exists in this location ("+str(apppath)+")")
sys.exit()
os.makedirs(os.path.join(apppath,"Contents","MacOS"))
f = open(os.path.join(apppath,"Contents","Info.plist"), "w")
f.write('''<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>English</string>
<key>CFBundleExecutable</key>
<string>main.sh</string>
<key>CFBundleGetInfoString</key>
<string>{:}</string>
<key>CFBundleIconFile</key>
<string>app.icns</string>
<key>CFBundleIdentifier</key>
<string>{:}</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>{:}</string>
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>{:}</string>
<key>CFBundleSignature</key>
<string>????</string>
<key>CFBundleVersion</key>
<string>{:}</string>
<key>NSAppleScriptEnabled</key>
<string>YES</string>
<key>NSMainNibFile</key>
<string>MainMenu</string>
<key>NSPrincipalClass</key>
<string>NSApplication</string>
</dict>
</plist>
'''.format(projectversion, bundleIdentifier, project, projectversion, version)
)
f.close()
# not sure what this file does
f = open(os.path.join(apppath,'Contents','PkgInfo'), "w")
f.write("APPL????")
f.close()
# create a link to the python app, but named to match the project
os.symlink(pythonapp,newpython)
# create a script that launches python with the requested app
shell = os.path.join(apppath,"Contents","MacOS","main.sh")
# create a short shell script
f = open(shell, "w")
f.write('#!/bin/sh\nexec "'+newpython+'" "'+script+'"\n')
f.close()
os.chmod(shell, os.stat(shell).st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
I think you need to do this:
win.setWindowTitle("MyApp")
Edit: This is for PyQt. I don't know the equivalent in PySide.