Java has the concept of packaging all of the code into a file called a Jar file. Does Python have an equivalent idea? If so, what is it? How do I package the files?
Python doesn't have any exact equivalent to a .jar file.
There are many differences, and without knowing exactly what you want to do, it's hard to explain how to do it. But the Python Packaging User Guide does a pretty good job of explaining just about everything relevant.
Here are some of the major differences.
A .jar file is a compiled collection of classes that can be dropped into your application, or installed anywhere on your CLASSPATH.
In Python:
A .py (or .pyc) module can be dropped into your application, or installed anywhere on your sys.path, and it can be imported and used.
A directory full of modules can be treated the same way; it becomes a package (or, if it doesn't contain an __init__.py, it merges with other directories of the same name elsewhere on sys.path into a single package).
A .zip archive containing any number of modules and packages can be stored anywhere, and its path added to your sys.path (e.g., at runtime or via PYTHONPATH) and all of its contents become importable.
Most commonly, you want things to be installed into a system, user, or virtualenv site-packages directory. The recommended way to do that is to create a pip-compatible package distribution; people then install it (and possibly automatically download it from PyPI or a private repo) via pip.
pip does a lot more than that, however. It also allows you to manage dependencies between packages. So ideally, instead of listing a bunch of prereqs that someone has to go download and install manually, you just make them dependencies, and someone just has to pip install your-library. And it keeps track of the state of your site-packages, so you can uninstall or upgrade a package without having to track down the specific files.
Meanwhile, in Java, most .jar files are cross-platform; build once, run anywhere. A few packages have JNI native code and can't be used this way, but it's not the norm.
In Python, many packages have C extensions that have to be compiled for each platform, and even pure-Python packages often need to do some install-time configuration. And meanwhile, "compiling" pure Python doesn't do anything that can't be done just as well at runtime. So in Python, you generally distribute source packages, not compiled packages.
However, .wheel is a binary package format. You can pip wheel to build binary packages for different targets from the source package; then, if someone tries to pip install your package, if there's a wheel for his system, that will be downloaded and installed.
Easy Install from setup_tools defines the .egg format for deploying Python libraries or applications. While similar to JAR, it is nowhere spread as universally as JARs in Java world. Many people just deploy the .py files.
A newer format, intended to supersede eggs, is wheel.
Though it's not a perfect susbstitute of jar due to portability issues, I would add the "auto-extracting" archive way.
One possibility is "makeself": https://makeself.io/
But if you don't need to package external files, and if you like KISS approach, the following is a nice and clean alternative:
The following is taken from Asim Jalis's website.
How to deploy a Python application as a zip file
Create a file __main__.py containing:
print "Hello world from Python"
Zip up the Python files (in this case just this one file) into app.zip by typing:
zip app.zip *
The next step adds a shebang to the zip file and saves it as app—at this point the file app is a zip file containing all your Python sources.
echo '#!/usr/bin/env python' | cat - app.zip > app
chmod 755 app
That’s it. The file app is now have a zipped Python application that is ready to deploy as a single file.
You can run app either using a Python interpreter as:
python app
Or you can run it directly from the command line:
./app
Reference: https://gist.github.com/asimjalis/4237534
Related
Say you have a project, and you want to export it so that you can run it on another machine that:
You don't have root access on
You cannot assume any python packages to be installed other than python itself (not even pip)
Is there a way to export the project, even if it is just a simple script, with everything that it imports, and then everything that the imports need etc.
For example, my project uses a library called python-telegram-bot. It has a list of requirements, and I have tried running pip -r requirements.txt --target myapp to install the requirements into the app's folder, but this is not recursive. For example, requests is not in the library, yet it is needed by the app. And if I manually add requests, there are things that requests needs that aren't part of that.
Is there a way to collect every last bit of requirements into a single folder so that my script functions on an entirely vanilla installation of python?
Pyinstaller creates an executable and you can either roll all dependencies into 1 file (makes it a bit slow to load though) or create a folder with all the packages, modules, imports etc. your script will need to run on any machine. You can Google the docs for pyinstaller, it's all pretty well covered.
Hope that helps, Kuda
I have a simple python shell script (no gui) who uses a couple of dependencies (requests and BeautifulfSoup4).
I would like to share this simple script over multiple computers. Each computer has already python installed and they are all Linux powered.
At this moment, on my development environments, the application runs inside a virtualenv with all its dependencies.
Is there any way to share this application with all the dependencies without the needing of installing them with pip?
I would like to just run python myapp.py to run it.
You will need to either create a single-file executable, using something like bbfreeze or pyinstaller or bundle your dependencies (assuming they're pure-python) into a .zip file and then source it as your PYTHONPATH (ex: PYTHONPATH=deps.zip python myapp.py).
The much better solution would be to create a setup.py file and use pip. Your setup.py file can create dependency links to files or repos if you don't want those machines to have access to the outside world. See this related issue.
As long as you make the virtualenv relocatable (use the --relocatable option on it in its original place), you can literally just copy the whole virtualenv over. If you create it with --copy-only (you'll need to patch the bug in virtualenv), then you shouldn't even need to have python installed elsewhere on the target machines.
Alternatively, look at http://guide.python-distribute.org/ and learn how to create an egg or wheel. An egg can then be run directly by python.
I haven't tested your particular case, but you can find source code (either mirrored or original) on a site like github.
For example, for BeautifulSoup, you can find the code here.
You can put the code into the same folder (probably a rename is a good idea, so as to not call an existing package). Just note that you won't get any updates.
I have a simple script that has a dependency on dnspython for parsing zone files. I would like to distribute this script as a single .py that users can run just so long as they have 2.6/2.7 installed. I don't want to have the user install dependencies site-wide as there might be conflicts with existing packages/versions, nor do I want them to muck around with virtualenv. I was wondering if there was a way to embed a package like dnspython inside the script (gzip/base64) and have that script access that package at runtime. Perhaps unpack it into a dir in /tmp and add that to sys.path? I'm not concerned about startup overhead, I just want a single .py w/ all dependencies included that I can distribute.
Also, there would be no C dependencies to build, only pure python packages.
Edit: The script doesn't have to be a .py. Just so long as it is a single executable file.
You can package multiple Python files up into a .egg. Egg files are essentially just zip archives with well defined metadata - look at the setuptools documentation to see how to do this. Per the docs you can make egg files directly executable by specifying the entry point. This would give you a single executable file that can contain your code + any other dependencies.
EDIT: Nowadays I would recommend building a pex to do this. pex is basically an executable zip file with non stdlib dependencies. It doesn't contain a python distribution (like py2app/py2exe) but holds everything else and can be built with a single command line invocation. https://pex.readthedocs.org/en/latest/
The simplest way is just to put your python script named __main__.py with pure Python dependencies in a zip archive, example.
Otherwise PyInstaller could be used to produce a stand-alone executable.
please don't do this. If you do DO NOT make a habit of it.
pydns is BDS licensed but if you try to "embed" a gpl module in this way you could get in trouble
you can learn to use setuptools and you will be much happier in the long run
setuptools will handle the install of dependencies you identified (I'm not sure if the pydns you are using is pure python so you might create problems for your users if you try to add it yourself without knowing their environment)
you can set a url or pypi so that people could upgrade your script with easy_install -U
In Trac on Admin -> Plugins there is an option to install Plug-ins. Now this option expect you to upload an Python egg.
This would be all well but for the fact that all the Trac plug-ins I found are either plain .py files or zip files and are incompatible with the upload function (I tried it).
This leaves my with a bunch of questions:
Are there any Trac plug-ins which come as an Python egg?
What is an (Trac compatible) Python egg?
Is it difficult to repackage an .py file into a Trac compatible Python egg?
If not: how is it done?
Haven't used trac for a year, but what I remember is that most plugins are available trough subversion and already packed as an egg (which is kind of an installer in the python world, but I am not very familiar with the concept).
Most plugins are available at http://trac-hacks.org/ and the easiest way to install a plugin is
easy_install http://svn.domain.tdl/path/to/plugin/
the folder should contain a setup.py and a setup.cfg file.
easy_install checks the files out from svn and installs the plugin. You can find details here: http://trac.edgewall.org/wiki/TracPlugins
If the plugin makes database changes you have to call
trac-admin upgrade
from console.
http://trac.edgewall.org/wiki/TracAdmin
If I remember right, the install through the webinterface installs the plugin locally (for the instance) while easy_install installs it globally (for all running trac sites) and is the more common way to install a plugin.
Hint: After every plugin install you have to restart trac
Hint2: Most plugins don't tell you how to install and only give a link to the root of their svn. You only have to browse the svn folder and locate the folder containing the setup.py.
The rest is done with easy_install.
Example:
Plugin: http://trac-hacks.org/wiki/GoogleChartPlugin
Wiki pages tells you:
You can check out GoogleChartPlugin from here using Subversion, or browse the source with Trac.
where here links to http://trac-hacks.org/svn/googlechartplugin/
The svn contains two versions. Browse to http://trac-hacks.org/svn/googlechartplugin/0.11/trunk/ and copy the path.
Then do
easy_install http://trac-hacks.org/svn/googlechartplugin/0.11/trunk/
Answers to your questions in order.
Python eggs are binary packages which contain the code for the application and some metadata. They're not very different from debs or rpms in this sense. The egg itself is basically just a zip file which contains all the above mentioned files with specific names and layouts. For more information on eggs (the format and how to create them), please refer to http://www.ibm.com/developerworks/library/l-cppeak3.html. It's probably a little dated since the future (and present) of python packaging is a little hazy.
A trac plugin is a python program that uses the Trac plugin API to extend the functionality of trac. It can be packaged as an egg.
If your package is properly laid out and contains a setuptools/distribute setup.py file, then issuing the command python setup.py bdist_egg will create a .egg file for you. For details on this please refer to this(a little dated but complete) and this (more upto date but still in progress). The Trac Growl plugin mentions this on it's documentation page.
Please see above point.
I have questions about egg files in Python.
I have much Python code organized by package and I'm trying to create egg files.
I'm following instructions, but they are very common.
According to that, it seems I need to have a setup.py file.
Would you please tell me what I need to put into setup.py file and where it should reside?
I suppose it's enough to create setup.py and then start "setup.py bdist_egg" for getting egg file. Could you please confirm?
Is it possible to include only .pyc files into egg file?
Having .egg file how I can just start the code from it without unpacking like java -jar <jar file> does?
You are reading the wrong documentation. You want this: https://setuptools.readthedocs.io/en/latest/setuptools.html#develop-deploy-the-project-source-in-development-mode
Creating setup.py is covered in the distutils documentation in Python's standard library documentation here. The main difference (for python eggs) is you import setup from setuptools, not distutils.
Yep. That should be right.
I don't think so. pyc files can be version and platform dependent. You might be able to open the egg (they should just be zip files) and delete .py files leaving .pyc files, but it wouldn't be recommended.
I'm not sure. That might be “Development Mode”. Or are you looking for some “py2exe” or “py2app” mode?
For #4, the closest thing to starting java with a jar file for your app is a new feature in Python 2.6, executable zip files and directories.
python myapp.zip
Where myapp.zip is a zip containing a __main__.py file which is executed as the script file to be executed. Your package dependencies can also be included in the file:
__main__.py
mypackage/__init__.py
mypackage/someliblibfile.py
You can also execute an egg, but the incantation is not as nice:
# Bourn Shell and derivatives (Linux/OSX/Unix)
PYTHONPATH=myapp.egg python -m myapp
rem Windows
set PYTHONPATH=myapp.egg
python -m myapp
This puts the myapp.egg on the Python path and uses the -m argument to run a module. Your myapp.egg will likely look something like:
myapp/__init__.py
myapp/somelibfile.py
And python will run __init__.py (you should check that __file__=='__main__' in your app for command line use).
Egg files are just zip files so you might be able to add __main__.py to your egg with a zip tool and make it executable in python 2.6 and run it like python myapp.egg instead of the above incantation where the PYTHONPATH environment variable is set.
More information on executable zip files including how to make them directly executable with a shebang can be found on Michael Foord's blog post on the subject.
I think you should use python wheels for distribution instead of egg now.
Wheels are the new standard of python distribution and are intended to
replace eggs. Support is offered in pip >= 1.4 and setuptools >= 0.8.