I'm building a package on Python 2.7.6 32bit Windows 32
The only definitive source of some components of a package is an svn 'share'. The common practice in this company is to include that into your project the using svn:externals.
The normal way to build this package is:
python setup.py bdist_wheel
All appears normal on my workstation (where I checked out the code with TortoiseSVN), but when I run the same process on Jenkins the bdist_wheel process does not include any .py file that was sourced via svn:externals.
After reading through the documentation, this appears to be because of a feature which identifies which scripts are part of the package based on which files are tracked by SVN. It appears that as a consequence of how Jenkins checks out the files, the bdist_wheel sees that I'm using SVN and assumes that it knows how to determine which files are tracked, but gets the answer wrong.
What I need is a way to stop the bdist_wheel command from trying to guess which files I care about (I actually want every .py file in the project to be included, regardless of how it's been brought in)
I tried tried specifying the files I needed using a MANIFEST.in file, but it did not work.
recursive-include externals *.py
In this example, 'externals' is a top-level directory in my source-tree which contains an init.py file and a bunch of svn:external'd directories. Only the init file can be seen in the built whl file.
Unfortunately this makes the .py files behave as if they were data, in the log I can see this:
copying build\lib\externals\security\credentials.py -> build\bdist.win32\wheel\foopackage-0.0.4.data\..\externals\security
That's obviously not a real solution!
Pip, Virtualenv and all relevant tools are at the latest stable versions.
It turns out that this problem is caused by Jenkins using a very old SVN standard (1.4) for it's own repositories. Switching to 1.7 corrects this behavior.
Related
How can I build an .egg file include a '.so' file?
here is my file tree:
--setup.py
--__init__.py
--mcpack.py`
--mcpack.so
how can I make a package with mcpack.so in it?
I've tried using pack_data, it turns out .so file is packaged but not findable in python file.
Put it in native_libs.txt and build the egg with bdist_egg, see this section of the documentation. You should, however, switch to wheels instead of eggs.
Shared object files are only distributable if you can guarantee the exact same tool chain that was used to build your .so is also present in every other distribution that is going to run it - this is highly unlikely and would break the distributable nature of egg files, that is unless you really only want to target a specific version of a Linux distribution.
Your egg, if installed in any other distribution than the one the .so was built on will not be able to be used.
If you want to build a distributable package containing native code have a look at wheel files and for wheels that can be distributed to all Linux OSs, have a look at manylinux.
We've had the case where we released a Python source distribution (sdist) file on PyPI, and there was an issue that was only noticed after making the release. Namely for some files, the permissions were set to not allow read by all, and then in some cases those files couldn't be read on user machines after they installed the package with sudo.
We're considering starting our own script to check an sdist for this issue and to run it before uploading to PyPI.
Is there a tool (or something in setuptools or twine or somewhere) to run this or other basic checks on an sdist before uploading it available already?
In this case, the application consists of one or more Python files, plus a settings.ini file. Now the Python files when being installed need to be installed in ~/.hg (as default) or prompted where the user want them installed. The installation also requires text to be appended to files like hgrc.
Is there already a specific Python package that does all of this, or if anyone has any experience in this area please share.
As far as I have looked, Python packaging refers to setuptools and easy_install.
The basis for packaging is a setup.py file. A problem with this is that such a setup file is used for a couple of dissimilar tasks:
Generating documentation.
Creating a release (source/binary).
Actually installing the software.
Combining these tasks in one file is a bit of a hazard and leads to problems now and then.
or distutils, but I am not sure if these packages support the notion of user prompting and deployment like appending text to existing files, and creating new ones.
I would include a custom script (bin/ command) which will poke the users' .hgrc and others. Doing it without the user consent would be rude.
User story
Install package: easy_install pkgname and this deploys myproject-init-hg (UNIX executable, can be also written in Python)
The installation finished and tells the user to run commmand myproject-init-hg
setup.py include mechanism to distribute and deploy bin/ style scripts.
I have a "standard" python package layout like this:
setup.py - using setuptools
README
src/moduleA
test/
However, when I execute setup.py it decides to create the directory src/moduleA.egg-info.
The question is, do I need to worry about the contents of this directory and check it in with the rest of my code, or should I just rely on setuptools/distribute to regenerate it? It seems that all the information in the .egg-info directory comes from the config in setup.py anyway.
The automatically generated bits don't need to be checked in, unless you're actually extending setuptools itself as part of your build process.
However, if you're putting files of your own in .egg-info (like i18n resources for EggTranslations), then those should definitely be checked in, since setuptools obviously isn't going to be able to regenerate them for you. ;-)
I have questions about egg files in Python.
I have much Python code organized by package and I'm trying to create egg files.
I'm following instructions, but they are very common.
According to that, it seems I need to have a setup.py file.
Would you please tell me what I need to put into setup.py file and where it should reside?
I suppose it's enough to create setup.py and then start "setup.py bdist_egg" for getting egg file. Could you please confirm?
Is it possible to include only .pyc files into egg file?
Having .egg file how I can just start the code from it without unpacking like java -jar <jar file> does?
You are reading the wrong documentation. You want this: https://setuptools.readthedocs.io/en/latest/setuptools.html#develop-deploy-the-project-source-in-development-mode
Creating setup.py is covered in the distutils documentation in Python's standard library documentation here. The main difference (for python eggs) is you import setup from setuptools, not distutils.
Yep. That should be right.
I don't think so. pyc files can be version and platform dependent. You might be able to open the egg (they should just be zip files) and delete .py files leaving .pyc files, but it wouldn't be recommended.
I'm not sure. That might be “Development Mode”. Or are you looking for some “py2exe” or “py2app” mode?
For #4, the closest thing to starting java with a jar file for your app is a new feature in Python 2.6, executable zip files and directories.
python myapp.zip
Where myapp.zip is a zip containing a __main__.py file which is executed as the script file to be executed. Your package dependencies can also be included in the file:
__main__.py
mypackage/__init__.py
mypackage/someliblibfile.py
You can also execute an egg, but the incantation is not as nice:
# Bourn Shell and derivatives (Linux/OSX/Unix)
PYTHONPATH=myapp.egg python -m myapp
rem Windows
set PYTHONPATH=myapp.egg
python -m myapp
This puts the myapp.egg on the Python path and uses the -m argument to run a module. Your myapp.egg will likely look something like:
myapp/__init__.py
myapp/somelibfile.py
And python will run __init__.py (you should check that __file__=='__main__' in your app for command line use).
Egg files are just zip files so you might be able to add __main__.py to your egg with a zip tool and make it executable in python 2.6 and run it like python myapp.egg instead of the above incantation where the PYTHONPATH environment variable is set.
More information on executable zip files including how to make them directly executable with a shebang can be found on Michael Foord's blog post on the subject.
I think you should use python wheels for distribution instead of egg now.
Wheels are the new standard of python distribution and are intended to
replace eggs. Support is offered in pip >= 1.4 and setuptools >= 0.8.