We Keep Coding

How to detect if module is installed in "editable mode"?

I'm pip-installing my module like so:
cd my_working_dir
pip install -e .
When I later import the module from within Python, can I somehow detect if the module is installed in this editable mode?
Right now, I'm just checking if there's a .git folder in os.path.dirname(mymodule.__file__)) which, well, only works if there's actually a .git folder there. Is there a more reliable way?

Another workaround:
Place an "not to install" file into your package. This can be a README.md, or a not_to_install.txt file. Use any non-pythonic extension, to prevent that file installation. Then check if that file exists in your package.
The suggested source structure:
my_repo
|-- setup.py
`-- awesome_package
|-- __init__.py
|-- not_to_install.txt
`-- awesome_module.py
setup.py:
# setup.py
from setuptools import setup, find_packages
setup(
name='awesome_package',
version='1.0.0',
# find_packages() will ignore non-python files.
packages=find_packages(),
)
The __init__.py or the awesome_module.py:
import os
# The current directory
__here__ = os.path.dirname(os.path.realpath(__file__))
# Place the following function into __init__.py or into awesome_module.py
def check_editable_installation():
'''
Returns true if the package was installed with the editable flag.
'''
not_to_install_exists = os.path.isfile(os.path.join(__here__, 'not_to_install.txt'))
return not_to_install_exists

Run pip list, and there is a column called "Editable project location". If that column has a value, specifically the directory from which you installed it, then the package is pip installed in editable mode.

I don't know of a way to detect this directly (e.g. ask setuptools).
You could try to detect that you package can not be reached through the paths in sys.path. But that's tedious. It's also not bullet proof -- what if it can be reached through sys.path but it's also installed as en editable package?
The best option is to look at the artifacts an editable install leaves in your site-packages folder. There's a file called my_package.egg-link there.
from pathlib import Path
# get site packages folder through some other magic
# assuming this current file is located in the root of your package
current_package_root = str(Path(__file__).parent.parent)
installed_as_editable = False
egg_link_file = Path(site_packages_folder) / "my_package.egg-link"
try:
linked_folder = egg_link_file.read_text()
installed_as_editable = current_package_root in linked_folder
except FileNotFoundError:
installed_as_editable = False
Note: to make this a bit more bullet-proof, read only the first line of the egg-link file and parse it using Path() as well to account for proper slashes etc.

Recently I had to test if various packages were installed in editable mode across different machines. Running pip show <package name> reveals not only the version, but other information about it, which includes the location of the source code. If the package was not installed in editable mode, this location will point to site-packages, so for my case it was sufficient with checking the output of such command:
import subprocess
def check_if_editable(name_of_the_package:str) -> bool:
out = subprocess.check_output(["pip", "show", f"{name_of_the_package}"]).decode()
return "site-packages" in out

Haven't found a definite source for this, but if the output of
$ pip show my-package -f
is something like this:
..
..
Files:
__editable__.my-package-0.0.5.pth
my-package-0.0.5.dist-info/INSTALLER
my-package-0.0.5.dist-info/METADATA
my-package-0.0.5.dist-info/RECORD
my-package-0.0.5.dist-info/REQUESTED
my-package-0.0.5.dist-info/WHEEL
my-package-0.0.5.dist-info/direct_url.json
my-package-0.0.5.dist-info/top_level.txt
then it's probably editable.

How to run a python script installed from distutils

I have a medium sized python command line program that runns well from my source code, and I've created a source distribution file and installed it into the virtual environment using "python setup.py install"
Since this is a pure Python program, and provided that the end users have installed Python, and the required packages, my idea is that i can distribute it through PyPi for all available platforms as a source distribution.
Upon install, I get an 'appname' directory within the virtualenv site-packages directory, and it also runs correctly when I write "python 'pathtovirtualenv'/Lib/sitepackages/'myappname'
But is this the way the end user is supposed to run distutils-distributed programs from the command line.
I fnd a lot of information on how to distribute a program using distutils, but not on how the end user is supposed to launch it after installing it.

Since you already created a setup.py, I would recommend looking at the entry_points:
entry_points={
'console_scripts': [
'scriptname=yourpackage.module:function',
],
},
Here, you have a package named yourpackage, and a module named module in it, and you refer to the function function. This function will wrapped by the script called scriptname, which will be installed in the users bin folder, which is normally in the $PATH, so the user can simply type scriptname after he installed your package via pip install.
To sum up: a user will install the package via pip install yourpackage and finally be able to call the function in module via script name.
Here are some docs on this topic:
https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation
http://www.scotttorborg.com/python-packaging/command-line-scripts.html

Well, I eventually figured it out.
Initially, I wanted to just use distutils, I like it when the end user can install it with minimum of extra dependencies. But I have now discovered that setuptools is the better option in my case.
My directory structure looks like this (Subversion):
trunk
|-- appname
| |-- __init__.py # an empty file
| |-- __main__.py # calls appname.main()
| |-- appname.py # contains a main() and imports moduleN
| |-- module1.py
| |-- module2.py
| |-- ...
|-- docs
| |-- README
| |-- LICENSE
| |-- ...
|-- setup.py
And my setyp.py basically looks like this:
# This setup file is to be used with setuptools source distribution
# Run "python setup sdist to deploy
from setuptools import setup, find_packages
setup( name = "appname",
...
include_package_data = True,
packages = find_packages(),
zip_safe = True,
entry_points = {
'console_scripts' : 'appname=appname.appname:main'
}
)
The next step now is to figure out how to install the contents of the docs directory on the users computer.
But right now, I'm thinking about adding --readme, --license, --changes, --sample (and so forth) options to the main script, to display them at run time.

Create a python executable using setuptools

I have a small python application that I would like to make into a downloadable / installable executable for UNIX-like systems. I am under the impression that setuptools would be the best way to make this happen but somehow this doesn't seem to be a common task.
My directory structure looks like this:
myappname/
|-- setup.py
|-- myappname/
| |-- __init__.py
| |-- myappname.py
| |-- src/
| |-- __init__.py
| |-- mainclassfile.py
| |-- morepython/
| |-- __init__.py
| |-- extrapython1.py
| |-- extrapython2.py
The file which contains if __name__ == "__main__": is myappname.py. This file has a line at the top, import src.mainclassfile.
When this is downloaded, I would like for a user to be able to do something like:
$ python setup.py build
$ python setup.py install
And then it will be an installed executable which they can invoke from anywhere on the command line with:
$ myappname arg1 arg2
The important parts of my setup.py are like:
from setuptools import setup, find_packages
setup(
name='code2flow',
scripts=['myappname/myappname.py'],
package_dir={'myappname': 'myappname'},
packages=find_packages(),
)
Current state
By running:
$ sudo python setup.py install
And then in a new shell:
$ myapp.py
I am getting a No module named error

The problem here is that your package layout is broken.
It happens to work in-place, at least in 2.x. Why? You're not accessing the package as myappname—but the same directory that is that package's directory is also the top-level script directory, so you end up getting any of its siblings via old-style relative import.
Once you install things, of course, you'll end up with the myappname package installed in your site-packages, and then a copy of myappname.py installed somewhere on your PATH, so relative import can't possibly work.
The right way to do this is to put your top-level scripts outside the package (or, ideally, into a bin directory).
Also, your module and your script shouldn't have the same name. (There are ways you can make that work, but… just don't try it.)
So, for example:
myappname/
|-- setup.py
|-- myscriptname.py
|-- myappname/
| |-- __init__.py
| |-- src/
| |-- __init__.py
| |-- mainclassfile.py
Of course so far, all this makes it do is break in in-place mode the exact same way it breaks when installed. But at least that makes things easier to debug, right?
Anyway, your myscriptname.py then has to use an absolute import:
import myappname.src.mainclassfile
And your setup.py has to find the script in the right place:
scripts=['myscriptname.py'],
Finally, if you need some code from myscriptname.py to be accessible inside the module, as well as in the script, the right thing to do is to refactor it into two files—but if that's too difficult for some reason, you can always write a wrapper script.
See Arranging your file and directory structure and related sections in the Hitchhiker's Guide to Packaging for more details.
Also see PEP 328 for details on absolute vs. relative imports (but keep in mind that when it refers to "up to Python 2.5" it really means "up to 2.7", and "starting in 2.6" means "starting in 3.0".
For a few examples of packages that include scripts that get installed this way via setup.py (and, usually, easy_install and pip), see ipython, bpython, modulegraph, py2app, and of course easy_install and pip themselves.

Setting up setup.py for packaging of a single .py file and a single data file without needing to create any folders

Project tree:
$.
├── happy_birthday-art.txt
├── happy_birthday.py
├── MANIFEST.in
├── README.rst
└── setup.py
setup.py
from setuptools import setup
setup(
name='Happy_birthday',
py_modules=['happy_birthday'],
data_files=['happy_birthday-art.txt'],
entry_points={
'console_scripts': ['happy_birthday = happy_birthday:main', ],},
long_description=open('README.rst').read(),
)
Now when I do python setup.py sdist and then pip install the created .tar.gz file in a virtual environment I get the following message:
warning: install_data: setup script did not provide a directory for 'happy-birthday-art.txt' -- installing right in '/home/username/.virtualenvs/happy_birthday'
The program uses that .txt file so it fails when trying to run it afterwards.
But I don't want to install happy_birthday-art.txt into a separate folder. I want to install it in the folder where happy_birthday.py is installed. Also, I don't want to have to use absolute paths in setup.py. How do I best set up my setup.py file?

If you have a single-file module like this, no folder will be created, your .py file will be moved directly into the directory which contains the other python modules (/usr/lib/pythonX.X/site-packages/, for example). That's why you have to create a directory:
$ .
|-- happy_birthday/
|-- __init__.py
|-- art.txt
|-- MANIFEST.in
|-- README.rst
|-- setup.py

http://docs.python.org/2/distutils/setupscript.html
"You can specify the data_files options as a simple sequence of files
without specifying a target directory, but this is not recommended,
and the install command will print a warning in this case. To install
data files directly in the target directory, an empty string should be
given as the directory."
However, here the target directory is NOT the site-packages folder, but the
prefix folder, that is the root of the venv. If you'd want the .txt reside in the site-packages dir bare, then it would look not only ugly but seem that really not supported. On the other hand it is possible to install it to another location in the env, for example in "share/doc/foo":
data_files=[('share/doc/foo', ['happy_birthday-art.txt'])],

Packaging resources with setuptools/distribute

I'm developing an Python egg that has several .txt dependencies (they're templates used to generate files by the egg itself), and I'm struggling to get those dependencies copied to site-packages during setup.py install. According to the distribute documentation...
Filesystem of my package:
setup.py
package
|--- __init__.py
|--- main.py
|--- binary (calls main.py with pkg_resources.load_entry_point)
|--- templates
|--file1.txt
|--file2.txt
In setup.py:
setup(
[...]
eager_resources = ['templates/file1.txt', 'templates/file2.txt']
)
Within my package:
from pkg_resources import resource_string
tpl = resource_string(__name__, 'templates/file1.txt')
...this combination of configuration and filesystem should result in file1.txt and file2.txt being available through pkg_resources.resource_string. Unfortunately, they're not being copied to site-packages during setup.py install. What am I missing?
Thanks!

The information can be found in the setuptools documentation for including package data: https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files
Basically, you just need to set include_package_data=True in your setup.py file. If you are using subversion or CVS, all versioned files will be included. If not, you can specify which files to include with a MANIFEST.in file.
I believe distribute supports this as well.
You can then access the files as you would without them being packaged. i.e. in main.py you could have:
import os.path
f = open(os.path.join(os.path.dirname(__file__),'templates','file1.txt'))
print f.read()
f.close()
and this would work in the packaged version as well. One caveat is that you will have to also set zip_safe = False in setup.py so that all the files are unzipped during installation.