We Keep Coding

pip: Cannot perform editable system-wide install of "pyproject.toml" project

Consider the following minimal Python project with setuptools packaging and a "pyproject.toml" file (see setuptools Build System Support):
> tree myproject
myproject
|-- myproject
| `-- __init__.py
|-- pyproject.toml
|-- setup.cfg
`-- setup.py
"setup.py" is only a minimal dummy file to enable support for editable installs, as described here:
from setuptools import setup
if __name__ == '__main__':
setup()
When performing an editable install (pip install -e) to a virtualenv, everything works as expected:
> ls venv/lib/python3.9/site-packages | grep myproject
myproject.egg-link
> cat venv/lib/python3.9/site-packages/easy-install.pth
/myproject
> python3
>>> import myproject
Hello world!
The same is true for a non-editable system-wide install:
> ls /usr/local/lib/python3.9/dist-packages | grep myproject
myproject
myproject-1.0.dist-info
> python3
>>> import myproject
Hello world!
For an editable system-wide install, however, pip succeeds but does not result in a usable module:
> ls /usr/local/lib/python3.9/dist-packages | grep myproject
(No output)
> python3
>>> import myproject
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'myproject'
I'm aware that there were some issues in the past with "pyproject.toml" and editable installs. However, these appear to be fixed since 2019.
I have a relatively recent Debian Bullseye system with pip 20.3.4 and setuptools 52.0.0.
There also is PEP 660, which has not been implemented by setuptools yet. However, the dummy "setup.py" file (see above) should work around that limitation.

This is a Debian-specific problem.
Having a "pyproject.toml" file (i.e. a PEP 518 package), enables build isolation by default. On Debian, that results in editable installs ending up in "/usr/lib/python3.9/site-packages" instead of "/usr/local/lib/python3.9/dist-packages":
> ls /usr/lib/python3.9/site-packages
easy-install.pth myproject.egg-link
> cat /usr/lib/python3.9/site-packages/easy-install.pth
/myproject
However, "/usr/lib/python3.9/site-packages" is not part of the default Debian Python module paths.
I reported this behavior as Debian bug #1004149. Until it has been fixed, one can work around it by using pip install --no-build-isolation -e.

pip: module installed as editable not found

I'm trying to install the source checkout of my own package as an editable package using pip, but it doesn't get picked up as installed.
Yes, there are a whole lot of related questions, but they're mostly about non-standard directory structures. This here seems like it should "just work" and partially even does.
# starting from an empty venv:
(env) PS C:\test> pip install -e C:\path\to\my\PySymCircuit
Obtaining file:///C:/path/to/my/PySymCircuit
Preparing metadata (setup.py) ... done
Collecting sympy
Using cached sympy-1.9-py3-none-any.whl (6.2 MB)
Collecting mpmath>=0.19
Using cached mpmath-1.2.1-py3-none-any.whl (532 kB)
Installing collected packages: mpmath, sympy, SymCircuit
Looks good, but:
(env) PS C:\test> python -c 'import symcircuit'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'symcircuit'
(env) PS C:\test> pip show SymCircuit
WARNING: Package(s) not found: SymCircuit
The link file that is created is correct (as well as the egg-info in the source directory), and when I try to uninstall the package, suddenly it is recognized. Just not when it matters...
(env) PS C:\test> cat .\env\Lib\site-packages\SymCircuit.egg-link
C:\path\to\my\PySymCircuit
.
(env) PS C:\test> pip uninstall SymCircuit
Found existing installation: SymCircuit 0.1.0
Uninstalling SymCircuit-0.1.0:
Would remove:
c:\test\env\lib\site-packages\symcircuit.egg-link
Proceed (Y/n)? Y
WARNING: Cannot remove entries from nonexistent file c:\test\env\lib\site-packages\easy-install.pth
Successfully uninstalled SymCircuit-0.1.0
Plain setup.py develop produces the same link, also not usable.
What am I missing?
A non-editable install from the same source tree works as intended, so I believe it has something to do with the linking mechanism?
Update: so apparently the link is just for pip, the important part is that setuptools should add the path to easy-install.pth. It doesn't, hence the error message when uninstalling.
Now the question is, why...
Update as requested:
Local directory structure: git clone https://github.com/martok/py-symcircuit.git
py-symcircuit
│ LICENSE
│ README.md
│ setup.py
├───.git
├ ...
├───symcircuit
│ __init__.py
│ bode.py
│ spice.py
│ system.py
└───SymCircuit.egg-info
dependency_links.txt
PKG-INFO
requires.txt
SOURCES.txt
top_level.txt
Setup.py:
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(
name="SymCircuit",
version="0.1.0",
author="Martok",
author_email="martok#martoks-place.de",
description="Symbolic electronic circuit analysis",
long_description=open("README.md","rt").read(),
long_description_content_type="text/markdown",
url="https://github.com/martok/py-symcircuit",
project_urls={
"Bug Tracker": "https://github.com/martok/py-symcircuit/issues",
},
license="MIT",
classifiers=[...],
packages=find_packages(),
python_requires='>=3.6',
install_requires=[
"sympy",
],
extras_require={
"EE": [
"networkx",
"numpy",
"mplotkit"
],
},
)
Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 23 2018, 23:31:17) [MSC v.1916 32 bit (Intel)] on win32
Package Version
---------------- ---------
pip 21.3.1
setuptools 59.4.0

single-sourcing package version for setup.cfg Python projects

For traditional Python projects with a setup.py, there are various ways of ensuring that the version string does not have to be repeated throughout the code base. See PyPA's guide on "Single-sourcing the package version" for a list of recommendations.
Many are trying to move away from setup.py to setup.cfg (probably under the influence of PEP517 and PEP518; setup.py was mostly used declaratively anyway, and when there was logic in setup.py, it was probably for the worse.) This means that most the suggestions won't work anymore since setup.cfg cannot contain "code".
How can I single-source the package version for Python projects that use setup.cfg?

There are a couple of ways to do this (see below for the project structure used in these examples):
1.
setup.cfg
[metadata]
version = 1.2.3.dev4
src/my_top_level_package/__init__.py
import importlib.metadata
__version__ = importlib.metadata.version('MyProject')
2.
setup.cfg
[metadata]
version = file: VERSION.txt
VERSION.txt
1.2.3.dev4
src/my_top_level_package/__init__.py
import importlib.metadata
__version__ = importlib.metadata.version('MyProject')
3.
setup.cfg
[metadata]
version = attr: my_top_level_package.__version__
src/my_top_level_package/__init__.py
__version__ = '1.2.3.dev4'
And more...
There are probably other ways to do this, by playing with different combinatons.
References:
https://setuptools.readthedocs.io/en/latest/userguide/declarative_config.html
https://docs.python.org/3/library/importlib.metadata.html
Structure assumed in the previous examples is as follows...
MyProject
├── setup.cfg
├── setup.py
└── src
└── my_top_level_package
└── __init__.py
setup.py
#!/usr/bin/env python3
import setuptools
if __name__ == '__main__':
setuptools.setup(
# see 'setup.cfg'
)
setup.cfg
[metadata]
name = MyProject
# See above for the value of 'version = ...'
[options]
package_dir =
= src
packages = find:
[options.packages.find]
where = src
$ cd path/to/MyProject
$ python3 setup.py --version
1.2.3.dev4
$ python3 -m pip install .
# ...
$ python3 -c 'import my_top_level_package; print(my_top_level_package.__version__)'
1.2.3.dev4
$ python3 -V
Python 3.6.9
$ python3 -m pip list
Package Version
------------- ----------
MyProject 1.2.3.dev4
pip 20.0.2
pkg-resources 0.0.0
setuptools 45.2.0
wheel 0.34.2
zipp 3.0.0

setuptools very simple (one source file module) configuration

I want to use setuptools to create a package consisting of two files: foo.py (script) and foo.conf.
Then I want to publish the package on my devpi-server and then install the package using pip.
Suppose I that initially I have my current working directory clean
$ ls -l
total 0
Then I issue pip install (or download?) command
$ pip install -i http://mydevpi.server foo
And get a dir with my two files created
$ tree
.
|
foo
|
|\_ foo.py
|
\_ foo.conf
So questions are:
what setuptools configuration should I use?
what exact pip command should I use to install the package the way I want? Will pip install -i http://mydevpi.server --target=. do the trick?

First write somethings as setup.py in foo directory like:
import setuptools
setuptools.setup(
name='foo_pip',
version='1',
packages=[''],
url='1',
license='1',
author='1',
author_email='1',
description='1'
)
(You can use distutils or setuptools)
Then python setup.py bdist_wheel -d TARGET and there will be a whl file in target directory, copy the path.
You can now install using pip install the_wheel_file_path --prefix="the_path_to_install"
Something like this
Processing .../TARGET/foo_pip-1-py2-none-any.whl
Installing collected packages: foo-pip
Successfully installed foo-pip-1
Then use it by import foo

Test on Publishing my own python package to PyPI but failed

I followed the instructions here to publish a simple project onto PyPI. I succeeded after some struggles.
However, after I installed the project thru pip install my-project, I cannot import it in Python like this: ImportError: No module named my-project...
Is it because of my file structure? It is like this:
My-project-folder
|- setup.py
|- test.py
|- README
|- my-project
~~|- file.py
~~|- file2.py

You're missing __init__.py inside my-project!