We Keep Coding

What is the best replacement for python setup.py install when Cython needs to be compiled?

With the latest version of setuptools, the python setup.py install command is being deprecated (see https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for more info).
(venv) [jon#dev02 py38]$ python setup.py install
running install
/home/jon/.jenkins/workspace/Farm_revision_linux_py36/TOXENV/py38/venv/lib64/python3.8/site-
packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is
deprecated. Use build and pip and other standards-based tools.
warnings.warn(
/home/jon/.jenkins/workspace/Farm_revision_linux_py36/TOXENV/py38/venv/lib64/python3.8/site-
packages/setuptools/command/easy_install.py:156: EasyInstallDeprecationWarning: easy_install
command is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
running bdist_egg
running egg_info
... etc
It is suggested that you can just pip install . to install a package from source, but this does not compile any Cython code. What is the best method for doing this?
The Cython docs are still recommending use of setup.py and I can't find any better suggestions. It appears that a developer install (pip install -e .) does compile the Cython files, or you can python setup.py build_ext --inplace after running pip install .. Neither of these options are ideal, so I would like to know if there is a suggested alternative.
EDIT:
The package setup.py file contains this code, which should compile the Cython files:
try:
import numpy
from Cython.Build import cythonize
cython_files = ["farm/rasters/water_fill.pyx"]
cython_def = cythonize(cython_files, annotate=True,
compiler_directives={'language_level': str(sys.version_info.major)})
include_dirs = [numpy.get_include()]
except ImportError:
cython_def = None
include_dirs = None
When installing with python setup.py install, the farm/rasters directory contains the following files:
water_fill.c
water_fill.cpython-38-x86_64-linux-gnu.so
water_fill.html
water_fill.pyx
When installing with pip install ., the directory does not contain the .so file and when we try and run water_fill tests, we get errors like this
________________________________ TestFlowDown1D.test_distribute_1d_trough_partial_3 _________________________________
farm/rasters/tests/test_flood_enhance.py:241: in test_distribute_1d_trough_partial_3
actual_arr = water_fill.water_fill_1d(arr, additional_value)
E AttributeError: 'NoneType' object has no attribute 'water_fill_1d'

I believe the warning should go away if you keep the setup.py but add a pyproject.toml.
This is described in https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html .
Here is the example pyproject.toml file they give:
[build-system]
requires = ["setuptools", "wheel", "Cython"]

Python creating pip package - module not found

I am trying to create a python package to distribute my code. I am not getting any error in creating a package, and installing created package.
However, after installation when I am trying to import the package I am getting error ModuleNotFoundError:
Following is the code
hello_world.py
class HelloWorld:
def print_msg(self):
print("Hello World")
setup.py
from setuptools import setup, find_packages
setup(
name = "HelloWorld",
version = "0.1",
packages = find_packages(),
)
create package
▶ python setup.py bdist_wheel
running bdist_wheel
running build
installing to build/bdist.macosx-10.14-x86_64/wheel
running install
running install_egg_info
running egg_info
writing HelloWorld.egg-info/PKG-INFO
writing dependency_links to HelloWorld.egg-info/dependency_links.txt
writing top-level names to HelloWorld.egg-info/top_level.txt
reading manifest file 'HelloWorld.egg-info/SOURCES.txt'
writing manifest file 'HelloWorld.egg-info/SOURCES.txt'
Copying HelloWorld.egg-info to build/bdist.macosx-10.14-x86_64/wheel/HelloWorld-0.1-py3.7.egg-info
running install_scripts
creating build/bdist.macosx-10.14-x86_64/wheel/HelloWorld-0.1.dist-info/WHEEL
creating 'dist/HelloWorld-0.1-py3-none-any.whl' and adding 'build/bdist.macosx-10.14-x86_64/wheel' to it
adding 'HelloWorld-0.1.dist-info/METADATA'
adding 'HelloWorld-0.1.dist-info/WHEEL'
adding 'HelloWorld-0.1.dist-info/top_level.txt'
adding 'HelloWorld-0.1.dist-info/RECORD'
removing build/bdist.macosx-10.14-x86_64/wheel
Installing package
~/PycharmProjects/test_dist ▶ pip install dist/HelloWorld-0.1-py3-none-any.whl
Processing ./dist/HelloWorld-0.1-py3-none-any.whl
Installing collected packages: HelloWorld
Successfully installed HelloWorld-0.1
~/PycharmProjects/test_dist ▶ pip freeze
HelloWorld==0.1
Error While importing module
>>> import HelloWorld
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'HelloWorld'

Where is hello_world.py? Is it at the root folder adjacent to setup.py? Or in some subdirectory? I suspect the former. That means you don't have any packages so find_packages() returns an empty list so setuptools don;t package any code into the package.
Your hello_world.py isn't a packages (a directory with file __init__.py), it's a standalone module and such modules must be packed using py_modules. This is how you should write your setup.py:
from setuptools import setup
setup(
name = "HelloWorld",
version = "0.1",
py_modules = ['hello_world'],
)

Tox fails because setup.py can't find the requirements.txt

I have added tox to my project and my tox.ini is very simple:
[tox]
envlist = py37
[testenv]
deps =
-r{toxinidir}/requirements_test.txt
commands =
pytest -v
But when I run tox, I get the following error:
ERROR: invocation failed (exit code 1), logfile: /path/to/my_project/.tox/py37/log/py37-2.log
========================================================================================= log start ==========================================================================================
Processing ./.tox/.tmp/package/1/my_project-0+untagged.30.g6909bfa.dirty.zip
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-req-build-ywna_4ks/setup.py", line 15, in <module>
with open(requirements_path) as requirements_file:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-req-build-ywna_4ks/requirements.txt'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-ywna_4ks/
You are using pip version 10.0.1, however version 19.2.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
========================================================================================== log end ===========================================================================================
__________________________________________________________________________________________ summary ___________________________________________________________________________________________
ERROR: py37: InvocationError for command /path/to/my_project/.tox/py37/bin/python -m pip install --exists-action w .tox/.tmp/package/1/my_project-0+untagged.30.g6909bfa.dirty.zip (exited with code 1)
Here is my setup.py:
-*- coding: utf-8 -*-
import os
import sys
from setuptools import setup, find_packages
import versioneer
here = os.path.abspath(os.path.dirname(__file__))
sys.path.insert(0, here)
requirements_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'requirements.txt')
with open(requirements_path) as requirements_file:
requires = requirements_file.readlines()
setup(
name='my_project',
version=versioneer.get_version(),
cmdclass=versioneer.get_cmdclass(),
maintainer='Hamed',
license='BSD',
py_modules=['my_project'],
packages=find_packages(),
package_data={'': ['*.csv', '*.yml', '*.html']},
include_package_data=True,
install_requires=requires,
long_description=open('README.md').read(),
zip_safe=False
]
},
)
python setup.py install works fine.
It seems that tox is looking for requirements in the tmp dir, but can't find it there. Is there something wrong with my configurations?
I am using tox==3.12.1, python==3.7.3, setuptools==41.0.1, and conda==4.6.9
I've tested this on Arch and SLES 12 and got the same result with both.

Based on the point from #phd, I found out that requirements.txt was not present in the source distribution. Adding requirements.txt to the MANIFEST.in solved the issue!

Complementing Hamed2005's answer:
I have my requirements split into different files (base_requirements.txt, dev_requirements.txt, etc), all of them in a requirements directory. In this case, you need to add this directory in the MANIFEST.in as
recursive-include requirements *

Install modules with setup.py and setup.cfg

I have the following directory structure:
/modules/
/modules/setup.py
/modules/setup.cfg
/modules/module1/
/modules/module1/__init__.py
/modules/module1/tool1/__init__.py
/modules/module1/tool1/tool1.py
/modules/module2/
/modules/module2/__init__.py
/modules/module2/tool2/__init__.py
/modules/module2/tool2/tool2.py
/modules/module2/tool3/__init__.py
/modules/module2/tool3/tool3.py
And I want to install these modules using setup.py and setup.cfg and import them later on like this:
import my_project.module1.tool1
import my_project.module2.tool2
import my_project.module2.tool3
These are my installation files:
setup.py
import setuptools
setuptools.setup(
setup_requires=['paramiko>=2.0.1'],
paramiko=True)
setup.cfg
[metadata]
name = my_project
summary = my project modules
[files]
packages =
module1
module2
It fails when I try to install the packages:
/modules# pip install -e .
Obtaining file:///modules
Installing collected packages: UNKNOWN
Found existing installation: UNKNOWN 0.0.0
Can't uninstall 'UNKNOWN'. No files were found to uninstall.
Running setup.py develop for UNKNOWN
Successfully installed UNKNOWN

Try using the find_packages function:
from setuptools import setup, find_packages
...
setup(
...
packages=find_packages(),
...

Could not find a version that satisfies the requirement

I am trying to create a library distributed with pip.
sudo python setup.py sdist upload -r pypitest
when I try to install it with
sudo pip install -i https://testpypi.python.org/pypi abce
It fails with
Could not find a version that satisfies the requirement pandas>=0.17 (from abce) (from versions: )
No matching distribution found for pandas>=0.17 (from abce)
I tried no for a day, but I can't make it work. When I install pandas with pip install pandas it installs the 0.18.1 version. What am I doing wrong?
The setup.py is the following:
#!/usr/bin/env python
import os
try:
from setuptools import setup
from setuptools import Extension
except ImportError:
from distutils.core import setup
from distutils.extension import Extension
cmdclass = { }
ext_modules = [ ]
try:
from Cython.Distutils import build_ext
ext_modules += [
Extension("abce.trade", [ "abce/trade.pyx" ]),
]
cmdclass.update({ 'build_ext': build_ext })
except ImportError:
ext_modules += [
Extension("abce.trade", [ "abce/trade.c" ]),
]
setup(name='abce',
version='0.5.07b',
author='Davoud Taghawi-Nejad',
author_email='Davoud#Taghawi-Nejad.de',
description='Agent-Based Complete Economy modelling platform',
url='https://github.com/DavoudTaghawiNejad/abce.git',
package_dir={'abce': 'abce'},
packages=['abce'],
long_description=open('README.rst').read(),
install_requires=['numpy>=1.9',
'pandas>=0.17',
'networkx>=1.9',
'flask>=0.10',
'bokeh>=0.11',
'matplotlib>=1.3'],
include_package_data=True,
ext_modules=ext_modules,
cmdclass=cmdclass)

On day later:
pip was searching the packages on piptest, but actually the ABCE package should come from piptest and the requirments should come from pip:
sudo pip install -i https://testpypi.python.org/pypi --extra-index-url https://pypi.python.org/pypi abce