is it possible to detect if current python code is running from package?
if yes - is it possible to get package metadata (name, version, description)?
package is created with this kind of setup.py
import os
from setuptools import setup, find_packages
setup(
name='my-pack-name',
description='my description ' + os.getenv('GIT_COMMIT', '*')[:7],
version=os.getenv('BUILD_VERSION', '0.0.0dev'),
packages=find_packages(),
)
build: python3 setup.py bdist_wheel -d ./artifact
install on target: pip3 install "my-pack-name-x.x.x.whl" --upgrade
now from my_pack_name/app.py that was inside my-pack-name-x.x.x.whl i want to detect that i'm running from installed package
and if so then get package metadata defined during setup.py execution
For Python >=3.8
https://docs.python.org/es/3.10/library/importlib.metadata.html
You can get the metadata for a package by:
from importlib.metadata import metadata
md = metadata("your package name")
author = md["Author"]
# etc ...
For Python <3.8
This is just an idea (not tested).
What about having the package metadata in a different module, and try relative import it in the app.py module?
# metadata.py
name='my-pack-name',
description='my description ' + os.getenv('GIT_COMMIT', '*')[:7],
version=os.getenv('BUILD_VERSION', '0.0.0dev')
In setup.py you could reuse that:
# setup.py
import os
from setuptools import setup, find_packages
import .metadata as md
setup(
name=md.name,
description=md.description + os.getenv('GIT_COMMIT', '*')[:7],
version=md.version,
packages=find_packages(),
)
And the in app.py
def get_metadata():
try:
import .metadata as md
except ImportError:
return None
else:
# return metadata here
That way if get_metadata returns None, you were not able to import the module, so you're not executing the app.py in your package, otherwise, you are in your package and as a bonus you got your metadata.
For the test if python code is running from an installed package (vs. from a development location), I use
if 'site-packages' in __file__:
...
I don't know if that's a good approach; seems to work thus far.
Related
I am trying to build a package for distribution which has cython code that I would like to compile into binaries before uploading to PyPI. To do this I am using pypa's build,
python -m build
in the project's root directory. This cythonizes the code and generates the binaries for my system then creates the sdist and wheel in the dist directory. However, the wheel is named "--py3-none-any.whl". When I unzip the .whl I do find the appropriate binaries stored,
(e.g., cycode.cp39-win_amd64.pyd). The problem is I plan to run this in a GitHub workflow where binaries are built for multiple python versions and operating systems. That workflow works fine but overwrites (or causes a duplicate version error) when uploading to PyPI since all of the wheels from the various OS share the same name. Then if I install from PyPI on another OS I get "module can't be found" errors since the binaries for that OS are not there and, since it was a wheel, the installation did not re-compile the cython files.
I am working with 64-bit Windows, MacOS, and Ubuntu. Python versions 3.8-3.10. And a small set of other packages which are listed below.
Does anyone see what I am doing wrong here? Thanks!
Simplified Package
Tests\
Project\
__init__.py
pycode.py
cymod\
__init__.py
_cycode.pyx
_build.py
pyproject.toml
pyproject.toml
[project]
name='Project'
version = '0.1.0'
description = 'My Project'
authors = ...
requires-python = ...
dependencies = ...
[build-system]
requires = [
'setuptools>=64.0.0',
'numpy>=1.22',
'cython>=0.29.30',
'wheel>=0.38'
]
build-backend = "setuptools.build_meta"
[tool.setuptools]
py-modules = ["_build"]
include-package-data = true
packages = ["Project",
"Project.cymod"]
[tool.setuptools.cmdclass]
build_py = "_build._build_cy"
_build.py
import os
from setuptools.extension import Extension
from setuptools.command.build_py import build_py as _build_py
class _build_cy(_build_py):
def run(self):
self.run_command("build_ext")
return super().run()
def initialize_options(self):
super().initialize_options()
import numpy as np
from Cython.Build import cythonize
print('!-- Cythonizing')
if self.distribution.ext_modules == None:
self.distribution.ext_modules = []
# Add to ext_modules list
self.distribution.ext_modules.append(
Extension(
'Project.cymod.cycode',
sources=[os.path.join('Project', 'cymod', '_cycode.pyx')],
include_dirs=[os.path.join('Project', 'cymod'), np.get_include()]
)
)
# Add cythonize ext_modules
self.distribution.ext_modules = cythonize(
self.distribution.ext_modules,
compiler_directives={'language_level': "3"},
include_path=['.', np.get_include()]
)
print('!-- Finished Cythonizing')
Given a Python package with the following structure.
Installed it with pip
pip install --upgrade git+git://github.com/balandongiv/driving_tools.git
The installed directory looks as below
As shown in figure above, the subfolder sub_file and the nickname_override.py are missing in the installation folder.
May I know what modification is required to amend this issue.
Modification to be made as per Balaitous
from setuptools import setup,find_packages
setup(name='ppackage',
version='0.0.111',
description='make life easier',
author='testx',
packages=['ppackage','ppackage.sub_file'],
)
In packages argument of setup fonction, all modules have to be explicitly mentioned. Module can be a python file or a folder containing __init__.py.
It is not recursive. Here you have two modules ppackage and ppackage.sub_folder.
See: https://docs.python.org/3/distutils/setupscript.html#listing-whole-packages
So you should have:
setup(
name=...,
packages=["ppackage", "ppackage.sub_folder"],
...
)
If you want to embed all modules in you package, you can use find_packages
from setuptools import find_packages
setup(
packages=find_packages(),
...
)
I've created a python package that is posted to pypi.org. The package consists of a single .py file, which is the same name as the package.
After installing the package via pip (pip install package_name) in a conda or standard python environment I must use the following statement to import a function from this module:
from package_name.package_name import function_x
How can I reorganise my package or adjust my installation command so that I may use import statement
from package_name import function_x
which I have successfully used when I install via python setup.py install.
My setup.py is below
setup(
name = "package_name",
version = "...",
packages=find_packages(exclude=['examples', 'docs', 'build', 'dist']),
)
Change your setup arguments from using packages to using py_modules e.g.
setup(
name = "package_name",
version = "..",
py_modules=['package_name'],
)
This is documented here https://docs.python.org/2/distutils/introduction.html#a-simple-example
I have the following directory structure:
/modules/
/modules/setup.py
/modules/setup.cfg
/modules/module1/
/modules/module1/__init__.py
/modules/module1/tool1/__init__.py
/modules/module1/tool1/tool1.py
/modules/module2/
/modules/module2/__init__.py
/modules/module2/tool2/__init__.py
/modules/module2/tool2/tool2.py
/modules/module2/tool3/__init__.py
/modules/module2/tool3/tool3.py
And I want to install these modules using setup.py and setup.cfg and import them later on like this:
import my_project.module1.tool1
import my_project.module2.tool2
import my_project.module2.tool3
These are my installation files:
setup.py
import setuptools
setuptools.setup(
setup_requires=['paramiko>=2.0.1'],
paramiko=True)
setup.cfg
[metadata]
name = my_project
summary = my project modules
[files]
packages =
module1
module2
It fails when I try to install the packages:
/modules# pip install -e .
Obtaining file:///modules
Installing collected packages: UNKNOWN
Found existing installation: UNKNOWN 0.0.0
Can't uninstall 'UNKNOWN'. No files were found to uninstall.
Running setup.py develop for UNKNOWN
Successfully installed UNKNOWN
Try using the find_packages function:
from setuptools import setup, find_packages
...
setup(
...
packages=find_packages(),
...
There is a Python package with a setup.py that reads thusly:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
name = 'fastahack',
ext_modules=[
Extension("fastahack.cfastahack",
sources=["fastahack/cfastahack.pyx", "lib/Fasta.cpp", "lib/split.cpp"],
libraries=["stdc++"],
include_dirs=["lib/"],
language="c++"),
],
package_data = {'lib': ['*.pyx', "*.c", "*.h", "README.rst"]},
package_dir = {"fastahack": "fastahack"},
cmdclass = {'build_ext': build_ext},
packages = ['fastahack', 'fastahack.tests'],
author = "Brent Pedersen",
author_email="bpederse#gmail.com",
#test_suite='nose.collector'
)
This setup.py can't be imported if Cython is not installed. As far as I know, importing setup.py is how tools like pip figure out the dependencies of a package. I want to set up this package so that it could be uploaded to PyPI, with the fact that it depends on Cython noted, so that Cython will be downloaded and installed when you try to "pip install fastahack", or when you try to "pip install" directly from the Git repository.
How would I package this module so that it installs correctly from the Internet when Cython is not installed? Always using the latest version of Cython would be a plus.
You can specify Cython as a build dependency using PEP-518 project specification.
In the file pyproject.toml (in the same directory as setup.py) insert:
[build-system]
requires = ["setuptools", "wheel", "Cython"]
Cython will then be installed before building your package.
Note that (currently) you need to pass --no-use-pep517 to pip install if you are installing your package locally as editable (ie with --editable or -e) setuptools v64 supports editable installs with pyproject.toml builds
My standard template for setup.py:
have_cython = False
try:
from Cython.Distutils import build_ext as _build_ext
have_cython = True
except ImportError:
from distutils.command.build_ext import build_ext as _build_ext
if have_cython:
foo = Extension('foo', ['src/foo.pyx'])
else:
foo = Extension('foo', ['src/foo.c'])
setup (
...
ext_modules=[foo],
cmdclass={'build_ext': build_ext}
And don't forget to provide extention .c files with package - that will allow users to build module without installing cython.
Use a try and except for the Cython import and modify your setup based on whether or not your import succeeds. Look at the setup.py of Pandas for an example