We Keep Coding

PyPI - allow importing a module using a different name

I recently created a simple python package / library and uploaded it to (PyPI). That's all fine - I can upload my project to pypi and download it again using pip, but I can't import it because it contains a full stop .
My package is called dcode.py (which obviously causes issues with importing), I'd like to be able to import it by just typing import dcode (without the .py). I've looked around for a solution but can't seem to find one - I'd like to be able to import it under an alias (dcode) instead of dcode.py (which obviously doesn't work).
Is there a way to do this? I'd assume there must be (for example, both the packages discord.py and py-cord can both just be imported using import discord. Does anyone know how to do this?
Just in case you need it, my pyproject.toml file looks like this:
[project]
name = "dcode.py"
version = "0.0.2"
authors = [
{ name = "Dylan Rogers", email = "opendylan#proton.me" },
]
description = "dcode - the Python package for everything"
readme = "README.md"
requires-python = ">=2.0"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
]
[project.urls]
"Homepage" = "https://github.com/dylanopen/dcode.py"
"Bug Tracker" = "https://github.com/dylanopen/dcode.py/issues"
And my entire project is on github.
Here is the PyPI project
I've tried importing dcode.py:
import dcode.py
But I just get this error:
File "/run/media/dylan/Programming/dcode.py-test/test.py", line 1, in <module>
import dcode_py
ModuleNotFoundError: No module named 'dcode_py'
If it helps, I'm using Manjaro Linux (no idea why it would help though, I just need to know how to allow users to import dcode.py by typing import dcode, like they import discord.py as discord)
Thank you so much for any help!

This is the recommended directory structure your project should have:
dcode.py/
├── LICENSE
├── pyproject.toml
├── README.md
├── src/
│ └── dcode/
│ ├── __init__.py
│ └── io/
│ ├── __init__.py
│ └── file.py
└── tests/
Here dcode (as defined by dcode.py/src/dcode) is the name of the only top-level importable package. And it also contains a sub-package dcode.io (as defined by dcode.py/src/dcode/io).
The name of the actual distribution project as seen on PyPI and that gets pip-installed is defined in the package metadata, in the pyproject.toml file:
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[project]
name = "dcode.py"
version = "0.0.1"
authors = [
{ name = "Dylan Rogers", email = "opendylan#proton.me" },
]
description = "dcode - the Python package for everything"
readme = "README.md"
requires-python = ">=3.8"
[project.urls]
"Homepage" = "https://github.com/dylanopen/dcode.py"
"Bug Tracker" = "https://github.com/dylanopen/dcode.py/issues"
[tool.setuptools.packages.find]
where = ["src"]
Do not push the dist directory containing the built sdist and wheel distribution packages artifacts into the source code repository.
Do not push any *.egg-info directory into the source code repository.
Place all the actual Python code inside a src directory.
The name of the project can contain a dot such as dcode.py, although I would recommend against it.
The name of the actual top-level import packages and modules can be completely different than the name of the distribution project.
If you declare a classifier for the license in the package metadata in pyproject.toml, then it should match the actual license file in the source code repository (you had a GNU GPL v3.0 license file but a MIT license classifier).
References:
https://packaging.python.org/en/latest/tutorials/packaging-projects/
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
https://sinoroc.gitlab.io/kb/python/packaging.html

How to find the namespace of a package with pip

I'm trying to write a custom package for python. It has the following folder structure:
E:\GITHUB\FUNCTIONSYNTHESIZER
│ .gitignore
│ LICENSE
│ README.md
│ setup.py
│
├───docs
│ docs.md
│ Example-6-Points.png
│
└────src
│ function_synth.py
│ __init__.py
│
└───__pycache__
Pip allows an install via the Folderpath of the setup.py. The setup.py follows the guidelines of Python.org an most importantly works in another package with a different name:
from distutils.core import setup
def README():
return open("README.md").read()
setup(name='function_synthesizer',
version = '0.0.2',
description = 'Generates a polynomial interpolation from a set up points',
author = 'David J. Kowalk',
author_email = '...',
url = 'https://github.com/davidkowalk/FunctionSynthesizer',
license = "MIT",
keywords = "math interpolation polynomial calculus",
long_description = README(),
long_description_content_type='text/markdown',
classifiers=[
Calssifiers here
],
package_dir = {'':'src'},
python_requires = ">=3.6",
install_requires=['numpy'],
project_urls={
'Bug Reports': '...',
'Funding': '...',
'Source': '...',
}
)
The __init__.py imports the different functions of function_synth.
from function_synth import solve
from function_synth import to_str
from function_synth import calculate
The package gets installed by pip under the name "function_synthesizer" as defined by the setup.py but when I run pip show function_synthesizer I get
Name: function-synthesizer
Version: ...
This name violates the package name syntax and most importantly isn't defined anywhere. I cannot find any reason for that dash to be there, and it seems to replace the underscore defined by the name argument for some reason. But even when I remove the underscore and the resulting dash python is unable to import the packages under the following names:
function_synthesizer
function-synthesizer (syntax error at the dash)
functionsynthesizer (after removing the dash)
function_synth (file name)
Is there anyway of fetching the namespace from pip and how can I define that namespace in the setup.py?

Issue was fixed by adding a packages attribute:
packages = ["function_synthesizer"],
package_dir = {'function_synthesizer':'src'},
The __init__.py imports the functions then via
from function_synthesizer.function_synth import function_name

How to Import python package from another directory?

I have a project that is structured as follows:
project
├── api
│ ├── __init__.py
│ └── api.py
├── instance
│ ├── __init__.py
│ └── config.py
├── package
│ ├── __init__.py
│ └── app.py
├── requirements.txt
└── tests
└── __init__.py
I am trying to call the config.py file from the package/app.py as shown below:
# package/app.py
from instance import config
# I've also tried
import instance.config
import ..instance.config
from ..instance import config
But I always get the following error:
Traceback (most recent call last):
File "/home/csymvoul/projects/project/package/app.py", line 1, in <module>
from instance import config
ModuleNotFoundError: No module named 'instance'
Modifying the sys.path is not something I want to do.
I know that this question is very much answered but the answers that were given, did not work for me.
EDIT: When moving the app.py to the root folder it works just fine. But I need to have it under the package folder.

You can add the parent directory to PYTHONPATH, in order to achieve that, you can use OS depending path in the "module search path" which is listed in sys.path. So you can easily add the parent directory like following:
import sys
sys.path.insert(0, '..')
from instance import config
Note that the previous code uses a relative path, so you must launch the file inside the same location or it will likely not work. To launch from anywhere, you can use the pathlib module.
from pathlib import Path
import sys
path = str(Path(Path(__file__).parent.absolute()).parent.absolute())
sys.path.insert(0, path)
from instance import config
However, the previous approach is more a hack than anything, in order to do things right, you'll first need to reshape your project structure according to this very detailed blog post python packaging, going for the recommended way with a src folder.
Your directory layout must look like this:
project
├── CHANGELOG.rst
├── README.rst
├── requirements.txt
├── setup.py
├── src
│   ├── api
│   │   ├── api.py
│   │   └── __init__.py
│   ├── instance
│   │   ├── config.py
│   │   └── __init__.py
│   └── package
│   ├── app.py
│   └── __init__.py
└── tests
└── __init__.py
Note that you don't really need the requirements.txt because you can declare the dependencies inside your setup.py.
A sample setup.py (adapted from here):
#!/usr/bin/env python
# -*- encoding: utf-8 -*-
from __future__ import absolute_import
from __future__ import print_function
import io
import re
from glob import glob
from os.path import basename
from os.path import dirname
from os.path import join
from os.path import splitext
from setuptools import find_packages
from setuptools import setup
def read(*names, **kwargs):
with io.open(
join(dirname(__file__), *names),
encoding=kwargs.get('encoding', 'utf8')
) as fh:
return fh.read()
setup(
name='nameless',
version='1.644.11',
license='BSD-2-Clause',
description='An example package. Generated with cookiecutter-pylibrary.',
author='mpr',
author_email='contact#ionelmc.ro',
packages=find_packages('src'),
package_dir={'': 'src'},
include_package_data=True,
zip_safe=False,
classifiers=[
# complete classifier list: http://pypi.python.org/pypi?%3Aaction=list_classifiers
'Development Status :: 5 - Production/Stable',
'Intended Audience :: Developers',
'License :: OSI Approved :: BSD License',
'Operating System :: Unix',
'Operating System :: POSIX',
'Operating System :: Microsoft :: Windows',
'Programming Language :: Python',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: PyPy',
# uncomment if you test on these interpreters:
# 'Programming Language :: Python :: Implementation :: IronPython',
# 'Programming Language :: Python :: Implementation :: Jython',
# 'Programming Language :: Python :: Implementation :: Stackless',
'Topic :: Utilities',
],
keywords=[
# eg: 'keyword1', 'keyword2', 'keyword3',
],
python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*',
install_requires=[
# eg: 'aspectlib==1.1.1', 'six>=1.7',
],
extras_require={
# eg:
# 'rst': ['docutils>=0.11'],
# ':python_version=="2.6"': ['argparse'],
},
setup_requires=[
# 'pytest-runner',
],
entry_points={
'console_scripts': [
'api = api.api:main',
]
},
)
The content of my api.py:
from instance import config
def main():
print("imported")
config.config()
The content of my config.py:
def config():
print("config imported successfully")
You can find all the previous here
Optional but recommended: create a virtual environment, I use venv (Python 3.3 <=) for that, inside the root of the project:
python -m venv .
And to activate:
source bin/activate
Now I can install the package:
Using pip install -e . (with the dot) command inside the root of the project
Your import from instance import config works now, to confirm you can run api.py with:
python src/api/api.py

In Python 3.3 onwards you don't need __init__.py files in your subdirectories for the purpose of imports. Having them can actually be misleading as it causes the creation of package namespaces in each folder containing an init file, as described here.
By removing all those __init__.py files you will be able to import files in the namespace package (including subdirectories) when running app.py, but that's still not want we want.
The Python interpreter still doesn't know how to reach your instance namespace. In order to do that you can use the PYTHONPATH environment variable, including a path that is parent to config.py. You can do that as suggested in #RMPR's answer with sys.path, or by directly setting the environment variable, for instance:
PYTHONPATH=/home/csymvoul/projects/project python3 /home/csymvoul/projects/project/package/app.py
Then importing the dependency like from instance import config.

Import modules in package in ROS2

I have created a package for ROS2 and I have added a Python repository I downloaded. The problem I am having is that in the original repository the modules from the own repo were imported directly while in mine I have to import them adding the ROS2 package name before the module, even though I am importing a module from the same repo, like:
import planner_pkg.SimpleOneTrailerSystem as SimpleOneTrailerSystem
while I would like:
import SimpleOneTrailerSystem
My ROS2 project structure is like:
ros2_ws
src
planner
planner_pkg
__init__.py
SimpleOneTrailerSystem.py
planner_node.py
...
package.xml
setup.py
package.xml
<?xml version="1.0"?>
<package format="2">
<name>planner_pkg</name>
<version>0.0.1</version>
<description>This package contains algorithm for park planner</description>
<maintainer email=""></maintainer>
<license>Apache License 2.0</license>
<exec_depend>rclpy</exec_depend>
<exec_depend>std_msgs</exec_depend>
<!-- These test dependencies are optional
Their purpose is to make sure that the code passes the linters -->
<test_depend>ament_copyright</test_depend>
<test_depend>ament_flake8</test_depend>
<test_depend>ament_pep257</test_depend>
<test_depend>python3-pytest</test_depend>
<export>
<build_type>ament_python</build_type>
</export>
</package>
setup.py:
from setuptools import setup
package_name = 'planner_pkg'
setup(
name=package_name,
version='0.0.0',
packages=[package_name],
data_files=[
('share/ament_index/resource_index/packages',
['resource/' + package_name]),
('share/' + package_name, ['package.xml']),
],
install_requires=['setuptools'],
zip_safe=True,
author='',
author_email='',
maintainer='',
maintainer_email='',
keywords=['ROS'],
classifiers=[
'Intended Audience :: Developers',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python',
'Topic :: Software Development',
],
description='Package containing examples of how to use the rclpy API.',
license='Apache License, Version 2.0',
tests_require=['pytest'],
entry_points={
'console_scripts': [
'planner_node = planner_pkg.planner_node:main',
],
},
)

First, according to the Module Search Path docs, when you do import something, Python looks for that something in the following places:
From the built-in modules
sys.path, which is a list containing:
The directory of the input script
PYTHONPATH, which is an environment variable containing a list of directories
The installation-dependent default
Second, when you build your ROS2 Python package (by calling colcon build invoking the ament_python build type), your Python codes will be copied over to an install folder with a tree structure like this:
install
...
├── planner_pkg
│   ├── bin
│   │   └── planner_node
│   ├── lib
│   │   └── python3.6
│   │   └── site-packages
│   │   ├── planner_pkg
│   │   │   ├── __init__.py
│   │   │   ├── planner_node.py
│   │   │   └── SimpleOneTrailerSystem.py
...
Now, when you do import SimpleOneTrailerSystem, Python will first search for it from the built-in modules, which for sure it won't find there. Next on the list is from sys.path. You can add a print(sys.path) at the top of planner_node.py to see something like this list:
['/path/to/install/planner_pkg/bin',
'/path/to/install/planner_pkg/lib/python3.6/site-packages',
'/opt/ros/eloquent/lib/python3.6/site-packages',
'/usr/lib/python36.zip',
'/usr/lib/python3.6',
...other Python3.6 installation-dependent dirs...
]
First on the sys.path list is the bin folder of the input script. There are only executables there, no SimpleOneTrailerSystem.py file/module, so that import will fail.
The next on the list would be the planner_pkg/lib/pythonX.X/site-packages, and as you can see from the tree structure above, there is a SimpleOneTrailerSystem.py module BUT it is under a planner_pkg folder. So a direct import like this
import SimpleOneTrailerSystem
will not work. You need to qualify it with the package folder like this:
import planner_pkg.SimpleOneTrailerSystem
There are 2 ways to get around this.
Modify sys.path before import-ing SimpleOneTrailerSystem
import sys
sys.path.append("/path/to/install/planner_pkg/lib/python3.6/site-packages/planner_pkg")
import SimpleOneTrailerSystem
This approach adds the path to the planner_pkg install directory to the sys.path list so that you don't need to specify it in subsequent imports.
Modify PYTHONPATH before running your ROS2 node
$ colcon build
$ source install/setup.bash
$ export PYTHONPATH=$PYTHONPATH:/path/to/install/planner_pkg/lib/python3.6/site-packages/planner_pkg
$ planner_node
This approach is almost the same as the first one, but there is no code change involved (and no rebuilding involved), as you only need to modify the PYTHONPATH environment variable.

I had the same problem and fixed it by modifying setup.py.
Add:
('lib/' + package_name, [package_name+'/SimpleOneTrailerSystem.py']),
to "data_files" list.
setup.py:
from setuptools import setup
package_name = 'planner_pkg'
setup(
name=package_name,
version='0.0.0',
packages=[package_name],
data_files=[
('share/ament_index/resource_index/packages',
['resource/' + package_name]),
('share/' + package_name, ['package.xml']),
('lib/' + package_name, [package_name+'/SimpleOneTrailerSystem.py']),
],
install_requires=['setuptools'],
zip_safe=True,
author='',
author_email='',
maintainer='',
maintainer_email='',
keywords=['ROS'],
classifiers=[
'Intended Audience :: Developers',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python',
'Topic :: Software Development',
],
description='Package containing examples of how to use the rclpy API.',
license='Apache License, Version 2.0',
tests_require=['pytest'],
entry_points={
'console_scripts': [
'planner_node = planner_pkg.planner_node:main',
],
},
)

For CMAKE packages with cpp and python code you can add the following lines to your CMakeLists.txt.
This will copy your python_pkg folder into your install environment to "lib/python{version}/site-packages" which is per default included in your python path.
# for python code
find_package(ament_cmake_python REQUIRED)
find_package(rclpy REQUIRED)
install(DIRECTORY
../path_to_python_pkg
DESTINATION lib/python3.8/site-packages
)
install(PROGRAMS
${PROJECT_NAME}/python_node.py
DESTINATION lib/${PROJECT_NAME}
)
ament_python_install_package(${PROJECT_NAME})
If you are using e.g. Dashing or Eloquent it is "python3.6", for Foxy or newer it is "python3.8"

New to creating a Python Package

I tried making one to post on my GitHub and help users download dependencies for the program, but in the end, all it does is generate duplicate files. I was hoping to have (in the end) a package where the user could enter:
>>> import my_package
>>> my_package.main
but that isn't working. I've looked at several different websites and different templates, but seem to be getting nowhere with this.
Directory structure
Kodimer_Project
├── LICENSE
├── MANIFEST.in
├── Makefile
├── README.md
├── requirements.txt
├── setup.py
└── wav2bin
├── __version__.py
├── imgs
│   ├── App_Icon.gif
│   ├── App_Icon.ico
│   └── __init__.py
└── src
├── DrawGraph.py
├── GraphicInterface.py
├── SplashScreen.py
├── __init__.py
├── __main__.py
└── helper_functions.py
setup.py
From: https://github.com/kennethreitz/setup.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Note: To use the 'upload' functionality of this file, you must:
# $ pip install twine
import io
import os
import sys
from shutil import rmtree
from setuptools import find_packages, setup, Command
# Package meta-data.
NAME = 'wav2bin'
DESCRIPTION = 'GUI graphing tool used concurrently with lab.'
URL = 'https://github.com/jvanderen1/Kodimer_Project'
EMAIL = 'jvanderen1#gmail.com'
AUTHOR = 'Joshua Van Deren'
# What packages are required for this module to be executed?
REQUIRED = [
'matplotlib',
'numpy',
'scipy'
]
# The rest you shouldn't have to touch too much :)
# ------------------------------------------------
# Except, perhaps the License and Trove Classifiers!
# If you do change the License, remember to change the Trove Classifier for that!
here = os.path.abspath(os.path.dirname(__file__))
# Import the README and use it as the long-description.
# Note: this will only work if 'README.md' is present in your MANIFEST.in file!
with io.open(os.path.join(here, 'README.md'), encoding='utf-8') as f:
LONG_DESCRIPTION = '\n' + f.read()
# Load the package's __version__.py module as a dictionary.
about = {}
with open(os.path.join(here, NAME, '__version__.py')) as f:
exec(f.read(), about)
class UploadCommand(Command):
"""Support setup.py upload."""
description = 'Build and publish the package.'
user_options = []
#staticmethod
def status(s):
"""Prints things in bold."""
print('\033[1m{0}\033[0m'.format(s))
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
try:
self.status('Removing previous builds…')
rmtree(os.path.join(here, 'dist'))
except OSError:
pass
self.status('Building Source and Wheel (universal) distribution…')
os.system('{0} setup.py sdist bdist_wheel --universal'.format(sys.executable))
self.status('Uploading the package to PyPi via Twine…')
os.system('twine upload dist/*')
sys.exit()
# Where the magic happens:
setup(
name=NAME,
version=about['__version__'],
description=DESCRIPTION,
long_description=LONG_DESCRIPTION,
author=AUTHOR,
author_email=EMAIL,
url=URL,
package_dir={'': 'wav2bin'},
packages=find_packages(exclude=('tests', 'docs')),
# If your package is a single module, use this instead of 'packages':
# py_modules=['mypackage'],
entry_points={
'gui_scripts': ['wav2bin = wav2bin.__main__:main'],
},
install_requires=REQUIRED,
python_requires='>=3',
include_package_data=True,
license='MIT',
classifiers=[
# Trove classifiers
# Full list: https://pypi.python.org/pypi?%3Aaction=list_classifiers
'License :: OSI Approved :: MIT License',
'Programming Language :: Python :: 3 :: Only',
'Natural Language :: English',
'Topic :: Scientific/Engineering :: Human Machine Interfaces',
'Topic :: Software Development :: User Interfaces'
],
# $ setup.py publish support.
cmdclass={
'upload': UploadCommand,
},
)
wav2bin/src/__main__.py
Snippet of code:
if __name__ == '__main__':
main()

After you have some experience with various packaging modules what you would usually do is decide how professional you want your packaging to be? Do you want to host it on pypi? Distribute it from github? Pass it along to friends?
That's how you pick your packaging method, but first you should probably get some experience with the existing packaging modules, the two most popular ones are:
setuptools which is what I usually go with and to which I linked a good tutorial
distutils an older api to distribute packages but it's still around and is also very good to know
If then you decide this is an overkill and you want a less professional approach you can always do it manually.
Either installation to the python package folder, which for pip usually means something like entering the packages root folder and entering either
pip install .
If you are certain of your package, or
pip install -e .
For installing in edit mode if you still wish to keep the package malleable
or having the package in your python path in some other manner before import is mandatory.