We Keep Coding

PyPI - allow importing a module using a different name

I recently created a simple python package / library and uploaded it to (PyPI). That's all fine - I can upload my project to pypi and download it again using pip, but I can't import it because it contains a full stop .
My package is called dcode.py (which obviously causes issues with importing), I'd like to be able to import it by just typing import dcode (without the .py). I've looked around for a solution but can't seem to find one - I'd like to be able to import it under an alias (dcode) instead of dcode.py (which obviously doesn't work).
Is there a way to do this? I'd assume there must be (for example, both the packages discord.py and py-cord can both just be imported using import discord. Does anyone know how to do this?
Just in case you need it, my pyproject.toml file looks like this:
[project]
name = "dcode.py"
version = "0.0.2"
authors = [
{ name = "Dylan Rogers", email = "opendylan#proton.me" },
]
description = "dcode - the Python package for everything"
readme = "README.md"
requires-python = ">=2.0"
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
]
[project.urls]
"Homepage" = "https://github.com/dylanopen/dcode.py"
"Bug Tracker" = "https://github.com/dylanopen/dcode.py/issues"
And my entire project is on github.
Here is the PyPI project
I've tried importing dcode.py:
import dcode.py
But I just get this error:
File "/run/media/dylan/Programming/dcode.py-test/test.py", line 1, in <module>
import dcode_py
ModuleNotFoundError: No module named 'dcode_py'
If it helps, I'm using Manjaro Linux (no idea why it would help though, I just need to know how to allow users to import dcode.py by typing import dcode, like they import discord.py as discord)
Thank you so much for any help!

This is the recommended directory structure your project should have:
dcode.py/
├── LICENSE
├── pyproject.toml
├── README.md
├── src/
│ └── dcode/
│ ├── __init__.py
│ └── io/
│ ├── __init__.py
│ └── file.py
└── tests/
Here dcode (as defined by dcode.py/src/dcode) is the name of the only top-level importable package. And it also contains a sub-package dcode.io (as defined by dcode.py/src/dcode/io).
The name of the actual distribution project as seen on PyPI and that gets pip-installed is defined in the package metadata, in the pyproject.toml file:
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
[project]
name = "dcode.py"
version = "0.0.1"
authors = [
{ name = "Dylan Rogers", email = "opendylan#proton.me" },
]
description = "dcode - the Python package for everything"
readme = "README.md"
requires-python = ">=3.8"
[project.urls]
"Homepage" = "https://github.com/dylanopen/dcode.py"
"Bug Tracker" = "https://github.com/dylanopen/dcode.py/issues"
[tool.setuptools.packages.find]
where = ["src"]
Do not push the dist directory containing the built sdist and wheel distribution packages artifacts into the source code repository.
Do not push any *.egg-info directory into the source code repository.
Place all the actual Python code inside a src directory.
The name of the project can contain a dot such as dcode.py, although I would recommend against it.
The name of the actual top-level import packages and modules can be completely different than the name of the distribution project.
If you declare a classifier for the license in the package metadata in pyproject.toml, then it should match the actual license file in the source code repository (you had a GNU GPL v3.0 license file but a MIT license classifier).
References:
https://packaging.python.org/en/latest/tutorials/packaging-projects/
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
https://sinoroc.gitlab.io/kb/python/packaging.html

Two python packages with the same structure use function of the same name from other package

I have created 2 packages, named A and B. Both packages have the same structure to them, as they do very similar things, e.g. their structure looks like this:
A/
__init__.py
subpackage1/
__init__.py
submodule1.py
subpackage2/
__init__.py
submodule2.py
setup.py
README.md
requirements.txt
They share the same subpackage, submodule and function names. Each module has a main function, which does the argparsing for me and calls a function with those params. In my setup.py, I specified additional entry points, so that I can call the modules from the command line:
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
with open('requirements.txt') as f:
requirements = f.readlines()
setuptools.setup(
name="A",
version="0.0.1",
author="Me",
author_email="me#myself.com",
description="Test package",
long_description=long_description,
long_description_content_type="text/markdown",
packages=setuptools.find_packages(),
entry_points ={
'console_scripts': [
'command1 = subpackage1.submodule1:main',
'command2 = subpackage2.submodule2:main'
]
},
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.6',
install_requires = requirements
)
When I install the package in a blank docker container, it works fine and I can call my functions with 'command1' and 'command2' from the command line.
As stated before, package B has exactly the same setup.py file, except for the name. If I install that as well, the package A now uses the entry points of package B instead of its own. That means, I call the function with the right name, but from the wrong package.
I want to have them side-by-side in my docker container. How do I have to adjust my packages, so that the system can differentiate between them?
I installed the packages via pip from wheels that I generated.

First impression, the directory structure seems wrong: the first red flag is that there shouldn't be a __init__.py in the same directory as setup.py, the second red flag is that the directories next to setup.py can not be sub-packages, they are top-level packages.
In your example the top-level packages are subpackage1 and subpackage2 in the project A, and in the project B as well. So in both cases, after installation the importable items are import subpackage1 and import subpackage2. It also means that when you install A, then B, the top-level packages of B overwrite those that were previously installed as part of A (since they have the exact same name).
What you probably want to do is to add a directory a in the project A right next to its setup.py, and move both subpackageN as well as the __init__.py into that a directory (same in the project B). So that directory structure looks like:
A/
a/
__init__.py
subpackage1/
__init__.py
submodule1.py
subpackage2/
__init__.py
submodule2.py
setup.py
README.md
requirements.txt
Then the imports will look like:
import a.subpackage1
import b.subpackage1
from a import subpackage2 as subpackagea2
from b import subpackage2 as subpackageb2
Following that the setup.py files should be adjusted accordingly:
# ...
setuptools.setup(
# ...
entry_points ={
'console_scripts': [
'commanda1 = a.subpackage1.submodule1:main',
'commanda2 = a.subpackage2.submodule2:main'
]
},
)

How to Import python package from another directory?

I have a project that is structured as follows:
project
├── api
│ ├── __init__.py
│ └── api.py
├── instance
│ ├── __init__.py
│ └── config.py
├── package
│ ├── __init__.py
│ └── app.py
├── requirements.txt
└── tests
└── __init__.py
I am trying to call the config.py file from the package/app.py as shown below:
# package/app.py
from instance import config
# I've also tried
import instance.config
import ..instance.config
from ..instance import config
But I always get the following error:
Traceback (most recent call last):
File "/home/csymvoul/projects/project/package/app.py", line 1, in <module>
from instance import config
ModuleNotFoundError: No module named 'instance'
Modifying the sys.path is not something I want to do.
I know that this question is very much answered but the answers that were given, did not work for me.
EDIT: When moving the app.py to the root folder it works just fine. But I need to have it under the package folder.

You can add the parent directory to PYTHONPATH, in order to achieve that, you can use OS depending path in the "module search path" which is listed in sys.path. So you can easily add the parent directory like following:
import sys
sys.path.insert(0, '..')
from instance import config
Note that the previous code uses a relative path, so you must launch the file inside the same location or it will likely not work. To launch from anywhere, you can use the pathlib module.
from pathlib import Path
import sys
path = str(Path(Path(__file__).parent.absolute()).parent.absolute())
sys.path.insert(0, path)
from instance import config
However, the previous approach is more a hack than anything, in order to do things right, you'll first need to reshape your project structure according to this very detailed blog post python packaging, going for the recommended way with a src folder.
Your directory layout must look like this:
project
├── CHANGELOG.rst
├── README.rst
├── requirements.txt
├── setup.py
├── src
│   ├── api
│   │   ├── api.py
│   │   └── __init__.py
│   ├── instance
│   │   ├── config.py
│   │   └── __init__.py
│   └── package
│   ├── app.py
│   └── __init__.py
└── tests
└── __init__.py
Note that you don't really need the requirements.txt because you can declare the dependencies inside your setup.py.
A sample setup.py (adapted from here):
#!/usr/bin/env python
# -*- encoding: utf-8 -*-
from __future__ import absolute_import
from __future__ import print_function
import io
import re
from glob import glob
from os.path import basename
from os.path import dirname
from os.path import join
from os.path import splitext
from setuptools import find_packages
from setuptools import setup
def read(*names, **kwargs):
with io.open(
join(dirname(__file__), *names),
encoding=kwargs.get('encoding', 'utf8')
) as fh:
return fh.read()
setup(
name='nameless',
version='1.644.11',
license='BSD-2-Clause',
description='An example package. Generated with cookiecutter-pylibrary.',
author='mpr',
author_email='contact#ionelmc.ro',
packages=find_packages('src'),
package_dir={'': 'src'},
include_package_data=True,
zip_safe=False,
classifiers=[
# complete classifier list: http://pypi.python.org/pypi?%3Aaction=list_classifiers
'Development Status :: 5 - Production/Stable',
'Intended Audience :: Developers',
'License :: OSI Approved :: BSD License',
'Operating System :: Unix',
'Operating System :: POSIX',
'Operating System :: Microsoft :: Windows',
'Programming Language :: Python',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: PyPy',
# uncomment if you test on these interpreters:
# 'Programming Language :: Python :: Implementation :: IronPython',
# 'Programming Language :: Python :: Implementation :: Jython',
# 'Programming Language :: Python :: Implementation :: Stackless',
'Topic :: Utilities',
],
keywords=[
# eg: 'keyword1', 'keyword2', 'keyword3',
],
python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*',
install_requires=[
# eg: 'aspectlib==1.1.1', 'six>=1.7',
],
extras_require={
# eg:
# 'rst': ['docutils>=0.11'],
# ':python_version=="2.6"': ['argparse'],
},
setup_requires=[
# 'pytest-runner',
],
entry_points={
'console_scripts': [
'api = api.api:main',
]
},
)
The content of my api.py:
from instance import config
def main():
print("imported")
config.config()
The content of my config.py:
def config():
print("config imported successfully")
You can find all the previous here
Optional but recommended: create a virtual environment, I use venv (Python 3.3 <=) for that, inside the root of the project:
python -m venv .
And to activate:
source bin/activate
Now I can install the package:
Using pip install -e . (with the dot) command inside the root of the project
Your import from instance import config works now, to confirm you can run api.py with:
python src/api/api.py

In Python 3.3 onwards you don't need __init__.py files in your subdirectories for the purpose of imports. Having them can actually be misleading as it causes the creation of package namespaces in each folder containing an init file, as described here.
By removing all those __init__.py files you will be able to import files in the namespace package (including subdirectories) when running app.py, but that's still not want we want.
The Python interpreter still doesn't know how to reach your instance namespace. In order to do that you can use the PYTHONPATH environment variable, including a path that is parent to config.py. You can do that as suggested in #RMPR's answer with sys.path, or by directly setting the environment variable, for instance:
PYTHONPATH=/home/csymvoul/projects/project python3 /home/csymvoul/projects/project/package/app.py
Then importing the dependency like from instance import config.

New to creating a Python Package

I tried making one to post on my GitHub and help users download dependencies for the program, but in the end, all it does is generate duplicate files. I was hoping to have (in the end) a package where the user could enter:
>>> import my_package
>>> my_package.main
but that isn't working. I've looked at several different websites and different templates, but seem to be getting nowhere with this.
Directory structure
Kodimer_Project
├── LICENSE
├── MANIFEST.in
├── Makefile
├── README.md
├── requirements.txt
├── setup.py
└── wav2bin
├── __version__.py
├── imgs
│   ├── App_Icon.gif
│   ├── App_Icon.ico
│   └── __init__.py
└── src
├── DrawGraph.py
├── GraphicInterface.py
├── SplashScreen.py
├── __init__.py
├── __main__.py
└── helper_functions.py
setup.py
From: https://github.com/kennethreitz/setup.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Note: To use the 'upload' functionality of this file, you must:
# $ pip install twine
import io
import os
import sys
from shutil import rmtree
from setuptools import find_packages, setup, Command
# Package meta-data.
NAME = 'wav2bin'
DESCRIPTION = 'GUI graphing tool used concurrently with lab.'
URL = 'https://github.com/jvanderen1/Kodimer_Project'
EMAIL = 'jvanderen1#gmail.com'
AUTHOR = 'Joshua Van Deren'
# What packages are required for this module to be executed?
REQUIRED = [
'matplotlib',
'numpy',
'scipy'
]
# The rest you shouldn't have to touch too much :)
# ------------------------------------------------
# Except, perhaps the License and Trove Classifiers!
# If you do change the License, remember to change the Trove Classifier for that!
here = os.path.abspath(os.path.dirname(__file__))
# Import the README and use it as the long-description.
# Note: this will only work if 'README.md' is present in your MANIFEST.in file!
with io.open(os.path.join(here, 'README.md'), encoding='utf-8') as f:
LONG_DESCRIPTION = '\n' + f.read()
# Load the package's __version__.py module as a dictionary.
about = {}
with open(os.path.join(here, NAME, '__version__.py')) as f:
exec(f.read(), about)
class UploadCommand(Command):
"""Support setup.py upload."""
description = 'Build and publish the package.'
user_options = []
#staticmethod
def status(s):
"""Prints things in bold."""
print('\033[1m{0}\033[0m'.format(s))
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
try:
self.status('Removing previous builds…')
rmtree(os.path.join(here, 'dist'))
except OSError:
pass
self.status('Building Source and Wheel (universal) distribution…')
os.system('{0} setup.py sdist bdist_wheel --universal'.format(sys.executable))
self.status('Uploading the package to PyPi via Twine…')
os.system('twine upload dist/*')
sys.exit()
# Where the magic happens:
setup(
name=NAME,
version=about['__version__'],
description=DESCRIPTION,
long_description=LONG_DESCRIPTION,
author=AUTHOR,
author_email=EMAIL,
url=URL,
package_dir={'': 'wav2bin'},
packages=find_packages(exclude=('tests', 'docs')),
# If your package is a single module, use this instead of 'packages':
# py_modules=['mypackage'],
entry_points={
'gui_scripts': ['wav2bin = wav2bin.__main__:main'],
},
install_requires=REQUIRED,
python_requires='>=3',
include_package_data=True,
license='MIT',
classifiers=[
# Trove classifiers
# Full list: https://pypi.python.org/pypi?%3Aaction=list_classifiers
'License :: OSI Approved :: MIT License',
'Programming Language :: Python :: 3 :: Only',
'Natural Language :: English',
'Topic :: Scientific/Engineering :: Human Machine Interfaces',
'Topic :: Software Development :: User Interfaces'
],
# $ setup.py publish support.
cmdclass={
'upload': UploadCommand,
},
)
wav2bin/src/__main__.py
Snippet of code:
if __name__ == '__main__':
main()

After you have some experience with various packaging modules what you would usually do is decide how professional you want your packaging to be? Do you want to host it on pypi? Distribute it from github? Pass it along to friends?
That's how you pick your packaging method, but first you should probably get some experience with the existing packaging modules, the two most popular ones are:
setuptools which is what I usually go with and to which I linked a good tutorial
distutils an older api to distribute packages but it's still around and is also very good to know
If then you decide this is an overkill and you want a less professional approach you can always do it manually.
Either installation to the python package folder, which for pip usually means something like entering the packages root folder and entering either
pip install .
If you are certain of your package, or
pip install -e .
For installing in edit mode if you still wish to keep the package malleable
or having the package in your python path in some other manner before import is mandatory.

Packaging stub files

Let's say I have very simple package with a following structure:
.
├── foo
│   ├── bar
│   │   └── __init__.py
│   └── __init__.py
└── setup.py
Content of the files:
setup.py:
from distutils.core import setup
setup(
name='foobar',
version='',
packages=['foo', 'foo.bar'],
url='',
license='Apache License 2.0',
author='foobar',
author_email='',
description=''
)
foo/bar/__init__.py:
def foobar(x):
return x
The remaining files are empty.
I install the package using pip:
cd foobar
pip install .
and can confirm it is installed correctly.
Now I want to create a separate package with stub files:
.
├── foo
│   ├── bar
│   │   └── __init__.pyi
│   └── __init__.pyi
└── setup.py
Content of the files:
setup.py:
from distutils.core import setup
import sys
import pathlib
setup(
name='foobar_annot',
version='',
packages=['foo', 'foo.bar'],
url='',
license='Apache License 2.0',
author='foobar',
author_email='',
description='',
data_files=[
(
'shared/typehints/python{}.{}/foo/bar'.format(*sys.version_info[:2]),
["foo/bar/__init__.pyi"]
),
],
)
foo.bar.__init__.pyi:
def foobar(x: int) -> int: ...
I can install this package, see that it creates anaconda3/shared/typehints/python3.5/foo/bar/__init__.pyi in my Anaconda root, but it doesn't look like it is recognized by PyCharm (I get no warnings). When I place pyi file in the main package everything works OK.
I would be grateful for any hints how to make this work:
I've been trying to make some sense from PEP 484 - Storing and distributing stub files but to no avail. Even pathlib part seem to offend my version of distutils
PY-18597 and https://github.com/python/mypy/issues/1190#issuecomment-188526651 seem to be related but somehow I cannot connect the dots.
I tried putting stubs in the .PyCharmX.X/config/python-skeletons but it didn't help.'
Some things that work, but don't resolve the problem:
Putting stub files in the current project and marking as sources.
Adding stub package root to the interpreter path (at least in some simple cases).
So the questions: How to create a minimal, distributable package with Python stubs, which will be recognized by existing tools. Based on the experiments I suspect one of two problems:
I misunderstood the structure which should be created by the package in the shared/typehints/pythonX.Y - if this is true, how should I define data_files?
PyCharm doesn't consider these files at all (this seem to be contradicted by some comments in the linked issue).
It suppose to work just fine, but I made some configure mistake and looking for external problem which doesn't exist.
Are there any established procedures to troubleshoot problems like this?

Problem is that you didn't include the foo/__init__.pyi file in your stub distribution. Even though it's empty, it makes foo a stub files package, and enables search for foo.bar.
You can modify the data_files in your setup.py to include both
data_files=[
(
'shared/typehints/python{}.{}/foo/bar'.format(*sys.version_info[:2]),
["foo/bar/__init__.pyi"]
),
(
'shared/typehints/python{}.{}/foo'.format(*sys.version_info[:2]),
["foo/__init__.pyi"]
),
],