We Keep Coding

Python cannot find newly installed module

I've created a module using the following setup.py
# -*- coding: utf-8 -*-
# Learn more: https://github.com/kennethreitz/setup.py
from setuptools import setup, find_packages
with open('README.md') as f:
readme = f.read()
with open('LICENSE') as f:
license = f.read()
setup(
name='mymod',
version='1.0a1',
description='test',
long_description=readme,
long_description_content_type="text/markdown",
author='Ray Salemi',
author_email='ray#raysalemi.com',
url='https://rayboston#bitbucket.org/rayboston/mymod',
license=license,
packages=find_packages(exclude=('tests', 'docs', 'examples'))
)
But when I try to install it using
% python setup.py install
I see that it gets installed in my site packages:
Processing mymod-1.0a1-py3.8.egg
Copying mymod-1.0a1-py3.8.egg to /Users/raysalemi/PycharmProjects/testenv/lib/python3.8/site-packages
Adding mymod 1.0a1 to easy-install.pth file
Installed /Users/raysalemi/PycharmProjects/testenv/lib/python3.8/site-packages/mymod-1.0a1-py3.8.egg
Processing dependencies for mymod==1.0a1
Finished processing dependencies for mymod==1.0a1
(testenv) (base) raysalemi#WriteNow mymod % cd ../testenv
(testenv) (base) raysalemi#WriteNow testenv % python
Python 3.8.3 (default, Jul 2 2020, 11:26:31)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import mymod
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'mymod'
How do I debug this? I can't see an error.
I'm running Big Sur 11.0.1 and
Python 3.8.3 from Anaconda
Pip shows the module is there
Package Version
---------- -------
pip 20.3.1
mymod 1.0a1
setuptools 41.2.0
The problem is that the package is being misnamed:
(testenv) (base) raysalemi#WriteNow site-packages % ls
__pycache__ mymod-1.0a0-py3.8.egg
easy-install.pth mymod-1.0a0.dist-info
easy_install.py setuptools
pip setuptools-41.2.0.dist-info
pip-20.3.1.dist-info src
pkg_resources
It is mymod-1.0a0-py3.8.egg instead of mymod

To debug you can run the setup:
python setup.py sdist --formats=gztar
and unzip the resulting .tar.gz file and check if all your source code files are in it.
(or use --formats=zip instead of gztar to get a simpler file to extract)
The resulting package is always of the form package_name-package_version, so the name you received is not incorrect. (In case you are wondering, you can find the valid package_version formatting rules here.)
You can later use this package by adding it to the requirements.txt file of the project you want to be dependent on it. E.g.
my-package>=1.2.0,<2.0.0
In your case, since the version is a pre-release (mymod-1.0a0-py3.8.egg ==> version is 1.0a0-py3.8.egg which means version 1.0 pre-relase version alpha0-py3.8).
The version 1.0a0-py3.8.egg < than version 1.0 (pre-release always < release with same number), so you will need something like >0,<2.0.
Personally, I put the source code in the repo under src/ and then select these files in setup.py using:
packages=find_namespace_packages(where="src")
There are other parameters I recommend using e.g. make sure environment has a new enough setuptools to recognize find_namespace_packages, take list of dependencies from requirements.txt files etc.:
from setuptools import setup, find_namespace_packages
with open('requirements.txt') as f:
required = f.read().splitlines()
setup(
name='your_project_name',
version='1.0.0',
description='your project description',
url='your repo url',
author='your username',
author_email='your email',
license='your license type',
package_dir={'': 'src'},
setup_requires='setuptools>=45.2.0',
packages=find_namespace_packages(where="src"),
install_requires=required,
data_files=['requirements.txt'],
include_package_data=True
)
See the full list of options and what they are for in the documentation.

I found my problem.
My source directory was named src not mymod. So there was a src directory in site-packages instead of a mymod directory. This is a surprise since the package is named in setup.py.

Why customer python package can not be imported?

I created my own python package named jjnsegutils and upload it to Pypi website. But after I successfully install it by pip install jjnsegutils, I still can not import it. Error shows: ModuleNotFoundError: No module named 'jjnsegutils'.
Details about the whole procedure are as follows.
Package structure and details
The structure of my package is:
jjnsequtils
├─ __init__.py
├─ myutil
├─ __init__.py
├─ myutil.py
├─ LICENSE
├─ README.md
├─ setup.py
Two __init__.py files are empty.
In my setup.py:
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="jjnsegutils", # Replace with your own username
version="0.0.10",
author="Jingnan",
author_email="jiajingnan2222#gmail.com",
description="A package of common utilities for Medical images segmentation and evaluation.",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/Ordgod/jjnsegutils",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
python_requires='>=3.6',
)
How do I generate the package?
I use command python setup.py sdist bdist_wheel to generate distribute files.
logging information:
running sdist
running egg_info
writing jjnsegutils.egg-info/PKG-INFO
writing dependency_links to jjnsegutils.egg-info/dependency_links.txt
writing top-level names to jjnsegutils.egg-info/top_level.txt
reading manifest file 'jjnsegutils.egg-info/SOURCES.txt'
writing manifest file 'jjnsegutils.egg-info/SOURCES.txt'
running check
creating jjnsegutils-0.0.10
creating jjnsegutils-0.0.10/jjnsegutils.egg-info
creating jjnsegutils-0.0.10/myutil
copying files to jjnsegutils-0.0.10...
copying README.md -> jjnsegutils-0.0.10
copying setup.py -> jjnsegutils-0.0.10
copying jjnsegutils.egg-info/PKG-INFO -> jjnsegutils-0.0.10/jjnsegutils.egg-info
copying jjnsegutils.egg-info/SOURCES.txt -> jjnsegutils-0.0.10/jjnsegutils.egg-info
copying jjnsegutils.egg-info/dependency_links.txt -> jjnsegutils-0.0.10/jjnsegutils.egg-info
copying jjnsegutils.egg-info/top_level.txt -> jjnsegutils-0.0.10/jjnsegutils.egg-info
copying myutil/__init__.py -> jjnsegutils-0.0.10/myutil
copying myutil/myutil.py -> jjnsegutils-0.0.10/myutil
Writing jjnsegutils-0.0.10/setup.cfg
Creating tar archive
removing 'jjnsegutils-0.0.10' (and everything under it)
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
copying build/lib/myutil.py -> build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/myutil
copying build/lib/myutil/__init__.py -> build/bdist.linux-x86_64/wheel/myutil
copying build/lib/myutil/myutil.py -> build/bdist.linux-x86_64/wheel/myutil
running install_egg_info
Copying jjnsegutils.egg-info to build/bdist.linux-x86_64/wheel/jjnsegutils-0.0.10-py3.7.egg-info
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.linux-x86_64/wheel/jjnsegutils-0.0.10.dist-info/WHEEL
creating 'dist/jjnsegutils-0.0.10-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'myutil.py'
adding 'myutil/__init__.py'
adding 'myutil/myutil.py'
adding 'jjnsegutils-0.0.10.dist-info/LICENSE'
adding 'jjnsegutils-0.0.10.dist-info/METADATA'
adding 'jjnsegutils-0.0.10.dist-info/WHEEL'
adding 'jjnsegutils-0.0.10.dist-info/top_level.txt'
adding 'jjnsegutils-0.0.10.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
How do I distribute the package?
I use command twine upload dist/* to upload the files in dist to Pypi.
How do I install the package?
I use command pip install jjnsegutils to install it into my own environment named py37 created by conda. It shows
Collecting jjnsegutils
Downloading jjnsegutils-0.0.10-py3-none-any.whl (11 kB)
Installing collected packages: jjnsegutils
Successfully installed jjnsegutils-0.0.10
Then I type $ python in my terminal to enter the python interractive terminal. and type:
>>> import jjnsegutils
But it shows:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'jjnsegutils'
What actions I have tried to diagnosize the issue?
I checked the package by command $ pip show jjnsegutils, it shows:
Name: jjnsegutils
Version: 0.0.10
Summary: A package of common utilities for Medical images segmentation and evaluation.
Home-page: https://github.com/Ordgod/jjnsegutils
Author: Jingnan
Author-email: jiajingnan2222#gmail.com
License: UNKNOWN
Location: /home/jjia/.conda/envs/py37/lib/python3.7/site-packages
Requires:
Required-by:
And I checked it futher by $ ls /home/jjia/.conda/envs/py37/lib/python3.7/site-packages, it shows:
... # other packages
jjnsegutils-0.0.10.dist-info
... # other packages
I thought there should be anotehr directory named jjnsegutils along with jjnsegutils-0.0.10.dist-info. But I did not see it. Is this the reason that I can not import my own package?
I make sure that the virtual environment is always the same one named py37 during the whole procedure. I used CentOS, python3.7.
Looking forward to any discussion and advice. Thanks very much!

You should import myutil.
packages is a list of all Python import packages that should be included in the Distribution Package. Instead of listing each package manually, we can use find_packages() to automatically discover all packages and subpackages. In this case, the list of packages will be example_pkg as that’s the only package present.
https://packaging.python.org/tutorials/packaging-projects/

Python creating pip package - module not found

I am trying to create a python package to distribute my code. I am not getting any error in creating a package, and installing created package.
However, after installation when I am trying to import the package I am getting error ModuleNotFoundError:
Following is the code
hello_world.py
class HelloWorld:
def print_msg(self):
print("Hello World")
setup.py
from setuptools import setup, find_packages
setup(
name = "HelloWorld",
version = "0.1",
packages = find_packages(),
)
create package
▶ python setup.py bdist_wheel
running bdist_wheel
running build
installing to build/bdist.macosx-10.14-x86_64/wheel
running install
running install_egg_info
running egg_info
writing HelloWorld.egg-info/PKG-INFO
writing dependency_links to HelloWorld.egg-info/dependency_links.txt
writing top-level names to HelloWorld.egg-info/top_level.txt
reading manifest file 'HelloWorld.egg-info/SOURCES.txt'
writing manifest file 'HelloWorld.egg-info/SOURCES.txt'
Copying HelloWorld.egg-info to build/bdist.macosx-10.14-x86_64/wheel/HelloWorld-0.1-py3.7.egg-info
running install_scripts
creating build/bdist.macosx-10.14-x86_64/wheel/HelloWorld-0.1.dist-info/WHEEL
creating 'dist/HelloWorld-0.1-py3-none-any.whl' and adding 'build/bdist.macosx-10.14-x86_64/wheel' to it
adding 'HelloWorld-0.1.dist-info/METADATA'
adding 'HelloWorld-0.1.dist-info/WHEEL'
adding 'HelloWorld-0.1.dist-info/top_level.txt'
adding 'HelloWorld-0.1.dist-info/RECORD'
removing build/bdist.macosx-10.14-x86_64/wheel
Installing package
~/PycharmProjects/test_dist ▶ pip install dist/HelloWorld-0.1-py3-none-any.whl
Processing ./dist/HelloWorld-0.1-py3-none-any.whl
Installing collected packages: HelloWorld
Successfully installed HelloWorld-0.1
~/PycharmProjects/test_dist ▶ pip freeze
HelloWorld==0.1
Error While importing module
>>> import HelloWorld
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'HelloWorld'

Where is hello_world.py? Is it at the root folder adjacent to setup.py? Or in some subdirectory? I suspect the former. That means you don't have any packages so find_packages() returns an empty list so setuptools don;t package any code into the package.
Your hello_world.py isn't a packages (a directory with file __init__.py), it's a standalone module and such modules must be packed using py_modules. This is how you should write your setup.py:
from setuptools import setup
setup(
name = "HelloWorld",
version = "0.1",
py_modules = ['hello_world'],
)

how to install and use open-tamil package?

I want to use the open-tamil package for processing tamil text.
I downloaded it as "open-tamil-master" package which has a setup.py file. but on running the file it says "error no commands supplied".
also available in https://github.com/arcturusannamalai/open-tamil
can some one say me how to install and use it??

To download and install (using setup.py) open-tamil you can do the following
git clone https://github.com/arcturusannamalai/open-tamil.git
cd open-tamil
sudo python setup.py install
it says "error no commands supplied"
That is because you need to tell setup.py what to do by supplying a command. In this case that is install.
Now you can import its modules etc:
tim#tim-N53SN:~$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import transliterate
>>> import tamil
>>>
The full list of what commands you can supply to setup.py:
Standard commands:
build build everything needed to install
build_py "build" pure Python modules (copy to build directory)
build_ext build C/C++ extensions (compile/link to build directory)
build_clib build C/C++ libraries used by Python extensions
build_scripts "build" scripts (copy and fixup #! line)
clean clean up temporary files from 'build' command
install install everything from build directory
install_lib install all Python modules (extensions and pure Python)
install_headers install C/C++ header files
install_scripts install scripts (Python or otherwise)
install_data install data files
sdist create a source distribution (tarball, zip file, etc.)
register register the distribution with the Python package index
bdist create a built (binary) distribution
bdist_dumb create a "dumb" built distribution
bdist_rpm create an RPM distribution
bdist_wininst create an executable installer for MS Windows
upload upload binary package to PyPI
check perform some checks on the package

Installing my own module from pyPI, using pip, is not working

Short summary
I can upload a package to pyPI, and install it with pip, but Python is not seeing it when I try to import it (it says there is no module with the relevant name). I am using Python 2.7 in Windows 7 (Anaconda distribution, using iPython interpreter).
Detailed summary
I am learning how to upload a package (mymathFoo) at the cheese shop (pyPI), and trying to make sure I can install it once it is uploaded. When I manually paste the package folder into my Lib/site-packages directory, it imports and works fine from my interpreter.
What I have done:
0. Write package
It's a silly little package with an add and subtract module (e.g., add(1,2)). The directory is structured as follows:
\mymathFoo
__init__.py
add.py #adds two numbers
subtract.py #subtract two numbers
setup.py #shown below
README.txt #simple description of package
The __init__.py file is as follows:
from add import add
from subtract import subtract
1. Register the package at pyPI
Namely, from the command line, I enter:
python setup.py register
Which returns:
running register
running check
Registering mymathFoo to http://pypi.python.org/pypi
Server response (200): OK
Note my setup.py file is:
from distutils.core import setup
setup(name="mymathFoo",
version="0.6.2",
url="http://mymathfoo.net",
maintainer='Math Foo',
maintainer_email='mathfoo#math.org',
py_modules=['add','subtract'],
description="Silly little math turd to add/subtract.")
Note if I change it to py_modules='mymathFoo' I get the same error as below.
2. Upload the package
At the command line, I entered:
python setup.py sdist bdist_wininst upload
And the response was:
running sdist
running check
warning: sdist: manifest template 'MANIFEST.in' does not exist (using default fi
le list)
writing manifest file 'MANIFEST'
creating mymathFoo-0.6.3
copying files to mymathFoo-0.6.3...
copying README.txt -> mymathFoo-0.6.3
copying add.py -> mymathFoo-0.6.3
copying setup.py -> mymathFoo-0.6.3
copying subtract.py -> mymathFoo-0.6.3
creating dist
creating 'dist\mymathFoo-0.6.3.zip' and adding 'mymathFoo-0.6.3' to it
adding 'mymathFoo-0.6.3\add.py'
adding 'mymathFoo-0.6.3\PKG-INFO'
adding 'mymathFoo-0.6.3\README.txt'
adding 'mymathFoo-0.6.3\setup.py'
adding 'mymathFoo-0.6.3\subtract.py'
removing 'mymathFoo-0.6.3' (and everything under it)
running bdist_wininst
running build
running build_py
creating build
creating build\lib
copying add.py -> build\lib
copying subtract.py -> build\lib
installing to build\bdist.win-amd64\wininst
running install_lib
creating build\bdist.win-amd64
creating build\bdist.win-amd64\wininst
creating build\bdist.win-amd64\wininst\PURELIB
copying build\lib\add.py -> build\bdist.win-amd64\wininst\PURELIB
copying build\lib\subtract.py -> build\bdist.win-amd64\wininst\PURELIB
running install_egg_info
Writing build\bdist.win-amd64\wininst\PURELIB\mymathFoo-0.6.3-py2.7.egg-info
creating 'c:\users\eric\appdata\local\temp\tmp65xxe2.zip' and adding '.' to it
adding 'PURELIB\add.py'
adding 'PURELIB\mymathFoo-0.6.3-py2.7.egg-info'
adding 'PURELIB\subtract.py'
removing 'build\bdist.win-amd64\wininst' (and everything under it)
running upload
Submitting dist\mymathFoo-0.6.3.zip to http://pypi.python.org/pypi
Server response (200): OK
Submitting dist\mymathFoo-0.6.3.win-amd64.exe to http://pypi.python.org/pypi
Server response (200): OK
Things seem to have worked. So far so good.
3. Install this package locally using pip.
Then I go to install this package with pip at the command line:
pip install mymathFoo
To which I get:
Downloading/unpacking mymathFoo
Downloading mymathFoo-0.6.3.zip
Running setup.py (path:c:\users\eric\appdata\local\temp\pip_build_Eric\mymathF
oo\setup.py) egg_info for package mymathFoo
Installing collected packages: mymathFoo
Running setup.py install for mymathFoo
Successfully installed mymathFoo
Cleaning up...
Running the above causes the following directory to be copied into my Lib/site-packages folder:
mymathFoo-0.6.3-py2.7.egg-info
4. Import the package (not)
This is where I run into the problem. I open my iPython interpreter (using Anaconda distribution of Python, Windows 7):
import mymathFoo
I get:
Traceback (most recent call last):
File "<ipython-input-7-b7486b6a0225>", line 1, in <module>
import mymathFoo
ImportError: No module named mymathFoo
What am I missing? Why is my poor little module invisible?
Update
Note if I collapse all the files into the root directory (which I ultimately do not want to do), the errors go away. Unfortunately I will often want directories in my root directory, and nothing I have tried based on comments has fixed this.
I'm still looking for an answer, the discussion here looks promising:
http://www.scotttorborg.com/python-packaging/index.html#
I will work through that and post any solutions I find.
Discussion of related, but not the same, issues
"ImportError: No module named httplib2" even after installation
how to install package with pypi in python 2.7?
Note
This is based on some work I am doing with the book Python 101, by Michael Driscoll (it is in draft form right now).

Your package does install, just not the way you intended it to:
$ pip install mymathFoo
Downloading/unpacking mymathFoo
Using download cache from /Users/stu/tmp/pip-cache/http%3A%2F%2Fpypi.counsyl.com%2Froot%2Fpypi%2F%2Bf%2F64ef17a9135f05820c2d96050ce4315d%2FmymathFoo-0.6.3.zip
Running setup.py egg_info for package mymathFoo
Installing collected packages: mymathFoo
Running setup.py install for mymathFoo
Successfully installed mymathFoo
Cleaning up...
$ python
Type "help", "copyright", "credits" or "license" for more information.
>>> import add
>>> import subtract
>>>
Your setup.py says to install two modules, "add" and "subtract", and not to install a package named mymathFoo. Put add.py and subtract.py in a folder named 'mymathFoo', and set py_modules=['mymathFoo'] in setup.py.
Lastly, you can test your packaging without hitting pypi. Just run python setup.py install, and try to import you package.

it works for me:
>>> import mymathFoo
>>> dir(mymathFoo)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__']
though your module exports nothing. It looks like you changed your package since #Stu-Gla's answer, as you removed the add.py and subtract.py from sources.
By the way, you do not need to register a dummy package to pypi in order to test it, you can also use pip to install a package locally:
pip install /path/to/sources # path where the setup.py is