We Keep Coding

Python packaging with setuptools does not include my source code when installing

I'm trying to make a python program into a package:
This is my setup.py
from setuptools import setup, find_packages
setup(
name='scroll',
version='2020.6.14',
# package_dir={'': 'scroll'},
packages=find_packages(),
install_requires=[
'Click',
],
entry_points='''
[console_scripts]
scroll=scroll:scroll
''',
# .... all other stuff
)
This is the module structure,
SCROLL
- scroll/
|
+--scroll.py
- setup.py
- MANIFEST.in
- venv/
When I run python setup.py sdist, a tar.gz file is created and when extracted, it contains the source code at projects\SCROLL\dist\scroll-2020.6.14\dist_scroll-2020.6\scroll-2020.6.14\scroll
But when I install the archive using pip install ./dist/scroll-2020.6.14.tar.gz,
Running scroll produces an ModuleNotFoundError: No module named 'scroll'
This is because during installation, the source code is not copied to SCROLL\venv\lib\python3.8\site-packages\.
Copying the scroll folder manually to site-packages solves this error
I have tried using MANIFEST.in file with contents below but the code is still not copied to site-packages
include scroll
recursive-include scroll *.py

I solved this by packaging the module using flit. I just found out it can create scripts too after asking the question.
For the how-to:
Refer to https://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-script-creation

ModuleNotFoundError when pip installing package, but not when using python setup.py install

I've restructured a project to the src directory structure. It looks like this:
root_dir/
src/
module1/
__init__.py
script1.py
script2.py
module2/
__init__.py
other_script1.py
other_script2.py
conftest.py
setup.py
tests/
conftest.py
some_tests/
conftest.py
test_some_parts.py
some_other_tests/
conftest.py
test_these_other_parts.py
My setup.py looks like this:
setup(
name='Project',
version=0.0,
author='Me',
install_requires=['pyodbc'],
tests_require=['pytest'],
setup_requires=['pytest-runner'],
test_suite='root_dir.Tests',
entry_points={
'console_scripts': ['load_data = module1.script1:main']
},
package_data={'Config': ['*.json']},
packages=find_packages('src'),
package_dir={'': 'src'})
I am running Anaconda3 on Windows 10. When I run python setup.py install, I am able to run the load_data script without any issue. However, from what I've been reading, it is preferable to use pip install . vice python setup.py install. When I pip install the package and attempt to run load_data, I get ModuleNotFoundError: No module named 'module1.script1'. I've attempted adding 'src' to the front of this, but this doesn't work either. I don't understand what the differences are or how to troubleshoot this.

When building the source distribution for the package not all files are included. Try creating a MANIFEST.in file with
recursive-include src/module1 *

Run a install-able Flask project error: could not import ""

I read these guides to create my project.
Make the Project Installable
Deploying with Setuptools
First, creating a tar zip file.I create many files in my project folder.
setup.py
from setuptools import setup, find_packages
setup(
name='WebMonitor',
version='1.0',
long_description=__doc__,
packages=find_packages(),
include_package_data=True,
zip_safe=False,
install_requires=['Flask>=0.12']
)
MANIFEST.in
include schema.sql
include __init__.py
include auth.py
include blog.py
include db.py
graft static
graft templates
global-exclude *.pyc
setup.cfg
[egg_info]
tag_build = .dev
tag_date = 1
[aliases]
release = egg_info -Db ''
Then I run this command python setup.py release sdist to build a release package.
Second, install and run this app.
Create a virtual environment with the command virtualenv env
Activate the env with env\Scripts\activate
Install the release package by pip install WebMonitor-1.0.tar.gz
set FLASK_APP=WebMonitor
Then run my app flask run -h 127.0.0.1 -p 5001. I get an error output:
Serving Flask app "WebMonitor"
Environment: production
WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
Debug mode: off
Usage: flask run [OPTIONS]
Error: Could not import "WebMonitor".
Does anyone know how to solve this problem? Thank you very much.

As it is production env
in cmd,
set FLASK_APP=WebMonitor.py
set FLASK_ENV=development

First ensure that you are at the root of the application file. in case it persist, Deactivate the virtual environment then activate it.

Ubuntu does not find module directory containing __init__.py

I have a python project with the following structure:
/project
/bin
executable
/app
__init__.py
a.py
b.py
c.py
From within the /project directory I try to run the executable, which depends on modules in the app package:
./bin/executable
However, when I try this, python fails to find any modules defined in the app package, giving the ImportError: No module named XYZ error.
From what I understood, the presence of __init__.py in the app directory should mark it as a module?
What am I doing wrong?

If you add the following to your executable (before you try importing your app module):
import sys; print sys.path
...
import app
You'll see that the current working directory is not included in the path. I suggest you create a proper Python package by adding a setup.py file. You can then install your package and your executable should work just fine.
Here is a simple setup.py file:
from setuptools import find_packages, setup
config = {
'name': 'app',
'version': '0.1.0',
'description': 'some app',
'packages': find_packages(),
}
setup(**config)
I prefer to install into a virtualenv:
virtualenv venv
source venv/bin/activate
python setup.py install
./bin/executable
Now, if you deactivate that virtualenv and try running your executable again, you will get your original ImportError.
Take a few minutes and read through the Python Packaging User Guide.
Also, you might want to use python setup.py develop instead of python setup.py install.

setup.py hangs during cleanup in chroot

I'm having an issue with running setup.py/pip in a chroot environment.
Here's the scoop:
Normal directory location:
/local/my_dir/project/src/qa/libs
Chroot-ed location
/src/qa/libs
Here's my setup.py file:
#!/usr/bin/env
from __future__ import (unicode_literals, print_function, division,
absolute_import)
from setuptools import find_packages, setup
test = [
'mock',
'pytest',
'pytest-cov',
]
setup(
name='libs',
version=0.1,
description='Some desc',
long_description=open('README').read(),
author='insert_author_here',
author_email='insert_email_here',
packages=find_packages(),
package_dir={},
include_package_data=True,
tests_require=test,
install_requires=[],
keywords=['qa', 'framework'],
extras_require={
'test': test,
}
)
When running python setup.py develop in the libs directory everything goes swimmingly during the install until the very end.
Installed /src/qa/libs
Processing dependencies for libs==0.1
Finished processing dependencies for libs==0.1 # <-- It hangs here
This doesn't happen when I'm not currently in chroot (required for the environment) and it seems like setuptools/distribute is getting stuck in a recursive filesystem looking for things to clean up. Any idea how to fix this?
Installing a requirements.txt file with pip doesn't have any problems like this, so I think it might be the structure of the setup.py file.

It turns out the hang occurred during during the bash script that created the virtualenv and installed this package. I figured this out by executing the script with the bash -x my_script command, which showed the actual executing command when the hang occurred.
The setup.py file correctly installs the package and exits successfully.