We Keep Coding

Application says module not found after moving to directory

I'm rather new to python and have this odd issue, which I can't seem to find an answer for.
When both app.py and mod_db were in the root directory, it works but when I shifted them to a sub directory
My directory structure:
demo_api
|
|-- demo-api
|
|-- __init__.py
|-- app.py
|-- mod_db.py
My main module
import json
from flask import Flask, request, Response
from .db_mod import insert_and_score
app = Flask(__name__)
#app.route('/')
def hello_world():
return 'Hello World!'
#app.route('/emotional_scoring', methods=['POST'])
def get_scoring():
json_obj = request.json
ret_json = insert_and_score(json_obj)
resp = Response(json.dumps(ret_json), mimetype='application/json', status=200)
return resp
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=False)
The error message is on this line
from .mod_db import insert_and_score
I've tried change the sub-directory name. I've tried doing a full path, i.e. from demo_app.mod_db import insert_and_score and the error is ModuleNotFoundError: No module named 'demo_app'
The issue is that it works find in my IDE (PyCharm) but when I do it on command line, these are the errors I encounter.

As stated by mfrackowiak, I just had to change it to
from db_mod import insert_and_score
and it worked. But PyCharm doesn't like this. So I think it might be a PyCharm issue.

Fix by removing relative import
It seems you want both the relative imports and Pycharm to be happy. Like mfrackowiak said, you want to use absolute importing.
from db_mod import insert_and_score
Tell PyCharm what the new "Sources Root" is
Now to make Pycharm happy, you will want to tell it that the demo_api subdirectory is a "Sources Root". You can do this by right clicking the directory and go to Mark Directory as > Sources Root. You can also find it in Preferences > Project Structure. You can do this for each subdirectory, as needed.
Why this is good to do
This is useful when you have a python project as a subdirectory of a repo with many configs, scripts, and other files and directories. You often don't want the python app to be mixed in with those, so you move it to an app/ folder. This confuses Pycharm, as by default, it uses the Content Root as the Sources Root as well. You can fix this by simply telling Pycharm explicitly what the Sources Root is.
Example:
my-awesome-project/ <---- Content Root
|--.venv/
| |--<venv stuff>
|--scripts/
| |--build.sh
| |--run.sh
|--docker/
| |--dev.Dockerfile
| |--prod.Dockerfile
|--app/ <---------------- Sources Root
| |--sub-mod1/
| | |--foo.py
| | |--bar.py
| |--sub-mod2/
| |--baz.py
|--.gitignore
|--.python-version
|--requirements.txt
|--dev.env
|--docker-compose.yml

Python logging: Set level in `main.py` and inherit to child loggers

I have a Python project that looks like this:
my_project
|- my_dir
| |- part_1.py
| |- part_2.py
| |- ...
|- main.py
Every file (main.py and part_1.py, part_2.py, ...) should be able to write to the same log file, with the following two requirements:
The file name should appear in the log message: a line in the log file should look like
ERROR - my_dir.part_1 - Something went wrong in part_1.py
The log level for all loggers should be set programatically from main.py:
# main.py
from my_dir import part_1, part_2, ...
...
logger.setLevel('DEBUG')
result_1 = part_1.my_function() # Logger in part_1.py should be set to DEBUG level
logger.setLevel('WARNING')
result_2 = part_2.my_other_function() # Logger in part_2.py should be set to WARNING level
I accomplish the first requirement by beginning each file in my_dir with
import logging
logger = logging.getLogger(__name__)
Setting the same level for all loggers works if I use a config file and do logging.config.fileConfig('config.ini'). But changes to the log level made in main.py are not propagated to the other loggers. What is the best way to handle this situation?

Run all tests from subdirectories in Python

I am at my wits end with trying to get all my unittest to run in Python. I have searched about 30 different posts and the unit test documentation but still cannot figure it out.
First I have two test classes that I can run each individually and all the tests pass:
File: unittest.subfolder1.TestObject1.py
class TestObject1(unittest.TestCase):
def test_case1(self):
...some code...
...some assertions...
if __name__ == '__main__':
unittest.main()
File: unittest.subfolder2.TestObject2.py
class TestObject2(unittest.TestCase):
def test_case1(self):
...some code...
...some assertions...
if __name__ == '__main__':
unittest.main()
Starting in the top level directory above 'unittest' I am trying to us unittest.discover to find and run all my tests:
import unittest
loader = unittest.TestLoader()
suite = loader.discover('unittest')
unittest.TextTestRunner().run(suite)
When I do this I get the error `ModuleNotFoundError: No module named 'subfolder1.TestObject1'
What am I doing wrong?

A good approach is to run all the tests in a subdirectory from the command line. In order to find the following files "TestObject1.py, TestObject2.py, ..." in subdirectories, you can run the following command in the command line:
python -m unittest discover -p 'Test*.py'
Additionally, the __init__.py is required within the import and module directories: Python unittest discovery with subfolders
The import unittest is required in the files unittest.subfolder1.TestObject1.py and unittest.subfolder2.TestObject2.py
It is also possible to define explicitly the directory where the discovery starts with the -s parameter:
python -m unittest discover [options]
-s directory Directory to start discovery ('.' default)
-p pattern Pattern to match test files ('test*.py' default)
In case you are using unittest2, it comes with a script unit2. The command line usage is:
unit2 discover unit2 -v test_module

Do not name your directory unittest, it may conflict with the standard library.
You also need to create a file named __init__.py in all of your directories (subfolder1, etc.), so they become packages and their content can be imported.

So I had to do my own workaround but at least I can get them all to run with the above file structure. It requires that I reinstantiate the TestLoader and the TestSuite each time I give it a new file path, so first I need to collect all relevant file paths in the unittest directory.
import os
import unittest
import traceback
class UnitTestLauncher(object):
def runTests(self):
#logging.INFO("Running unit tests...")
lsPaths = []
#Find all relevant subdirectories that contain unit tests
#Exclude 'unittest' directory, but include subdirectories, with code `path != 'unittest'`
for path,subdirs,files in os.walk('unittest'):
if "pycache" not in path and path != 'unittest':
lsPaths.append(path)
#loop through subdirectories and run individually
for path in lsPaths:
loader = unittest.TestLoader()
suite = unittest.TestSuite()
suite = loader.discover(path)
unittest.TextTestRunner().run(suite)
This solution is not perfect and each different directory comes out as a line of output so you have to look through each line manually for failed tests.

Old question but, oh, so current. I am new to Python, coming from strong typed languages and while the language itself is ok(ish), the conventions, tools and workarounds to make everything work in the ecosystem can drive you nuts. I struggled myself with running unit tests from separate subdirectories and this is the way I solved it.
First, the code you test, package it into a package. Organize your directories like this:
Work
|
+---PkToTest
| |
| +--- __init__.py
| +--- a.py
| +--- <other modules>.py
|
+---Tests (for PKToTest)
|
+--- test_a.py
PkToTest becomes a package due to the init.py file. In test_a.py make sure your sys.path will contain the path to PkToTest (absolute path not relative). I did that by:
import sys
sys.path.insert(0, "<absolute path to parent of PkTotest directory>")
import unittest
from PkToTest import a
class aTestSuite(unittest.TestCase):
def test1(self):
self.assertEqual(a.fnToTest(), ...)

Testing All Subdirectories
Given a structure of:
my_package
|
|
controller
|-- validator.py
|
validator
|-- controller.py
|
test
|-- controller
|
|-- __init__.py (MAKE SURE THIS EXISTS OR unittest MODULE WOULD NOT KNOW)
|-- test_controller.py
|
|-- validator
|
|-- __init__.py (MAKE SURE THIS EXISTS OR unittest MODULE WOULD NOT KNOW)
|-- test_validator.py
|
then just run
python -m unittest discover -s my_package/test
What this does is to test and -s means to start with the my_package/test as the starting directory

In my project all folders are folders (not modules) and they have the structure:
Folder > Subfolder > Subfolder > Tests > test_xxxx.py
Folder > Subfolder > Subfolder > xxxx.py
So i modified the answer from here, and also took a part from How do I run all Python unit tests in a directory? and came up with this:
import os, unittest
testFolderPaths = []
for path, subdirs, files in os.walk(os.getcwd()):
for file in files:
if file.startswith("test_") and file.endswith(".py"):
testFolderPaths.append(path)
for path in testFolderPaths:
print(f"Running tests from {path}...")
loader = unittest.TestLoader()
suite = loader.discover(path)
runner = unittest.TextTestRunner()
result = runner.run(suite)
print(f"RUN {result.testsRun} Tests. PASSED? {result.wasSuccessful()}")
If any tests fail it will throw and error showing which one exactly failed.

How to load config file in python from an external module

I have a config module (myConfig.py) present in a library for which i created a standard distribution package using setuptools.
--configPackage
|
| ---- myConfig.py
| ---- __init__.py
myConfig.py is a key value pair like this:
MY_NAME = 'myname'
MY_AGE = '99'
Now I have another python project where I import this config module like this
import configPackage.myConfig as customConfig
If this config file was native to my python project and had not come from an external project then I would have done something like this:
app = Flask(__name__)
app.config.from_object('app.configPackage.config')
where config is actually config.py file under configPackage.
and any key value pair in config.py could then be accessed as
myName = app.config['MY_NAME']
My problem is that I am not able to load the external config file in the above mentioned way for a native config file. What I have tried is this which doesn't work:
import configPackage.myConfig as customConfig
app = Flask(__name__)
app.config.from_object(customConfig)
myName = app.config['MY_NAME']
I get the following error
model_name = app.config['model_name']
KeyError: 'model_name'
Which means that it is not able to load the config file from external module correctly. Can anyone tell me the right way to accomplish this ?

Try this,
in your __init__.py of configPackage import myConfig as shown below
import configPackage.myConfig as myConfig
And in your other app try
app.config.from_object('configPackage.myConfig')

Import Error in flask application

I have an application. Below is o/p of tree command -
app
|-- main
| |-- lib
| | |-- constants.py
| | |-- helper.py
| | `-- __init__.py
| `-- src
| |-- __init__.py
| `-- web.py
web.py
from flask import Flask, request
app = Flask(__name__)
from lib.helper import endpoints
.....
Some code
.....
if __name__ == '__main__':
app.run('0.0.0.0', 5433, debug=True)
I am getting this error
ImportError: No module named lib.helper.
where am I doing wrong?

from flask import Flask, request
app = Flask(__name__)
import sys
from os.path import abspath, dirname
sys.path.insert(0, dirname(dirname(abspath(__file__))))
from lib.helper import endpoints
.....
Some code
.....
if __name__ == '__main__':
app.run('0.0.0.0', 5433, debug=True)

Lib module is outside of src folder, need to go up one folder up and use that model.
from ..lib.helper

Or else a fully qualified namespace as
from app.main.lib