We Keep Coding

python issue while importing a module from a file

the below is my main_call.py file
from flask import Flask, jsonify, request
from test_invoke.invoke import end_invoke
from config import config
app = Flask(__name__)
#app.route("/get/posts", methods=["GET"])
def load_data():
res = "True"
# setting a Host url
host_url = config()["url"]
# getting request parameter and validating it
generate_schedule= end_invoke(host_url)
if generate_schedule == 200:
return jsonify({"status_code": 200, "message": "success"})
elif generate_schedule == 400:
return jsonify(
{"error": "Invalid ", "status_code": 400}
)
if __name__ == "__main__":
app.run(debug=True)
invoke.py
import requests
import json
import urllib
from urllib import request, parse
from config import config
from flask import request
def end_invoke(schedule_url):
headers = {
"Content-Type":"application/json",
}
schedule_data = requests.get(schedule_url, headers=headers)
if not schedule_data.status_code // 100 == 2:
error = schedule_data.json()["error"]
print(error)
return 400
else:
success = schedule_data.json()
return 200
tree structure
test_invoke
├── __init__.py
├── __pycache__
│   ├── config.cpython-38.pyc
│   └── invoke.cpython-38.pyc
├── config.py
├── env.yaml
├── invoke.py
└── main_call.py
However when i run, i get the no module found error
python3 main_call.py
Traceback (most recent call last):
File "main_call.py", line 3, in <module>
from test_invoke.invoke import end_invoke
ModuleNotFoundError: No module named 'test_invoke'

Python looks for packages and modules in its Python path. It searches (in that order):
the current directory (which may not be the path of the current Python module...)
the content of the PYTHONPATH environment variable
various (implementation and system dependant) system paths
As test_invoke is indeed a package, nothing is a priori bad in using it at the root for its modules provided it is accessible from the Python path.
But IMHO, it is always a bad idea to directly start a python module that resides inside a package. Better to make the package accessible and then use relative imports inside the package:
rename main_call.py to __main__.py
replace the offending import line with from .invoke import end_invoke
start the package as python -m test_invoke either for the directory containing test_invoke or after adding that directory to the PYTHONPATH environment variable
That way, the import will work even if you start your program from a different current directory.

You are trying to import file available in the current directory.
So, please replace line
from test_invoke.invoke import end_invoke with from invoke import end_invoke

Is there a way to fix the pythonpath when using aws sam?

I'm trying to import a custom module that I created, but it breaks my API just to import it.
data directory:
-src
--order
----__init__.py
----app.py
----validator.py
----requirments.txt
--__init__.py
on my app.py I have this code:
import json
from .validator import validate
def handler(event, context):
msg = ''
if event['httpMethod'] == 'GET':
msg = "GET"
elif event['httpMethod'] == 'POST':
pass #msg = validate(json.loads(event['body']))
return {
"statusCode": 200,
"body": json.dumps({
"message": msg,
}),
}
I get this error:
Unable to import module 'app': attempted relative import with no known parent package
However, if I remove line 2 (from .validator import validate) from my code, it works fine, so the problem is with that import, and honestly, I can't figure what is going on. I have tried to import using:
from src.order.validator import validate
but it doesn't work either.

was able to solve my issue by generating a build through the command: sam build, and zipping my file, and putting it on the root folder inside aws-sam, it's not a great solution because I have to rebuild at every small change, but at least it's a workaround for now

It seems app.py has not been loaded as part of the package hierarchy (i.e. src and order packages have not been loaded). You should be able to run
from src.order import app
from the parent directory of src and your code will work. If you run python app.py from the terminal — which I assume is what you did — app.py will be run as a standalone script — not as part of a package.
However, I do not believe you need the .validator in your case since both modules are in the same directory. You should be able to do
from validator import validate

Import path in pytest

I have Python Flask project with following structure of my project (I removed uneccesary things):
server
src
service
__init__.py
User.py
tests
pytest.ini
service
test_user.py
Where pytest.ini contains:
[pytest]
python_paths = ../src
And test_user.py:
from service.User import UserService
def test_empty_db():
service = UserService()
users = service.get_all() # Get all users from DB.
assert len(users) = 0 # There should be users.
Now, I would like to run this test. When I run pytest or pytest server from root of project, everything is ok. However, when I want to run the specified file pytest server/tests/service/test_user.py error appears:
from services.User import UserService
E ModuleNotFoundError: No module named 'service'
Is there any way to fix it?

You can try to make your imports more absolute inside your test_user.py like: from server.src.service.User import …
Or you can also try adding a __init__.py to your tests directory.

Your error message is showing services.User while you code is showing service.User, could it be a typo?

More elegant solution for python imports across app/tests?

I'm trying to keep my code reasonably organized by putting tests in a separate directory from my app. However, imports work either for the app or for the tests, but not both. Here's a contrived example that illustrates my current problem:
myapp/
app/
main.py
settings.py
moods.py
test/
test_moods.py
Contents of files as follows:
main.py
import settings
from moods import Moods
words = Moods(settings.EXAMPLE)
print(words.excited())
settings.py
EXAMPLE = "Wow$ Python$"
DELIM = "$"
moods.py
import settings
class Moods:
def __init__(self, text):
self.text = text
def excited(self):
return self.text.replace(settings.DELIM, "!!!")
test_moods.py
import sys, os, unittest
sys.path.insert(0, os.path.abspath('..'))
from app.moods import Moods
class TestMood(unittest.TestCase):
def setUp(self):
self.words = Moods("Broken imports$ So sad$")
def test_mood(self):
self.assertEqual(self.words.excited(), "Broken imports!!! So sad!!!")
with self.assertRaises(AttributeError):
self.words.angry()
if __name__ == "__main__":
unittest.main()
In the current state, from myapp/ I can run the following successfully:
>> python3 app/main.py
Wow!!! Python!!!
But when I try to run tests, the import fails in moods.py:
>> python3 -m unittest discover test/
ImportError: Failed to import test module: test_moods
[ stack trace here pointing to first line of moods.py ]
ImportError: No module named 'settings'
If I modify line 1 of moods.py to read from app import settings, the test will pass, but normal execution of the app fails because it sees no module app.
The only solutions I can think of are
Putting the tests in the same directory as the code (which I was trying to avoid)
Adding the sys.path.insert(0, os.path.abspath('..')) to each file in my app to make the imports work the same as the test ones do (which seems messy)
Is there are more elegant way to solve this import problem?

You should have both app and test directory in PYTHONPATH.
One way is to replace this:
sys.path.insert(0, os.path.abspath('..'))
from app.moods import Moods
with this:
sys.path.insert(0, os.path.abspath('../app'))
from moods import Moods
Or you can set the PYTHONPATH environament variable before running the tests.
Why? Because when you run main.py, then app is in the path, not app/.., so you want to have the same when running tests.

Optimal file structure organization of Python module unittests?

Sadly I observed that there are are too many ways to keep your unittest in Python and they are not usually well documented.
I am looking for an "ultimate" structure, one would accomplish most of the below requirements:
be discoverable by test frameworks, including:
pytest
nosetests
tox
the tests should be outside the module files and in another directory than the module itself (maintenance), probably in a tests/ directory at package level.
it should be possible to just execute a test file (the test must be able to know where is the module that is supposed to test)
Please provide a sample test file that does a fake test, specify filename and directory.

Here's the approach I've been using:
Directory structure
# All __init__.py files are empty in this example.
app
package_a
__init__.py
module_a.py
package_b
__init__.py
module_b.py
test
__init__.py
test_app.py
__init__.py
main.py
main.py
# This is the application's front-end.
#
# The import will succeed if Python can find the `app` package, which
# will occur if the parent directory of app/ is in sys.path, either
# because the user is running the script from within that parect directory
# or because the user has included the parent directory in the PYTHONPATH
# environment variable.
from app.package_a.module_a import aaa
print aaa(123, 456)
module_a.py
# We can import a sibling module like this.
from app.package_b.module_b import bbb
def aaa(s, t):
return '{0} {1}'.format(s, bbb(t))
# We can also run module_a.py directly, using Python's -m option, which
# allows you to run a module like a script.
#
# python -m app.package_a.module_a
if __name__ == '__main__':
print aaa(111, 222)
print bbb(333)
module_b.py
def bbb(s):
return s + 1
test_app.py
import unittest
# From the point of view of testing code, our working modules
# are siblings. Imports work accordingly, as seen in module_a.
from app.package_a.module_a import aaa
from app.package_a.module_a import bbb
class TestApp(unittest.TestCase):
def test_aaa(self):
self.assertEqual(aaa(77, 88), '77 89')
def test_bbb(self):
self.assertEqual(bbb(99), 100)
# Simiarly, we can run our test modules directly as scripts using the -m option,
# or using nose.
#
# python -m app.test.test_app
# nosetests app/test/test_app.py
if __name__ == '__main__':
unittest.main()