We Keep Coding

Python, absolute/relative import not working as expected

I apologize for the millionth post about this topic.
I thought I had a good grip of the whole absolute/relative import mechanism - I even replied to a couple of questions about it myself - but I'm having a problem with it and I can't figure out how to solve it.
I'm using Python 3.8.0, this is my directory structure:
project_folder
scripts/
main.py
models/
__init__.py
subfolder00/
subfolder01/
some_script.py --> contains def for some_function
I need to import some_function from some_script.py when running main.py, so I tried:
1) relative import
# in main.py
from ..models.subfolder00.subfolder01.somescript import some_function
but when I run (from the scripts/ folder)
python main.py
this fails with error:
ImportError: attempted relative import with no known parent package
This was expected, because I'm running main.py directly as a script, so its _name_ is set to _main_ and relative imports are bound to fail.
However, I was expecting it to work when running (always from within the scripts folder):
python -m main
but I'm getting always the same error.
2) absolute import
I tried changing the import in main.py to:
# in main.py
from models.subfolder00.subfolder01.somescript import some_function
and running, this time from the main project folder:
python scripts/main.py
so that - I was assuming - the starting point for the absolute import would be the project folder itself, from which it could get to models/....
But now I'm getting the error:
ModuleNotFoundError: No module named 'models'
Why didn't it work when using the -m option in the case of relative import, and it's not working when using absolute ones either? Which is the correct way to do this?

I think quite likely you missed python's official doc ( that even come offline )
https://docs.python.org/3/tutorial/modules.html
you'll need a dummy __init__.py within your module, at same level of some_script.py

I think your "absolute" import may not have been absolute in the truest sense.
Prior to running the python scripts/main.py command, you would have needed to setup PYTHONPATH environment variable to include the path to project_folder.
Alternatively I do something like this in main.py:
import sys
import os
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)),'..','models','subfolder00','subfolder01'))
from somescript import some_function
Maybe it is a little pedantic, but it makes sense to me.

python import packages that know nothing about my system configuration

I have a project I am working on, let's call it Project, which lives in the directory Project somewhere wholly unknown to me (really it lives both on my local system and on a couple Docker build systems). In that project, I have some source files, source/module1.py and source/module2.py. I also have some example files, some test files, and an init.py So my directory looks something like this:
Project
__init__.py
/source
module1.py
module2.py
/test
testRunner.py
/examples
awesomeExample.py
However, module1 needs some stuff from module2. My naive self thought this could be done by putting an import statement in module1:
import module2
# Do some other interesting stuff
And this works, but only when I am running / importing module 1 from the source directory. If I am, for example, running some unit tests in another directory test/testRunner.py, either from the test directory or in the main Project directory, the import will fail. Same with trying to use it when running an example in the examples directory.
So here is my problem: in general, I don't know where the calling script lives. It might be in the examples directory, it might be in the test directory, or it might be in the main Project directory (for example when trying to import stuff with an init.py). How do I ensure that module1 can always import module2 in each of these scenarios?
I am not looking for a solution like "add all those directories to your python path". Initially I just added Project to my python path on my local machine, and then did all my imports relative to that (import Project.source.module2), but this (predictably) caused my builds to fail on the Docker instances. I don't just want this to work on my local machine, but also on the Docker instances I'm using to build and test this software, and on any user's machine that subsequently installs it (i.e. by doing a pip install Project. What is the most robust way to make sure this dependency is satisfied? How can I make sure module1 can import module2 regardless of where module1 itself is imported from? Any python 3.x.x solution is welcome.

I figured out a way to do it (credit here) - it's a little inelegant, but extremely robust. Works on my local machine independent of whether using an import statement or running a script directly, as well as my build servers which use github actions and Travis CI.
Basically, I added a file in the source directory, called context.py with the following contents:
import os
import sys
fileLocation = os.path.dirname(os.path.abspath(__file__))
sourceLocation = os.path.abspath(os.path.join(fileLocation, '..', 'source/'))
sys.path.insert(0, sourceLocation)
This finds out the current file being executed from python, and then uses that to add to the python path. And then in my module1.py file, at the top I have:
import context
import module2
Now, whenever module1 is imported, it successfully imports module2. More elegant answers or comments on why this works and in which cases it might fail are appreciated.

(Python Unittest) Module being tested cannot import its dependencies: ModuleNotFoundError

I am working on developing unittests for a project that has been already completed, however I am having a hard time running my unittests without modifying the original code. The module I am trying to test has other dependencies in the same folder that will not import when the unittests are run. Here is what my directory looks like:
root
|--main_folder
|--module1.py
|--module2.py
|--tests
|--test_module1.py
The original code in module1.py successfully imports module2.py on its own like this: from module2 import Practices where Practices is a function from module2.
The issue I am running into is that in order to run test_module1.py (which I am doing by calling python3 -m unittest from the root directory), I have to modify module1.py itself such that it says: from main_folder.module2 import Practices.
If I run the test file without modifying module1.py, I get the error ModuleNotFoundError: No module named 'module2'.
Ideally I cannot modify the code in this way, and I am trying to find a way to make my tests work without touching the application itself. How should I go about this? module1.py runs normally when I run the application without modifying the file, however modifying it so that the tests work breaks the main application. What can I do to make my tests independent of the code for the main app?
(For some more background, the test_module1.py file works by calling from main_folder.module1 import fun1 where fun1 is the function I am trying to test)

Try running your tests using one of the following commands (replacting the actual paths):
if your tests import the modules "from main_folder import ..."
env PYTHONPATH=/root python3 -m unittest
or if your tests import directly "import module1":
env PYTHONPATH=/root/main_folder python3 -m unittest
As a side note, you might need to have existing
main_folder/__init__.py
file, to get the main_folder recognized as package, depending of the python version you're using. If you currently don't have such file, try creating it (empty, no need to put code inside it) and check if the issue persists.

Python site-packages import

I have a site-packages folder in an environment. This is containing the exemplary site-package 'test'.
Now in this "\lib\site-package\test" there are two files. init.py and testwrapper.py.
init.py:
import testwrapper
Testclass = testwrapper.Testclass()
When I run this init.py in an IDE it perfectly initiates this "Testclass" and I can use it with as an example Testclass.exampleFunction().
When I now run the python console and type import test, it finds the site-package and imports it. But I can not use Testclass.exampleFunction() as I can in the IDE, because Testclass is unknown. I can use it if I type in the code from the init.py manually, thus in the console:
import test
import testwrapper
Testclass = testwrapper.Testclass()
Testclass.exampleFunction()
This works just fine in the console. But as far as I understood, if I import the site-package by using import test the init.py shall automatically be loaded and started?
Thanks for the help in understanding guys.

Python - organising code and test suite

I am very new to python, coming from a php background and cant figure out the best way to organise my code.
Currently I am working through the project euler exercises to learn python. I would like to have a directory for my solution to the problem and a directory that mirrors this for tests.
So ideally:
Problem
App
main.py
Tests
maintTest.py
Using php this is very easy as i can just require_once the correct file, or amend the include_path.
How can this be achieved in python? Obviously this is a very simplistic example - therefore some advice on how this is approached on a larger scale would also be extremely grateful.

This depends on which test runner you want to use.
pytest
I recently learned to like pytest.
It has a section about how to organize the code.
If you can not import your main into the code then you can use the tricks below.
unittest
When I use unittest I do it like this:
with import main
Problem
App
main.py
Tests
test_main.py
test_main.py
import sys
import os
import unittest
sys.path.append(os.path.join(os.path.dirname(__file__), 'App'))
import main
# do the tests
if __name__ == '__main__':
unittest.run()
or with import App.main
Problem
App
__init__.py
main.py
Tests
test.py
test_main.py
test.py
import sys
import os
import unittest
sys.path.append(os.path.dirname(__file__))
test_main.py
from test import *
import App.main
# do the tests
if __name__ == '__main__':
unittest.run()

I have always loved nosetests, so here is my solution:
Problem
App
__init__.py
main.py
Tests
__init__.py
tests.py
Then, open the command prompt, CD to /path/to/Problem and type:
nosetests Tests
it will automatically recognize and run the tests. However, read this:
Any python source file, directory or package that matches the
testMatch regular expression (by default: (?:^|[b_.-])[Tt]est) will be
collected as a test (or source for collection of tests).
[...]
Within a test directory or package, any python source file matching testMatch will be examined for test cases. Within a test module, functions and classes whose names match testMatch and TestCase subclasses with any name will be loaded and executed as tests.
This basically means that your tests (both your files and function/methods/classes) have to begin with the "test" or "Test" word.
More on Nosetests usage here: Basic Usage.