We Keep Coding

How to import base classes correctly

I am trying to write a library (sort of) for my project. I would like to keep it as modularized as possible. I have a main.py in the root directory and I have two packages required for running the main. I am calling them package1 and package2. The packages in each of them will inherit from their own base packages. The overall tree is as following:
.
├── main.py
├── package1
│   ├── base_package1.py
│   ├── __init__.py
│   └── main_package1.py
└── package2
├── base_package2.py
├── __init__.py
└── main_package2.py
In the file for each package, I am also writing the code to test them. But to test package 2, I also need package 1. My question is about how to import package 1 to package 2. main.py is currently empty. I will share the contents of the remaining files here.
# package1/__init__.py
from .main_package1 import *
# package1/base_package1.py
class BasePackage1:
def __init__(self):
print("init base package 1")
# package1/main_package1.py
from base_package1 import BasePackage1
class MainPackage1(BasePackage1):
def __init__(self):
super().__init__()
print("init main package 1")
if __name__ == "__main__":
# Test MainPackage1
pkg = MainPackage1()
# package2/base_package2.py
class BasePackage2:
def __init__(self):
print("init base package 2")
# package2/main_package2.py
from base_package2 import BasePackage2
class MainPackage2(BasePackage2):
def __init__(self):
super().__init__()
print("init main package 2")
if __name__ == "__main__":
# Option 1
import sys
sys.path.append("../")
from package1 import MainPackage1
# Error:
# ModuleNotFoundError: No module named 'base_package1'
# Option 2
from ..package1 import MainPackage1
# Error:
# ImportError: attempted relative import with no known parent package
What is the correct way to do this? Is there a standard way to do this?
Update: I have found a solution that works for now and posted it as an answer below. If you have a better solution, please post it.

In package2/base_package2.py if you want to reference package1.base_package1 you can import that directly as your current reference point is where main.py is. Absolute imports (those which don't start with a dot) start from your working directory, so you can import package1 from anywhere in package2 and it will be the same.
The reason why you get
ImportError: attempted relative import with no known parent package
is because you are running package2/base_package.py as a main file, which would run as its own script. However, you have provided three files with paths given as package2/base_package.py. I'm assuming that the ones with __main__ are actually main_package1 and main_package2 respectively.

After trying a few other things, I found a solution that seems to be work. I will post it here but will not select it as the accepted answer in case someone comes up with a better and more elegant solution.
In package2/main_package2.py, I did the following import:
import sys
sys.path.append("../package1")
from main_package1 import MainPackage1

Understanding python import system and project structuring

Suppose I have a project organized as follows:
ProjectRoot/
__init__.py
test.py
A/
__init__.py
a_test.py
B/
__init__.py
b_test.py
And suppose that a_test depends on b_test. So the source code is relatively simple:
#
# a_test.py
#
from B.b_test import b_test_class
class a_test_class:
def func(self):
print("a_test_class")
b_instance = b_test_class()
b_instance.func()
if __name__ == "__main__":
a_instance = a_test_class()
a_instance.func()
#
# b_test.py
#
class b_test_class:
def func(self):
print("b_test_class")
#
# test.py
#
from A.a_test import a_test_class
if __name__ == "__main__":
a_instance = a_test_class()
a_instance.func()
As long as I launch test.py script, everything works as intended. Python loads all modules without any troubles and executes them. Now the question comes: how do I launch a_test.py without having test.py? So, basically, what I want to achieve is to cd into projectRoot/A and execute a_test.py. This results in getting ImportError: No module named 'B'
Currently I've been able to create a project with following structure:
ProjectRoot/
customLibModuleA/
...
customLibModuleB/
...
mainApp.py
And what I want to be able to create is following:
ProjectRoot/
customLibModuleA/ #custom protocol implementation
...
customLibModuleB/ #custom logging functions
...
application1/ #server
...
application2/ #client
...
How do I expected to manage complex projects? Any good references to project structuring manuals and styleguides are welcome.

Here's my temporal solution since no one provided pythonic approach.
Folder structure looks like that:
ProjectRoot/
__init__.py
customLibModuleA/ #custom protocol implementation
__init__.py
...
customLibModuleB/ #custom logging functions
__init__.py
...
application1/ #server
__init__.py
__path_setup__.py
server.py
...
application2/ #client
__init__.py
__path_setup__.py
client.py
...
__path_setup__.py content is:
import sys
import os
os.sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), ".."))
Application scripts have some setup code preceding imports (server.py):
#
#settings up the environment
#
if __name__ == "__main__":
exec(open("./__path_setup__.py").read())
#
# system and libraries imports
#
import customLibModuleA.whatever
...
Hight quality pythonic solution to this problem is still welcome.

Python unittest with TestLoader results in ModuleNotFoundError

i try to run a unittest with unittest.TestLoader and unittest.TextTestRunner but get an ModuleNotFoundError everytime i try to run the 'main' test file (here: test_all.py). I have the following file structure:
src.
test_all.py
dir1.
__init__.py
module1.py
submodule1.py
test_module1.py
dir2.
__init__.py
module2.py
submodule2.py
test_module2.py
dir3.
...
The test_all.py file looks like this:
# test_all.py
import os
import unittest
loader = unittest.TestLoader()
suite = loader.discover(os.getcwd())
runner = unittest.TextTestRunner()
runner.run(suite)
And finally the structure of the single testcases look like this:
# test_module1.py
import unittest
from module1 import Module1
class Module1TestCase(unittest.TestCase):
def setUp(self):
# do something
def test_something(self):
# test test
def tearDown(self):
# do something
if __name__ == '__main__':
unittest.main()
So, running the test_all.py always results in an ModuleNotFoundError referencing to the from module1 import Module1 inside the TestCase test_module1.py (and the same in the following TestCases). As far as i can tell there are no circular dependencies. Maybe adding the current Path to the PythonPath would work, but it really makes no sense to me: on the one hand i run the test_all.pyas main in the current directory and on the other the unittest.TestLoader.discover() already takes the current path.
PS: I know that putting all the TestCases in one folder is way better. But first i want to figure out why this is not working. Thanks!

Imports break moving from python 3.4 to 3.5

I have a module that works fine in python 3.5+ but not in 3.4. The only possible change that may be effecting it is how circular imports are handled in 3.5+. I cannot find any circular imports though so there may be something else going on.
module/
module/
__init__.py
file_a.py
from module import settings
from module.file_b import SomeBClass
def stuff():
settings.init()
stuff = SomeBClass()
def run():
stuff()
def main():
run()
file_b.py
from module.settings import config, properties
class SomeBClass():
....
file_c.py
class SomeClass():
connect to db...
settings.py
from module.file_c import SomeClass
def init():
global config
global properties
config = SomeClass()
properties = config.get_it()
when running I get the following error:
File "/home/somewhere/module/module/file_b.py", line 11, in <module>
from module.settings import config, properties
ImportError: cannot import name 'config'
I have tried running the module with python -mv to see if something gets imported more than once but I cannot see anything alarming.
Anyone have experience dealing with the differences between 3.4 and 3.5+? Does trying to access attributes from globals in the settings.init cause issues?

Mock function called on import

I have a module I need to test that calls a function on import but I cannot call this function for various reasons. So I am mocking this function but even mocking it calls import.
For example I am testing mod1.py that looks like this:
import os
def bar():
return 'foo'
def dont_call():
os.listdir("C:\\tmp")
dont_call()
And my test looks something like this:
import mock
#mock.patch("mod1.dont_call")
def test_mod1(mock_dont_call):
import mod1
assert mod1.bar()=='foo'
if __name__=="__main__":
test_mod1()
The problem is os.listdir is called.
I cannot change mod1 so what can I do?
I am using python2.7.
To put this in context I am testing a module that opens a database connection on import which I do not agree with but I can see the reasoning behind it. Unfortunately I cannot access this database on my QA machine.

If you want code to 'not' be executed on import put them inside the following condition:
In mod1.py, do the following:
if __name__=="__main__":
dont_call()
This is because, by default when you import a python module, all the code in it gets executed. By adding the above condition, you are explicitly stating that dont_call() is to be called only when the file it run as a script and not when it is imported in other modules.

The workaround I found was to mock what dont_call was calling giving me something like this:
import mock
#mock.patch("os.listdir")
def test_mod1(mock_dont_call):
import mod1
assert mod1.bar()=='foo'
if __name__=="__main__":
test_mod1()

Check your dir
$tree.
test_shot/
├── mod1.py
├── __pycache__
│ └── mod1.cpython-310.pyc
└── test.py
Below code works fine for me.
mod1.py
import os
def bar():
return 'foo'
def dont_call():
os.listdir(".")
def call_this():
print('called this')
call_this()
dont_call()
test.py
import mock
#mock.patch("mod1.dont_call")
def test_mod1(mock_dont_call):
import mod1
assert mod1.bar()=='foo'
if __name__=="__main__":
test_mod1()
Here is output:
$cd test_shot
$python3 test.py
called this