We Keep Coding

How to debug a python package?

I'm creating my first package and I can not figure out how to create a simple test.py file to debug my code.
Consider the following file structure.
|--MyPackage
|--test.py
|--PackageName
|--__init__.py
|--module.py
Inside test.py I have: from .src.module import SomeClass
And of course this gives me the dreaded "attempted relative import with no known parent package" message.
I know I could just install the package and then import it with from PackageName.module import SomeClass but then I'm using the code installed on my system and not the code that I am actively editing.
There has got to be some kind of standard way for testing and debugging a package right? Despite all the searching I've done, I can't seem to find any kind of solution.
I'm running test.py with python3 test.py
If it's helpful, here is a screenshot of my actual project folder structure:

Relative imports for the trillionth time

I have the following data structure:
project/
folder_a/
file.py
folder_b/
useful_functions.py
I am running my file.py and attempting to import a set of functions I have written within useful_functions.py.
At first I tried the following:
from ..folder_b.useful_functions import function_a
But got the following error:
ValueError: attempted relative import beyond top-level package
I then removed the two dots at the beginning of the import statement which initially worked. I have since revisited this project with no changes and I am faced with a new error message. The following code:
from folder_b.useful_functions import function_a
Gives me the following error message:
ModuleNotFoundError: No module named 'folder_b'
I find it very strange that one time it works and later with no changes the import fails. I really would like to solve this with relative imports as I would like the code to work other machines with different absolute file paths.
Thanks in advance.

Well for the trillionth time running a script directly inside a package is considered an antipattern. I do not know how spyder runs stuff but presumably does
python file.py # from inside folder_a
This should be changed to
python -m folder_a.file # from inside project
This would resolve relative imports among packages inside project folder alright and will not need any sys.path or PYTHONPATH hacks

Python: Import scrypt from subfolder to another subfolder

I am working on some python project (2.7) and I have issue with imports. When I run main.py it start scripts from tests folder etc. and save output to logs and everything works fine.
/root------------
-logs
-staticCfg
-config.py
-tests
-systemTest
-scrypt1.py
-scrypt2.py
-userTest
-uScrypt1.py
main.py
My static variables (email, name etc.) are located in config.py. I need to import config.py in scrypt1.py or scrypt2.py. I tryed adding __init__.py to tests, systemTest and staticCfg folder but I always get an error.
In my scrypt1.py:
import staticCfg as cfg
...
or
from staticCfg import *
...
I get the error:
ImportError: No module named staticCfg

The import mechanism of Python can be a bit tricky.
You can refer to the documentation for more information: Python Import Mechanism
When you use absolute imports (your import does not start with a .) as you do, the import path will start from your main script (the one you launch). In your case, it's scrypt1.py. So starting from this location, python can't find the package staticCfg.
For me, the simplest solution is to create a main script in your root directory and call scrypt1.py from there (imported using from tests.systemTet import scrypt1.py). In this case, the base package will be your root folder and you will have access to the package staticCfg from all your script files as you wanted to do.

you may add root folder to PYTHONPATH.

Importing Module Between Directories

UPDATE (TO SHOW FINAL CODE)
Since this seemed hard to explain, I shared the project. For those that come to this question, you can see the full project here:
https://github.com/jeffnyman/pacumen
Calling out the files that were problematic for me:
graphical_pacman
https://github.com/jeffnyman/pacumen/blob/master/displays/graphical_pacman.py
layout
https://github.com/jeffnyman/pacumen/blob/master/mechanics/layout.py
With the __init__.py and setup.py files in place, I'm now able to run commands like these:
python displays/graphical_pacman.py
python mechanics/layout.py
All imports now resolve correctly when those commands are executed. You can see all the import statements I use in each file and the placement of the various __init__.py files.
ORIGINAL QUESTION
I cannot get what seems to be a simple thing to work: importing modules between directories. This is all in Python 3 so I don't want to have __init__.py files all over the place if I can help it, which many of the answers here suggest is the "right" way.
I have a structure like this:
project
displays
graphical_pacman.py
mechanics
layout.py
The layout.py file has a top level function called get_layout() that I want to call from graphical_pacman.py.
Going to the minimum code necessary, in graphical_pacman.py I have:
import layout
if __name__ == '__main__':
board = layout.get_layout("test_maze.lay")
Shows up fine in IDE, even autocompletes it for me. Running graphical_pacman.py gets me this:
File "displays/graphical_pacman.py", line 3, in <module>
import layout
ModuleNotFoundError: No module named 'layout'
Then I tried this:
from mechanics.layout import get_layout
if __name__ == '__main__':
board = mechanics.layout.get_layout("test_maze.lay")
Can't do that either:
File "displays/graphical_pacman.py", line 3, in <module>
from mechanics.layout import get_layout
ModuleNotFoundError: No module named 'mechanics'
I tried this:
from mechanics import layout
if __name__ == '__main__':
board = layout.get_layout("test_maze.lay")
I tried this:
from layout import get_layout
if __name__ == '__main__':
board = get_layout("test_maze.lay")
Doesn't work. Got this:
File "displays/graphical_pacman.py", line 3, in <module>
from layout import get_layout
ModuleNotFoundError: No module named 'layout'
I tried using relative imports (with the . in front of things) but that also doesn't work. I've also just tried using the * for my import (essentially importing everything). Also doesn't work. When I say "doesn't work" I get some variation of the above errors.
I've tried all of this running the command python graphical_pacman.py from within the displays directory and at the root project directory. The same errors occur each time.
I also tried using a sys.path, such as this sys.path.insert(0, '../mechanics'). I also tried variations on sys.path.append('../') based on other answers I've seen here. Again, all I get are variations on the above errors.
What am I missing?

2nd Update:
I have sent a pull request, check it out.
Here's what I did: I created a package out of the project root directory and installed it using pip. Details follow:
Files Added:
pacumen
__init__.py
setup.py
displays
__init__.py
library
__init__.py
mechanics
__init__.py
Contents of pacumen/__init__.py:
from . import displays
from . import library
from . import mechanics
Contents of pacumen/setup.py:
import setuptools
setuptools.setup(
name='pacumen',
version='0.01dev',
packages=['displays',
'library',
'mechanics',
],
author='jefferyman',
author_email='something#something.com',
description='Pacman. Duh.',
long_description=open('README.md').read(),
)
Contents of pacumen/displays/__init__.py:
from . import graphical_helpers
from . import graphical_pacman
from . import graphical_support
Contents of pacumen/library/__init__.py:
from . import structures
from . import utilities
Contents of pacumen/mechanics/__init__.py:
from . import grid
from . import layout
Changes in files:
pacumen/mechanics/layout.py:
from mechanics.grid import Grid
Make sure your virtual environment is active. (Instructions for that further down).
Finally, navigate to the project root directory and install your project root as a package:
pip install --editable . # or pip install -e . (Note the period at the end)
Now as long as you activate the virtual environment, you should not have any import problems. Do make sure you use the import statements of the style:
from mechanics.grid import Grid
Virtual Environment creation and activation:
For those reading, now is a good time to make a virtual environment if one isn't already made. If made make sure to activate it. If not, navigate to the root of the project dir (pacumen) and run
$ python -m venv venvdir
Then, once it is created, run:
<project-root>$ .venvdir\Scripts\activate # for windows.
OR
<project-root>$ source venvdir/bin/activate # for linux bash
Update: First things to check are that there are no circular imports, which means you are importing something in graphical_pacman from mechanics and something else in mechanics from graphical_pacman. Also make sure your module names don't conflict with built-in python module's names. If it's all good on these fronts,
Have you tried this?
from project.mechanics import layout
If that didn't work place __init__.py files in project, displays, mechanics.
In project/displays/__init__.py add from . import graphical_pacman.
In project/mechanics/__init__.py add from. import layout.
If neither of these worked, create a package out of this, navigate to the package root and install it in you environment with pip install -e .

I have something that works so I have an answer but the full answer is that what I'm doing can't be done in Python, as far as I can tell. Part of the problem here is that as I learned more, I realized my question was not worded well enough in terms of how I wanted things to execute.
Here's how things can work. If in graphical_pacman.py, I do this:
if __name__ == '__main__':
sys.path.append('../')
from mechanics import layout
board = layout.get_layout("test_maze.lay")
This finally works!
BUT ...
I have to run python graphical_pacman.py from within the displays directory rather than at the project root. (Meaning, I can't do this command from the project root: python displays\graphical_pacman.py.)
BUT ...
Execution still runs into an error:
Traceback (most recent call last):
File "graphical_pacman.py", line 18, in <module>
from mechanics import layout
File "../mechanics/layout.py", line 4, in <module>
from grid import Grid
ModuleNotFoundError: No module named 'grid'
And that's coming from this line in layout.py:
from grid import Grid
If I change that line to a relative import:
from .grid import Grid
NOW my python graphical_pacman.py command works when run from the displays directory.
BUT ...
Now I can't just run python layout.py, which also needs to be possible. That's apparently because the relative import is now in there, and so I get this error:
Traceback (most recent call last):
File "layout.py", line 4, in <module>
from .grid import Grid
ModuleNotFoundError: No module named '__main__.grid'; '__main__' is not a package
The problem I'm finding is I need to sometimes run files individually. You'll see mechanics\layout.py has an if __name__ == '__main__': section. So does displays\graphical_support.py. Those are necessary so people can see how things work by running those files directly. But eventually those will be used/imported by a file that's in the root of the directory that is orchestrating everything. And sometimes files (like layout) will depend on others (like grid).
So it seems the answer is that Python can't do what I'm trying to do, which is basically just seek flexibility.
I suppose I could handle all of this by just having all of the files in one directory, which seems to be the best answer here, and seems to be how other projects have done it. That seems like a bad approach but I'm weighing that against fighting with an import mechanism particularly since I've now discovered that the mechanism works in different ways depending on execution.

exe-file created by pyinstaller, not find self-defined modules while running

I create two python files, and the directory/file relations is as follows:
mytest---
|---mycommon.py
|---myMainDir---
|----myMain.py
In mycommon.py:
def myFunc(a):
...
And in myMain.py:
import sys
sys.path.append(os.path.join(os.path.dirname(os.path.abspath('__file__')), '..'))
import mycommon.py
mycommon.myFunc("abc")
Then I created exe using pyinstaller:
pyinstall.py -F mytest\myMainDir\myMain.py
MyMain.exe is created, but when run, is tells that can not find mycommon module.

PyInstaller's official manual describes this issue:
Some Python scripts import modules in ways that PyInstaller cannot detect: for example, by using the __import__() function with variable data, or manipulating the sys.path value at run time. If your script requires files that PyInstaller does not know about, you must help it.
It also suggests what should be done in such a case:
If Analysis recognizes that a module is needed, but cannot find that module, it is often because the script is manipulating sys.path. The easiest thing to do in this case is to use the --paths= option to list all the other places that the script might be searching for imports:
pyi-makespec --paths=/path/to/thisdir --paths=/path/to/otherdir myscript.py
These paths will be added to the current sys.path during analysis.
Therefore, please specify the --paths argument while building the application. The manual states that specifying the -p argument is equivalent:
-p dir_list, --paths=dir_list
Set the search path(s) for imported modules (like using PYTHONPATH). Use this option to help PyInstaller to search in the right places when your code modifies sys.path for imports. Give one or more paths separated by ; (under Windows) or : (all other platforms), or give the option more than once to give multiple paths to search.

Also I had to fight a bit to get pyinstaller correctly import python scripts in a subfolder where the path to the subfolder was set relatively via sys.path.insert.
The answer by Yoel was correct for me but I needed careful setting of paths in Windows. Here is what I did:
My main py is:
D:\_Development\pCompareDBSync\pCompareDBSync\pCompareDBSync.py
My imported py is:
D:\_Development\pCompareDBSync\pCompareDBSync\py\pCompareNVR.py
(I have many of such imported py's in folder .\py\ but here i just use a single one as example)
So my main PY, I have the following include:
sys.path.insert(0, 'py')
try:
from pCompareNVR import fgetNV_sN_dict
from pCompareNVR import findNVRJobInDBSync
from pCompareNVR import getNVRRecords
from pCompareNVR import saveNVRRecords
from pCompareNVR import compareNVRs
except Exception as e:
print('Can not import files:' + str(e))
input("Press Enter to exit!")
sys.exit(0)
pyinstaller --onefile pCompareDBSync.py
-> pCompareDBSync.exe that did NOT include py/pCompareNVR.py
I had to include the absolute pad to the main PY and the imported PY's:
pyinstaller --onefile --paths=D:\_Development\pCompareDBSync\pCompareDBSync\ --paths=D:\_Development\pCompareDBSync\pCompareDBSync\py pCompareDBSync.py
-> pCompareDBSync.exe that did now include py/pCompareNVR.py -> OK
And that solved this issue for me!

I am having the same issue as OP (and as this comes up a lot in google searches I though I would add my experience).
Similar folder layout, save for a common folder containing mycommon.py in the same location. I am running PyInstaller from myMainDir as part of a CI build step.
I had tried the suggested solutions: setting --paths, declaring the hidden imports in the spec file etc. I still could not get it working.
I ended up 'solving' (read hacking) the problem by adding a step in the build script to copy the common folder into myMainDir before running PyInstaller.