We Keep Coding

Importing from above path on python for windows

I have the script I want to run in the following structure
scripts/
project/
main.py
libraries/
a.py
In main.py I need to import things from a.py. How can I import things in subfolders that are two or more folders above main.py?

The proper way to handle this would be putting everything that needs to know about each other under a shared package, then the individual sub-packages and sub-modules can be accessed through that package. But this will also require moving the application's entrypoint to either the package, or a module that's a sibling of the package in the directory and can import it. If moving the entrypoint is an issue, or something quick and dirty is required for prototyping, Python also implements a couple other methods for affecting where imports search for modules which can be found near the end of the answer.
For the package approach, let's say you have this structure and want to import something between the two modules:
.
├── bar_dir
│   └── bar.py
└── foo_dir
└── foo.py
Currently, the two packages do not know about each other because Python only adds the entrypoint file's parent (either bar_dir or foo_dir depending on which file you run) to the import search path, so we have to tell them about each other in some way, this is done through the top level package they'll both share.
.
└── top_level
├── __init__.py
├── bar_dir
│   ├── __init__.py
│   └── bar.py
└── foo_dir
├── __init__.py
└── foo.py
This is the package layout we need, but to be able to use the package in imports, the top packagehas to be initialized.
If you only need to run the one file, you can do for example python -m top_level.bar_dir.bar but a hidden entry point like that could be confusing to work with.
To avoid that, you can define the package as a runnable module by implementing a __main__.py file inside of it, next to __init__.py, which is ran when doing python -m top_level. The new __main__.py entrypoint would then contain the actual code that runs the app (e.g. the main function) while the other modules would only have definitions.
The __init__.py files are used to mark the directories as proper packages and are ran when the package is imported to initialize its namespace.
With this done the packages now can see each other and can be accessed through either absolute or relative imports, an absolute import would being with the top_level package and use the whole dotted path to the module/package we need to import, e.g. from top_level.bar_dir import bar can be used to import bar.
Packages also allow relative imports which are a special form of a from-style import that begins with one or more dots, where each dot means the import goes up one package - from the foo module from . import module would attempt to import module from the foo_dir package, from .. import module would search for it in the top_level package etc.
One thing to note is that importing a package doesn't initialize the modules under it unless it's an explicit import of that module, for example only importing top_level won't make foo_dir and bar_dir available in its namespace unless they're imported directly through import top_level.foo_dir/top_level.bar_dir or the package's __init__.py added them to the package's namespace through its own import.
If this doesn't work in your structure, an another way is to let Python know where to search for your modules by adding to its module search path, this can be done either at runtime by inserting path strings into the sys.path list, or through the PYTHONPATH environment variable.
Continuing with the above example with a scenario and importing bar from foo, an entry for the bar_dir directory (or the directory above it) can be added to the sys.path list or the aforementioned environment variable. After that import bar (or from bar_dir import bar if the parent was added) can be used to import the module, just as if they were next to each other. The inserted path can also be relative, but that is prone to breakage with a changing cwd.

Import two modules with same name at top of PYTHONPATH elements

I'll shorten the notation. I have
PYTHONPATH=/path1/dir1:/path2/dir2
Structures:
/path1/dir1/
README
muggle.py
...
utils/
/path2/dir2/
__init__.py
utils/
__init__.py
pkg2/
__init__.py
mod2.py
dir1 has a module utils, but is not, itself a package: no __init__.py
dir2 has a module utils, and does have __init__.py
My boiler-plate code (before dir1 was part of the environment) has imports from dir2 of the form
from utils.pkg2.mod2 import func2
The problem comes in that I'm now adapting this code to call functions that import from utils in dir1; I cannot alter that part of the environment.
What can I do to make my code go for the dir2/utils module? Unfortunately, this also needs to be adaptable to Python 2.6.6 and later.
I have search existing questions on SO and elsewhere; all the answers I've found depend on some package "handle" that I do not have.

This import statement is incorrect:
from utils.pkg2.mod2 import func2
If it has ever worked correctly, that was relying on resolving with the current working directory, implicit relative imports in Python 2.x, or a manually munged PYTHONPATH / sys.path.
This is the type of import for which PEP8 said:
Implicit relative imports should never be used and have been removed in Python 3.
So what to do instead? sys.path should be augmented with top-level directories, not intra-package directories, i.e.:
PYTHONPATH=/path1/dir1:/path2
And change imports like this:
from dir2.utils.pkg2.mod2 import func2
Now the sub-package dir2.utils is namespaced from the top-level package utils.

How to cross-import modules in subdirectory so they work as sub-module and as a stand-alone?

I have a Python module that normally works as a stand-alone.
file1.py
file2.py
file3.py
However, I also want it to be part of a different project, in which the module is placed in a separate subdirectory.
__init.py__
build.py
└── compiler
└── __init__.py
└── file1.py
└── file2.py
└── file3.py
Since the module scripts use plenty of cross-imports, this is not possible. Once placed in a subdirectory, the imports no longer find the respective files because it looks in the top directory only.
To remedy the problem, I tried various things. I appended the subdirectory as an additional path in the top-most build.py script.
sys.path.append('compiler')
It did not solve the problem. Cross-imports are still not working.
I also tried relative imports but that breaks the stand-alone version of the module. So, I tried exception handling to catch them
try:
from file1 import TestClass
except ImportError:
from .file1 import TestClass
That did not work either and resulted, despite my best efforts in ImportError: attempted relative import with no known parent package errors.
I also tried all sorts of variations of these things, but none of it worked.
I know it has to be possible to do something like this and I am surprised that this is so hard to do. My Internet searches all came back with the same suggestions—the ones I outlined above, none of which worked in my case, particularly because they do not take into account the ability to run code as a stand-alone and as a sub-module.
I can't be the first person trying to write a module that can be used as a stand-alone package or as a sub-module in other projects. What am I missing here?

Relative imports, as the error tells you, require a parent package. Think of .file1 as shorthand for <this_module>.file1. If there is no <this_module>, you can't ask it for file1. In order to properly use relative imports you'll have to make a wrapper project to contain the shared module so it can be properly namespaced.
So your standalone module will instead look like this, matching the consumer:
__init.py__
standalone.py
└── compiler
└── __init__.py
└── file1.py
└── file2.py
└── file3.py
The other option is to make your shared module truly installable with a setup.py or pyproject.toml or whatever your favorite method is. Then you install it in the consuming project instead of directly including it.

Can a Python script in a (sub)module import from upstream in its directory hierarchy?

I realize there are a slew of posts on SO related to Python and imports, but it seems like a fair number of these posts are asking about import rules/procedures with respect to creating an actual Python package (vs just a project with multiple directories and python files). I am very new to Python and just need some more basic clarification on what is and is not possible with regard to access/importing within the context of multiple py files in a project directory.
Let's say you have the following project directory (to be clear, this is not a package that is somewhere on sys.path, but say, on your Desktop):
myProject/
├── __init__.py
├── scriptA.py
└── subfolder
├── __init__.py
└── scriptB.py
└── subsubfolder
├── __init__.py
└── scriptC.py
└── foo.py
Am I correct in understanding that the only way scriptC.py could import and use methods or classes within scriptB.py if scriptC.py is run directly via $ python scriptC.py and from within the subsubfolder directory is if I add the parent directory and path to scriptB.py to the Python path at runtime via sys.path ?
It is possible, however, for scriptC.py to import foo.py or for scriptB.py to import scriptC.py or foo.py without dealing with sys.path, correct? Adjacent py files and py files in subdirectories are accessible just by using relative import paths, you just can't import python scripts that live in parent or sibling directories (without using sys.path) ?

What's Possible
Anything.
No, really. See the imp module, the the imputil module -- take a look at how the zipimport module is written if you want some inspiration.
If you can get a string with your module's code in a variable, you can get a module into sys.modules using the above, and perhaps hack around with its contents using the ast module on the way.
A custom import hook that looks in parent directories? Well within the range of possibilities.
What's Best Practice
What you're proposing isn't actually good practice. The best-practice approach looks more like the following:
myProject/
├── setup.py
└── src/
├── moduleA.py
└── submodule/
├── __init__.py
├── moduleB.py
└── subsubmodule/
├── __init__.py
└── moduleC.py
Here, the top of your project is always in myProject/src. If you use setup.py to configure moduleA:main, submodule.moduleB:main and submodule.subsubmodule.moduleC:main as entry points (perhaps named scriptA, scriptB and scriptC), then the functions named main in each of those modules would be invoked when the user ran the (automatically generated by setuptools) scripts so named.
With this layout (and appropriate setuptools use), your moduleC.py can absolutely import moduleA, or import submodule.moduleB.
Another approach, which doesn't involve entrypoints, to invoke the code in your moduleC.py (while keeping the module's intended hierarchy intact, and assuming you're in a virtualenv where python setup.py develop has been run) like so:
python -m submodule.subsubmodule.moduleC

Intrapackage module loads in python

I am just starting with python and have troubles understanding the searching path for intra-package module loads. I have a structure like this:
top/ Top-level package
__init__.py Initialize the top package
src/ Subpackage for source files
__init__.py
pkg1/ Source subpackage 1
__init__.py
mod1_1.py
mod1_2.py
...
pkg2/ Source subpackage 2
__init__.py
mod2_1.py
mod2_2.py
...
...
test/ Subpackage for unit testing
__init__.py
pkg1Test/ Tests for subpackage1
__init__.py
testSuite1_1.py
testSuite1_2.py
...
pkg2Test/ Tests for subpackage2
__init__.py
testSuite2_1.py
testSuite2_2.py
...
...
In testSuite1_1 I need to import module mod1_1.py (and so on). What import statement should I use?
Python's official tutorial (at docs.python.org, sec 6.4.2) says:
"If the imported module is not found in the current package (the package of which the current module is a submodule), the import statement looks for a top-level module with the given name."
I took this to mean that I could use (from within testSuite1_1.py):
from src.pkg1 import mod1_1
or
import src.pkg1.mod1_1
neither works. I read several answers to similar questions here, but could not find a solution.
Edit: I changed the module names to follow Python's naming conventions. But I still cannot get this simple example to work.

The module name doesn't include the .py extension. Also, in your example, the top-level module is actually named top. And finally, hyphens aren't legal for names in python, I'd suggest replacing them with underscores. Then try:
from top.src.pkg1 import mod1_1

Problem solved with the help of http://legacy.python.org/doc/essays/packages.html (referred to in a similar question). The key point (perhpas obvious to more experienced python developers) is this:
"In order for a Python program to use a package, the package must be findable by the import statement. In other words, the package must be a subdirectory of a directory that is on sys.path. [...] the easiest way to ensure that a package was on sys.path was to either install it in the standard library or to have users extend sys.path by setting their $PYTHONPATH shell environment variable"
Adding the path to "top" to PYTHONPATH solved the problem.To make the solution portable (this is a personal project, but I need to share it across several machines), I guess having a minimal initialization code in top/setup.py should work.