We Keep Coding

Why does importing modules result in a circular import error

I have this for the layout of my project:
projectFolder /
setup/
__init__.py
setup.py
Utils /
__init__.py
cloudmanager.py
startup.py
I am trying to import the Setup Module inside my cloudmanager.py script (which is nested in one more directory). I can easily import both the setup module and the Utils module inside my startup.py script since it's in the root directory.
I've tried (inside my cloudmanager.py script):
from . import setup
Which gives me the error of:
ImportError: cannot import name 'setup' from partially initialized module 'Utils' (most likely due to a circular import)
and I've tried:
from .. import setup
Which gives me the error of:
ValueError: attempted relative import beyond top-level package
Any help? There are questions like this out there but they steer towards to using OS, which I'd like to avoid...

Okay, so the reason you're getting an error importing .. setup is indeed that you can't do relative imports when the parent directory is a package. A package is any directory with a __init__.py file in it.
You could solve this by doing one of two things:
You could make sure the root of your project is in the Python path and import everything in the root.
You could make your project's root directory itself a package and then use relative imports.
Option 1: Importing from the project root
If your projectFolder folder lives at /home/you/projects/projectFolder, make sure your PYTHONPATH has /home/you/projects/projectFolder in it. For example, when you run your main script, you could set it before hand. In bash (assuming a Unix environment):
export PYTHONPATH=/home/you/projects/projectFolder
python /home/you/projects/projectFolder/startup.py
You could also do that inside startup.py, if you want to avoid changing the external environment:
# startup.py
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__)))
If you do that in startup.py, the directory of startup.py will always be in the Python path.
Once you one of those, you can base all your imports on the relative location of your project. Eg:
import setup.setup
import Utils.cloudmanager
(That will work in every file you import after the sys.path mutation runs)
Option 2: Relative imports
If you make your project's root a Python package, you can use relative imports entirely. Eg, you'd have these files:
projectFolder/__init__.py
projectFOlder/setup/__init__.py
projectFolder/setup/setup.py
projectFolder/Utils/__init__.py
projectFolder/Utils/cloudmanager.py
If you do that, inside cloudmanager.py, you could run from .. import setup just fine.
What do you do?
Both of these are valid options. In general relative imports have less ambiguity, since they avoid name collisions, but they're a newer feature of Python so option #1 is more common, in general.

Try to use:
import setup.setup

Python: Import scrypt from subfolder to another subfolder

I am working on some python project (2.7) and I have issue with imports. When I run main.py it start scripts from tests folder etc. and save output to logs and everything works fine.
/root------------
-logs
-staticCfg
-config.py
-tests
-systemTest
-scrypt1.py
-scrypt2.py
-userTest
-uScrypt1.py
main.py
My static variables (email, name etc.) are located in config.py. I need to import config.py in scrypt1.py or scrypt2.py. I tryed adding __init__.py to tests, systemTest and staticCfg folder but I always get an error.
In my scrypt1.py:
import staticCfg as cfg
...
or
from staticCfg import *
...
I get the error:
ImportError: No module named staticCfg

The import mechanism of Python can be a bit tricky.
You can refer to the documentation for more information: Python Import Mechanism
When you use absolute imports (your import does not start with a .) as you do, the import path will start from your main script (the one you launch). In your case, it's scrypt1.py. So starting from this location, python can't find the package staticCfg.
For me, the simplest solution is to create a main script in your root directory and call scrypt1.py from there (imported using from tests.systemTet import scrypt1.py). In this case, the base package will be your root folder and you will have access to the package staticCfg from all your script files as you wanted to do.

you may add root folder to PYTHONPATH.

Python Relative Import within Brother/Sister Folder for AWS CodeBuild

I am building on AWS CodeBuild using Python 2.7, but I believe this is much more a generic python import problem. I have a directory setting, shown below. I am running test.py inside the test folder. I would like to import the dependency mainScript.py as part of this testing. However, I cannot seem to get the relative dependencies right and I am having a lot of difficulty importing the mainScript within the test folder. Below is a layout of my directory folder
main
src
mainScript.py
test
test.py
If for example my directory setup was something like
main
test
test.py
mainScript.py
I could have my import be done the following way
from mainScript import *
I have confirmed this works. But, I like it in its own src folder. I have tried all these These are the following relative path attempts I have tried
from ..src/mainScript import * #SyntaxError: invalid syntax
from ..src.mainScript import * #ValueError: attempted relative import beyond top-level package
from mainScript import * #ModuleNotFoundError: No module named 'mainScript'
from src.mainScript import * #ModuleNotFoundError: No module named 'src'
from src/mainScript import * #SyntaxError: invalid syntax
I have been struggling for a bit and I couldn't quite find a question with someone asking about accessing a brother/sister folder script. Thank you in advance for your help.

Python treats directories as packages if they contain a __init__.py file. Updating your structure to the following should do the trick:
__init__.py
src
__init__.py
mainScript.py
test
__init__.py
test.py
Now, from test.py, you could do from ..src import *. For more details on init.py, you can look here: What is __init__.py for?

In addition to adding the init.py files. It ended up being that I had to run python with the -m argument in my command, which was added in Python 2.4.
PEP 338
Python 2.4 adds the command line switch -m to allow modules to be
located using the Python module namespace for execution as scripts.
The motivating examples were standard library modules such as pdb and
profile, and the Python 2.4 implementation is fine for this limited
purpose.
So the command to launch from the top directory is:
python -m test.test
This seems to work and get the right namespace. Then in your test.py file you would import the mainScript the following way
from src.mainScript import *

python: import problems with directory structure that keeps scripts and modules separate

I have the following directory structure:
app/
bin/
script1.py
script2.py
lib/
module1/
__init__.py
module1a.py
module1b.py
__init__.py
module2.py
Dockerfile
My problem is that I want to execute script1.py and script2.py, but inside those scripts, I want to import the modules in lib/.
I run my scripts from the root app/ directory (i.e. adjacent to Dockerfile) by simply executing python bin/script1.py. When I import modules into my scripts using from lib.module1 import module1a, I get ImportError: No module named lib.module1. When I try to import using relative imports, such as from ..lib.module1 import module1a, I get ValueError: Attempted relative import in non-package.
When I simply fire up the interpreter and run import lib.module1 or something, I have no issues.
How can I get this to work?

In general, you need __init__.py under app and bin, then you can do a relative import, but that expects a package
If you would structure your python code as python package (egg/wheel) then you could also define an entry point, that would become your /bin/ file post install.
here is an example of a package - https://python-packaging.readthedocs.io/en/latest/minimal.html
and this blog explains entry points quite well - https://chriswarrick.com/blog/2014/09/15/python-apps-the-right-way-entry_points-and-scripts/
if so, that way you could just do python setup.py install on your package and then have those entry points available within your PATH, as part of that you would start to structure your code in a way that would not create import issues.

You can add to the Python path at runtime in script1.py:
import sys
sys.path.insert(0, '/path/to/your/app/')
import lib.module1.module1a

you have to add current dir to python path.
use export in terminal
or sys.path.insert in your python script both are ok

Importing files in different directories in Python

I have a directory structure like this:
dir/
frontend.py
dir1/main.py
dir2/backend.py
How do I import backend in main in Python?
How do I import frontend in main in Python?
Have tried all the answers on Stackoverflow. Nothing seems to work.

In any folder from which you want to import source files you need to have existing init.py file.
I would advise structure like:
dir/
main.py
dir1/frontend.py
dir1/__init__.py
dir2/backend.py
dir2/__init__.py
Then you import them in following fashion (in main.py):
import dir1.frontend
import dir2.backend

There is only one rule when it comes to importing files in a Python project.
You have to import the package relative to the directory from where the project is run.
For example in the question main.py should have something like this:
from dir.frontend import *
from dir.dir2.backend import *
But then you'll have to have something like main.py under dir/ which imports dir/dir1/main.py and then run python main.py.
So, try to keep the main.py always in the head directory so you don't have to worry about such a situation as above.
ONLY ONE RULE: Everything has to be imported relatively to the directory from where the project is run.