We Keep Coding

To be imported is it necessary for a directory to be in sys.path?

I have this package I designed which have the following structure,
General/
__init__.py
Vector/
__init__.py
module_1.py
module_A.py
In module_A there is a just a function prints a string.
In module_1 there is,
from ...General import module_A
module_A.function()
Then I create a separate python file First.py which only has the line import General.Vector.module_1.
I put this file in to the parent directory of General and locate cmd to the parent directory and run python First.py
Then I get the error ValueError: attempted relative import beyond top-level package
But if I change module_1 to
from .. import module_A
module_A.function()
Then this works.
I searched for a solution and from this post, I got a good understanding about relative imports. Then it was pointed out to me that if I add my package to sys.path then python First.py would work regardless of where the First.py is located.
So I tried to add my package to sys.path as this post suggests.
I changed my First.py to
import sys
sys.path.insert(0,'/path/to/General')
import General.Vector.module_1
But that didn't work.
1 I like to know the reason for this two different behaviours for module_1's changes.
2 I like to know how to add my package to sys.path.

python: import problems with directory structure that keeps scripts and modules separate

I have the following directory structure:
app/
bin/
script1.py
script2.py
lib/
module1/
__init__.py
module1a.py
module1b.py
__init__.py
module2.py
Dockerfile
My problem is that I want to execute script1.py and script2.py, but inside those scripts, I want to import the modules in lib/.
I run my scripts from the root app/ directory (i.e. adjacent to Dockerfile) by simply executing python bin/script1.py. When I import modules into my scripts using from lib.module1 import module1a, I get ImportError: No module named lib.module1. When I try to import using relative imports, such as from ..lib.module1 import module1a, I get ValueError: Attempted relative import in non-package.
When I simply fire up the interpreter and run import lib.module1 or something, I have no issues.
How can I get this to work?

In general, you need __init__.py under app and bin, then you can do a relative import, but that expects a package
If you would structure your python code as python package (egg/wheel) then you could also define an entry point, that would become your /bin/ file post install.
here is an example of a package - https://python-packaging.readthedocs.io/en/latest/minimal.html
and this blog explains entry points quite well - https://chriswarrick.com/blog/2014/09/15/python-apps-the-right-way-entry_points-and-scripts/
if so, that way you could just do python setup.py install on your package and then have those entry points available within your PATH, as part of that you would start to structure your code in a way that would not create import issues.

You can add to the Python path at runtime in script1.py:
import sys
sys.path.insert(0, '/path/to/your/app/')
import lib.module1.module1a

you have to add current dir to python path.
use export in terminal
or sys.path.insert in your python script both are ok

Python: importing a sub-package module from both the sub-package and the main package

Here we go with my first ever stackoverflow quesion. I did search for an answer, but couldn't find a clear one. Here's the situation. I've got a structure like this:
myapp
package/
__init.py__
main.py
mod1.py
mod2.py
Now, in this scenario, from main.py I am importing mod1.py, which also needs to be imported by mod2.py. Everything works fine, my imports look like this:
main.py:
from mod1 import Class1
mod2.py:
from mod1 import Class1
However, I need to move my main.py to the main folder structure, like this:
myapp
main.py
package/
__init.py__
mod1.py
mod2.py
And now what happens is that of course I need to change the way I import mod1 inside main.py:
from package.mod1 import Class1
However, what also happens is that in order not to get an "ImportError: No module named 'mod1'", I have make the same type of change inside mod2.py:
from package.mod1 import Class1
Why is that? mod2 is in the same folder/pakcage as mod1, so why - upon modifying main.py - am I expected to modify my import inside mod2?

The reason this is happening is because how python looks for modules and packages when you run a python script as the __main__ script.
When you run python main.py, python will add the parent directory of main.py to the pythonpath, meaning packages and modules within the directory will be importable. When you moved main.py, you changed the directory that was added to the pythonpath.
Generally, you don't want to rely on this mechanism for importing your modules, because it doesn't allow you to move your script and your package and modules are only importable when running that main script. What you should do is make sure your package is installed into a directory that is already in the pythonpath. There are several ways of doing this, but the most common is to create a setup.py script and actually install your python package for the python installation on your computer.

python module and imports

If this is my directory tree
temp
├── __init__.py
└── __main__.py
0 directories, 2 files
And I have the following code in __init__.py and in __main__.py
__init__.py
"""Initializes the module"""
CONSTANT = 1
sys.exit("what is happening here")
__main__.py
# from . import CONSTANT
# from temp import CONSTANT
if __name__ == "__main__":
print "This should never run"
I am getting two problems here that I am trying to figure out
On running python . in the temp directory I get the output
This should never run, shouldn't the module be initialized first with the __init__.py file resulting in the abort?
Second how do I go about doing imports in python modules? Neither of the two options I have mentioned above works. I can neither do from . import CONSTANT nor from temp import CONSTANT in the code above. What is the right way to do relative imports?
I am running this on Python 2.7.5, apologies if this has already been asked before.

You should be running it from out of the temp directory. If someDir contains your temp directory, then:
someDir $ python -m temp #someDir/temp/__init__.py is your file.
On running python . in the temp directory I get the output This should never run, shouldn't the module be initialized first with the init.py file resulting in the abort?
If you run it from outside, __init__.py will be called. And sys.exit will be called too.
Second how do I go about doing imports in python modules? Neither of the two options I have mentioned above works. I can neither do from . import CONSTANT nor from temp import CONSTANT in the code above. What is the right way to do relative imports?
You are doing it just fine. Just import sys in your __init__.py file. And fix the spelling of CONSTANT.
Also why do I need the -m flag? Isn't it ok to just do python temp from the parent directory of temp?
You need the -m flag to tell that you are using packages. If you dont use it you wont be able to do relative imports.

When you tell Python to run a directory, Python does not treat the directory as a package. Instead, Python adds that directory to sys.path and runs its __main__.py. __init__.py is not executed, and relative imports will not view the directory as a package.
If you want to run a package's __main__.py and treat it as part of the package, with __init__.py executed and all, go to the directory containing the package and run
python -m packagename

You are running inside temp; this is not considered a package and __init__.py is not loaded. Only if the parent of the current directory is on the module loading path and you explicitly load temp as a module, is __init__.py loaded.
Because temp is not a package you can't use relative imports here. Instead, every Python file inside of the directory is considered a top-level module all by themselves.
You'd have move to the parent of the temp directory, then run:
python -m temp
for Python to import temp as a package and then run the __main__ module in that package.

python packaging for relative imports

First off all: I'm sorry, I know there has been lots of question about relative imports, but I just didn't find a solution. If possible I would like to use the following directory layout:
myClass/
__init__.py
test/
demo.py
benchmark.py
specs.py
src/
__init__.py
myClass.py
Now my questions are:
How do the test files from within the package properly import myClass.py?
How would you import the package from outside, assuming you take myClass as submodule in libs/myClass or include/myClass?
So far I couldn't find an elegant solution for this. From what I understand Guido's Decision it should be possible to do from ..src import myClass but this will error:
ValueError: Attempted relative import in non-package
Which looks as it doesn't treat myClass as packages. Reading the docs:
The __init__.py files are required to make Python treat the directories as containing packages;
It seems I'm missing something that specifies where the scripts of the package are, should I use .pth ?

ValueError: Attempted relative import in non-package
Means you attempt to use relative import in the module which is not package. Its problem with the file which has this from ... import statement, and not the file which you are trying to import.
So if you are doing relative imports in your tests, for example, you should make your tests to be part of your package. This means
Adding __init__.py to test/
Running them from some outside script, like nosetests
If you run something as python myClass/test/demo.py, relative imports will not work too since you are running demo module not as package. Relative imports require that the module which uses them is being imported itself either as package module, from myClass.test.demo import blabla, or with relative import.

After hours of searching last night I found the answer to relative imports in python!! Or an easy solution at the very least. The best way to fix this is to have the modules called from another module. So say you want demo.py to import myClass.py. In the myClass folder at the root of the sub-packages they need to have a file that calls the other two. From what I gather the working directory is always considered __main__ so if you test the import from demo.py with the demo.py script, you will receive that error. To illustrate:
Folder hierarchy:
myClass/
main.py #arbitrary name, can be anything
test/
__init__.py
demo.py
src/
__init__.py
myClass.py
myClass.py:
def randomMaths(x):
a = x * 2
y = x * a
return y
demo.py:
from ..src import myClass
def printer():
print(myClass.randomMaths(42))
main.py:
import test.demo
demo.printer()
If you run demo.py in the interpreter, you will generate an error, but running main.py will not. It's a little convoluted, but it works :D

Intra-package-references describes how to myClass from test/*. To import the package from outside, you should add its path to PYTHONPATH environment variable before running the importer application, or to sys.path list in the code before importing it.
Why from ..src import myClass fails: probably, src is not a python package, you cannot import from there. You should add it to python path as described above.