Currently, I'm playing around with absolute and relative imports for my tests. Below is the example of the file structure.
File Structure:
project/
main_script.py
input1.csv
input2.csv
Testing/
test_script.py
test_input1.csv
test_input2.csv
test_output.csv
Code in main_script.py
def function1():
{function definition here}
def function2():
{function definition here}
def function3():
{function definition here}
Code in test_script.py
import pytest
from ..main_script import function1, function2, function3
The output I get when running test_script:
ImportError: attempted relative import with no known parent package
From the examples I've read from different sources and tutorials I have watched, this should have been the solution to my problem but it's coming out with more errors. Why is that?
You can try any one of these ways -
Use absolute import
Use standard way of import and remove from keyword
Example: import main_script.function1
Put this inside your package's init.py -
For relative imports to work in Python 3.6 -
import os, sys; sys.path.append(os.path.dirname(os.path.realpath(file)))
And now use normal import statement like :
from main_script import function1
Reference Attached
Do let me know if this helps out.
Happy Learning!!
Related
This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed 3 months ago.
I have a directory and module design that looks like this:
MyProject --- - script.py
|
- helpers --- - __init__.py
|
- class_container.py
|
- helper.py
# class_container.py
class MyClass:
pass
# helper.py
from class_container import MyClass
def func():
# some code using MyClass
# script.py
from helpers.helper import func
When I run script.py:
ModuleNotFoundError: No module named 'class_container'
I tried changing the code in helper.py such as:
# helper.py
from helpers.class_container import MyClass
def func():
# some code using MyClass
Then running script.py started working. But when I explicitly run helper.py:
ModuleNotFoundError: No module named 'helpers'
I want to be able run both script.py and helper.py separately without needing to change the code in any module.
Edit: I figured a solution which is changing helper.py such as:
from pathlib import Path
import sys
sys.path.append(str(Path(__file__).parent))
from class_container import MyClass
def func():
# some code using MyClass
Basically I added the directory of helper.py to sys.path by using sys and pathlib modules and __file__ object. And unlike import statement's behaviour, __file__ will not forget it's roots when imported/used from a different module (i.e it won't become the path of script.py when imported into it. It'll always be the path of helper.py since it was initiated there.).
Though I'd appreciate if someone can show another way that doesn't involve messing with sys.path, it feels like an illegal, 'unpythonic' tool.
You have to create a __init__.py file in the MyProject/helpers directory. Maybe you already have created it. If not, create an empty file.
Then in the MyProject/helpers/helper.py, access the module helpers.class_container like this.
from helpers.class_container import MyClass
def func():
# some code using MyClass
You can also use a relative import like this.
from .class_container import MyClass
If you want to run the MyProject/helpers/helper.py independently, add test code in helper.py like this.
from helpers.class_container import MyClass
def func():
# some code using MyClass
if __name__ == '__main__':
func()
And run like this in the MyProject directory.(I assume a Linux environment.)
$ python3 -m helpers.helper
The point is to differentiate Python modules from Python scripts and treat them differently.
I am having a lot of trouble understanding the python module import system.
I am trying to create a simple folder structure as follows.
SomeModule
__init__.py
AnotherModule
AnotherModule.py
__init__.py
Utils
Utils.py
__init__.py
To use SomeModule i can do:
SomeModule.Foo()
Now inside AnotherModule.py I would like to import my Utils directory.
How come I have to do
import SomeModule.AnotherModule.Utils.Foo
why cannot I just do
import Utils.Foo
To shorten up the actual function name that you'll have to call in your code, you can always do:
from SomeModule.AnotherModule.Utils import *
While this still won't allow you to get away with a shorter import statement at the top of your script, you'll be able to access all of the functions within .Utils just by calling their function name (i.e. foo(x) instead of SomeModule.AnotherModule.Utils.foo(x).
Part of the reason for the lengthy import statement goes to the comment from #wim . Have a look by typing import this in a python interpreter.
put
import sys
import SomeModule.AnotherModule
sys.modules['AnotherModule'] = SomeModule.AnotherModule
in SomeModules __init__.py
i've run through many posts about this, but still doesn't seem to work. The deal is pretty cut. I've the got the following hierarchy.
main.py
DirA/
__init__.py
hello.py
DirB/
__init__.py
foo.py
bla.py
lol.py
The__init__.py at DirA is empty. The respective one at DirB just contains the foo module.
__all__.py = ["foo"]
The main.py has the following code
import DirA
import DirB
hey() #Def written at hello.py
foolish1() #Def written at foo.py
foolish2() #Def written at foo.py
Long story short, I got NameError: name 'foo' is not defined. Any ideas? Thanks in advance.
You only get what you import. Therefore, in you main, you only get DirA and DirB. You would use them in one of those ways:
import DirA
DirA.something_in_init_py()
# Importing hello:
import DirA.hello
DirA.hello.something_in_hello_py()
# Using a named import:
from DirA.hello import something_in_hello_py
something_in_hello_py()
And in DirB, just make the __init__.py empty as well. The only use of __all__ is for when you want to import *, which you don't want because, as they say, explicit is better than implicit.
But in case you are curious, it would work this way:
from DirB import *
something_in_dirb()
By default the import * will import everything it can find that does not start with an underscore. Specifying a __all__ restricts what it imported to the names defined in __all__. See this question for more details.
Edit: about init.
The __init__.py is not really connected to the importing stuff. It is just a special file with the following properties:
Its existence means the directory is a python package, with several modules in it. If it does not exist, python will refuse to import anything from the directory.
It will always be loaded before loading anything else in the directory.
Its content will be available as the package itself.
Just try it put this in DirA/__init__.py:
foo = 42
Now, in your main:
from DirA import foo
print(foo) # 42
It can be useful, because you can import some of your submodules in the __init__.py to hide the inner structure of your package. Suppose you build an application with classes Author, Book and Review. To make it easier to read, you give each class its own file in a package. Now in your main, you have to import the full path:
from myapp.author import Author
from myapp.book import Book
from myapp.review import Review
Clearly not optimal. Now suppose you put those exact lines above in your __init__.py, you may simplify you main like this:
from myapp import Author, Book, Review
Python will load the __init__.py, which will in turn load all submodules and import the classes, making them available on the package. Now your main does not need to know where the classes are actually implemented.
Have you tried something like this:
One way
from DirA import hello
Another way
from DirA.hello import hey
If those don't work then append a new system path
You need to import the function itself:
How to call a function from another file in Python?
In your case:
from DirA import foolish1, foolish2
I am using someone elses project to add some functionality to mine, and there is a python script I want to import. The problem comes with the import structure of their directories: I have placed their project directory in a subfolder under my main project (needs to stay there so I can keep their project out of my version control) it looks like this:
myproject/
myscript.py
theirproject/
__init__.py
baz.py
secondlayer/
__init__.py
all.py
foo.py
bar.py
all.py is simply a list of import statements which import additional scripts from the secondlayer directory like this:
from secondlayer.foo import *
from secondlayer.bar import * #etc
I would like to import:
from theirproject.secondlayer.all import *
but that fails when python complains "no module named secondlayer.foo"
I have also tried the following:
from theirproject.secondlayer import all
I can get it to work when I place my script in theirproject/ and import all without the "theirproject" prefix, but I really cant have it be like that. I can get further through the import process by importing foo, bar, etc individually like this:
from theirproject.secondlayer import foo
from theirproject.secondlayer import bar #etc
But then those scripts fail to import more stuff from still other scripts (like baz.py) at the same level as secondlayer, so im stuck.
Whats the right way to do this in python 2.7.6?
If you change
from secondlayer.foo import *
from secondlayer.bar import *
to user relative imports like this
from .foo import *
from .bar import *
or like this
from foo import *
from bar import *
it works.
Plus you could do these imports in the __init__.py at secondlayer level so that the import from myscript.py becomes
from theirproject.secondlayer.all import *
See if you have the necessary permissions to import the package from your directory and its corresponding sub directories.
For reference, you may like to see this and its corresponding linked questions :
Python Imports do not work
I ended up solving my problem by adding theirproject/ to my PYTHONPATH. I upvoted junnytony's answer - it helped point me in the right direction, so thanks!
Let's say I have such a directory structure:
- Documents/
- thesis_program/
- __init__.py
- classes.py
- utils.py
- GE_Test.py
- GE_Test_fail.py
classes.py and utils.py contains some classes and functions.
GE_Test.py and GE_Test_fail.py contains the exactly same code, except the import part.
In GE_Test.py I import classes and utils this way:
from utils import execute
from classes import Grammatical_Evolution
While in GE_Test_fail.py, I import classes and utils this way:
from thesis_program.utils import execute
from thesis_program.classes import Grammatical_Evolution
And unexpectedly I get a different result. Is there anything wrong here?
Do I import the modules correctly?
I can ensure that the result should be the same, because I generate the random number with certain seed
Also classes.py is somehow depended on utils.py since I have several common functions in utils.py. I suspect that utils is also a name used by the system. So in the second case (GE_Test_fail.py) The system utils override my utils.py. But it doesn't seem make sense for me.
The complete source code of classes.py and utils.py is available here (if it helps to discover what's wrong): https://github.com/goFrendiAsgard/feature-extractor
And also, the screenshots: https://picasaweb.google.com/111207164087437537257/November25201204?authuser=0&authkey=Gv1sRgCOKxot2a2fTtlAE&feat=directlink
add below mentioned lines to your test files which are going outside of your thesis folder.
import sys sys.path.insert(0,"/path to your thesis folder/thesis_program")
and maintain everything else; for example in GE_Test.py.. .
import sys
sys.path.insert(0,"/path to your thesis folder/thesis_program")
from utils import execute
from classes import Grammatical_Evolution
EDIT:
Or use this to make it more dynamic
(caution: don't try to find the path by os.path.abspath('./thesis_program') because it may not be always possible that you find your test_files and your thesis_folder are in the same dir; if you can fix them permanently in your code like above; then you are free to use them from anywhere on your system)
import os, sys
lib_path = os.path.abspath('./thesis_program')
sys.path.insert(0,lib_path)