Let's say I have such a directory structure:
- Documents/
- thesis_program/
- __init__.py
- classes.py
- utils.py
- GE_Test.py
- GE_Test_fail.py
classes.py and utils.py contains some classes and functions.
GE_Test.py and GE_Test_fail.py contains the exactly same code, except the import part.
In GE_Test.py I import classes and utils this way:
from utils import execute
from classes import Grammatical_Evolution
While in GE_Test_fail.py, I import classes and utils this way:
from thesis_program.utils import execute
from thesis_program.classes import Grammatical_Evolution
And unexpectedly I get a different result. Is there anything wrong here?
Do I import the modules correctly?
I can ensure that the result should be the same, because I generate the random number with certain seed
Also classes.py is somehow depended on utils.py since I have several common functions in utils.py. I suspect that utils is also a name used by the system. So in the second case (GE_Test_fail.py) The system utils override my utils.py. But it doesn't seem make sense for me.
The complete source code of classes.py and utils.py is available here (if it helps to discover what's wrong): https://github.com/goFrendiAsgard/feature-extractor
And also, the screenshots: https://picasaweb.google.com/111207164087437537257/November25201204?authuser=0&authkey=Gv1sRgCOKxot2a2fTtlAE&feat=directlink
add below mentioned lines to your test files which are going outside of your thesis folder.
import sys sys.path.insert(0,"/path to your thesis folder/thesis_program")
and maintain everything else; for example in GE_Test.py.. .
import sys
sys.path.insert(0,"/path to your thesis folder/thesis_program")
from utils import execute
from classes import Grammatical_Evolution
EDIT:
Or use this to make it more dynamic
(caution: don't try to find the path by os.path.abspath('./thesis_program') because it may not be always possible that you find your test_files and your thesis_folder are in the same dir; if you can fix them permanently in your code like above; then you are free to use them from anywhere on your system)
import os, sys
lib_path = os.path.abspath('./thesis_program')
sys.path.insert(0,lib_path)
Related
My project structure like this:
/project
main.py
/a_module
__init__.py
/sub_module
__init__.py
some_file.py
main.py
from a_module import main_api
a_module/__init__.py
from sub_module import sub_api
sub_module/__init__.py
from some_file import detail_api
In a_module/__init__.py gives Unable to import 'sub_module' error.
Why I cannot import 'sub_module'?
When I change to the relative path solve the error.
from .sub_module import sub_api
But I don't understand, does __init__.py design for public the API of the module? Why don't treat sub_module as a module instead of a directory? it's such a bad design to me...
__init__.py is executed when you import the package that contains it. But it's not your problem. Your problem is that module imports are always absolute unless explicitly relative. That means that they must chain from some directory in sys.path. By default this includes the working directory, so when you run main.py from within project, it can find a_module, and nothing else.
from sub_module import sub_api
In a_module/__init__.py doesn't work though, because imports are always absolute unless explicitly relative. So that import says "starting from some sys.path root, find a top level package named sub_module and import sub_api from it". Since no such module exists you get an error. from .sub_module import sub_api works because you opted into relative imports, so it doesn't start over from sys.path.
For an example of why you would do this, I'll give you something that broke in our own code back in the Python 2 days before absolute import by default was the law (from __future__ import absolute_import enabled the Py3 behavior, which is how we fixed it, but despite what the docs say, it was never enabled by default in Py2, the only enabled by default behavior was relative imports). Our layout was:
teamnamespace/
module.py
math/
mathrelatedsubmodule.py
othermathsubmodule.py
Now, we innocently thought hey, we'll put all our packages under a single shared top level namespace, and subpackages cover broad categories within them, and since we had a lot of additional utilities for basic mathematics, we put them under teamnamespace.math. Problem was, for the non-math modules, like teamnamespace.module, when they did:
import math # or
from math import ceil
it defaulted to relative lookup, and imported teamnamespace.math as math (a thoroughly useless import, since it was a namespace package only, all the functionality was in the sub-modules), not the built-in math module. In fact, without the Python 3 behavior, there was no reasonable way to get the built-in math module from a module under teamnamespace. Whereas with the Python 3 behavior, you can get either one or both (by aliasing one or the other with as, with no ambiguity:
# Gets built-in
import math
# Gets teamnamespace.math
from . import math
So, this is a set of questions about how to use __init__.py in packages/sub-packages. I have searched, and surprisingly not found a decent answer to this.
If I have the following structure (which is just a simplified example obviously):
my_package/
__init__.py
module1.py
my_sub_package/
__init__.py
module2.py
the contents of module1.py is
my_string = 'Hello'
the contents of module2.py is
my_int = 42
First question: importing multiple modules from a package by default
What should be in the __init__.py files?
I can leave them empty, in which case, import my_package does nothing really (obviously it imports the package, but the package effectively contains nothing). This is fine obviously, and what should happen in most cases.
What I'd like in this case though is for import my_package to allow me to use my_package.module1.my_string and my_package.my_sub_package.module2.my_int.
I can add __all__ = [ 'module1' ] to my_package/__init__.py and __all__ = [ 'module2' ] to my_package/my_sub_package/__init__.py, but this only affects imports using a wildcard as I understand it (so only from my_package import * and from my_package.my_sub_package import *).
I can achieve this by putting
import my_package.module1
import my_package.my_sub_package
in my_package/__init__.py and
import my_package.my_sub_package.module2
in my_package/my_sub_package/__init__.py, but is this a bad idea? It creates a (seemingly) infinite series of my_package.my_package.my_package.... when I do this in the Python interpreter (3.5.5).
Separate, but highly related, question: using modules to keep files reasonably sized
If I wanted instead to be able to do the following
import my_package
print(my_package.my_string)
print(str(my_package.my_sub_package.my_int))
i.e. I wanted to use module1 and module2 purely for separating code into smaller more readable files if I actually had lots of modules in each package (which obviously doesn't apply in this trivial example, but can easily)
is doing from my_package.module1 import * in my_package/__init__.py and from my_package.my_sub_package.module2 import * in my_package/my_sub_package/__init__.py a reasonable way to do that? I don't like the use of the wildcard import, but it seems like it would be impractically verbose to import everything defined in a (real) module, listing them all.
Third (also highly related) question: avoiding writing the package names in multiple places
Is there a way I can achieve the above without having to put the names of the packages into the source code in them? I ask because I'd like to avoid having to change it in multiple places if I renamed the package (again, simple in this trivial example, can be done by an IDE or script in reality, but would be nice to know how to avoid).
In my_package/__init__.py, use
from . import my_sub_package
etc.
See for example NumPy's __init__.py, which has from . import random, and allows
import numpy as np
np.random.random
Wildcard imports inside a single package tend to be common, provided you have __all__ defined in the modules and subpackages you import from.
Again an example from NumPy's __init__.py, which has several wildcard imports.
Here's part of that __init__.py:
from . import core
from .core import *
from . import compat
from . import lib
from .lib import *
from . import linalg
from . import fft
from . import polynomial
from . import random
from . import ctypeslib
from . import ma
from . import matrixlib as _mat
from .matrixlib import *
from .compat import long
Notice also the two core import lines. Both numpy.core and the core definitions (functions, classes etc) are then available.
When in doubt how to do something, or whether something is good practice, have a look at a few well-known libraries or packages. That can help gaining some valuable insights.
I am having a lot of trouble understanding the python module import system.
I am trying to create a simple folder structure as follows.
SomeModule
__init__.py
AnotherModule
AnotherModule.py
__init__.py
Utils
Utils.py
__init__.py
To use SomeModule i can do:
SomeModule.Foo()
Now inside AnotherModule.py I would like to import my Utils directory.
How come I have to do
import SomeModule.AnotherModule.Utils.Foo
why cannot I just do
import Utils.Foo
To shorten up the actual function name that you'll have to call in your code, you can always do:
from SomeModule.AnotherModule.Utils import *
While this still won't allow you to get away with a shorter import statement at the top of your script, you'll be able to access all of the functions within .Utils just by calling their function name (i.e. foo(x) instead of SomeModule.AnotherModule.Utils.foo(x).
Part of the reason for the lengthy import statement goes to the comment from #wim . Have a look by typing import this in a python interpreter.
put
import sys
import SomeModule.AnotherModule
sys.modules['AnotherModule'] = SomeModule.AnotherModule
in SomeModules __init__.py
i've run through many posts about this, but still doesn't seem to work. The deal is pretty cut. I've the got the following hierarchy.
main.py
DirA/
__init__.py
hello.py
DirB/
__init__.py
foo.py
bla.py
lol.py
The__init__.py at DirA is empty. The respective one at DirB just contains the foo module.
__all__.py = ["foo"]
The main.py has the following code
import DirA
import DirB
hey() #Def written at hello.py
foolish1() #Def written at foo.py
foolish2() #Def written at foo.py
Long story short, I got NameError: name 'foo' is not defined. Any ideas? Thanks in advance.
You only get what you import. Therefore, in you main, you only get DirA and DirB. You would use them in one of those ways:
import DirA
DirA.something_in_init_py()
# Importing hello:
import DirA.hello
DirA.hello.something_in_hello_py()
# Using a named import:
from DirA.hello import something_in_hello_py
something_in_hello_py()
And in DirB, just make the __init__.py empty as well. The only use of __all__ is for when you want to import *, which you don't want because, as they say, explicit is better than implicit.
But in case you are curious, it would work this way:
from DirB import *
something_in_dirb()
By default the import * will import everything it can find that does not start with an underscore. Specifying a __all__ restricts what it imported to the names defined in __all__. See this question for more details.
Edit: about init.
The __init__.py is not really connected to the importing stuff. It is just a special file with the following properties:
Its existence means the directory is a python package, with several modules in it. If it does not exist, python will refuse to import anything from the directory.
It will always be loaded before loading anything else in the directory.
Its content will be available as the package itself.
Just try it put this in DirA/__init__.py:
foo = 42
Now, in your main:
from DirA import foo
print(foo) # 42
It can be useful, because you can import some of your submodules in the __init__.py to hide the inner structure of your package. Suppose you build an application with classes Author, Book and Review. To make it easier to read, you give each class its own file in a package. Now in your main, you have to import the full path:
from myapp.author import Author
from myapp.book import Book
from myapp.review import Review
Clearly not optimal. Now suppose you put those exact lines above in your __init__.py, you may simplify you main like this:
from myapp import Author, Book, Review
Python will load the __init__.py, which will in turn load all submodules and import the classes, making them available on the package. Now your main does not need to know where the classes are actually implemented.
Have you tried something like this:
One way
from DirA import hello
Another way
from DirA.hello import hey
If those don't work then append a new system path
You need to import the function itself:
How to call a function from another file in Python?
In your case:
from DirA import foolish1, foolish2
I have a directory in my Python 3.3 project called /models.
from my main.py I simply do a
from models import *
in my __init__.py:
__all__ = ["Engine","EngineModule","Finding","Mapping","Rule","RuleSet"]
from models.engine import Engine,EngineModule
from models.finding import Finding
from models.mapping import Mapping
from models.rule import Rule
from models.ruleset import RuleSet
This works great from my application.
I have a model that depends on another model, such that in my engine.py I need to import finding.py in engine.py. When I do: from finding import Finding
I get the error No Such Module exists.
How can I import class B from file A in the same module/directory?
Since you are using Python 3, which disallows these relative imports (it can lead to confusion between modules of the same name in different packages).
Use either:
from models import finding
or
import models.finding
or, probably best:
from . import finding # The . means "from the same directory as this module"
Apparently I can do: from .finding import Finding and this works.
And the answer below reflects this as well so I guess this is reasonably correct.
I've fixed up my file naming and moved my tests to a different directory and I am running smoothly now. Thanks!