I have a following project structure:
project
|----app.py
|----package
|---__init__.py
|---module.py
|---module2.py
|---module3.py
|---....
My __init__.py file currently is empty. In module.py I have a definition of a class:
class UsefulClass:
...
And in other modules similar definitions as well. My app.py looks like this:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from package.module import UsefulClass
from package.module2 import UsefulClass2
...
usefulclass = UsefulClass()
usefulclass2 = UsefulClass2()
....
My question is: how can I replace this from package.module... import UsefulClass statements? Even now, I have only 4 modules defined and this imports starting to look ugly. Can I import them in __init__.py file and then just use import package in app.py? I have tried that and it gives me an error.
I am looking for a clean and elegant solution.
In Python 3:
package/__init__.py:
from .foo import bar
package/foo.py:
bar=0
app1.py:
import package
print(package.bar)
app2.py:
from package import bar
print(bar)
Either way, this prints 0, just as you want.
In Python 2, just change from .foo import bar to from foo import bar.
(In fact, the 2.x code often works in Python 3, but it's not correct, and in some edge cases it will fail. For example, if you have a bar.py at the same level as the app, you'll end up with bar being that module, instead of 0.)
In real life, you probably want to specify a __all__ from each package and module that you might ever from foo import … (if for no other reason than to allow to test things at the interactive interpreter with from foo import *).
It sounds like you're saying you already tried this, and got an error. Without knowing exactly what you tried, and what the error was, and which Python version you're using, I have no idea what in particular you might have gotten wrong, but presumably you got something wrong.
The .foo specifies a package-relative import. Saying from .foo import bar means "from the foo module in the same package as me, import bar". If you leave off the dot, you're saying "from the foo module in the standard module path, import bar".
The tutorial section on Intra-package References (and surrounding sections) gives a very brief explanation. The reference docs for import and the import system in general give most of the details, but the original proposal in PEP 328 explains the details, and the rationale behind the design, a lot more simply.
The reason you need to leave off the dot in 2.x is that 2.x didn't have any way to distinguish relative and absolute imports. There's only from foo import bar, which means "from the foo module of the same package as me, or, if there is no such module, the one in the standard module path, import bar".
Related
In a python project I would like to globber imports into a single file called common_imports.py in order to reduce number of import statements in python files.
Instead of writing
file1.py
import foo
import bar
import baz
[...]
file2.py
import foo
import bar
import baz
[...]
I would like to write
file1.py
from common_imports import *
[...]
file2.py
from common_imports import *
[...]
common_imports.py
import foo
import bar
import baz
However, this gives me a lot of pylint false positives.
I can disable pylint warnings in the common_imports.py file by adding a pylint disable comment. I can disable wildcard imports. Unfortunately, I can disable unused imports only globally but not specific for all imports from common_imports.py.
Somebody has an idea howto get pylint on the track?
Summarising my comments above into a proper answer:
TL;DR:
While the reusable code motive is commendable, it's not fit for purpose here. Listen to the linter, and save your hard-earned respect among your colleagues. :-)
Pythonic Viewpoint:
Don't
Why? Python convention, in all its organisational glory and documented structure, states that if you use a library in a module, import it in the module. Plain and simple.
Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants.
-- PEP8 - Imports
At a lower level, the sys.modules dict, which tracks imports, will only import a library if it hasn’t been imported already. So from an efficiency point of view, there is no gain.
Maintainer's Viewpoint:
Don't
Why? If (when) the code is changed / optimised in a module, thus alleviating the need for a specific import … "remind me where I look to find where that library is imported? Oh ya, here. But this other module needs that import, but not this new library I’m using to optimise this code. Where should I import that? Ugh!!!"
You've lost the hard-earned respect of following maintainers.
Real goal: I have a module that is common between two packages (say, bar and bar2). I want to use exact same test files for both cases so I want to change the test imports to not name the package explicitly. (Why? This can be useful during the process of extracting modules from a mega-package into separate packages.)
My idea was to add another module that imports a particular package and provides an "alias" for it. It almost worked, but I got a problem.
Initially I had:
# test.py:
from bar import some_function
If I do nothing magical, there will be two versions of test.py: one with from bar import some_function and another with from new_bar import some_function. I want to avoid this and have the test code files remain the same.
After I added indirection:
#foo.py:
import bar as baz
#test.py:
from .foo import baz # Works!
from .foo.baz import some_function # ModuleNotFoundError: No module named 'cur_dir.foo.baz'; 'cur_dir.foo' is not a package
I can make foo a package:
#foo/__init__.py:
import bar as baz
#test.py:
from .foo import baz # Works!
from .foo.baz import some_function # ModuleNotFoundError: No module named 'cur_dir.foo.baz'
The error changes a bit, but still remains.
I know that I can work around the problem by writing
# test.py:
from .foo import baz
some_function = baz.some_function
Is there any other way? I want my imports to be "normal".
Is there a way to create an "alias" for a package that can be used with the standard import mechanism?
The import statement only looks at actual modules and their paths, not at aliases inside the loaded modules. An actual module alias in Python's module registry, sys.modules, is required.
import sys
import os
sys.modules["os_alias"] = os # alias `os` to `os_alias`
import os_alias # alias import works now
from os_alias import chdir # even as from ... import ...
Once a module alias has been added to sys.modules, it is available for import in the entire application.
Note that module aliasing can lead to subtle bugs when submodules of aliased modules are loaded. In specific, if the submodules are not aliased explicitly, separate versions are created that are not identical. This means that any tests based on object identity, including isinstance(original.submodule.someclass(), alias.submodule.someclass), will fail if the versions are mixed.
To avoid this, you must alias all submodules of any aliased package.
My project structure like this:
/project
main.py
/a_module
__init__.py
/sub_module
__init__.py
some_file.py
main.py
from a_module import main_api
a_module/__init__.py
from sub_module import sub_api
sub_module/__init__.py
from some_file import detail_api
In a_module/__init__.py gives Unable to import 'sub_module' error.
Why I cannot import 'sub_module'?
When I change to the relative path solve the error.
from .sub_module import sub_api
But I don't understand, does __init__.py design for public the API of the module? Why don't treat sub_module as a module instead of a directory? it's such a bad design to me...
__init__.py is executed when you import the package that contains it. But it's not your problem. Your problem is that module imports are always absolute unless explicitly relative. That means that they must chain from some directory in sys.path. By default this includes the working directory, so when you run main.py from within project, it can find a_module, and nothing else.
from sub_module import sub_api
In a_module/__init__.py doesn't work though, because imports are always absolute unless explicitly relative. So that import says "starting from some sys.path root, find a top level package named sub_module and import sub_api from it". Since no such module exists you get an error. from .sub_module import sub_api works because you opted into relative imports, so it doesn't start over from sys.path.
For an example of why you would do this, I'll give you something that broke in our own code back in the Python 2 days before absolute import by default was the law (from __future__ import absolute_import enabled the Py3 behavior, which is how we fixed it, but despite what the docs say, it was never enabled by default in Py2, the only enabled by default behavior was relative imports). Our layout was:
teamnamespace/
module.py
math/
mathrelatedsubmodule.py
othermathsubmodule.py
Now, we innocently thought hey, we'll put all our packages under a single shared top level namespace, and subpackages cover broad categories within them, and since we had a lot of additional utilities for basic mathematics, we put them under teamnamespace.math. Problem was, for the non-math modules, like teamnamespace.module, when they did:
import math # or
from math import ceil
it defaulted to relative lookup, and imported teamnamespace.math as math (a thoroughly useless import, since it was a namespace package only, all the functionality was in the sub-modules), not the built-in math module. In fact, without the Python 3 behavior, there was no reasonable way to get the built-in math module from a module under teamnamespace. Whereas with the Python 3 behavior, you can get either one or both (by aliasing one or the other with as, with no ambiguity:
# Gets built-in
import math
# Gets teamnamespace.math
from . import math
i've run through many posts about this, but still doesn't seem to work. The deal is pretty cut. I've the got the following hierarchy.
main.py
DirA/
__init__.py
hello.py
DirB/
__init__.py
foo.py
bla.py
lol.py
The__init__.py at DirA is empty. The respective one at DirB just contains the foo module.
__all__.py = ["foo"]
The main.py has the following code
import DirA
import DirB
hey() #Def written at hello.py
foolish1() #Def written at foo.py
foolish2() #Def written at foo.py
Long story short, I got NameError: name 'foo' is not defined. Any ideas? Thanks in advance.
You only get what you import. Therefore, in you main, you only get DirA and DirB. You would use them in one of those ways:
import DirA
DirA.something_in_init_py()
# Importing hello:
import DirA.hello
DirA.hello.something_in_hello_py()
# Using a named import:
from DirA.hello import something_in_hello_py
something_in_hello_py()
And in DirB, just make the __init__.py empty as well. The only use of __all__ is for when you want to import *, which you don't want because, as they say, explicit is better than implicit.
But in case you are curious, it would work this way:
from DirB import *
something_in_dirb()
By default the import * will import everything it can find that does not start with an underscore. Specifying a __all__ restricts what it imported to the names defined in __all__. See this question for more details.
Edit: about init.
The __init__.py is not really connected to the importing stuff. It is just a special file with the following properties:
Its existence means the directory is a python package, with several modules in it. If it does not exist, python will refuse to import anything from the directory.
It will always be loaded before loading anything else in the directory.
Its content will be available as the package itself.
Just try it put this in DirA/__init__.py:
foo = 42
Now, in your main:
from DirA import foo
print(foo) # 42
It can be useful, because you can import some of your submodules in the __init__.py to hide the inner structure of your package. Suppose you build an application with classes Author, Book and Review. To make it easier to read, you give each class its own file in a package. Now in your main, you have to import the full path:
from myapp.author import Author
from myapp.book import Book
from myapp.review import Review
Clearly not optimal. Now suppose you put those exact lines above in your __init__.py, you may simplify you main like this:
from myapp import Author, Book, Review
Python will load the __init__.py, which will in turn load all submodules and import the classes, making them available on the package. Now your main does not need to know where the classes are actually implemented.
Have you tried something like this:
One way
from DirA import hello
Another way
from DirA.hello import hey
If those don't work then append a new system path
You need to import the function itself:
How to call a function from another file in Python?
In your case:
from DirA import foolish1, foolish2
I was trying out one of the Python standard library modules, let's call it foo.bar.baz.
So I wrote a little script starting with
import foo.bar.baz
and saved it as foo.py.
When I executed the script I got an ImportError. It took me a while (I'm still learning Python), but I finally realized the problem was how I named the script. Once I renamed foo.py to something else, the problem went away.
So I understand that the import foo statement will look for the script foo.py before looking for the standard library foo, but it's not clear to me what it was looking for when I said import foo.bar.baz. Is there some way that foo.py could have the content for that statement to make sense? And if not, why didn't the Python interpreter move on to look for a directory hierarchy like foo/bar with the appropriate __init__.py's?.
An import statement like import foo.bar.baz first imports foo, then asks it for bar, and then asks foo.bar for baz. Whether foo will, once imported, be able to satisfy the request for bar or bar.baz is unimportant to the import of foo. It's just a module. There is only one foo module. Both import foo and import foo.bar.baz will find the same module -- just like any other way of importing the foo module.
There is actually a way for foo to be a single module, rather than a package, and still be able to satisfy a statement like import foo.bar.baz: it can add "foo.bar" and "foo.bar.baz" to the sys.modules dict. This is exactly what the os module does with os.path: it imports the right "path" module for the platform (posixpath, ntpath, os2path, etc), and assigns it to the path attribute. Then it does sys.modules["os.path"] = path to make that module importable as os.path, so a statement like import os.path works. There isn't really a reason to do this -- os.path is available without importing it as well -- but it's possible.