I have this package structure:
widget/
__init__.py
core.py
extension.py
__init__.py is empty.
core.py contains this:
import widget.extension as extension
It works, but it feels like I'm side-stepping the package and just importing it from the global path (i.e. climbing out of it only to look back into it). If I just import extension in core.py it doesn't work. Does this matter? Am I wrong in the first place? Should I instead be pulling both of these modules into __init__.py?
I'm presuming you are using Python 3; in Python 2, import extension would work as Python 2 will first look for a local, relative import before looking for a global, absolute reference.
You have two more options:
from widget import extension
and
from . import extension
The latter imports relative to the current package, which allows you to rename your widget package without having to update all your internal imports. What you use is a matter of style and taste.
Related
I've tried to create a new module in Python. It's git link : https://github.com/Sanmitha-Sadhishkumar/strman
After uploading and installing that using pip, I found that I could access that module as
import strman.strman as s
s.func_name
What are the changes to be made to access that as
import strman
strman.func_name
In your __init__.py file, you want to use a relative import.
from .strman import *
You have a strman package (the outer directory) and within it a strman module (the strman.py file). That's a perfectly common pattern. But without the relative import your __init__.py wasn't importing from deep enough in the hierarchy.
More generally, whenever you import from a sibling module within a project, you almost always should use a relative import, because it's explicit and avoids various complications, such as the example in your case.
You have to move the files into the main directory. Your directory will look like this
\myProject
- init.py
- strman.py
- main.py # <-- this would be the file your programming is
Read more about modules in the docs
I want to ask you something that came to my mind doing some stuff.
I have the following structure:
src
- __init__.py
- class1.py
+ folder2
- __init__.py
- class2.py
I class2.py I want to import class1 to use it. Obviously, I cannot use
from src.class1 import Class1
cause it will produce an error. A workaround that works to me is to define the following in the __init__.py inside folder2:
import sys
sys.path.append('src')
My question is if this option is valid and a good idea to use or maybe there are better solutions.
Another question. Imagine that the project structure is:
src
- __init__.py
- class1.py
+ folder2
- __init__.py
- class2.py
+ errorsFolder
- __init__.py
- errors.py
In class1:
from errorsFolder.errors import Errors
this works fine. But if I try to do in class2 which is at the same level than errorsFolder:
from src.errorsFolder.errors import Errors
It fails (ImportError: No module named src.errorsFolder.errors)
Thank you in advance!
Despite the fact that it's slightly shocking to have to import a "parent" module in your package, your workaround depends on the current directory you're running your application, which is bad.
import sys
sys.path.append('src')
should be
import sys,os
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)),os.pardir))
to add the parent directory of the directory of your current module, regardless of the current directory you're running your application from (your module may be imported by several applications, which don't all require to be run in the same directory)
One correct way to solve this is to set the environment variable PYTHONPATH to the path which contains src. Then import src.class1 will always work, regardless of which directory you start in.
from ..class1 import Class1 should work (at least it does here in a similar layout, using python 2.7.x).
As a general rule: messing with sys.path is usually a very bad idea, specially if this allows a same module to be imported from two different paths (which would be the case with your files layout).
Also, you may want to think twice about your current layout. Python is not Java and doesn't require (nor even encourage) a "one class per module" approach. If both classes need to work together, they might be better in a same module, or at least in modules at the same level in the package tree (note that you can use the top-level package's __init__ as a facade to provide direct access to objects defined in submodules / subpackages). NB : I'm not saying that your current layout is necessarily wrong, just that it might not be the simplest one.
There is also a solution that does not involve the use of the environment variable PYTHONPATH.
In src/ create setup.py with these contents:
from setuptools import setup
setup()
Now you can do pip install -e /path/to/src and import to your hearts content.
-e, --editable <path/url> Install a project in editable mode (i.e. setuptools "develop mode") from a local project path or a VCS url.
No, Its not good. Python takes modules in two ways:
Python looks for its modules and packages in $PYTHONPATH.
Refer: https://docs.python.org/2/using/cmdline.html#envvar-PYTHONPATH
To find out what is included in $PYTHONPATH, run the following code in python (3):
import sys
print(sys.path)
all folder containing init.py are marked as python package.(if they are subdirectories under PYTHONPATH)
By help of these two ways you can full-fill the need to create a python project.
It seems that I am still missing some basics of python. I was trying to understand submodules importing, which I feel I have not understood yet. But I have also stumbled upon something new.
I am having following two packages in two different PyDev projects:
package1
|
+--mod1.py
|
+--mod2.py
package2
|
+--__init__.py
|
+--modx.py
|
+--mody.py
In mod1, I can do import mod2. But in __init__ and modx, I cannot do import mody (Eclipse says "unresolved imports"). In __init__, I can do import .mody or from .mody import vary. In modx, I cannot do import .mody. (In fact I never saw use of . in import statement as prefix to the module. Earlier I only came across import mod and from mod import var, but never saw import .mod and from .mod import var.) Why this might be happening? I must be unaware some context which is causing this behaviour. But then I dont know what is it?
PS: I am using Python 3.4
There is a subtle different between how Python is treating both of those packages.
package1 is treated as a namespace package in that it does not contain an __init__.py file.
package2 is treated as a regular package in that it does contain an __init__.py file.
So I'll give a quick breakdown of why each step is happening:
In mod1, I can do import mod2.
This is happening due to how namespace packages are handled using absolute imports. You're most likely executing python mod1.py from the directory in which the file is stored, right (in my attempt to re-create your folder structure and test it myself locally, I did the same)? So package1 becomes your current working directory with your mod2 file being at the root of that directory.
With namespace packages, Python will default look to sys.path in an attempt to find the imports you have requested. Since your current working directory is automatically added to and included in sys.path, Python can successfully find your import mod2 request without any difficulty.
Thanks to ShadowRanger for correcting my initial response to this where I had misunderstood exactly how Python is including the current working directory in its search.
In init, I can do import .mody or from .mody import vary.
This is because Python is treating this as a regular package. The name of your regular package in this case is package2. When you use the . notation, you are asking Python to start searching for the import from the current package (which in this case is your parent package2). So you have to use import .mody to find the mody package within the current package.
If you used .. then it would import from the parent of the current package and so on.
The dot notation is useful as you are explicitly declaring that you wish to search from the current package only - so if there was another package2 package on your PYTHONPATH, Python would know which one to choose.
But in init and modx, I cannot do import mody (Eclipse says "unresolved imports").
With __init__.py this is because you have not used the dot notation and have not told Python that you wish to search for these modules in the current package. So it's looking to the Python standard library and to your PYTHONPATH for these packages and not finding them (hence your error in Eclipse). By using the dot notation, you are stating that you wish to include the current package in the search and, thus, Python will then be able to locate those files.
Using the dot notation like this, to import via from . import mody, is to use a relative import.
With modx you also have to use a relative import (see next section).
In modx, I cannot do import .mody. Why this might be happening?
This is because you're not using a relative / absolute import. You'll be using a relative import in this case. A relative import is the from . import mody syntax you've seen already. Using a relative or absolute import behaviour is default in Python.
It is now the default behaviour as, with the old Python import behaviour, suppose Python's own standard library had a package called mody. When you'd use import mody it would previously have imported mody from your package and not the standard library. This wasn't always desirable. What if you specifically wanted the standard library version?
So now your imports must be made using from . import mody or from .mody import vary syntax so as the import is very clear. If you use import and not the from... syntax, Python will assume it's a standard library or PYTHONPATH import.
By the way, sources for a lot of the above information came from the following sites:
https://docs.python.org/3/reference/import.html
https://docs.python.org/2.5/whatsnew/pep-328.html
Python modules are optional "additions" to Python that can be imported using the import command like so:
import package1
package1.mod1 # Can be accessed using this
To import individual parts of the package, use from like so:
from package1 import mod1
mod1 # Can be accessed using this
If you want to import every part of a module and use it without package., use:
from package1 i
My Directory structure in /VM/repo/project is:
__init__.py
scripts/
getSomething.py
__init__.py
classes/
project.py
db.py
__init__.py
getSomething.py
from ..classes import project
from ..classes import db
project.py
class PROJECT:
def __init__(self):
stuff
db.py
class DB:
def __init__(self):
stuff
When I try to run
python getSomething.py
I get the error
Traceback (most recent call last):
File "scripts/getSomething.py", line 4, in < module >
from ..classes import project
ValueError: Attempted relative import in non-package
What am I missing here?
As stated in the error, you're running getSomething as a main module. But you can't do package relative imports when you aren't in a package. The main module is never in a package. So, if you were to import getSomething as part of a package...:
# /VM/repo/main.py
from project.scripts import getSomething
Then you would not have the import errors.
Perhaps it is helpful to have a quick discussion on python modules and packages. In general, a file that contains python source code and has the .py extension is a module. Typically that module's name is the name of the file (sans extension), but if you run it directly, that module's name is '__main__'. So far, this is all well known and documented. To import a module you just do import module or import package.module and so on. This last import statement refers to something else (a "package") which we'll talk about now...
Packages are directories that you can import. Many directories can't be imported (e.g. maybe they have no python source files -- modules -- in them). So to resolve this ambiguity, there is also the requirement that the directory has an __init__.py file. When you import a directory on your filesystem, python actually imports the __init__.py module and creates the associated package from the things in __init__.py (if there are any).
Putting all of this together shows why executing a file inside a directory that has an __init__.py is not enough for python to consider the module to be part of a package. First, the name of the module is __main__, not package.filename_sans_extension. Second, the creation of a package depends not just on the filesystem structure, but on whether the directory (and therefore __init__.py was actually imported).
You might be asking yourself "Why did they design it this way?" Indeed, I've asked myself that same question on occasion. I think that the reason is because the language designers want certain guarantees to be in place for a package. A package should be a unit of things which are designed to work together for a specific purpose. It shouldn't be a silo of scripts that by virtue of a few haphazard __init__.py files gain the ability to walk around the filesystem to find the other modules that they need. If something is designed to be run as a main module, then it probably shouldn't be part of the package. Rather it should be a separate module that imports the package and relies on it.
First off all: I'm sorry, I know there has been lots of question about relative imports, but I just didn't find a solution. If possible I would like to use the following directory layout:
myClass/
__init__.py
test/
demo.py
benchmark.py
specs.py
src/
__init__.py
myClass.py
Now my questions are:
How do the test files from within the package properly import myClass.py?
How would you import the package from outside, assuming you take myClass as submodule in libs/myClass or include/myClass?
So far I couldn't find an elegant solution for this. From what I understand Guido's Decision it should be possible to do from ..src import myClass but this will error:
ValueError: Attempted relative import in non-package
Which looks as it doesn't treat myClass as packages. Reading the docs:
The __init__.py files are required to make Python treat the directories as containing packages;
It seems I'm missing something that specifies where the scripts of the package are, should I use .pth ?
ValueError: Attempted relative import in non-package
Means you attempt to use relative import in the module which is not package. Its problem with the file which has this from ... import statement, and not the file which you are trying to import.
So if you are doing relative imports in your tests, for example, you should make your tests to be part of your package. This means
Adding __init__.py to test/
Running them from some outside script, like nosetests
If you run something as python myClass/test/demo.py, relative imports will not work too since you are running demo module not as package. Relative imports require that the module which uses them is being imported itself either as package module, from myClass.test.demo import blabla, or with relative import.
After hours of searching last night I found the answer to relative imports in python!! Or an easy solution at the very least. The best way to fix this is to have the modules called from another module. So say you want demo.py to import myClass.py. In the myClass folder at the root of the sub-packages they need to have a file that calls the other two. From what I gather the working directory is always considered __main__ so if you test the import from demo.py with the demo.py script, you will receive that error. To illustrate:
Folder hierarchy:
myClass/
main.py #arbitrary name, can be anything
test/
__init__.py
demo.py
src/
__init__.py
myClass.py
myClass.py:
def randomMaths(x):
a = x * 2
y = x * a
return y
demo.py:
from ..src import myClass
def printer():
print(myClass.randomMaths(42))
main.py:
import test.demo
demo.printer()
If you run demo.py in the interpreter, you will generate an error, but running main.py will not. It's a little convoluted, but it works :D
Intra-package-references describes how to myClass from test/*. To import the package from outside, you should add its path to PYTHONPATH environment variable before running the importer application, or to sys.path list in the code before importing it.
Why from ..src import myClass fails: probably, src is not a python package, you cannot import from there. You should add it to python path as described above.