I have a simple python package that I am able to use and import, but that is behaving unexpectedly. The package structure is
package_name
__init__.py
modfile1.py
modfile2.py
in __init__.py I have
import package_name.modfile1
import package_name.modfile2
If I open a python interpreter and
import package_name
I then see the following autocomplete choices (via jedi)
package_name.modfile1 package_name.modfile2 package_name.package_name
If I use package_name.package_name this can continue recursively. Why is the package listed as a member of itself?
This is normal.
package_name.modfile1 would even exist if __init__.py were empty. It's just a module name, that you could import (with import package_name.modfile1)
same for package_name.modfile2
in __init__.py you import package_name.modfile1
So if you imported package_name, then the tab completion will offer you the symbols from __init__.py within package_name
Thus you get package_name.package_name.modfile1 and package_name.package_name.modfile2
Just comment the two imports from __init__.py and these symbols will disappear.
You could change the lines
import package_name.modfile1
import package_name.modfile2
to
import package_name.modfile1 as modfile1
import package_name.modfile2 as modfile2
To have a more consistent naming.
If you keep the __init__.py file empty
You have to use either
import package_name.modfile1
or
from package_name import modfile1
If you have it populated with my last suggestion you can just use.
import package_name
and then directly access package_name.modfile1
Please note, that adding imports to the __init__.py file has one small side effect.
whenever you package_name modified1 and modified2 will always be imported into RAM. Even if you imported package_name.modfile1 then package_name.modfile2 would also be loaded into RAM.
Therefore I personally keep the init files almost always empty and import explicitly what I want.
exceptions being, where I know, that modfile1 and modfile2 will always be used together.
Just do whatever fits best your purpose.
Related
I have a custom python module I'm working on and am confused how I should import modules into other modules. I want to use bits and pieces of some modules within others and keep getting an error: ImportError: cannot import name NameOfModule
I'm assuming there's some sort of circular reference that's causing the issue but I'm not sure if I need to add something to __init__.py, or if there's a specific way of importing the modules into each other, or if I should change my folder structure?
If I want to be able to use some function from mod1.py within mod2.py how should I go about setting up the import statements?
My current folder structure is:
FolderName
-__init__.py
-mod1.py
-mod2.py
-mod3.py
-mod4.py
Sample code:
__init__.py is empty
mod1.py: from . import mod2
mod2.py: from . import mod1
You should be using relative imports for files within the current module, like so:
from . import mod2
Or:
from .mod2 import foo
And unless you have a VERY good reason, you should be using Python 3.
It seems that I am still missing some basics of python. I was trying to understand submodules importing, which I feel I have not understood yet. But I have also stumbled upon something new.
I am having following two packages in two different PyDev projects:
package1
|
+--mod1.py
|
+--mod2.py
package2
|
+--__init__.py
|
+--modx.py
|
+--mody.py
In mod1, I can do import mod2. But in __init__ and modx, I cannot do import mody (Eclipse says "unresolved imports"). In __init__, I can do import .mody or from .mody import vary. In modx, I cannot do import .mody. (In fact I never saw use of . in import statement as prefix to the module. Earlier I only came across import mod and from mod import var, but never saw import .mod and from .mod import var.) Why this might be happening? I must be unaware some context which is causing this behaviour. But then I dont know what is it?
PS: I am using Python 3.4
There is a subtle different between how Python is treating both of those packages.
package1 is treated as a namespace package in that it does not contain an __init__.py file.
package2 is treated as a regular package in that it does contain an __init__.py file.
So I'll give a quick breakdown of why each step is happening:
In mod1, I can do import mod2.
This is happening due to how namespace packages are handled using absolute imports. You're most likely executing python mod1.py from the directory in which the file is stored, right (in my attempt to re-create your folder structure and test it myself locally, I did the same)? So package1 becomes your current working directory with your mod2 file being at the root of that directory.
With namespace packages, Python will default look to sys.path in an attempt to find the imports you have requested. Since your current working directory is automatically added to and included in sys.path, Python can successfully find your import mod2 request without any difficulty.
Thanks to ShadowRanger for correcting my initial response to this where I had misunderstood exactly how Python is including the current working directory in its search.
In init, I can do import .mody or from .mody import vary.
This is because Python is treating this as a regular package. The name of your regular package in this case is package2. When you use the . notation, you are asking Python to start searching for the import from the current package (which in this case is your parent package2). So you have to use import .mody to find the mody package within the current package.
If you used .. then it would import from the parent of the current package and so on.
The dot notation is useful as you are explicitly declaring that you wish to search from the current package only - so if there was another package2 package on your PYTHONPATH, Python would know which one to choose.
But in init and modx, I cannot do import mody (Eclipse says "unresolved imports").
With __init__.py this is because you have not used the dot notation and have not told Python that you wish to search for these modules in the current package. So it's looking to the Python standard library and to your PYTHONPATH for these packages and not finding them (hence your error in Eclipse). By using the dot notation, you are stating that you wish to include the current package in the search and, thus, Python will then be able to locate those files.
Using the dot notation like this, to import via from . import mody, is to use a relative import.
With modx you also have to use a relative import (see next section).
In modx, I cannot do import .mody. Why this might be happening?
This is because you're not using a relative / absolute import. You'll be using a relative import in this case. A relative import is the from . import mody syntax you've seen already. Using a relative or absolute import behaviour is default in Python.
It is now the default behaviour as, with the old Python import behaviour, suppose Python's own standard library had a package called mody. When you'd use import mody it would previously have imported mody from your package and not the standard library. This wasn't always desirable. What if you specifically wanted the standard library version?
So now your imports must be made using from . import mody or from .mody import vary syntax so as the import is very clear. If you use import and not the from... syntax, Python will assume it's a standard library or PYTHONPATH import.
By the way, sources for a lot of the above information came from the following sites:
https://docs.python.org/3/reference/import.html
https://docs.python.org/2.5/whatsnew/pep-328.html
Python modules are optional "additions" to Python that can be imported using the import command like so:
import package1
package1.mod1 # Can be accessed using this
To import individual parts of the package, use from like so:
from package1 import mod1
mod1 # Can be accessed using this
If you want to import every part of a module and use it without package., use:
from package1 i
I am trying to import a module from a python file that is in a sibling folder. I read several similar questions here and tried to apply solutions listed there, but I wasn't able to solve the problem.
The structure is as follows:
parentfolder/gfolder/codefolder/fileA.py
parentfolder/gfolder/utilfolder/util.py
gfolder, codefolder and utilfolder all have an __init__.py.
I'm trying to do in fileA.py:
import gfolder.utilfolder.util as util
I also tried adding before the import statement:
sys.path.append(".../parentfolder/")
And that didn't work either:
import gfolder.utilfolder.util as util
ModuleNotFoundError: No module named 'gfolder'
The solution in a similar question says to include __init.py__ in the directories, which I already have.
EDIT:
Now both sys.append and sys.insert work and the problem was that I included a slash at the end of the path. When I took it out, everything worked.
First of all, let me describe you the differences between a Python module & a Python package so that both of us are on the same page. ✌
A module is a single .py file (or files) that are imported under one import and used. ✔
import aModuleName
# Here 'aModuleName' is just a regular .py file.
Whereas, a package is a collection of modules in directories that give a package hierarchy. A package contains a distinct __init__.py file. ✔
from aPackageName import aModuleName
# Here 'aPackageName` is a folder with a `__init__.py` file
# and 'aModuleName', which is just a regular .py file.
Therefore, when we have a project directory named proj-dir of the following structure ⤵
proj-dir
--|--__init__.py
--package1
--|--__init__.py
--|--module1.py
--package2
--|--__init__.py
--|--module2.py
🔎 Notice that I've also added an empty __init__.py into the proj-dir itself which makes it a package too.
👍 Now, if you want to import any python object from module2 of package2 into module1 of package1, then the import statement in the file module1.py would be
from proj-dir.package2.module2 import object2
# if you were to import the entire module2 then,
from proj-dir.package2 import module2
I hope this simple explanation clarifies your doubts on Python imports' mechanism. 😊
As Andrew Cox answerd int the following thread Import a module from a relative path
You can add the subdirectory to your Python path so that it imports as a normal script
import sys
sys.path.insert(0, <path to gfolder>)
import gfolder
you can also add the directory to the PATH var of the Linux system (I use it while I'm working on a project, at the end i modified the PATH to it's origin value)
if you maintain the following structre than it is working out side the box
parentfolder/gfolder/codefolder/fileA.py
parentfolder/gfolder/utilfolder/util.py
parentfolder/gfolder/main.py
run main.py
I have this package structure:
widget/
__init__.py
core.py
extension.py
__init__.py is empty.
core.py contains this:
import widget.extension as extension
It works, but it feels like I'm side-stepping the package and just importing it from the global path (i.e. climbing out of it only to look back into it). If I just import extension in core.py it doesn't work. Does this matter? Am I wrong in the first place? Should I instead be pulling both of these modules into __init__.py?
I'm presuming you are using Python 3; in Python 2, import extension would work as Python 2 will first look for a local, relative import before looking for a global, absolute reference.
You have two more options:
from widget import extension
and
from . import extension
The latter imports relative to the current package, which allows you to rename your widget package without having to update all your internal imports. What you use is a matter of style and taste.
I'd like to have modules/packages structure like following:
/__init__.py
/mymodule.py
/mymodule/
/mymodule/__init__.py
/mymodule/submodule.py
And then use modules like:
import mymodule
import mymodule.submodule
But it seems like file "mymodule.py" conflicts with "mymodule" directory.
What's the correct naming convention here?
If you want to make a package, you have to understand how Python translates filenames to module names.
The file mymodule.py will be available as the mymodule, assuming the interpreter finds it in a directory in the Python search path. If you're on a case-insensitive filesystem, it might also be importable with different capitalization (but you should avoid using such system-dependent behavior).
A package is a directory with an __init__.py file in it. There's been some movement recently to allow packages without those files, but I'm going to ignore that less-common case for this answer. A package becomes a module inside Python, with its code coming from the __init__.py file. So the file mypackage/__init__.py can be imported as mypackage.
There's no meaning to an __init__.py file directly in the Python search path (well, I suppose you could import it an an __init__ module, but this is probably a bad idea).
So, for your situation, here's the appropriate filesystem layout:
toplevel/
mymodule/
__init__.py # put code here for mymodule
submodule.py # put code here for mymodule.submodule
Only the toplevel folder should be in the Python search path.
You are dealing with a package. The package structure you should have is:
/some-parent-directory # This needs to be on sys.path
/mymodule # This is not really a module - it's a package
__init__.py # import mymodule
# init is loaded when you `import mymodule` or anything below it
some.py # import mymodule.some
implementation.py # import mymodule.implementation
files.py # import mymodule.files
/submodule
__init__.py # import mymodule.submodule
# init is loaded when you `import mymodule.submodule` or anything below it
submodule_impl.py # import mymodule.submodule.submodule_impl
goes.py # import mymodule.submodule.goes
here.py # import mymodule.submodule.here
As long as the parent directory is on sys.path you will be able to call import mymodule or from mymodule.submodule import something without issue.
If you want something to be available from the root level of a package (i. e. from mymodule import SomeItem or from a sub-package from mymodule.submodule import AnotherItem) then you can import it into the appropriate __init__.py file.
So, for example, let's say you wanted the class CustomClass defined in the submodule_impl.py module to be importable directly from submodule. Your submodule/__init__.py would have to contain the following:
from .submodule_impl import CustomClass
Then you would be able to import CustomClass directly from submodule (i. e. from mymodule.submodule import CustomClass)