Can't import package from parallel directory - python

I have 3 folders in my project: scripts , src, tests. In scripts, I have a script to populate my local db with test data (populate_db.py). In the tests folder, I have a file called generators.py, which I'm trying to import into populate_db.py , but I'm not having any luck. I have this:
code_path = Path("../")
sys.path.append(code_path / "tests")
from generators import UserFactory
but I get the error ModuleNotFoundError: No module named 'generators'. Strange thing, is I do
sys.path.append(code_path / "src")
from users.models import User
and that imports fine. Not sure what I've done differently/wrong. I also tried a relative import: from ..generators import UserFactory, but got attempted relative import with no known parent package. I do have __init__.py in the parent and tests directories (though there are no python files in the parent, only in the 3 children).

Try this:
from .tests.generators import UserFactory

I found that the correct way to add a path to sys.path when using the pathlib.Path is to convert it to a string:
sys.path.append(str(code_path / "tests"))

Related

python file needs to imports a class from another python file that is in the parent folder

i'm struggling with something that feels like it should be simple.
my current dir looks like this:
root/
└─ __init__.py (tried with it and without)
└─ file_with_class.py
└─ tests_folder/
└─ __init__.py (tried with it and without)
└─ unittest_for_class.py
unittest_for_class.py needs to import the class from file_with_class to test it, i tried to import it in various ways i found online but i just keep getting errors like:
(class name is same as file name lets say its called file_with_class)
File "tests_folder/unittest_for_class.py", line 3, in <module>
from ..file_with_class import file_with_class
ValueError: Attempted relative import in non-package
File "tests_folder/unittest_for_class.py", line 3, in <module>
from file_with_class import file_with_class
ImportError: No module named file_with_class
and others..
what is the correct way to import a class from a .py file that is in the parent folder ?
As a short explanation
import * from ..parent works if your program started at the parent level.
You import submodules which can have cross relations to other submodules or files in the package -> They are only relative inside a package not the os structure.
Option 1
you actually enter via a script in your parent folder and import your mentioned file as a submodule. Nicest, cleanest and intended way, but then your file is no standalone.
Option 2 - Add the parent dictionary to your path
sys.path.append('/path/to/parent')
import parent
This is a little bit dirty as you now have an extra path for your imports but still one of most easiest ones without much trickery.
Further Options and theory
There are quite a few posts here covering this topic relative imports covers quite a few good definitions and concepts in the answers.
Option 3 - Deprecated and not future proof importlib.find_loader
import os
import importlib
current = os.getcwd() # for rollback
os.chdir("..") # change to arbitrary path
loader = importlib.find_loader("parent") # load filename
assert loader
parent = loader.load_module() # now this is your module
assert parent
os.chdir(current) # change back to working dictionary
(Half of an) Option 4
When working with an IDE this might work, Spyder allows the following code. Standard python console does NOT.
import os
current = os.getcwd()
os.chdir("..")
import parent
os.chdir(current)
Following up on #Daraan's answer:
You import submodules which can have cross relations to other submodules or files in the package -> They are only relative inside a package not the os structure.
I've written an experimental, new import library: ultraimport which allows to do just that, relative imports from anywhere in the file system. It will give you more control over your imports.
You could then write in your unittest_for_class.py:
import ultraimport
MyClass = ultraimport("__dir__/../file_with_class.py", "MyClass")
# or to import the whole module
file_with_class = ultraimport("__dir__/../file_with_class.py")
The advantage is that this will always work, independent of sys.path, no matter how you run your script and all the other things that were mentioned.
You can add the parent folder to the search path with sys.path.append() like so:
import sys
sys.path.append('/path/to/parentdir')
from file_with_class import file_with_class
...
See also the tutorial for how Python modules and packages are handled
Just keep from file_with_class import file_with_class. Then run python -m test_folder.unittest_for_class. This supports running the script as if it is a module.

Python ImportError. How can I import modules among packages?

I was working on a project with this type of folder tree:
-main.py
-Message_handlers
-__init__.py
-On_message.py
-Other_modules.py
-Exceptions_handler
-__init__.py
-Run_as_mainException.py
Message_handlers and Exceptions_handler are two packages, now my problem is that from inside the module On_message.py I can't import the class Run_as_mainException (inside of the module Run_as_mainException.py) using this code
# This is On_message.py
from ..Exceptions_handler.Run_as_mainException import Run_as_main_Exception
This line gives the error: ImportError: attempted relative import with no known parent package
p.s. Every file has a class inside with the same name of the file, example:
# This is Run_as_mainExample.py
class Run_as_mainExample:
def __init__(self):
# rest of the code
Can anyone help me please?
You have to assume that you're running everything from the level where main.py is located. This means, imagine that you execute python main.py, and you want to import Run_as_main_Exception from main.py, what should you do?
from Exceptions_handler.Run_as_mainException import Run_as_main_Exception
Try using that line in your On_message.py file, and you shouldn't have any problem when running your script from that exact location (remember, the same level where main.py is located)

Python import from submodule of package exported by another file

I have imports.py containing:
import os as exported_os
and foo.py containing:
from imports import exported_os
print(exported_os.path.devnull) # works
from imports.exported_os.path import devnull # doesn't
Is there a way to make the second import work? I tried adding __path__ to imports.py and fiddling with it but couldn't get anything.
Actual usecase: os is some_library_version_n and exported_os is some_library_version (I'm trying to avoid having many instances of some_library_version_n across different files).
One approach
Directory structure:
__init__.py
foo.py
imports/
├ __init__.py
└ exported_os/
├ __init__.py
└ path.py
imports/exported_os/__init__.py:
from . import path
from os import * # not necessary for the question
# but it makes `exported_os` more like `os`
# e.g., `exported_os.listdir` will be callable
imports/exported_os/path.py:
from os.path import *
In this way, you can use exported_os as if it is os with a submodule path. Different with import, from takes modules and classes.
Another approach
imports.py:
import os
import sys
ms = []
for m in sys.modules:
if m.startswith('os'):
ms.append(m)
for m in ms:
sys.modules['imports.exported_os' + m[2:]] = sys.modules[m]
Or, by explicitly extending sys.modules you can use exported_os as if os with its submodules.
Why you cannot simply change the name of os
If you open .../lib/python3.9/os.py you can find the following line:
sys.modules['os.path'] = path
So even if you copy .../lib/python3.9/os.py to .../lib/python3.9/exported_os.py, the following does not work:
from exported_os.path import devnull
But if you change the line sys.modules['os.path'] to sys.modules['exported_os.path'] it works.
The error you are getting would be something like:
ModuleNotFoundError: No module named 'imports.exported_os'; 'imports' is not a package
When you code from imports import exported_os, then imports can refer to a module implemented by file imports.py. But when the name imports is part of a hierarchy as in from imports.exported_os.path import devnull, then imports must be a package implemented as a directory in a directory structure such as the following:
__init__.py
imports
__init__.py
exported_os
__init__.py
path.py
where directory containing the top-most __init__.py must be in the sys.path search path.
So, unless you want to rearrange your directory structure to something like the above, the syntax (and selective importing) you want to use is really not available to you without getting into the internals of Python's module system.
Although this is not a solution to your wanting to be able to do an from ... import ... due to your unique versioning issue, let me suggest an alternate method of doing this versioning. In your situation you could do the following. Create a package, my_imports (give it any name you want):
my_imports
__init__.py
The contents of __init__.py is:
import some_library_version_n as some_library_version
Then in foo.py and in any other file that needs this module:
from my_imports import *
This is another method of putting the versioning dependency in one file. If you had other similar dependencies, you would, of course, add them to this file and you could import from my_imports just the names you are interested. You still have the issue that you are importing the entire module some_library_version.
However, we could take this one step further. Suppose the various versions of your library had components A, B and C that you might be interested in importing individually or all together. Then you could do the following. Let's instead name the package some_library_version, since it will only be dealing with this one versioning issue:
some_library_version/init.py
from some_library_version_n import A
from some_library_version_n import B
from some_library_version_n import C
foo.py
from some_library_version import A, C
Most of the answers added are accured but dont add context of why works in that way, GyuHyeon explains it well but it just resumes it into import is a fancy file include system that checks into the std libraries, then the installed ones and finaly into the context provided, context is added on where is called and the from given.
This example gives the various method of importing a specific function dirname(), the lib os is just a folder, if you imagine that os is in your working folder the import path whoud be the same or './os' and beause python, everything is a class, so import will search for the .os/__init__.py so if your library dont have one importing the subdirs it will have no efect.
from os.path import dirname as my_fucntion # (A_2)
from os import path as my_lib # (B_2)
from os.path import dirname # (C_1)
from os import path # (B_1)
import os # (A_1)
if __name__ == '__main__':
print(os.path.dirname(__file__)) # (A_1)
print(path.dirname(__file__)) # (B_1)
print(dirname(__file__)) # (C_1)
print(my_lib.dirname(__file__)) # (B_2)
print(my_fucntion(__file__)) # (A_2)
You could try to go with sys.path.append(...), e.g.:
import sys
sys.path.append(<your path to devnull goes here>)
Maybe not so nice, but you could use the pathlib library and path joins to construct the path (but some assumptions on file structure unfortunately have to be made if you the files are in separate folder structures):
from pathlib import Path
from os import path
sys.path.append(path.join(str(Path(__file__).parents[<integer that tells how many folders to go up>]), <path to devnull>))
Instead of pathlib you could also use the dirname function from os.path.
After appending to the system path, you could just use:
import devnull

gRPC generated python script fails to find a local module in the same directory? [duplicate]

In python 2 I can create a module like this:
parent
->module
->__init__.py (init calls 'from file import ClassName')
file.py
->class ClassName(obj)
And this works. In python 3 I can do the same thing from the command interpreter and it works (edit: This worked because I was in the same directory running the interpreter). However if I create __ init __.py and do the same thing like this:
"""__init__.py"""
from file import ClassName
"""file.py"""
class ClassName(object): ...etc etc
I get ImportError: cannot import name 'ClassName', it doesn't see 'file' at all. It will do this as soon as I import the module even though I can import everything by referencing it directly (which I don't want to do as it's completely inconsistent with the rest of our codebase). What gives?
In python 3 all imports are absolute unless a relative path is given to perform the import from. You will either need to use an absolute or relative import.
Absolute import:
from parent.file import ClassName
Relative import:
from . file import ClassName
# look for the module file in same directory as the current module
Try import it this way:
from .file import ClassName
See here more info on "Guido's decision" on imports in python 3 and complete example on how to import in python 3.

Python strange import behavior: import x.y works, from x import y doesn't

I am having a simple directory structure, like that:
MyProject
--main.py
--lib #that's a directory/package
----__init__.py
----view.py
----common_lib.py
----other irrelevant modules...
In main.py:
from lib import view
causes the following error:
ImportError: cannot import name view
If instead, I write it like:
from lib.view import *
This import passes successfully, but next failure happens in view.py, in that:
from common_lib import Comments, Locations, ScreenData, ProgressSignal
causes:
ImportError: No module named 'common_lib'
And as it appears from the directory structure, common_lib.py resides in the same directory as view.py, how can it happen that it cannot be found?
How does it come that 'from x import y' doesn't work, and 'from x.y import *' works?
'__init__.py' is completely empty BTW. And I am using Python 3.3 32-bit.
What is more annoying, this same program worked 2 days ago.I was testing some code in IDLE and when I thought the code was ready to include in the project, pasted it into PyDev, I was shocked by this error. I changed nothing about my directories or modules.
Also, still more strangeness, running view.py and common_lib.py as standalone (without being imported) runs just fine. It should produce the error if any issues really existed.
Thanks. Any advice is highly appreciated.
Since main.py is still at the top level, you need to use lib.common_lib:
from lib.common_lib import Comments, Locations, ScreenData, ProgressSignal
because the previous line from lib import view does not start looking for modules from inside lib.
Given:
+--main.py # from lib import view
+--lib
+--__init__.py
+--common_lib.py # Comments, etc.
+--view.py # from .common_lib import Comments, etc.
This works:
from lib import view
And this works from view.py with a relative import to indicate common_lib is in the same package.
from .common_lib import Comments, Locations, ScreenData, ProgressSignal
Works for me:
danielallan#MacBook:~$mkdir myproject
danielallan#MacBook:~$cd myproject/
danielallan#MacBook:myproject$mkdir lib
danielallan#MacBook:myproject$cd lib
danielallan#MacBook:lib$touch __init__.py
danielallan#MacBook:lib$touch view.py
danielallan#MacBook:lib$touch common_lib.py
danielallan#MacBook:lib$cd ..
In [1]: from lib import view
In [2]: view
Out[2]: <module 'lib.view' from 'lib/view.pyc'>
What happens when you try that on your machine? Are you sitting in the wrong directory, or is your path not configured to find these files?

Categories