ModuleNotFoundError: struggling to with imports within an imported program - python

I have a program, program1.py, that has this structure:
Program
--program1.py
--__init__.py
--data\
----__init__.py
----helper_data.py
--classes\
----__init__.py
----helper_class.py
In helper_class.py, there is an import statement from data.helper_data import *. When I run program1, this works perfectly.
I have a second program, program2.py. I have put program1.py on my PYTHONPATH. In program2.py, I use import program1. It finds the program, but when running the imports from program1.py, I get the following error stemming from the classes.helper_class: ModuleNotFoundError: No module named 'data.helper_data'.
I think I vaguely understand what's going on, but I can't figure out the fix or the search terms to find the answer. I've tried changing the import in program1 to from ..data.helper_data import * and get an error saying I've tried a relative import beyond the parent-level package. I've also tried from .data.helper_data import * and get the same ModuleNotFoundError.
What can I do?

I think You have to import "sys" package.
import sys
sys.path.append('E:\ToDataScientist') # this is where the "Program" folder exists
from Program.data.helper_data import aa # "aa" is the class or function in helper_data
from Program.data.helper_data import * # include all from helper_data

Related

Import problems

On the following path:
main
system.py
|-----pid
init.py
pid.py
tunning_methods.py
pid.py has the following relevant import lines:
import tunning_methods
tunning_methods.py does not import any other file as it does not need it.
init.py is empty, and system.py has the following import lines:
import pid.tunning_methods
import pid.pid
every file has a function to test it on the format:
if __name__ == "__main__":
testfunc()
now when i try to run pid.py it runs just fine, however when i try to run system.py it raises an error saying the is no such module named tunning_methods.py on the pid.py file. I was able to solve that by importing "import pid.tunning_methods" on the pid.py it was like on the system vision of import path, instead of the pid, however the pid now does not run, more than that im having a lot of trouble with imports, this is just a piece of application that goes in to a much bigger one, its possible to do this for every import, however i want an easier solution, any ideas ?
tried to do the imports on the description and it did not work as i was intending.
for anyone having the same problem the suggestion that Michael gave worked, a relative import from . import tunning_methods.

Python circular import in custom package and __init__.py

I get ImportError: cannot import name 'Result' from partially initialized module 'libs.elastic_search_hunt' (most likely due to a circular import) error when I try to run tests.
But I does not see any circular imports in my code.
I have a package, named elastic_search_hunt which contains 3 modules:
elastic_query.py
elastic_query_result.py
search_processor.py
And I also have __init__.py file with following text:
from libs.elastic_search_hunt.elastic_query import Query
from libs.elastic_search_hunt.search_processor import SearchProcessor
from libs.elastic_search_hunt.elastic_query_result import Result
__all__ = ['Query', 'SearchProcessor', 'Result'] # I guess it does not have any effect
elastic_query.py has only external imports.
elastic_query_result.py the same.
search_processor.py has those import:
from . import Query
from . import Result
Then I have a test file, which imports Query class:
from libs.elastic_search_hunt import Query
When I run tests, I get this errors:
test_query.py:2: in <module>
from libs.elastic_search_hunt import Query
..\src\libs\elastic_search_hunt\__init__.py:2: in <module>
from libs.elastic_search_hunt.search_processor import SearchProcessor
..\src\libs\elastic_search_hunt\search_processor.py:4: in <module>
from . import Result
E ImportError: cannot import name 'Result' from partially initialized module 'libs.elastic_search_hunt' (most likely due to a circular import)
But where is any circular import in my code?
I only can assume that when I import Query from tests, it also import search_processor from the __init__.py module which in turn loads Query one more time. But the error is about Result in elastic_query_result module and I see only one import of Result.
When i delete search_processor from __init__.py everything works fine.
I have read a lot of issues about circular imports, but all of them was quite obvious and does not touch the __init__.py. What am I missing?
TL;DR: replace from . import Query with from .elastic_query import Query
Explanation:
When you import something from libs.elastic_search_hunt module it loads __init__.py at first. Since every module executes at first import __init__.py also being executed.
Then Python executes code from __init__.py and at second line
from libs.elastic_search_hunt.search_processor import SearchProcessor
it imports search_processor.py. Since it's first import - file must be executed - therefore all your imports in that file must be executed right now as well:
As you mentioned you have the following imports in your file:
from . import Query
from . import Result
At this point you tell python to load libs.elastic_search_hunt entire module and take Query, Result from it. So Python does.
It makes an attempt to load libs/elastic_search_hunt/__init__.py but wait... it is still not loaded completely. So it must load it, but in order to load it properly it must firstly load search_processor which requires elastic_search_hunt/__init__.py to be loaded.... oh well, there's a loop.
So in order to avoid such behaviour you should explicitly say from which module exactly you wish to load Query and Result, therefore change
from . import Query
from . import Result
to
from .elastic_query import Query
from .elastic_query_result import Result
Example: Failed
Example: Success

Python import: attempted relative import

I had some errors on import from top level folderin Python
(version Python 3.6.9)
My folder structure is below:
-topfolder
alltestfunc.py
-subfolder
update_test.py
I am trying to import alltestfunc.py in update_test.py , that's all.
The errors which i had:
ModuleNotFoundError: No module named 'alltestfunc'
ValueError: attempted relative import beyond top-level package
etc..
Thanx a lot
Are you calling the code like this?
python topfolder/subfolder/update_test.py
This would make the update_test.py the __main__ file, or the starting point of your program. From there on out you cannot do an import one level up, only from subdirectories.
If you have some script on the topfolder level that is called and then executes the update_test.py you should be good.

Failure with more than one co-dependant/circular python import

I am trying to import python code from one submodule to another, and I am getting an error, but only when I have more than co-dependant import from the package.
From my understanding, this "circular" importing is okay to do in Python. From the way I'd like the code to be organized, I do need these "co-dependant" imports and unless I really have to change it, I'd like to keep my code structured in the same submodules it currently is.
My directory/file structure:
./subm1/
./subm1_file.py
./subm2/
./subm2_file.py
./subm_main/
./subm_main_file.py
# subm1_file.py
# -------------
import subm_main.subm_main_file
print(subm_main.subm_main_file.test)
# subm2_file.py
# -------------
import subm_main.subm_main_file
print(subm_main.subm_main_file.test)
# subm_main_file.py
# -------------
import os
import sys
current_path = os.path.dirname(__file__)
sys.path.append(os.path.join(current_path+".."))
import subm1.subm1_file
import subm2.subm2_file
test = "test_variable"
I am running $ python subm_main_file.py while inside the subm_main directory
this works if I only use one module, so if in subm_main_file.py I comment out import subm1.subm1_file, it will run and print the test variable, and same if i comment out import subm2.subm2_file the error always comes one the second import. So in the code I show, the error is in subm2_file.py.
Error:
AttributeError: module 'subm_main' has no attribute 'subm_main_file'
It seems very strange to me that this works one time but not the second time with both import statements uncommented out. What can I do to fix this error (and hopefully keep my code organized in its current state)?
I cannot comment yet on a post but maybe your issue is that you need to place an init.py file inside the root and sub-directories if you want to keep your folder/code structure unchanged...

chained imports from subdirectory in python

I am using someone elses project to add some functionality to mine, and there is a python script I want to import. The problem comes with the import structure of their directories: I have placed their project directory in a subfolder under my main project (needs to stay there so I can keep their project out of my version control) it looks like this:
myproject/
myscript.py
theirproject/
__init__.py
baz.py
secondlayer/
__init__.py
all.py
foo.py
bar.py
all.py is simply a list of import statements which import additional scripts from the secondlayer directory like this:
from secondlayer.foo import *
from secondlayer.bar import * #etc
I would like to import:
from theirproject.secondlayer.all import *
but that fails when python complains "no module named secondlayer.foo"
I have also tried the following:
from theirproject.secondlayer import all
I can get it to work when I place my script in theirproject/ and import all without the "theirproject" prefix, but I really cant have it be like that. I can get further through the import process by importing foo, bar, etc individually like this:
from theirproject.secondlayer import foo
from theirproject.secondlayer import bar #etc
But then those scripts fail to import more stuff from still other scripts (like baz.py) at the same level as secondlayer, so im stuck.
Whats the right way to do this in python 2.7.6?
If you change
from secondlayer.foo import *
from secondlayer.bar import *
to user relative imports like this
from .foo import *
from .bar import *
or like this
from foo import *
from bar import *
it works.
Plus you could do these imports in the __init__.py at secondlayer level so that the import from myscript.py becomes
from theirproject.secondlayer.all import *
See if you have the necessary permissions to import the package from your directory and its corresponding sub directories.
For reference, you may like to see this and its corresponding linked questions :
Python Imports do not work
I ended up solving my problem by adding theirproject/ to my PYTHONPATH. I upvoted junnytony's answer - it helped point me in the right direction, so thanks!

Categories