I am having trouble importing a module from within another module. I understand that that sentence can be confusing so my question and situation is exactly like the one suggested here: Python relative-import script two levels up
So lets say my directory structure is like so:
main_package
|
| __init__.py
| folder_1
| | __init__.py
| | folder_2
| | | __init__.py
| | | script_a.py
| | | script_b.py
|
| folder_3
| | __init__.py
| | script_c.py
And I want to access code in script_b.py as well as code from script_c.py from script_a.py.
I have also followed exactly what the answer suggested with absolute imports.
I included the following lines of code in script_a.py:
from main_package.folder_3 import script_c
from main_package.folder1.folder2 import script_b
When I run script_a.py, I get the following error:
ModuleNotFoundError: No module named 'main_package'
What am I doing wrong here?
This is because python doesn't know where to find main_package in script_a.py.
There are a couple of ways to expose main_package to python:
run script_a.py from main_package's parent directory (say packages). Python will look for it in the current directory (packages), which contains main_package:
python main_package/folder_1/folder_2/script_a.py
add main_package's parent directory (packages) to your PYTHONPATH:
export PYTHONPATH="$PYTHONPATH:/path/to/packages"; python script_a.py
add main_package's parent directory (packages) to sys.path in script_a.py
In your script_a.py, add the following at the top:
import sys
sys.path.append('/path/to/packages')
Related
A have several DAGs of similar structure and I wanted to use advice described in Airflow docs as Modules Management:
This is an example structure that you might have in your dags folder:
<DIRECTORY ON PYTHONPATH>
| .airflowignore -- only needed in ``dags`` folder, see below
| -- my_company
| __init__.py
| common_package
| | __init__.py
| | common_module.py
| | subpackage
| | __init__.py
| | subpackaged_util_module.py
|
| my_custom_dags
| __init__.py
| my_dag1.py
| my_dag2.py
| base_dag.py
In the case above, these are the ways you could import the python
files:
from my_company.common_package.common_module import SomeClass
from my_company.common_package.subpackage.subpackaged_util_module import AnotherClass
from my_company.my_custom_dags.base_dag import BaseDag
That works fine in Airflow.
However I used to validate my DAGs locally by running (also as advised by a piece of documentation - DAG Loader Test):
python my_company/my_custom_dags/my_dag1.py
When using the imports, it complains:
Traceback (most recent call last):
File "/[...]/my_company/my_custom_dags/my_dag1.py", line 1, in <module>
from my_company.common_package.common_module import SomeClass
ModuleNotFoundError: No module named 'my_company'
How should I run it so that it understands the context and reckognizes the package?
It works when run this way:
PYTHONPATH=. python my_company/my_custom_dags/my_dag1.py
It seems that when entry point is my_dag1.py that's inside my_company/my_custom_dags Python considers it as its "working directory" and only looks for modules within that path. When I add . to PYTHONPATH it can also look at entire directory structure and reckognize module my_company and below.
(I'm not expert in Python, so above explanation might be somewhat innacurrate, but this is my understanding. I'm also not sure if that's indeed the cleanest way to make it work)
Hello Python Programmers
I'm getting a weird module import error during unittest. Not able to find the root cause.
Here is how my directory structure looks like
Main_folder
|
|
Module_x
| ABC.py
|
|
Module_y
| DEF.py
|
|
test
| unit_tests
| test_ABC.py
In test_ABC.py I'm importing the following
from Module_x import ABC
I get the error as "No module name Module_x"
I've created __init__.py file at each folder
I don't get any error if I use the same line from DEF.py
Please help if you're aware about why I am getting this issue?
I am using Python 3.5 Anaconda Distribution
Thanks
This is not a weird problem but a common one.
You can add your root into PYTHONPATH to solve this problem: PYTHONPATH=/path/to/project_root python test_ABC.py or something else similar.
I have a python 2.7 project which I have structured as below:
project
|
|____src
| |
| |__pkg
| |
| |__ __init__.py
|
|____test
|
|__test_pkg
| |
| |__ __init__.py
|
|__helpers
| |
| |__ __init__.py
|
|__ __init__.py
I am setting the src folder to the PYTHONPATH, so importing works nicely in the packages inside src. I am using eclipse, pylint inside eclipse and nosetests in eclipse as well as via bash and in a make file (for project). So I have to satisfy lets say every stakeholder!
The problem is importing some code from the helpers package in test. Weirdly enough, my test is also a python package with __init__.py containing some top level setUp and tearDown method for all tests. So when I try this:
import helpers
from helpers.blaaa import Blaaa
in some module inside test_pkg, all my stakeholders are not satisfied. I get the ImportError: No module named ... and pylint also complains about not finding it. I can live with pylint complaining in test folders but nosetests is also dying if I run it in the project directory and test directory. I would prefer not to do relative imports with dot (.).
The problem is that you can not escape the current directory by importing from ..helpers.
But if you start your test code inside the test directory with
python3 -m test_pkg.foo
the current directory will be the test directory and importing helpers will work. On the minus side that means you have to import from . inside test_pkg.
I would understand this behaviour.
In Pycharm I can successfully run (green play button) this main.py which includes src.v0.Det.py
src
|
+--v0
| |
| +--Det.py (class det)
| __init__.py
|
+-- main.py
__init__.py
main.py
------------
from src.v0.Det import det
....
But when I run on unix using: python main.py I get "No module named src.v0.Det".
I can obviously remove "src" and it works but I don't want to change the file. How can I run this file like in Pycharm?
Riccardo
export PYTHONPATH="${PYTHONPATH}:<.....path.to.src.without.src....>"
I'm looking for the best way to access the root directory of a sub-package in order to retrieve common configuration files (which in my case are JSON files).
As an example, the directory structure may look as follows:
| README
| setup.py
| proj_name
| __init__.py
| pack_01
| __init__.py
| file_01.py
| file_02.py
| pack_02
| __init__.py
| file_03.py
| file_04.py
| conf
| conf_01.json
| conf_02.json
| conf_03.json
In this example, the package config files are in the proj_name/conf directory.
When importing say file_01 using a command such as import proj_name.pack_01.file_01, the attribute __file__ would point to proj_name/pack_01/file_01.py and a line such as root_dir=dirname(dirname(__file__)) is required; this however implies a knowledge of the directory structure when writing the sub-package.
Is there a way to access the root package (proj_name in this case) and its directory using a variable such as __root_package__ or something similar? What is the tidiest way to achieve this in python? And would the method still work when building an egg using a setup.py utility?
Thank you in advance!