Can't import module from different directory python - python

So below I have is my file structure:
/
api.py
app.py
code/
data_api/
weather_data/
get_data_dir/
get_data.py
master_call_dir/
master_call.py
My current problem is that I'm importing master_call.py from inside app.py I do this with the following code:
-- app.py
import sys
sys.path.append('./code/data_api/weather_data/master_call_dir')
from master_call import master_call_interface as master_call
Importing master_call itself works perfectly fine, the issue is that inside off master_call I import get_data.py. I do this with the following code:
-- master_call.py
import sys
sys.path.append("../get_data_dir")
from get_data import get_data_module
When printing out sys.path I can see ../get_data_dir inside of it but Python still isn't able to find the get_data.py file in /get_data_dir.
Does anyone know how to fix this?
(there is an __init__.py file in every directory, I removed it here for readability)


So I figured it out. The problem is while the directory of the master_call.py file is /code/data_api/weather_data/master_call the current working directory (CWD) is / since that is where app.py is located.
To fix it we simply fetch absolute file path in each python script with
current_path = os.path.dirname(os.path.realpath(__file__)).replace('\\', '/')
and pre-pend it to any file path we're appending with sys.path.append
So it'd look like this in practice:
import sys
import os
current_path = os.path.dirname(os.path.realpath(__file__)).replace('\\', '/')
sys.path.append(f'{current_path}./code/data_api/weather_data/master_call_dir')
from master_call import master_call_interface as master_call

Related

How can I import module from another folder which is not same directory structure

I have two directory on same level
1.Scripts->(Inside it we have app.py file)
2. Keymaker-->(Inside it we have keymaker.py file)
Now I want to import keymaker.py in app.py, So how Can I do it

Write these lines in app.py file.
import sys
sys.path.append('path/to/Keymaker/dir')
After that you can import anything from keymaker.py file using
from keymaker import *

Add empty __init__.py file to Keymaker directory and do from Keymaker import keymaker

File import issue in Python

Here's my folder structure:
src
->deployment_pipeline
->__init__.py, train_pipeline.py
src
->dags
->__init__.py,airflow_dag.py
src
->db_connector_mlflow
-> __init__.py, db_connector_mlflow.py
Now, I am trying to import a function start_final_train from train_pipeline.py(which is inside the folder deployment_pipeline) to airflow_dag.py and from db_connector_mlflow.py(which is inside the folder db_connector_mlflow) to airflow_dag.py
My import statement:
from deployment_pipeline import start_final_train
But I keep getting this error:
ModuleNotFoundError: No module named 'deployment_pipeline'

imports must be globally installed or be in the same directory or subdirectory thereof as your main file.
If you move your main file to the src folder and run everything from there it works out.
Your main file should import:
from dags.airflow_dag import <stuff you need from airflow_dag.py>
....
You should keep the same structure in airflow_dag.py to import your function (as if you were importing from src):
from deployment_pipeline.train_pipeline import start_final_train

Python Importing files from other folders (within the same project) using __init__.py?

Trying to import aFile.py from within the bSubFile.py but getting an error saying 'exceptions.ValueError, Attempted relative import in non-package'
My file structure is as follows:
app/
- __init__.py
FolderA/
- __init__.py
- aFile.py
FolderB/
- __init__.py
- bFile.py
SubfolderB/
- __init__.py
- bSubFile.py
I am trying to import aFile from bSubFile.py
tried:
from ..FolderA import aFile
class bSubFile():
...
and:
from ...FolderA import aFile
class bSubFile():
...
but I always get 'Attempted relative import in non-package', I must be missing something very obvious. Thanks!

You can add the other path to your system path. This may be not the most elegant way, but it works.
import sys
import os
# get the folder of the current file, go one directory back and add "FolderA"
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "FolderA"))
# now you can import everything that is in FolderA directly
import aFile

Python access to Parent Directory

I have a file in the directory
app
a
Ulil.py
b
main.py
I want to import Ulil.py (at app\a) into main.py (at app\b).
How do i go about doing this. I need to move the file around as well so I don't want to put the entire path. I just to be in app\b and access only app\a. Folder name stays the same.

You can add the directory to sys.path:
import sys
sys.path.insert( 0, '../a' )
import Util

First, create an empty file called __init__.py in a
Second, in main.py in b import it using
import sys
sys.path.insert( 0, '../' )
from a.Util import *

Python import file with dependencies in the same folder

I have a file ref.py that depends on a text file ex.txt, that is in the same directory \home\ref_dir . So it works normally when I run the file ref.py, but if I try to import ref.py to another file work.py in a different directory \home\work_dir , I do the following
import sys
sys.path.append('\home\ref_dir')
import ref
But then I get an error, that the program cannot find ex.txt
How can I solve this issue without using absolute paths. Any ideas?

Use the os module to get access to the current absolute path of the module that you're in, and then get the dirname from that
You would want to open ex.txt in your file like this.
import os
with open('%s/ex.txt' % os.path.dirname(os.path.abspath(__file__)) as ex:
print ex.read()

Categories