I am trying to import a file from a folder named pytorch_net from a folder named AI_physicist into a script named models.py. I have tried to change the folder locations of the files, get an init.py file into the main AI_physicist folder, and change the sys.path.append command to get only the folder with the files inside of it. I have also looked at other posts and have tried their solutions but to no avail.
Here are the posts that I have found and have tried:
What does os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)) mean? python
import python module using sys.path.append
file structure:
C:-----
Users----
trevo----
Desktop----
AI_physicist----
pytorch_net----
theory_learning----
Here is the error in its entirety:
No module named 'pytorch_net'
File "C:\Users\trevo\OneDrive\Desktop\AI_physicist\theory_learning\models.py", line 13, in <module>
from pytorch_net.net import MLP
File "C:\Users\trevo\OneDrive\Desktop\AI_physicist\theory_learning\theory_model.py", line 24, in <module>
from AI_physicist.theory_learning.models import Loss_Fun_Cumu, get_Lagrangian_loss
And lastly here is the code that I am using:
models.py:
# coding: utf-8
# In[ ]:
import numpy as np
import torch
import torch.nn as nn
from torch.autograd import grad
import sys, os
sys.path.append(os.path.join(os.path.dirname("__file__"), '..', '..'))
from AI_physicist.pytorch_net.net import MLP
from AI_physicist.pytorch_net.util import get_criterion, MAELoss, to_np_array, to_Variable, Loss_Fun
from AI_physicist.settings.global_param import PrecisionFloorLoss, Dt
from AI_physicist.theory_learning.util_theory import logplus
Any help on what I would be missing would be very appreciated, thank you!
Try to put __init__.py not init.py in that folder.
Alternatively:
sys.path.append(f'{os.path.dirname(__file__)}\\folder_name')
Please look on the difference between my and yours sys.append.
That should always work. Bare in mind these will work only if your script is in a subfolder of your main folder with main.py in.
Related
I am brand new to working with python so this question might be basic. I am attempting to import five helper files into a primary script, the directory setup is as follows and both the script I'm calling from and the helper scripts are located within src here in this path-
/Users/myusername/Desktop/abd-datatable/src
I am currently importing as-
import helper1 as fdh
import helper2 as hdh
..
import helper5 as constants
The error I see is
File "/Users/myusername/Library/Application Support/JetBrains/Toolbox/apps/PyCharm-P/ch-0/22.77756/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
ModuleNotFoundError: No module named 'src'
I have also attempted the following for which the import was still unsuccessful:
import src.helper1 as fdh
..
import src.helper5 as constants
and
from src import src.helper1 as fdh
...
from src import helper5 as constants
and tried adding the following to the head of the script-
import sys
import os
module_path = os.path.abspath(os.getcwd())
if module_path not in sys.path:
sys.path.append(module_path)
Would be very grateful for any pointers on how to debug this!
Seems likely that it's a path issue.
In your case, if you want to import from src, you would need to add Users/myusername/Desktop/abd-datatable to the path.
os.getcwd() is getting the path you invoke the script from, so it would only work if you are invoking the script from Users/myusername/Desktop/abd-datatable.
I have two scripts in the same folder:
5057_Basic_Flow_Acquire.xyzpy
5006_Basic_Flow_Execute.xyzpy
I need to call function run() from 5057_Basic_Flow_Acquire file in the 5006_Basic_Flow_Execute file.
I tried two approaches:
1.
import time
import os
import sys
import types
dir_path = os.path.dirname(os.path.realpath(__file__))
#os.chdir(str(dir_path))
sys.path.append(str(dir_path))
sensor = __import__('5057_Basic_Flow_Acquire')
import time
import os
import sys
import types
import importlib
import importlib.util
dir_path = os.path.dirname(os.path.realpath(__file__))
spec = importlib.util.spec_from_file_location('run', dir_path+'\\5057_Basic_Flow_Acquire')
module = importlib.util.module_from_spec(spec)
Pycharm is reporting this type of errors:
case:
ModuleNotFoundError: No module named '5057_Basic_Flow_Acquire'
case
AttributeError: 'NoneType' object has no attribute 'loader'
So in both cases module was not found.
Does someone have any suggestion?
You should rename your files to something like :
5057_Basic_Flow_Acquire_xyz.py
5006_Basic_Flow_Execute_xyz.py
so that they are recognized as modules and can be imported.
Then, in your 5057_Basic_Flow_Acquire_xyz module, you can import the run function with the following line :
from 5006_Basic_Flow_Execute_xyz import run
Both files are in the same directory, so you should be able to import without changing your PATH environment variable as the current directory is automatically added at the top of the list.
So below I have is my file structure:
/
api.py
app.py
code/
data_api/
weather_data/
get_data_dir/
get_data.py
master_call_dir/
master_call.py
My current problem is that I'm importing master_call.py from inside app.py I do this with the following code:
-- app.py
import sys
sys.path.append('./code/data_api/weather_data/master_call_dir')
from master_call import master_call_interface as master_call
Importing master_call itself works perfectly fine, the issue is that inside off master_call I import get_data.py. I do this with the following code:
-- master_call.py
import sys
sys.path.append("../get_data_dir")
from get_data import get_data_module
When printing out sys.path I can see ../get_data_dir inside of it but Python still isn't able to find the get_data.py file in /get_data_dir.
Does anyone know how to fix this?
(there is an __init__.py file in every directory, I removed it here for readability)
So I figured it out. The problem is while the directory of the master_call.py file is /code/data_api/weather_data/master_call the current working directory (CWD) is / since that is where app.py is located.
To fix it we simply fetch absolute file path in each python script with
current_path = os.path.dirname(os.path.realpath(__file__)).replace('\\', '/')
and pre-pend it to any file path we're appending with sys.path.append
So it'd look like this in practice:
import sys
import os
current_path = os.path.dirname(os.path.realpath(__file__)).replace('\\', '/')
sys.path.append(f'{current_path}./code/data_api/weather_data/master_call_dir')
from master_call import master_call_interface as master_call
Here's my folder structure:
src
->deployment_pipeline
->__init__.py, train_pipeline.py
src
->dags
->__init__.py,airflow_dag.py
src
->db_connector_mlflow
-> __init__.py, db_connector_mlflow.py
Now, I am trying to import a function start_final_train from train_pipeline.py(which is inside the folder deployment_pipeline) to airflow_dag.py and from db_connector_mlflow.py(which is inside the folder db_connector_mlflow) to airflow_dag.py
My import statement:
from deployment_pipeline import start_final_train
But I keep getting this error:
ModuleNotFoundError: No module named 'deployment_pipeline'
imports must be globally installed or be in the same directory or subdirectory thereof as your main file.
If you move your main file to the src folder and run everything from there it works out.
Your main file should import:
from dags.airflow_dag import <stuff you need from airflow_dag.py>
....
You should keep the same structure in airflow_dag.py to import your function (as if you were importing from src):
from deployment_pipeline.train_pipeline import start_final_train
The files I need in the directory look like
face/util/load_data.py
face/preprocessing.py
preprocessing.py defines the classes FaceDetector, FaceAligner and the method clip_to_range
I want to import these classes into load_data.py
I am trying to execute this statement inside load_data.py
from preprocessing import FaceDetector, FaceAligner, clip_to_range
I am getting the error
Traceback (most recent call last):
File "utils/load_data.py", line 7, in <module>
from preprocessing import FaceDetector, FaceAligner, clip_to_range
ImportError: cannot import name 'FaceDetector'
Can you please tell me how to correctly import these classes?
You can try adding __init__.py
Which will allow you to import face module, then you can import like,
from face.preprocessing import FaceDetector, FaceAligner, clip_to_range
Update:
Another way is to insert module in sys.path,
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
import preprocessing
move preprocessing.py to util directory.
Look for the file in the path of the executed file.
If you do not want to change the directory structure. Add the following code.
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
Import is possible regardless of directory structure.
You will can solved.