We Keep Coding

Executing external python script function

I have two scripts in the same folder:
5057_Basic_Flow_Acquire.xyzpy
5006_Basic_Flow_Execute.xyzpy
I need to call function run() from 5057_Basic_Flow_Acquire file in the 5006_Basic_Flow_Execute file.
I tried two approaches:
1.
import time
import os
import sys
import types
dir_path = os.path.dirname(os.path.realpath(__file__))
#os.chdir(str(dir_path))
sys.path.append(str(dir_path))
sensor = __import__('5057_Basic_Flow_Acquire')
import time
import os
import sys
import types
import importlib
import importlib.util
dir_path = os.path.dirname(os.path.realpath(__file__))
spec = importlib.util.spec_from_file_location('run', dir_path+'\\5057_Basic_Flow_Acquire')
module = importlib.util.module_from_spec(spec)
Pycharm is reporting this type of errors:
case:
ModuleNotFoundError: No module named '5057_Basic_Flow_Acquire'
case
AttributeError: 'NoneType' object has no attribute 'loader'
So in both cases module was not found.
Does someone have any suggestion?

You should rename your files to something like :
5057_Basic_Flow_Acquire_xyz.py
5006_Basic_Flow_Execute_xyz.py
so that they are recognized as modules and can be imported.
Then, in your 5057_Basic_Flow_Acquire_xyz module, you can import the run function with the following line :
from 5006_Basic_Flow_Execute_xyz import run
Both files are in the same directory, so you should be able to import without changing your PATH environment variable as the current directory is automatically added at the top of the list.

Cloned github repository url lead to error in google colab

I'm working on a google colab available at this link: https://colab.research.google.com/github/lyricstopaintings/lyricstopaintings/blob/main/Lyrics%20inspired%20AI%20paintings.ipynb
On github: https://github.com/lyricstopaintings/lyricstopaintings
I would like to store all the files regarding my project on my own github repository, so I cloned a few public repositories for example:
https://github.com/openai/CLIP
https://github.com/kostarion/guided-diffusion
The following script works fine,
import pathlib, shutil, os, sys
useCPU = False ##param {type:"boolean"}
if not is_colab:
# If running locally, there's a good chance your env will need this in order to not crash upon np.matmul() or similar operations.
os.environ['KMP_DUPLICATE_LIB_OK']='TRUE'
PROJECT_DIR = os.path.abspath(os.getcwd())
USE_ADABINS = True
if is_colab:
if not google_drive:
root_path = f'/content'
model_path = '/content/models'
else:
root_path = os.getcwd()
model_path = f'{root_path}/models'
multipip_res = subprocess.run(['pip', 'install', 'lpips', 'datetime', 'timm', 'ftfy', 'einops', 'pytorch-lightning', 'omegaconf'], stdout=subprocess.PIPE).stdout.decode('utf-8')
print(multipip_res)
if is_colab:
subprocess.run(['apt', 'install', 'imagemagick'], stdout=subprocess.PIPE).stdout.decode('utf-8')
try:
from CLIP import clip
except:
if not os.path.exists("CLIP"):
gitclone("https://github.com/openai/CLIP")
sys.path.append(f'{PROJECT_DIR}/CLIP')
try:
from guided_diffusion.script_util import create_model_and_diffusion
except:
if not os.path.exists("guided-diffusion"):
gitclone("https://github.com/kostarion/guided-diffusion")
sys.path.append(f'{PROJECT_DIR}/guided-diffusion')
**... other modules and packages in the same structure**
import torch
from dataclasses import dataclass
from functools import partial
import cv2
import pandas as pd
import gc
import io
import math
import timm
from IPython import display
import lpips
from PIL import Image, ImageOps
import requests
from glob import glob
import json
from types import SimpleNamespace
from torch import nn
from torch.nn import functional as F
import torchvision.transforms as T
import torchvision.transforms.functional as TF
from tqdm.notebook import tqdm
from CLIP import clip
from resize_right import resize
from guided_diffusion.script_util import create_model_and_diffusion, model_and_diffusion_defaults
from datetime import datetime
import numpy as np
import matplotlib.pyplot as plt
import random
from ipywidgets import Output
import hashlib
from functools import partial
if is_colab:
os.chdir('/content')
from google.colab import files
else:
os.chdir(f'{PROJECT_DIR}')
from IPython.display import Image as ipyimg
from numpy import asarray
from einops import rearrange, repeat
import torch, torchvision
import time
from omegaconf import OmegaConf
import warnings
warnings.filterwarnings("ignore", category=UserWarning)
but when I change the links
https://github.com/openai/CLIP --> https://github.com/lyricstopaintings/lyricstopaintings/tree/main/CLIP
https://github.com/kostarion/guided-diffusion --> https://github.com/lyricstopaintings/lyricstopaintings/tree/main/guided-diffusion
got the following error:
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-3-b4c7e2b3bdd9> in <module>()
107 import torchvision.transforms.functional as TF
108 from tqdm.notebook import tqdm
--> 109 from CLIP import clip
110 from resize_right import resize
111 from guided_diffusion.script_util import create_model_and_diffusion, model_and_diffusion_defaults
ModuleNotFoundError: No module named 'CLIP'
What is wrong with my approach? I'm new on this field, so sorry if this is some basic thing.

The Github repository should be referred to as follows
https://github.com/lyricstopaintings/lyricstopaintings
But this contains folders for the cloned repositories so the path needs to include the folder as follows.
sys.path.append(f'{PROJECT_DIR}/lyricstopaintings/CLIP')
I couldn't get from CLIP import clip working so I changed it to import clip.
I cut down the code to be a minimal example because I couldn't find some of the functions such as subprocess.run and gitclone so I replaced these with other functions.
import pathlib, shutil, os, sys
PROJECT_DIR = os.path.abspath(os.getcwd())
!pip install -q ftfy
try:
import clip
except:
if not os.path.exists("CLIP"):
!git clone -q "https://github.com/lyricstopaintings/lyricstopaintings"
sys.path.append(f'{PROJECT_DIR}/lyricstopaintings/CLIP')
import clip
clip.available_models()
#
['RN50',
'RN101',
'RN50x4',
'RN50x16',
'RN50x64',
'ViT-B/32',
'ViT-B/16',
'ViT-L/14',
'ViT-L/14#336px']

Python ModuleNotFoundError - No module named 'pytorch_net'

I am trying to import a file from a folder named pytorch_net from a folder named AI_physicist into a script named models.py. I have tried to change the folder locations of the files, get an init.py file into the main AI_physicist folder, and change the sys.path.append command to get only the folder with the files inside of it. I have also looked at other posts and have tried their solutions but to no avail.
Here are the posts that I have found and have tried:
What does os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir)) mean? python
import python module using sys.path.append
file structure:
C:-----
Users----
trevo----
Desktop----
AI_physicist----
pytorch_net----
theory_learning----
Here is the error in its entirety:
No module named 'pytorch_net'
File "C:\Users\trevo\OneDrive\Desktop\AI_physicist\theory_learning\models.py", line 13, in <module>
from pytorch_net.net import MLP
File "C:\Users\trevo\OneDrive\Desktop\AI_physicist\theory_learning\theory_model.py", line 24, in <module>
from AI_physicist.theory_learning.models import Loss_Fun_Cumu, get_Lagrangian_loss
And lastly here is the code that I am using:
models.py:
# coding: utf-8
# In[ ]:
import numpy as np
import torch
import torch.nn as nn
from torch.autograd import grad
import sys, os
sys.path.append(os.path.join(os.path.dirname("__file__"), '..', '..'))
from AI_physicist.pytorch_net.net import MLP
from AI_physicist.pytorch_net.util import get_criterion, MAELoss, to_np_array, to_Variable, Loss_Fun
from AI_physicist.settings.global_param import PrecisionFloorLoss, Dt
from AI_physicist.theory_learning.util_theory import logplus
Any help on what I would be missing would be very appreciated, thank you!

Try to put __init__.py not init.py in that folder.
Alternatively:
sys.path.append(f'{os.path.dirname(__file__)}\\folder_name')
Please look on the difference between my and yours sys.append.
That should always work. Bare in mind these will work only if your script is in a subfolder of your main folder with main.py in.

importing the right module but getting an error in python

import sys
from subprocess import run, PIPE
import shlex
from src.detector_main import detect_main
def main():
# print command line arguments
for arg in sys.argv[1:]:
print(arg)
if __name__ == "__main__":
# main()
print(sys.argv)
This is my main module. If you see the from src.detector_main import detect_main, it is supposed to import detect_main from src/detector_main.py.
In my detector_main.py, I have a bunch of imports,
import ast
import os
import fpdf
import sys
from Detector.class_coupling_detector import detect_class_cohesion
from Detector.cyclomatic_complexity_detector import detect_cyclomatic_complexity
from Detector.long_lambda_detector import detect_long_lambda
from Detector.long_list_comp_detector import detect_long_list_comp
from Detector.pylint_output_detector import detect_pylint_output
from Detector.shotgun_surgery_detector import detect_shotgun_surgery
from Detector.useless_exception_detector import detect_useless_exception
# from tools.viz_generator import add_viz
def detect_main(directory):
# Get stats for files in directory
stats_dict = get_stats(directory)
....
Running my main module gives me this error:
File "pyscent.py", line 5, in <module>
from src.detector_main import detect_main
File "C:\Users\user\Desktop\proj\src\detector_main.py", line 5, in <module>
from Detector.class_coupling_detector import detect_class_cohesion
ModuleNotFoundError: No module named 'Detector'
I don't get this because I am following the exact path.
I don't get this because I am following the right path.

In your example you import the Detector.class_coupling_detector module in file that is in the same directory as Detector but your cwd is not the src directory.
Due to this you should either use absolute import from src.Detector... or relative import from .Detector...
Here is some information about difference between this two ways to import

guidelines for importing modules in python

I know that there have been a number of questions concerning importing modules in python, but my question seems to be somewhat different.
I'm trying to understand when you have to import a whole module as opposed to when you have to import a specific entry in the module. It seems that only one of the two ways work.
For example if I want to use basename, importing os.path doesn't do the trick.
>>> import os.path
>>> basename('scripts/cy.py')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'basename' is not defined
Instead I need to import basename from os.path as in
>>> from os.path import basename
>>> basename('scripts/cy.py')
'cy.py'
Going the other way, if I want to use shutil.copyfile, importing copyfile from shutil doesn't work
>>>
>>> from shutil import copyfile
>>>
>>> shutil.copyfile('emma','newemma')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'shutil' is not defined
Instead I must import shutil as in
>>>
>>> import shutil
>>>
>>> shutil.copyfile('emma','newemma')
'newemma'
>>>
The only way I have been able to get this right is through experimentation.
Are there some guidelines to avoid experimentation?

If you import
import os.path
then you have to use full namespace os.path
os.path.basename()
If you import with from
from shutil import copyfile
then you don't have to use full namespace shutil
copyfile(...)
That's all.
If you use as
import os.path as xxx
then you have to use xxx instead of os.path
xxx.basename()
If you use from and as
from os.path import basename as xxx
then you have to use xxx instead of basename
xxx()

you could use form module import *
from time import *
sleep(2)
which allow you to call its submodules, instead of:
from time import sleep
sleep(2)
or:
import time
time.sleep(2)
this imports every sub-module of the package
https://docs.python.org/2/tutorial/modules.html#importing-from-a-package