We Keep Coding

force subprocss to return a value before importing a module

Objective : To Create UnitTestCases for main.py
How can we run subprocess.run as unittest case
File : main.py
import os
_ENV = os.environ['server']
BASE_PATH = '/home/test'
SERVER_URL = {
'DEV':{'ENV_URL':'https://dev.net'},
'UAT':{'ENV_URL':'https://uat.net'},
'PROD':{'ENV_URL':'https://prod.net'}
}[_ENV]
if _CYBERARK=='DYNAMIC' and _ENV.upper() in ('DEV','UAT','PROD') :
TOKEN = json.load(open(BASE_PATH + "secrets/cyberark_env.json"))
APPID = CONFIGURE_TOKEN.get('desc').get('appID')
## --> error on this because its running .sh files
VALUE=subprocess.run(["sh",BASE_PATH + "bin/cyberark.sh",APPID],capture_output=True,text=True).stdout.strip()
In my test_main.py
I am trying to import main.py to my test.py without any issues, but i find it challenging on how we to manually make VALUE variable return as 'ABCD'
Similar to below scripts
Testing : test_main.py
import unittest, sys
from unittest import mock
os.environ['server']='DEV'
# Similar to os.environ , i want to create temporarily VALUE before import main is called.
sys.path.insert(1, 'C:/home/src')
import main.py as conf
class test_tbrp_case(unittest.TestCase):
def test_port(self):
#function yet to be created
pass
if __name__=='__main__':
unittest.main()
Is this achieveable or i need to relogic my whole function

How to resolve this: pyreportjasper: ImportError: DLL load failed while importing jpy: The specified module could not be found

I want to use Jasper Reports with a python-flask app.
I have followed the installation instructions on this page:
https://pypi.org/project/pyreportjasper/
It fails on the line import jpy, I am getting the error:
ImportError: DLL load failed while importing jpy: The specified module could not be found.
I am using 32 bit python Python: 3.8.6rc1.
32 bit JDK: javac 1.8.0_261
When I run pip install pyreportjasper, it says: Requirement already satisfied: jpy in c:\users....
Any ideas?

It seems that you need to import jpyutil and then initialize the JVM prior to calling import jpy. The following fixed this error for me. One step closer.
import jpyutil
jpyutil.init_jvm(jvm_maxmem='512M')
import jpy
UPDATE: so while the above statement is true, it doesn't directly address the issue of running the example code at the project page: https://pypi.org/project/pyreportjasper/
As I dug into the code I found that jasperpy.py is attempting to import jpy BEFORE the __init__ method is run and therefore BEFORE the JVM is initialized via the jpyutil.init_jvm() method call. So I edited jasperpy.py and moved import jpy from line 14 to line 56, directly after the call to jpyutil.init_jvm() and before the call to jpy.get_type('java.io.File') and I am able to successfully run Jasper reports from python via the example code shown. Here is what my jasperpy.py looks like now.
# -*- coding: utf-8 -*-
# GNU GENERAL PUBLIC LICENSE
#
# Copyright (c) 2020 Jadson Bonfim Ribeiro <contato#jadsonbr.com.br>
#
import os
import subprocess
import re
import xml.etree.ElementTree as ET
import tempfile
import jpyutil
#import jpy
import json
from requests import Request, Session
FORMATS = (
'pdf',
'rtf',
'xls',
'xlsx',
'docx',
'odt',
'ods',
'pptx',
'csv',
'html',
'xhtml',
'xml',
'jrprint',
)
EXECUTABLE = 'jasperstarter'
class JasperPy:
_FORMATS_JSON = ('pdf')
_FORMATS_METHODS_REQUEST = ('GET', 'POST', 'PUT')
def __init__(self, resource_dir=False, jvm_maxmem='512M', jvm_classpath=None):
self.WINDOWS = True if os.name == 'nt' else False
self.SCRIPT_DIR = os.path.abspath(os.path.dirname(__file__))
self.LIBS = os.path.join(self.SCRIPT_DIR, 'jasperstarter', 'lib')
if not os.path.isdir(self.LIBS):
raise NameError('Unable to find lib in {0}'.format(self.LIBS))
self.CLASSPATH = os.path.join(self.LIBS, 'jasperstarter.jar')
if not os.path.exists(self.CLASSPATH):
raise NameError('Unable to find jasperstarter in {0}'.format(self.LIBS))
if jvm_classpath is None:
jpyutil.init_jvm(jvm_maxmem=jvm_maxmem, jvm_classpath=[self.CLASSPATH])
else:
jpyutil.init_jvm(jvm_maxmem=jvm_maxmem, jvm_classpath=[self.CLASSPATH, jvm_classpath])
# IMPORT jpy HERE AFTER init_jvm
import jpy
self.jvFile = jpy.get_type('java.io.File')
self.jvArrays = jpy.get_type('java.util.Arrays')
self.jvReport = jpy.get_type('de.cenote.jasperstarter.Report')
self.jvConfig = jpy.get_type('de.cenote.jasperstarter.Config')
self.jvDsType = jpy.get_type('de.cenote.jasperstarter.types.DsType')
self.jvApplicationClasspath = jpy.get_type('de.cenote.tools.classpath.ApplicationClasspath')
self.jvHashMap = jpy.get_type('java.util.HashMap')
self.jvLocale = jpy.get_type('java.util.Locale')
self.jvJasperFillManager = jpy.get_type('net.sf.jasperreports.engine.JasperFillManager')
self.jvDb = jpy.get_type('de.cenote.jasperstarter.Db')
self.jvJsonQueryExecuterFactory = jpy.get_type('net.sf.jasperreports.engine.query.JsonQueryExecuterFactory')
self.jvJasperExportManager = jpy.get_type('net.sf.jasperreports.engine.JasperExportManager')
.
.
.
.
This file is located in your Python directory under Lib\site-packages\pyreportjasper-2.0.2-py3.8.egg\pyreportjasper. While this is currently a hack to make it work, there obviously needs to be a fix to the package which I will attempt to address on the pyreportjasper github page.

Is there a way to manage huge many import statements in Python?

I am working on a Flask-RESTFul API where I have so many import statements in my app.py file as below.
from flask import Flask
from flask_restful import Api
from apis.ChambAvail import ChambAvail
from apis.ChambAvailBal import ChambAvailBal
from apis.ChambTwInhibit import ChambTwInhibit
from apis.CurrentWip import CurrentWip
from apis.Detail import Detail
from apis.IncomingWip import IncomingWip
from apis.Info import Info
from apis.Manage import Manage
from apis.MfCount import MfCount
from apis.OtherHold import OtherHold
from apis.PqeDue import PqeDue
from apis.QualInhibit import QualInhibit
from apis.ReleaseBef import ReleaseBef
from apis.RfInhibit import RfInhibit
from apis.SketchDetailedLot import SketchDetailedLot
from apis.SketchLotDetail import SketchLotDetail
from apis.SketchMainDetail import SketchMainDetail
from apis.SketchMainRC import SketchMainRC
from apis.SketchDesignInfo import SketchDesignInfo
from apis.SketchOneDetail import SketchOneDetail
from apis.SketchOneLotDetail import SketchOneLotDetail
from apis.SketchOneMain import SketchOneMain
from apis.SketchOneNearestLot import SketchOneNearestLot
from apis.SketchTimeList import SketchTimeList
from apis.SketchWsgList import SketchWsgList
from apis.TakeEquipDetail import TakeEquipDetail
from apis.TakeEquipLotDetail import TakeEquipLotDetail
from apis.TakeEquipMain import TakeEquipMain
from apis.TakeOneDetail import TakeOneDetail
from apis.TakeOneEquipData import TakeOneEquipData
from apis.TakeOneMain import TakeOneMain
from apis.TakeOneRADetail import TakeOneRADetail
from apis.UpdBufInputCapacity import UpdBufInputCapacity
from apis.UpdEohInputCapacity import UpdEohInputCapacity
from dbmanager import datasource
from logmanager.setlogger import logger
from utils import config_reader
env = 'test'
app = Flask(__name__)
api = Api(app)
api.add_resource(SketchMainRC, '/SketchMainrc')
api.add_resource(SketchDesignInfo, '/SketchDesigninfo')
api.add_resource(SketchWsgList, '/Sketchwsglist')
api.add_resource(SketchMainDetail, '/Sketchmaindetail')
api.add_resource(SketchLotDetail, '/Sketchlotdetail')
api.add_resource(SketchOneMain, '/SketchOneMain')
api.add_resource(SketchOneDetail, '/SketchOnedetail')
api.add_resource(SketchOneLotDetail, '/SketchOnelotdetail')
api.add_resource(SketchOneNearestLot, '/SketchOnenearestlot')
api.add_resource(SketchDetailedLot, '/Sketchdetailedlot')
api.add_resource(SketchTimeList, '/Sketchtimelist')
api.add_resource(TakeEquipMain, '/TakeequipMain')
api.add_resource(TakeEquipDetail, '/Takeequipdetail')
api.add_resource(TakeEquipLotDetail, '/Takeequiplotdetail')
api.add_resource(TakeOneMain, '/TakeOneMain')
api.add_resource(TakeOneDetail, '/TakeOnedetail')
api.add_resource(TakeOneRADetail, '/TakeOneradetail')
api.add_resource(TakeOneEquipData, '/TakeOneequipData')
api.add_resource(ChambAvail, '/chambavail')
api.add_resource(ChambAvailBal, '/chambavailbal')
api.add_resource(ChambTwInhibit, '/chambtwinhibit')
api.add_resource(CurrentWip, '/currentwip')
api.add_resource(IncomingWip, '/incomingwip')
api.add_resource(Detail, '/detail')
api.add_resource(Manage, '/manage')
api.add_resource(MfCount, '/mfcount')
api.add_resource(OtherHold, '/otherhold')
api.add_resource(PqeDue, '/pqedue')
api.add_resource(QualInhibit, '/qualinhibit')
api.add_resource(ReleaseBef, '/releasebef')
api.add_resource(RfInhibit, '/rfinhibit')
api.add_resource(Info, '/info')
api.add_resource(InputCapacity, '/inputcapacity')
api.add_resource(InputCapacity, '/inputcapacity')
app.ls_sf_db_env = 'test'
app.ls_config = config_reader.get_config('test')
# Initialise the connection pool
datasource.initial_all_fab_engine(app)
if __name__ == '__main__':
logger.info('Starting the server')
app.run(port=5000, debug=True)
Below is my project structure.
These are just some of the APIs I completed & there are 100 more. I am new to Python, particularly to Flask-RESTFul API Development. I have separate classes for each API, I thought code looks cleaner that way. But in my app.py file, these import statements are piling up. Is there a better way to add/manage these import statements & also the add.resource() statements in a better way in my app.py file ? Could anyone let me know if there is a way I can import all these statements differently & keep my app.py cleaner ?
Any help is appreciated.

I'd do some path-based magic.
The following snippet will list all .py files in the apis/ subdir, import them as modules, extract a class that is named same as the file, and register it under /ClassName endpoint.
from importlib import import_module
from pathlib import Path
HERE = Path(__file__).parent
for api_class_file in HERE.glob("apis/*.py"):
# apis/ChambAvail.py -> ChambAvail
api_class_name = api_class_file.stem
if api_class_name.startswith("__"):
# skip special files such as __init__ and __main__
continue
# equivalent to "import apis.ChambAvail"
api_module = import_module(f"apis.{api_class_name}")
# equivalent to "apis.ChambAvail.ChambAvail
api_class = getattr(api_module, api_class_name)
api.add_resource(api_class, f"/{api_class_name}")

Cannot import class from module (circular reference issue?)

I've seen countless issues here on stack about this but still can't figure out why I cant get mine to work. I have a 2 .ipynb files and I'm looking to import a class from one file to the other as follows:
CV_Screening_Interface:
from joblib import dump, load
import sys
import pandas as pd
import os
import import_ipynb
import docx
import readDocx ***(This is another ipynb file)***
from docx import Document
import string
model = load('model.joblib')
class CV:
def __init__(self,university,major,masters,company,certification,GPA):
self.university = university
self.major = major
self.masters = masters
self.company = company
self.certification = certification
self.GPA = GPA
#And a bunch of other functions
Now in FirstProgram
from tkinter import *
from tkinter import filedialog
from docx import Document
import io
import import_ipynb
import CV_Screening_Interface
#Till here works fine
When I try to import class CV I get an import error
from CV_Screening_Interface import CV
OR
test = CV_Screening_Interface.CV()
ImportError: cannot import name 'CV' from 'CV_Screening_Interface' (CV_Screening_Interface.ipynb)
I checked PYTHONPATH, I have an empty init.py in the directory already. What's weird is that importing the module works, but importing the class in the module doesn't.
Note in CV_Screening_Interface CV class works perfectly fine so I don't think there is any issue with it specifically. Probably need a if name="main" inside it?

Ok so this solved my problems:
I created a new ipynb called ClassFile.ipynb and it only contains the Class CV without importing any packages
Then i converted ClassFile.ipynb to ClassFile.py and imported it to FirstProgram and it worked.

Python importing multiple scripts

I've been working on a python script that will require multiple libraries to be imported.
At the moment my directory structure is
program/
main.py
libs/
__init__.py
dbconnect.py
library01.py
library02.py
library03.py
My dbconnect.py which has the following contents
import psycopg2
class dbconnect:
def __init__(self):
self.var_parameters = "host='localhost' dbname='devdb' user='temp' password='temp'"
def pgsql(self):
try:
var_pgsqlConn = psycopg2.connect(self.var_parameters)
except:
print("connection failed")
return var_pgsqlConn
I am able to import and use this in my main.py using
from libs.dbconnect import dbconnect
class_dbconnect = dbconnect()
var_pgsqlConn = class_dbconnect.pgsql()
This works as expected however I am trying to import all of the library scripts each which have similar contents to bellow
def library01():
print("empty for now but this is library 01")
I have added to my __init__.py script
__all__ = ["library01", "library02"]
Then in my main.py I tried to import and use them as bellow
from libs import *
library01()
I am getting the following error
TypeError: 'module' object is not callable

I'll suppose content in your library0x.py are different (the functions/class have different names)
The best way is to import all your subfiles content in the __init__.py
# __init__.py
from .dbconnect import *
from .library01 import *
from .library02 import *
from .library03 import *
Then you can use the following :
from libs import library01, library02
If you want to restrict for some reasons importation with the wildcard (*) in your library0x.py files, you can define a __all__ variable containing all the names of the function you will import with the wildcard :
# library01.py
__all__ = ["library01"]
def a_local_function():
print "Local !"
def library01():
print "My library function"
Then, by doing from .library01 import *, only the function library01 will be import.
EDIT: Maybe i missunderstand the question : here are some ways to import the function library01 in the file library01.py :
# Example 1:
from libs.library01 import library01
library01()
# Example 2:
import libs.library01
libs.library01.library01()
# Example 3:
import libs.library01 as library01
library01.library01()

In your case library01 is a module which contains a function named library01. You import the library01 module and try to call it as a function. That's the problem. You should call the function like this:
library01.library01()