We Keep Coding

Python - Is there way to refer ghostscript's DLLs without installing?

I found this link for Magick.NET
Is there way to refer ghostscript's DLLs without installing?
It mentions function:
MagickNET.SetGhostscriptDirectory
Is there an equivalent for Python?
I tried setting path, but in python program, it fails to load gsdll64.dll when copied to other machine:
My code:
import os, time
import sys
import camelot.io as camelot
import traceback
sys.path.insert(0, r'C:\gs\gs9.56.1\bin')
sys.path.insert(0, r'C:\gs\gs9.56.1\lib')
print('path',sys.path)
#check ghostscript lib - as done in ghostscript_backend.py
import ctypes
from ctypes.util import find_library
mylib = find_library("".join(("gsdll", str(ctypes.sizeof(ctypes.c_voidp) * 8), ".dll")))
print(mylib,os.getcwd())
if mylib is None:
print('gsdll not loaded')

In site-packages\ghostscript\_gsprint.py
The code is written to get dll path from windows registry.
In function __win32_finddll, I added code to check and load dll from my folder:
LooseVrsn = "9.56.1" #from dll->Properties->details->ProductVersion
dll_path = os.path.join(os.getcwd(),'gsdll64.dll')
dlls.append((LooseVrsn, dll_path))

How to resolve this: pyreportjasper: ImportError: DLL load failed while importing jpy: The specified module could not be found

I want to use Jasper Reports with a python-flask app.
I have followed the installation instructions on this page:
https://pypi.org/project/pyreportjasper/
It fails on the line import jpy, I am getting the error:
ImportError: DLL load failed while importing jpy: The specified module could not be found.
I am using 32 bit python Python: 3.8.6rc1.
32 bit JDK: javac 1.8.0_261
When I run pip install pyreportjasper, it says: Requirement already satisfied: jpy in c:\users....
Any ideas?

It seems that you need to import jpyutil and then initialize the JVM prior to calling import jpy. The following fixed this error for me. One step closer.
import jpyutil
jpyutil.init_jvm(jvm_maxmem='512M')
import jpy
UPDATE: so while the above statement is true, it doesn't directly address the issue of running the example code at the project page: https://pypi.org/project/pyreportjasper/
As I dug into the code I found that jasperpy.py is attempting to import jpy BEFORE the __init__ method is run and therefore BEFORE the JVM is initialized via the jpyutil.init_jvm() method call. So I edited jasperpy.py and moved import jpy from line 14 to line 56, directly after the call to jpyutil.init_jvm() and before the call to jpy.get_type('java.io.File') and I am able to successfully run Jasper reports from python via the example code shown. Here is what my jasperpy.py looks like now.
# -*- coding: utf-8 -*-
# GNU GENERAL PUBLIC LICENSE
#
# Copyright (c) 2020 Jadson Bonfim Ribeiro <contato#jadsonbr.com.br>
#
import os
import subprocess
import re
import xml.etree.ElementTree as ET
import tempfile
import jpyutil
#import jpy
import json
from requests import Request, Session
FORMATS = (
'pdf',
'rtf',
'xls',
'xlsx',
'docx',
'odt',
'ods',
'pptx',
'csv',
'html',
'xhtml',
'xml',
'jrprint',
)
EXECUTABLE = 'jasperstarter'
class JasperPy:
_FORMATS_JSON = ('pdf')
_FORMATS_METHODS_REQUEST = ('GET', 'POST', 'PUT')
def __init__(self, resource_dir=False, jvm_maxmem='512M', jvm_classpath=None):
self.WINDOWS = True if os.name == 'nt' else False
self.SCRIPT_DIR = os.path.abspath(os.path.dirname(__file__))
self.LIBS = os.path.join(self.SCRIPT_DIR, 'jasperstarter', 'lib')
if not os.path.isdir(self.LIBS):
raise NameError('Unable to find lib in {0}'.format(self.LIBS))
self.CLASSPATH = os.path.join(self.LIBS, 'jasperstarter.jar')
if not os.path.exists(self.CLASSPATH):
raise NameError('Unable to find jasperstarter in {0}'.format(self.LIBS))
if jvm_classpath is None:
jpyutil.init_jvm(jvm_maxmem=jvm_maxmem, jvm_classpath=[self.CLASSPATH])
else:
jpyutil.init_jvm(jvm_maxmem=jvm_maxmem, jvm_classpath=[self.CLASSPATH, jvm_classpath])
# IMPORT jpy HERE AFTER init_jvm
import jpy
self.jvFile = jpy.get_type('java.io.File')
self.jvArrays = jpy.get_type('java.util.Arrays')
self.jvReport = jpy.get_type('de.cenote.jasperstarter.Report')
self.jvConfig = jpy.get_type('de.cenote.jasperstarter.Config')
self.jvDsType = jpy.get_type('de.cenote.jasperstarter.types.DsType')
self.jvApplicationClasspath = jpy.get_type('de.cenote.tools.classpath.ApplicationClasspath')
self.jvHashMap = jpy.get_type('java.util.HashMap')
self.jvLocale = jpy.get_type('java.util.Locale')
self.jvJasperFillManager = jpy.get_type('net.sf.jasperreports.engine.JasperFillManager')
self.jvDb = jpy.get_type('de.cenote.jasperstarter.Db')
self.jvJsonQueryExecuterFactory = jpy.get_type('net.sf.jasperreports.engine.query.JsonQueryExecuterFactory')
self.jvJasperExportManager = jpy.get_type('net.sf.jasperreports.engine.JasperExportManager')
.
.
.
.
This file is located in your Python directory under Lib\site-packages\pyreportjasper-2.0.2-py3.8.egg\pyreportjasper. While this is currently a hack to make it work, there obviously needs to be a fix to the package which I will attempt to address on the pyreportjasper github page.

How to set different keras backend in different conda environments

How do I set different keras backends in different conda environments? Because in a specific environment, if I change the backend to tensorflow in keras.json, then in another python environment, keras backend will be tensorflow too. There is only one keras.json in my documents.

For using different keras backends in different environments in Anaconda - 'env1' and 'env2'
Activate the first environment 'env1'
Import keras from python with the default backend (if it fails to load tensorflow for example, install tensorflow in that environment) successfully
In the ~ folder, a '.keras' folder will be created which will contain keras.json file
For the other environment create a copy of the '.keras' folder as '.keras1'
Change the keras.json file in that folder as per requirements (the 'backend' field)
For using that config in 'env2' go to '~/anaconda3/envs/env2/lib/pythonx.x/site-packages/keras/backend' and edit the __init__.py file
Make the changes marked with ##
You will be able to import keras with different backends in env1 and env2
from __future__ import absolute_import
from __future__ import print_function
import os
import json
import sys
import importlib
from .common import epsilon
from .common import floatx
from .common import set_epsilon
from .common import set_floatx
from .common import cast_to_floatx
from .common import image_data_format
from .common import set_image_data_format
# Set Keras base dir path given KERAS_HOME env variable, if applicable.
# Otherwise either ~/.keras or /tmp.
if 'KERAS_HOME' in os.environ:
_keras_dir = os.environ.get('KERAS_HOME')
else:
_keras_base_dir = os.path.expanduser('~')
if not os.access(_keras_base_dir, os.W_OK):
_keras_base_dir = '/tmp'
_keras_dir = os.path.join(_keras_base_dir, '.keras1')##
# Default backend: TensorFlow.
_BACKEND = 'tensorflow'
# Attempt to read Keras config file.
_config_path = os.path.expanduser(os.path.join(_keras_dir, 'keras.json'))
if os.path.exists(_config_path):
try:
with open(_config_path) as f:
_config = json.load(f)
except ValueError:
_config = {}
_floatx = _config.get('floatx', floatx())
assert _floatx in {'float16', 'float32', 'float64'}
_epsilon = _config.get('epsilon', epsilon())
assert isinstance(_epsilon, float)
_backend = _config.get('backend', _BACKEND)
_image_data_format = _config.get('image_data_format',
image_data_format())
assert _image_data_format in {'channels_last', 'channels_first'}
set_floatx(_floatx)
set_epsilon(_epsilon)
set_image_data_format(_image_data_format)
_BACKEND = _backend
# Save config file, if possible.
if not os.path.exists(_keras_dir):
try:
os.makedirs(_keras_dir)
except OSError:
# Except permission denied and potential race conditions
# in multi-threaded environments.
pass
if not os.path.exists(_config_path):
_config = {
'floatx': floatx(),
'epsilon': epsilon(),
'backend': _BACKEND,
'image_data_format': image_data_format()
}
try:
with open(_config_path, 'w') as f:
f.write(json.dumps(_config, indent=4))
except IOError:
# Except permission denied.
pass
# Set backend based on KERAS_BACKEND flag, if applicable.
if 'KERAS_BACKEND' in os.environ:
_backend = os.environ['KERAS_BACKEND']
_BACKEND = _backend
# Import backend functions.
if _BACKEND == 'cntk':
sys.stderr.write('Using CNTK backend\n')
from .cntk_backend import *
elif _BACKEND == 'theano':
sys.stderr.write('Using Theano backend.\n')
from .theano_backend import *
elif _BACKEND == 'tensorflow':
sys.stderr.write('Using TensorFlow backend.\n')
from .tensorflow_backend import *
else:
# Try and load external backend.
try:
backend_module = importlib.import_module(_BACKEND)
entries = backend_module.__dict__
# Check if valid backend.
# Module is a valid backend if it has the required entries.
required_entries = ['placeholder', 'variable', 'function']
for e in required_entries:
if e not in entries:
raise ValueError('Invalid backend. Missing required entry : ' + e)
namespace = globals()
for k, v in entries.items():
# Make sure we don't override any entries from common, such as epsilon.
if k not in namespace:
namespace[k] = v
sys.stderr.write('Using ' + _BACKEND + ' backend.\n')
except ImportError:
raise ValueError('Unable to import backend : ' + str(_BACKEND))
def backend():
"""Publicly accessible method
for determining the current backend.
# Returns
String, the name of the backend Keras is currently using.
# Example
```python
>>> keras.backend.backend()
'tensorflow'
```
"""
return _BACKEND

Here is what I did for my own purposes, same logic as Kedar's answer, but on a Windows install (and Keras version) for which locations and file names may differ :
1/ Set a specific keras.json file, in a folder of your targeted Anaconda environment. Modify the "backend" value.
2/ Then force the 'load_backend.py' (the one specific to your anaconda env.) to load this specific keras.json.
Also, force de "default backend" to the one you want in that very same file.
=======================================================
IN DETAILS :
1.1 Open the Anaconda environment folder for which you want a specific backend. In my case it's C:\ProgramData\Anaconda3\envs\[MyAnacondaEnvironment]\
1.2 Here create a folder .keras, and in that folder copy or create a file keras.json (I copied mine from C:\Users\[MyWindowsUserProfile]\.keras\keras.json).
Now in that file, change the backend for the one you want, I've chosen 'cntk' for some tests. The file's content should now look like that :
{
"floatx": "float32",
"epsilon": 1e-07,
"backend": "cntk",
"image_data_format": "channels_last"
}
And the file's name and location looks like C:\ProgramData\Anaconda3\envs\[MyAnacondaEnvironment]\.keras\keras.json
2.1 Now open the file 'load_backend.py' specific to the environment you are customizing, located here (in my case) C:\ProgramData\Anaconda3\envs\[MyAnacondaEnvironment]\Lib\site-packages\keras\backend
2.2 Here line 17 to 25 in my Keras version (2.3.1), the file usually loads the backend from the configuration file it locates with the help of your environment variables or of your current Windows user for instance. That's why currently your backend is cross environment.
Get rid of this by forcing 'load_backend.py' to look which backend to load directly in your environment specific configuration file (the one you created at step 1.2)
For instance, line 26 of that 'load_backend.py' file (line 26 in my case, anyway right after the attempt to load the configuration file automatically) add that line (and customize it for your own location) :
_keras_dir = 'C:\ProgramData\Anaconda3\envs\[MyAnacondaEnvironment]\.keras' ##Force script to get configuration from a specific file
3.1 Then replace (line 28 in my case, anyway right after you forced _keras_dir path) the default backend _BACKEND = 'tensorflow' by _BACKEND = 'cntk'.
You should be done

One solution is to create different users for different environments and put different keras.json files for both:
$HOME/.keras/keras.json
This way you'll be able to change any keras parameter independently.
If you only need to change the backend, it is easier to use KERAS_BACKEND env variable. The following command will use tensorflow, not matter what's in keras.json:
$ KERAS_BACKEND=tensorflow python -c "from keras import backend"
Using TensorFlow backend.
So can you start a new shell terminal, run export KERAS_BACKEND=tensorflow in it and all subsequent commands will use tensorflow. You can go further and set this variable per conda env activation as discussed in this question (if you need it permanently):
$PREFIX/etc/conda/activate.d

How can I import one Jupyter notebook into another

Apparently it's possible to import one Jupyter notebook into another. The linked page has quite a bit of code to do it. Am I supposed to add that code to the importing notebook? The page isn't clear about it. It's supposed to be a general solution, so it doesn't make sense to add all that code to all notebooks that import other notebooks. Any help would be appreciated. Thanks.

Yes, you can add all of that code to a notebook if you want.
And yes, you shouldn't do so as a general solution.
A notebook is a complicated structure, as alluded to in the details of the text (I think it's JSON). It can contain python code, it can contain magics - cython, bash, latex and more - which would not be understood by the Python kernel. Essentially you have to replicate a portion of the functionality of the normal Python import process, as natively Python won't understand there is Python code inside an Ipython notebook.
However ... normally if you have a significant amount of Python code you would split it into modules, and then import the modules. These work as normal, because it is a normal Python import.
For example, once the code has been loaded to tell it how to understand what a notebook is, the actual import is only
import nbpackage.mynotebook
We can use the same technique with the module import code - find_notebook and NotebookLoader can be put into a helper module (e.g. helper.py), and all you would have to do is, from within your notebook, use from helper import NotebookFinder.
I suspect you'd still have to call sys.meta_path.append(NotebookFinder()) from inside your notebook along with the import.
Here is a specific example of how you can use the import capabilities to create an API drawn from a notebook:
Create a notebook. We'll call it scanner.ipynb:
import os, sys
def scanner(start):
for root, dirs,files in os.walk(start):
# remove any already processed file
if 'done' in dirs:
dirs.remove('done')
for names in files:
name, ext = os.path.splitext(names)
# only interested in media files
if ext == '.mp4' or ext == '.mkv':
print(name)
Create a regular python file called reuse.py. This is your general re-usable Ipython import module:
#! /usr/env/bin python
# *-* coding: utf-8 *-*
import io, os, sys, types
from IPython import get_ipython
from nbformat import read
from IPython.core.interactiveshell import InteractiveShell
def find_notebook(fullname, path=None):
"""find a notebook, given its fully qualified name and an optional path
This turns "foo.bar" into "foo/bar.ipynb"
and tries turning "Foo_Bar" into "Foo Bar" if Foo_Bar
does not exist.
"""
name = fullname.rsplit('.', 1)[-1]
if not path:
path = ['']
for d in path:
nb_path = os.path.join(d, name + ".ipynb")
if os.path.isfile(nb_path):
return nb_path
# let import Notebook_Name find "Notebook Name.ipynb"
nb_path = nb_path.replace("_", " ")
if os.path.isfile(nb_path):
return nb_path
class NotebookLoader(object):
"""Module Loader for Jupyter Notebooks"""
def __init__(self, path=None):
self.shell = InteractiveShell.instance()
self.path = path
def load_module(self, fullname):
"""import a notebook as a module"""
path = find_notebook(fullname, self.path)
print ("importing Jupyter notebook from %s" % path)
# load the notebook object
with io.open(path, 'r', encoding='utf-8') as f:
nb = read(f, 4)
# create the module and add it to sys.modules
# if name in sys.modules:
# return sys.modules[name]
mod = types.ModuleType(fullname)
mod.__file__ = path
mod.__loader__ = self
mod.__dict__['get_ipython'] = get_ipython
sys.modules[fullname] = mod
# extra work to ensure that magics that would affect the user_ns
# actually affect the notebook module's ns
save_user_ns = self.shell.user_ns
self.shell.user_ns = mod.__dict__
try:
for cell in nb.cells:
if cell.cell_type == 'code':
# transform the input to executable Python
code = self.shell.input_transformer_manager.transform_cell(cell.source)
# run the code in themodule
exec(code, mod.__dict__)
finally:
self.shell.user_ns = save_user_ns
return mod
class NotebookFinder(object):
"""Module finder that locates Jupyter Notebooks"""
def __init__(self):
self.loaders = {}
def find_module(self, fullname, path=None):
nb_path = find_notebook(fullname, path)
if not nb_path:
return
key = path
if path:
# lists aren't hashable
key = os.path.sep.join(path)
if key not in self.loaders:
self.loaders[key] = NotebookLoader(path)
return self.loaders[key]
Create your specific API file that connects the loader above with the notebook above. Call it scan_api.py:
# Note the python import here
import reuse, sys
# This is the Ipython hook
sys.meta_path.append(reuse.NotebookFinder())
import scanner
# And now we can drawn upon the code
dir_to_scan = "/username/location"
scanner.scanner(dir_to_scan)

Simple Solution with 2 lines of code
Use 'nbimporter' package in python for importing another notebook A (or its function) in notebook B. You can install 'nbimporter' by using the command - pip install nbimporter
Assume there are two Notebooks A.ipynb and B.ipynb. we are trying to import Notebook A (or its function) inside B Code sample below should solve the issue.
Inside Notebook B.ipynb
import nbimporter
import A # or do this --> from A import func1

From *folder_name* import *variable* Python 3.4.2

File setup:
...\Project_Folder
...\Project_Folder\Project.py
...\Project_folder\Script\TestScript.py
I'm attempting to have Project.py import modules from the folder Script based on user input.
Python Version: 3.4.2
Ideally, the script would look something like
q = str(input("Input: "))
from Script import q
However, python does not recognize q as a variable when using import.
I've tried using importlib, however I cannot figure out how to import from the Script folder mentioned above.
import importlib
q = str(input("Input: "))
module = importlib.import_module(q, package=None)
I'm not certain where I would implement the file path.

Repeat of my answer originally posted at How to import a module given the full path?
as this is a Python 3.4 specific question:
This area of Python 3.4 seems to be extremely tortuous to understand, mainly because the documentation doesn't give good examples! This was my attempt using non-deprecated modules. It will import a module given the path to the .py file. I'm using it to load "plugins" at runtime.
def import_module_from_file(full_path_to_module):
"""
Import a module given the full path/filename of the .py file
Python 3.4
"""
module = None
try:
# Get module name and path from full path
module_dir, module_file = os.path.split(full_path_to_module)
module_name, module_ext = os.path.splitext(module_file)
# Get module "spec" from filename
spec = importlib.util.spec_from_file_location(module_name,full_path_to_module)
module = spec.loader.load_module()
except Exception as ec:
# Simple error printing
# Insert "sophisticated" stuff here
print(ec)
finally:
return module
# load module dynamically
path = "<enter your path here>"
module = import_module_from_file(path)
# Now use the module
# e.g. module.myFunction()

I did this by defining the entire import line as a string, formatting the string with q and then using the exec command:
imp = 'from Script import %s' %q
exec imp