We Keep Coding

How to bundle Anaconda-installed paraview package using `pyinstaller`

Consider this python test file main.py
# main.py
from paraview.simple import *
def paraTest():
Cone()
SetProperties(Resolution=32)
Shrink()
Show()
Render()
if __name__ == '__main__':
paraTest()
that depends on paraview package installed from Anaconda using
conda install -c conda-forge paraview
according to
https://anaconda.org/conda-forge/paraview.
The program works as expected when directly run with python.
The Problem
When I package it using pyinstaller with the Terminal command
pyinstaller main.py, and then run the executable, I get the following error:
Traceback (most recent call last):
File "paraview/__init__.py", line 161, in <module>
ImportError: Failed to load vtkRemotingCore: No module named paraview.modules.vtkPVVTKExtensionsCore
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 1, in <module>
from paraview.simple import *
File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
File "paraview/__init__.py", line 163, in <module>
ModuleNotFoundError: No module named '_paraview_modules_static'
[14370] Failed to execute script 'main' due to unhandled exception!
My Attempts
I understand there may be some modules not recognized and included by pyinstaller, so I copied all module files from {miniconda_root}/miniconda3/lib/python3.9/site-packages/paraview/modules to my package {MyPackage_root}/dist/main/paraview/modules, but the same error persists.
Environment info
MacOS 11.2 (Big Sur)
python==3.9
paraview==5.9.1
pyinstaller==4.4
Thanks in advance!

Python external on Azure ImportError No Module Found

If i locally run my code with python main.py, then this error occurs locally. If i run it with python3 main.py, it works as expected. I suspect my azure app service is not running the correct version of python, however, it is set to python 3.4
StdErr:
2018-06-30 19:47:13.785205: Unhandled exception in wfastcgi.py: Traceback (most recent call last):
File "D:\Python34\Scripts\wfastcgi.py", line 711, in main
env, handler = read_wsgi_handler(response.physical_path)
File "D:\Python34\Scripts\wfastcgi.py", line 568, in read_wsgi_handler
return env, get_wsgi_handler(handler_name)
File "D:\Python34\Scripts\wfastcgi.py", line 541, in get_wsgi_handler
handler = handler()
File ".\ptvs_virtualenv_proxy.py", line 120, in get_venv_handler
handler = get_wsgi_handler(os.getenv('WSGI_ALT_VIRTUALENV_HANDLER'))
File ".\ptvs_virtualenv_proxy.py", line 89, in get_wsgi_handler
raise ValueError('"%s" could not be imported%s' % (handler_name, last_tb))
ValueError: "main.app" could not be imported: Traceback (most recent call last):
File ".\ptvs_virtualenv_proxy.py", line 73, in get_wsgi_handler
handler = __import__(module_name, fromlist=[name_list[0][0]])
File "D:\home\site\wwwroot\main.py", line 2, in <module>
from newspaper import Article
ImportError: No module named 'newspaper'
UPDATE
So I went to the debug console in kudo and i ran python
-m pip install --upgrade -r d:\home\site\wwwroot\requirements.txt.
Here is the error
ERROR: b"'xslt-config' is not recognized as an internal or external command,\r\noperable program or batch file.\r\n" ** make sure the development packages of libxml2 and libxslt are installed ** Using build configuration of libxslt
One of the requirements is Pillow which requires libxml2 and libxslt. I've tried using wheels however, I've had no success.
UPDATE II
I tried to install virtualenv since it does not exist and I receive a permissions error.
UPDATE III
I ran this command
D:\home\python364x64>python -m pip install newspaper3k
I test everything locally, and it works perfect. I push and receive this error below.
Error occurred while reading WSGI handler:
Traceback (most recent call last):
File "D:\home\python364x64\wfastcgi.py", line 791, in main
env, handler = read_wsgi_handler(response.physical_path)
File "D:\home\python364x64\wfastcgi.py", line 633, in read_wsgi_handler
handler = get_wsgi_handler(os.getenv("WSGI_HANDLER"))
File "D:\home\python364x64\wfastcgi.py", line 600, in get_wsgi_handler
handler = __import__(module_name, fromlist=[name_list[0][0]])
File ".\app.py", line 8, in <module>
from newspaper import Article
File "D:\home\python364x64\lib\site-packages\newspaper\__init__.py", line 10, in <module>
from .api import (build, build_article, fulltext, hot, languages,
File "D:\home\python364x64\lib\site-packages\newspaper\api.py", line 12, in <module>
import feedparser
File "D:\home\python364x64\lib\site-packages\feedparser.py", line 316
raise KeyError, "object doesn't have key 'category'"
^
SyntaxError: invalid syntax
StdOut:
StdErr:

Per my experience, we have no permission to modify D:\home environment.
So , It's better to use python extension. Please follow the steps in this case: Azure Flask Routes Not found and use command python -m pip install newspaper to install additional packages in python extension.
Please make sure the package you want to install must match with version of python extension. Otherwise, it will throw error as follows.
Hope it helps you.

ImportError: No module named _tkinter_finder

I am packaging my application using pyinstaller. It created the binary file, but it is not working. When I load the application on terminal I see this error
[9854] LOADER: Running BC_GUI_Client.py
Traceback (most recent call last):
File "FALCON_BC_CLIENT/BC_GUI_Client.py", line 145, in <module>
File "PIL/ImageTk.py", line 127, in __init__
File "PIL/ImageTk.py", line 192, in paste
File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyimod03_importers.py", line 687, in load_module
module = imp.load_module(fullname, fp, filename, ext_tuple)
ImportError: No module named _tkinter_finder
[9854] Failed to execute script BC_GUI_Client
I am using python 2.7 and installed pyinstaller 3.3
Anything wrong here?

I was able to resolve a simiar issue by running pyinstaller with the following option added in the command:
--hidden-import='PIL._tkinter_finder'
Your command should look like this:
pyinstaller module.py --hidden-import='PIL._tkinter_finder'

configparser not working in Python 3.4, NoSectionError but works fine in PyCharm

I worked on a Python 3.4 script in PyCharm 4.5. (repo: https://github.com/Djidiouf/bbot )
In it, I used import configparser without any problem for retrieving some values in a config.cfg:
config = configparser.RawConfigParser()
config.read('config.cfg')
server = config.get('bot_configuration', 'server')
channel = config.get('bot_configuration', 'channel')
botnick = config.get('bot_configuration', 'botnick')
port = config.getint('bot_configuration', 'port')
Now, I want to deploy it on a Debian 8.1 server. On that server, I have both Python 2.7 and Python 3.4 versions installed.
But when I ran the script on the machine, I have the following issue:
djidiouf#linuxserver:/home/djidiouf# /usr/bin/python3.4 /home/djidiouf/bbot/bbot.py
Traceback (most recent call last):
File "/usr/lib/python3.4/configparser.py", line 1116, in _unify_values
sectiondict = self._sections[section]
KeyError: 'bot_configuration'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/djidiouf/bbot/bbot.py", line 25, in <module>
import modules.steam # Contains specific Steam-Valve related functions
File "/home/djidiouf/bbot/modules/steam.py", line 13, in <module>
import modules.connection
File "/home/djidiouf/bbot/modules/connection.py", line 37, in <module>
server = config.get('bot_configuration', 'server')
File "/usr/lib/python3.4/configparser.py", line 754, in get
d = self._unify_values(section, vars)
File "/usr/lib/python3.4/configparser.py", line 1119, in _unify_values
raise NoSectionError(section)
configparser.NoSectionError: No section: 'bot_configuration'
I made a lot of research on Internet about that but I didn't find anything valuable. Some people made mention about a corrupt config.cfg file but mine is a LF EOL file with UTF8 without BOM as needed. I also recreated it again with vi on the machine.
I also tried to install configparser with pip (even if, if I understand it well, it's now build-in in Python 3) : pip3 install configparser But when I do that, I have the following error:
djidiouf#linuxserver:/home/djidiouf# sudo pip3 install -v configparser
[sudo] password for djidiouf:
Downloading/unpacking configparser
Ignoring link https://pypi.python.org/packages/source/c/configparser/configparser-3.5.0b1.tar.gz#md5=d60ca2c714acb4adaf5818c6a1ffd78b (from https://pypi.python.org/simple/configparser/), version 3.5.0b1 is a pre-release (use --pre to allow).
Ignoring link https://pypi.python.org/packages/source/c/configparser/configparser-3.5.0b2.tar.gz#md5=ad2a71db8bd9a017ed4735eac7acfa07 (from https://pypi.python.org/simple/configparser/), version 3.5.0b2 is a pre-release (use --pre to allow).
Using version 3.3.0r2 (newest of versions: 3.3.0r2, 3.3.0r1, 3.2.0r3, 3.2.0r2, 3.2.0r1)
Downloading configparser-3.3.0r2.tar.gz
Downloading from URL https://pypi.python.org/packages/source/c/configparser/configparser-3.3.0r2.tar.gz#md5=dda0e6a43e9d8767b36d10f1e6770f09 (from https://pypi.python.org/simple/configparser/)
Running setup.py (path:/tmp/pip-build-v7av3db7/configparser/setup.py) egg_info for package configparser
Traceback (most recent call last):
File "<string>", line 3, in <module>
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 5, in <module>
import distutils.core
File "/usr/lib/python3.4/distutils/core.py", line 18, in <module>
from distutils.config import PyPIRCCommand
File "/usr/lib/python3.4/distutils/config.py", line 7, in <module>
from configparser import ConfigParser
File "/tmp/pip-build-v7av3db7/configparser/configparser.py", line 397
_KEYCRE = re.compile(ur"%\(([^)]+)\)s")
^
SyntaxError: invalid syntax
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 3, in <module>
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 5, in <module>
import distutils.core
File "/usr/lib/python3.4/distutils/core.py", line 18, in <module>
from distutils.config import PyPIRCCommand
File "/usr/lib/python3.4/distutils/config.py", line 7, in <module>
from configparser import ConfigParser
File "/tmp/pip-build-v7av3db7/configparser/configparser.py", line 397
_KEYCRE = re.compile(ur"%\(([^)]+)\)s")
^
SyntaxError: invalid syntax
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip-build-v7av3db7/configparser
Exception information:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 290, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/usr/lib/python3/dist-packages/pip/req.py", line 1230, in prepare_files
req_to_install.run_egg_info()
File "/usr/lib/python3/dist-packages/pip/req.py", line 326, in run_egg_info
command_desc='python setup.py egg_info')
File "/usr/lib/python3/dist-packages/pip/util.py", line 716, in call_subprocess
% (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command python setup.py egg_info failed with error code 1 in /tmp/pip-build-v7av3db7/configparser
Storing debug log for failure in /root/.pip/pip.log
Here is a config example, I just changed the values and rename the file accordingly to config.cfg :
[bot_configuration]
server = your.irc.server.com
channel = ##your_channel
botnick = bbot
port = 6667
admins = You,Him
https://github.com/Djidiouf/bbot/blob/master/config_example.cfg
Finally, I just want to say that everything is working on PyCharm 4.5, I just made a git clone, rename the config_example.cfg, change the values, tried to run the script with Python 3.4.

The install error is from _KEYCRE = re.compile(ur"%\(([^)]+)\)s"), that is causing a syntax error as the ur prefix is not supported in python3, pip is trying to instal a python2 package.
Your first error is a keyError 'bot_configuration' which is not because of not having configparser installed, configparser is part of the standard library.
The bot_configuration KeyError is because python cannot find your config file, if you use config.read('foobar') in pycharm you will see the exact same output

I figured out that I had some inconsistency about running the script. When I was in another folder than my script, nothing worked but when I was in it, it worked. Perhaps because the config file is shared and used in the main script and in a module.
In fact it's due to the need to have an absolute path in config.read('/some/path/file.cfg')
Finally, I find that answer: https://stackoverflow.com/a/13800583/3301370
So here is the solution in the case of config.cfg being on the project root directory /config.cfg :
/bbot.py
import os
config.read(os.path.join(os.path.abspath(os.path.dirname(__file__)), 'config.cfg'))
However, as the config file is shared and used in the main script and in a subpath, module, a correct path must be set for the sub module too:
/modules/connection.py
import os
config.read(os.path.join(os.path.abspath(os.path.dirname(__file__)), '..', 'config.cfg'))
It works fine in PyCharm because when you hit the run button for your script, it runs it with the path of the project. The fact that having a relative path for both config.read() statements worked only because of that.
The problem with installing configparser with pip is unrelated to that specific problem and can be ignore in that case as configparser is a built-in library in Python3.

import jenkins in python fails on mac

I installed the jenkins package on a mac using sudo easy_install python-jenkins. The installation went fine, no errors, but when I open python and type in import jenkins I get the following error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.macosx-10.7-intel/egg/jenkins.py", line 9, in <module>
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 431, in LoadLibrary
return self._dlltype(name)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 353, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Library/Python/2.7/site-packages/lookup3.so, 6): image not found
Googling this did not yield any helpful result.
Traceback after Editing
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "jenkins.py", line 9, in <module>
lookup3 = cdll.LoadLibrary(os.path.join(get_python_lib(), "lookup3.dynlib"))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 431, in LoadLibrary
return self._dlltype(name)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 353, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Library/Python/2.7/site-packages/lookup3.dynlib, 6): image not found

The module in question isn't compatible with Mac OS X dynamic library conventions.
It tries to load a lookup3.so file, but on Mac it'll be lookup3.dynlib instead. You can verify this by looking at the /Library/Python/2.7/site-packages directory. Then just edit the jenkins.py module and replace the .so extension with .dynlib.
I'd also report this to the issue tracker of the module; a simple if platform.system() == 'Darwin': switch would keep things loadable across platforms.
Update: Actually, the module stubbornly expects the lookup3.so module to be located in the site-packages directory instead of with the module itself. It is thus not easy_install compatible. It'll work when you move all the files (unaltered) out of the .egg directory straight into your /Library/Python/2.7/site-packages/ directory.

OSError: dlopen(/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/lookup3.so, 6): image not found
From the directory,
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages,
ln -s lookup3.cpython-36m-darwin.so lookup3.so
worked for me.

You can try:
pip uninstall jenkins
pip install python-jenkins
worked for me.

I found the lookup.so jenkins.py is trying to load in:
/Users/MY-USER/Documents/Telnyx/dev/tdm-get-ips/venv/lib/python3.7/site-packages/lookup3.cpython-37m-darwin.so
so i changed
python3.7/site-packages/jenkins.py
Line:
lookup3 = cdll.LoadLibrary(os.path.join(get_python_lib(), "lookup3.so"))
for:
lookup3 = cdll.LoadLibrary(os.path.join(get_python_lib(), "lookup3.cpython-37m-darwin.so"))
And it started working properly.