I want to install SciPy, NumPy and MatplotLib globally so that the libraries can be accessed from network computers. Basically it should be in some network drive without full installation of the libraries on each remote computer.
Now I copied the site-package directory from one working installation into the network drive, but not surprisingly I now get "DLL load failed" (multiarray) errors when trying to import scipy from remote computers. I suppose there are DLLs missing.
What would be the easiest solution to this problem? Does anyone know which DLLs I need to copy to all remote computers or can I even have these DLLs on the network drive too inside PYTHONPATH?
In the docs, see : http://docs.python.org/install/index.html
"Alternate installation: Windows (the prefix scheme)"
"Modifying Python’s Search Path"
"Custom Installation"
may help you.
There is also the PYTHONHOME var used during install.
But frankly, I'm afraid you will not succeed doing this because the packages are compiled during installation, and compilers depends on CPU, vary from one PC to another (one has the minGW, other has the intel, third one has the borland ... ) and I think python packages do not use as much place, so that it makes sense to install on each computer, more than trying to deal with ONE install for everyone.
Unless you make sure everybody has the same C and fortran compilers... but even so, it will not be easy, i'm afraid.
Related
For my work I need to write a GUI using PySide6 for a remote system. The OS is RHEL 7.9 and I have neither admin privileges nor PIP working (blocked by admins), so i can't install anything by myself (and i'm not allowed to anyways).
The script runs perfectly on Windows and Fedora, but it doesn't work on RHEL 7.9:
Since the machine doesn't allow pip, I've included PySide6 in my virtual environment, but there are missing libraries in the system itself, like CXXABI_1.3.9 and GLIBC_2.33 that Shiboken6 needs.
It also didn't work in compiled form (with PyInstaller) because the GLIBC_2.29 is missing.
Naively I copied libstdc++.so.6 and libc.so.6 from a Fedora machine to RHEL and redirected the linking to the libraries with the LD_LIBRARY_PATH environment variable, but because of other dependencies it didn't work.
Is there a solution to make the script work cross-platform and independently?
This won't answer your question about the missing libraries, but I hope it helps solve your current issue with PySide.
I've had a similar problem before, you should always develop on the target platform to get comparable results.
This means that you theoretically have to write, compile and package your program on the RHEL machine. You also need to always develop on the older platform. Forward compatibility is not always guaranteed. I therefore suggest, that you install CentOS 7 in a virtual machine and if your program is not too complicated try to use PySide2 instead of PySide6.
One way may be to put the (binary) files in the user bin.
The path for that (at least on my system) is /home/.local/bin
Make sure this is in your $PATH variable, if it isn't already (it should be).
If these are just Python modules, you can just drag the source file into the aforementioned path (make sure they have a shebang so they run). If not, you may have to compile them and then put them there.
Explaination of local bin
Note: This is from my experience
The local bin (/home/.local/bin) is where you can put programs you want to run without installing them globally. It's similar to doing ./file.py, except you don't need to be in the same directory as the file (and you don't have to include ./) I assume, if you include a compiled version of the dependencies you want, it should work.
On my work computer, I do not have permissions to install new software, edit PATH, etc. I have Python installed to local folder (and everything works, including pip to install and update packages. But, I cannot install the Cuda Computing Toolkit.
On another computer that I have those permissions, I have tensorflow installed, verified, and by using ProcessExplorer I know what 3 DLLs I need to have installed for tensorflow to work with my Python app.
Typically, when I have this problem, I have found it is acceptable to install DLLs in the local path relative to the binary loading/requiring them; for instance: I put the Visual Studio runtime DLL in the same folder as python.exe and I do not need to actually run the installer.
However, I cannot figure out where the cuda DLLs need to go relative to tensorflow for them to be loaded. I have searched the source with no luck. I have set breakpoints in a Python debugger with no luck. Can anybody point me to where the DLLs need to go, or at least where they are loaded?
I am new to Python having come from a proprietary compiled language (Xojo) that produces self-contained executables.
I understand that Python is an interpreted language. I understand that it requires an interpreter (let’s stick with CPython) and presumably it requires a number of accessory frameworks/C libraries in order to run. What I don’t understand is why is it so hard to create a folder containing the interpreter and all required files and libraries and simply bundle these up with my script to distribute.
I have discovered that there are a bunch of tools that attempt to do this (py2app, cx_freeze, etc) but many of them seem either broken, not maintained or really buggy.
I guess my question is: is there any documentation that describes the exact things I need to bundle with a “Hello World” script to get it running? This seems to be a really straightforward problem to solve but it hasn’t been (which suggests that it is far more complex than I appreciate).
My understanding is that PyInstaller works fine for making a single exe for distribution. But barring packaging tools like that, in general, there isn't an obvious "bare minimum"; the modules don't have documented dependencies, so it's usually best to ship the whole standard library.
Typically, if you need a redistributable version, you use the embedded Python zip redistributable, shipping Python alongside your main application.
The exact list of files/libraries depends on how the python interpreter is built. In windows for example, you can obtain CPython binaries built from Visual Studio, Cygwin and Mingw-w64. They have different dependency of cause. In Linux distributions, python is normally installed by default.
Below is the list of .dll and .exe files that you can find in the official CPython release for windows.
libcrypto-1_1-x64.dll python.exe python37.dll sqlite3.dll
libssl-1_1-x64.dll pythonw.exe python3.dll vcruntime140.dll
The total size of this ZIP file release is only 6.7 MB. So it would be easy to bundle it in your main executable. You can use whatever bundler at hand, not necessary those designed for python. Quoting from the documentation here:
extracting the embedded distribution to a subdirectory of the application installation is sufficient to provide a loadable Python interpreter.
I feel the absolute best way to experience Python for beginners in thonny and an esp32.
A very good way to get started with python is to use Anaconda https://www.anaconda.com/distribution/#download-section - this distribution contains the CPython interpreter and the most commonly used packages. For quite a while you will get along without installing more packages.
To be able to make a simple distributable piece of code just include a requirements.txt along with your code which should list down the packages (and versions) you are using in your code.
More on that here : https://www.idiotinside.com/2015/05/10/python-auto-generate-requirements-txt/
pip freeze generates a superset of all packages in your running environment so you would ideally go with the second smarter option in the link : pipreqs
So, in short along with your code just an additional requirements.txt should be fine using which people can install all required packages as
pip install -r requirements.txt
and they are good to go to run your code.
For advanced scenarios you might want to look up creating virtual environments using conda.
What is a conda environment?
https://docs.conda.io/projects/conda/en/latest/user-guide/concepts.html#conda-environments
How to create/manage a conda environment
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
All the best in your Python journey!
I thought it is an easy question but I spent a lot of google time to find the answer with no luck. Hope you can help me.
My company has a large SW system on windows which is portable, meaning copy some folders, add some folder to windows path and you are ready to go.
No registry, no dll in system directory, no shortcuts, Nothing!
I want to start using python 3.x in our system in the same paradigm. I also want the ability to add to this distribution a pip/conda 3rd packages from time to time.
I don't want to install python msi on all the systems.
I don't want to pack it to standalone executable like py2exe and pyinstaller or use special python distribution like PyWin32.
Somehow, I couldn't find a formal official solution for that.
The closest thing was here but no pip is supported, python is minimal, and the system isolation is "almost".
3.8. Embedded Distribution New in version 3.5.
The embedded distribution is a ZIP file containing a minimal Python
environment. It is intended for acting as part of another application,
rather than being directly accessed by end-users.
When extracted, the embedded distribution is (almost) fully isolated
from the user’s system, including environment variables, system
registry settings, and installed packages. The standard library is
included as pre-compiled and optimized .pyc files in a ZIP, and
python3.dll, python36.dll, python.exe and pythonw.exe are all
provided. Tcl/tk (including all dependants, such as Idle), pip and the
Python documentation are not included.
Note The embedded distribution does not include the Microsoft C
Runtime and it is the responsibility of the application installer to
provide this. The runtime may have already been installed on a user’s
system previously or automatically via Windows Update, and can be
detected by finding ucrtbase.dll in the system directory. Third-party
packages should be installed by the application installer alongside
the embedded distribution. Using pip to manage dependencies as for a
regular Python installation is not supported with this distribution,
though with some care it may be possible to include and use pip for
automatic updates. In general, third-party packages should be treated
as part of the application (“vendoring”) so that the developer can
ensure compatibility with newer versions before providing updates to
users.
Any ideas?
Thanks.
How about... installing Python in one machine and replicate that installation on others computers?
Usually, I install Python in a Windows Virtualbox machine (Microsoft usually give it for free to try it or for testing old Internet Explorer versions).
Then I copy the Python directory to my Windows machine (the real host) and usually works. This makes possible to using various python versions.
Did you try to complete the Python Embedded Distribution? Usually they not come with Tkinter, but once I could copy files and put in this distribution in a way that works. Try it too.
You can install pip with get-pip.py
I downloaded Snappy library sources for working with compression and everything was great on one machine, but it didn't work on another machine. They have completely same configurations of hardware/OS + python 2.7.3.
All I was doing is "./configure && make && make install".
There were 0 errors during any of these processes and it installed successfully to the default lib directory, but python cant see it anyhow. help('modules') and pip freeze doesn't show snappy on the second machine and as the result I cant import it.
I tried even 'to break' structure and install it to different lib catalogs, but even that didn't work. I don't think if its related to system environment variables, since python should have completely same configuration on any of these machines (Amazon EC2).
Anyone knows how to fix this issue?
I just found python-snappy on github and installed it via python. Not a permanent solution, but at least something.