I'm building a C++ python module on MacOS.
On my machine I have python, installed in directory /Library/Frameworks/Python.framework/Versions/3.9. I'm specifing python includes directory as -I/Library/Frameworks/Python.framework/Versions/3.9/Headers and libraries directory as -L/Library/Frameworks/Python.framework/Versions/3.9/lib -lpython3.9.
So it builds good and runs good on my machine and other machines, where python is installed into /Library/Frameworks/Python.framework/Versions/3.9.
But when I move my library to machine, where python is installed into another folder, like /usr/local/Cellar/python#3.9/3.9.12/Frameworks/Python.framework/Versions/3.9, I'm getting import module error.
otool -L _mymodule.so command returns hardcoded absolute path to /Library/Frameworks/Python.framework/Versions/3.9. I know, that I can change the path using install_name_tool, but I don't want to ask and explain my module users to do that, as well as to make a symlink.
How can I make module more portable and python install path independent?
Related
I am quite new on python environment and on pyinstaller, but I really need your help.
Context
I have a python app which loads plugins within code. Indeed, we provide new features for our users in the form of plugins, and we want them to be able to install and use those that interest them. We don't want to build a new executable file every new plugin.
We bundled our app in a executable with pyinstaller, a one folder, because it seems easier to add plugins.
pyinstaller --clean --paths=lib/python3.8/site-packages src/main.py
My plugins have sometimes external dependencies (lib, etc.), so foreach plugin here is my setup.py (with requirements, values from requirements.txt file) :
setup(name='package_name',version='0.0.1',include_package_data=True,install_requires=requirements,extras_require={'all': requirements})
My issue
I know how to import installed modules dynamically to my code:
module=__import__(module_name,globals(),locals(),[name],0)
This part works.
But I have difficulties to install completely and dynamically those plugins within my code.
The solution must work on a linux ans Windows machine.
What I tried
Let's imagine I want to import my test.whl plugin, which has a dependency with 'test2' package
Install the plugin with pip:
subprocess.run(["pip", "install", "--upgrade", package, "-t", my_one_folder_location])
Pro: if python and pip are installed on the Host machine it works
Cons : You need to have python and "pip" (not "pip3" etc) on the user computer and the path. You have to be careful to have the same python version than on the package, etc. Not a recommanded solution...
I also tried to use the python installed in the pyinstaller folder but did not succeed.
Import every plugins in a specific directory:
PyInstaller understands the “egg” distribution format often used for Python packages. If your script imports a module from an “egg”, PyInstaller adds the egg and its dependencies to the set of needed files.
I tried to put my .wheel packages in a folder "egg". Then my program detect the plugin, but it doesn't import my lib dependencies
__main__:Unexpected error: no module named 'test2'
I'm going around in circles and can't find any solutions: What is the best way to import my external .whl packages and all it's dependencies in a pyinstaller solution?
I am using PyCharm to develop a python project, which uses an external library called win10toast. I have installed win10toast using PyCharm. However, when I tried to run the .py file using cmd (i.e Externally running the python file), an error shows up:
ModuleNotFoundError: No module named 'win10toast'.
I have python 3.6.4. I installed win10toast using PyCharm.
from win10toast import ToastNotifier
I expect the program to run without any error, but currently I am getting the ModuleNotFound error.
Python can be tricky to run properly because it is sensitive to where you installed your dependencies (such as external libraries and packages). If you installed Python to one directory, but accidentally installed the external library to another directory, when you run your .py program, it will be unable to call from the external library because it doesn't exist in the same library that Python is running from.
Lookup where you installed Python on your computer and then find where you installed the external library. Once your find where you installed the external library, move its entire package content to the same directory where Python is installed. Or better yet, reinstall the external library with pip into the same directory as Python.
If you're on Mac, Python and its related dependencies are usually stored somewhere in /User/bin. If you're on Windows, it will be stored somewhere in your C:// directory (possibly somewhere in C:\Users\username\Local\AppData). If you're on Linux, it will be stored somewhere in /usr/bin. Whatever you do, don't move Python from wherever it is because sometimes that can mess up your system for certain operating systems like Mac, which comes with its own version of Python (Python 2.7 I believe, which is outdated anyway).
Lastly, you may have two different versions of Python on your computer, which is common; Python 2.7 and Python 3+. If you wrote your program in one version, but ran it from the other, the external library can only be called from whichever Python version you installed it to. Try running your .py program with python3 instead of python (or vice versa) and see what happens. If it works with one python version over the other, that tells you that the external library is installed in the other version's directory.
That should solve your issue.
I would suggest that you not use PyCharm to install packages, at least not
if the result deviates in the slightest from doing a "pip install" at the command line. I see no reason to involve PyCharm in configuring Python installations. It's just asking for trouble.
I admit that I'm not familiar with the practice I'm suggesting you avoid. I've been using PyCharm since pretty much the week it came out (was an avid user of the IntelliJ Python plugin before that), and have never once considered doing anything but installing Python modules at the command line. That way, I'm sure right where those modules are going (into which base Python install or venv). Also, I know I'm doing all that I can to minimize the differences that I might see between running code in PyCharm and running it at the command line. I'm making my suggestion based solely on this practice having never gone wrong for me.
I have multiple base Python versions installed, and dozens of venvs defined on top of those. PyCharm is great at allowing me to indicate which of these I want to apply to any project or Run/Debug configuration, and utilizing them seamlessly. But agin, I administer these environments at the command line exclusively.
I still experience issues in switching between the command line and PyCharm in terms of one module referencing others in a single source tree. My company has come up with a simple solution to this that insures that all of our Python scripts still run when moving away from PyCharm and its logic for maintaining the Python Path within a project. I've explained the mechanism before on S.O. I'd be happy to find that if anyone is interested.
The library win10toast installed in the directory: YOUR_PYCHARM_WORKSPACE\PycharmProjects\YOUR_PROJECT_NAME\venv\Lib\site-packages
but when you are running your program using cmd, pycharm interpreter uses site-packages directory that you installed python at there. for Ex: C:\Python27\Lib\site-packages
So, you can install the win10toast library to this windows directory using pip.
I'm using software that is built for Windows with no version for Linux. They have dedicated Python library called ArcPy, that has scripts of all of the tools available in this software.
On Ubuntu 16.04 I'm trying to import this package to my Python, so I can use all those scripts. Inside /usr/local/lib/python2.7/site-packages I created Desktop.pth into which I echoed:
/media/adam/somedisk/Program\ Files\ \(x86\)/ArcGIS/Desktop10.5/bin
/media/adam/somedisk/Program\ Files\ \(x86\)/ArcGIS/Desktop10.5/arcpy
/media/adam/somedisk/Program\ Files\ \(x86\)/ArcGIS/Desktop10.5/ArcToolBox/Scripts
Now entering Python shell and typing import arcpy returns ImportError: No module named arcpy. I know I typed the paths with correct escaping, because I can cd them.
Is it a correct way of importing Python packages across OSs? What went wrong here?
You can not use arcpy unless, either ArcGIS Server or ArcGIS engine is installed in the machine. ArcPy does not work without the binaries.
I am attempting to install IronPython. I downloaded the stable 2.7.5 installer from here:
http://ironpython.net/download/
I ran the installer and all is well. It is not located at:
C:\Program Files (x86)\IronPython 2.7
I have a previous installation of Python 2.7 from my ArcGIS installation, which packages and installs python along with it. My installation location for python is:
C:\Python27\ArcGIS10.2
So, when I test and:
import clr
I receive and error:
ImportError: No module named clr
My Environment Variable Path names are all correct.
I'm just wondering if python IDLE knows where to grab IronPython, or did I install wrong?
The name of the IronPython intepreter is ipy.exe. If you not starting this, you are not using IronPython. So you should see the application name in the window title and when starting the shell. When you use it in an IDE, you should tell the IDE that you are using IronPython, if it support it. Just setting some environment variables will not help. If you are targeting the correct directory and ipy.exe will be used, everything should work just fine.
I'm embedding python in a C application. I have downloaded the standard python dist and placed it relative to program and used this to link & build the C application.
This works fine on my devel machine and the application runs against this version of python.
When I bundle up the application and the python distro and deploy it, Im getting the following error:
ImportError: No module named site
This can be fixed by setting PYTHONHOME to the path to this 'local' python distribution, but I don't want to mess around with any python installation the user may already have, so would rather not set this variable.
Any ideas how to correctly bundle the python interpreter & lib without interfering with any possible versions of python that may already be on a target machine?
Just add
Py_SetPythonHome(pathToPython);
before Py_Initialize call. pathToPython should be the path to python distribution, for Windows it is the folder that contains Lib and DLLs folders.