I am building a python module using C++ with pybind11. This project will depend on two separate packages, that provide bindings for native C++ libraries:
opencv (provided by the pip package opencv-python)
imgui (provided by the pip package pyimgui).
This packages, once installed, consist mainly of dynamic libraries, plus some glue code in python.
My question concerns the installation/build process on the final user computer, i.e when pip install . is launched (in build isolation mode): how to find the path to the third parties dynamic libraries when linking?
In the end, they will be located somewhere in the virtual environment of the user; but I need to link them during the pip build (in isolation mode), and I also need to ensure that they will be found when the module will be used.
Note: as far a opencv is concerned, I could link to a local installation of OpenCV (i.e require the user to apt install opencv before, and use find_package(OpenCV)), but I suspect this is not the way to go. Concerning imgui, this solution would not apply.
Related
I want to package a python module containing python source and a native c++ library. Cppyy is used to dynamically generate the bindings so the library is really just a normal library. The build system for the library is meson and should not be replaced. The whole thing is in a git repository. I only care about Linux.
My question is how to get from this to “pip install url_to_package builds/installs everything.” in the least complicated way possible.
What I’ve tried:
Extending setuptools with a custom build command:
…that executes meson compile and copies the result in the right place. But pip install will perform its work in some random split-off temporary directory and I can’t find my C++ sources from there.
The Meson python module:
…can build my library and install files directly into some python env. Does not work with pip and has very limited functionality.
Wheels:
…are incredibly confusing and overkill for me. I will likely be the only user of this module. Actually, all I want is to easily use the module in projects that live in different directories…
Along the way, I also came across different CMake solutions, but those are disqualified because of my build system choice. What should I do?
I'm trying to build a speech recognition application that works in a browser. Currently using pyodide with a web worker. I have my own package, built alongside pyaudio, that I use for the web worker.
Is there a way that I can include a brew instillation, specifically portaudio, inside of my python package so that when I build the package, portaudio is included in the wheel file? I need portaudio included for this to work in the browser.
Thank you!
I'm understanding two different questions here, so I'll try to answer them both.
Can I have a python build fetch a Homebrew project during buildtime
To my knowledge, the answer is no. The Python distribution system is separate from Homebrew, and they can't interact in this fashion.
Even if they could, this wouldn't necessarily be desirable:
What happens if the user isn't on macOS (or Linux)? Then the build would fail.
The prefix that Homebrew will install the package in isn't very deterministic. The user might be using a custom prefix, or they might be on Apple Silicon (which has a different default prefix to Intel).
Your python package might run into some difficulty locating the package.
What about if they don't have Homebrew installed? They might have another package manager like MacPorts or Fink, or maybe none at all.
Can I bundle portaudio into the build distribution?
Maybe? Even if you could, I almost certainly wouldn't recommend it.
Bundling dependencies increases the size of the distribution unnecessarily.
It would take a reasonable amount of effort to setup, assuming you can do it.
All these reasons are why for the majority of projects that have a similar setup, you will find that they recommend installing certain packages with their system package manager first, before building the Python source code.
This allows them to choose whatever package manager they have installed, and it should also be a quick and painless process.
Therefore, just change your installation instructions to the following:
# On macOS
brew install portaudio
pip install ...
I've seen 2 ways of installing OpenCV (there might be more ways which I don't know):
Installing from the source
Installing with pip: pip install opencv-python
My question is, why we need to install OpenCV from the source while we can simply install it using pip? Since people are using both of them, both must be useful. If so, there are any conditions for selecting one of them?
I will list out the differences between both
1.
Installation using pip
Installation is done at the default location where all the python packages resides.
Installation from Source
Installation location is provided by the developer.
2.
Installation using pip
In terms of performance, the packages installed might run slower because of the hidden conflicts between features.
Installation from Source
The developer can select the optimization flags during the compilation of packages which are responsible for the fast performance of library.
3.
Installation using pip
The developers can neither add nor remove features provided in the installation done by pip.
Installation from Source
The developer has all the rights to add or remove the features during the installation of library.
4.
Installation using pip
The package manager will do the work on behalf of developer. Package Manager is also responsible for taking care of library updation.
Installation from Source
The developers are responsible for feature selection and updation of library. They must be aware of new package updates, latest security patches etc, to keep themselves updated about the library.
Hope this helps you!
OpenCV is always under development, and the thing is some parts of the library is not going to published, due to compatibility and copyright issues, but if you use the source then you can have all the capabilities that you need. SURF & SIFT are examples of this problem.
I'm writting an application in c++ which uses pybind11 library to provide python support.
I use some non standard python libraries installed with pip.
Now I'd like to build a Windows installer which would contain complete application with all required python modules and proper python environment.
I add extracted python-embed.zip provided by https://www.python.org/ but it lacks of modules which are required by external modules I use.
For example one of modules I use in my application requires pkg_resources.
I would add it with pip to my installation, but there are no modules available for pip matching this name.
I'm not sure how to solve these problems in right way.
Why we have to install the python packages before using them?
I am currently working on a small python mysql program. What i tried to download the python connector module from mysql webpage and simply unzip it and place it in the same folder of my code.
And I can import the module properly.
So what is the meaning of installing those packages? Can I use those packages like matplotlib, numpy without installing them ?
Is it possible to have all the required packages installed on a folder so that i can move it to another computer and run my program with only CPython installed (I don't want to install any package on this computer)?
it's not that simple :-)
some packages have dependencies, you also need to download and extract their dependencies (you need pacakge x,and package x uses y) pakcage manager handles that
some package have some c code (they need to be compiled before use (ujson or postgres module) package manager handles that
when your share your code instead of sharing dependencies you simply add a file containing the list of dependencies (requirements.txt) and other user can simply install all dependencies using package manager
Installing a python package enables us to use it anywhere on our system. If we just place the package in the same directory as our script then it may well work, but only for scripts in that directory.
Some packages also rely on others to function properly, and the installation of a package may well install those pre-requisite packages for you. You may be able to do this manually, but you'd have to put them all in the same directory as your script every time you wanted to run it.
So installing the packages is the easiest way to use them.
You don't have to install them, and in some cases you wouldn't install them on your system; if you had split your code across two files and imported one file at the top of the other for example.
In fact, you don't really need install package on your system.
But if you install it, you can use these packages every where on your system.
Also, you can create a requirement.txt file to enable install all packages that you need on other computer. You can check this manual https://pip.pypa.io/en/stable/user_guide/#requirements-files