I'm writting an application in c++ which uses pybind11 library to provide python support.
I use some non standard python libraries installed with pip.
Now I'd like to build a Windows installer which would contain complete application with all required python modules and proper python environment.
I add extracted python-embed.zip provided by https://www.python.org/ but it lacks of modules which are required by external modules I use.
For example one of modules I use in my application requires pkg_resources.
I would add it with pip to my installation, but there are no modules available for pip matching this name.
I'm not sure how to solve these problems in right way.
Related
I am building a python module using C++ with pybind11. This project will depend on two separate packages, that provide bindings for native C++ libraries:
opencv (provided by the pip package opencv-python)
imgui (provided by the pip package pyimgui).
This packages, once installed, consist mainly of dynamic libraries, plus some glue code in python.
My question concerns the installation/build process on the final user computer, i.e when pip install . is launched (in build isolation mode): how to find the path to the third parties dynamic libraries when linking?
In the end, they will be located somewhere in the virtual environment of the user; but I need to link them during the pip build (in isolation mode), and I also need to ensure that they will be found when the module will be used.
Note: as far a opencv is concerned, I could link to a local installation of OpenCV (i.e require the user to apt install opencv before, and use find_package(OpenCV)), but I suspect this is not the way to go. Concerning imgui, this solution would not apply.
The book "Learning Python" by Mark Lutz states that in Python one can use various types of external modules, which include .py files, .zip archives, C/C++ compiled libraries and others. My question is, how does one usually handle installation of each type of module?
For example, I know that to use a .py module, I simply need to locate it with import. What about something like .dll or .a? Or for example, I found an interesting library on GitHub, which has no installation manual. How do I know which files to import?
Also, are there any ways of installing modules besides pip?
The answer depends on what you want to do.
You can use Ninja for example to use C++ modules and cython for C and there are various packages for almost any type of compiled code.
You can install packages via pip using the pypi package repository or by using cloned repositories that have a setup.py file inside.
Any other python based repo can be imported either by a custom build script (that they will provide) or by directly importing the relevant Python files. This will require you the dive into the code and check what are the relevant files.
Also, are there any ways of installing modules besides pip?
Yes, according to Installing Python Modules (Legacy version) modules packaged using distutils should be downloaded, unpacked and command
python setup.py install
or similar should be run. Beware that
The entire distutils package has been deprecated and will be removed
in Python 3.12.
I'm trying to create and distribute (with pip) a Python package that has Python code, and C++ code compiled to a .pyd file with Pybind11 (using Visual Studio 2019). I also want to include .pyi stub files, for VScode and other editors. I can't find much documentation on doing this correctly.
I'd like to be able to just install the package via pip as normal, and write from mymodule.mysubmodule import myfunc etc like a normal Python package, including autocompletes, type annotations, VScode intellisense etc using the .pyi files I'd write.
My C++ code is in multiple cpp and header files. It uses a few standard libraries, and a few external libraries (such as boost). It defines a single module, and 2 submodules. I want to be able to distribute this on Windows and Linux, and for x86 and x64. I am currently targeting Python 3.9, and the c++17 standard.
How should I structure and distribute this package? Do I include the c++ source files, and create a setup.py similar to the Pybind11 example? If so, how do I include the external libraries? And how do I structure the .pyi stub files? Does this mean whoever tries to install my package would need a c++ compiler as well?
Or, should I compile my c++ to a .pyd/.so file for each platform and architecture? If so, is there a way to specify which one gets installed through pip? And again, how would I structure the .pyi stubs?
Generating .pyi stubs
The pybind11 issue mentions a couple of tools (1, 2) to generate stubs for binary modules. There could be more, but I'm not aware of others. Unfortunately both are far from being perfect, so you probably still need to check and adjust the generated stubs manually.
Distribution of .pyi stubs
After correction of stubs you just include those .pyi files in you distribution (e.g. in wheel or as sources) along with py.typed indication file or, alternatively, distribute them separately as standalone package (e.g. mypackage-stubs).
Building wheels
Wheels allows users of your library to install it in binary form, i.e. without compilation. Wheels makes use of older compilers in order to be compatible with greater number of systems/platforms, so you might face some troubles with a C++17 library. (C++11 is old enough and should have no problems with wheels).
Building wheels for various platforms is tedious, the pybind11's python_example uses cibuildwheels package to do that, I would recommend this route if you are already using CI.
If wheels are missing for target platform the pip will attempt to install from source. This would require compiler and 3rd party libraries you are using to be already installed.
Maybe conda?
If your setup is complex and requires a number of 3rd party libraries it might be worth to write a conda recipe and use conda-forge to generate binary versions of the package. Conda is superior to pip, since it can manage non-python dependencies as well.
I'm trying to write a sublime text plugin which would make some windows api calls. I did some research and found out that this python library provides the API's that I need to use.
So, I'm trying to to use this library. When I add import statement for it in my sublime text plugin it gives me error
ImportError: No module named win32api
I'd assume that it's because sublime text comes with inbuilt python and I haven't actually installed these libraries on my system it's throwing up these errors. How do I add such libraries in my sublime plugin also How would I distribute such plugin?
In order to use a third-party library in Sublime, you'll need to include it in your plugin directory, and you'll need to include the correct version for the Python you're working with - 2.6 for ST2, 3.3 for ST3. In order for the next step to succeed properly, install the version of Python you're targeting on your system from python.org. For example, if you're working with ST3, make sure you install Python 3.3.2 on your system.
Then, download the correct version of pywin32 from Christoph Gohlke's Python Extension Packages for Windows repository. Install it on your system, then try copying c:\Python33\Lib\site-packages\win32\ to your Packages/MyPlugin folder in Sublime. The installer makes several directories in site-packages, so depending on the system calls you're planning on making you might need to copy other directories as well - win32com, win32comext, pywin32_system32, and/or pythonwin.
You should now be able to import win32 or import .win32 in your plugin. Good luck!
I need to support some software that is using an old Python version (2.4). So I have downloaded and compiled Python 2.4 and installed it in a virtualenv. So far, all OK and normal procedure.
But the software is trying to import an rpm module. And I cannot find a source for that module (it is not part of the standard Python library, afaict).
Typically, once the virtualenv is enabled (source env/bin/activate) I can install required software using easy_install. But easy_install rpm is failing to find anything. There is a pyrpm module, but it is not the same thing (it installs a module called "pyrpm"). And google searches are useless, as they all link to articles on how to build rpms...
If I were using the system python (on Ubuntu) I could install the python-rpm package. But that is for Python 2.7. How do I install the equivalent for Python 2.4?
[My impression is that the rpm libraries, used by many Linux systems, include a Python library, which is packaged as python-dev by the distro. But I can't see how to access that for an arbitrary python version.]
I AM NOT LOOKING FOR AN RPM THAT CONTAINS PYTHON 2.4. I AM LOOKING FOR A MODULE NAMED rpm THAT IS USED BY SOFTWARE WRITTEN FOR PYTHON 2.4.
It's right there, in the python-rpm RPM package:
http://rpmfind.net/linux/rpm2html/search.php?query=python-rpm
You will probably want to download the package contents, extract them, and then use
python setup.py install
From your active environment.
Of course, as it's pre compiled, you might have trouble getting the C extension to run.
I'm not familiar enough with RPM's to know whether you can get the source from there.
No guarantees the package will work with your python version though.
there's no simple way to do this; the python library is part of the system rpm package and interfaces to C code, so is closely tied to the rpm package installed on your machine.
instead, it's much simpler to install an old OS in a VM (eg CentOS 5) that uses Python 2.4. then everything is consistent and works.
the sources for the rpm module can be found here: http://www.rpm.org/wiki/Download
After you download the wanted version read and follow the INSTALL instructions in order to compile it on your target OS. Afterwards make sure you add the correct path to the 'site-packages' folder the installation chose into your PYTHONPATH environment variable.
To test start your python interpreter and run 'import rpm'
HTH,
Ran