Building PyCrypto with fastmath (gmp or mpir) via pip on Windows - python

I installed PyCrypto on Windows via pip but i was not able to build Crypto.PublicKey._fastmath because GMP was not found.
I know there is a binary version on voidspace but i would like to build the latest version of PyCrypto

The following one is a way to achieve your goal. There are other, probably better ways (e.g. based on Visual Studio), but this one has worked for me. Additionally, it does not use pip.
All operations are carried out on a command prompt.
Install Mingw, including MSYS and the Development Toolkit. This will give you a fairly complete Unix-like development environment.
Ensure that Mingw binaries are in PATH environment variable. You need MinGW\bin and MingGW\msys\1.0\bin.
Download MPIR sources in a temporary directory. It is important you do not use 2.5.1 because of a bug that will break the build. 2.5.0 is fine.
Build the MPIR library. This is fairly straightforward: execute bash configure followed by make.
HACK #1 Copy libmpir.a from mpir-2.5.0\.libs into C:\Python2.7.1\libs. This is necessary because distutils is broken and I could not find a way to direct it to the correct library location.
HACK #2 Edit C:\Python2.7.1\Lib\distutils\cygwincompiler.py and remove any occurrance of the string -mno-cygwin. The reason is explained here.
Download PyCrypto sources and unpack them in another temporary directory.
Set CPPFLAGS environment variable to the MPIR directory, which contains mpir.h.
HACK 3 Edit setup.py and add the following line in build_extension method:
self.__add_compiler_option(os.environ['CPPFLAGS'])
Run bash configure. You should see two lines saying:
checking for __gmpz_init in -lgmp... no
checking for __gmpz_init in -lmpir... yes
Execute python setup.py build -c mingw32. You should see no errors.
Execute python setup.py test to verify that everything is fine.
Execute python setup.py install to copy the files into your local Python repository.
Alternatively, run python setup.py bdist_wininst to create an installer.
I really hate all the various hacks, and I'd love to hear if they can be avoided.

Related

How to install python modules using cppyy?

I want to package a python module containing python source and a native c++ library. Cppyy is used to dynamically generate the bindings so the library is really just a normal library. The build system for the library is meson and should not be replaced. The whole thing is in a git repository. I only care about Linux.
My question is how to get from this to “pip install url_to_package builds/installs everything.” in the least complicated way possible.
What I’ve tried:
Extending setuptools with a custom build command:
…that executes meson compile and copies the result in the right place. But pip install will perform its work in some random split-off temporary directory and I can’t find my C++ sources from there.
The Meson python module:
…can build my library and install files directly into some python env. Does not work with pip and has very limited functionality.
Wheels:
…are incredibly confusing and overkill for me. I will likely be the only user of this module. Actually, all I want is to easily use the module in projects that live in different directories…
Along the way, I also came across different CMake solutions, but those are disqualified because of my build system choice. What should I do?

Importing libraries

I have a general question using third party libraries, but I will exemplify it on two examples to make it clearer and more "answerable":
I want to use pyfmi in Python. Trying to install it through pip tells me:
"Exception: FMI Library cannot be found. Please specify its location, either using the flag to the setup script '--fmil-home' or specify it using the environment variable FMIL_HOME."
I figured out that I had to download the tar.gz from jmodelica.org and extract the files, create a build directory, use cmake, make and make install commands. All runs through without a hitch. But trying to install through pip gives me the same error message. So my question is:
How does one do this? Do they mean by setup script the setup.py file? How can I access that one if I am installing through pip?
An which one is the fmi home directory? Is it the untarred file in my Downloads-Folder or one of the files in it:
builddir
Config.cmake
install
src
Test
ThirdParty
CMakeLists.txt
FMILIB_Acknowledgements.txt
FMILIB_License.txt
FMILIB_Readme.txt
LICENSE.md
README.md
? What is that flag and where to put it "exactly".
Thanks a lot.
PyFMI requires (as noted on the PyPI site) that FMI Library is installed prior to trying to install PyFMI from source.
During the installation (invoking python setup.py install) of PyFMI, the environment variable "FMIL_HOME" is checked to see if that points to an installation of FMI Library, if so, this will be used during the installation. So in your case, you need to set this environment variable. The other option is to install manually (using python setup.py install --fmil-home="/path/to/fmil") where the added path should point to FMI Library.
The third option is to see if there are binary installers for your platform (these include FMIL). Check PyPI, Anaconda and Christoph Gohlke's site.

how can I create a debian package to distribute python modules without using distutils?

I'm getting myself thoroughly confused how to correctly package my python3 based application.
My application uses a Makefile to install stuff to the correct file locations e.g.
/usr/lib/rhythmbox/plugins/foo/myfoo.plugin
/usr/lib/rhythmbox/plugins/foo/myfoo/po/translation.mo
/usr/lib/rhythmbox/plugins/foo/myfoo_module.py
/usr/lib/rhythmbox/plugins/foo/myfoo_module2.py
I'm not using a python distutils setup.py type installation - just a straightforward sudo make install based method.
When I try to Debian package my rules are very straightforward:
#!/usr/bin/make -f
%:
dh $# --parallel --with autoreconf,python3
override_dh_autoreconf:
dh_autoreconf -- ./autogen.sh
override_dh_auto_configure:
dh_auto_configure -- --libdir="\$${exec_prefix}/lib"
My debian/control file "build-depends" is again straightword:
Build-Depends: debhelper (>= 9),
dh-autoreconf,
dh-python (>= 1.20130903),
gir1.2-glib-2.0,
gir1.2-gstreamer-1.0,
gir1.2-gtk-3.0,
gir1.2-peas-1.0,
gir1.2-rb-3.0,
gobject-introspection (>= 0.10.0),
intltool (>= 0.35.0),
libglib2.0-dev,
python3
That works just fine - I can run debuild -us -uc and it creates my .deb and when I install via sudo dpkg -i myfoo-0.1_all.deb everything is installed in the correct file locations.
Except one small matter - each python module should be byte-compiled so that on installation there is a subfolder /usr/lib/rhythbox/plugins/foo/__pycache__ containing myfoo_module.pyc and myfoo_module2.pyc
Now I know I cannot byte-compile during the build process - Debian rules forbid having a .deb with byte-compiled modules. So somehow I need to get debhelper to work for me.
Looking at the Debian packaging guide they mention stuff like cdbs, distutils etc - any debhelper examples always use syntax like:
override_dh_auto_install:
dh_auto_install
python setup.py install --root=$(CURDIR)/debian/$(DEB_SOURCE) --install-layout=deb)
... but I'm not using a distutils setup.py to install my application.
So I'm must be missing something pretty obvious - any thoughts?
You don't need to use distutils or a setup.py to create a valid deb for a Python program; dh_python3 would just make things easier if you were using distutils. Since you're not, you should simply proceed as if you were using something other than Python.
Since I don't know exactly what your structure is, I can't describe the exact steps, but in general you should install the right files to the right places with dh_install and create appropriate maintainer scripts to do whatever work is necessary for byte-compiling (on install or update) and removing the byte-compiled files (on remove or purge). The maintainer scripts that dh_python3 produces may provide a good starting point (which you can probably find in most files matching /var/lib/dpkg/info/python3-*.postinst and .postrm on your system).

How can I install a python package without pip or virtualenv

I have to deploy a python application to a production server (Ubuntu) that I do not control nor do I have permissions to apt-get, pip, virtualenv, etc. Currently, its the server is running python 2.6+. I need to install pycrypto as a dependency for the application but given my limited permissions, I'm not sure as to how to do it. The only think I have permissions to do is wget a resource and unpack it or things along those lines.
First off, is it possible to use it without getting it installed in the aforementioned approach? If not, could I download the package then drop in __init__.py files in the pycrypto dir so python knows how to find it like so:
/my_app
/pycrypto
/__init__.py
/pycrypto.py
According to PEP370, starting with python 2.6 you can have a per-user site directory (see the What's new in Python 2.6?).
So you can use the --user option of easy_install to install the directory for each user instead of system-wide. I believe a similar option exists for pip too.
This doesn't require any privileges since it only uses current user directories.
If you don't have any installer installed you can manually unpack the package into:
~/.local/lib/python2.6/site-packages
Or, if you are on Windows, into:
%APPDATA%/Python/Python26/site-packages
In the case of pycrypto, the package requires building before installation because it contains some C code. The sources should contain a setup.py file. You have to build the library running
python setup.py build
Afterwards you can install it in the user directory by giving:
python setup.py install --user
Note that the building phase might require some C library to already be installed.
If you don't want to do this, the only option is to ship the library together with your application.
By the way: I believe easy_install doesn't really check whether you are root before performing a system wide install. It simply checks whether it can write in the system-wide site directory. So, if you do have the privileges to write there, there's no need to use sudo in the first place. However this would be really odd...
Use easy_install. It should be installed already on Ubuntu for python 2.6+. If not take a look at these install instructions.

Building NumPy on RedHat

I installed a local version of Python 2.7 in my home directory (Linux RedHat) under ~/opt using the --prefix flag.
More specifically, Python was placed in ~/home/opt/bin.
Now, I want to install NumPy, but I am not really sure how I would achieve this. All I found in the INSTALL.txt and online documentation was the command to use the compiler.
I tried gfortran, and it worked without any error message:
python setup.py build --fcompiler=gnu95
However, I am not sure how to install it for my local version of Python.
Also, I have to admit that I don't really understand how this whole approach works in general. E.g., what is the setup.py build doing? Is it creating module files that I have to move to a specific folder?
I hope anyone can give me some help here, and I would also appreciate a few lines of information how this approach works, or maybe some resources where I can read it up (I didn't find anything on the NumPy pages).
Your local version of python should keep all of it's files somewhere in ~/opt (presumably). As long as this is the python installation that gets used when you issue the command
python setup.py build --fcompiler=gnu95
you should be all set because in the sys module, there are a bunch of constants which the setup script uses to determine where to put the modules once they are built.
So -- running python setup.py build issues all of the necessary commands to build the module (compiling the C/Fortran code into shared object libraries that python can load dynamically and copying the pure python code to create the proper directory structure). The module is actually built somewhere in the build subdirectory which gets created during the process if it doesn't already exist. Once the library has been built (successfully), installing it should be as simple as:
python setup.py install
(You might need to sudo if you don't have write privileges in the install directory).

Categories