My code requires cryptography moduleI use Cython to distribute my code, say foo.pyx
I want to perform following task using cryptography
import cryptography.fernet
key = b"My Key"
fer = Fernet(key)
content = input().encode()
fer.encrypt(content)
Next, I want to distribute this as .pyd
Again, I don't want users to pip install cryptography on their own pc.
Is there any way to distribute foo.px file as foo.pyd file including Cryptography with it?
My setup.py file is:
from setuptools import setup
from Cython.Build import cythonize
setup(
ext_modules = cythonize("foo.pyx")
)
Kindly Note: My platform is Windows 10
Related
I am trying to build a package for distribution which has cython code that I would like to compile into binaries before uploading to PyPI. To do this I am using pypa's build,
python -m build
in the project's root directory. This cythonizes the code and generates the binaries for my system then creates the sdist and wheel in the dist directory. However, the wheel is named "--py3-none-any.whl". When I unzip the .whl I do find the appropriate binaries stored,
(e.g., cycode.cp39-win_amd64.pyd). The problem is I plan to run this in a GitHub workflow where binaries are built for multiple python versions and operating systems. That workflow works fine but overwrites (or causes a duplicate version error) when uploading to PyPI since all of the wheels from the various OS share the same name. Then if I install from PyPI on another OS I get "module can't be found" errors since the binaries for that OS are not there and, since it was a wheel, the installation did not re-compile the cython files.
I am working with 64-bit Windows, MacOS, and Ubuntu. Python versions 3.8-3.10. And a small set of other packages which are listed below.
Does anyone see what I am doing wrong here? Thanks!
Simplified Package
Tests\
Project\
__init__.py
pycode.py
cymod\
__init__.py
_cycode.pyx
_build.py
pyproject.toml
pyproject.toml
[project]
name='Project'
version = '0.1.0'
description = 'My Project'
authors = ...
requires-python = ...
dependencies = ...
[build-system]
requires = [
'setuptools>=64.0.0',
'numpy>=1.22',
'cython>=0.29.30',
'wheel>=0.38'
]
build-backend = "setuptools.build_meta"
[tool.setuptools]
py-modules = ["_build"]
include-package-data = true
packages = ["Project",
"Project.cymod"]
[tool.setuptools.cmdclass]
build_py = "_build._build_cy"
_build.py
import os
from setuptools.extension import Extension
from setuptools.command.build_py import build_py as _build_py
class _build_cy(_build_py):
def run(self):
self.run_command("build_ext")
return super().run()
def initialize_options(self):
super().initialize_options()
import numpy as np
from Cython.Build import cythonize
print('!-- Cythonizing')
if self.distribution.ext_modules == None:
self.distribution.ext_modules = []
# Add to ext_modules list
self.distribution.ext_modules.append(
Extension(
'Project.cymod.cycode',
sources=[os.path.join('Project', 'cymod', '_cycode.pyx')],
include_dirs=[os.path.join('Project', 'cymod'), np.get_include()]
)
)
# Add cythonize ext_modules
self.distribution.ext_modules = cythonize(
self.distribution.ext_modules,
compiler_directives={'language_level': "3"},
include_path=['.', np.get_include()]
)
print('!-- Finished Cythonizing')
I am writing setup.py for my small python package, which is python wrapper on c++ tool, and it's depends on shared libraries like icu-uc, boost_program_options, and some custom ones. Python wrapper build with swig. I use Extension(libraries=[...]) to specify libraries to link.
My setup.py looks like this:
package_name = Extension(
name='_package_name',
sources=['some_souces.cpp',
'some_swig_interfaces.i'
],
swig_opts=['-c++'],
libraries=['icuuc', ...],
)
setup(
name='package_name',
version='0.1.0',
....
ext_modules=[package_name],
py_modules=['package_name']
)
and I build this with python3 setup.py bdist_wheel
And when I build and install generated file locally it works fine, but when I truing to use pip3 install package_name.whl on computer with other version of icu-uc or other libraries it's crashes at runtime throwing ImportError: libicuuc.so.60: cannot open shared object file: No such file or directory. So basically my question is how to build more generic .whl package that don't stick to library version but looks for any match?
I have a setup.py as follows (other source files are available too)
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
ext_modules = [
Extension("fastsim",
["src/fastsim.pyx", "src/selection.c"],
libraries=["m"],)
]
setup(
name="fastsimlib",
cmdclass={"build_ext": build_ext},
ext_modules=ext_modules
)
This works fine as long as there is cython installed in the target machine. I am wondering if I can get rid of the cython dependency by creating an egg file so that I only to run easy_install or pip.
To clarify, I want to create a distribution package so that I can install the library in a machine that does not have cython installed. Or even better, can I only distribute the binary .so file (so that no cython or c code in the package)? The OS and other information of the target machine is given(Ubuntu).
I would like to install my Python module in development mode. As I have seen in many examples python setup.py develop is supposed to do that. But the develop command does not exist for my setup.py file:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
from Cython.Distutils import build_ext
import os
src = ["_NetworKit.pyx"] # list of source files
modules = [Extension("_NetworKit",
src,
language = "c++",
extra_compile_args=["-fopenmp", "-std=c++11", "-O3", "-DNOGTEST"],
extra_link_args=["-fopenmp", "-std=c++11"],
libraries=["NetworKit-Core-O"],
library_dirs=["../"])]
for e in modules:
e.cython_directives = {"embedsignature" : True}
setup(name="_NetworKit",
cmdclass={"build_ext": build_ext},
ext_modules=modules,
py_modules = ["NetworKit.py"])
(Note the Cython extension module).
What am I missing? Do I need to modify the setup.py?
The develop command is a part of setuptools. Install setuptools and replace the first line in setup.py with this:
from setuptools import setup
There is a Python package with a setup.py that reads thusly:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
name = 'fastahack',
ext_modules=[
Extension("fastahack.cfastahack",
sources=["fastahack/cfastahack.pyx", "lib/Fasta.cpp", "lib/split.cpp"],
libraries=["stdc++"],
include_dirs=["lib/"],
language="c++"),
],
package_data = {'lib': ['*.pyx', "*.c", "*.h", "README.rst"]},
package_dir = {"fastahack": "fastahack"},
cmdclass = {'build_ext': build_ext},
packages = ['fastahack', 'fastahack.tests'],
author = "Brent Pedersen",
author_email="bpederse#gmail.com",
#test_suite='nose.collector'
)
This setup.py can't be imported if Cython is not installed. As far as I know, importing setup.py is how tools like pip figure out the dependencies of a package. I want to set up this package so that it could be uploaded to PyPI, with the fact that it depends on Cython noted, so that Cython will be downloaded and installed when you try to "pip install fastahack", or when you try to "pip install" directly from the Git repository.
How would I package this module so that it installs correctly from the Internet when Cython is not installed? Always using the latest version of Cython would be a plus.
You can specify Cython as a build dependency using PEP-518 project specification.
In the file pyproject.toml (in the same directory as setup.py) insert:
[build-system]
requires = ["setuptools", "wheel", "Cython"]
Cython will then be installed before building your package.
Note that (currently) you need to pass --no-use-pep517 to pip install if you are installing your package locally as editable (ie with --editable or -e) setuptools v64 supports editable installs with pyproject.toml builds
My standard template for setup.py:
have_cython = False
try:
from Cython.Distutils import build_ext as _build_ext
have_cython = True
except ImportError:
from distutils.command.build_ext import build_ext as _build_ext
if have_cython:
foo = Extension('foo', ['src/foo.pyx'])
else:
foo = Extension('foo', ['src/foo.c'])
setup (
...
ext_modules=[foo],
cmdclass={'build_ext': build_ext}
And don't forget to provide extention .c files with package - that will allow users to build module without installing cython.
Use a try and except for the Cython import and modify your setup based on whether or not your import succeeds. Look at the setup.py of Pandas for an example