We Keep Coding

Publishing to Pypi from gitlab: 422 Unprocessable Entity

I'm trying to push a project package to pypi (it's my first time doing so) from gitlab, and I'm following their turorial [here][1]. However, when I am trying python3 -m twine upload --repository gitlab dist/* the console returns me HTTPError: 422 Unprocessable Entity from https://gitlab.com/MyProfile/mylib
I don't know what is wrong ?
my .pypirc file looks like this
[distutils]
index-servers =
gitlab
[gitlab]
repository = https://gitlab.com/FiiireFlyyy/firelib
username = my_token
password = my_token_password
and here is my setup.py
# Always prefer setuptools over distutils
from setuptools import setup, find_packages
# To use a consistent encoding
from codecs import open
from os import path
# The directory containing this file
HERE = path.abspath(path.dirname(__file__))
# Get the long description from the README file
with open(path.join(HERE, 'README.md'), encoding='utf-8') as f:
long_description = f.read()
# This call to setup() does all the work
setup(
name="firelib",
version="0.1.0",
description="Demo library",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://gitlab.com/FiiireFlyyy/firelib",
author="Willy Lutz",
author_email="myemail0willy#gmail.com",
license="no licence",
classifiers=[
"Intended Audience :: Developers",
"License :: OSI Approved :: no licence",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Operating System :: OS Independent"
],
packages=["firelib"],
include_package_data=True,
install_requires=["os", "shutil"]
)
My file tree is :
firelib (pycharm project folder)
|-----dist (directory)
|-----firelib (package)
|-----firelib.egg-info (directory)
|-----venv
|-----LICENCE
|-----README
|-----setup.py
|-----requirements.txt
I'm really bad with config stuff. Can someone help me ?
Edit : i put the real neames of packages insteand of 'myProfile' and 'mylib'
[1]: https://docs.gitlab.com/ee/user/packages/pypi_repository/

Cythonize installs .so files to wrong location

I'd like to ask a question about how to configure setup.py with Cython, setuptools extensions etc. I'm trying cythonize a submodule with Cython.Distutils's build_ext. But the issue isn't Cython.Distutils's Extentsion module, (if it exists) because it isn't loaded. Only build_ext. So I create a setuptools.extension Extension in a list, and then cythonize the list of Extension objects. There's only one Extension in the list, an its as follows.
Extension("distance", ["kmerdb/distance.pyx"], include_dirs=[np.get_include()])
I've tried different methods of installation, from python setup.py install to pip install -e . to generating the wheel file and installing the wheel. I couldn't find anything that worked...
Okay, so I know very little about the process and that's probably why I am getting hung up but I've searched the whole site, no luck. So here goes.
I'm running the following shell script to install my package locally, and it works fine. The problem is the installation doesn't move the .so and the .py file into the kmerdb module I'm trying to export. Any suggestions? Thanks
>python setup.py sdist bdist_wheel
>/bin/auditwheel repair --plat manylinux2014_x86_64 dist/kmerdb-*linux_x86_64.whl
>mv wheelhouse/* dist
>rm dist/*linux_x86_64.whl
>pip install dist/kmerdb-*-manylinux2014_x86_64.whl
>ls ~/.pyenv/versions/kdb/lib/python3.10/site-packages/kmerdb-0.6.5-py3.10-linux-x86_64.egg/
distance.cpython-310-x86_64-linux-gnu.so distance.py kmerdb ...
Again, the files distance.cpython-310-x86_64-linux-gno.so is not moved into the kmerdb module, my Python packages, which I'm trying to install locally and configure for .whl upload to PyPI.
Python 3.10.1 (main, Jan 1 2022, 21:28:19) [GCC 11.1.0] on linux
Cython==0.29.26
setup.py
Extension("distance", ["kmerdb/distance.pyx"], include_dirs=[np.get_include()], define_macros=[("NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION")],),
setup(
name=NAME,
version=VERSION,
description=DESCRIPTION,
long_description=long_description,
long_description_content_type='text/markdown',
author=AUTHOR,
author_email=EMAIL,
python_requires=REQUIRES_PYTHON,
url=URL,
download_url=CURRENT_RELEASE,
keywords = ["k-mer", "kmer", "k-merdb", "kmerdb", "kdb"],
classifiers=[
"Development Status :: 1 - Planning",
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Topic :: Scientific/Engineering",
"Topic :: Scientific/Engineering :: Bio-Informatics",
"Topic :: Software Development :: Libraries :: Python Modules",
],
packages=find_packages(exclude=["tests", "*.tests", "*.tests.*", "tests.*"]),
package_dir={'kmerdb': 'kmerdb'},
package_data={'kmerdb': ['CITATION']},
# If your package is a single module, use this instead of 'packages':
#py_modules=['kmerdb'],
#scripts=['bin/kmerdb', 'bin/kmerdb_report.R'],
entry_points={
'console_scripts': ['kmerdb=kmerdb:cli'],
},
install_requires=REQUIRED,#['Cython==0.29.21', 'numpy==1.18.1'],
extras_require=EXTRAS,
include_package_data=True,
license='GPLv3+',
test_suite='test',
# tests_require=['mamba', 'expect'],
ext_modules=cythonize(extensions),
library_dirs=["."],
zip_safe=False,
)

I'm going to self answer here. The issue stemmed from an improperly specified Extension.
Extension("kmerdb.distance", ["kmerdb/distance.pyx"], include_dirs=[np.get_include()])
All I had to do was include the module name for the fully specified submodule hierarchy. Fixed it!

Use different version of python package in the same environment

I need to have different versions of elasticseach-dsl installed in the same virtualenv. I would like to be able to import them using some sort of alias:
import elasticsearch_dsl1
import elasticsearch_dsl2
import elasticsearch_dsl5
I was thinking to create packages called elasricsearch_dsl* and in their setup.py add install requirements for the proper version of elasticsearch_dsl but when I install the packages it results in a conflict because all my packages require different versions of the same library.
I have no clue on how to proceed and if this is possible.
Thanks a lot to everyone

Short answer:
It is not possible.

I was able to find a solution to my problem.
I downloaded the source code of the elasticsearch-dsl libraries of different versions
Then changed the import statements to use elasticsearch1, elasticsearch2 etc.libraries instead of elasticsearch v1.x and v2.x
# example `elasticsearch-dsl` v5.4.0
from elasticsearch import Elasticsearch # --> from elasticsearch5 import Elasticsearch
Modified the setup.py to change the dependencies from elasticsearch to elasticsearch1, elasticsearch2 and so on for each version of the elasticsearch-dsl
I've been very lucky because of the existence of elasticsearch1, elasticsearch2... otherwise I would have to repeat the same procedure for them. And more lucky for the compatibility of dependencies of the various versions of the library.
I'm not proud of this hack but it worked.
setup.py
An example of a setup.py file for elasticsearch-dls==7.3.0, then elasticsearch-dsl7
from os.path import join, dirname
from setuptools import setup, find_packages
VERSION = (7, 3, 0)
__version__ = VERSION
__versionstr__ = ".".join(map(str, VERSION))
f = open(join(dirname(__file__), "README"))
long_description = f.read().strip()
f.close()
install_requires = [
"six",
"python-dateutil",
"elasticsearch7", # before "elasticsearch>=7.0.0,<8.0.0" <---
# ipaddress is included in stdlib since python 3.3
'ipaddress; python_version<"3.3"',
]
tests_require = [
"mock",
"pytest>=3.0.0",
"pytest-cov",
"pytest-mock<3.0.0",
"pytz",
"coverage<5.0.0",
]
setup(
name="elasticsearch-dsl7", # <---
description="Python client for Elasticsearch",
license="Apache-2.0",
url="https://github.com/elasticsearch/elasticsearch-dsl-py",
long_description=long_description,
version=__versionstr__,
author="Honza Král",
author_email="honza.kral#gmail.com",
maintainer="Seth Michael Larson",
maintainer_email="seth.larson#elastic.co",
packages=find_packages(where=".", exclude=("test_elasticsearch_dsl*",)),
python_requires=">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*",
classifiers=[
"Development Status :: 4 - Beta",
"License :: OSI Approved :: Apache Software License",
"Intended Audience :: Developers",
"Operating System :: OS Independent",
"Programming Language :: Python",
"Programming Language :: Python :: 2",
"Programming Language :: Python :: 2.7",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.4",
"Programming Language :: Python :: 3.5",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: PyPy",
],
install_requires=install_requires,
test_suite="test_elasticsearch_dsl.run_tests.run_all",
tests_require=tests_require,
extras_require={"develop": tests_require + ["sphinx", "sphinx_rtd_theme"]},
)

Manual installation of Python package

I wrote a small python program and now I want to install and use it on my system.
Program structure:
projectname
|--src
| |--main.py #import file1
| |--file1.py #import file2
| |--file2.py
|--LICENSE
|--README.md
|--setup.py
I did sudo python setup.py install, but now I only able to run it with /usr/bin/main.py. What should I do, to be able to run it by only typing a projectname?
EDIT: Setup.py content:
setuptools.setup(
name="projectname",
version="0.0.1",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
entry_points={
"console_scripts": [
"projectname = projectname.src/main:main"
]
},
python_requires='>=3.6',
)

You can use setuptools' entry points in your setup.py.
Example from the documentation:
setup(
# other arguments here...
entry_points={
"console_scripts": [
"foo = my_package.some_module:main_func",
"bar = other_module:some_func",
],
"gui_scripts": [
"baz = my_package_gui:start_func",
]
}
)
With your structure, create file src/__init__.py:
from . import main
and lets say that this is src/main.py:
def main():
print("Hello world")
Finally, setup.py can be:
import setuptools
setuptools.setup(
name="projectname",
version="0.0.1",
packages=setuptools.find_packages(),
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
],
entry_points={
"console_scripts": [
"projectname = src:main.main"
]
},
python_requires='>=3.6',
)

sdist and bdist_wheel freezes at build

I am trying to build a wheel and source distribution on windows 10 for a project of mine, but the build freezes at removing 'release-exporter-1.0' (and everything under it) for sdist and creating build\bdist.win-amd64\wheel\release_exporter-1.0.dist-info\WHEEL for bdist_wheel.
Following is my setup.py
setup(
name='release-exporter',
version=version(),
install_requires=get_requirements('requirements.txt'),
packages=find_packages(),
url='https://github.com/akshaybabloo/release-exporter',
license='MIT',
author='Akshay Raj Gollahalli',
author_email='akshay#gollahalli.com',
description='Release exporter for GitHub and GitLab.',
keywords="changelog releases",
classifiers=[
'License :: OSI Approved :: MIT License',
'Programming Language :: Python :: 3 :: Only',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Intended Audience :: Developers',
'Environment :: Console',
'Development Status :: 5 - Production/Stable',
'Topic :: Utilities'
],
entry_points={
'console_scripts': [
'rex = release_exporter:main'
]
},
)
I have tried to manually give the package name from ['release_exporter'] to using find_packages() of setup tools. I also thought the keyword should be a string, so I changed it from ['changelog', 'releases'] to "changelog releases". But still no luck.
Also, I have tried to set --verbose flag by doing python setup.py sdist --verbose, but it doesn't work. I think the bug is not rectified yet -> https://bugs.python.org/issue7202
Any help would be appreciated.
Update
I am using Python 3.6.3

Its a bug in Python 3.6.3 -> https://docs.python.org/3.6/whatsnew/changelog.html#build. Just in case if anyone has a problem, update your Python to Python 3.4.4.