I current have a setup.py file for an application I've written. Using setuptools I've easily been able to install pip-available requirements as such:
install_requires = [
'argparse',
'multiprocessing',
'requests',
'numpy',
'termcolor',
'prettytable'
]
The problem is that I also need to install MySQLdb, which is not installed via pip. When setting up locally, I had to download the tarball, uncompress, install, symlink, etc... To put it short, it was a PITA.
Is there anyway to automate this within my setup.py file? Rather than downloading the tarball and including it as a package? Even then, how would I run a setup.py within my own setup.py?
You can actually install MySQLdb via pip, but the package is named MySQL-python.
Now, your users will still need the package's C dependencies installed (libmysqlclient), but this is easily installed with a package manager. It would also be reasonably easy to provide a non-setup.py install script (e.g. a bash script) that installs the appropriate system dependencies (libmysqlclient) and calls your setup.py:
#!/bin/bash
apt-get install -y libmysqlclient-dev # Improve me! Check the platform first
python setup.py install
Just don't try to do too much in your setup.py. No one expects a setup.py script to install system packages, so you should refrain from doing that in yours.
Now, if requiring users to install a system package is too much (if they don't have root access, it can be), you should use a pure-Python MySQL client instead.
One such client is pymysql, which of course can be installed via pip.
Related
I want to create a python package which will be cloned from its git repo when a build runs, so I will have the source inside the build agent. I would then like to run the python package as a command line tool, the package is called environment_manager.
Initially I thought I would follow a tutorial for creating a simple setup.py although this has proved to be a lot more difficult than I thought it would be and whenever I run python setup.py install --force I am not able to use my installed package, generally either module not found or the command is not recognised when I type it.
I have found that if I simply install with pip install . then I am actually able to use the tool from the command line and it works. I don't understand what the difference is, or why this only works when doing the pip install method.
Below is the setup.py file, I cannot see what is wrong with it:
from setuptools import setup, find_packages, find_namespace_packages
import pathlib
here = pathlib.Path(__file__).parent.resolve()
# Get the long description from the README file
long_description = (here / 'README.MD').read_text(encoding='utf-8')
setup(
name='environment_manager',
version='1.0.0',
package_dir={'': 'src'},
packages=find_namespace_packages(where='src', include='environment_manager.*'),
python_requires='>=3.8, <4',
install_requires=['boto3', 'botocore', 'pyyaml'],
extras_require={
'dev': ['pre-commit', 'black', 'pylint'],
'test': ['pytest', 'pytest-mock', 'coverage'],
},
entry_points={
'console_scripts': [
'environment-manager=environment_manager.environment_controller:main',
],
}
)
My project structure looks like:
environment_manager
/src
conf/
environment_manager/
environment_controller.py
config_parser.py
command.py
test/
unit_tests.py
I thought the correct way to install and run the tool from the command line was to use setup.py and setuptools but it seems like it is a lot easier and actually works if I just install it with pip.
Is installing it with pip over setup.py correct (as both ways the package appears when I type pip list) and are there any issues with my setup.py script? The script was taken from the pypa sample project and I removed most of what I didnt need.
setup.py is a python file, which usually tells you that the module/package you are about to install has been packaged and distributed with Distutils, which is the standard for distributing Python Modules. This allows you to easily install Python packages. Often it's enough to write: $ pip install .
In other words setup.py is a packaging file while pip is a package manager, therefore you should have setup.py file to be able to install with pip.
pip is a package manager which helps install, manage, and uninstall Python packages. It searches for them on PyPI, downloads them, and then runs their setup.py script.
Since you mentioned that you can run your binary executable after a pip install, but not a setup.py install, it is likely that each of them is installing the binary to separate locations.
One thing I would check is that you are using python and pip from the same version of Python, e.g:
% python --version
Python 3.8.6
% pip --version
pip 20.1.1 from /usr/lib/python3.8/site-packages/pip (python 3.8)
If these have different Python versions listed, they are likely installing to two separate directories - one in your PATH environment variable, and one which is not.
Next, I would check pip list -v after each install method, as this should list a Location header telling you where the package has been installed.
I have developed a tool that my team can use after running the setup.py script. The tool requires this library: https://github.com/c2nes/javalang
How can I make my python setup script install this library on their computer regardless of what OS they are on. They can't run my tool without that library (Some people are on windows, mac, and linux.)
pip can install projects on Github as a dependency too!
All you need to do is, in your requirements.txt, add a line like following:
..
git+https://github.com/c2nes/javalang.git
then install the dependency using:
$ pip install -r requirements.txt
What you are looking for exists on PyPI. Instead of git+https://.. line above, just say: javalang. Oh and BTW, unless they are running old versions of Python, they should already have pip installed. If they don't use your operating systems package manager or get-pip.py as you said.
I was wondering how the above "yum install package" & "python setup.py install" are used differently in CentOS? I used yum install ... all the time. However, when I try to do python setup.py install, I always get: this setup.py file couldn't be found even though its path shows up under echo $PATH, unless I try to use it in its current directory or use the absolute path.
When you type python setup.py install, your shell will check your $PATH for the python command, and run that. Then, python will be examining its arguments, which are setup.py install. It knows that it can be given the name of a script, so it looks for the file called setup.py so it can be run. Python doesn't use your $PATH to find scripts, though, so it should be a real path to a file. If you just give it the name setup.py it will only look in your current directory.
The source directory for a python module should not, ideally, be in your $PATH.
yum install is a command that will go to a package repository, download all the files needed to install something, and then put them in the right place. yum (and equivalents on other distributions, like apt for Debian systems) will also fetch and install any other packages you need, including any that aren't python modules.
Python has a package manager, too. You may also find using pip install modulename or pip install --user modulename (if you don't have administrative rights) easier than downloading and installing the module by hand. You can often get more recent versions of modules this way, as the ones provided by an operating system (through yum) tend to be older, more stable versions. Sometimes the module is not available through yum at all. pip can't install any extra packages that aren't python modules, though.
If you don't have pip already (it comes with Python3, but might need installing separately for Python2, depending on how it was set up), then you can install it by following the instructions here: https://pip.pypa.io/en/stable/installing/
I am running a local pypi server. I can install packages from this server by either specifying it with the -i option of the pip command or by setting the PIP_INDEX_URL environment variable. When I install a package that has prerequisites, setup.py has historically honored the PIP_INDEX_URL environment variable, pulling the additional packages from my local server.
However, on a couple of systems that have been recently installed, it is behaving differently. Running, for instance, python setup.py develop fails because it tries to install prerequisites packages from pypi.python.org.
I have updated all of the related python packages (python, distribute, virtualenv, pip, etc...) on all the systems I'm testing on and continue to see this discrepancy. On my "original" system, setup.py downloads prerequisites from the pypi server specified in my PIP_INDEX_URL environment variable. On the newer systems, I can't seem to make it honor this variable.
What am I missing?
Create setup.cfg in the same folder as your setup.py with following content:
[easy_install]
allow_hosts = *.myintranet.example.com
From: http://pythonhosted.org/setuptools/easy_install.html#restricting-downloads-with-allow-hosts
You can use the --allow-hosts (-H) option to restrict what domains EasyInstall will look for links and downloads on.
--allow-hosts=None prevents downloading altogether.
I ran into the same issue. Fundamentally, setup.py is using setuptools which leverages easy_install, not pip. Thus, it ignores any pip-related environment variables you set.
Rather than use python setup.py develop you can run pip (from the top of the package) pip install -e . to produce the same effect.
I created some python app using autobahn and packaged it using baazar builddeb. In python setup.py file I added requires tag with all the required dependencies. Is it possible to tell debian package installer to install these packages?
I added some of deps to debian/control>Depends but:
dpkg -i my_package does not install dependencies. Just shows the error and I need to install these deps manually.
some packages does not exists in standard Ubuntu repos. For example autobahn. And in general I'd like to have installed all python dependencies by pip/easy_install
I am using DistUtilsExtra.auto.setup with personalized install action. So I think I could run easy_install packages there. Is it good idea?
Thank you.
Create debian packages from pypi using the python-stdeb package, then depend on them like any other package.
See http://pypi.python.org/pypi/stdeb