I would like to release a python module I wrote which depends on several packages. What's the easiest way to make it so these packages are programmatically downloaded just in case they are not available on the system that's being run? Most of these modules should be available by easy_install or pip or something like that. I simply want to avoid having the user install each module separately.
thanks.
pip uses requirements files, which have a very straightforward format.
For more Python packaging tooling recommendations, see the latest from the Python Packaging Authority (PyPA).
See the setuptools docs on how to declare your dependencies -- this will allow easy_install to find, download and install all of them (and transitive closure thereof) if everything's available in PyPi, or otherwise if you specify the dependencies' URLs.
Related
I'm trying to build a speech recognition application that works in a browser. Currently using pyodide with a web worker. I have my own package, built alongside pyaudio, that I use for the web worker.
Is there a way that I can include a brew instillation, specifically portaudio, inside of my python package so that when I build the package, portaudio is included in the wheel file? I need portaudio included for this to work in the browser.
Thank you!
I'm understanding two different questions here, so I'll try to answer them both.
Can I have a python build fetch a Homebrew project during buildtime
To my knowledge, the answer is no. The Python distribution system is separate from Homebrew, and they can't interact in this fashion.
Even if they could, this wouldn't necessarily be desirable:
What happens if the user isn't on macOS (or Linux)? Then the build would fail.
The prefix that Homebrew will install the package in isn't very deterministic. The user might be using a custom prefix, or they might be on Apple Silicon (which has a different default prefix to Intel).
Your python package might run into some difficulty locating the package.
What about if they don't have Homebrew installed? They might have another package manager like MacPorts or Fink, or maybe none at all.
Can I bundle portaudio into the build distribution?
Maybe? Even if you could, I almost certainly wouldn't recommend it.
Bundling dependencies increases the size of the distribution unnecessarily.
It would take a reasonable amount of effort to setup, assuming you can do it.
All these reasons are why for the majority of projects that have a similar setup, you will find that they recommend installing certain packages with their system package manager first, before building the Python source code.
This allows them to choose whatever package manager they have installed, and it should also be a quick and painless process.
Therefore, just change your installation instructions to the following:
# On macOS
brew install portaudio
pip install ...
I am looking for a way to express something that resembles NPM's peerDependencies in setuptools.
My Python library is a plugin that should work with another Python library that I don't want to have as a dependency. Instead I'd like my end user to be responsible for it and install it by themselves. I can't find a proper way to express this in Python's setuptools (or any other build for that matter), to let my user "know" about the library
From my current understanding, this is a close approximation between the tools:
NPM
setuptools
dependencies
install_requires
optionalDependencies
extras_require
peerDependencies
???
I have two possible solutions, both of which I find lacking:
Use extras_require anyway
Specify my requirements under an extras_require will do the job, but it feels like abusing it, because the user shouldn't install these extras.
Just document it
Inform the user that they need to install that package separately. Feels lame too
To my understanding, there is no need for that, as Python handles dependencies different anyway.
With npm/node you can have multiple versions of one package withing the same environment, if it required by one of the sub-dependencies (How does NPM handle version conflicts?)
This is not possible in Python. Only a single version of a library can be installed in the one environment. This means that the general issue that you are trying to solve with peer-dependencies, can not happen.
My recommendation is to specify your dependencies as normal, but with a loose version specifier.
Why we have to install the python packages before using them?
I am currently working on a small python mysql program. What i tried to download the python connector module from mysql webpage and simply unzip it and place it in the same folder of my code.
And I can import the module properly.
So what is the meaning of installing those packages? Can I use those packages like matplotlib, numpy without installing them ?
Is it possible to have all the required packages installed on a folder so that i can move it to another computer and run my program with only CPython installed (I don't want to install any package on this computer)?
it's not that simple :-)
some packages have dependencies, you also need to download and extract their dependencies (you need pacakge x,and package x uses y) pakcage manager handles that
some package have some c code (they need to be compiled before use (ujson or postgres module) package manager handles that
when your share your code instead of sharing dependencies you simply add a file containing the list of dependencies (requirements.txt) and other user can simply install all dependencies using package manager
Installing a python package enables us to use it anywhere on our system. If we just place the package in the same directory as our script then it may well work, but only for scripts in that directory.
Some packages also rely on others to function properly, and the installation of a package may well install those pre-requisite packages for you. You may be able to do this manually, but you'd have to put them all in the same directory as your script every time you wanted to run it.
So installing the packages is the easiest way to use them.
You don't have to install them, and in some cases you wouldn't install them on your system; if you had split your code across two files and imported one file at the top of the other for example.
In fact, you don't really need install package on your system.
But if you install it, you can use these packages every where on your system.
Also, you can create a requirement.txt file to enable install all packages that you need on other computer. You can check this manual https://pip.pypa.io/en/stable/user_guide/#requirements-files
Does Python have anything similar to apt or Maven where a single repository can house different versions of a library as opposed to just the current version?
For example: My site-packages folder does not group libraries by version. So instead of:
/Library/Python/2.7/site-packages/tox/1_2_3
We have:
/Library/Python/2.7/site-packages/tox
...which presumably contains the latest version of tox which may or may not be compatible with every piece of software on my system that wants to use tox. Is there a versioned approach to this? If not, is it possible to create one?
No, there is no way to install multiple versions of a package in the same environment, nor import multiple versions in the same process. The commonly accepted way to handle specific versions for separate projects is to set up a virtualenv for that project and install the specific requirements.
Is there a way to install/generate the documentation for the packages installed using pip?
I wish to install all the required packages for a project, as well as all the associated documentation (e.g. Django documentation when installing django, dateutil documentation with dateutil etc.).
Pip requirements files are a great way of quickly installing the required packages for a project, but it would be even better if I could also install all the associated docs as well.
Ubuntu Python packages install documentation to /usr/share/docs, but pip does not appear to do the same.
Documentation for these packages is important to me for when I need to work on projects offline.
I think you're looking for an equivalent to the way that ruby automatically (unless suppressed) generates rdoc from installed gems/packages.
In python, there is a standardized mechanism for annotating code with documentation -- docstrings with optional formatting. However there isn't a standardized way of generating/storing documentation from python code. Each python package may have a different mechanism, so there couldn't be a way for pip to generate it.