I have a Python project stored in GitHub that is going to use tweepy, a Twitter Python library stored in GitHub too. In the INSTALL file says that I can use git to bundle it in my project but I have no idea on how to do this. I thought to use git submodule but this will fetch all the project, not only the sources that is what I need. How could I do it? Is there a best way to work with external libraries in a project?
You can't get just a subdirectory with git-submodule (see this question).
Bundling is also bad practice (it makes updating the library code in the case of a library bug more difficult, and results in useless duplication). tweepy is on PyPI, so the best way would not be bundling it, but to require it in your setup.py or list it in your requirements.txt (depends on whether you're packaging it for PyPI or just building need to have an easy way to install your projects dependencies). You can specify what versions of the library are permissible when doing so, so tweepy can't change out from under your feet.
If you're planning on making changes to tweepy, the polite thing would be to submit those changes to the project, rather than making the changes within your project (or perhaps maintaining your own fork on GitHub if tweepy won't accept the changes). Note that pip has ways to install git versions of a package if you need a version that's not available in PyPI.
Related
The language around packaging python projects in the documentations suggests that it is exclusively for distributing or exporting the project. So what should I do with projects I intend to use personally?
Should I package them anyway? If not, what steps should I take so that I can run my code on any machine with the right version of python? Would packaging a project even accomplish that? Is there even any way to "package" all of a project's files and relevant libraries together, with the end product being a folder/file rather than an install-able package?
I'm sorry if this is basic, I'm very confused, thank you for your time.
For personal use I do not package, except I am very sure there is no need to modify it anymore. And that is very rarely my case. Once it is packaged and published, public or private package repository you get the package and to make changes is more complicated but possible. I prefer to have the project repository and be able to edit and push the changes to remote locations.
Many packaging tools like poetry make it easy to build but also to install requirements and keep track of them. So, there is no hustle with managing requirements.
I am working on a python project that I have hosted on github. I plan to release an initial version to the community. The problem is this: I want people to be able to update it newer versions as and when available by running some command like this:
cf-cli.py update
What are some of the most native ways to this? --that which does not require any additional installation other than python itself.
Add it to PyPI - the Python Package Index.
The Python Package Index is a repository of software for the Python programming language.
Your users can install it and upgrade it as new versions come out as and when they want to without any issues.
You will have to do some basic work like creating a .pypirc file and make accounts on PyPI Live and PyPI Test.
You'll also have to make a setup.py and setup.cfg and of course a LICENSE.txt.
After all this you'll have to upload them to PyPI and you're done.
Here's an in depth tutorial that covers the steps in much greater detail.
If one creates a useful Python package, how/where does one publish/advertise it for other people to use?
I've put it on hithub, but even Google does not find it after a few weeks.
The package is neat & complete, I made it for my personal use and would be a shame not to share it with others :)
Here is the PyPI guide. https://python-packaging-user-guide.readthedocs.org/en/latest/distributing.html
PyPI is the place for putting your Python packages up for others to find. The built-in tool pip references it to install packages for you, and at least one IDE uses pip in the background to give you a GUI for doing this. (PyCharm)
So, to make the package available to a pip install, you have to register it in the Python Package Index (PyPI): https://pypi.python.org/pypi
There's also the test environment, where you can upload your packages to test if your setup is ok before going to the real deal: https://testpypi.python.org/pypi
You create an account in one of the servers and will be able to upload your package. But, before that, you will have to build your package using setuptools. Here's the documentation for packaging and distributing: https://packaging.python.org/distributing/
The proccess can be little boring, so I wrote a little tool to make it simpler. Maybe it's of some use to you: https://github.com/hugollm/foster
I have a virtualenv that serves a few separate projects. I'd like to write a utility library that each of these projects can use. It seems to make sense to stick that in the virtualenv. By all means shoot me down now but please give an alternative if you do.
Assuming I'm not completely crazy though, Where's the best place to stick my library?
My virtualenv sticks everything I install with pip in lib/pyton2.7/site-packages. I wonder if it would make more sense to follow suit or to hack in a separate home (further up the directory tree) so if things do ever clash, my work isn't overwritten by pip (et al).
If your project follows the standard packaging practices with setuptools, then all you have to do is run python setup.py develop inside the virtualenvs that you want the library to be used for. A .egg-link file will be created pointing to your package from which your other libraries will use much like any other packages, with the added benefit that your latest changes will be available to all packages at the same time (if that's your intention). If not, then either you could call python setup.py install or use multiple versions at different locations on the file system.
To get yourself started, take a look at Getting Started With setuptools and setup.py (you can skip the part on registering your package on pypi if this is your private work).
Another relevant stackoverflow thread: setup.py examples? The Hitchhiker's Guide to Packaging can also be quite useful.
My package has a dependency on the latest version of the jsonpickle package. Older versions are installable via pip, however I need the latest version (i.e on Github) for it to work. Is it generally considered OK in this situation to bundle the latest version of jsonpickle in my code? Is there some other solution? I'd rather not ask my users not to clone from github.
I'm thinking of organising my package like this:
My package
|
__init__.py
file1.py
file2.py
\
jsonpickle (latest)
i.e Doing what was done here: Python: importing a sub‑package or sub‑module
As kag says, this is generally not a good idea. It's not that it's "frowned upon" as being unfriendly to the other packages, it's that it causes maintenance burdens for you and your users. (Imagine that there's a bug that's fixed in jsonpickle that affects your users, but you haven't picked up the fix yet. If you'd done things normally, all they'd have to do is upgrade jsonpickle, but if you're using an internal copy, they have to download the jsonpickle source and yours, hack up your package, and install it all manually.)
Sometimes, it's still worth doing. For example, the very popular requests module includes its own copy of other packages like urllib3. And yes, it does face both of the costs described above. But it also means that each version of request can rely on an exact specific version of urllib3. Since requests makes heavy use of urllib3's rarely-used interfaces, and even has workarounds for some of its known bugs, that can be valuable.
In your case, that doesn't sound like the issue. You just need a bleeding-edge version of jsonpickle temporarily, until the upstream maintainers upload a new version to PyPI. The problem isn't that you don't want your users all having different versions; it's that you don't want to force them to clone the repo and figure out how to install it manually. Fortunately, pip takes care of that for you, by wrapping most of the difficulties up in one line:
pip install git+https://github.com/foo/bar
It's not a beautiful solution, but it's only temporary, right?
It's generally not the best idea to bundle some dependency with your project. Some projects do it anyway, or bundle it as an alternative if there's no system package available. (This is mostly found in C projects, not Python.)
You didn't mention what the "latest" means exactly. Is this the latest in pypi?
The best way to make sure a specific version, or greater than a baseline version, of a package is installed is to properly specify the requirement in setup.py requires section. Read more about requires here [1]. This way pip can take care of resolving dependencies, and if it's available in pypi it will be automatic.
[1] http://docs.python.org/2/distutils/setupscript.html#relationships-between-distributions-and-packages