How to install python package without copy everything into lib/site-packages? - python

I want to develop a common python package, I got other packages depends on it. For example:
packageA/
packageB/
packageC/
commonPackage/
packageA, packageB and packageC can all be executed directly, but they are all depend on commonPackage. I want to install the commonPackage into lib/site-packages, but I don't want it copys the source code. Instead, I want it creates a commonPackage.pth in lib/site-packages with the path of where the commonPackage at. So that when I modify commonPackage or update it from SVN, I don't need to install it again. Here comes the problem, how can I write the setup.py or use the options of python setup.py install so that it would do what I want?

Oops, I just find exactly what I want here. The develop command of setuptools do what I said. Here you type
python setup.py develop
It creates .pth rather than copying everything into site-packages.

You can always take a look at virtualenv which will allow you to create a python environment for each of your projects - this is the ideal way to develop/build/deploy your app without having load up your site-packages directory with all and sundry.
There's a good tutorial here:
http://iamzed.com/2009/05/07/a-primer-on-virtualenv/
Good luck !

Related

How to reverse py setup.py build or py setup.py install and save package contents locally?

This is the first time I'm writing a prioprietary, local library so I'm totally new in this matter. I've been looking for a proper solution but maybe I'm missing something. Anyway...
The main code is in some non-Python folder. I install this package in three steps:
First (excluding the last one substep)
Second (py setup.py build)
py setup.py install
I'm using this method because when I tried pip it didn't work for me.
Because it's a new package I still make many changes. After installation, I do it in the source code in site-packages Python folder.
My question is - how to reverse this process of installation in order to save source code files to a selected, non-Python folder? I prefer to avoid manually copying files or using pip uninstall and yet repeating the process described above to install package after editing the code.
In other words I would love to work on my package locally and automatically update it in site-packages OR the other way round.
I will be grateful for any response.
Python version I'm writing in - 3.10.1. IDE - VSC.

Proper way to customize third-party python package in project [duplicate]

I installed some package via pip install something. I want to edit the source code for the package something. Where is it (on ubuntu 12.04) and how do I make it reload each time I edit the source code and run it?
Currently I am editing the source code, and then running python setup.py again and again, which turns out to be quite a hassle.
You should never edit an installed package. Instead, install a forked version of package.
If you need to edit the code frequently, DO NOT install the package via pip install something and edit the code in '.../site_packages/...'
Instead, put the source code under a development directory, and install it with
$ python setup.py develop
or
$ pip install -e path/to/SomePackage
Or use a vcs at the first place
$ pip install -e git+https://github.com/lakshmivyas/hyde.git#egg=hyde
Put your changes in a version control system, and tell pip to install it explicitly.
Reference:
Edit mode
You can edit the files installed in /usr/local/lib/python2.7/dist-packages/. Do note that you will have to use sudo or become root.
The better option would be to use virtual environment for your development. Then you can edit the files installed with your permissions inside your virtual environment and only affect the current project.
In this case the files are in ./venv/lib/pythonX.Y/site-packages
The path could be dist-packages or site-packages, you can read more in the answer to this question
Note that, as the rest of the people have mentioned, this should only be used sparingly, for small tests or debug, and being sure to revert your changes to prevent issues when upgrading the package.
To properly apply a change to the package (a fix or a new feature) go for the options described in other answers to contribute to the repo or fork it.
I too needed to change some things inside a package. Taking inspiration from the previous answers, You can do the following.
Fork the package/repo to your GitHub
clone your forked version and create a new branch of your choice
make changes and push code to the new branch on your repository
you can easily use pip install -e git+repositoryurl#branchname
There are certain things to consider if its a private repository
If you are doing the custom module that you want hot loading, you can put your running code also inside the module. Then you can use python -m package.your_running_code. In this way, you can change the module in the package and reflect the result of your running code immediately.

setup.py + virtualenv = chicken and egg issue?

I'm a Java/Scala dev transitioning to Python for a work project. To dust off the cobwebs on the Python side of my brain, I wrote a webapp that acts as a front-end for Docker when doing local Docker work. I'm now working on packaging it up and, as such, am learning about setup.py and virtualenv. Coming from the JVM world, where dependencies aren't "installed" so much as downloaded to a repository and referenced when needed, the way pip handles things is a bit foreign. It seems like best practice for production Python work is to first create a virtual environment for your project, do your coding work, then package it up with setup.py.
My question is, what happens on the other end when someone needs to install what I've written? They too will have to create a virtual environment for the package but won't know how to set it up without inspecting the setup.py file to figure out what version of Python to use, etc. Is there a way for me to create a setup.py file that also creates the appropriate virtual environment as part of the install process? If not — or if that's considered a "no" as this respondent stated to this SO post — what is considered "best practice" in this situation?
You can think of virtualenv as an isolation for every package you install using pip. It is a simple way to handle different versions of python and packages. For instance you have two projects which use same packages but different versions of them. So, by using virtualenv you can isolate those two projects and install different version of packages separately, not on your working system.
Now, let's say, you want work on a project with your friend. In order to have the same packages installed you have to share somehow what versions and which packages your project depends on. If you are delivering a reusable package (a library) then you need to distribute it and here where setup.py helps. You can learn more in Quick Start
However, if you work on a web site, all you need is to put libraries versions into a separate file. Best practice is to create separate requirements for tests, development and production. In order to see the format of the file - write pip freeze. You will be presented with a list of packages installed on the system (or in the virtualenv) right now. Put it into the file and you can install it later on another pc, with completely clear virtualenv using pip install -r development.txt
And one more thing, please do not put strict versions of packages like pip freeze shows, most of time you want >= at least X.X version. And good news here is that pip handles dependencies by its own. It means you do not have to put dependent packages there, pip will sort it out.
Talking about deploy, you may want to check tox, a tool for managing virtualenvs. It helps a lot with deploy.
Python default package path always point to system environment, that need Administrator access to install. Virtualenv able to localised the installation to an isolated environment.
For deployment/distribution of package, you can choose to
Distribute by source code. User need to run python setup.py --install, or
Pack your python package and upload to Pypi or custom Devpi. So the user can simply use pip install <yourpackage>
However, as you notice the issue on top : without virtualenv, they user need administrator access to install any python package.
In addition, the Pypi package worlds contains a certain amount of badly tested package that doesn't work out of the box.
Note : virtualenv itself is actually a hack to achieve isolation.

How to "build" a python script with its dependencies

I have a simple python shell script (no gui) who uses a couple of dependencies (requests and BeautifulfSoup4).
I would like to share this simple script over multiple computers. Each computer has already python installed and they are all Linux powered.
At this moment, on my development environments, the application runs inside a virtualenv with all its dependencies.
Is there any way to share this application with all the dependencies without the needing of installing them with pip?
I would like to just run python myapp.py to run it.
You will need to either create a single-file executable, using something like bbfreeze or pyinstaller or bundle your dependencies (assuming they're pure-python) into a .zip file and then source it as your PYTHONPATH (ex: PYTHONPATH=deps.zip python myapp.py).
The much better solution would be to create a setup.py file and use pip. Your setup.py file can create dependency links to files or repos if you don't want those machines to have access to the outside world. See this related issue.
As long as you make the virtualenv relocatable (use the --relocatable option on it in its original place), you can literally just copy the whole virtualenv over. If you create it with --copy-only (you'll need to patch the bug in virtualenv), then you shouldn't even need to have python installed elsewhere on the target machines.
Alternatively, look at http://guide.python-distribute.org/ and learn how to create an egg or wheel. An egg can then be run directly by python.
I haven't tested your particular case, but you can find source code (either mirrored or original) on a site like github.
For example, for BeautifulSoup, you can find the code here.
You can put the code into the same folder (probably a rename is a good idea, so as to not call an existing package). Just note that you won't get any updates.

Make an installer for python project

I downloaded an open source project http://gmapcatcher.googlecode.com/files/GMapCatcher-0.7.2.0.tar.gz and I am trying to modify a few things in the code but don't know how to test the code!
I tried to make an installer for the project but nothing worked till now maybe I didn't follow the right steps or I am missing somthing.
my question is how can I modify the code and test it ? and how can I make an installer for this project (I know there is an installer already in google but I want to make it myself).
Looks like the package has a setup.py for the use with distutils. The setup.py works kind of like Makefile for python. The way you use it is (in the directory where setup.py is located:
$ python setup.py command
Where "command" is... well... a command. Type
$ python setup.py --help
for more information. The two basic commands are build and install. install installs, as the name suggests, the package to your system. It is not probably going to do anything like create shortcuts on your desktop or anything like that. It simply installs the Python package into your Python installation. Judging by the contents of setup.py, it seems they're somehow using py2exe (google it; being a newbie I can only include one hyperlink in my answer) to prepare the Windows installer.
To simply run the software, however, it seems all you need to do is to unpack it and do
$ python maps.py
in the package's root directory - provided you have all the necessary dependencies already installed, of course.

Categories