We Keep Coding

Enable module modification in virtualenv

I have 10 django projects that use over 50 django apps. Each app is separated in its own project and added to pypi and is getting use by few project. Every thing if fine except every time i work on a project and i want to change some code that is in one of my modules (that happens a lot) I have to open the module project, make my changes, test and publish to pypi then come back to my project update requirements.txt file and get the updated module from pip.
I'm looking for a way to be able to edit module right away from all of my projects. For example instead of getting it from pypi i want to get it from git and be able to commit to the git repository in my venv folder!
I know it seems a little bit crazy but i could save a lot of time! publisher and user of all of the modules is me so I don't mind the user to be able to change as well.
Any thought or suggestion will be appreciated. Also any none pip solution will be fine as well like writing a custom shell script.

I don't know about editing in your venv folder, which I think is not a good practice, but you can install from github by pip. You can use 'pip install git+https://github.com/urltoproject/repository.git'. Fill in the necessary details yourself of course. This also works with other systems like gitlab. You could have a separate development requirement file and a production requirement file to separate the two environments, or you install on the commandline directly with pip.

What does install django-app as a user library means?

I just finished my first Django-app and I now have a .tar.gz file. (It's my first one ever: :'))
After having the .tar.gz file I was like ok... now what's next? How do I install this? I found this question: How do I install Python/Django Modules? And I was reading #Miku answer to the question and I was thinking: Ok, if I just do pip install --user django-polls/dist/django-polls-0.1.tar.gz (from Advanced tutorial: How to write reusable apps documentation!) that means that my django proyect should be able to find my app after I add the app to
INSTALLED_APPS?
Because I was following the tutorial and I had the app folder right next to manage.py file. And know pip says my app is installed and not folder has been created in my project directory.
If this is a really stupid question, please, pardon me, I'm really new to django. :-)

When you install a python package as a library, it's installed to the python's packages folder, so it can be used from anywhere in the system.
So yes, your Django project should be able to find your app when you add its name in INSTALLED_APPS. If your app doesn't have its own models, you don't even have to include it in the installed apps, and just import from it when you need to use it.
I'm not sure what system you are on, but in Linux, usually installed python packages are located in /usr/lib/{PYTHON VERSION}/site-packages/
(just for a reference)

find django/contrib/admin/templates

I have trouble to see django/contrib/admin/templates folder. It seems like it is hidden in /usr/lib/python2.7/dist-packages/ folder, ctrl+h wont help ( appearencely all django files are hidden).
"locate django/contrib/admin/templates" in terminal shows bunch of files, but how can i see those files in GUI? I use Ubuntu 12.10
Thanks in advance

To see where your django installation resides, run this at the command line:
python -c "
import sys
sys.path = sys.path[1:]
import django
print(django.__path__)"
On my system, this returns
['/usr/local/lib/python2.7/site-packages/django']
Source: Django Docs

You should not mess with your system-specific python setup because it is used as a dependency for other programs (which are use python). For example, a manual update of a package in /usr/lib/python2.7/site-packages/ can break a program and also requires root permissions.
Instead, you should create a virtualenv and install django in it:
# create an isolated python environment
virtualenv ~/your_env
# activate this environment, this means that you don't need to mess with your /usr system anymore
source ~/your_env/bin/activate
# use python's standard package manager to install django in the virtualenv
# does not require special permissions
pip install Django
# it will install in: ~/your_env/lib/python2.7/site-packages/
virtualenvs are isolated, safe, and work with your regular user permissions.

Should be here: /usr/lib/python2.7/site-packages/django/contrib/admin/templates

Since, everyone is posting my comment's suggestion, might as well post it myself. Try looking at:
/usr/lib/python2.6/site-packages/django/

I think you should be looking in site-packages. Assuming you're using django 1.4 it should be -
/usr/lib/python2.7/site-packages/django/contrib/admin/templates

If you are using Python3, Django is located at your venv. In my case templates are located at <project_root>/venv/lib/python3.5/site-packages/django/contrib/admin/templates/.

How do you arrange your site-packages folder in python

I just downloaded django and installed it into C:\Python27\Lib\site-packages So now my site-packages directory seems rather unorganized with 5 separate folders a bunch ot text files and a python script. Should I create a sub directory in site-packages for the django package and store everything in there? If I do that will the python intrepeter know to look in the Django directory in the site-packages directory for the django package? Or will I have to change some environment variables around?
Thank you.

I'm not sure if you're asking it because this is something you need to know because of some complex setting you have. In case you are just want to install Django, you should follow the install instructions from their site:
pip install Django
the directory order in site-packages is an internal mechanism of your Python build and I don't think you should change it, unless you want to create your own package and want to learn more on how python find installed packages (and if this is the case I would start reading this document: http://docs.python.org/2/distutils/index.html)
As suggested in the comments to your question, and also in the documentation of Django, you should consider using virtualenv, especially when developing a webapp which will get deployed on a server with a specific Django version. Over time you'll see that you want to create other apps that require different versions of modules, and to still support the already existing apps you will need to use virtualenv.

Best practice for installing python modules from an arbitrary VCS repository

I'm newish to the python ecosystem, and have a question about module editing.
I use a bunch of third-party modules, distributed on PyPi. Coming from a C and Java background, I love the ease of easy_install <whatever>. This is a new, wonderful world, but the model breaks down when I want to edit the newly installed module for two reasons:
The egg files may be stored in a folder or archive somewhere crazy on the file system.
Using an egg seems to preclude using the version control system of the originating project, just as using a debian package precludes development from an originating VCS repository.
What is the best practice for installing modules from an arbitrary VCS repository? I want to be able to continue to import foomodule in other scripts. And if I modify the module's source code, will I need to perform any additional commands?

Pip lets you install files gives a URL to the Subversion, git, Mercurial or bzr repository.
pip install -e svn+http://path_to_some_svn/repo#egg=package_name
Example:
pip install -e hg+https://rwilcox#bitbucket.org/ianb/cmdutils#egg=cmdutils
If I wanted to download the latest version of cmdutils. (Random package I decided to pull).
I installed this into a virtualenv (using the -E parameter), and pip installed cmdutls into a src folder at the top level of my virtualenv folder.
pip install -E thisIsATest -e hg+https://rwilcox#bitbucket.org/ianb/cmdutils#egg=cmdutils
$ ls thisIsATest/src
cmdutils

Are you wanting to do development but have the developed version be handled as an egg by the system (for instance to get entry-points)? If so then you should check out the source and use Development Mode by doing:
python setup.py develop
If the project happens to not be a setuptools based project, which is required for the above, a quick work-around is this command:
python -c "import setuptools; execfile('setup.py')" develop
Almost everything you ever wanted to know about setuptools (the basis of easy_install) is available from the the setuptools docs. Also there are docs for easy_install.
Development mode adds the project to your import path in the same way that easy_install does. An changes you make will be available to your apps the next time they import the module.
As others mentioned, you can also directly use version control URLs if you just want to get the latest version as it is now without the ability to edit, but that will only take a snapshot, and indeed creates a normal egg as part of the process. I know for sure it does Subversion and I thought it did others but I can't find the docs on that.

You can use the PYTHONPATH environment variable or symlink your code to somewhere in site-packages.

Packages installed by easy_install tend to come from snapshots of the developer's version control, generally made when the developer releases an official version. You're therefore going to have to choose between convenient automatic downloads via easy_install and up-to-the-minute code updates via version control. If you pick the latter, you can build and install most packages seen in the python package index directly from a version control checkout by running python setup.py install.
If you don't like the default installation directory, you can install to a custom location instead, and export a PYTHONPATH environment variable whose value is the path of the installed package's parent folder.