We Keep Coding

Best practice for common functions used by several Python projects

now I have a folder named my_funcs which have __init__.py and some .py files containing some functions and classes I wrote that I want to use for several projects.
So I want to know the best practice for these projects to direct import from this folder.
one solution is to sys.path.append('.../my_funcs'), in this case I will have to put this in front of the import statement for every .py file.
Any suggestions? BTW, I'm on Windows

Best is to use PYTHONPATH. Set it to the path where your common modules are found, before running Python. Then you can just do import my_funcs for example.

Checkout PEP370, "Per user site-packages directory".
You set PYTHONUSERBASE to somewhere under your control, and you can see
$ PYTHONUSERBASE=/home/me/.local
$ python -m site --user-site
/home/me/.local/lib/python2.7/site-packages
Your personal directory now appears in sys.path. easy_install respects this (using the --user option) so you can install "real" packages there, also, but make them available only to you.
Depending on your configuration, sounds like you could move your my_funcs directory under the site-packages directory and you're done!

How to switch between test code and production code in python?

I have a project that is constantly undergoing development. I have installed a release of the project in my python distribution's site-packages directory using the setup.py script for the project.
However, when I make changes to the project I would like my test scripts to find the files that are under the project's directory and not those that it finds in site-packages. What is the proper way to do this? I only know of one approach which is to modify the search path in the test script itself using sys.path, but then it means that I cannot use the same scripts to test the "installed" version of my codes without editing the sys.path again.

I'm not quite sure what you are asking but you could use
python setup.py develop to create a develop version of your project
https://pythonhosted.org/setuptools/setuptools.html#development-mode
Under normal circumstances, the distutils assume that you are going to
build a distribution of your project, not use it in its “raw” or
“unbuilt” form. If you were to use the distutils that way, you would
have to rebuild and reinstall your project every time you made a
change to it during development.
Another problem that sometimes comes up with the distutils is that you
may need to do development on two related projects at the same time.
You may need to put both projects’ packages in the same directory to
run them, but need to keep them separate for revision control
purposes. How can you do this?
Setuptools allows you to deploy your projects for use in a common
directory or staging area, but without copying any files. Thus, you
can edit each project’s code in its checkout directory, and only need
to run build commands when you change a project’s C extensions or
similarly compiled files. You can even deploy a project into another
project’s checkout directory, if that’s your preferred way of working
(as opposed to using a common independent staging area or the
site-packages directory).

Use "Editable" package installation like:
pip install -e path/to/SomeProject
Assuming we are in the same directory with setup.py, the command will be:
pip install -e .

Managing PYTHONPATH for development environment

I'm developing several different Python packages with my team. Say that we have ~/src/pkg1, ~/src/pkg2, and ~/src/pkg3. How do we add these to PYTHONPATH without each of us having to manage dot-files?
We could add, say, ~/src/site/sitecustomize.py, which is added once to PYTHONPATH, but is it "guaranteed" that there won't be a global sitecustomize.py.
virtualenv seems like the wrong solution, because we don't want to have to build/install the packages after each change.

You have a lot of options...
1) Why not dotfiles?
You could centralize the management of dotfiles with a centralized repository and optionally with version control. I use a Dropbox folder named dotfiles but many people use github or other services like that to manage dotfiles.
If you do that, you will guarantee every people on your development team to share some dotfiles. So you could define a dotfile say .python_proys which export the appropriate PATH and PYTHONPATH which by convention every developer should source in their environment.
Suppose pkg1 is only an script, pkg2 is an script and also a module and pk3 is only a module. Then, python_proys Example:
export PATH=$PATH:~/src/pkg1:~/src/pkg2
export PYTHONPATH=$PYTHONPATH:~/src/pkg2:~/src/pkg3
And then, every developer have to source this dotfile somewhere by convetion. Each one will do the way he like. One could source the dotfile manually before using the packages. Another one could source it in his .bashrc or .zshenv or whatever dotfile apply to him.
The idea is to have one centralized point of coordination and only one dotfile to maintain: the .python_proys dotfile.
2) Use symlinks
You could define a directory in your home, like ~/dist (for modules) and ~/bin (for scripts) and set symbolic links there to the specific pakages in ~/src/, and make every developer have this PATH and PYTHONPATH setting:
export PATH=$PATH:~/bin
export PYTHONPATH=$PYTHONPATH:~/dist
So, using the same example at Why not dotfiles?, where pkg1 is only an script, pkg2 is an script and also a module and pkg3 is only a module, then you could symlink like:
cd ~/bin
ln -s ../src/pkg1
ln -s ../src/pkg2
cd ~/dist
ln -s ../src/pkg2
ln -s ../src/pkg3
Those commands could be do automatically with an script. You could write a bootstrap script, or simply copy and paste the commands and save it in a shell script. In any way, maintain it and centralize it the same way i explain it before.
This way the .dotfiles will not change, only the script defining the symlinks.

I suggest looking into creating a name.pth path configuration file as outlined in thesitemodule's documentation. These files can hold multiple paths that will be added tosys.pathand be easily edited since they're simply text files.

First, you don't add a python module to PYTHONPATH, you just add the path component.
If you want all your team to be working on some python package, you can install the package as editable with the -e option in a virtual environment.
This way you can continue development and you don't have to mess with the PYTHONPATH. Keep in mind that the working directory is always included in the PYTHONPATH, so unless you have an external requirement; you don't need a virtual environment, just the source in your working directory.
Your workflow would be the following:
Create virtual environment
Create a .pth file, to modify your PYTHONPATH.
Work as usual.
This would be my preferred option. If you have a standard layout across your projects, you can distribute a customized bootstrap script which will create the environment, and then adjust the PYTHONPATH automatically. Share this bootstrap script across the team, or add it as part of the source repository.

I assume that your other modules are at predictable path (relative to $0)
We can compute absolute path of $0
os.path.realpath(sys.argv[0])
then arrive at your module path and append it
sys.path.append(something)

Should I add my Python project to the site-packages directory, or append my project to PYTHONPATH?

I have a Python project which contains three components: main executable scripts, modules which those scripts rely on, and data (sqlite3 databases, flat files, etc.) which those scripts manipulate. The top level has an __init__.py file so that other programs can also borrow from the modules if needed.
The question is, is it more "Pythonic" or "correct" to move my project into the default site-packages directory, or to modify PYTHONPATH to include one directory above my project (so that the project can be imported from)? On the one hand, what I've described is not strictly a "package", but a "project" with data that can be treated as a package. So I'm leaning in the direction of modifying PYTHONPATH (after all, PYTHONPATH must exist for a reason, right?)

Definitely do not add your project to site-packages this is going to spoil your system Python installation and will fire back at the moment some other app would come there or you would need to install something.
There are at last two popular options for installing python apps in isolated manner
Using virtualenv
See virtualenv project. It allows
creation of new isolating python environment - python for this environment is different from system one and has it's own PYTHONPATH setup this allows to keep all installed packages private for it.
activation and deactivation of given virtualenv from command line for command line usage. After activate, you can run pip install etc. and it will affect only given virtualenv install.
calling any Python script by starting by virtualenv Python copy - this will use related virtualenv (note, that there is no need to call any activate)
using zc.buildout
This package provides command buildout. With this, you can use special configuration file and this allows creation of local python environment with all packages and scripts.
Conclusions
virtualenv seems more popular today and I find it much easier to learn and use
zc.buildout might be also working for you, but be prepared for a bit longer learning time
installing into system Python directories shall be always reserved for very special cases (pip, easy_install), better avoid it.
installing into private directories and manipulatig PYTHONPATH is also an option, but you would repeat, what virtualenv already provides

Separate Python paths for development and production

I'm using Python with a Cygwin environment to develop data processing scripts and Python packages I'd like to actively use the scripts while also updating the packages on which those scripts depend. My question is what is the best practice, recommendation for managing the module loading path to isolate and test my development changes but not affect the working of a production script.
Python imports modules in the following order (see M. Lutz, Learning Python)
Home directory.
PYTHONPATH directories.
Standard library directories.
The contents of any *.pth file.
My current solution is to install my packages in a local (not in /usr/lib/python2.x/ ) site-packages directory and add a *.pth file in the global site-packages directory so these are loaded by default. In the development directory I then simply modify PYTHONPATH to load the packages I'm actively working on with local changes.
Is there a more standard way of handling this situation? Setting up a virtualenv or some other way of manipulating the module load path?

This is just my opinion, but I would probably use a combination of virtualenvs and Makefiles/scripts in this case. I haven't done it for your specific use case, but I often set up multiple virtualenvs for a project, each with a different python version. Then I can use Makefiles to run my code or tests in one or all of my virtualenvs. Seems like it wouldn't be too hard to set up a makefile that would let you type make devel to run in the development envionment, and make production for the production environment.
Alternatively, you could use git branches to do this. Keep your production scripts on master, and use feature branches to isolate and test changes while still having your production scripts just a git checkout master away.