We Keep Coding

import error that doesn't replicate in different clones of the same repository

I have a big python project. When I try to execute a certain script I get an import error
something like:
from my_package import xyz
ImportError: cannot import name 'xyz'
Where my_package is an existing directory in the code base, and xyz.py is within this directory. Yes, I checked, and this directory is in the search path.
What really boggles my mind is that on a different machine, when I clone the same repository, activate the same virtual environment, and try the exact same script - I don't get this error.
I try to figure what is wrong and how to fix it, and the fact that it happens on a remote server, and does not replicate on my own computer just seem weird.
Any clue what could go wrong and where should I look for the bug?
EDIT: some additional information -
When I ran the script as it should i.e.
python ./myscript.py
It gives the above error. But when I run the problematic import command within the interpreter:
python
from my_package import xyz
I get no error whatsoever.

The three potential errors I can think of are the following:
Does the git project use submodules? If so, you need to make sure you run a git clone with the --recursive flag, since it might not have downloaded all the submodules correctly.
How are you setting up the virtual environments? Usually they shouldn't be stored within the git repository, but should instead be created on each machine. The dependencies can then be installed from the project's requirements.txt file. Sharing the same virtual environment between multiple machines could mean that it doesn't behave correctly on one of them.
Have you compared the versions of Python used between both machines. If it's a built-in module that's failing to import then perhaps your second machine isn't using a new enough version of Python.

(Dumb noob) Will changing directory in cmd.exe prior to installing a third party module, cause that module to be stored in that directory?

I want to install python, and then some additional modules, to my computer (windows).
I've been told that the best way to install python and packages is to make a folder in your drive somewhere BEFORE installing anything, (ex. 'Python') then pointing all your downloads (using cd in the command line) to this folder so that everything is in once place, and you don't have to go on a wild goose chase to make sure everything you want access to is in your PATH when you go to import modules etc.
Do I have the right idea?
I have had trouble importing modules in the past because they were not in the Path.
Will changing the directory in the command line before typing:
pip install somemodule
cause that module to be saved to where I just changed the directory to?

pip always installs the libraries in a fixed directory, usually in the user folder. You can check this by the command pip show <installed-package-name>
So you can use any package you already installed to get the pip directory. The location might vary based on your python version and env name.
Example: c:\users\<user>\appdata\roaming\python\python37\site-packages

Import forked module in Python instead of installed module

I would like to make changes (and possibly contribute if its any good) to a public project on GitHub. I've forked and cloned the module but Im unclear how to get my program to import the local library instead of the 'official' installed module.
I tried cloning it into my project folder but when I imported it and tried to use it things got weird calmap\calmap.plot()
I also tried doing sys.path.append and the folder location. But it seems to still import the official one instead of the forked.
I'm assuming that I could put my program inside the module folder so that module would be found first but I can't image thats the 'correct' way to do it.
|
|-->My_Project_Folder/
|
|-->Forked_Module/
|-->docs/
|-->Forked_Module/
|-->__init__.py

If you're already using anaconda, then you can create a new environment just for the development of this feature.
First, create a new environment:
# develop_lib is the name of the environment.
# You can pick anything that is memorable instead.
# You can also use whatever python version you require ...
conda create -n develop_lib python3.5
Once you have the environment, then you probably want to enter that environment in your current session:
source activate develop_lib
Ok, now that you have the environment set up, you'll probably need to install some requirements for whatever third party library you're developing. I don't know what those dependencies are, but you can install them in your environment using conda install (if they're available) or using pip. Now you're ready to start working with the library that you want to update. python setup.py develop should be available assuming that the package has a standard build process. After you've run that, things should be good to go. You can make changes, run tests, etc.

If you use sys.path.append() the new "path" will be used if none of the previous contains the module you are importing. If you want that the "added path" has precedence over all the older, you have to use
sys.path.insert(0, "path")
In this way, if you print the sys.path you will see that the added path is at the beginning of the list and the module you are importing will be loaded from the path you have specified.

to import from the forked repo instead of python package you should
make a virtual environment for the cloned project then activate it, that way the environment is isolated from the globally installed packages.
1- you need to fork your repo;
2- create a virtual env and activate it;
3- clone your repo.
now if you print your import you will see the path of the forked repo.
import any_module
print(any_module)

Access a Python Package from local git repository

I have a local git repository on my machine, let's say under /develop/myPackage.
I'm currently developing it as a python package (a Django app) and I would like to access it from my local virtualenv. I've tried to include its path in my PYTHONPATH (I'm on a Mac)
export PATH="$PATH:/develop/myPackage"
The directory already contains a __init__.py within its root and within each subdirectory.
No matter what I do but I can't get it work, python won't see my package.
The alternatives are:
Push my local change to github and install the package within my virtualenv from there with pip
Activate my virtualenv and install the package manually with python setup.py install
Since I often need to make changes to my code the last two solution would require too much work all the time even for a small change.
Am I doing something wrong? Would you suggest a better solution?

Install it in editable mode from your local path:
pip install -e /develop/MyPackage
This actually symlinks the package within your virtualenv so you can keep on devving and testing.

The example you show above uses PATH, and not PYTHONPATH. Generally, the search path used by python is partially predicated on the PYTHONPATH environment variable (PATH has little use for this case.)
Try this:
export PYTHONPATH=$PYTHONPATH:/develop/myPackage
Though in reality, you likely want it to be pointing to the directory that contains your package (so you can do 'import myPackage', rather than importing things within the package. That being said, you likely want:
export PYTHONPATH=$PYTHONPATH:/develop/
Reference the python docs here for more information about Python's module/package search path: http://docs.python.org/2/tutorial/modules.html#the-module-search-path
By default, Python uses the packages that it was installed with as it's default path, and as a result PYTHONPATH is unset in the environment.

Python: using downloaded modules

I am new to Python and mostly used my own code. But so now I downloaded a package that I need for some problem I have.
Example structure:
root\
externals\
__init__.py
cowfactory\
__init__.py
cow.py
milk.py
kittens.py
Now the cowfactory's __init__.py does from cowfactory import cow. This gives an import error.
I could fix it and change the import statement to from externals.cowfactory import cow but something tells me that there is an easier way since it's not very practical.
An other fix could be to put the cowfactory package in the root of my project but that's not very tidy either.
I think I have to do something with the __init__.py file in the externals directory but I am not sure what.

Inside the cowfactory package, relative imports should be used such as from . import cow. The __init__.py file in externals is not necessary. Assuming that your project lies in root\ and cowfactory is the external package you downloaded, you can do it in two different ways:
Install the external module
External Python packages usually come with a file "setup.py" that allows you to install it. On Windows, it would be the command "setup.py bdist_wininst" and you get a EXE installer in the "dist" directory (if it builds correctly). Use that installer and the package will be installed in the Python installation directory. Afterwards, you can simply do an import cowfactory just like you would do import os.
If you have pip or easy_install installed: Many external packages can be installed with them (pip even allows easy uninstallation).
Use PYTHONPATH for development
If you want to keep all dependencies together in your project directory, then keep all external packages in the externals\ folder and add the folder to the PYTHONPATH. If you're using the command line, you can create a batch file containing something like
set PYTHONPATH=%PYTHONPATH%:externals
yourprogram.py
I'm actually doing something similar, but using PyDev+Eclipse. There, you can change the "Run configurations" to include the environment variable PYTHONPATH with the value "externals". After the environment variable is set, you can simply import cowfactory in your own modules. Note how that is better than from external import cowfactory because in the latter case, it wouldn't work anymore once you install your project (or you'd have to install all external dependencies as a package called "external" which is a bad idea).
Same solutions of course apply to Linux, as well, but with different commands.

generally, you would use easy_install our pip to install it for you in the appropriate directory. There is a site-packages directory on windows where you can put the package if you can't use easy_install for some reason. On ubuntu, it's /usr/lib/pythonX.Y/dist-packages. Google for your particular system. Or you can put it anywhere on your PYTHONPATH environment variable.
As a general rule, it's good to not put third party libs in your programs directory structure (although there are differing opinions on this vis a vis source control). This keeps your directory structure as minimalist as possible.

The easiest way is to use the enviroment variable $PYTHONPATH. You set it before running your scripts as follows:
export $PYTHONPATH=$PYTHONPATH:/root/externals/
You can add as many folders as you want (provided their separate by :) and python will look in all those folders when importing.