I am new to Python and mostly used my own code. But so now I downloaded a package that I need for some problem I have.
Example structure:
root\
externals\
__init__.py
cowfactory\
__init__.py
cow.py
milk.py
kittens.py
Now the cowfactory's __init__.py does from cowfactory import cow. This gives an import error.
I could fix it and change the import statement to from externals.cowfactory import cow but something tells me that there is an easier way since it's not very practical.
An other fix could be to put the cowfactory package in the root of my project but that's not very tidy either.
I think I have to do something with the __init__.py file in the externals directory but I am not sure what.
Inside the cowfactory package, relative imports should be used such as from . import cow. The __init__.py file in externals is not necessary. Assuming that your project lies in root\ and cowfactory is the external package you downloaded, you can do it in two different ways:
Install the external module
External Python packages usually come with a file "setup.py" that allows you to install it. On Windows, it would be the command "setup.py bdist_wininst" and you get a EXE installer in the "dist" directory (if it builds correctly). Use that installer and the package will be installed in the Python installation directory. Afterwards, you can simply do an import cowfactory just like you would do import os.
If you have pip or easy_install installed: Many external packages can be installed with them (pip even allows easy uninstallation).
Use PYTHONPATH for development
If you want to keep all dependencies together in your project directory, then keep all external packages in the externals\ folder and add the folder to the PYTHONPATH. If you're using the command line, you can create a batch file containing something like
set PYTHONPATH=%PYTHONPATH%:externals
yourprogram.py
I'm actually doing something similar, but using PyDev+Eclipse. There, you can change the "Run configurations" to include the environment variable PYTHONPATH with the value "externals". After the environment variable is set, you can simply import cowfactory in your own modules. Note how that is better than from external import cowfactory because in the latter case, it wouldn't work anymore once you install your project (or you'd have to install all external dependencies as a package called "external" which is a bad idea).
Same solutions of course apply to Linux, as well, but with different commands.
generally, you would use easy_install our pip to install it for you in the appropriate directory. There is a site-packages directory on windows where you can put the package if you can't use easy_install for some reason. On ubuntu, it's /usr/lib/pythonX.Y/dist-packages. Google for your particular system. Or you can put it anywhere on your PYTHONPATH environment variable.
As a general rule, it's good to not put third party libs in your programs directory structure (although there are differing opinions on this vis a vis source control). This keeps your directory structure as minimalist as possible.
The easiest way is to use the enviroment variable $PYTHONPATH. You set it before running your scripts as follows:
export $PYTHONPATH=$PYTHONPATH:/root/externals/
You can add as many folders as you want (provided their separate by :) and python will look in all those folders when importing.
Related
I want to install python, and then some additional modules, to my computer (windows).
I've been told that the best way to install python and packages is to make a folder in your drive somewhere BEFORE installing anything, (ex. 'Python') then pointing all your downloads (using cd in the command line) to this folder so that everything is in once place, and you don't have to go on a wild goose chase to make sure everything you want access to is in your PATH when you go to import modules etc.
Do I have the right idea?
I have had trouble importing modules in the past because they were not in the Path.
Will changing the directory in the command line before typing:
pip install somemodule
cause that module to be saved to where I just changed the directory to?
pip always installs the libraries in a fixed directory, usually in the user folder. You can check this by the command pip show <installed-package-name>
So you can use any package you already installed to get the pip directory. The location might vary based on your python version and env name.
Example: c:\users\<user>\appdata\roaming\python\python37\site-packages
There is a popular Python question called Importing files from different folder.
But the top answer there mentions adding stuff to "some_file.py", and that will obviously only apply to imports inside that file.
What if I want to specify an additional dir to import from, project-wide?
I don't want to modify PYTHONPATH, as I believe a per-project solution is cleaner.
I don't want to use python packages for this, because I feel they'll probably just complicate stuff. E.g. maybe I'll need to manually recompile .py files into .pyc files every time I make a change to the code in the other folder.
What if I want to specify an additional dir to import from, project-wide?
Solution 1: Package installed in develop mode
Create a regular package with setup.py and install it with -e option:
python -m pip install -e /path/to/dir_with_setup_py/
-e, --editable Install a project in editable mode (i.e. setuptools "develop mode") from a local project
path or a VCS url.
Now, as soon as you update your code, the new version will be used at import
without reinstalling anything.
Solution 2: Dynamically modify sys.path
You can add as many directories dynamically to the search path for Python packages as you want. Make this the very first lines of code you execute:
import sys
sys.path.append('my/path/to/my/file1')
sys.path.append('my/path/to/my/file2')
or to make the first to be found:
sys.path.insert(0, 'my/path/to/my/file1')
sys.path.insert(0, 'my/path/to/my/file2')
Now the files:
my/path/to/my/file1/myscript1.py
my/path/to/my/file2/myscript2.py
can be imported anywhere in your project:
import myscript1
import myscript2
No need to modify sys.path again as long as this Python process is running.
now I have a folder named my_funcs which have __init__.py and some .py files containing some functions and classes I wrote that I want to use for several projects.
So I want to know the best practice for these projects to direct import from this folder.
one solution is to sys.path.append('.../my_funcs'), in this case I will have to put this in front of the import statement for every .py file.
Any suggestions? BTW, I'm on Windows
Best is to use PYTHONPATH. Set it to the path where your common modules are found, before running Python. Then you can just do import my_funcs for example.
Checkout PEP370, "Per user site-packages directory".
You set PYTHONUSERBASE to somewhere under your control, and you can see
$ PYTHONUSERBASE=/home/me/.local
$ python -m site --user-site
/home/me/.local/lib/python2.7/site-packages
Your personal directory now appears in sys.path. easy_install respects this (using the --user option) so you can install "real" packages there, also, but make them available only to you.
Depending on your configuration, sounds like you could move your my_funcs directory under the site-packages directory and you're done!
I am a newbie to Python and I have been going through the exercises in LPTHW:
http://learnpythonthehardway.org/book/ex46.html
The explanation of packaging the author gave was very vague, so I did some researching, but I was still left with some questions in the end.
Firstly, what happens if you have a module that imports another module? Do you put the module that is imported into a subdirectory of the directory containing the first module, or in another separate directory in the root?
Secondly, once you have installed the entire package how do you run any scripts that are in the bin folder?
Firstly, what happens if you have a module that imports another
module? Do you put the module that is imported into a subdirectory of
the directory containing the first module, or in another separate
directory in the root?
A module is just a file with Python code - you don't need to put it in a separate directory. The easiest option is to put the file in the same directory that contains the module that will be doing the import.
If you put it in some other directory and that directory is not a package you will not be able to import it at all (unless you manually mess around with PYTHONPATH which you shouldn't).
Secondly, once you have installed the entire package how do you run
any scripts that are in the bin folder?
The way to do this is to specify the scripts for your application in the setup.py file using console_scripts. This way, when your package is installed, your scripts will be built correctly and placed in the right directories on the target platform so that you can directly execute them (without being aware of their location on the filesystem).
As an example, you can browse the setup.py file for the django project.
If you are installing a package, you don't need to know where the actual script is located - simply install the package and execute the command directly from your prompt.
I am trying to import ZipCodeDatabase in helloworld.py.
helloworld.py exists at /google-app-engine/helloworld
ZipCodeDatabase module exists /usr/local/lib/python/python2.7/dist-packages
PYTHONPATH = /usr/local/lib/python/python2.7/dist-packages;/usr/local/lib/python/
When compiling helloworld I am still getting "ZipCodeDatabase module not found". Why isn't it being picked from the PYTHONPATH?
I highly doubt you've got a module called ZipCodeDatabase. That naming convention is typically reserved for a class that resides within a module. Modules are usually lowercase or lower_snake_case, to represent the file containing the module. I'm assuming you've installed pyzipcode here, but it may be a different module.
# assuming pyzipcode.py in the dist-packages directory
$ python -c 'from pyzipcode import ZipCodeDatabase'
If I'm wrong above, then are you sure you're running the version of python that has the ZipCodeDatabase module installed?
Some troubleshooting steps:
$ which python
$ python --version
$ python -c 'import ZipCodeDatabase'
$ ls -l /usr/local/lib/python2.7/dist-packages/ | grep -i zip
Also, is it really necessary for you to specify the PYTHONPATH line? Typically, the site-packages folder (and by extension I assume the dist-packages folder on Ubuntu) is included in the default PYTHONPATH, along with the current directory of the python module you're using.
How did you install the ZipCodeDatabase? Did you just drop the file in there? Try putting it alongside your helloworld.py file and try importing it then. Also, a full stack trace is useful information here, especially when others are trying to diagnose the problem you're having.
Edit:
Ok, now that I know you're using google app engine (should have been obvious from your use of paths - I'm sorry), it looks like it doesn't use the site-packages or dist-packages to load modules. You should create a sub-directory in your project with the relevant third party libraries, and add that sub-directory to your path. Disclaimer: I've never used GAE so I might be missing the mark with this.
Check out this answer for how to structure your project and add the extra directory to your path from within the application.