tl;dr - My python package requires import package.package instead of working with just import package. How do I get it to work with the latter?
I am trying to set up my first python package and am running into some issues with the import part of the process.
My python package file setup looks like this on my computer:
my-package
- build
- dist
- package
- package.egg-info
- LICENSE
- README.md
- Setup.py
Inside package is the following:
__init__.py
package.py
__init__.py reads name = 'package', and package.py contains all of the content of the package.
EDIT: I have attempted using various different versions of __init__.py, including adding import package, import package.package, or import package.package as package below the line name = 'package', but all of resulted in the same issue.
Using the Packaging Python Projects tutorial, I've been able to get my package upload to TestPyPi, but when I install the package to my computer, none of the functions/methods are available, and when I run "import package" and do help(package), I get the following:
Help on package package:
NAME
package
PACKAGE CONTENTS
package
DATA
name = 'package'
FILE
url/to/package
When I run import package.package and help(package), I can access the methods/functions, and get the expected help text for the package's content.
My question is, how to I configure the package file on my computer in such a way that once it is upload to TestPyPi and then downloaded, import package works, instead of needing to run import package.package?
When you write import package, you can access names in package/__init__.py as package.foo.
So, inside __init__.py, if you import all the necessary functions/variables/etc from package.py, those names will be visible to clients that just import package.
So, if you have this in package/__init__.py:
from .package import (foo, bar, baz)
Then in your other code you can do this:
from package import foo
And you dont have to worry about from package.package import foo.
Related
I have a python package defined via __init__.py in a folder named python_utils that has many python scripts in it.
python_utils/
__init__.py
print_pretty.py
script_1.py
...
For convenience this __init__.py imports all of them using:
from .pretty_print import pretty_print
from .scritp_1 import func1
from .script_2 import func2
....
The problem I have is that some of the python scripts depend on packages that may not be available at run time. So when I try to import a very trivial function from the package (from python_utils import print_pretty), it fails because of the dependencies the other scripts have. I also tried to import directly with from python_utils.print_pretty import print_pretty same problem. I can get rid of the __init__.py to solve the problem, but it saves me a lot of typing when importing packages. Is there a way I can import only pretty_print without getting the dependencies from the other scripts?
My directory is structured like this
>project
>tests
>test_to_json.py
>app.py
>x.db
in my test_to_json.py I want to access a class from app.py
How would I import it? I've tried from ..app import myClass but that gives me ImportError: attempted relative import with no known parent package. Is there something I'm missing?
You cannot use .. relative pathing in python. That specific type of relative python is simply not allowed. . is allowed, though. The resolution for this problem is usually done by converting your project into a python package.
Extensive tutorials for doing so can be found here, but I will give an example of how to convert your project into a package.
Step 1
The new file structure should look like this:
>project
>tests
>__init__.py #Note this file
>test_to_json.py
>__init__.py #Note this file
>setup.py #Note this file
>app.py
>x.db
Step 2
Write your setup.py.
Here is an generic setup.py that should work for your project:
from setuptools import setup, find_packages
setup(
name='project_package', #the name of your desired import
version='0.0.1',
author='An Awesome Coder',
packages=find_packages(),
description='An awesome package that does something',
install_requires=[], # a list of python dependencies for your package
)
find_packages() looks for all the __init__.py files in your package to identify submodules.
Step 3
Install your package. In the folder with your new setup.py, run pip install -e . This will install your package on your computer, basically adding the files to your python system path.
Step 4
From any python terminal on your computer you should now be able to import your package using the package_name you specified.
import project_package
from project_package.app import myClass
myClass()
.
.
.
I'm using the following python project structure in order to allow packaging (based on this):
In each package I import all the submodules using
__init__.py
When i'm inside config.py and trying to
import settings
it does not identify settings as a package of modules..
How should I refer to that so it will recognise it?
So the problem was that because everything was inside as_manager package, I should have import as_manager.settings ...
I have created a Python package with the following directory structure:
/
LICENSE
MANIFEST.IN
README.rst
VERSION
docs/
multitool/
__init__.py
core/
__init__.py
classes.py
utils.py
libs/
multitool.py
tests/
tools/
__init__.py
hashtool.py
webtool.py
setup.py
The goal is to create a command line application (multitool.py) that 3rd parties can add to by adding their own files to the tools directory. This is accomplished by having them subclass a class that I've created. For example, these are the first few lines of hashtool.py:
import multitool
class HashTool(multitool.core.classes.CLITool):
All of this works as long as I run it from the project directory itself:
$ ./multitool.py -h <---works
$ ./multitool/multitool.py -h <---works
The problem comes when I try to create and install it as a package. The install runs and installs the script. However, when you run the script, it cannot find any of the modules in the package:
$ multitool.py
import core
ImportError: No module named core
I've tried changing the import to multitool, multitool.core, .multitool, ..multitool, and others with the same result.
However, I am able to do imports from the Python interpreter:
Type "help", "copyright", "credits" or "license" for more information.
>>> import multitool
>>> import multitool.core
>>> import multitool.core.classes
>>> from multitool import core
>>>
Here is the relevant portion of my setup.py
setup(
name = 'multitool',
version = __version__,
license = 'GPLv2',
packages = find_packages(exclude=['test/']),
scripts = ['multitool/multitool.py'],
include_package_data = True,
....
)
What am I doing wrong? How can I import my own code and the files from the tools directory in the script that I install with the package?
Updated
MrAlias's edited comment below worked. The confusion was that the script was the same name as the package itself and was not in a separate directory. Moving the script to its own bin/ directory solved the problem.
First off, when you install the package you are importing core without identifying it is being apart of the multitool package. So:
import core
should be,
from multitool import core
That way the interpreter knows the module to import core from.
[Edit]
As for the directory structure of the installed package, scripts need to go into a separate directory from the module itself. The way the shown directory structure is Distutils will install the script you named into both a place your system looks for executables as well as in the package itself, which is likely where all the confusion is coming from.
import multitool
class HashTool(multitool.core.classes.CLITool):
Importing a package does not import its subpackages and submodules. Try this:
import multitool.core.classes
class HashTool(multitool.core.classes.CLITool):
For my Python application, I have the following directories structure:
\myapp
\myapp\utils\
\myapp\utils\GChartWrapper\
\myapp\model\
\myapp\view\
\myapp\controller\
One of my class in \myapp\view\ must import a class called GChartWrapper. However, I am getting an import error...
myview.py
from myapp.utils.GChartWrapper import *
Here is the error:
<type 'exceptions.ImportError'>: No module named GChartWrapper.GChart
args = ('No module named GChartWrapper.GChart',)
message = 'No module named GChartWrapper.GChart'
What am I doing wrong? I really have a hard time to import modules/classes in Python...
The __init__.py file of the GChartWrapper package expects the GChartWrapper package on PYTHONPATH. You can tell by the first line:
from GChartWrapper.GChart import *
Is it necessary to have the GChartWrapper included package in your package directory structure?
If so, then one thing you could do is adding the path where the package resides to sys.path at run time. I take it myview.py is in the myapp\view directory? Then you could do this before importing GChartWrapper:
import sys
import os
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..', 'utils')))
If it is not necessary to have it in your directory structure, it could be easier to have it installed at the conventional location. You can do that by running the setup.py script that's included in the GChartWrapper source distribution.
You don't import modules and packages from arbritary paths. Instead, in python you use packages and absolute imports. That'll avoid all future problems.
Example:
create the following files:
MyApp\myapp\__init__.py
MyApp\myapp\utils\__init__.py
MyApp\myapp\utils\charts.py
MyApp\myapp\model\__init__.py
MyApp\myapp\view\__init__.py
MyApp\myapp\controller\__init__.py
MyApp\run.py
MyApp\setup.py
MyApp\README
The files should be empty except for those:
MyApp\myapp\utils\charts.py:
class GChartWrapper(object):
def __init__(self):
print "DEBUG: An instance of GChartWrapper is being created!"
MyApp\myapp\view\__init__.py:
from myapp.utils.charts import GChartWrapper
def start():
c = GChartWrapper() # creating instance of the class
MyApp\run.py:
from myapp.view import start
start()
That's all! When you run your entry point (run.py) it calls a function on the view, and that creates an instance of the GChartWrapper class. Using this structure you can import anything anywhere and use it.
To complement, in MyApp\setup.py you write an installation program for the MyApp\myapp package. Use distutils to write it:
from distutils.core import setup
setup(name='MyApp',
version='1.0',
description='My Beautiful Application',
author='Martin',
author_email='martin#xxxxxxx.com',
url='http://stackoverflow.com/questions/1003843/',
packages=['myapp'],
scripts=['run.py']
)
That is enough. Now when people download the MyApp folder, they can just install it using setup.py and run it using run.py. Distutils can generate packages in a number of formats including windows installable .EXE
It's the standard way of distributing python packages/applications.
You can change the path where python looks for files.
At the top of your source file, add:
import sys
sys.path.append("..")
Or alternatively change the environment variable:
export PYTHONPATH=..
Or starting in python 2.5 (again assuming myview is in myapp\view:
from __future__ import absolute_import
from ..utils.GChartWrapper import *
See: http://docs.python.org/whatsnew/2.5.html#pep-328-absolute-and-relative-imports
GChartWrapper is also available from PyPI so you can use easy_install or pip to install the module:
sudo pip install GChartWrapper==0.9
It will then be automatically added to your PYTHONPATH and then you can remove it from your /myapp/utils directory. If you can't use sudo, look at using virtualenv (and virtualenvwrapper).