I'm self-taught in the Python world, so some of the structural conventions are still a little hazy to me. However, I've been getting very close to what I want to accomplish, but just ran into a larger problem.
Basically, I have a directory structure like this, which will sit outside of the normal python installation (this is to be distributed to people who should not have to know what a python installation is, but will have the one that comes standard with ArcGIS):
top_directory/
ArcToolbox.tbx
scripts/
ArcGIStool.py (script for the tool in the .tbx)
pythonmod/
__init__.py
general.py
xlrd/ (copied from my own python installation)
xlwt/ (copied from my own python installation)
xlutils/ (copied from my own python installation)
So, I like this directory structure, because all of the ArcGIStool.py scripts call functions within the pythonmod package (like those within general.py), and all of the general.py functions can call xlrd and xlwt functions with simple "import xlrd" statements. This means that if the user desired, he/she could just move the pythonmod folder to the python site-packages folder, and everything would run fine, even if xlrd/xlwt/xlutils are already installed.
THE PROBLEM:
Everything is great, until I try to use xlutils in general.py. Specifically, I need to "from xlutils.copy import copy". However, this sets off a cascade of import errors. One is that xlutils/copy.py uses "from xlutils.filter import process,XLRDReader,XLWTWriter". I solved this by modifying xlutils/copy.py like this:
try:
from xlutils.filter import process,XLRDReader,XLWTWriter
except ImportError:
from filter import process,XLRDReader,XLWTWriter
I thought this would work fine for other situations, but there are modules in the xlutils package that need to import xlrd. I tried following this advice, but when I use
try:
import xlrd
except ImportError:
import os, sys, imp
path = os.path.dirname(os.path.dirname(sys.argv[0]))
xlrd = imp.load_source("pythonmod.xlrd",os.path.join(path,"xlrd","__init__.py"))
I get a new import error: In xlrd/init.py, the info module is called (from xlrd/info.py), BUT when I use the above code, I get an error saying that the name "info" is not defined.
This leads me to believe that I don't really know what is going on, because I thought that when the init.py file was imported it would run just like normal and look within its containing folder for info.py. This does not seem to be the case, unfortunately.
Thanks for your interest, and any help would be greatly appreciated.
p.s. I don't want to have to modify the path variables, as I have no idea who will be using this toolset, and permissions are likely to be an issue, etc.
I realized I was using imp.load_source incorrectly. The correct syntax for what I wanted to do should have been:
imp.load_source("xlrd",os.path.join(path,"xlrd","__init__.py"))
In the end though, I ended up rewriting my code to not need xlutils at all, because I continued to have import errors that were causing many more problems than were worth dealing with.
Related
I am struggling a bit to get a package to work, so I would like to just get the simplest possible test case to work.
Here is my current attempt.
Inside of a folder called Python_experiment I have two files: a jupyter notebook with the code
from .pleasejustwork import eat_muffin
and a file called pleasejustwork.py with the code
def eat_muffin():
print('i ate a muffin')
When I run the line from the jupyter notebook I get the error "attempted relative import with no known parent package". What am I missing?
The syntax you’re using is for a python package. To achieve this, place a file, __init__.py in the directory. It can be empty. If you’d like to use that package from anywhere, write a setup.py script to build and install your package. There are a lot of good tutorials on how do that, like in the python docs. Then you could do
from python_experiment.pleasejustwork import eatmuffin or from another file in the package from .pleasejustwork import eatmuffin
If you’re just trying to learn about modules, you can simply take out that leading period
from pleasejustwork import eatmuffin
I recently was asked to deliver a python project as part of an interview process.
I tested my project on Windows and MacOSX, with Pycharm, Spyder, jupyter notebook and command line and everything works fine.
However, the reviewer was unable to make the project work on his side, because of module import issues according to him.
My modules are organized like this:
my_project/
my_module.py
main_module.py
my_package/
__init__.py
my_submodule_1.py
my_submodule_2.py
my_submodule_1.py:
import my_module
import my_submodule_2
I haven't added any path related to this project in PYTHONPATH.
The project main function is located in main_module.py.
The reviewer seem to have problems with the modules imported in my_submodule_1.py.
Could anyone shed some light on possible mistakes here and why it would work on my side and not on his?
Your my_submodule_1 module is doing an implicit relative import when it imports my_submodule_2 directly.
That's not legal in Python 3. It is allowed in Python 2, though its usually a bad idea to use it. You can get the Python 3 semantics by putting from __future__ import absolute_import above the other import statements in your file. To fix the import, you'd want to change import my_submodule_2 to either import my_package.my_submodule_2 (an absolute import) or from . import my_submodule2 (an explicit relative import).
If your interviewer is using Python 3 and you're using Python 2, there are likely to be other issues with your code (especially if you're doing any sort of text processing), so I'd make sure you're testing your code in the version they expect!
I think since my_module.py is not in same directory as my_submodule1.py ,and on the reviewer pc the sys.path doesn't have that location of my_module.py, that's why it getting problem in importing the module from its parent directory.
if u give the details of error that the reviewer is getting it might help finding the right solution.
I'm following a tutorial to call python code from a C++ program from the python docs.
Everything works just fine when trying to call the multiply example. Now if I add a line to the python source code importing a library, lets say openpyxl,
from openpyxl import load_workbook
I receive an error from python
ImportError: No module named openpyxl
I thought if I import a system library, I wouldn't have any problems, but I also get an error if I try to import datetime.
I don't have any error if I import the file from the python console. The openpyxl library is installed in my system.
So my question is: how to import python source code that needs to import packages?
EDIT: Ok, I forgot to mention something, I have not been completely honest with you guys, I'm sorry.
Trying to run the example I run into a problem: I couldn't make python found my multiply.py file, and the line PyImport_Import always return null.
My solution was to add the path in which I knew my python source was by using PySys_SetPath. The problem is that I just realized that this function doesn't append a new directory, it just overwrites the PYTHONPATH. So now python can find multiply.py, but absolutly anything else.
Of course I've deleted that line but now I have another question, why does python can't find my source if the file is just in the same directory of the C++ compiled program?
The I realized that my sys.path from my python console was a little different from the path showed in my embedded python: the first one had at the beginning of the list an empty string ''. I'm not a python expert, but when I add that line to my path I could import the multiply.py so it seems that was the reason I couldn't import modules that were located to relative to my executable was the missing of this empty path -but still don't know what it means-.
I have to thank to #paul-evans who give me the idea of adding the path to find my files.
This is what PYTHONPATH is for. You can set it as an environment variable containing a list module directories, or in the code itself something like:
import sys
sys.path.append("path/to/openpyxl/module")
First of all let me tell you that I'm a new user and I'm just starting to learn Python in College so my apologies if this question is answered in other topic, but I searched and I can't seem to find it.
I received a file work.pyc from my teacher and he says I have to import it in my Wing IDE using the command from work import *, the question is I don't know where to put the file to import it.
It just says ImportError: No module named work.
Thank you
There are several options for this.
The most straightforward is to just place it in the same folder as the py file that wants to import it.
You may also want to have a look at this
if you're using the python interpreter (the one that lets you directly input python code into it and executes) you'll have to do this:
sys.path.append('newpath')
from work import *
where newpath is the path on your filesystem containing your work.pyc file
If you're working on a script called main.py in the folder project, one option is to place it at project/work.pyc
This will make the module importable because it's in the same working directory as your code.
The way Python resolves import statements works like this (simplified):
The Python interpreter you're using (/usr/bin/python2.6 for example, there can be several on your system) has a list of search paths where it looks for importable code. This list is in sys.path and you can look at it by firing up your interpreter and printing it out like this:
>>> import sys
>>> from pprint import pprint
>>> pprint(sys.path)
sys.path usually contains the path to modules from the standard library, additional installed packages (usually in site-packages) and possibly other 3rd party modules.
When you do something like import foo, Python will first look if there is a module called foo.py in the directory your script lives. If not, it will search sys.path and try to import it from there.
As I said, this explanation is a bit simplified. The details are explained in the section about the module search path.
Note 1:
The *.pyc you got handed is compiled Python bytecode. That means it's contents are binary, it contains instructions to be executed by a Python virtual machine as opposed to source code in *.py that you will normally deal with.
Note 2:
The advice your teacher gave you to do from work import * is rather bad advice. It might be ok to do this for testing purposes in the interactive interpreter, but your should never do that in actual code. Instead you should do something like from work import chop, hack
Main reasons:
Namespace pollution. You're likely to import things you don't need but still pollute your global namespace.
Readability. If you ever read someone elses code and wonder where foo came from, just scroll up and look at the imports, and you'll see exactly where it's being imported from. If that person used import *, you can't do that.
Let's say I have vtk module in my Python site packages, and from application with own Python distribution I want to access this module.
I tried couple of things like:
import sys
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
import vtk
lut = vtk.vtkLookupTable()
but it fails to load module properly:
AttributeError: 'module' object has no attribute 'vtkLookupTable'
If I do same from default Python interpreter all is fine.
Now I thought to make a wrapper of vtk in this application site packages, with simple __init__.py resolving paths, so that when I do import vtk it will hopefully load right thing, but I have no experience with Python packages to try to make this work
To put it simple, how can I wrap module from arbitrary folder, in Python site packages by making folder with same name as referenced package and simple __init__.py file?
Remove these lines:
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
The site-packages will already be on your python path. Adding a package/folder within that python path (especially at the first level), will just mess with your imports. How is this vtk package structured?
/path/to/site-packages/
vtk/
__init__.py
vtk.py
In this case, to access a function within vtk:
from vtk import vtk
lut = vtk.vtkLookupTable()
It all comes down to how the folder is arranged. You could also do this:
import vtk
lut = vtk.vtk.vtkLookupTable()
Do not try to hack python importing by creating proxy modules simply because you're not understanding how python importing is working. The error was quite clear. The attribute vtkLookupTable did not exist on whatever it was you imported. You imported the wrong thing. Fix it.
You should very very very very rarely have to manipulate the sys.path manually. When you do have to, you should know that it's the right reason - not to work around something you're not fully understanding.
I had trouble with python paths when I first started with python. It can be frustrating, but coming to understand how it works is necessary. What can help you is something like the following:
import vtk
print dir(vtk)
That will print the attributes of vtk, so you can explore exactly what is in the package or module in cases like this where you think you're importing the right thing.
After re-reading your question, it seems like this is a different python install you're talking about. The answer is to install this package into the other python install, or include this package as a top level import by copying the folder into the root level of your application.
"C:\Python27\Lib\site-packages" is already on your python path. So appending path is unnecessary. Remove:
import sys
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
Create a new folder called 'vtk\' in "C:\Python27\Lib\site-packages", then create a new python file named __init__.py in "C:\Python27\Lib\site-packages\vtk" and put your own module vtk.py in this directory.
Using:
import vtk
or
from vtk import vtk
to use your own module.