I have a PyQt application which needs to show some images. I saw in:
How can I include package_data without a MANIFEST.in file?
A way to include the image files in the package installation.
It works fine in the aspect that /usr/local/lib/python2.7/dist-packages/mypacakge/images/ now contains all the files. Unfortunately images are still not being shown by the app.
My current code addresses the images files as relative paths (i.e: images/foo.jpeg) which I guess is wrong. But I have no idea what is the right way to address them (unless I want to use the absolute /usr/local/lib/python2.7/dist-packages/mypacakge/images/foo.jpeg which I believe I don't). This seems like a common enough issue, yet I can't find any resources/answers on how to do that.
Module pkg_resources has support to locate a file contained in a package:
import pkg_resources
path = pkg_resources.resource_filename('<package name>', 'images/foo.jpeg')
Related
In my project, no matter where I am or where the file I'm trying to import from is in, I have to specify the path, which is fine.
Ex: import protos.example as example, even when I'm already in the protos directory. So when the generated files are made, they naturally just say import example_pb2 as example__pb2, but with the way it is forcing me to put the full path, that natural way of generating doesn't work because it has to be import protos.generated.example_pb2.
How can I change this to where it automatically searches the current directory before needing to specify the location?
Found the fix. I'm using Pycharm , not sure how this works in other IDEs. Right click the folder that has the generated files and mark it as sources. That solves the generated files not being able to see each other. Then in any file where you reference the generated files, put
import sys
sys.path.append(r'generated')
and then your imports should work as expected
I am creating a tool I plan to put on Github. It is very general however I want to also include some examples of its usage. As such I have set up the following folder structure.
This tool is for python 3.
repository/
commonTool.py
commonTool2.py
specificUsage/
runTheSpecificUsage.py
helpRunTheSpecificUsage.py
Now both of the scripts in the specificUsage/ folder will import the methods in commonTool.py and commonTool2.py.
My issue is ideally the user would be able to go
python repository/specificUsage/runTheSpecificUsage.py
However I cant get this to work. It never is able to import the functions that are in the folder above it. I have tried a variety of various posts on how to import a file from a super folder with no luck.
How should I be setting up these folders? Should I have one init.py or two? Where?
You can create a setup.py at the top level along with commonTool.py and commonTool2.py. Inside your setup.py, put the following line:
import setuptools
setuptools.setup(packages=setuptools.find_packages())
Then put an __init__.py in the same level, as well as in specificUsage/. You should be able to import from specificUsage like this:
import commonTool
After setting up your files, from the top level run:
pip install -e .
You may also want to consider that Python has strong naming conventions for modules and packages and frowns upon camelcase in general. You can read more here if you like.
Imagine some hypothetical python code:
from turboencabulator import panametricfan
If i try to run this on my computer (Windows, 64-bit, Windows 10), it fails to find the turboencabulator module:
What is it looking for?
When Python is attempting to load a module, what is it looking for? Is it a .py file? Is it a .dll (since this is Windows). Is it a folder?
Windows has modules; they're called assemblies
Windows has the notion of assemblies. If you declare you want to use an assembly called turboencabulator, it will look for an assembly manifest:
turboencabulator.manifest
You can also create a folder called turboencabulator, where it will look for an assemebly manifest there:
turboencabulator
|--- turboencabulator.manifest
That manifest will then in turn declare all the files that make up that assembly.
For example, if i need to use the Microsoft Visual C Runtime 8.0, i would declare a dependency on the module
Microsoft.VC80.CRT
Then Windows will go hunting for:
Microsoft.VC80.CRT.manifest file in the folder
Microsoft.VC80.CRT folder
which then must contain Microsoft.VC80.CRT.manifest
Some other places (like WinSxS, maybe the path, who knows)
And then the manifest is simply a text file that lets you know that msvcrt is comprised of 3 dlls:
<file name="msvcr80.dll" />
<file name="msvcp80.dll" />
<file name="msvcm80.dll" />
What is python looking for?
When i specify i want to import a module:
from RetroEncabulator import GirdleSpring
what is python looking for?
a binary?
a zip?
is it a named item that has been registered in the registry?
the name of a module manifest file that declares all the files that make up the module?
What, exactly, is python looking for?
Bonus Questions
where does it look
can it be configured to look places
can modules be registered somewhere, so it doesn't have to go hunting? (as correctly written Windows applications are supposed to do)
Bonus Chatter
in reality i'm from numpy
in reality i'm from cntk
And neither of these "things" are "found".
Bonus Reading
Python - Modules
Additional Investigation
To contrast the heresay so far, i decided to actually spy on python to see what it goes and looks for:
HKEY_CURRENT_USER/Software/Python/PythonCore/2.7/Modules/turboencabulator
HKEY_LOCAL_MACHINE/Software/Python/PythonCore/2.7/Modules/turboencabulator
D:\DevelopPy\Bunny\turboencabulator
D:\DevelopPy\Bunny\turboencabulator.pyd
D:\DevelopPy\Bunny\turboencabulator.py
D:\DevelopPy\Bunny\turboencabulator.pyw
D:\DevelopPy\Bunny\turboencabulator.pyc
D:\Programs\Python27\DLLs\turboencabulator
D:\Programs\Python27\Lib\turboencabulator
D:\Programs\Python27\Lib\site-packages\turboencabulator
I see it looking in the registry. Does that mean it can be registered?
I see it looking for files besides py (pyc, pyw, pyd). Are those compiled?
I see it looking in the Python\DLLs folder. Does that means that there can be DLLs?
I see it looking in Lib? Are those for intermediate object binaries?
Man, it really has been four years without being able to figure it out. Sure would be super if the SO community was helpful. Well-earned reputation.
When you include import statement in your code, python tries to find the python file or package directory with __init__.py file in it the following order
current working directory
local user site packages
common site packages
You can see the path by the following code
import sys
print(sys.path)
You can also add your own path.
Note: You must add this line before importing other modules
import sys
sys.path.append("your/complete/path/to/directory")
I used below solution for importing dependencies.
I found this solution works if I run the code in Pycharm but not in Terminal.
The error message in Terminal is "cannot find graphics.primitive".
I'm using Mac and Python 3.5.
Why I see different behaviors from the Terminal and Pycharm?
How may I make the solution work for both?
http://chimera.labs.oreilly.com/books/1230000000393/ch10.html#_solution_169
Making a Hierarchical Package of Modules
Problem
You want to organize your code into a package consisting of a hierarchical collection of modules.
Solution
Making a package structure is simple. Just organize your code as you wish on the file-system and make sure that every directory defines an init.py file. For example:
graphics/
__init__.py
primitive/
__init__.py
line.py
fill.py
text.py
formats/
__init__.py
png.py
jpg.py
Once you have done this, you should be able to perform various import statements, such as the following:
import graphics.primitive.line
from graphics.primitive import line
import graphics.formats.jpg as jpg
You need to make sure that the graphics package is in the Python search path. PyCharm does this by extending sys.path as follows:
import sys
sys.path.extend(['/Users/hackworth/Development/graphics_parent_dir', '/Applications/PyCharm.app/Contents/helpers/pycharm', '/Applications/PyCharm.app/Contents/helpers/pydev'])
You can do the same in your code replacing /Users/hackworth/graphics_parent_dir with the appropriate path, or you can include the full path to graphics_parent_dir in the PYTHONPATH environment variable. See the Python documentation for details.
Another option would be to place the graphics package into a location the is searched by default on your system.
Let's say I have vtk module in my Python site packages, and from application with own Python distribution I want to access this module.
I tried couple of things like:
import sys
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
import vtk
lut = vtk.vtkLookupTable()
but it fails to load module properly:
AttributeError: 'module' object has no attribute 'vtkLookupTable'
If I do same from default Python interpreter all is fine.
Now I thought to make a wrapper of vtk in this application site packages, with simple __init__.py resolving paths, so that when I do import vtk it will hopefully load right thing, but I have no experience with Python packages to try to make this work
To put it simple, how can I wrap module from arbitrary folder, in Python site packages by making folder with same name as referenced package and simple __init__.py file?
Remove these lines:
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
The site-packages will already be on your python path. Adding a package/folder within that python path (especially at the first level), will just mess with your imports. How is this vtk package structured?
/path/to/site-packages/
vtk/
__init__.py
vtk.py
In this case, to access a function within vtk:
from vtk import vtk
lut = vtk.vtkLookupTable()
It all comes down to how the folder is arranged. You could also do this:
import vtk
lut = vtk.vtk.vtkLookupTable()
Do not try to hack python importing by creating proxy modules simply because you're not understanding how python importing is working. The error was quite clear. The attribute vtkLookupTable did not exist on whatever it was you imported. You imported the wrong thing. Fix it.
You should very very very very rarely have to manipulate the sys.path manually. When you do have to, you should know that it's the right reason - not to work around something you're not fully understanding.
I had trouble with python paths when I first started with python. It can be frustrating, but coming to understand how it works is necessary. What can help you is something like the following:
import vtk
print dir(vtk)
That will print the attributes of vtk, so you can explore exactly what is in the package or module in cases like this where you think you're importing the right thing.
After re-reading your question, it seems like this is a different python install you're talking about. The answer is to install this package into the other python install, or include this package as a top level import by copying the folder into the root level of your application.
"C:\Python27\Lib\site-packages" is already on your python path. So appending path is unnecessary. Remove:
import sys
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
Create a new folder called 'vtk\' in "C:\Python27\Lib\site-packages", then create a new python file named __init__.py in "C:\Python27\Lib\site-packages\vtk" and put your own module vtk.py in this directory.
Using:
import vtk
or
from vtk import vtk
to use your own module.