I used below solution for importing dependencies.
I found this solution works if I run the code in Pycharm but not in Terminal.
The error message in Terminal is "cannot find graphics.primitive".
I'm using Mac and Python 3.5.
Why I see different behaviors from the Terminal and Pycharm?
How may I make the solution work for both?
http://chimera.labs.oreilly.com/books/1230000000393/ch10.html#_solution_169
Making a Hierarchical Package of Modules
Problem
You want to organize your code into a package consisting of a hierarchical collection of modules.
Solution
Making a package structure is simple. Just organize your code as you wish on the file-system and make sure that every directory defines an init.py file. For example:
graphics/
__init__.py
primitive/
__init__.py
line.py
fill.py
text.py
formats/
__init__.py
png.py
jpg.py
Once you have done this, you should be able to perform various import statements, such as the following:
import graphics.primitive.line
from graphics.primitive import line
import graphics.formats.jpg as jpg
You need to make sure that the graphics package is in the Python search path. PyCharm does this by extending sys.path as follows:
import sys
sys.path.extend(['/Users/hackworth/Development/graphics_parent_dir', '/Applications/PyCharm.app/Contents/helpers/pycharm', '/Applications/PyCharm.app/Contents/helpers/pydev'])
You can do the same in your code replacing /Users/hackworth/graphics_parent_dir with the appropriate path, or you can include the full path to graphics_parent_dir in the PYTHONPATH environment variable. See the Python documentation for details.
Another option would be to place the graphics package into a location the is searched by default on your system.
Related
I'm building a pyd file via pybind11. There's no python-only code in the project (not even in the final wheel). I'm packing this prebuilt pyd into a wheel package. It's all working well so far, but I've got one small issue. Every time I want to use the functions from the pyd file (which is actually the primary "source" of code) I have to type the following: from python_lib_name.pyd_name import pyd_exported_class example: from crusher.crusher import Shard. This looks pretty lame, is there any way to prevent this and let the developer simply use the from crusher import Shard syntax?
Also, I'm not sure what information to provide, obviously, I wouldn't like to throw the whole project at you unnecessarily. I'm glad to provide any details by editing.
how about this: put your pyd in a folder and include a __init__.py that import the stuff from the pyd
my_project
|---my_pyd_lib
| |---my_pyd_lib.pyd
| |---__init__.py
|---#the other stuff
and in that init just put
#__init__.py
from .my_pyd_lib import *
the dot in the import make it a relative import useful to make sure that import the thing in the same folder as that file, it also make sure you import your thing if it happens to share a name with some other package/module you happens to have installed
One way to make a pyd file simply importable is to use the Python interpreter's -m flag. For example, if your file is called foo.pyd and is located in the current directory, you can import it by running the following command:
python -m foo
This will cause the Python interpreter to load the module from foo.pyd and then execute its contents as if they were written in a normal .py file. You can also use this technique to import modules that are located in different directories; just specify the path to the module as an argument to -m.
Another way to make a pyd file simply importable is to convert it into a normal .py file.
I am creating a tool I plan to put on Github. It is very general however I want to also include some examples of its usage. As such I have set up the following folder structure.
This tool is for python 3.
repository/
commonTool.py
commonTool2.py
specificUsage/
runTheSpecificUsage.py
helpRunTheSpecificUsage.py
Now both of the scripts in the specificUsage/ folder will import the methods in commonTool.py and commonTool2.py.
My issue is ideally the user would be able to go
python repository/specificUsage/runTheSpecificUsage.py
However I cant get this to work. It never is able to import the functions that are in the folder above it. I have tried a variety of various posts on how to import a file from a super folder with no luck.
How should I be setting up these folders? Should I have one init.py or two? Where?
You can create a setup.py at the top level along with commonTool.py and commonTool2.py. Inside your setup.py, put the following line:
import setuptools
setuptools.setup(packages=setuptools.find_packages())
Then put an __init__.py in the same level, as well as in specificUsage/. You should be able to import from specificUsage like this:
import commonTool
After setting up your files, from the top level run:
pip install -e .
You may also want to consider that Python has strong naming conventions for modules and packages and frowns upon camelcase in general. You can read more here if you like.
First, I need to describe the environment I'm writing for. I'm writing Python code that will be loaded and executed by a Python runtime running within a CAD application. The CAD application uses Python as its scripting engine. As a result, I don't have access to the Python runtime and as a good citizen to all other scripts shouldn't modify any system settings. My script is just one of many that are loaded and running.
This all works fine except when I want to use non-standard libraries. In that case, I need to install a local copy of the library for my script to access. The problem I'm having is that most libraries expect to be installed and added to the sys path which is something I shouldn't do because it could create conflicts with what other scripts are doing. What I'm attempting to do instead is to set up a local copy of the library(s) and then edit their source so their imports are relative and they don't depend on the sys path. That way my program will have its own local copy of the libraries and not depend on anything else and won't disturb any other scripts.
I'm using the -t option of PIP to install Requests and PyOpenSSL into a "Packages" subfolder in my script folder. Here's an abbreviated list of what I have.
RequestsTest/
RequestsTest.py
Packages/
OpenSSL/
cryptography/
x509/
__init__.py
base.py
hazmat/
__init__.py
backends/
__init__.py
interfaces.py
openssl/
__init__.py
backend.py
x509.py
OpenSSL/
__init__.py
SSL.py
Requests/
chardet/
__init__.py
requests/
__init__.py
urllib3/
__init__.py
request.py
contrib/
__init__.py
pyopenssl.py
util/
__init__.py
request.py
ssl_.py
Although it's tedious to track down the various import statements and make then relative, it does seem to work. However, I'm having problems with one particular set of imports.
In Packages/Requests/urllib3/contrib/pyopenssl.py it contains the following imports, which I've modified:
from ....OpenSSL.OpenSSL import SSL
from ....OpenSSL.cryptography import x509
They were originally:
from OpenSSL import OpenSSL.SSL
from cryptography import x509
I get the error "ImportError: No module named 'OpenSSL'" for the first line and "ImportError: No module named 'cryptography'" for the second line. I'm fairly certain the path is correct because if I change the number of dots I get the no module named error but it lists the full path of the what it's trying to load and not just the name of the module.
I would appreciate some help with this specific issue but can also use some overall advice of how to set up and use private copies of libraries. Remember that my program is just one of many that the system is loading changing the system or setting up a virtual environment is not an option.
Check out the localimport module, which seems to be a solution for your particular use case. From the README:
Given your Python script, application or plugin comes with a directory that contains modules for import, you can use localimport to keep the global importer state clean.
app.py
res/modules/
some_package/
__init__.py
# app.py
with localimport('res/modules') as _importer:
import some_package
assert 'some_package' not in sys.modules
The tagline is "Isolated import of Python Modules for embedded applications." so it seems pretty relevant.
When using that module, the following may help keep things neat:
Put your actual script logic into its own file.
Have a wrapper script (which will be the one loaded by the CAD software) which does localimport as mentioned in the README then does a relative import of your module. If your module gets big enough maybe put it into its own package and consume it in the same way as everything else (just do from RequestsTest import * in the body of the with localimport(): ....
Try to have a clear boundary between the source code you write and the final organized set of files required to use that source code in the context of the CAD Python runtime. It is OK to have a build/packaging step that creates the localimport script, downloads the required packages, etc. It's better even because then it is automated and not something that was done manually that someone in the future may have to recreate.
I developed a python3 package in Pycharm but am running into some confusing behavior when I try to import modules in my test cases. The problem seems to be with the directory path to the internal package modules. It is a bit difficult to explain the issue, but here is the gist.
So the python package name is pyugend. Now when I try and import a module--inside the package--named Models into a test case, pycharm forces me to reference the path as pyugend.pyugend.Models. So I need to reference pyugend twice.
However, when I build, install, and import this package into a jupyter notebook or some script, then I run into errors about the pyugend package not finding the internal modules. The only way to fix these errors is to change the paths inside of the module to references like pyugend.Models.
So basically, to run tests inside of pycharm I have make sure all of the internal package imports use a directory path like from pyugend.pyugend.Models import ... But when I want to use the package outside of pycharm then I actually have to go into the package, convert all of the import pyugend.pyugend... references to just single import pyugend.Models ... references.
I have included a picture of the directory structure as well as a picture of the __init__.py.
You can add the linesys.path.append(os.path.dirname(os.path.abspath(__file__))) before the imports in __init.py__ and then change the imports to from pyugend.Models import Base_model and so on, to enable consistent behavior wherever you use the package.
Let's say I have vtk module in my Python site packages, and from application with own Python distribution I want to access this module.
I tried couple of things like:
import sys
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
import vtk
lut = vtk.vtkLookupTable()
but it fails to load module properly:
AttributeError: 'module' object has no attribute 'vtkLookupTable'
If I do same from default Python interpreter all is fine.
Now I thought to make a wrapper of vtk in this application site packages, with simple __init__.py resolving paths, so that when I do import vtk it will hopefully load right thing, but I have no experience with Python packages to try to make this work
To put it simple, how can I wrap module from arbitrary folder, in Python site packages by making folder with same name as referenced package and simple __init__.py file?
Remove these lines:
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
The site-packages will already be on your python path. Adding a package/folder within that python path (especially at the first level), will just mess with your imports. How is this vtk package structured?
/path/to/site-packages/
vtk/
__init__.py
vtk.py
In this case, to access a function within vtk:
from vtk import vtk
lut = vtk.vtkLookupTable()
It all comes down to how the folder is arranged. You could also do this:
import vtk
lut = vtk.vtk.vtkLookupTable()
Do not try to hack python importing by creating proxy modules simply because you're not understanding how python importing is working. The error was quite clear. The attribute vtkLookupTable did not exist on whatever it was you imported. You imported the wrong thing. Fix it.
You should very very very very rarely have to manipulate the sys.path manually. When you do have to, you should know that it's the right reason - not to work around something you're not fully understanding.
I had trouble with python paths when I first started with python. It can be frustrating, but coming to understand how it works is necessary. What can help you is something like the following:
import vtk
print dir(vtk)
That will print the attributes of vtk, so you can explore exactly what is in the package or module in cases like this where you think you're importing the right thing.
After re-reading your question, it seems like this is a different python install you're talking about. The answer is to install this package into the other python install, or include this package as a top level import by copying the folder into the root level of your application.
"C:\Python27\Lib\site-packages" is already on your python path. So appending path is unnecessary. Remove:
import sys
sys.path.append("C:\Python27\Lib\site-packages")
sys.path.append("C:\Python27\Lib\site-packages\vtk")
Create a new folder called 'vtk\' in "C:\Python27\Lib\site-packages", then create a new python file named __init__.py in "C:\Python27\Lib\site-packages\vtk" and put your own module vtk.py in this directory.
Using:
import vtk
or
from vtk import vtk
to use your own module.