I am trying to use the tensorflow object detection APIs for one of my personal projects. Here you can find the link to the repo for the same that needs to be cloned before using it. In this repo, there are multiple directories and are arranged in this fashion:
models
+ research
+ object_detection
+
+ ....
+ ....
Now the object_detection is the one that is of use to me. So, I create a new directory in the parent directory research for my own models and try to use the functionalities given in the object_detection directory. Within the object_detction directory is a sub-directory utils which contains a bunch of .py files for certain utilities. Here is a sample link for such a file in that directory. Now when I try to use this file in my code, it says no module found: object_detection. This is a relative import error but I am not getting how to resolve it. I just want to make sure that I can use each of the functionality without modifying the imports and this is doable because there is an __init__ file in each sub-directory but somehow this isn't working. Please help!!
As per the installation instructions, please add slim/ and models/ to your PYTHONPATH.
Just for clarification, I assume that you already installed tensorflow/models. Otherwise install it (and all its dependencies) either as explained here or in an virtual environment.
Either way you can import code from the object_detection API like so:
from object_detection import <myExample>
For your example that would be:
from object_detection.utils import label_map_util
Related
I have connected a Github repository to my Databricks workspace, and am trying to import a module that's in this repo into a notebook also within the repo. The structure is as such:
Repo_Name
Checks.py
Test.ipynb
The path to this repo is in my sys.path(), yet I still get ModuleNotFoundError: No module named 'Checks'. When I try to do import Checks. This link explains that you should be able to import any modules that are in the PATH. Does anyone know why it might still not be working?
I have tried doing the same and got a similar error even after following the procedure as given in the link provided in the question.
I have the following python files in my GIT repo (3 files with .py extension).
Now when I add the path /Workspace/Repos/<username>/repro0812 to sys.path and try to import the sample module from this repo, it throws the same error.
This is because, for some reason, this file is not being rendered as a python file. When I open the repo, you can actually see the difference.
There was no problem while I import the other 2 python modules check and sample2. The following is an image for refernce.
Check and make sure that the file is being considered as a .py file after adding your repo.
I am trying to use the nets from the official mnist directory of tensorflows model repository. On my windows system I receive this error:
C:\Users\ry\Desktop\NNTesting\models\official\mnist>mnist_test.py
Traceback (most recent call last):
File "C:\Users\ry\Desktop\NNTesting\models\official\mnist\mnist_test.py",line 24, in <module>
from official.mnist import mnist
ModuleNotFoundError: No module named 'official'
I have followed their official directions and set my python path using
set PYTHONPATH="PYTHONPATH:"%cd%"
and can confirm that
PYTHONPATH="$PYTHONPATH:C:\Users\ry\Desktop\NNTesting\models"
and I have also installed the dependencies successfully. Does anyone have experience using these models on a windows system and can help me with this pathing issue? I'm not sure what I have done incorrectly here.
Thanks
pip install tf-models-official
For Google Colab I needed to add the the model dir also to the Systems path:
!git clone https://github.com/tensorflow/models.git
import os
os.environ['PYTHONPATH'] += ":/content/models"
import sys
sys.path.append("/content/models")
if anyone has this problem make sure that the python path variable doesn't have quotations in it. For some reason, the readme has quotations around it.
Here is the correct way to set it
PYTHONPATH=path\to\models
I had exactly the same question as you did, and the following solution solved this problem.
There is an error in the tensorflow/models/official README.md
https://github.com/tensorflow/models/tree/master/official
Wrong
export PYTHONPATH="$PYTHONPATH:/path/to/models"
Correct
export PYTHONPATH=$PYTHONPATH:/path/to/models
The Official Models are made available as a Python module. To run the models and associated scripts, add the top-level /models folder to the Python path with the command: export PYTHONPATH="$PYTHONPATH:/path/to/models"
FROM README
I had the same problem. Did you use windows 10? Make sure you run the command prompt as "administrator". I used it in VS code at first, no warning, and didn't work. But it worked when I run a separate prompt window as "administrator".
set PYTHONPATH=path\to\models
then run the model.
I was setting up to run the NMT model and ran into the same problem.
It took me bit to figure out exactly which folder should be added to PYTHONPATH.
I tried several folders inside my example's directory with no luck.
I finally understood what that import was trying to tell me...
"from official.transformer.utils import tokenizer"
means
"add the parent of directory 'official' to PYTHONPATH".
For me, this was just the top-level 'models-master' directory that I obtained from GitHub. Once I added /path/to/models-master, I was past this obstacle.
Go to models folder and do
export PYTHONPATH=$PYTHONPATH:$PWD
add the model directory to PYTHONPATH.
import os
os.environ['PYTHONPATH'] += ':/content/models/research/:/content/models/research/slim/'
jupyter. Clone from official git and manually appended the path to sys.
!git clone https://github.com/tensorflow/models.git
import sys
sys.path.append("C:/Windows/System32/models")
sys.path
I came up with the situation where I want to use 2 separate projects in tensorflow. The problem is that I use a (quite heavy loaded) virtual environment and I would like to keep using it. In those projects there are two folders named (not too surprisingly) utils. So, I guess a conflict occurs when I import from the second folder.
Possible solutions that I could think of:
Create a new virtual environment or even better clone the existing one like in here I guess. Con is that it takes about 1.7GB and I am running out of space (but I could deal with that somehow if that's the best choice).
Try to debug the conflicting cases using folder prefixes like instead of:
from utils import label_map_util I could use from project2.utils import label_map_util. This seems to work but has the con that I must search every file for conflicting imports. Like inside label_map_util.py where a similar import crashes (there are various folders in the project).
Is there any better solution? Am I missing something obvious here?
Also (I am fairly new to python) should this conflict be dealt automatically by python? I mean when inside a module in a folder and trying to import another module existing in the same folder or subfolder shouldn't this get priority over modules in other parallel folders (possibly in other projects etc)?
Edit:
To make it more clear I have 2 projects say projectA and projectB meaning 2 folders with various subfolders, which both contains a folder named utils. projectA was running just fine before the installation of projectB.
When I installed projectB and tried to run a module in it it contained imports from utils folder. Like
from utils import label_map_util
from utils import visualization_utils as vis_util
etc. I am using pyCharm and my IDE did not recognized the import as valid. I tried to debug it in interactive mode and my conclusions were:
There is a label_map_util.py module in folder utils in projectB folder.
For some reason import tries to find this module in projectA. So, when there is no module with that name it complains giving error:
ImportError: cannot import name 'label_map_util'
in that sense import utils works fine but imports the wrong module (from projectA).
I used below solution for importing dependencies.
I found this solution works if I run the code in Pycharm but not in Terminal.
The error message in Terminal is "cannot find graphics.primitive".
I'm using Mac and Python 3.5.
Why I see different behaviors from the Terminal and Pycharm?
How may I make the solution work for both?
http://chimera.labs.oreilly.com/books/1230000000393/ch10.html#_solution_169
Making a Hierarchical Package of Modules
Problem
You want to organize your code into a package consisting of a hierarchical collection of modules.
Solution
Making a package structure is simple. Just organize your code as you wish on the file-system and make sure that every directory defines an init.py file. For example:
graphics/
__init__.py
primitive/
__init__.py
line.py
fill.py
text.py
formats/
__init__.py
png.py
jpg.py
Once you have done this, you should be able to perform various import statements, such as the following:
import graphics.primitive.line
from graphics.primitive import line
import graphics.formats.jpg as jpg
You need to make sure that the graphics package is in the Python search path. PyCharm does this by extending sys.path as follows:
import sys
sys.path.extend(['/Users/hackworth/Development/graphics_parent_dir', '/Applications/PyCharm.app/Contents/helpers/pycharm', '/Applications/PyCharm.app/Contents/helpers/pydev'])
You can do the same in your code replacing /Users/hackworth/graphics_parent_dir with the appropriate path, or you can include the full path to graphics_parent_dir in the PYTHONPATH environment variable. See the Python documentation for details.
Another option would be to place the graphics package into a location the is searched by default on your system.
I developed a python3 package in Pycharm but am running into some confusing behavior when I try to import modules in my test cases. The problem seems to be with the directory path to the internal package modules. It is a bit difficult to explain the issue, but here is the gist.
So the python package name is pyugend. Now when I try and import a module--inside the package--named Models into a test case, pycharm forces me to reference the path as pyugend.pyugend.Models. So I need to reference pyugend twice.
However, when I build, install, and import this package into a jupyter notebook or some script, then I run into errors about the pyugend package not finding the internal modules. The only way to fix these errors is to change the paths inside of the module to references like pyugend.Models.
So basically, to run tests inside of pycharm I have make sure all of the internal package imports use a directory path like from pyugend.pyugend.Models import ... But when I want to use the package outside of pycharm then I actually have to go into the package, convert all of the import pyugend.pyugend... references to just single import pyugend.Models ... references.
I have included a picture of the directory structure as well as a picture of the __init__.py.
You can add the linesys.path.append(os.path.dirname(os.path.abspath(__file__))) before the imports in __init.py__ and then change the imports to from pyugend.Models import Base_model and so on, to enable consistent behavior wherever you use the package.