We Keep Coding

Locating a data file installed by another Python library not working when installed to site-packages without "-e"

I have a project packaged as:
wcnlp-tools (project root)
nlu-spacy
setup.py
...
spacy(package name)
...
nlu-wcnlp
setup.py
...
wcnlp (package name)
...
wcnlp depends on spacy.
When I use:
pip install -e .
to install both libraries, it works all fine. However, if I don't use -e to install both, it reprots an error.
pip install .
The line which caused this error is:
abspath = os.path.abspath(os.path.dirname(__file__))
read_yaml_file('os.path.join(abspath,"../../../../nlu-wcnlp/wcnlp/configs/spacy_config.yml")')
The error message is:
No such file or directory: '/Users/minmin/nlp/test/wcnlp-tools/ven/lib/python3.7/site-packages/spacy/lang/en/../../../../nlu-wcnlp/wcnlp/configs/spacy_config.yml'
The actual path, if correctly resolved, should be:
/Users/minmin/nlp/test/wcnlp-tools/ven/lib/python3.7/site-packages/spacy/lang/en/../../../../site-packages/wcnlp/configs/spacy_config.yml
So the differences between the two:
site-packages
VS
nlu-wcnlp
What's possible reason? Should I change my file path code, or setup scripts? Note that 'pip install -e .' works all fine.
ADDITIONS:
The error originates from nlu-wcnlp, but it's traced to the project nlu-spacy, which contains the error.
Error trace:
File "/Users/minmin/nlp/test/wcnlp-tools/ven/lib/python3.7/site-packages/wcnlp/nlp_utils.py", line 4, in <module>
from spacy.lang.en.stop_words import STOP_WORDS
File "/Users/minmin/nlp/test/wcnlp-tools/ven/lib/python3.7/site-packages/spacy/lang/en/__init__.py", line 32, in <module>
CONFIG = read_yaml_file(SPACY_CONFIG_FILE)
File "/Users/minmin/nlp/test/wcnlp-tools/ven/lib/python3.7/site-packages/wcnlp/utils/fileio.py", line 10, in read_yaml_file
with open(filename) as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/minmin/nlp/test/wcnlp-tools/ven/lib/python3.7/site-packages/spacy/lang/en/../../../../wcnlp-tools/wcnlp/configs/spacy_config.yml

When using __file__ to search for data files associated with a package, you should always use a package relative to the __file__ attribute for a module associated with the package that provides the data file. Do not try to use a relative path that spans the space between two packages, and thus assumes that it can correctly predict where those packages will be installed relative to each other.
So, this is unreliable (makes assumptions about installation locations that are not guaranteed to hold):
SPACEY_CONFIG_FILE = os.path.join(os.path.dirname(__file__),
'/../../../../site-packages/wcnlp/configs/spacy_config.yml'
...but this is okay:
import wcnlp
SPACEY_CONFIG_FILE = os.path.join(os.path.dirname(wcnlp.__file__),
'configs/spacy_config.yml')

OpenALPR Python binding failing when trying to load dll

After downloading the precompiled binaries for Windows of the openALPR library, running the setup.py included in the source code to install it, and running the python_test.bat included in the precompiled binaries directory I get the following error:
File "C:\Users\rhenriquez\AppData\Local\Programs\Python\Python37-32\lib\site-packages\openalpr\openalpr.py", line 51, in __init__
self._openalprpy_lib = ctypes.cdll.LoadLibrary("libopenalprpy.dll")
File "C:\Users\rhenriquez\AppData\Local\Programs\Python\Python37-32\lib\ctypes\__init__.py", line 434, in LoadLibrary
return self._dlltype(name)
File "C:\Users\rhenriquez\AppData\Local\Programs\Python\Python37-32\lib\ctypes\__init__.py", line 356, in __init__
self._handle = _dlopen(self._name, mode)
OSError: OSError: [WinError 193] %1 is not a valid Win32 application
When I started to run the script at first the error was WinError 126 and then I noticed that the dll that the script was trying to call (openalprpy.dll) didn't had "lib" at the beginning of its name, so I added it to the dll's name. Then it started to send me WinError 193, and I've been clueless on what else to do or what am I doing wrong from then on.
Any help would be appreciated.

EDIT: So I tried this with the 32-bit version and it gives the same error you are encountering. This appears to be an issue with trying to import a 32-bit .dll while using 64-bit python, or vice-versa, as seen in Python Ctypes Load Library
/EDIT
I did get it to work on my system... with a few modifications, this package is not as "plug and play" as it should be.
I don't know where I went right, so I'll just list what I did:
Download the pre-compiled biniaries from releases (I used openalpr-2.3.0-win-64bit.zip) https://github.com/openalpr/openalpr/releases
Download the project itself, https://github.com/openalpr/openalpr
Unzip both.
Goto the bindings folder in openalpr-master cd C:\openalpr-master\openalpr-master\src\bindings\python and run python setup.py install to make the bindings.
Then navigate to the project folder in site-packages, most likely C:\Users\rhenriquez\AppData\Local\Programs\Python\Python37-32\lib\site-packages\openalpr\ and open openalpr.py in IDLE,
here you can change line 51 from self._openalprpy_lib = ctypes.cdll.LoadLibrary("libopenalprpy.dll") to self._openalprpy_lib = ctypes.cdll.LoadLibrary("openalprpy.dll") since it appears other links are broken if you change the file name.
I also found it beneficial to change line 90 to except Exception: since it did not want to import correctly and was not raising an ImportError.
After that the python_test.bat worked correctly.
Namespace(config='openalpr.conf', country='us', plate_image='samples/us-1.jpg', runtime_data='runtime_data')
Using OpenALPR 2.3.0
Image size: 497x372
Processing Time: 561.825989
Plate #1
Plate Confidence
- THECAR 92.207481
- THEGAR 81.348961
- HECAR 80.229317
- TMECAR 78.159492
- THE0AR 77.702461
- THECAB 77.389000
- THEAR 76.510017
Press any key to continue . . .

ImportError: cannot import name 'input_reader_pb2'

working on win10 64-bit
when i trying to train my model by E:\projectx\model-master\models-master>python train.py --logtostderr --train_dir=training/ --pipeline_config_path=training/ssd_mobilenet_v1_pets.config
this error appear
File "train.py", line 49, in
from object_detection.builders import dataset_builder
File "C:\Users\DELL\AppData\Local\Programs\Python\Python36\lib\site-packages\object_detection-0.1-py3.6.egg\object_detection\builders\dataset_builder.py",
line 27, in
from object_detection.data_decoders import tf_example_decoder
File "C:\Users\DELL\AppData\Local\Programs\Python\Python36\lib\site-packages\object_detection-0.1-py3.6.egg\object_detection\data_decoders\tf_example_decoder.py",
line 27, in
from object_detection.protos import input_reader_pb2
ImportError: cannot import name 'input_reader_pb2'
i do run protoc 3.4 protoc object_detection/protos/*.proto --python_out=.
but the error still exist
i check all thing and it is should be fine and work, it is going me mad!!
please help....
thanks

As you already compiled all .proto files in object_detection/protos/ . You should see python files which have '_pb2' like eval_pb2. If you can see these then go to models/research directory and run these code one by one:
python setup.py build
python setup.py install

From inside the object_detection folder:
protoc ../object_detection/protos/*.proto --python_out=.
This command will generate a *_pb2.py for each .proto file in the object_detection/protos/ folder.
Note: Is important to specify a path above object_detection, otherwise errors would occur, most likely:
object_detection/protos/ssd_anchor_generator.proto: File not found.
protos/anchor_generator.proto:8:1: Import "object_detection/protos/ssd_anchor_generator.proto" was not found or had errors.
protos/anchor_generator.proto:17:5: "FlexibleGridAnchorGenerator" is not defined.
If the error persists probably you are in the wrong folder, take a look at the output message to see from where it is trying to import the file and execute the command in the right directory.
As last resort: Download the object_detection module from https://github.com/tensorflow/models/tree/master/research
place it in your working directory, enter it and re-execute the command above.
It will surely works as local modules have import priority over sys.path.
If not, the error message would be probably different from the one reported and the problem lies on the tensorflow installation or the protobuf compiler; as here where the problem was caused by the protoc version.
other useful links: https://github.com/tensorflow/models/issues/5264

Maybe you haven't added module slim to PYTHONPATH.It can be done by running below code inside models/research directory.
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

Seems that you didnt compile protobuf.
For solving it:
Download the latest protoc exe here: https://github.com/google/protobuf/releases (in your case should be win32)
Rename that folder to "protoc"
Put that folder inside models/research
in models/research via console, launch:
protoc/bin/protoc object_detection/protos/*.proto --python_out=.
I dont know exactly if that command will work on windows, but you have to be sure that you are using the protoc compiler that you downloaded (v.3.6) instead of the protoc in your enviroment.

ImportError: No module named stack

I have a code in python that I have been working on and it builds and runs very well on my pc (Windows). I had to run the same code on my other machine which runs ubuntu,so I had to install all the packages on prior to runing the code. The problem is I ran into this error which I couldn't figure out. The error is triggered by one of the installed packages.
from qalsadi import analex
File "/usr/local/lib/python2.7/dist-packages/qalsadi/analex.py", line 14, in <module>
import pyarabic.araby as araby # basic arabic text functions
File "/usr/local/lib/python2.7/dist-packages/pyarabic/araby.py", line 28, in <module>
from stack import *
ImportError: No module named stack
I used the following command, "sudo pip install pyarabic", to install it. However, still the file stack.py doesn't exist among it's files. I searched in the folder /usr/local/lib/python2.7/dist-packages/pyarabic. The folder contains the following: araby.py and init.py and the coresponding pyc files only. I'v insalled and uninstalled it a number of times using "pip" but still the file is not there.

Check your pyarabic folder. Usually it's in Python27\Lib\site-packages\pyarabic.
There, there should be stack.py. If it doesn't exists, re-download pyarabic and then reinstall it.

After installation of pyarabic import STACK in this manner:
from pyarabic.stack import Stack

for window users
open cmd prompt and type the following to install the stack variable to python 3.x-
pip install pyarabic
To install and run with this code-
from pyarabic.stack import Stack

It seems like stack is not part of the Python Package Index so most probably it is a script you installed manually. The problem can be that the folder containing stack.py is not on your PYTHONPATH.
Open a terminal (Ctrl+ Alt + t) and edit the .bashrc file:
sudo gedit ~/.bashrc
Add the following line:
export PYTHONPATH=$PYTHONPATH:/path/to/the/folder/of/your/module
where you should substitute the part after the : to the full path to the directory
where stack.py can be found.
I hope this helps.

How can I use Django with MySQL in MAMP stack?

I have difficulty especially in installing MySQLdb module (MySQL-python-1.2.3c1), to connect to the MySQL in MAMP stack.
I've done a number of things such as copying the mysql include directory and library (including plugin) from a fresh installation of mysql (version 5.1.47) to the one inside MAMP (version 5.1.37).
Now, the MySQLdb module build and install doesnt give me error.
The error happens when I'm calling 'import MySQLdb' from python shell (version 2.6).
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.macosx-10.6-universal/egg/MySQLdb/__init__.py", line 19, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 7, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so, 2): Symbol not found: _mysql_affected_rows
Referenced from: /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
Expected in: flat namespace
in /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
Any idea, what else do I need to do to make it works?
Thanks a bunch,
Robert
=========
Add the system response after using virtualenv as suggested by Hank Gay below...
(MyDjangoProject)MyMacPro:MyDjangoProject rhenru$ which python
/Users/rhenru/Workspace/django/MyDjangoProject/bin/python
After I run python in virtualenv, importing MySQLdb:
>>> import MySQLdb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.macosx-10.6-universal/egg/MySQLdb/__init__.py", line 19, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 7, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so, 2): Symbol not found: _mysql_affected_rows
Referenced from: /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
Expected in: flat namespace
in /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
import sys and sys.path
>>> import sys
>>> print sys.path
['', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/site-packages/distribute-0.6.10-py2.6.egg', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/site-packages/pip-0.7.1-py2.6.egg', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python26.zip', '/Library/Python/2.6/site-packages/PyXML-0.8.4-py2.6-macosx-10.6-universal.egg', '/Library/Python/2.6/site-packages/pydot-1.0.2-py2.6.egg', '/Library/Python/2.6/site-packages/pyparsing-1.5.2-py2.6.egg', '/Library/Python/2.6/site-packages/vobject-0.8.1c-py2.6.egg', '/Library/Python/2.6/site-packages/pytz-2010h-py2.6.egg', '/Library/Python/2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg', '/Library/Python/2.6/site-packages/distribute-0.6.12-py2.6.egg', '/Library/Python/2.6/site-packages/pip-0.7.1-py2.6.egg', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/plat-darwin', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/plat-mac', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/plat-mac/lib-scriptpackages', '/Users/rhenru/Workspace/django/MyDjangoProject/Extras/lib/python', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/lib-tk', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/lib-old', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/site-packages', '/Library/Python/2.6/site-packages', '/Library/Python/2.6/site-packages/PIL', '/Library/Python/2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode']

How are you installing MySQL-Python? I just tested in a fresh virtualenv and pip install mysql-python seems to have done the trick.
UPDATE:
pip is sort of like a package manager for Python packages.
By default, pip installs to your current site-packages directory, which is on your $PYTHONPATH. This lets other libraries/applications (like Django) access it. pip also works well with virtualenv (it should; Ian Bicking wrote them both), which is a nifty library that lets you sandbox an application. This is nice because it means you can try out new things without polluting (or even needing write access to) the global site-packages directory.
It probably seems like yak-shaving right now, but I'd say it's worth the effort to get up to speed on pip and virtualenv (you may also want to look into virtualenvwrapper, but we'll skip that for now; it's just sugar for virtualenv). It will lead to a slightly more complicated deployment scenario than putting everything in the global site-packages, but for development it's really no harder, and there are lots of good guides to deploying using a virtualenv.
I'd recommend something like the following:
curl -0 http://python-distribute.org/distribute_setup.py
python distribute_setup.py
easy_install pip
pip install virtualenv
virtualenv --distribute MyDjangoProject --no-site-packages
cd MyDjangoProject
source bin/activate (this activates the sandbox that virtualenv created)
pip install django mysql-python
At this point, you should have a totally functional Django+MySQL install (if I missed any steps, just comment and I'll try to add it in). You can start your Django project like this: django-admin.py startproject MyDjangoProject. cd into your project's directory, edit your settings.py file to point to your MySQL database, and run the dev server to test it out like so: ./manage.py runserver (you may need to chmod u+x your manage.py file). Voila! You should be able to access your site on localhost:8000. When you're done working on the project, you can just use deactivate to exit the virtualenv sandbox.
Try not to hold all this against Django: a lot of it is just best practices stuff for working with Python libraries. You could get by with a lot less, but this way it's more reproducible and you're less likely to accidentally mess up one of this project's dependencies when working on a different project.

I had this problem and it turned out to be due to an errant configuration:
export VERSIONER_PYTHON_PREFER_32_BIT=yes
I can't recall what I had this enabled for (some package that required 32-bit), probably related to Google AppEngine. But Setting it to 'no' solved by issues.
Otherwise I just installed everything using homebrew and pip.