I would like to give TextCat a try. It would be most convenient for me if I could run it from Python, as I would like to see how well it does on a private dataset.
I gave languagedet, but according to
from languagedet.mixed import MixedDetector
det = MixedDetector()
print(det.available)
much less than the 69 languages claimed on TextCats website are available via languagedet.
I also tried pylibtextcat, but I get:
Collecting pylibtextcat
Using cached pylibtextcat-0.2.tar.bz2
Building wheels for collected packages: pylibtextcat
Running setup.py bdist_wheel for pylibtextcat ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-1dkslney/pylibtextcat/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpyct9pyfepip-wheel- --python-tag cp35:
running bdist_wheel
running build
running build_ext
building 'textcat' extension
creating build
creating build/temp.linux-x86_64-3.5
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION="0.2" -I/usr/include/python3.5m -c libtextcat.c -o build/temp.linux-x86_64-3.5/libtextcat.o -Wall -Wextra
libtextcat.c:7:32: fatal error: libtextcat/textcat.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Failed building wheel for pylibtextcat
Running setup.py clean for pylibtextcat
Failed to build pylibtextcat
Installing collected packages: pylibtextcat
Running setup.py install for pylibtextcat ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-1dkslney/pylibtextcat/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-lwxglu50-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_ext
building 'textcat' extension
creating build
creating build/temp.linux-x86_64-3.5
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION="0.2" -I/usr/include/python3.5m -c libtextcat.c -o build/temp.linux-x86_64-3.5/libtextcat.o -Wall -Wextra
libtextcat.c:7:32: fatal error: libtextcat/textcat.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-1dkslney/pylibtextcat/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-lwxglu50-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-1dkslney/pylibtextcat/
when I try to install it (and I have libexttextcat-2.0-0, libexttextcat-data, libexttextcat-dev installed).
Can I use TextCat with Python?
Seems not to be the same, but nltk has:
from nltk.classify import textcat
text = "This is a simple example."
cls = textcat.TextCat()
distances = cls.lang_dists(text) # a dict of 437 elements
cls.guess_language(text) # a str
Related
when I try to pip3 install tslearn on my ElementaryOS (similar to ubuntu) machine I get this error
building 'tslearn.cycc' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/tslearn
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/bruno/.local/lib/python3.6/site-packages/numpy/core/include -I/usr/include/python3.6m -c tslearn/cycc.c -o build/temp.linux-x86_64-3.6/tslearn/cycc.o
/usr/lib/gcc/x86_64-linux-gnu/7/cc1: error while loading shared libraries: libisl.so.19: cannot open shared object file: No such file or directory
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-5p7o5dn8/tslearn/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-tqrjth8y-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-5p7o5dn8/tslearn/
I've tried the solutions on this thread but nothing works. Any ideas? Thanks
In the end I fixed it by installing libisl, then going into the install location, locating libisl.so, and creating a symlink to point libisl.so.19 to it - by doing ln -s libisl.so libisl.so.19
I'm trying to use import and use mmh3 for hashing however I get errors when I try to do so.
These are the error i get:
Collecting mmh3
Using cached mmh3-2.5.1.tar.gz (9.8 kB)
Building wheels for collected packages: mmh3*emphasized text*
Building wheel for mmh3 (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /PYSPARK-ENV/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6vdrtr2d/mmh3/setup.py'"'"'; file='"'"'/tmp/pip-install-6vdrtr2d/mmh3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-vr4gj2gj
cwd: /tmp/pip-install-6vdrtr2d/mmh3/
Complete output (9 lines):
running bdist_wheel
running build
running build_ext
building 'mmh3' extension
creating build
creating build/temp.linux-x86_64-3.6
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c mmh3module.cpp -o build/temp.linux-x86_64-3.6/mmh3module.o
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
ERROR: Failed building wheel for mmh3
Running setup.py clean for mmh3
Failed to build mmh3
Installing collected packages: mmh3
Running setup.py install for mmh3 ... error
ERROR: Command errored out with exit status 1:
command: /PYSPARK-ENV/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6vdrtr2d/mmh3/setup.py'"'"'; file='"'"'/tmp/pip-install-6vdrtr2d/mmh3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-rvdzd4bs/install-record.txt --single-version-externally-managed --compile --install-headers /PYSPARK-ENV/include/site/python3.6/mmh3
cwd: /tmp/pip-install-6vdrtr2d/mmh3/
Complete output (9 lines):
running install
running build
running build_ext
building 'mmh3' extension
creating build
creating build/temp.linux-x86_64-3.6
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c mmh3module.cpp -o build/temp.linux-x86_64-3.6/mmh3module.o
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /PYSPARK-ENV/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6vdrtr2d/mmh3/setup.py'"'"'; file='"'"'/tmp/pip-install-6vdrtr2d/mmh3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-rvdzd4bs/install-record.txt --single-version-externally-managed --compile --install-headers /PYSPARK-ENV/include/site/python3.6/mmh3 Check the logs for full command output.
Note: you may need to restart the kernel to use updated packages.
Then if I try and just import it, it will say mmh3 not found
Even though it all works on my local machine
Any help would be greatly appreciated.
I am trying to install ujson which is required dependency for the installation of RASA. The command I triggered is follows:
sudo pip3 install ujson
And got the following error message.
WARNING: The directory '/home/kamaldeep/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting ujson
Downloading ujson-1.35.tar.gz (192 kB)
|████████████████████████████████| 192 kB 217 kB/s
Building wheels for collected packages: ujson
Building wheel for ujson (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uo5b9gzb/ujson/setup.py'"'"'; _file__='"'"'/tmp/pip-install-uo5b9gzb/ujson/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, _file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-918warx5
cwd: /tmp/pip-install-uo5b9gzb/ujson/
Complete output (14 lines):
Warning: 'classifiers' should be a list, got type 'filter'
running bdist_wheel
running build
running build_ext
building 'ujson' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/python
creating build/temp.linux-x86_64-3.7/lib
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I./python -I./lib -I/usr/include/python3.7m -c ./python/ujson.c -o build/temp.linux-x86_64-3.7/./python/ujson.o -D_GNU_SOURCE
In file included from ./python/ujson.c:39:0:
./python/py_defines.h:39:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for ujson
Running setup.py clean for ujson
Failed to build ujson
Installing collected packages: ujson
Running setup.py install for ujson ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uo5b9gzb/ujson/setup.py'"'"'; _file__='"'"'/tmp/pip-install-uo5b9gzb/ujson/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, _file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-c2p8ewru/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.7/ujson
cwd: /tmp/pip-install-uo5b9gzb/ujson/
Complete output (14 lines):
Warning: 'classifiers' should be a list, got type 'filter'
running install
running build
running build_ext
building 'ujson' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/python
creating build/temp.linux-x86_64-3.7/lib
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I./python -I./lib -I/usr/include/python3.7m -c ./python/ujson.c -o build/temp.linux-x86_64-3.7/./python/ujson.o -D_GNU_SOURCE
In file included from ./python/ujson.c:39:0:
./python/py_defines.h:39:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uo5b9gzb/ujson/setup.py'"'"'; _file__='"'"'/tmp/pip-install-uo5b9gzb/ujson/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, _file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-c2p8ewru/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.7/ujson Check the logs for full command output.
enter image description here
Please let me know the resolution. Thanks in advance.
You need to install the python developer package, since your are building the module from source.
sudo aptitude install python-dev
It’s Super Easy!
sudo apt-get update
sudo apt-get install python3-ujson
I have errors almost everytime I use pip (the package installer of Python) and I really don't manage to figure out how to fix them.
Below are the errors while trying to install pyv8 :
$ pip3 install pyv8
Collecting pyv8
Downloading https://files.pythonhosted.org/packages/c6/fb/5c0512a373e3d0aad0b683f439e7bdd67f95a69e6473b76dc2cfdca2c43f/PyV8-0.5.zip
Building wheels for collected packages: pyv8
Running setup.py bdist_wheel for pyv8 ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-wmzf4xop/pyv8/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-h9c9ol37 --python-tag cp37:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
copying PyV8.py -> build/lib.linux-x86_64-3.7
running build_ext
building '_PyV8' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBOOST_PYTHON_STATIC_LIB -Ilib/python/inc -Ilib/boost/inc -Ilib/v8/inc -I/usr/include/python3.7m -c src/Exception.cpp -o build/temp.linux-x86_64-3.7/src/Exception.o
In file included from src/Exception.cpp:1:
src/Exception.h:6:10: fatal error: v8.h: Aucun fichier ou dossier de ce type
#include <v8.h>
^~~~~~
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Failed building wheel for pyv8
Running setup.py clean for pyv8
Failed to build pyv8
Installing collected packages: pyv8
Running setup.py install for pyv8 ... error
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-wmzf4xop/pyv8/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-wig6t3n9/install-record.txt --single-version-externally-managed --compile --user --prefix=:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
copying PyV8.py -> build/lib.linux-x86_64-3.7
running build_ext
building '_PyV8' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DBOOST_PYTHON_STATIC_LIB -Ilib/python/inc -Ilib/boost/inc -Ilib/v8/inc -I/usr/include/python3.7m -c src/Exception.cpp -o build/temp.linux-x86_64-3.7/src/Exception.o
In file included from src/Exception.cpp:1:
src/Exception.h:6:10: fatal error: v8.h: Aucun fichier ou dossier de ce type
#include <v8.h>
^~~~~~
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-wmzf4xop/pyv8/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-wig6t3n9/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-install-wmzf4xop/pyv8/
Sorry if I can't give more precision
I'm trying to install GDAL with python.But it failed with error.
The command I use is pip install GDAL.
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr/ -I../../ogr/ogrsf_frmts -I../../gnm -I../../apps -I/usr/include/python2.7 -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/include -c extensions/gdal_wrap.cpp -o build/temp.linux-x86_64-2.7/extensions/gdal_wrap.o -std=c++11 -I/usr/include/gdal
extensions/gdal_wrap.cpp:3177:27: fatal error: cpl_vsi_error.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
and
----------------------------------------
Failed building wheel for GDAL
Running setup.py clean for GDAL
Failed to build GDAL
Installing collected packages: GDAL
Running setup.py install for GDAL ... error
Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_spRXy/GDAL/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-NxpUaO-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
copying gdal.py -> build/lib.linux-x86_64-2.7
...
creating build/temp.linux-x86_64-2.7/extensions
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr/ -I../../ogr/ogrsf_frmts -I../../gnm -I../../apps -I/usr/include/python2.7 -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/include -c extensions/gdal_wrap.cpp -o build/temp.linux-x86_64-2.7/extensions/gdal_wrap.o -std=c++11 -I/usr/include/gdal
extensions/gdal_wrap.cpp:3177:27: fatal error: cpl_vsi_error.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_spRXy/GDAL/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-NxpUaO-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-_spRXy/GDAL/
I've already tried sudo apt-get install build-essential, but stil the same error occurs.
Here is the answer I found that worked:
"you might have to change the gdal version to the version installed on your host. So I had to do this since I have gdal==1.11.2 on my host:"
pip install gdal==1.11.2 --global-option=build_ext --global-option="-I/usr/include/gdal/"
Where the 1.11.2 should be updated to your gdal_version, which can be found in the line # define GDAL_RELEASE_NAME of the /usr/include/gdal/gdal_version.h file (at least on my system running Kubuntu).
Link to original github page with this answer from Basaks, mentioned in the comment above by Craicerjack.