I have installed,
pip install thrift
pip install PyHive
pip install thrift-sasl
and
since pip install sasl failed I downloaded sasl‑0.2.1‑cp27‑cp27m‑win_amd64.whl file and installed it in my Windows 8.1 PC.
Then I wrote this code,
from pyhive import hive
cursor = hive.connect('192.168.1.232', port=10000, auth='NONE')
cursor.execute('SELECT * from sample_07 LIMIT 5',async=True)
print cursor.fetchall()
this gives the error:
Traceback (most recent call last):
File "C:/DigInEngine/scripts/UserManagementService/fd.py", line 37, in <module>
cursor = hive.connect('192.168.1.232', port=10000, auth = 'NONE')
File "C:\Python27\lib\site-packages\pyhive\hive.py", line 63, in connect
return Connection(*args, **kwargs)
File "C:\Python27\lib\site-packages\pyhive\hive.py", line 104, in __init__
self._transport.open()
File "C:\Python27\lib\site-packages\thrift_sasl\__init__.py", line 72, in open
message=("Could not start SASL: %s" % self.sasl.getError()))
thrift.transport.TTransport.TTransportException: Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: Unable to find a callback: 2
and this code gives,
from sqlalchemy import create_engine
engine = create_engine('hive://192.168.1.232:10000/default')
try:
connection = engine.connect()
except Exception, err:
print err
result = connection.execute('select * from sample_07;')
engine.dispose()
this error,
Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no
mechanism available: Unable to find a callback: 2
I have downloaded Hortonworks sandbox from here and use it in a separate server.
NOTE: I went through this as well but the accepted answer is not working for me, because importing ThriftHive from hive gives Import error although I have pip installed hive. So I decided to use pyhive or sqlalchemy
How can I connect to hive and execute a query easily?
Here are steps to build SASL on Windows, but your mileage may vary: A lot of this depends on your particular system's paths and available libraries.
Please also note that these instructions are specific to Python 2.7 (which I see you are using from the paths in your question).
The high-level overview is that you're installing this project: https://github.com/cyrusimap/cyrus-sasl. In order to do that, you have to use the legacy C++ compiler that was used to build Python 2.7. There are a couple of other steps to getting this to work.
Pre-build Steps:
Install Microsoft Visual C++ Compiler for Python 2.7. Use the default installation paths - take note of where it got installed for the next 2 steps (2 options are included in the list below)
Copy this file to whichever of the include locations is appropriate for your install
Make a unistd.h file from this answer in the same include directory
Build steps:
git clone https://github.com/cyrusimap/cyrus-sasl
Open the "VS2013 x64 Native Tools Command Prompt" that's installed with the Compiler from step 1
Change directory to the directory created by step 4, then the lib sub-directory
nmake /f ntmakefile STATIC=no prefix=C:\sasl64
nmake /f ntmakefile prefix=C:\sasl64 STATIC=no install see note below
copy /B C:\sasl64\lib\libsasl.lib /B C:\sasl64\lib\sasl2.lib
pip install thrift_sasl --global-option=build_ext \
--global-option=-IC:\\sasl64\\include \
--global-option=-LC:\\sasl64\\lib
'Include' locations:
"C:\Program Files (x86)\Common Files\Microsoft\Visual C++ for Python\9.0\VC\include\stdint.h"
"%USERPROFILE%\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\include"
Here's a reference to these same steps, with some additional annotations and explanations: http://java2developer.blogspot.co.uk/2016/08/making-impala-connection-from-python-on.html.
Note
The referenced instructions also executed step (8) in the include and win32\include sub-directories, you may have to do that as well.
While using pyhive no authentication can be passed as auth="NOSASL", instead of "None", so your code should look like this:
from pyhive import hive
cursor = hive.connect('192.168.1.232', port=10000, auth='NOSASL')
cursor.execute('SELECT * from sample_07 LIMIT 5',async=True)
print cursor.fetchall()
Related
https://raw.githubusercontent.com/paramiko/paramiko/master/demos/demo_server.py
I see the above demo_server of paramiko. But I don't see the instructions on how to run it. I run the following ./demo_server.py command. But once I run ssh robey#127.0.0.1 -p 2200, the server fails. Could anybody let me know the complete steps on how to run this example? Thanks.
$ python3 ./demo_server.py
Read key: 60733844cb5186657fdedaa22b5a57d5
Listening for connection ...
Got a connection!
*** Caught exception: <class 'ImportError'>: Unable to import a GSS-API / SSPI module!
Traceback (most recent call last):
File "./demo_server.py", line 140, in <module>
t = paramiko.Transport(client, gss_kex=DoGSSAPIKeyExchange)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/paramiko/transport.py", line 445, in __init__
self.kexgss_ctxt = GSSAuth("gssapi-keyex", gss_deleg_creds)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/paramiko/ssh_gss.py", line 107, in GSSAuth
raise ImportError("Unable to import a GSS-API / SSPI module!")
ImportError: Unable to import a GSS-API / SSPI module!
$ ssh robey#127.0.0.1 -p 2200
kex_exchange_identification: read: Connection reset by peer
I managed to get it to work, make sure to go through these steps (Thank you Sandeep for the pip insight), chances are you may be missing the Kerberos dependencies:
You may need to perform pip install gssapi in the CLI if that has not already been done (I'm using the Windows Command Prompt, Linux/WSL might need pip3 instead depending on your version of python)
From there you will need to import the gssapi library into the code at the top with the other imported libraries, so just call
import gssapi in demo_server.py
After running demo_server.py again, the CLI should eventually say something like it is missing files located in Program Files\ MIT\ Kerberos\ bin, as Kerberos is a dependency for gssapi, you can install it from here:
https://web.mit.edu/KERBEROS/dist/
Make sure you do custom install if you're not sure where it will be downloaded, so that you can set up the file location where the CLI says is missing in step 3 (Should automatically say Program Files\ MIT). I unchecked the boxes for auto-start and tickets, but not sure as to what your preferences may be. After all that, a computer restart is required.
When importing pyodbc
❯ python
>>> import pyodbc
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/pcosta/Documents/test/myenv/lib/python3.7/site-packages/pyodbc.cpython-37m-darwin.so, 2): Library not loaded: /usr/local/opt/unixodbc/lib/libodbc.2.dylib
Referenced from: /Users/pcosta/Documents/test/myenv/lib/python3.7/site-packages/pyodbc.cpython-37m-darwin.so
Reason: image not found
I know why this is happening, as I don't have libodbc.2.dylib in the expected location. The reason is I do not have permission to write to /usr/local/, so I have Homebrew installing into ~/.brew. This mostly works fine. I am even able to get both tsql and isql working as expected by following the steps outlined here: https://github.com/mkleehammer/pyodbc/wiki/Connecting-to-SQL-Server-from-Mac-OSX.
So I do have libodbc.2.dylib, it's just that it lives in /Users/pcosta/.brew/lib, not /usr/local/opt/unixodbc/lib.
The main questions is can I get pyodbc to look for libodbc.2.dylib (and other associated files) in another directory?
I have all the files needed and have configured them correctly, I just need to repoint pyodbc somehow.
Thanks!
Thanks in part to guidance from this GitHub issue I was able to come to some solution.
Assuming you have brew install unixodbc:
Add the following paths (to .zshrc, .bashrc, or .bash_profile):
export LDFLAGS="-L/Users/pcosta/homebrew/opt/unixodbc/lib $LDFLAGS"
export CPPFLAGS="-I/Users/pcosta/homebrew/opt/unixodbc/include $CPPFLAGS"
export PKG_CONFIG_PATH="/Users/pcosta/homebrew/opt/unixodbc/lib/pkgconfig $PKG_CONFIG_PATH"
Run pip install --no-binary pyodbc pyodbc to bypass the binary and build yourself
I successfully installed pyodbc module for python 2.7. However, when input the following code to connect to teradata,
import pyodbc
conn = pyodbc.connect('DRIVER={Teradata};DBCNAME=<tdwc>;UID=<UID>;PWD=<UID>;QUIETMODE=YES;')
I got the following error;
Traceback (most recent call last):
File "", line 1, in
pyodbc.connect('DRIVER={Teradata};DBCNAME=;UID=;PWD=;QUIETMODE=YES;')
Error: ('00000', '[00000] [iODBC][Driver Manager]dlopen(/Library/Application Support/teradata/client/ODBC/lib/tdata.dylib, 6): Library not loaded: libtdparse.dylib\n Referenced from: /Library/Application Support/teradata/client/ODBC/lib/tdata.dylib\n Reason: image not found (0) (SQLDriverConnect)')
What should I do to have this fixed? Any ideas?
Basically pyodbc is not straight forward and gives good enough debugging time for developers.
Follow the below steps,
You might have done them already, then just verify
Install iodbc for mac http://www.iodbc.org/dataspace/iodbc/wiki/iODBC/Downloads
Install Teradata ODBC Driver for Mac OS X, http://downloads.teradata.com/download/connectivity/teradata-odbc-driver-for-mac-os-x
Also install unixodbc for mac, “brew install unixodbc”
Download pyodbc source and change the setup.py file as below,
elif sys.platform == 'darwin':
# OS/X now ships with iODBC.
settings['libraries'].append('iodbc')
settings['libraries'].append('odbc')
you will be adding the last line of including “odbc” for build
Build and install pyodbc
Setup the traditional LD_LIBRARY_PATH for mac as below ( I used TD version 15.00, you have to point to the one as you installed)
export DYLD_LIBRARY_PATH=/Library/Application\
Support/teradata/client/ODBC/lib:/Library/Application\
Support/teradata/client/15.00/lib/
Export ODBCINI path
export ODBCINI=/Library/Application\
Support/teradata/client/15.00/odbc/odbc.ini
Now pyodbc cursor happily will fetch record sets for you
Good day,
I'm a student and I would just like to ask for a minute of your time.
I'm working on a barcode reader connected via USB port to a board name Arduino Yun. This board runs a version of embedded linux derived from OpenWrt using a microprocessor named Atheros AR9331
I would like to ask you, what's necessary to make the Python Evdev binding (python-evdev.readthedocs.org/en/latest/), to be able to run in this type of MIPS microarchitecture? At the momento, it's only for Ubuntu and ArchLinux.
I'm kind of guessing that cross compilation would be needed, or the indication of the usage of a specific C compiler inside this linux.
The current python version supported for OpenWrt is 2.7.3
I already know , if you compile C code in your PC, the resulting executable will only run in this type of architecture. If you use that compiled program inside the microprocessor, it wont work.
I've used this binding without trouble within ubuntu in my PC. I followed the instructions, python setup.py install, with a previous installation of setuptools, and it worked just fine.
But regarding OpenWrt, this was not the case.
The python script I'm using requires this library within the first line of code in order to reach the data from the device (it works like a keyboard /dev/input/event0):
#!/usr/bin/env python
from evdev import InputDevice, ecodes, list_devices
from select import select
I've seen suggestions of copying the entire library inside the arduino, and run the script inside the same folder. But it doesn't work, since the evdev module has files created with the architecture of the PC and not the MIPS.
So, what are the messages displayed for the error?
If you run python setup.py install in Openwrt to try to install the evdev binding, this appears on screen:
File "setup.py", line 10, in <module>
from setuptools.command.develop import develop
ImportError: No module named setuptools.command.develop
It's obvious from here that you need the module aforementioned. So, I tried to install it with this script (pypi.python.org/pypi/setuptools):
python ez_setup.py
And the output shows this:
Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-11.3.1.zip
Traceback (most recent call last):
File "ez_setup.py", line 332, in <module>
sys.exit(main())
File "ez_setup.py", line 327, in main
downloader_factory=options.downloader_factory,
File "ez_setup.py", line 287, in download_setuptools
downloader(url, saveto)
File "ez_setup.py", line 209, in download_file_curl
_clean_check(cmd, target)
File "ez_setup.py", line 169, in _clean_check
subprocess.check_call(cmd)
File "/usr/lib/python2.7/subprocess.py", line 511, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['curl','https://pypi.python.org/packages/source/s/setuptools/setuptools-11.3.1.zip', '--silent', '--output', '/mnt/sda1/evdev-0.4.6/setuptools-11.3.1.zip']' returned non-zero exit status 60
I pressume this output is due to the fact that pypi doesn't exist for the python 2.7.3 in OpenWrt , only for newer versions and other architectures. Evedv binding is requiring the setuptools module in order to make things easier and standard, but if the binding is not supported for the target architecture, what's needed to be able to use it anyways?
Thanks for your time,
Good day everyone,
The solution was provided by Georgi Valkov. He is the creator of the python-evdev binding. I contacted him directly, and he was so kind that he cross compiled a version for the OpenWrt / Yun .
You can install the package using the openwrt package manager - opkg. The installation process is along the lines of:
$ opkg update
$ opkg install /path/to/python-evdev_0.4.7-1_ar71xx.ipk
To verify that the install was successful:
$ opkg files python-evdev
/usr/lib/python2.7/site-packages/evdev-0.4.7-py2.7.egg-info
/usr/lib/python2.7/site-packages/evdev/genecodes.py
/usr/lib/python2.7/site-packages/evdev/ff.py
/usr/lib/python2.7/site-packages/evdev/_input.so
/usr/lib/python2.7/site-packages/evdev/device.py
/usr/lib/python2.7/site-packages/evdev/events.py
/usr/lib/python2.7/site-packages/evdev/__init__.py
/usr/lib/python2.7/site-packages/evdev/ecodes.py
/usr/lib/python2.7/site-packages/evdev/_ecodes.so
/usr/lib/python2.7/site-packages/evdev/util.py
/usr/lib/python2.7/site-packages/evdev/uinput.py
/usr/lib/python2.7/site-packages/evdev/_uinput.so
This works just fine. Thanks.
PS. If someone needs the file, please contact me. Georgi sent me this address, but I didn't download the file from there because he sent it to me over email.
https://github.com/gvalkov/openwrt-packages-yun/blob/master/lang/python-evdev/Makefile
In the output, you can see that curl returned the status code 60. According to man curl
60 Peer certificate cannot be authenticated with known CA certifi‐
cates.
According to the setuptools page, you can instead use python ez_setup.py --insecure but obviously do that at your own risk. Alternatively you could do what the advanced instructions say and manually download the setuptools tarball, verify its md5 hash yourself, and install it using its setup.py .
I'm trying to run a python script using python 2.6.4. The hosting company has 2.4 installed so I compiled my own 2.6.4 on a similar server and then moved the files over into ~/opt/python. that part seems to be working fine.
anyhow, when I run the script below, I am getting ImportError: No module named _sqlite3 and I'm not sure what to do to fix this.
Most online threads mention that sqlite / sqlite3 is included in python 2.6 - so I'm not sure why this isn't working.
-jailshell-3.2$ ./pyDropboxValues.py
Traceback (most recent call last):
File "./pyDropboxValues.py", line 21, in
import sqlite3
File "/home/myAccount/opt/lib/python2.6/sqlite3/__init__.py", line 24, in
from dbapi2 import *
File "/home/myAccount/opt/lib/python2.6/sqlite3/dbapi2.py", line 27, in
from _sqlite3 import *
ImportError: No module named _sqlite3
I think I have everything set up right as far as the directory structure.
-jailshell-3.2$ find `pwd` -type d
/home/myAccount/opt
/home/myAccount/opt/bin
/home/myAccount/opt/include
/home/myAccount/opt/include/python2.6
/home/myAccount/opt/lib
/home/myAccount/opt/lib/python2.6
/home/myAccount/opt/lib/python2.6/distutils
/home/myAccount/opt/lib/python2.6/distutils/command
/home/myAccount/opt/lib/python2.6/distutils/tests
/home/myAccount/opt/lib/python2.6/compiler
/home/myAccount/opt/lib/python2.6/test
/home/myAccount/opt/lib/python2.6/test/decimaltestdata
/home/myAccount/opt/lib/python2.6/config
/home/myAccount/opt/lib/python2.6/json
/home/myAccount/opt/lib/python2.6/json/tests
/home/myAccount/opt/lib/python2.6/email
/home/myAccount/opt/lib/python2.6/email/test
/home/myAccount/opt/lib/python2.6/email/test/data
/home/myAccount/opt/lib/python2.6/email/mime
/home/myAccount/opt/lib/python2.6/lib2to3
/home/myAccount/opt/lib/python2.6/lib2to3/pgen2
/home/myAccount/opt/lib/python2.6/lib2to3/fixes
/home/myAccount/opt/lib/python2.6/lib2to3/tests
/home/myAccount/opt/lib/python2.6/xml
/home/myAccount/opt/lib/python2.6/xml/parsers
/home/myAccount/opt/lib/python2.6/xml/sax
/home/myAccount/opt/lib/python2.6/xml/etree
/home/myAccount/opt/lib/python2.6/xml/dom
/home/myAccount/opt/lib/python2.6/site-packages
/home/myAccount/opt/lib/python2.6/logging
/home/myAccount/opt/lib/python2.6/lib-dynload
/home/myAccount/opt/lib/python2.6/sqlite3
/home/myAccount/opt/lib/python2.6/sqlite3/test
/home/myAccount/opt/lib/python2.6/encodings
/home/myAccount/opt/lib/python2.6/wsgiref
/home/myAccount/opt/lib/python2.6/multiprocessing
/home/myAccount/opt/lib/python2.6/multiprocessing/dummy
/home/myAccount/opt/lib/python2.6/curses
/home/myAccount/opt/lib/python2.6/bsddb
/home/myAccount/opt/lib/python2.6/bsddb/test
/home/myAccount/opt/lib/python2.6/idlelib
/home/myAccount/opt/lib/python2.6/idlelib/Icons
/home/myAccount/opt/lib/python2.6/tmp
/home/myAccount/opt/lib/python2.6/lib-old
/home/myAccount/opt/lib/python2.6/lib-tk
/home/myAccount/opt/lib/python2.6/hotshot
/home/myAccount/opt/lib/python2.6/plat-linux2
/home/myAccount/opt/lib/python2.6/ctypes
/home/myAccount/opt/lib/python2.6/ctypes/test
/home/myAccount/opt/lib/python2.6/ctypes/macholib
/home/myAccount/opt/share
/home/myAccount/opt/share/man
/home/myAccount/opt/share/man/man1
And finally the contents of the sqlite3 directory:
-jailshell-3.2$ find `pwd`
/home/myAccount/opt/lib/python2.6/sqlite3
/home/myAccount/opt/lib/python2.6/sqlite3/__init__.pyo
/home/myAccount/opt/lib/python2.6/sqlite3/dump.pyc
/home/myAccount/opt/lib/python2.6/sqlite3/__init__.pyc
/home/myAccount/opt/lib/python2.6/sqlite3/dbapi2.pyo
/home/myAccount/opt/lib/python2.6/sqlite3/dbapi2.pyc
/home/myAccount/opt/lib/python2.6/sqlite3/dbapi2.py
/home/myAccount/opt/lib/python2.6/sqlite3/dump.pyo
/home/myAccount/opt/lib/python2.6/sqlite3/__init__.py
/home/myAccount/opt/lib/python2.6/sqlite3/dump.py
I feel like I need to add something into the sqlite3 directory - maybe sqlite3.so? But I don't know where to get that.
What am I doing wrong here? Please remember that I'm using a shared host so that means installing / compiling on another server and then copying the files over. Thanks! :)
Update
Just wanted to confirm that the answer from #samplebias did work out very well. I needed to have the dev package installed on the machine I was compiling from to get it to add in sqlite3.so and related files. Also found the link in the answer very helpful. Thanks #samplebias !
Python's build system uses a setup.py file to compile all of the native extensions, including sqlite3. It searches common operating system paths for the sqlite3 include and library dirs. If the sqlite3 development package is not installed Python will skip compiling the _sqlite3.so extension, but the pure Python portion of the sqlite3 package will still be installed.
You would need to have the operating system's sqlite3 development package installed when you compile Python and at runtime: sqlite3-devel on Centos, both libsqlite3-0 and libsqlite3-dev on Ubuntu.
Here's an example of the _sqlite3.so extension linkage on my Ubuntu system:
% ldd /usr/lib/python2.6/lib-dynload/_sqlite3.so | grep sqlite3
libsqlite3.so.0 => /usr/lib/libsqlite3.so.0 (0x00007f29ef3be000)
% dpkg -S /usr/lib/libsqlite3.so.0
libsqlite3-0: /usr/lib/libsqlite3.so.0
In general, the first thing to do is to ask your host. I seems a bit odd that SQLite is not installed (or installed properly). So they'll likely fix it quite fast if you ask them.
For python 2.4, you need sqlite and bindings, pysqlite 2 or aspw
None of the files listed in the sqlite folder is the _sqlite3.pyd Python shared library. Are you sure you compiled it when compiling Python? What does the build log say? I think there's a configure flag that needs to be passed.
Alternatively, just install pysqlite