I'm having trouble with using impyla library on windows
I installed impyla library
pip install impyla
Error occured when I tried to import impyla libary in python code
from impala.dbapi import connect # error occured
from impala.util import as_pandas
conn = connect(host='10.xx.xx.xx', database='xx_xx', port=21050)`
Traceback (most recent call last): ...
File "D:/test/test.py", line 14, in
from impala.dbapi import connect
File "C:\Anaconda3\lib\site-packages\impala\dbapi.py", line 28, in
import impala.hiveserver2 as hs2
File "C:\Anaconda3\lib\site-packages\impala\hiveserver2.py", line 32, in
from impala._thrift_api import (
File "C:\Anaconda3\lib\site-packages\impala_thrift_api.py", line 73, in
include_dirs=[thrift_dir])
File "C:\Anaconda3\lib\site-packages\thriftpy\parser__init__.py", line 30, in load
include_dir=include_dir)
File "C:\Anaconda3\lib\site-packages\thriftpy\parser\parser.py", line 496, in parse
url_scheme))
thriftpy.parser.exc.ThriftParserError: ThriftPy does not support generating module with path in protocol 'c'
when I tried to print include_dir, which was
D:/test\thrift
I just cannot import libray at all
help me
I had the same problem with thriftpy, the problem on windows is an absolute path is like C:\foo\bar.thrift
But, the way the thrift library parses the file, it detects the C: as if it were a protocol like http: or https:
Its pretty easy to workaround you just have to strip the first two characters from the path with a slice like path[2:]
Just slice when you call thriftpy.load or in the library file
File "C:\Anaconda3\lib\site-packages\thriftpy\parser__init__.py", line 30
path = "C:\foo\bar.thrift"
thrift.load(path[2:], module_name, include_dirs=include_dirs,
include_dir=include_dir)
OR
You can go a bit deeper and make the same change that I already submitted as a patch on the github page... perhaps it will be incorporated in the next version of thrift.
File "C:\Anaconda3\lib\site-packages\thriftpy\parser\parser.py", line 488
- if url_scheme == '':
+ if len(url_scheme) <= 1:
My justification of why this change is valid is in the pull request. If its incorporated then you wont have to worry about making the same change again when you update the library. If not then just strip the two characters again.
Update:
Thriftpy Version 1: the parser fix is now on line 547: elif len(url_scheme) <= 1:
Thriftpy Version 2: the fix has already been merged.
I was encountering the same error with impyla on an Anaconda Python 3.6 distribution on Windows. Instead of installing using pip, I was able to get it working using:
conda install -c anaconda impyla
https://anaconda.org/anaconda/impyla
Related
I have this very simple scapy program, which does a arp_ping to my subnet
from scapy.all import srp, Ether, ARP, conf
def arp_ping(subnet):
conf.verb = 1
answered, unanswered = srp(Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(pdst=subnet),timeout=2,verbose=False, inter=0.1)
return [rcv.sprintf(r"%Ether.src% - %ARP.psrc%") for snd, rcv in answered]
if __name__=="__main__":
subnet = '192.168.1.0/24'
for i in arp_ping(subnet):
print i
EDIT
After reinstalling atleast PyWin32 and WinPcap with choco I now have a this NameError:
Traceback (most recent call last):
File "arp_sender.py", line 1, in <module>
from scapy.all import srp, Ether, ARP, conf
File "C:\Python27\lib\site-packages\scapy\all.py", line 16, in <module>
from scapy.arch import *
File "C:\Python27\lib\site-packages\scapy\arch\__init__.py", line 83, in <module>
from scapy.arch.windows import *
File "C:\Python27\lib\site-packages\scapy\arch\windows\__init__.py", line 465, in <module>
conf.iface = get_working_if()
File "C:\Python27\lib\site-packages\scapy\arch\windows\__init__.py", line 463, in get_working_if
return LOOPBACK_NAME
NameError: global name 'LOOPBACK_NAME' is not defined
Checking for depencies issue
Running scapy.bat to check for depencies results into this message:
INFO: Can't load Python libreadline or completer
INFO: Can't import matplotlib. Won't be able to plot.
INFO: Can't import PyX. Won't be able to use psdump() or pdfdump().
WARNING: No match between your pcap and windows network interfaces found. You probably won't be able to send packets. Deactivating unneeded interfaces and restarting Scapy might help.Check your winpcap and powershell installation, and access rights.
INFO: Could not get readline console. Will not interpret ANSI color codes.
WARNING: No default IPv4 routes found. Your Windows release may no be supported and you have to enter your routes manually
Traceback (most recent call last):
File "C:\Python27\Scripts\\scapy", line 25, in <module>
interact()
File "C:\Python27\lib\site-packages\scapy\main.py", line 300, in interact
scapy_builtins = __import__("all",globals(),locals(),".").__dict__
File "C:\Python27\lib\site-packages\scapy\all.py", line 16, in <module>
from scapy.arch import *
File "C:\Python27\lib\site-packages\scapy\arch\__init__.py", line 83, in <module>
from scapy.arch.windows import *
File "C:\Python27\lib\site-packages\scapy\arch\windows\__init__.py", line 465, in <module>
conf.iface = get_working_if()
File "C:\Python27\lib\site-packages\scapy\arch\windows\__init__.py", line 463, in get_working_if
return LOOPBACK_NAME
NameError: global name 'LOOPBACK_NAME' is not defined
My guess is that:
WARNING: No match between your pcap and windows network interfaces found. You probably won't be able to send packets. Deactivating unneeded interfaces and restarting Scapy might help.Check your winpcap and powershell installation, and access rights.
is causing the issue, but I'm unsure how to resolve this.
You are using an old scapy version, update it via
pip install scapy --upgrade to get 2.4.0
I can see that because recent versions do not need libreadline anymore
In the latest version, the only dependency you need to install is:
Winpcap or Npcap
IPython (if asked)
You may have run your Python code as non-administrator.
I just encountered the same problem when I used the PyCharm IDE, which was not opened in the administrator mode (by default).
After I restarted the IDE as the administrator, the error message was gone.
Similarly, when you want to use the scapy library in Linux, you have to specify "sudo ..."
I am trying to use the python 3 bindings to VirtualBox but there appears to be broken dependencies. It seems odd to me that this hasn't been fixed over the ~4 years that people have been having this issue. Perhaps I'm missing something obvious. It's been known to happen.
I have installed the virtualbox host modules, sdk, and extensions through my OS's pacakage manager. Then, through pip:
pip install pyvbox
The imports work:
from virtualbox import VirtualBox, Session, Manager, WebServiceManager
But then any attempt to instantiate anything results in an exception complaining about a missing vboxapi.
box = VirtualBox()
Traceback:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/usr/lib/python3.6/site-packages/virtualbox/library_ext/vbox.py", line 22, in __init__
manager = virtualbox.Manager()
File "/usr/lib/python3.6/site-packages/virtualbox/__init__.py", line 130, in __init__
with import_vboxapi() as vboxapi:
File "/usr/lib/python3.6/contextlib.py", line 82, in __enter__
return next(self.gen)
File "/usr/lib/python3.6/site-packages/virtualbox/__init__.py", line 45, in import_vboxapi
import vboxapi
File "/home/$USER/.eclipse/org.eclipse.platform_4.6.3_155965261_linux_gtk_x86_64/plugins/org.python.pydev_5.7.0.201704111357/pysrc/_pydev_bundle/pydev_import_hook.py", line 20, in do_import
module = self._system_import(name, *args, **kwargs)
ModuleNotFoundError: No module named 'vboxapi'
There is a vboxapi on PyPi, but it won't install as there is no code associated with it, nor any useful information on the PyPi page:
https://pypi.python.org/pypi/vboxapi
Here are a couple links to the valiant efforts of braver souls than I. It is not immediately clear to me which is the correct solution or if either are still relevant, given that they are from 3 and 4 years ago, respectively.
https://github.com/GreatFruitOmsk/vboxapi-py3
https://github.com/jbuergel/vboxapi-py3
Also from 3 years ago, word of a vboxapi.diff and intergration into vboxapi:
https://www.virtualbox.org/pipermail/vbox-dev/2014-April/012231.html
I'm the current maintainer of the pyvbox package.
The VirtualBox SDK already supports Python 3, I use Python 3.5 to develop the library. I recommend uninstalling and reinstalling the latest version of the SDK (which at the time of writing this is 5.1.22).
You can find the SDK on the VirtualBox downloads page. Unzip the archive and run the vboxapisetup.py file using your system Python with the following command:
python vboxapisetup.py install
You don't need to install this in any virtualenv, as pyvbox will search your system libraries in addition to virtualenv installations for better ease of use.
If you have problems using the pyvbox package after running these steps, please open an issue and include as much information as possible including the steps you took, OS, where your system Python is located, which version of VirtualBox & SDK you're using, and I'll help you as best I can.
Yes you can, it is possible, very tricky to setup but it work fine for me now (Ubuntu 18.04 / python3.6 / virtualbox 6.0) .
The error:
ModuleNotFoundError: No module named 'vboxapi'
mean that python3 does not find vboxapi module, now there is two methods to "force-install" the vboxapi package to python3:
First Method [easy]: Assuming pyvbox is already installed and work fine with python2.7, in that case you can simply copy the package from python2.7 dist-packages to python3 dist-package with:
sudo cp -r /usr/lib/python2.7/dist-packages/vboxapi /usr/lib/python3/dist-packages
Second method [more tricky]: Go to VirtualBox, then download the last Software Developer Kit (SDK), actually the 6.0.4
Unzip the archive and run the vboxapisetup.py file using Python3 with the following command:
sudo python3 vboxapisetup.py install
You will get this issue:
Traceback (most recent call last):
File "vboxapisetup.py", line 90, in <module>
main(sys.argv)
File "vboxapisetup.py", line 63, in main
raise Exception("No VBOX_INSTALL_PATH defined, exiting")
Exception: No VBOX_INSTALL_PATH defined, exiting
You may directly edit the current file vboxapisetup.py and replace line 57, from vboxDest = os.environ.get("VBOX_MSI_INSTALL_PATH", None) to vboxDest = "/usr/lib/virtualbox"
Then run agin:
sudo python3 vboxapisetup.py install
And now you will get something like that:
running install
running build
running build_py
copying vboxapi/__init__.py -> build/lib/vboxapi
running install_lib
creating /usr/local/lib/python3.6/dist-packages/vboxapi
copying build/lib/vboxapi/__init__.py -> /usr/local/lib/python3.6/dist-packages/vboxapi
copying build/lib/vboxapi/VirtualBox_constants.py -> /usr/local/lib/python3.6/dist-packages/vboxapi
byte-compiling /usr/local/lib/python3.6/dist-packages/vboxapi/__init__.py to __init__.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/vboxapi/VirtualBox_constants.py to VirtualBox_constants.cpython-36.pyc
running install_egg_info
Removing /usr/local/lib/python3.6/dist-packages/vboxapi-1.0.egg-info
Writing /usr/local/lib/python3.6/dist-packages/vboxapi-1.0.egg-info
which mean that we are ok with vboxapi package installation !
Now, let's try again to load virtualbox() inside python3:
from virtualbox import VirtualBox, Session, Manager, WebServiceManager
box = VirtualBox()
this probably will raise this new issue:
Traceback (most recent call last):
File "virtualbox_python3_test.py", line XX, in <module>
vbox = virtualbox.VirtualBox()
File "/usr/local/lib/python3.6/dist-packages/virtualbox/library_ext/vbox.py", line 22, in __init__
manager = virtualbox.Manager()
File "/usr/local/lib/python3.6/dist-packages/virtualbox/__init__.py", line 143, in __init__
self.manager = vboxapi.VirtualBoxManager(mtype, mparams)
File "/usr/local/lib/python3.6/dist-packages/vboxapi/__init__.py", line 989, in __init__
self.platform = PlatformXPCOM(dPlatformParams)
File "/usr/local/lib/python3.6/dist-packages/vboxapi/__init__.py", line 750, in __init__
import xpcom.vboxxpcom
File "/usr/lib/virtualbox/sdk/bindings/xpcom/python/xpcom/vboxxpcom.py", line 78, in <module>
raise Exception('Cannot find VBoxPython module (tried: %s)' % (', '.join(_asVBoxPythons),))
Exception: Cannot find VBoxPython module (tried: VBoxPython3_6m, VBoxPython3m, VBoxPython)
If you dig you will find a lot of questions (question 1,question 2,question 3, question 4 etc...) relative to this issue on the web ...
But according to my dig & research, if you are lucky (and have a Virtualbox built with python3 native support) you can try:
cd /usr/lib/virtualbox/
sudo cp VBoxPython3_5m.so VBoxPython3_6m.so
But if you got the following error:
cp: cannot stat 'VBoxPython3_5m.so': No such file or directory
It mean that you don't have native python3 support in Virtualbox...
This could be solved like this:
Go here and download the python3-virtualbox-5.2.16 binary package (we don't care about the VirtualBox version...)
Now open python3-virtualbox-5.2.16-lp150.4.11.1.x86_64.rpm archive, browse it to /./usr/lib/virtualbox/, then extract the file VBoxPython3_6m.so, then drop this file in your current working directory, after that from this directory you have to do:
sudo cp VBoxPython3_6m.so /usr/lib/virtualbox/
And now, you can use python3 binding for virtualbox !
I have installed qpid-0.22 on sles11 sp2 X86_64, the broker works fine.
Then I installed qpid-python client and set the env variable.
PYTHONPATH=/home/zdx/qpid/qpid-0.22/python/:/usr/local/lib/python2.7:/usr/local/lib/python2.7/site-packages:/home/zdx/qpid/qpid-0.22/python
But the python client doesn't work, including qpid-config tool and qpid-python client test examples.
When I ran this kind of script, it showed following exception:
Traceback (most recent call last):
File "/usr/local/bin/qpid-config", line 31, in
from qpid.messaging import Connection
File "/usr/local/lib/python2.7/site-packages/qpid/init.py", line 20, in
import connection
File "/usr/local/lib/python2.7/site-packages/qpid/connection.py", line 20, in
import datatypes, session
File "/usr/local/lib/python2.7/site-packages/qpid/session.py", line 26, in
from ops import Command, MessageTransfer
ImportError: cannot import name MessageTransfer
It indicate that class or module MessageTransfer does not exist in ops module,
and I look into the python module ops.py, there is none class MessageTransfer.
what is the problem with it? thanks.
Even though you installed the command line tools properly, sometimes you will get this error.
This means that you need to install the python-qpid bindings and their libraries.
If you have epel repository in your /etc/yum.repos.d/ , You can directly install the package by using yum like this.
# yum search python-qpid
In search results, select package according to your Operating system (32-bit/64-bit).
And then install the package.
# yum install python-qpid..... (python-qpid-proton.x86_64, etc..)
If you don't have epel, first get epel into your /etc/yum.repos.d/ and then install the package
I will first state I have searched for this problem, and found the exact same problem here ( ImportError: No module named 'requests' ) but that hasn't helped me.
I am using macports on osx (mountain lion). I have successfully installed and run a few scripts without any issues.
from the macports page, I have installed requests via the method it detailed and as far as I can tell, it has installed successfully:
daves-mbp:~ Dave$ port search requests
arpwatch #2.1a15 (net)
Monitor ARP & RARP requests
http_ping #29jun2005 (net, www)
Sends HTTP requests every few seconds and times how long they take
httping #2.0 (net, www)
Ping-like tool for http-requests
py-requests #1.2.3 (python, devel)
Python HTTP for Humans.
py26-requests #1.2.3 (python, devel)
Python HTTP for Humans.
py27-requests #1.2.3 (python, devel)
Python HTTP for Humans.
py31-requests #1.2.3 (python, devel)
Python HTTP for Humans.
py32-requests #1.2.3 (python, devel)
Python HTTP for Humans.
py33-requests #1.2.3 (python, devel)
Python HTTP for Humans.
webredirect #0.3 (www)
small webserver which redirects all requests
Found 10 ports.
I have python 2.7, so I installed it via:
daves-mbp:~ Dave$ sudo port install py27-requests
Password:
---> Computing dependencies for py27-requests
---> Fetching archive for py27-requests
---> Attempting to fetch py27-requests-1.2.3_0.darwin_12.noarch.tbz2 from http://jog.id.packages.macports.org/macports/packages/py27-requests
---> Attempting to fetch py27-requests-1.2.3_0.darwin_12.noarch.tbz2.rmd160 from http://jog.id.packages.macports.org/macports/packages/py27-requests
---> Installing py27-requests #1.2.3_0
---> Activating py27-requests #1.2.3_0
---> Cleaning py27-requests
---> Updating database of binaries: 100.0%
---> Scanning binaries for linking errors: 100.0%
---> No broken files found.
daves-mbp:~ Dave$
I think that looks good. Using macports is there something else I have to do before using it? I thought the python setup.py install (in the aforementioned post) may have solved my problem, however, when I search for requests in my filesystem, the only reference is burried in a path (that macports says is a store for user installed modules. And besides, there is no setup.py within that or it's parent directory.
I have restarted my terminal window (that fixed another problem earlier), but it made no difference here.
Any help is appreciated
edit:
which python reports /opt/local/bin/python
the first lines of the python interpreter DID report:
daves-mbp:~ Dave$ python
Python 2.7.2 (default, Jun 20 2012, 16:23:33)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
but now I have done something and it's responding with new errors:
daves-mbp:~ Dave$ python
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 548, in <module>
main()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 530, in main
known_paths = addusersitepackages(known_paths)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 266, in addusersitepackages
user_site = getusersitepackages()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 241, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.py", line 231, in getuserbase
USER_BASE = get_config_var('userbase')
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sysconfig.py", line 516, in get_config_var
return get_config_vars().get(name)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sysconfig.py", line 449, in get_config_vars
import re
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/re.py", line 105, in <module>
import sre_compile
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sre_compile.py", line 14, in <module>
import sre_parse
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sre_parse.py", line 17, in <module>
from sre_constants import *
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sre_constants.py", line 18, in <module>
from _sre import MAXREPEAT
ImportError: cannot import name MAXREPEAT
In trying to sort this out, I have broken python, and eventually I got it going again.
I think initially I had not run one of the port select --set... commands. Once I realised this might be the case, I did so, but that produced the errors at the top. MAXREPEATS, a circular reference perhaps? No idea.
I have read here (macports didn't place python_select in /opt/local/bin) and here (How do I uninstall python from OSX Leopard so that I can use the MacPorts version?) about the --set command not working and to try sudo port select python python26 (i used python27) instead.
I checked the PATH and python didn't appear, so I updated that as well.
I got my python interpreter back and low-and-behold imports requests now works.
I think at the end of it all, there were two errors:
I used --set instead of the newer command, and
my path wasn't set
edit: Actually, after more debugging, I found the error was on the first line of my script, I had defined which python to use (which was the default apple one, which doesn't include the module). Once I updated the shebang line, it worked.
I've spent several hours attempting to resolve my problems setting up GeoIP in Django to no avail and was hoping to get some guidance on what the problem(s) might be.
I'm working on an existing Django application that required some geolocation abilities, specifically getting a users IP and lat/long and then placing that info on a map marker. GeoIP and the associated libraries appeared to be the best solution for the first step.
I installed GeoIP on a Mac using Homebrew. I then manually created a folder in the root directory of my project with the GeoIPv6.data and GeoLiteCity.dat files. After this, I added the path in my settings file:
import os
DEBUG = True
TEMPLATE_DEBUG = DEBUG
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
GEOIP_PATH = os.path.join(BASE_DIR, 'geoip'),
I then opened a command shell for the project and received the following error:
>>> from django.contrib.gis.geoip import GeoIP
Traceback (most recent call last):
File "<console>", line 1, in <module>
ImportError: No module named geoip
I can't seem to remedy this problem. One issue that may be the cause is extracting the two dat.gz files was an issue. Neither could be unzipped from the command line – neither are .zip files – and had to use Stuffit Expander to open these. The resulting dat files in my project IDE (pyCharm) have a VLC (?) icon on each. Perhaps this is part of the issue (finding a way to uncompress the file was a challenge in itself). I'm not sure as the module was not even found.
Any help would be extremely appreciated in resolving this issue as I can't progress any further without figuring out what's wrong.
Many thanks.
*Edit. Okay, it first appeared an imroper import statement might have been the problem. Fixed it: from django.contrib.gis.utils.geoip import GeoIP. Unfortunately, once fixed, the following error results:
>>> from django.contrib.gis.utils.geoip import GeoIP
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/contrib/gis/utils/geoip.py", line 68, in <module>
lgeoip = CDLL(lib_path)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 353, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/usr/local/lib/libGeoIP.dylib, 6): no suitable image found. Did find:
/usr/local/lib/libGeoIP.dylib: mach-o, but wrong architecture
This is perplexing. Not sure what the problem is exactly.
The arch mismatch message means that the libgeoIP library you installed was built with a different CPU architecture than the architecture that the Python interpreter is running in. From the paths involved, you appear to be using a newer (non-Apple-supplied) Python 2.7. Chances are that the libgeoIP was built as 64-bit (-arch x86_64) while the Python you installed is a 32-bit -only Python 2.7 (-arch i386). Or possibly the reverse. In either case, you could try reinstalling GeoIP with the universal option so that it contains both archs. Or you could try reinstalling Python 2.7 using a 64-bit/32-bit version such as downloadable from python.org. You can see for sure which archs are involved by using the file command:
$ file /usr/local/lib/libGeoIP.dylib
$ file $(python2.7 -c 'import sys;print(sys.executable)')
There needs to be at least one common architecture between the two.
The proper way to import GeoIP on Django 1.4 and up (including the current 1.9 release) is:
from django.contrib.gis.geoip import GeoIP
Its not an issue with your dat files. Its a problem with the import statement, and it finding the GeoIP module.
From the docs:
https://docs.djangoproject.com/en/1.4/ref/contrib/gis/geoip/
In Django 1.4, the GeoIP object was moved out of
django.contrib.gis.utils and into its own module,
django.contrib.gis.geoip. A shortcut is still provided in utils, but
will be removed in Django 1.6.
If you are using django 1.3.x, try:
from django.contrib.gis.utils import GeoIP
Note the difference in the import paths between these two versions:
https://docs.djangoproject.com/en/1.3/ref/contrib/gis/geoip/
https://docs.djangoproject.com/en/1.4/ref/contrib/gis/geoip/
Previously i was facing this issue : from django.contrib.gis.geoip import GeoIP ImportError: cannot import name GeoIP
Solution :
yum install GeoIP-devel -y