Install python CV2 on spark cluster(data bricks) - python

i want to install pythons library CV2 on a spark cluster using databricks community edition and i'm going to:
workspace-> create -> library , as the normal procedure and then selecting python in the Language combobox, but in the "PyPi Package" textbox , i tried "cv2" and "opencv" and had no luck. Does anybody has tried this? Do you know if cv2 can be installed on the cluster through this method? and if so, which name should be used in the texbox?

Try to download numpy first followed by opencv-python it will work.
Steps:
Navigate to Install Library-->Select PyPI----->In Package--->numpy
(after installation completes, proceed to step 2)
Navigate to Install Library-->Select PyPI----->In Package--->opencv-python

The PyPi package you want is https://pypi.python.org/pypi/opencv-python -- so just put opencv-python in the textbox and install.

Related

Error: Import cv cannot be resolved: How to install OpenCV correctly?

To keep a long story short. I copied this code from https://www.geeksforgeeks.org/detect-an-object-with-opencv-python/ (not really import but I still mentioned it)
import cv2
from matplotlib import pyplot as plt
# Opening image
img = cv2.imread("image.jpg")
# OpenCV opens images as BRG
# but we want it as RGB and
# we also need a grayscale
# version
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
# Creates the environment
# of the picture and shows it
plt.subplot(1, 1, 1)
plt.imshow(img_rgb)
plt.show()
Anyways this line below has this error that says "Import "cv2" could not be resolved"
import cv2
When I run this code below to download opencv. I get the message that the "Requirement already satisfied: "
pip install opencv-python
Despit me closing vscode the reopening I still get that message that "Import "cv2" could not be resolved" Ive updated pip, I've printed the version of cv2 in cmd which I get 4.5.5, I've deleted python and python intelligence. I just cant to figure out why opencv isn't working. Any suggestions? (I'm a newbie lol)
According to your description, I think there should be multiple versions of Python in your system.
You could use "ctrl+shift+P" to choose your python interpreter.
You can also use pip's command to install opencv into the current Python package floder.
pip install -t FloderPath opencv-python
Two reasons come to mind:
1.) Either the path to the opencv library is not set in the editor so it cannot find it. That happens sometimes if you changed settings in the runtime environment.
2.) Or you created a virtual environment prior to installing opencv. In that case, install opencv for your virtual environment as well.
It is difficult to tell what is going on with so little information. The above causes are just the most likely. Check your runtime environment and your library paths.
u need to do several things to clear it up, open cmd and follow this,
step 0: of course u must check python is added in ur system variable path,two path is essential, for me two dir is like this (add for ur installation, if have added this just skip this step)
C:\Python\Scripts\
C:\Python\
step 1: clear current package from python site package
pip uninstall opencv-python
step 2: clear cache from pip , for fresh install
pip cache dir
u will get a list of dir printed, now browse to that dir using explorer and delete everything in there.
step 3: check pip package install dir for ur python, it should be like "..\python\lib\site-packages" , to check this type into cmd
pip list -v
this will give u dir reference of all site packages, and u should check whether it is
"..\python\lib\site-packages" or not.
step 4: reinstall the opencv , u can install community contrib version of opencv, this is extended package of opencv-python with extra modules
pip install opencv-python
or
pip install opencv-contrib-python
step 5: type python in cmd, if python idle responded in cmd, then ur system found python, then type import cv2 and make sure it is imported. If it is imported successfully then u need to make sure ur vs code python plugin in up to date and configured well, for me i have added python path to system variable and didn't have to configure the plugin, it works well.
let me know if u have the problem unsolved.

No module named 'xlsxwriter' after installing module into virtual environment [duplicate]

I recently downloaded the xlsxwriter version 0.6.4 and installed it on my computer. It correctly added it to my C:\Python27\Lib\site-packages\xlsxwriter folder, however when I try to import it I get the error ImportError: No module named xlsxwriter. The traceback is File "F:\Working\ArcGIS\ArcGIS .py\Scripts\Append_Geodatabase.py".
However if I try to import numpy (I can't remember what numby is, however it is located in the same site-packages folder C:\Python27\Lib\site-packages\numpy) it has no problem.
Any idea of what could be causing this issue?
Thanks for the help.
Here are some easy way to get you up and running with the XlsxWriter module.The first step is to install the XlsxWriter module.The pip installer is the preferred method for installing Python modules from PyPI, the Python Package Index:
sudo pip install xlsxwriter
Note
Windows users can omit sudo at the start of the command.
Even if it looks like the module is installed, as far as Python is concerned it isn't since it throws that exception.
Try installing the module again using one of the installation methods shown in the XlsxWriter docs and look out for any installation errors.
If there are none then run a sample program like the following:
import xlsxwriter
workbook = xlsxwriter.Workbook('hello.xlsx')
worksheet = workbook.add_worksheet()
worksheet.write('A1', 'Hello world')
workbook.close()
I have the same issue. It seems that pip is the problem. Try
pip uninstall xlsxwriter
easy_install xlsxwriter
I managed to resolve this issue as follows...
Be careful, make sure you understand the IDE you're using! - Because I didn't.
I was trying to import xlsxwriter using PyCharm and was returning this error.
Assuming you have already attempted the pip installation (sudo pip install xlsxwriter) via your cmd prompt, try using another IDE e.g. Geany - & import xlsxwriter.
I tried this and Geany was importing the library fine. I opened PyCharm and navigated to 'File>Settings>Project:>Project Interpreter' xlslwriter was listed though intriguingly I couldn't import it! I double clicked xlsxwriter and hit 'install Package'... And thats it! It worked!
Hope this helps...
sudo pip install XlsxWriter
Make sure that X and W are in uppercase.
I am not sure what caused this but it went all well once I changed the path name from Lib into lib and I was finally able to make it work.
I installed it by using a wheel file that can be found at this location:
https://pypi.org/project/XlsxWriter/#files
I then ran pip install "XlsxWriter-1.2.8-py2.py3-none-any.whl"
Processing ./XlsxWriter-1.2.8-py2.py3-none-any.whl
Installing collected packages: XlsxWriter
Successfully installed XlsxWriter-1.2.8
in VSCode:
instead of activating your environment with script use python select interpreter
from VSCode(press ctrl + shift + p)
and then select your environment from the list (marked with recommended)
I found the same error when using xlsxwriter in my test.py application. First, check if you have xlsxwriter module installed or not.
sudo pip install xlsxwriter
Then check the python version you are using, The following worked for me
python2 test.py
Using this on Raspberry Pi 4. I had a similar issue. I followed the install step:
sudo pip install xlsxwriter
None of the IDEs could find the module. I had to use Add/Remove Software under preferences in the GUI, search for xlsxwriter, select one by clicking on it and make sure the box is checked, and then click apply, then select the other one (it showed two for me) and click apply for that one. After that, it worked fine.

How to install gdal on databricks cluster?

I am trying to install the package GDAL on an Azure Databricks cluster. In no way I can get it to work.
Approaches that I've tried but didn't work:
Via the library tab of the corresponding cluster --> Install New --> PyPi (under Library Source) --> Entered gdal under Package
Tried all approaches mentioned on https://forums.databricks.com/questions/13738/gdal-installation.html. None of them worked.
Details:
Runtime: 6.1 (includes Apache Spark 2.4.4, Scala 2.11) (When using runtime 3.5 I got GDAL to work, however an update to a higher runtime was necessary for other reasons.)
We're using python 3.7.
Finally we got it working by using an ML runtime in combination with the answer given in forums.databricks.com/answers/21118/view.html. Apparently the ML-runtimes contain conda, which is needed for the answer given in the previous link.
I have already replied similar type of question.
Please check the below link would help you to install the required library:
How can I download GeoMesa on Azure Databricks?
For your convenience I am pasting the Answer again... just you need to choose your required library from the search area.
You can install GDAL Library directly into your Databricks cluster.
1) Select the Libraries option then a new window will open.
2) Select the maven option and click on 'search packages' option
3) Search the required library and select the library/jar version and choose the 'select' option.
Thats it.
After the installation of the library/jar, restart your cluster. Now import the required classes in your Databricks notebook.
I hope it helps. Happy Coding..
pip install https://manthey.github.io/large_image_wheels/GDAL-3.1.0-cp38-cp38-manylinux2010_x86_64.whl
Looks like you are able to use this whl file and install the package but when running tasks like GDAL.Translate it will not actually run. This is the farthest I've gotten.
The above URL was found when I was searching for the binaries that GDAL needs. As a note you will have to run this every time you start your cluster.

'ImportError: No module named pillow' in PyCharm

I'm getting an error while using PyCharm which doesn't allow me to import the pillow module even though I have it installed as a package in the project interpreter. Any help is greatly appreciated!
http://imgur.com/a/DfjC3
While the name of the package is pillow, it is a replacement for PIL and uses the PIL for the name of the base module
the usual way to use pillow is
from PIL import Image
im = Image.open("filename")
See the tutorial, and the documentation
You try to run code with default Python interpreter (/Library/Frameworks/Python.framework/Versions/3.2/bin/python3). You need to configure PyCharm to run Your code with anaconda (~/anaconda/bin/python)
And now (Like #JamesK say) read Pillow tutorial and documentation:
import PIL not import Pillow
For anybody still having trouble with this, I did the following which solved my problem.
Open up your Project Interpreter (⌘ + , on Mac).
At the bottom of this page you'll see the + symbol to the left of the anaconda logo. This will create a pop-up that allows you to search for available packages.
In this new window, search for 'Pillow'.
Click and Install Package.
You should now be able to use "from PIL import Image" or "import Pillow as pil" etc.
After running this command on your terminal
pip install pillow
and you are sure it was installed, but still having same problem of PIL module not found.
Go to your IDE and make sure existing interpreter is set to python interpreter and not anaconda

How do I install modules on qpython3 (Android port of python)

I found this great module on within and downloaded it as a zip file. Once I extracted the zip file, i put the two modules inside the file(setup and the main one) on the module folder including an extra read me file I needed to run. I tried installing the setup file but I couldn't install it because the console couldn't find it. So I did some research and I tried using pip to install it as well, but that didn't work. So I was wondering if any of you could give me the steps to install it manually and with pip (keep in mind that the setup.py file needs to be installed in order for the main module to work).
Thanks!
The cleanest and simplest way I have found is to use pip from within QPython console as in This Answer
import pip
pip.main(['install', 'networkx'])
Step1: Install QPython.
Step2: Install AIPY for QPython.
Step3: Then go to QPython-->QPYPI-->AIPY and install from there Numpy, SciPy, Matplotlib, openCV etc.
Extract the zip file to the site-packages folder.
Find the qpyplus folder in that Lib/python3.2/site-packages extract here that's it.Now you can directly use your module from REPL terminal by importing it.

Categories