I'm trying to install OpenCV into my python environment (Windows), and I'm almost all of the way there, but still having some issues with autocomplete and Pycharm itself importing the library. I've been through countless other related threads, but it seems like most of them are either outdated, for prebuilt versions, or unanswered.
I'm using Anaconda and have several environments, and unfortunately installing it through pip install opencv-contrib-python doesn't include everything I need. So, I've built it from source, and the library itself seem to be working fine. The build process installed some things into ./Anaconda3/envs/cv/Lib/site-packages/cv2/: __init__.py, some config py files, and .../cv2/python-3.8/cv2.cp38-win_amd64.pyd. I'm not sure if it did anything else.
But here's where I'm at:
In a separate environment, a pip install opencv-contrib-python both runs and has autocomplete working
In this environment, OpenCV actually runs just fine, but the autocomplete doesn't work and Pycharm complains about everything, eg: Cannot find reference 'imread' in '__init__.py'
Invalidate Caches / Restart doesn't help
Removing and re-adding the environment doesn't help
Deleting the user preferences folder for Pycharm doesn't help
Rebuilding/Installing OpenCV doesn't help
File->Settings->Project->Project Interpreter is set correctly
Run->Edit Configuration->Python Interpreter is set correctly
So my question is: how does Pycharm get or generate that autocomplete information? It looks like the pyd file is just a dll in disguise, and looking through the other environment's site-packages/cv2 folder, I don't see anything interesting. I've read that __init__.py has something to do with it, but again the pip version doesn't contain anything (except there's a from .cv2 import *, but I'm not sure how that factors in). The .whl file you can download is a zip that only contains the same as what 'pip install' gets.
Where does the autocomplete information get stored? Maybe there's some way to copy it from one environment to another? It would get me almost all the way there, which at this point would be good enough I think. Maybe I need to rebuild it with another flag I missed?
Got it finally! Figures that would happen just after posting the question...
Turns out .../envs/cv/site-packages/cv2/python-3.8/cv2.cp38-win_amd64.pyd needed to be copied to .../envs/cv/DLLs/. Then PyCharm did it's magic and is now all good.
Alternatively add the directory containing the .pyd file to the interpreter paths.
I had exactly this problem with OpenCV 4.2.0 compiled from sources, installed in my Conda environment and PyCharm 2020.1.
I solved this way:
Select project interpreter
Click on the settings button next to it and then clicking on the Show paths for selected interpreter
adding the directory containing the cv2 library (in my case in the Conda Python library path - e.g. miniconda3/lib/python3.7/site-packages/cv2/python-3.7). In general check the site-packages/cv2/python-X.X directory)
Related
Short description: two computers in the same network, in the new one only those python scripts work that use native packages.
I have Pycharm in my old computer and it has worked fine. Now I got a new computer, installed the most recent version of Python and Pycharm, then opened one of my old projects. Both the old and the new computer are in the same network and the project is on a shared folder. So I did the following:
File - Open - selected the project. Got a message that there is no interpreter
Add local interpreter - selected the latest Python 311 exe. So location of the venv is the same as in the old computer (because it's a network folder) but Base interpreter is pointing to the C drive of my new computer.
PyCharm creates a virtual environment and the code runs fine.
I select another project which uses imported packages such as pandas. Again, same steps as above, add local interpreter. Venv is created.
I go to File - Setting - Project and see that pip, setuptools and wheel are listed as Packages. If I double click one of these, I can re-install and get a note that installation is succesful, so nothing seems to be wrong in the connection (after all, both the old and the new computer are in the same network.
I click the plus sign to add a new one, search pandas. Installation fails. Same thing if I try e.g. numpy.
Error message has lots of retrying, then "could not find the version that satisfies the requirement pandas (from versions: none", "not matching distribution found for pandas" (pip etc. have the latest versions).
After few hours of googling for solutions, I have tried the following:
Complety uninstall and reinstall python and PyCharm. Checked that PATH was included in the installation.
Tried launching pip command from shell
Changed http proxy to auto-detect
Typed 'import pandas' in PyCharm, then used the dropdown in the yellow bulb but there is no install option
Started a new project in the new computer, tried to install pandas
All failed. I'm surprised that changing computers is this difficult. Please let me know if there are other options than staying in the old computer...
If you want to use venv in the network, please use SSH interpreter. Pycharm supports this method. Shared folders are not a recommended usage, For pycharm, it will consider this as a local file. If the file map is not downloaded locally, it will make an error.
Another way is to reinstall the project environment on the new computer through requirement.txt. Reasonable use of requirements.txt can effectively avoid many project bugs caused by environment migration or different dependent versions. Before installing some scientific module such as pandas, it is recommended to install visual studio build tools, such as gcc ...
This took a while but here is what happened. Package installation did not work in project settings. Neither did it work when you select Python Packages tab at the bottom of the screen. The only thing that worked was to select the Terminal tab and manually install (pip install) there. We use a trusted repository but for other users, the easier package installation methods work. Not sure why they do not for me but at least there is this manual workaround.
EDIT- Things I tried
*This question is the same, but the solutions there did not solve my problem. I didn't have a DLLs folder so I made one, tried both with and without deleting the file in site-packages/cv2/
*I checked this and this and couldn't make it work.
*I also tried to delete opencv-python from my base interpreter and only download it on the venv, but still didn't work.
*I also tried to invalidate cache/ restart multiple times.
*I also tried to delete python and pycharm completely. When I upgraded to 3.10 I couldn't even install opencv. When I tried 3.8 again the same problem accoured.
I wanted to download ML libraries to Pycharm (python 3.8). Initially I just added them (e.g. opencv-python) in the python interpreter itself (Settings > Project > Python interpreter) and it worked fine (So now my regular interpreter has a bunch of libraries on it).
Then I read it's better practice to make a new venv and download the libraries there. I created a new environment:
Then I entered the new interpreter and added opencv-python (plus sign > searched for this > Install package):
After that I made a short script that needs cv2. Pycharm couldn't import cv2, but the script runs correctly (displays the image) when "run" is pressed.
If I switch interpreter to the regular one it recognizes the cv2 methods and autocomplete me.
I solved it using anaconda.
The first time I tried it didn't work, so it's probably a problem with having non-ascii username (again). I had the .conda folder in a path which contained non-ascii, as well as the anaconda3 itself installed in a different place
To install it correctly:
Uninstall both anaconda, pycharm (uninstall other pythons if you have them).
2. Create a new user with ascii only (e.g. Ben).
In this user:
Install anaconda (I guess miniconda will work too)
Create a venv (I used this and this tutorials)
Install your packages using conda-forge like in the tutorials
What I was doing before
When I start using Anaconda in pycharm before, I installed Anaconda and after that I installed pycharm. Running the pycharm and in the following menu (following image) I did choose the ~/Anadaconda/python.exe path for the pycharm interpreter and I used this interpreter for all my project. I hadn't any problem.
Question 1: Is this procedure correct?
There is something called Virtualenv Environment in the picture. As far as I know, this makes a project and its dependencies isolated. I didn't fully understand what is the use of that though.
Question 2: What is the use of isolating the projects? Why should I do that? (an example would be helpful)
Of Course, there is a benefit and use of that, so 3rd question comes here.
There is a checkbox that says Make it available for all project Is still this option follows being isolated?
Real Problem
The problem starts a week ago when I want to install the new version of Anaconda 3.5.1. and I did. but I wish I didn't.
After doing that, for every package I import to my project, I get an error, it seems like, it doesn't recognize them at all.
for e.g. Numpy:
module = self._system_import(name, *args, **kwargs)
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
For sklearn:
ImportError:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try git clean -xdf (removes all
files not under version control). Otherwise reinstall numpy.
Original error was: DLL load failed: The specified module could not be found.
Tired reinstall both pycharm and anaconda for a few times. I'm getting crazy about this.
The procedure you use to setup the interpreter is correct. The use of isolating projects is that you don't get in trouble with package dependencies. Some projects could require different versions of, for instance the sk-learn package. Isolating projects by using environments and installing seperate packages for every project prevents any issues with this. "Make available to projects" has nothing to do with isolation of packages inside that environment.
The problem here is that you probably did not tick the option 'Add to path' while installing Anaconda. PyCharm does not automatically activate your environments when you do not add Anaconda to path. Using PyCharm with an unactivated environment is unsupported, and can lead to import-errors like the ones you specify. So to solve your error, use anaconda prompt, activate your environment and launch PyCharm from there, or re-install anaconda/mini-conda with the option "add-to-path" checked. A detailed discussion of this problem can be found here.
Yes, a virtual environment is exactly that.
Using a virtual environment is kind of a clean slate. You wouldnt want you to use a environment for a web crawler that has Django in it. And in your Django site you wouldnt want to have BeautifulSoup in it. This is the best way of managing environments within different projects. I would highly recommend setting up a virtual environment for each project you create.
Now when you are getting these errors. First do you have pip updated to the latest version? When installing numpy are you using a CLI or the using the pycharm wizard thingy?
Sometimes when you get errors installing you can use easy install instead of pip and it can resolve the issue.
In short, my question is, how do I install the latest version of scikit-image into my usr/lib/python3/dist-packages so I can actually use it? I think there is a problem with my understanding of how third-party modules are installed. As a newb, I don’t know how to rectify that, hence this post.
I need help to understand how to install packages in python3 up until now I have used pip/pip3/apt-get/synaptic etc and it has worked fine for many packages. However, I have hit several barriers (Skimage, opencv, plantcv in python3). I must emphasise, the problem I am having is using these packages in python3, not 2.7.
For example, I want to use the latest version of scikit-image (0.14) with python3. (http://scikit-image.org/) I have tried using the installation instructions and have not yet successfully managed to install it. I have navigated to my usr/lib/python3/dist-packages and copied scikit-image into this directory (I have all the dependencies installed in here already).
Image of my folder for dist-packages as proof
As you can see, the folder containing skimage is in the directory I want to be installed in, how do I actually install it? Do I have to extract skimage out of the folder into the directory and then run the install command? If I navigate to usr/lib/python3/dist-packages/scikit-image and then run pip install -e . I get an error, stating that I need numpy. If I write a python script using python3 I can clearly see I have it installed (and I have been using it for a long time). So, there must be a problem in how I have this package in my file system. I think a janky workaround would be to copy all the modules into my working directory and Import them that way as if they were modules I have made myself, but this obviously negates the whole point of installing packages.
This has also happened with another package called plantcv. Where I went into the directory usr/lib/python3/dist-packages then cloned the source from git hub and installed as per instructions. When I import plantcv in my python3 script. It Imports fine. But, there is nothing in it, as python cannot see the modules which are inside this folder at usr/lib/python3/dist-packages/plantcv/plantcv.
There is clearly some comprehension here that I am missing, as I have a similar problem for two packages now. Please, Internet. Help me understand what I am missing!
You simply need to copy the folder in /usr/lib/python3/dist-packages/package-name
However, there are certain things that are specific to python packages. The folder named package name should be a valid package. A good indicator of that is it will contain a file "__init__.py". It is very likely that every sub-directory inside this package directory will contain a "__init__.py" file. It depends on whether there are modules inside these sub-directories.
In your code simply import the package like the following.
import package-name
where package-name can be skimage
I'm working on a script that will plot data onto a map using the Basemap library. I'm trying to import Shapely as well for use in this same script. I'm working with Anaconda2 for Python2.7 in a Windows 7 environment. I used conda install to download the tar.bz2 file from the Conda Packages site (using Windows command line) and it looked like it all installed correctly.
When I open Pycharm and look at my accessible site-packages, I can see this package. However, when I try to use it within my script, I get an error saying that the package does not exist. I ran the script through the debugger to see if it would shed any more light, but I got the same error. Here's a screenshot of my available site packages when I go to Settings-->Project Interpreter from within Pycharm.
Screenshot of PyCharm site packages available
Is there something special I need to do in order to access this package from within a fresh Python file? I was trying to say "import shapely" or "import osx-64-shapely", but both give me the same "package does not exist" error message. I've been able to successfully use other third-party libraries within Python, so I'm not quite sure what the error is here....
I'm new to SO - if you need more details or there's some piece of info I didn't include, please let me know. Thank you for your help!
EDIT: I am NOT asking what the difference is between conda and pip, or how to use pip within PyCharm. I have used both successfully before to install third-party libraries. What I am asking is what might cause a third-party library that appears to have installed successfully from the command line become inaccessible from within PyCharm when I attempt to import it.
I'm not familiar with Shapely but I was astonished to see the name osx-64-shapely as a site-package for your python installation which is in windows 7. Are you sure you downloaded the right file? :)