Short description: two computers in the same network, in the new one only those python scripts work that use native packages.
I have Pycharm in my old computer and it has worked fine. Now I got a new computer, installed the most recent version of Python and Pycharm, then opened one of my old projects. Both the old and the new computer are in the same network and the project is on a shared folder. So I did the following:
File - Open - selected the project. Got a message that there is no interpreter
Add local interpreter - selected the latest Python 311 exe. So location of the venv is the same as in the old computer (because it's a network folder) but Base interpreter is pointing to the C drive of my new computer.
PyCharm creates a virtual environment and the code runs fine.
I select another project which uses imported packages such as pandas. Again, same steps as above, add local interpreter. Venv is created.
I go to File - Setting - Project and see that pip, setuptools and wheel are listed as Packages. If I double click one of these, I can re-install and get a note that installation is succesful, so nothing seems to be wrong in the connection (after all, both the old and the new computer are in the same network.
I click the plus sign to add a new one, search pandas. Installation fails. Same thing if I try e.g. numpy.
Error message has lots of retrying, then "could not find the version that satisfies the requirement pandas (from versions: none", "not matching distribution found for pandas" (pip etc. have the latest versions).
After few hours of googling for solutions, I have tried the following:
Complety uninstall and reinstall python and PyCharm. Checked that PATH was included in the installation.
Tried launching pip command from shell
Changed http proxy to auto-detect
Typed 'import pandas' in PyCharm, then used the dropdown in the yellow bulb but there is no install option
Started a new project in the new computer, tried to install pandas
All failed. I'm surprised that changing computers is this difficult. Please let me know if there are other options than staying in the old computer...
If you want to use venv in the network, please use SSH interpreter. Pycharm supports this method. Shared folders are not a recommended usage, For pycharm, it will consider this as a local file. If the file map is not downloaded locally, it will make an error.
Another way is to reinstall the project environment on the new computer through requirement.txt. Reasonable use of requirements.txt can effectively avoid many project bugs caused by environment migration or different dependent versions. Before installing some scientific module such as pandas, it is recommended to install visual studio build tools, such as gcc ...
This took a while but here is what happened. Package installation did not work in project settings. Neither did it work when you select Python Packages tab at the bottom of the screen. The only thing that worked was to select the Terminal tab and manually install (pip install) there. We use a trusted repository but for other users, the easier package installation methods work. Not sure why they do not for me but at least there is this manual workaround.
Related
EDIT- Things I tried
*This question is the same, but the solutions there did not solve my problem. I didn't have a DLLs folder so I made one, tried both with and without deleting the file in site-packages/cv2/
*I checked this and this and couldn't make it work.
*I also tried to delete opencv-python from my base interpreter and only download it on the venv, but still didn't work.
*I also tried to invalidate cache/ restart multiple times.
*I also tried to delete python and pycharm completely. When I upgraded to 3.10 I couldn't even install opencv. When I tried 3.8 again the same problem accoured.
I wanted to download ML libraries to Pycharm (python 3.8). Initially I just added them (e.g. opencv-python) in the python interpreter itself (Settings > Project > Python interpreter) and it worked fine (So now my regular interpreter has a bunch of libraries on it).
Then I read it's better practice to make a new venv and download the libraries there. I created a new environment:
Then I entered the new interpreter and added opencv-python (plus sign > searched for this > Install package):
After that I made a short script that needs cv2. Pycharm couldn't import cv2, but the script runs correctly (displays the image) when "run" is pressed.
If I switch interpreter to the regular one it recognizes the cv2 methods and autocomplete me.
I solved it using anaconda.
The first time I tried it didn't work, so it's probably a problem with having non-ascii username (again). I had the .conda folder in a path which contained non-ascii, as well as the anaconda3 itself installed in a different place
To install it correctly:
Uninstall both anaconda, pycharm (uninstall other pythons if you have them).
2. Create a new user with ascii only (e.g. Ben).
In this user:
Install anaconda (I guess miniconda will work too)
Create a venv (I used this and this tutorials)
Install your packages using conda-forge like in the tutorials
I was trying to install a SQL-Python connector and kept receiving an error that it could not find the python installation. It said that it only accepted the version used by the installer from the python website so I downloaded that and tried to install it in the same path where my python was (C:\Python38) but then saw that it was installing a 32-bit version of python there so I cancelled the install mid-install (this was a mistake). I think I did that at the wrong time because now the folder still exists but it had deleted some parts of my python installation. So now I'm in a weird place where I can't install python (When I open the 64-bit installer it only gives me the option to Modify, Repair or Uninstall, and all of those shoot up errors where it says that it can't find the python installation). The 32-bit version does allow me to try and install but I get an error saying "The File 'python.dll' cannot be installed because the file cannot be found in cabinet file 'cab1.cab.'"
If I try and uninstall python from the Windows uninstaller I also get errors that there is no python version to install.
I tried just deleting the folder from my system but that doesn't work either. Not sure how to bring my computer back to a state where I can do a fresh python install. Is there any way?
What I did, and I am not sure that it was the cause of solving the problem, was to delete the 'python.exe' and 'python3.exe' files that where in 'C:\Users\Usuario\AppData\Local\Microsoft\WindowsApps'. As the user don't have access to delete them from the files explorer, I entered from cmd as administrator and deleted them with the 'del' command.
Again I am not sure if that was the cause but I think it worked for me while I was trying to delete all python related files to reinstall.
I'm trying to install OpenCV into my python environment (Windows), and I'm almost all of the way there, but still having some issues with autocomplete and Pycharm itself importing the library. I've been through countless other related threads, but it seems like most of them are either outdated, for prebuilt versions, or unanswered.
I'm using Anaconda and have several environments, and unfortunately installing it through pip install opencv-contrib-python doesn't include everything I need. So, I've built it from source, and the library itself seem to be working fine. The build process installed some things into ./Anaconda3/envs/cv/Lib/site-packages/cv2/: __init__.py, some config py files, and .../cv2/python-3.8/cv2.cp38-win_amd64.pyd. I'm not sure if it did anything else.
But here's where I'm at:
In a separate environment, a pip install opencv-contrib-python both runs and has autocomplete working
In this environment, OpenCV actually runs just fine, but the autocomplete doesn't work and Pycharm complains about everything, eg: Cannot find reference 'imread' in '__init__.py'
Invalidate Caches / Restart doesn't help
Removing and re-adding the environment doesn't help
Deleting the user preferences folder for Pycharm doesn't help
Rebuilding/Installing OpenCV doesn't help
File->Settings->Project->Project Interpreter is set correctly
Run->Edit Configuration->Python Interpreter is set correctly
So my question is: how does Pycharm get or generate that autocomplete information? It looks like the pyd file is just a dll in disguise, and looking through the other environment's site-packages/cv2 folder, I don't see anything interesting. I've read that __init__.py has something to do with it, but again the pip version doesn't contain anything (except there's a from .cv2 import *, but I'm not sure how that factors in). The .whl file you can download is a zip that only contains the same as what 'pip install' gets.
Where does the autocomplete information get stored? Maybe there's some way to copy it from one environment to another? It would get me almost all the way there, which at this point would be good enough I think. Maybe I need to rebuild it with another flag I missed?
Got it finally! Figures that would happen just after posting the question...
Turns out .../envs/cv/site-packages/cv2/python-3.8/cv2.cp38-win_amd64.pyd needed to be copied to .../envs/cv/DLLs/. Then PyCharm did it's magic and is now all good.
Alternatively add the directory containing the .pyd file to the interpreter paths.
I had exactly this problem with OpenCV 4.2.0 compiled from sources, installed in my Conda environment and PyCharm 2020.1.
I solved this way:
Select project interpreter
Click on the settings button next to it and then clicking on the Show paths for selected interpreter
adding the directory containing the cv2 library (in my case in the Conda Python library path - e.g. miniconda3/lib/python3.7/site-packages/cv2/python-3.7). In general check the site-packages/cv2/python-X.X directory)
What I was doing before
When I start using Anaconda in pycharm before, I installed Anaconda and after that I installed pycharm. Running the pycharm and in the following menu (following image) I did choose the ~/Anadaconda/python.exe path for the pycharm interpreter and I used this interpreter for all my project. I hadn't any problem.
Question 1: Is this procedure correct?
There is something called Virtualenv Environment in the picture. As far as I know, this makes a project and its dependencies isolated. I didn't fully understand what is the use of that though.
Question 2: What is the use of isolating the projects? Why should I do that? (an example would be helpful)
Of Course, there is a benefit and use of that, so 3rd question comes here.
There is a checkbox that says Make it available for all project Is still this option follows being isolated?
Real Problem
The problem starts a week ago when I want to install the new version of Anaconda 3.5.1. and I did. but I wish I didn't.
After doing that, for every package I import to my project, I get an error, it seems like, it doesn't recognize them at all.
for e.g. Numpy:
module = self._system_import(name, *args, **kwargs)
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
For sklearn:
ImportError:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try git clean -xdf (removes all
files not under version control). Otherwise reinstall numpy.
Original error was: DLL load failed: The specified module could not be found.
Tired reinstall both pycharm and anaconda for a few times. I'm getting crazy about this.
The procedure you use to setup the interpreter is correct. The use of isolating projects is that you don't get in trouble with package dependencies. Some projects could require different versions of, for instance the sk-learn package. Isolating projects by using environments and installing seperate packages for every project prevents any issues with this. "Make available to projects" has nothing to do with isolation of packages inside that environment.
The problem here is that you probably did not tick the option 'Add to path' while installing Anaconda. PyCharm does not automatically activate your environments when you do not add Anaconda to path. Using PyCharm with an unactivated environment is unsupported, and can lead to import-errors like the ones you specify. So to solve your error, use anaconda prompt, activate your environment and launch PyCharm from there, or re-install anaconda/mini-conda with the option "add-to-path" checked. A detailed discussion of this problem can be found here.
Yes, a virtual environment is exactly that.
Using a virtual environment is kind of a clean slate. You wouldnt want you to use a environment for a web crawler that has Django in it. And in your Django site you wouldnt want to have BeautifulSoup in it. This is the best way of managing environments within different projects. I would highly recommend setting up a virtual environment for each project you create.
Now when you are getting these errors. First do you have pip updated to the latest version? When installing numpy are you using a CLI or the using the pycharm wizard thingy?
Sometimes when you get errors installing you can use easy install instead of pip and it can resolve the issue.
The 3.6 installer suggests C:\Users\MyUserName\AppData\Local\Programs\Python\Python36-32 which is unlike any other software on Windows.
I remember that earlier versions installed to C:\PythonXY which is also unusual on Windows. Is any of that really a good idea?
In particular, I don't see why I would want to install this only for my user account. The checkbox "Install launcher for all users (recommended)" is default checked which seems incompatible with installing into %APPDATA%.
What is a good path to install Python to?
I'm a complete Python amateur and I don't want to cause myself problems. I am fearful of adding a space to the path for example.
Clicking further through the installer it turns out there is a checkbox to install for all users. This immediately sets a Program Files (x86) based path to the checkbox.
This is a very subjective question in most cases, I'm not really sure if its even fit for SO.
Here are the different characteristics of each path:
C:\PythonXY:
Requires administrator rights
All users have access to it - better if you want only one install of a python version on your system
More practical to write in command line (might be needed for multiple python installs, although using venvs or conda envs solves this problem)
Program Files is the same, except that there are spaces in the path (probably a bad idea)
AppData:
Doesn't require administrator rights
Only for one user - good if the other users don't want/need it
Might be painful to write in command line
I have admin rights and am the only user on my computer, so I chose the first option, but it really is case-dependent.
EDIT Please see the comments below for rectifications on paths.
Installing Python in c:\Program Files\ in 64-bit Windows 10 has problems because of the new "security features" of Windows 10. Subfolders of c:\Programs Files\ have restricted permissions on them which are not compatible with the installation processes for some Python packages. Installing in a directory directly below the root (for example c:\Python36\ for Python 3.6) avoids these problems. It may be "less secure", but it actually works, which installing under c:\Program Files\ sometimes does not. For example, I have found that if you try to install the matplotlib package in Python 3.7 installed under c:\Program Files\, some of the packages that matplotlib is dependent on are blocked from installing, and the package will not run, but if it is installed in c:\Python37\, it runs fine. Another example of overly enthusiastic Microsoft "security."
Installing in the %appdata% folder for a single user is fine if you are the only person who is going to be using Python, but if your computer is going to be used by another user (for example, if it is a work computer which will be "inherited" by a new employee that takes over your position), Python (and all necessary Python packages) would have to be reinstalled for that user.
I think the best path to install python is in C:\Users\<Username>\PythonXX (especially if you are using PC with multiple users and also if you are using your organization PC.
If you install to C:\PythonXX or to C:\Program Files\PythonXX, you will always have permissions issue. As soon as you try to install some packages/libraries using 'pip install ', permissions will not allow to install the packages.
If you leave the installation to be continued in C:\Users\<Username>\AppData\Local\Programs\Python\PythonXX as suggested by python installer, which also doesn't have any permissions issue, but path is not so easy to remember.
Few more important points:
If you are using a personal laptop and you are the only user (you will be administrator as well), then installing python to C:\PythonXX is also a best option.
Always check the option, add python path to Path variable during installation, so that you can also use python from command prompt/powershell.
Always check the option, install pip, so that you can use pip to install/update packages/libraries.
In my case because the Path variable was already set, I would only go to the python shell and type:
import os
os.path