I am going completely mental as I fail to understand and keep my python environment running.
So :
I have a python environment running under Miniconda 3 and that I manage through an environment.yaml file
Environment is used to develop within VSCode
It installs a bunch of external package through conda and then installs my local development package in editable mode (simplified version of the environment.yaml)
name: DLL_ETL
channels:
- conda-forge
- msys2
- defaults
dependencies:
- pandas
- numpy
- pip:
- -e .
- datatable
I update it regularly through "mamba env update"
However, since today this is what I see :
Pip states that the package install is sucessful
In the environment, when running Python from the command line, I can import my package
Package is installed pointing to the correct location (a one-drive)
When opening up an interactive window in VSCode with that environment as kernel : package cannot be found
What am I doing wrong?
What I tried :
Deleting the environment and reinstalling from scratch
Deleting the pycache and dll.egg-info
Manually removing and installing again through pip (and not through mamba, even if it runs pip underneath).
conda list shows the package as properly installed
As can be understood :
Yes I am running a virtual environment
Yes I am in the correct directory (especially when updating through environment.yaml)
Even when manually installing
Yes the directory with my local install is in my Python path and python is running
I managed to find the solution, but no idea why. I found a comment somewhere (I believe : pip: module installed as editable not found) that there is an issue in how some files are created when installing as editable.
More specifically, when installing as editable, two files are created :
___editable___dll_etl_1_0_0_finder.py
editable.dll_etl-1.0.0.pth
both in the environment
Miniconda3\envs\DLL_ETL\Lib\site-packages
When comparing two environments I have, I noticed that the .pth created in the one that broke was not formatted similarly, the .py didn't even exist. More specifically
The original file
What it should look like
When creating the files manually, everything seems to work fine. I haven't tried to update since. Likely it's an issue with setuptools if I understand correctly.
Restart the jupyter kernel.
Add the path to the package above the import statement.
import sys
// path is the directory where the package is located, not the full path of the package
sys.path.append("path/to/package")
import dll_etl
I'm having trouble with using 'requests' module on my Mac. I use python34 and I installed 'requests' module via pip. I can verify this via running installation again and it'll show me that module is already installed.
15:49:29|mymac [~]:pip install requests
Requirement already satisfied (use --upgrade to upgrade): requests in /opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages
Although I can import 'requests' module via interactive Python interpreter, trying to execute 'import requests' in PyCharm yields error 'No module named requests'. I checked my PyCharm Python interpreter settings and (I believe) it's set to same python34 as used in my environment. However, I can't see 'requests' module listed in PyCharm either.
It's obvious that I'm missing something here. Can you guys advise where should I look or what should I fix in order to get this module working? I was living under impression that when I install module via pip in my environment, PyCharm will detect these changes. However, it seems something is broken on my side ...
In my case, using a pre-existing virtualenv did not work in the editor - all modules were marked as unresolved reference (running naturally works, as this is outside of the editor's config, just running an external process (not so easy for debugging)).
Turns out PyCharm did not add the site-packages directory... the fix is to manually add it.
On Pycharm professional 2022.3
Open File -> Settings -> Python Interpreter, open the drop-down and pick "Show All..." (to edit the config) (1), right click your interpreter (2), click "Show Interpreter Paths" (3).
In that screen, manually add the "site-packages" directory of the virtual environment [looks like .../venv/lib/python3.8/site-packages (4) (I've added the "Lib" also, for a good measure); once done and saved, they will turn up in the interpreter paths.
The other thing that won't hurt to do is select "Associate this virtual environment with the current project", in the interpreter's edit box.
If you are using PyCharms CE (Community Edition), then click on:
File->Default Settings->Project Interpreter
See the + sign at the bottom, click on it. It will open another dialog with a host of modules available. Select your package (e.g. requests) and PyCharm will do the rest.
This issue arises when the package you're using was installed outside of the environment (Anaconda or virtualenv, for example). In order to have PyCharm recognize packages installed outside of your particular environment, execute the following steps:
Go to
Preferences -> Project -> Project Interpreter -> 3 dots -> Show All ->
Select relevant interpreter -> click on tree icon Show paths for the selected interpreter
Now check what paths are available and add the path that points to the package installation directory outside of your environment to the interpreter paths.
To find a package location use:
$ pip show gym
Name: gym
Version: 0.13.0
Summary: The OpenAI Gym: A toolkit for developing and comparing your reinforcement learning agents.
Home-page: https://github.com/openai/gym
Author: OpenAI
Author-email: gym#openai.com
License: UNKNOWN
Location: /usr/local/lib/python3.7/site-packages
...
Add the path specified under Location to the interpreter paths, here
/usr/local/lib/python3.7/site-packages
Then, let indexing finish and perhaps additionally reopen your project.
Open python console of your pyCharm. Click on Rerun.
It will say something like following on the very first line
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 /Applications/PyCharm.app/Contents/helpers/pydev/pydevconsole.py 52631 52632
in this scenario pyCharm is using following interpretor
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
Now fire up console and run following command
sudo /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 -m pip install <name of the package>
This should install your package :)
Pycharm is unable to recognize installed local modules, since python interpreter selected is wrong. It should be the one, where your pip packages are installed i.e. virtual environment.
I had installed packages via pip in Windows. In Pycharm, they were neither detected nor any other Python interpreter was being shown (only python 3.6 is installed on my system).
I restarted the IDE. Now I was able to see python interpreter created in my virtual environment. Select that python interpreter and all your packages will be shown and detected. Enjoy!
Using dual python 2.7 and 3.4 with 2.7 as default, I've always used pip3 to install modules for the 3.4 interpreter, and pip to install modules for the 2.7 interpreter.
Try this:
pip3 install requests
This is because you have not selected two options while creating your project:-
** inherit global site packages
** make available to all projects
Now you need to create a new project and don't forget to tick these two options while selecting project interpreter.
The solution is easy (PyCharm 2021.2.3 Community Edition).
I'm on Windows but the user interface should be the same.
In the project tree, open External libraries > Python interpreter > venv > pyvenv.cfg.
Then change:
include-system-site-packages = false
to:
include-system-site-packages = true
Before going further, I want to point out how to configure a Python interpreter in PyCharm: [SO]: How to install Python using the "embeddable zip file" (#CristiFati's answer). Although the question is for Win, and has some particularities, configuring PyCharm is generic enough and should apply to any situation (with minor changes).
There are multiple possible reasons for this behavior.
1. Python instance mismatch
Happens when there are multiple Python instances (installed, VEnvs, Conda, custom built, ...) on a machine. Users think they're using one particular instance (with a set of properties (installed packages)), but in fact they are using another (with different properties), hence the confusion. It's harder to figure out things when the 2 instances have the same version (and somehow similar locations)
Happens mostly due to environmental configuration (whichever path comes 1st in ${PATH}, aliases (on Nix), ...)
It's not PyCharm specific (meaning that it's more generic, also happens outside it), but a typical PyCharm related example is different console interpreter and project interpreter, leading to confusion
The fix is to specify full paths (and pay attention to them) when using tools like Python, PIP, .... Check [SO]: How to install a package for a specific Python version on Windows 10? (#CristiFati's answer) for more details
This is precisely the reason why this question exists. There are 2 Python versions involved:
Project interpreter: /Library/Frameworks/Python.framework/Versions/3.4
Interpreter having the Requests module: /opt/local/Library/Frameworks/Python.framework/Versions/3.4
well, assuming the 2 paths are not somehow related (SymLinked), but in latest OSX versions that I had the chance to check (Catalina, Big Sur, Monterey) this doesn't happen (by default)
When dealing with this kind of error, it always helps (most likely) displaying the following information (in a script or interpreter console):
import os
import sys
print(sys.executable)
print(sys.version)
print(os.getcwd())
print(getattr(os, "uname", lambda: None)())
print(sys.path)
2. Python's module search mechanism misunderstanding
According to [Python.Docs]: Modules - The Module Search Path:
When a module named spam is imported, the interpreter first searches for a built-in module with that name. These module names are listed in sys.builtin_module_names. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
The directory containing the input script (or the current directory when no file is specified).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
The installation-dependent default (by convention including a site-packages directory, handled by the site module).
A module might be located in the current dir, or its path might be added to ${PYTHONPATH}. That could trick users into making them believe that the module is actually installed in the current Python instance ('s site-packages). But, when running the current Python instance from a different dir (or with different ${PYTHONPATH}) the module would be missing, yielding lots of headaches
For a fix, check [SO]: How PyCharm imports differently than system command prompt (Windows) (#CristiFati's answer)
3. A PyCharm bug
Not very likely, but it could happen. An example (not related to this question): [SO]: PyCharm 2019.2 not showing Traceback on Exception (#CristiFati's answer)
To fix, follow one of the options from the above URL
4. A glitch
Not likely, but mentioning anyway. Due to some cause (e.g.: HW / SW failure), the system ended up in an inconsistent state, yielding all kinds of strange behaviors
Possible fixes:
Restart PyCharm
Restart the machine
Recreate the project (remove the .idea dir from the project)
Reset PyCharm settings: from menu select File -> Manage IDE Settings -> Restore Default Settings.... Check [JetBrains]: Configuring PyCharm settings or [JetBrains.IntelliJ-Support]: Changing IDE default directories used for config, plugins, and caches storage for more details
Reinstall PyCharm
Needless to say that the last 2 options should only be attempted as a last resort, and only by experts, as they might mess up other projects and not even fix the problem
Not directly related to the question, but posting:
[SO]: Run / Debug a Django application's UnitTests from the mouse right click context menu in PyCharm Community Edition? (a PyCharm related investigation from a while ago)
[SO]: ImportError: No module named win32com.client (#CristiFati's answer)
If you go to pycharm project interpreter -> clicked on one of the installed packages then hover -> you will see where pycharm is installing the packages. This is where you are supposed to have your package installed.
Now if you did sudo -H pip3 install <package>
pip3 installs it to different directory which is /usr/local/lib/site-packages
since it is different directory from what pycharm knows hence your package is not showing in pycharm.
Solution: just install the package using pycharm by going to File->Settings->Project->Project Interpreter -> click on (+) and search the package you want to install and just click ok.
-> you will be prompted package successfully installed and you will see it pycharm.
If any one faces the same problem that he/she installs the python packages but the PyCharm IDE doesn't shows these packages then following the following steps:
Go to the project in the left side of the PyCharm IDE then
Click on the venv library then
Open the pyvenv.cfg file in any editor then
Change this piece of code (include-system-site-packages = flase) from false to true
Then save it and close it and also close then pycharm then
Open PyCharm again and your problem is solved.
Thanks
This did my head in as well, and turns out, the only thing I needed to do is RESTART Pycharm. Sometimes after you've installed the pip, you can't load it into your project, even if the pip shows as installed in your Settings. Bummer.
For Anaconda:
Start Anaconda Navigator -> Enviroments -> "Your_Enviroment" -> Update Index -> Restart IDE.
Solved it for me.
After pip installing everything I needed. I went to the interpreter and re-pointed it back to where it was at already.
My case: python3.6 in /anaconda3/bin/python using virtualenv...
Additionally, before I hit the plus "+" sign to install a new package. I had to deselect the conda icon to the right of it. Seems like it would be the opposite, but only then did it recognize the packages I had/needed via query.
In my case the packages were installed via setup.py + easy_install, and the they ends up in *.egg directories in site_package dir, which can be recognized by python but not pycharm.
I removed them all then reinstalled with pip install and it works after that, luckily the project I was working on came up with a requirements.txt file, so the command for it was:
pip install -r ./requirement.txt
I just ran into this issue in a brand new install/project, but I'm using the Python plugin for IntelliJ IDEA. It's essentially the same as PyCharm but the project settings are a little different. For me, the project was pointing to the right Python virtual environment but not even built-in modules were being recognized.
It turns out the SDK classpath was empty. I added paths for venv/lib/python3.8 and venv/lib/python3.8/site-packages and the issue was resolved. File->Project Structure and under Platform Settings, click SDKs, select your Python SDK, and make sure the class paths are there.
pip install --user discord
above command solves my problem, just use the "--user" flag
I fixed my particular issue by installing directly to the interpreter. Go to settings and hit the "+" below the in-use interpreter then search for the package and install. I believe I'm having the issue in the first place because I didn't set up with my interpreter correctly with my venv (not exactly sure, but this fixed it).
I was having issues with djangorestframework-simplejwt because it was the first package I hadn't installed to this interpreter from previous projects before starting the current one, but should work for any other package that isn't showing as imported. To reiterate though I think this is a workaround that doesn't solve the setup issue causing this.
If you are having issues with the underlying (i.e. pycharm's languge server) mark everything as root and create a new project. See details: https://stackoverflow.com/a/73418320/1601580 this seems to happy to me only when I install packages as in editable mode with pip (i.e. pip install -e . or conda develop). Details: https://stackoverflow.com/a/73418320/1601580
--WINDOWS--
if using Pycharm GUI package installer works fine for installing packages for your virtual environment but you cannot do the same in the terminal,
this is because you did not setup virtual env in your terminal, instead, your terminal uses Power Shell which doesn't use your virtual env
there should be (venv) before you're command line as shown instead of (PS)
if you have (PS), this means your terminal is using Power Shell instead of cmd
to fix this, click on the down arrow and select the command prompt
select command prompt
now you will get (venv) and just type pip install #package name# and the package will be added to your virtual environment
On windows I had to cd into the venv folder and then cd into the scripts folder, then pip install module started to work
cd venv
cd scripts
pip install module
instead of running pip install in the terminal -> local use terminal -> command prompt
see below image
pycharm_command_prompt_image
In your pycharm terminal run pip/pip3 install package_name
I have installed python via conda, created a new environment, installed PyCharm and configured it to use this environment.
When I start a python console in PyCharm, I am greeted with a
C:\Users\9973879\AppData\Local\Continuum\miniconda3\envs\foo\lib\site-packages\IPython\core\history.py:226: UserWarning: IPython History requires SQLite, your history will not be saved
Indeed if I try to import sqlite3, I get a
ImportError: DLL load failed: The specified module could not be found.
However Project Interpreters > Interpreter Paths does list C:\Users\9973879\AppData\Local\Continuum\miniconda3\envs\foo\{DLLs, Lib, site-packages}. Also, importing this package from a python interpreter launched from the command line — in that very same virtual environment — succeeds.
EDIT
It seems the problem comes from the PATH environment variable. When calling conda activate foo, conda adds a bunch of folders to the PATH, containing various libraries (in particular sqlite3.dll).
When pointing to an existing environment (or creating a new one), PyCharm does automatically add folders to the PYTHONPATH as pointed out before, but does not seem to add anything to the PATH.
I tested this assumption by running PyCharm from the command line after calling conda activate foo, and it works. Of course, having to launch PyCharm from a specific conda environment is not great as I am stuck with this specific environment.
How could PyCharm be configured to automatically update the environment as conda activate does? Shouldn't it do it straight out of the box?
I fixed the issue by adding DLL to my < path_to_environment >/DLLs location
The issue seems to be solved by upgrading to the latest 4.6.7 version of conda. The relevant line from the release notes might be
revert conda prepending to PATH in cli main file on windows (#8307)
I am not sure why it affected PyCharm and not plain python in my case.
I am using Anaconda 4.3.1 64 Bit on Windows 10 64 Bit. I have successfully installed Tensorflow (CPU) according to this. The test program runs in the command line, but not in PyCharm. The problem is that it works only with activate tensorflow.
I followed the steps from Pycharm anaconda import tensor flow library issue ("You need to do these following steps:"). It seems now that tensorflow is loaded, but I get a new error:
ImportError:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try git clean -xdf (removes all
files not under version control). Otherwise reinstall numpy.
How can I use TensorFlow in PyCharm? (BTW: Tensorflow works with PyCharm on my Fedora VM.)
Edit:
Can I use TensorFlow with Jupyter? When running "Jupyter (tensorflow)" from the Anaconda menu, a console window opens and immediately closes.
When PyCharm is open:
if you navigate to preferences: Project: Project Interpreter. You can there either create your own virtualenv (and then manually install the required packages) or make sure you have selected the anaconda python interpreter for your project.
If following official instructions, you have created a virtual environment called tensorflow. The environment is located in Anaconda3\envs\tensorflow directory, where Anaconda3 is the Anaconda installation directory. You just need to point PyCharm to the python.exe which is located there.
The settings path in PyCharm is something like this:
Settings->Project Interpreter->Add Local->Virtualenv Environment->Existing environment
Point the interpreter to the python.exe in the Anaconda3\envs\tensorflow directory.
I recommend installing babun and creating a virtualenv with virtualenvwrapper there and install on that environment.
After doing this, just choose the python binary in the relevant directory of the virtualenv you created, i.e. ~/.virtualenvs/myenv/bin/python or ~/.virtaulenvs/myenv/usr/bin/python
Before importing anything else, do this
import sys
print(sys.path)
import os
print(os.environ)
print(os.environ['CUDA_VISIBLE_DEVICES'])
from the command line (when TF works) and from PyCharm (when it doesn't). If you see any relevant differences, adjust accordingly (define the environment variables in PyCharm, etc.)
I'm trying to import NumPy on PyCharm.
Using the PyCharm terminal and Miniconda I've launched the command:
conda install numpy
And this was the output:
Fetching package metadata: ....
Solving package specifications: ....................
# All requested packages already installed.
# packages in environment at C:\Users\...\Miniconda3:
#
numpy 1.10.4 py35_0
So I run my project but the terminal said:
ImportError: No module named 'numpy'
On my project bar I can see two different folders, the one with my project and another one with the external libraries.
Under External libraries > Extendend definitions there is a NumPy folder so I guess that the installation goes well.
Go to
ctrl-alt-s
click "project:projet name"
click project interperter
double click pip
search numpy from the top bar
click on numpy
click install package button
if it doesnt work this can help you:
https://www.jetbrains.com/help/pycharm/installing-uninstalling-and-upgrading-packages.html
In PyCharm go to
File → Settings, or use Ctrl + Alt + S
< project name > → Project Interpreter → gear symbol → Add Local
navigate to C:\Miniconda3\envs\my_env\python.exe, where my_env is the environment you want to use
Alternatively, in step 3 use C:\Miniconda3\python.exe if you did not create any further environments (if you never invoked conda create -n my_env python=3).
You can get a list of your current environments with conda info -e and switch to one of them using activate my_env.
It seems that each project may have a separate collection of python libraries in a project specific computing environment. To get this working with numpy I went to the terminal at the bottom of the pycharm window and ran pip install numpy and once the process finished running the install and indexing my python project was able to import numpy from the line of code import numpy as np. It seems you may need to do this for each project you setup in numpy.
I have encountered problem installing numpy package to pycharm and finally figured out. I hope it would be helpful for someone having the same problem in installing numpy and other packages on pycharm.
Pycharm Setting :
Go to File => Setting => Project => Project Interpreter. On this window select the appropriate project interpreter. After this, a list of packages under the selected project interpreter will be shown.
From the list select pip and check if the version column and the latest version column are the same. If different upgrade the version to the latest version by selecting the pip and using the upward triangle sign on the right side of the lists.
Once the upgrading completed successfully, you can now add new packages from the plus sign.
I hope this would be clear and useful for someone.
Another option is to open the terminal at the pycharm & install it with pip
sudo pip install numpy
I added Anaconda3/Library/Bin to the environment path and PyCharm no longer complained with the error.
Stated by https://intellij-support.jetbrains.com/hc/en-us/community/posts/360001194720/comments/360000341500
In general, the cause of the problem could be the following:
You started a new project with the new virtual environment. So probably you install numpy from the terminal, but it is not in your venv. So
either install it from PyCahrm Interface: Settings -> Project Interpreter -> Add the package
or activate your venv and -> pip install numPy