We Keep Coding

Use anaconda in pycharm (Import libraries error, updating anaconda and virtual environment)

What I was doing before
When I start using Anaconda in pycharm before, I installed Anaconda and after that I installed pycharm. Running the pycharm and in the following menu (following image) I did choose the ~/Anadaconda/python.exe path for the pycharm interpreter and I used this interpreter for all my project. I hadn't any problem.
Question 1: Is this procedure correct?
There is something called Virtualenv Environment in the picture. As far as I know, this makes a project and its dependencies isolated. I didn't fully understand what is the use of that though.
Question 2: What is the use of isolating the projects? Why should I do that? (an example would be helpful)
Of Course, there is a benefit and use of that, so 3rd question comes here.
There is a checkbox that says Make it available for all project Is still this option follows being isolated?
Real Problem
The problem starts a week ago when I want to install the new version of Anaconda 3.5.1. and I did. but I wish I didn't.
After doing that, for every package I import to my project, I get an error, it seems like, it doesn't recognize them at all.
for e.g. Numpy:
module = self._system_import(name, *args, **kwargs)
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
For sklearn:
ImportError:
Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
If you're working with a numpy git repo, try git clean -xdf (removes all
files not under version control). Otherwise reinstall numpy.
Original error was: DLL load failed: The specified module could not be found.
Tired reinstall both pycharm and anaconda for a few times. I'm getting crazy about this.

The procedure you use to setup the interpreter is correct. The use of isolating projects is that you don't get in trouble with package dependencies. Some projects could require different versions of, for instance the sk-learn package. Isolating projects by using environments and installing seperate packages for every project prevents any issues with this. "Make available to projects" has nothing to do with isolation of packages inside that environment.
The problem here is that you probably did not tick the option 'Add to path' while installing Anaconda. PyCharm does not automatically activate your environments when you do not add Anaconda to path. Using PyCharm with an unactivated environment is unsupported, and can lead to import-errors like the ones you specify. So to solve your error, use anaconda prompt, activate your environment and launch PyCharm from there, or re-install anaconda/mini-conda with the option "add-to-path" checked. A detailed discussion of this problem can be found here.

Yes, a virtual environment is exactly that.
Using a virtual environment is kind of a clean slate. You wouldnt want you to use a environment for a web crawler that has Django in it. And in your Django site you wouldnt want to have BeautifulSoup in it. This is the best way of managing environments within different projects. I would highly recommend setting up a virtual environment for each project you create.
Now when you are getting these errors. First do you have pip updated to the latest version? When installing numpy are you using a CLI or the using the pycharm wizard thingy?
Sometimes when you get errors installing you can use easy install instead of pip and it can resolve the issue.

Installing third party modules in python3 - Ubuntu

In short, my question is, how do I install the latest version of scikit-image into my usr/lib/python3/dist-packages so I can actually use it? I think there is a problem with my understanding of how third-party modules are installed. As a newb, I don’t know how to rectify that, hence this post.
I need help to understand how to install packages in python3 up until now I have used pip/pip3/apt-get/synaptic etc and it has worked fine for many packages. However, I have hit several barriers (Skimage, opencv, plantcv in python3). I must emphasise, the problem I am having is using these packages in python3, not 2.7.
For example, I want to use the latest version of scikit-image (0.14) with python3. (http://scikit-image.org/) I have tried using the installation instructions and have not yet successfully managed to install it. I have navigated to my usr/lib/python3/dist-packages and copied scikit-image into this directory (I have all the dependencies installed in here already).
Image of my folder for dist-packages as proof
As you can see, the folder containing skimage is in the directory I want to be installed in, how do I actually install it? Do I have to extract skimage out of the folder into the directory and then run the install command? If I navigate to usr/lib/python3/dist-packages/scikit-image and then run pip install -e . I get an error, stating that I need numpy. If I write a python script using python3 I can clearly see I have it installed (and I have been using it for a long time). So, there must be a problem in how I have this package in my file system. I think a janky workaround would be to copy all the modules into my working directory and Import them that way as if they were modules I have made myself, but this obviously negates the whole point of installing packages.
This has also happened with another package called plantcv. Where I went into the directory usr/lib/python3/dist-packages then cloned the source from git hub and installed as per instructions. When I import plantcv in my python3 script. It Imports fine. But, there is nothing in it, as python cannot see the modules which are inside this folder at usr/lib/python3/dist-packages/plantcv/plantcv.
There is clearly some comprehension here that I am missing, as I have a similar problem for two packages now. Please, Internet. Help me understand what I am missing!

You simply need to copy the folder in /usr/lib/python3/dist-packages/package-name
However, there are certain things that are specific to python packages. The folder named package name should be a valid package. A good indicator of that is it will contain a file "__init__.py". It is very likely that every sub-directory inside this package directory will contain a "__init__.py" file. It depends on whether there are modules inside these sub-directories.
In your code simply import the package like the following.
import package-name
where package-name can be skimage

Importing libraries

I have a general question using third party libraries, but I will exemplify it on two examples to make it clearer and more "answerable":
I want to use pyfmi in Python. Trying to install it through pip tells me:
"Exception: FMI Library cannot be found. Please specify its location, either using the flag to the setup script '--fmil-home' or specify it using the environment variable FMIL_HOME."
I figured out that I had to download the tar.gz from jmodelica.org and extract the files, create a build directory, use cmake, make and make install commands. All runs through without a hitch. But trying to install through pip gives me the same error message. So my question is:
How does one do this? Do they mean by setup script the setup.py file? How can I access that one if I am installing through pip?
An which one is the fmi home directory? Is it the untarred file in my Downloads-Folder or one of the files in it:
builddir
Config.cmake
install
src
Test
ThirdParty
CMakeLists.txt
FMILIB_Acknowledgements.txt
FMILIB_License.txt
FMILIB_Readme.txt
LICENSE.md
README.md
? What is that flag and where to put it "exactly".
Thanks a lot.

PyFMI requires (as noted on the PyPI site) that FMI Library is installed prior to trying to install PyFMI from source.
During the installation (invoking python setup.py install) of PyFMI, the environment variable "FMIL_HOME" is checked to see if that points to an installation of FMI Library, if so, this will be used during the installation. So in your case, you need to set this environment variable. The other option is to install manually (using python setup.py install --fmil-home="/path/to/fmil") where the added path should point to FMI Library.
The third option is to see if there are binary installers for your platform (these include FMIL). Check PyPI, Anaconda and Christoph Gohlke's site.

how install caffe for all system users?

I installed caffe with anaconda in my home directory and it works.
Now I want that also the other system users can use caffe and run it using the python wrapper.
Thus, I first installed anaconda in /opt/anaconda and added this folder in the PATH variable in /etc/bash.bashrc (in a similar way I did with my .bashrc file). Then I copied my caffe directory to /opt/caffe and added the proper path in the PYTHONPATH variable in /etc/bash.bashrc.
As result, if another user runs python he correctly uses the anaconda version. Moreover he can run import caffe, but when it does so, an error is obtained because the python wrapper doesn t find the protobuf library.
I though that all the requirements have been installed for all the users and not just for me.
How can I correctly permit other users to run caffe using the copy in /opt/caffe?
Thanks in advance

This is an answer for those who've compiled Caffe up to make distribute and went for extra style points with make pycaffe. I am proud of you. Now you would like to let all users of your Debian-based OS import caffe from Python successfully (e.g. avoiding the cannot find libcaffe.so errors upon import). I respect that.
This GH Issue has the answer. make sure that you adapt the ABSOLUTE caffe path to your particular installation:
echo <YOUR_CAFFE_ABSPATH>/distribute/lib > /etc/ld.so.conf.d/caffe.conf
ldconfig
And for the Python wrapper, the package also has to be added to PYTHONPATH. You can do this by extending said path, or by copying the package into an already existing path, in my case as follows:
cp -r <YOUR_CAFFE_ABSPATH>/distribute/python/caffe /usr/lib/python3/dist-packages
That's it! Now import caffe should work at any location.
Disclaimers:
I've tested it on a GPU-compatible Ubuntu18 Docker image (this one nvcr.io/nvidia/cuda:11.1-cudnn8-devel-ubuntu18.04) as root
I'm answering here because it is one of the top-ranked Google results for "install caffe for all users" and the GH issue isn't, so hopefully this helps someone!
I'd also be happy to hear from other solutions or remarks to make our Caffe life easier.
Cheers,
Andres

How do I change the kernel/python version for iPython?

I have installed iPython using pip in OS X 10.10, and it gave me the "ipython" and "ipython2" commands, which run great, but which use OS X's default python version 2.7.9. I downloaded and installed the latest release of Python3.4 and can load it with the command "python3," but cannot find a way to get iPython to use this version of python. The iPython Web site states the package can be used with python versions 3.3 and above, but I cannot find any instruction on how to change the default python version used.
So far I have found that the jupyter package for iPython has a kernel specification in /usr/local/share/jupyter/kernels/, which is just a folder called "python2" containing a json file that points to the system's python 2.7.6, but altering this to point to the new python3.4 installation does not work. My guess is this configuration is for the ipython notebook.
I've also tried the approach here: ipython reads wrong python version
In doing so I've duplicated the ipython2 command in /user/local/bin/ and edited it to use the python3 interpreter located at /Library/Frameworks/Python.framework/Versions/3.4/bin/python3, however, this gives me an error "ImportError: No module named 'IPython'," which suggests the python3 installation does not have ipython installed.
To tackle this, I've tried uninstalling ipython and reinstalling it using pip, but it just targets the system's Python 2.7 installation and does nothing for python3.
Does anyone know how to configure iPython to use a different python version, or even install a separate ipython installation for python3? Ultimately it would be nice to quickly switch back and forth depending on my needs.

I just found the answer. In essence, this stems from not understanding the python installation layout and how resources are separated between installed interpreters. It appears each python version will have its own repository of tools, and the current "pip" command I had installed on the system was mapped for use with python 2.7, so all libraries, tools, and other details it managed where available only to python 2.7. This included iPython.
I thought that installing python3 would make all these libraries available to the new interpreter by default, but it appears the system keeps them all separate (which makes sense). The main issue here was continuing to use "pip" which targeted the old installation, instead of the new "pip3" command included with python3. By using pip3 to reinstall iPython, I was able to get iPython3 installed properly.
Unfortunately this setup means needing to re-download other libraries, but that's not too difficult.
It might seem like a trivial issue in hindsight, but this had me completely stuck. I hope this helps someone else in this situation.