We Keep Coding

Tensorflow installation issues

System information
OS Platform and Distribution - Windows 10
TensorFlow version: latest
Python version: 3.6.4
Installed using virtualenv? pip? conda?: - virtualenv
Greetings,
I hope this is the correct place to submit an inquiry of this nature, if it is not, please forgive my confusion & please point me in the right direction. I greatly appreciate your time & consideration.
I am new to Python & Tensorflow. I've done some coding with C in the past, mostly when I was in college. I am determined to learn Python & to utilize both Python & Tensorflow for AI & Machine Learning purposes.
I've had difficulties in getting Tensorflow to install properly. I started by installing the latest version of Python which didn't seem to like my attempts at installing Tensorflow, I then went with Python 3.6.4-amd64. I installed that, created a fresh directory for my environments, then installed pip & virtual env, then created a virtual environment to setup with Tensorflow.
One of the confusing issues I keep encountering is that when I install pip & virtualenv, and eventually Tensorflow, it keeps sending it by default to C:\user\username\appdata\roaming\python etc, my question is, how do I prevent it from doing that? I am trying to install in the direct being utilized in the command prop, I call up the fresh directory I created for my virtual environment, then activate the virtual environment, and no matter what I do it keeps sending all new install files into the appdata/roaming user directory sub folders.
This is causing the incredibly annoying issue of making it impossible for me to proceed with utilizing Tensorflow because I get nothing but errors on missing files, path directory etc etc. I even tried manually moving some of the files over to the virtual environment directory and that worked in some cases, but did not solve the overall problem.
Okay, now that I've made it painfully apparent how much of an uneducated newbie I am with all of this, may someone please give me some advice. The first step is admitting you need help, and I clearly do as I've spent several hours with my eyes glued to various articles and tutorials that have left me with more questions than answers. I truly appreciate any help you're willing to provide. Just a loner trying to figure this all out & increase my knowledge along the way. Thanks for your time,

Just moving the folder is not enough.
Once you have moved it, you must replace the original with a symbolic link to the new location. This will make windows think the data is still located on your C drive, while it actually is on your D drive.
Do note, this does work with AppData, but not with Program Files nor with the Windows folder, as it will break things like Windows Update.
To create the Directory Junction (Symbolic Link) do the following:
Open a cmd window with administrative privileges.
Navigate to c:\Users\username\appdata
execute the following command: mklink /d local d:\appdata\local
replace d:\appdata\local with the actual path of where you moved the appdata to.
If you cannot move/delete the original copy, create a 2nd user, make it administrator, login with it, and retry the option. This should ensure that no files are in use.
With the above issue fixed, follow the tensorflow installation steps in Anaconda provided here.
Hope this answers your question. Happy Learning.

OpenCV built from source: Pycharm doesn't get autocomplete information

I'm trying to install OpenCV into my python environment (Windows), and I'm almost all of the way there, but still having some issues with autocomplete and Pycharm itself importing the library. I've been through countless other related threads, but it seems like most of them are either outdated, for prebuilt versions, or unanswered.
I'm using Anaconda and have several environments, and unfortunately installing it through pip install opencv-contrib-python doesn't include everything I need. So, I've built it from source, and the library itself seem to be working fine. The build process installed some things into ./Anaconda3/envs/cv/Lib/site-packages/cv2/: __init__.py, some config py files, and .../cv2/python-3.8/cv2.cp38-win_amd64.pyd. I'm not sure if it did anything else.
But here's where I'm at:
In a separate environment, a pip install opencv-contrib-python both runs and has autocomplete working
In this environment, OpenCV actually runs just fine, but the autocomplete doesn't work and Pycharm complains about everything, eg: Cannot find reference 'imread' in '__init__.py'
Invalidate Caches / Restart doesn't help
Removing and re-adding the environment doesn't help
Deleting the user preferences folder for Pycharm doesn't help
Rebuilding/Installing OpenCV doesn't help
File->Settings->Project->Project Interpreter is set correctly
Run->Edit Configuration->Python Interpreter is set correctly
So my question is: how does Pycharm get or generate that autocomplete information? It looks like the pyd file is just a dll in disguise, and looking through the other environment's site-packages/cv2 folder, I don't see anything interesting. I've read that __init__.py has something to do with it, but again the pip version doesn't contain anything (except there's a from .cv2 import *, but I'm not sure how that factors in). The .whl file you can download is a zip that only contains the same as what 'pip install' gets.
Where does the autocomplete information get stored? Maybe there's some way to copy it from one environment to another? It would get me almost all the way there, which at this point would be good enough I think. Maybe I need to rebuild it with another flag I missed?

Got it finally! Figures that would happen just after posting the question...
Turns out .../envs/cv/site-packages/cv2/python-3.8/cv2.cp38-win_amd64.pyd needed to be copied to .../envs/cv/DLLs/. Then PyCharm did it's magic and is now all good.

Alternatively add the directory containing the .pyd file to the interpreter paths.
I had exactly this problem with OpenCV 4.2.0 compiled from sources, installed in my Conda environment and PyCharm 2020.1.
I solved this way:
Select project interpreter
Click on the settings button next to it and then clicking on the Show paths for selected interpreter
adding the directory containing the cv2 library (in my case in the Conda Python library path - e.g. miniconda3/lib/python3.7/site-packages/cv2/python-3.7). In general check the site-packages/cv2/python-X.X directory)

Installing third party modules in python3 - Ubuntu

In short, my question is, how do I install the latest version of scikit-image into my usr/lib/python3/dist-packages so I can actually use it? I think there is a problem with my understanding of how third-party modules are installed. As a newb, I don’t know how to rectify that, hence this post.
I need help to understand how to install packages in python3 up until now I have used pip/pip3/apt-get/synaptic etc and it has worked fine for many packages. However, I have hit several barriers (Skimage, opencv, plantcv in python3). I must emphasise, the problem I am having is using these packages in python3, not 2.7.
For example, I want to use the latest version of scikit-image (0.14) with python3. (http://scikit-image.org/) I have tried using the installation instructions and have not yet successfully managed to install it. I have navigated to my usr/lib/python3/dist-packages and copied scikit-image into this directory (I have all the dependencies installed in here already).
Image of my folder for dist-packages as proof
As you can see, the folder containing skimage is in the directory I want to be installed in, how do I actually install it? Do I have to extract skimage out of the folder into the directory and then run the install command? If I navigate to usr/lib/python3/dist-packages/scikit-image and then run pip install -e . I get an error, stating that I need numpy. If I write a python script using python3 I can clearly see I have it installed (and I have been using it for a long time). So, there must be a problem in how I have this package in my file system. I think a janky workaround would be to copy all the modules into my working directory and Import them that way as if they were modules I have made myself, but this obviously negates the whole point of installing packages.
This has also happened with another package called plantcv. Where I went into the directory usr/lib/python3/dist-packages then cloned the source from git hub and installed as per instructions. When I import plantcv in my python3 script. It Imports fine. But, there is nothing in it, as python cannot see the modules which are inside this folder at usr/lib/python3/dist-packages/plantcv/plantcv.
There is clearly some comprehension here that I am missing, as I have a similar problem for two packages now. Please, Internet. Help me understand what I am missing!

You simply need to copy the folder in /usr/lib/python3/dist-packages/package-name
However, there are certain things that are specific to python packages. The folder named package name should be a valid package. A good indicator of that is it will contain a file "__init__.py". It is very likely that every sub-directory inside this package directory will contain a "__init__.py" file. It depends on whether there are modules inside these sub-directories.
In your code simply import the package like the following.
import package-name
where package-name can be skimage

What path to install Python 3.6 to on Windows?

The 3.6 installer suggests C:\Users\MyUserName\AppData\Local\Programs\Python\Python36-32 which is unlike any other software on Windows.
I remember that earlier versions installed to C:\PythonXY which is also unusual on Windows. Is any of that really a good idea?
In particular, I don't see why I would want to install this only for my user account. The checkbox "Install launcher for all users (recommended)" is default checked which seems incompatible with installing into %APPDATA%.
What is a good path to install Python to?
I'm a complete Python amateur and I don't want to cause myself problems. I am fearful of adding a space to the path for example.
Clicking further through the installer it turns out there is a checkbox to install for all users. This immediately sets a Program Files (x86) based path to the checkbox.

This is a very subjective question in most cases, I'm not really sure if its even fit for SO.
Here are the different characteristics of each path:
C:\PythonXY:
Requires administrator rights
All users have access to it - better if you want only one install of a python version on your system
More practical to write in command line (might be needed for multiple python installs, although using venvs or conda envs solves this problem)
Program Files is the same, except that there are spaces in the path (probably a bad idea)
AppData:
Doesn't require administrator rights
Only for one user - good if the other users don't want/need it
Might be painful to write in command line
I have admin rights and am the only user on my computer, so I chose the first option, but it really is case-dependent.
EDIT Please see the comments below for rectifications on paths.

Installing Python in c:\Program Files\ in 64-bit Windows 10 has problems because of the new "security features" of Windows 10. Subfolders of c:\Programs Files\ have restricted permissions on them which are not compatible with the installation processes for some Python packages. Installing in a directory directly below the root (for example c:\Python36\ for Python 3.6) avoids these problems. It may be "less secure", but it actually works, which installing under c:\Program Files\ sometimes does not. For example, I have found that if you try to install the matplotlib package in Python 3.7 installed under c:\Program Files\, some of the packages that matplotlib is dependent on are blocked from installing, and the package will not run, but if it is installed in c:\Python37\, it runs fine. Another example of overly enthusiastic Microsoft "security."
Installing in the %appdata% folder for a single user is fine if you are the only person who is going to be using Python, but if your computer is going to be used by another user (for example, if it is a work computer which will be "inherited" by a new employee that takes over your position), Python (and all necessary Python packages) would have to be reinstalled for that user.

I think the best path to install python is in C:\Users\<Username>\PythonXX (especially if you are using PC with multiple users and also if you are using your organization PC.
If you install to C:\PythonXX or to C:\Program Files\PythonXX, you will always have permissions issue. As soon as you try to install some packages/libraries using 'pip install ', permissions will not allow to install the packages.
If you leave the installation to be continued in C:\Users\<Username>\AppData\Local\Programs\Python\PythonXX as suggested by python installer, which also doesn't have any permissions issue, but path is not so easy to remember.
Few more important points:
If you are using a personal laptop and you are the only user (you will be administrator as well), then installing python to C:\PythonXX is also a best option.
Always check the option, add python path to Path variable during installation, so that you can also use python from command prompt/powershell.
Always check the option, install pip, so that you can use pip to install/update packages/libraries.

In my case because the Path variable was already set, I would only go to the python shell and type:
import os
os.path

Install and run tensorflow

I have never used tensorflow (or python). I installed Python 3.5.2 from https://www.python.org/downloads/. Then I followed instructions to install tensorflow according to this page, for a Windows 64-bit installation:
https://www.tensorflow.org/versions/r0.12/get_started/os_setup.html#download-and-setup
I followed the steps for Pip installation on windows as a CPU-only install. Then I followed Test the TensorFlow installation/ Run TensorFlow from command line which also seemed mostly fine.
But I hit a brick wall when I try the section Run a TensorFlow demo model. Specifically, the command python -m tensorflow.models.image.mnist.convolutional returns a Error while finding spec for 'tensorflow.models.image.mnist.convolutional' (ImportError: No module named 'tensorflow.models').
I don't really understand the installation nor what I'm missing. Can you tell me what I've screwed up?

Tensorflow models have been moved to a different github repository.
Therefore, they wont be available out of the box anymore when TF is installed.
You can find the file right here:
https://github.com/tensorflow/models/blob/master/tutorials/image/mnist/convolutional.py
You can download this file and execute it manually.
Sorry for out of date documentation.

To run the example the way you've described I think you'd have to add the tensorflow directory (or directory containing all 3rd party packages) to the PYTHONPATH environment variable as in the answer to this question or be in a directory where python can find the tensorflow module. Alternatively, you can run the example using:
python path/to/convolutional.py
from the command line which passes the script to python.exe and executes it. Make sure to either put in the full path or cd to the directory containing the script.
Using Anaconda as mentioned in your second link might be the best bet as setting PYTHONPATH will be taken care of for you for 3rd party packages installed either using conda or pip.