I have been trying to get the gdal library work using Python 2.7 and Anaconda in Windows 8 environment.
Besides gdal, I have also installed libgdal (frankly, I don't really understand the difference between the two). I now seem to have gdal 2.1.0 and 2.0.2 as well as libgdal 2.1.0.
However, when I run my Py code, there is a gdal error:
'gdalwarp' is not recognized as an internal or external command,
operable program or batch file.
I have already set the GDAL_DATA environmental variable to point to
C:\Anaconda\pkgs\libgdal-2.1.0-vc9_0\Library\share\gdal
I have also added a path, although I am not entirely sure where this should point to:
C:\Anaconda\pkgs\libgdal-2.1.0-vc9_0\Library\bin
I have tried the same with gdal 2.0.2 without success. gdalwarp.exe does seem to exist under libgdal 2.1.0 and gdal 2.0.2.
Any ideas? Is there an issue with the installation or have I not set the environmental variables correctly?
FYI, I have tried various installation commands, notably:
conda install gdal
conda install -c conda-forge gdal
conda install -c anaconda gdal
Addendum: I have found a manual solution: I set the GDAL_DATA and PATH variables in the terminal (pointing to libgdal 2.1.0) before running the code...
However, there is still an issue when I run my Py code: it is supposed to convert a tiff file to shp with gdal_polygonize:
cmd = 'gdal_polygonize.py %s -f "ESRI Shapefile" %s'%(dst_tif, dst_shp)
There is no error but the shapefile is not created (which leads to an error later on in the code). Any ideas as to why gdal is still not working correctly?
I have tried pointing the env variables to osgeo:
set PATH=%PATH%;C:\Anaconda2\Lib\site-packages\osgeo\scripts
set GDAL_DATA=C:\Anaconda2\Lib\site-packages\osgeo\data\gdal
Gdalinfo works but the gdal_polygonize used in my Py code does not appear to work.
The key is the activation script which is (potentially) executed when activating the environment. Not every GDAL build for Conda contains this. In my experience recent Conda-Forge builds are really good.
With your requirements of py27 and GDAL 2.1 i can get it working by following these steps:
1) Create a new environment: conda create -n gdaltest python=2.7
2) Activate: activate gdaltest
3) Install GDAL: conda install gdal=2.1 -c conda-forge
4) Reactivate environment: deactivate + activate gdaltest
This forces the just installed activation script to be executed, this sets the environment variables.
If i start python and run os.system("gdalinfo"), i can see its picked up correctly. And running os.environ['GDAL_DATA'] confirms the path is set correctly.
You can view the (de)activation script yourself at:
C:\Miniconda3\envs\<env name>\etc\conda\activate.d\gdal-activate.bat
A few years ago this didn't work as well as it does today, so make sure you have a recent Conda version (4.3.x) etc.
The benefit of this method is, that when switching environments, your paths are also changed accordingly. A "hard coded" GDAL_DATA path could potentially cause some compatibility issues if you mix and match different GDAL versions (although normally is should work OK).
Related
I want to import MeCab and use it, but that error comes out repeatedly.
MeCab-python is well installed.
I've tried Brew install and so on, and it's still the same.
I'd appreciate it if you could help me if you knew the solution.
Hmm. On a fresh macOS 13.1 Ventura, I just did this:
Installed miniconda https://docs.conda.io/en/latest/miniconda.html#latest-miniconda-installer-links
conda create -n mecab-tutorial and answering questions
conda activate mecab-tutorial
conda install python ipython and saying 'yes'
python -m pip install mecab-python3 unidic-lite
Then I was able to run this script:
import MeCab
wakati = MeCab.Tagger("-Owakati")
wakati.parse("pythonが大好きです").split()
# ['python', 'が', '大好き', 'です']
Looking at the path in the error in the screenshot, it looks like you installed a global Python in /Library which I don't want to try because I very much prefer to keep my Python environments local, hence using Conda above to create a custom environment to install MeCab packages in to ensure no cross-contamination later.
There's no guarantee that this will work if you try it since the error you're seeing seems to be a compiler issue (Python is finding MeCab, just having trouble calling a C++ function inside the binary), but if possible, can you try the Conda approach above?
I have installed Anaconda from its site and was working fine for sometime, however I needed to install Plotly and used the below steps mentioned in another site.
I just got it up and running on spyder 3.0 using the following steps. (windows 10)
Download plotly using pip usig command line (python -m pip install plotly) this requires downloading python 3.5 separately from Spyder as well. (so far I haven’t had any conflicts)
In Spyder, goto->Tools ->PYTHONPATH Manager -> addPath -> insert path to Plotly library (mine was in python\python36-32\Lib\site-packages), then synchronize
Restart Spyder
test it out with import plotly.plotly \n import plotly.graph_objs as go in a new .py scrypt
Hope it works out for you.
Cheers
After the above steps I was able to import plotly in Spyder and didn't face any issues, however after I restarted my machine I'm unable to run Anaconda navigator or Spyder.
I'm able to launch Anaconda prompt but any command executed returns different kinds of errors like
"conda install anaconda-navigator"
environment variables:
conda info could not be constructed.
KeyError('pkgs_dirs')
"spyder"
ImportError: cannot import name 'constants' from partially initialized module 'zmq.backend.cython' (most likely due to a circular import) (C:\Python\Lib\site-packages\zmq\backend\cython_init_.py)
anaconda-navigator
ImportError: DLL load failed while importing shell: The specified module could not be found.
I tried every solution on internet like uninstalling and reinstalling, deleting all the trace files on anaconda and even the Environment variables seem to be fine
echo %PATH% command returns
C:\ProgramData\Anaconda3;C:\ProgramData\Anaconda3\Library\mingw-w64\bin;C:\ProgramData\Anaconda3\Library\usr\bin;C:\ProgramData\Anaconda3\Library\bin;C:\ProgramData\Anaconda3\Scripts;C:\ProgramData\Anaconda3\bin;C:\ProgramData\Anaconda3\condabin;C:\Python\Scripts;C:\Python;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0;C:\windows\System32\OpenSSH;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\nodejs;C:\Program Files\Microsoft VS Code\bin;C:\Users\saish\AppData\Local\Programs\Python\Python38;C:\Program Files\Java\jre1.8.0_301\bin;C:\apache-zookeeper-3.7.0\bin;C:\Spark\spark-3.0.2-bin-hadoop2.7\bin;C:\hadoop\bin;C:\Users\saish\AppData\Local\Microsoft\WindowsApps;C:\Users\saish\AppData\Roaming\np
where python command returns
C:\ProgramData\Anaconda3\python.exe
C:\Users\saish\AppData\Local\Microsoft\WindowsApps\python.exe
Please suggest any new measures other than the ones I have mentioned. Thanks in advance
Per this github issue, you may have a conflict between dependencies of packages anaconda installed and the one you installed manually. Check your pythonpath and see if removing the pip folder from the pythonpath fixes the issue.
I'm trying to use reticulate to run some simple Python code in an RMarkdown document. I've found that if Matplotlib is in the conda environment, I get errors when trying to run a python code chunk, but I can run Python from R directly. Here's a simple example of what I see:
---
title: "Reticulate Test"
date: "9/21/2020"
output: html_document
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(reticulate)
use_condaenv('Toy_MPL') # this environment contain matplotlib and produces the error
#use_condaenv('Toy') # this environment does not contain matplotlib and no error
```
```{r}
# this works regardless of which environment I use
pysys <- import('sys')
pysys$version
```
[1] "3.8.5 (default, Sep 4 2020, 02:23:17) \n[Clang 10.0.0 ]"
```{python, engine.path = '/opt/miniconda2/envs/Toy_MPL/bin/python'}
# if Toy_MPL conda environment is used, the error is generated
# if Toy conda environment is used, I get the same output as above
import sys
print(sys.version)
```
Error in py_call_impl(callable, dots$args, dots$keywords) :
TypeError: use() got an unexpected keyword argument 'warn'
My first thought was that reticulate was not seeing the various system libraries that are installed in the conda environment lib/ folder when Matplotlib is installed - there are a LOT of dependencies that come along with Matplotlib. I tried the following, but none worked:
Set LD_LIBRARY_PATH in .Renviron to point to the correct library path.
Call use_python() in addition to or instead of use_condaenv()
set engine.path in the Python code chunk
I tried downgrading matplotlib to v3.2 (suggested here), but that caused a new set of errors:
Error in if (has_compatible_arch && has_preferred_numpy) valid_python_versions <- c(valid_python_versions, : missing value where TRUE/FALSE needed
Checking NumPy, I see I have v1.19.1 (other errors suggest needing >1.6). And, reinstalling matplotlib v3.3.1 does not prevent the error. After this "fix" I end up having to rebuild the entire environment.
traceback() gives me a CPP stack trace from the reticulate.so which is not interpretable.
My interpretation is that the environment created for RMarkdown does not point to the correct library locations, but I cannot determine how to set it correctly.
System info:
Mac OS Catalina 10.15.6
RStudio v1.3.1073
reticulate v1.16
conda v4.8.4
Python in conda environments v3.8.5
Matplotlib in Toy_MPL environment v3.3.1
In my original question, I referred to this question in which there was a suggestion to downgrade matplotlib to version 3.2.0 because reticulate was not up to date with changes in matplotlib. I followed up further on that suggestion and have found a resolution (for now).
TL;DR
Removing pip and conda installed versions of matplotlib, and then installation of matplotlib version 3.2.2 with conda (NOT pip) resolves the problem. Installing matplotlib with pip leads to other errors.
Details
In the response to the other question, the suggestion was to do:
pip install matplotlib==3.2
I tried this and ended up with other errors that I also could not track down. So, I uninstalled matplotlib and then reinstalled it with
pip install matplotlib==3.3.1
in hopes of getting back to where I was. This also did not work and the new errors persisted. I then removed matplotlib completely and with pip and reinstalled version 3.3.1 with conda:
pip uninstall matplolib
conda install matplotlib
This got me back to matplotlib version 3.3.1 and the original error I mention in my question. I then tried installing matplotlib version 3.2 with conda:
conda install matplotplib==3.2
The installed version is 3.2.2 and not 3.2.0, as suggested in the response, but when I did this, the original problem seems to be resolved.
There is clearly a difference in dependency resolution between pip and conda in this case, and conda provides a version of matplotlib that plays nicely with reticulate. I do not at this point know what the difference is, however.
I ran into a similar issue as well. I'm new to python so I used the Anaconda Navigator to manage my python environments and packages. I fixed my problem by doing the following.
Remove the reticulate package from R
Open the Anaconda Navigator and remove the r-reticulate environment from environments page. I also removed an additional conda environment that I had created with reticulate.
Re install the reticulate package and run your code
This won't help too much if you still want to use matplotlib with reticulate, but it should at least get your script running again if it doesn't use matplotlib.
I created a virtual environment called env using
python -m venv env
.\env\Scripts\activate.bat
pip install tensorflow
I verified tensorflow is in the env\Lib\site-packages folder
Next I loaded VS Code and created a workspace, added a python file, it prompted me to install pylint,
I typed in python: select interpreter and I browsed to C:\Users\admin\env\Scripts folder
This is the command line at the beginning of the script
(env) PS C:\Users\admin\env\project> cd 'c:\Users\admin\env\project'; & 'C:\Users\admin\env\Scripts\python.exe' 'c:\Users\admin\.vscode\extensions\ms-python.python-2020.8.106424\pythonFiles\lib\python\debugpy\launcher' '54436' '--' 'c:\Users\admin\env\project\face_gan.py'
This is the error I get when debugging the python file:
ImportError: Keras requires TensorFlow 2.2 or higher. Install TensorFlow via `pip install tensorflow`
PS C:\Users\admin\env\project> & C:/Users/admin/env/Scripts/Activate.ps1
When I type in pip install tensorflow in VS Code terminal, it shows its already installed
(env) PS C:\Users\admin\env\project> pip install tensorflow
Requirement already satisfied: tensorflow in c:\users\admin\env\lib\site-packages (2.3.0)
I don't understand this, is it not running in virtual environment?
Why is it executing C:/Users/admin/env/Scripts/Activate.ps1 at the end of the debugging session, not at the beginning
Lastly, is running python from the virtual environment folder C:\Users\admin\env\Scripts the same as using the activate.bat file or the source command? Does it automatically defer to using the C:\Users\admin\env\Lib folder, or is it still trying to use the default python installation to look for Tensorflow?
What step did I miss to make it use the virtual environment correct in VS Code?
First question: executing C:/Users/admin/env/Scripts/Activate.ps1 after debugging command make no difference. It just because it's the first command of the terminal. You can run it again to make a try.
Second question: Yes, that's the same. In your case, it will add 'C:\Users\admin\env' and 'C:\Users\admin\env\lib\site-packages' path to the PYTHONPATH variable.
You can through these codes to get the PYTHONPATH(the default search path for module files) variable value:
import sys
print(sys.path)
If you import 'tensorflow' directly. you will find you can import it correctly. It's a version problem. You should downgrade the version of the packages, and you can refer to this comment to get some useful information.
C:\Users\%USER%>conda info
Current conda install:
platform : win-32
conda version : 3.16.0
conda-build version : not installed
python version : 2.7.10.final.0
requests version : 2.7.0
root environment : C:\Users\%USER%\Miniconda (writable)
default environment : C:\Users\%USER%\Miniconda
envs directories : C:\Users\%USER%\Miniconda\envs
package cache : C:\Users\%USER%\Miniconda\pkgs
channel URLs : https://repo.continuum.io/pkgs/free/win-32/
https://repo.continuum.io/pkgs/free/noarch/
https://repo.continuum.io/pkgs/pro/win-32/
https://repo.continuum.io/pkgs/pro/noarch/
config file : None
is foreign system : True
# NOTE:
# root directory 'C:\Users\%USER%\Miniconda' is uninitialized
I see my root directory is not initialized.
Additionally when I try to install a package (pandas eg) I get the following error:
C:\Users\%USER%>conda install pandas
Error: This installation of conda is not initialized. Use 'conda create -n
envname' to create a conda environment and 'source activate envname' to
activate it.
# Note that pip installing conda is not the recommended way for setting up your
# system. The recommended way for setting up a conda system is by installing
# Miniconda, see: http://repo.continuum.io/miniconda/index.html
I assume they are related but I am not sure the course of action.
I had anaconda/python27 up and running but wanted to update matplotlib and thought to reinstall python with conda to be "up to date".
The error is documented here, but to be honest I do not really get it.
I regret this as it is a bit tough to get up and running again!
I had a similar problem, but with Anaconda on Windows 8.1.
The python env variables are set from a previous python install.
The installer did not update (user)PYTHONHOME, (system)PYTHON_DIR, and (system)PYTHONPATH, so I manually set them to /Users/me/Anaconda/.
conda update conda
gave the error mentioned above. I added a new conda environment...
conda create -n notebook jupyter
activate notebook
which succeeded, but...
conda info --envs
...gives...
notebook C:\Users\<me>\Anaconda\envs\notebook
root * C:\Users\<me>\Anaconda
...and...
conda info
...
# NOTE: root directory 'C:\Users\me\Anaconda' is uninitialized.
I suspect that the installer failed due to the presence of the environment variables. I deleted the environment variables and the /Users/me/Anaconda directory, then installed miniconda.
All seems to be well now.
Note:
I had previously installed python at least three different ways.
{Chocolatey, Anaconda(all)[failed], python msi}