I want to install a library called "Scrapy" for Python on Windows. The developers recommend installing the package using Anaconda instead of Python3. However, will i be able to use the package in Python3 or will i need to use Anaconda for Web Scraping?
Anaconda is loved because it simplify package management and deployment in Python(and R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, FORTRAN)
You get to keep your environment(program dependency) organized when using Anaconda.
With Anaconda, you can use any Python version, and libraries you need for a specific task. A workflow would be as follow(Assuming you have install Anaconda, and its available on your system path):
conda create -n scrap python=3.6
conda activate scrap
conda install -c conda-forge scrapy
Here we create environment called scrap with python version 3.6. We then activate it and install scrapy fron a conda channel forge.
While in this environment(scrap), you have access to Python 3.6 and scrapy. The best thing about this is that this is separate from your other Python and packages.
To get out of your environment,
conda deactivate
While in your environment, you can use both conda and pip to install packages to that environment. Always try finding packages in conda(plus it’s channels) before using pip, becsusr conda will check for packages compatabilities before installation. It will know which packages to upgrade or downgrade to avoid conflicts.
In few cases where a package is not in conda, then use pip.
Read more:
https://conda.io/docs/index.html
Anaconda is a python distribution which contains additional packages to the python it ships with. To have lightweight version of python (without many additional packages), you can install Miniconda. Anaconda and Miniconda come with the conda package manager. You can use it to install and update packages specific to its python distribution.
To install scrapy package using Anaconda / Miniconda, in the Windows Command Prompt simply type in:
conda install scrapy
In Anaconda you can install almost all python packages using conda install or pip install.
You have to goto Anaconda prompt and type in pip install Scrapy
Anaconda just simplified the installation, but you can also install scrapy through PyPi as
pip install scrapy
Related
I'm struggling to find any documentation on this.
I started using pyenv a while back on my M1 Mac to manage python versions.
Recently I had some issues getting tensorflow installed, so followed some instructions online which recommended using miniforge
pyenv local miniforge
python -v venv venv
source venv/bin/activate
conda install -c apple tensorflow-deps
pip install tensorflow-macos
pip install tensorflow-metal
However, I'm really not sure what this is doing. It's setting pyenv local to miniforge (rather than a python version). It's setting up a virtual environment and activating. Then using the conda command to install something tensorflow related, and then using pip to install some more tensorflow stuff.
What exactly is happening when I set miniforge with pyenv? I only thought you could set python versions here, and though miniforge was just an installer of sorts.
I think you want to check out the anaconda docs. The api is the same. https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
I stumbled on this post:
Install of R in miniconda
https://community.rstudio.com/t/install-of-r-in-miniconda/19755
Is it possible to install R in miniconda?
As I understand it miniconda is a package manager for Python only according to this definition:
Miniconda
Miniconda is a free minimal installer for conda. It is a small, bootstrap version of Anaconda that includes only conda, Python, the packages they depend on, and a small number of other useful packages, including pip, zlib and a few others. Use the conda install command to install 720+ additional conda packages from the Anaconda repository.
https://docs.conda.io/en/latest/miniconda.html
Or is it possible to use it with R?
I know Anaconda 3 offers R support.
Thanks.
Miniconda (the light, non-GUI version of Anaconda) gives you access to conda on the command line, and with that you can install R, along with some common packages, as follows:
conda install r r-essentials --channel conda-forge
And any further R packages you need:
conda install r-<package-name> --channel conda-forge
The "Conda" ecosystem is language-agnostic, it delivers whatever you ask for (if it exists in the repository) and installs the necessary platform binaries, but I would suggest creating a virtual environment specific to each "language platform", to ensure isolation.
Example:
conda create -n r_env r-essentials r-base
conda activate r_env
conda list
And work within this environment to run R and install new packages.
To leave the virtual environment:
conda deactivate
I'm using python 3.7 (environment created by anaconda) and trying to run python code that uses some google libraries but I don't really know how to install them.
From PyCharm IDE (Settings -> Project Interpreter -> Available Packages) I cannot find those packages to install.
And from terminal, running 'pip install --upgrade oauth2client' or 'pip3 install --upgrade oauth2client' doesn't seem to work either.
What I don't understand is: to install packages/libraries on python 3.x, should i only use pip3? But what if there is more than one python environment? On which one pip will install those libraries?
If you're using Anaconda, you should also have an Anaconda Prompt (py37) program that functions like the terminal. Typing pip install --upgrade oauth2client into Anaconda Prompt (py37) will install the upgrade into your current python environment. Alternatively, you can use conda install with more options (including specifying which environment you want).
I have installed a fresh anaconda v4.4. I realized that python packages can be installed using both conda and pip. What is the effect of using pip to install python packages instead of conda when using anaconda? Will the pip-installed libraries cease to function? I am using python v3
EDIT: I don't think the question is a duplicate of What is the difference between pip and conda?
That question explains the difference between pip and conda but does not talk about the effect of using pip when conda can be used.
Everything might keep working if you use pip to install vs conda. However, Conda cannot manage dependencies that pip has installed - it cannot upgrade them, or remove them. More importantly, conda will install a package even if its already been installed with pip! Try this test:
conda create -n testenv python=3
conda activate testenv
pip install numpy
conda install scipy
You will see from the third command that conda will want to re-install NumPy, even though it has already been installed with pip. This can cause problems if there are C libraries whose linking is different, or something like that. In general, whenever possible, use conda to install packages into conda environments.
I have the anaconda distribution of python installed on my machine. Before installing anaconda, i had pip installed. Right now, my system uses the previous version of pip rather than the anaconda version. In particular
arjuns-mbp:~ Arjun$ which pip
/usr/local/bin/pip
arjuns-mbp:~ Arjun$ which easy_install
/Users/Arjun/anaconda/bin/easy_install
arjuns-mbp:~ Arjun$ which python
/Users/Arjun/anaconda/bin/python
The result is that if i call a pip install, it won't add it to my anaconda version of python. So far i haven't had an issue simply using easy_install to add packages to anaconda, but it would be nice to know what causes this discrepancy and how to fix it
EDIT
i tried using conda install pip, it doesn't work. When I do
conda list
pip showed up before and after a conda install
You probably need to conda install pip.
pip should already come with Anaconda automatically. How did you install Anaconda? No matter, though, you can just conda install pip to make sure you have a version of pip that is tied to the Anaconda distribution.
From there you can create environments as much as you like to install whatever new trial-run software tools using conda create -n env_name python pip