We Keep Coding

What would cause the packages/libraries in Spyder to be different from the ones when using pip?

This problem occurred when I was trying to import pandas:
df = pd.read_html('https://www.macrotrends.net/stocks/charts/BMO/Bank-of-Montreal/dividend-yield-history')
I got the error message:
raise ImportError("lxml not found, please install it")
ImportError: lxml not found, please install it
So then I pip lxml to install, same error.
then I pip3 lxml, same issue...
And this led me down a rabbit hole of installing and re-installing lxml, pandas, html5lib, html
and then deleting all of Python from my machine as well as Spyder, then re-installing everything and still having the same issue of that error "lxml not found..."
So then I did 'pip list' on command line, lxml and pandas and everything else I installed is there.
I found some code to find all the packages/libraries installed on Spyder using this:
import pkg_resources
dists = [str(d).replace(" ","==") for d in pkg_resources.working_set]
for i in dists:
print(i)
And I get:
runfile('C:/Users/Preston_Jarvis/untitled0.py', wdir='C:/Users/Preston_Jarvis')
zipp==3.4.1
yapf==0.31.0
wheel==0.36.2
webencodings==0.5.1
wcwidth==0.2.5
And so on...
Pandas is there, but lxml or html5lib or hmtl
What could be causing this?
Thank you

How did you install Spyder? Did you use the installer or install it with pip/conda? Where are you installing your packages - what environment?
It appears that the environment where you install packages with pip (e.g. lxml) is different than what Spyder is running with. See https://docs.spyder-ide.org/current/faq.html#using-existing-environment
Note that for Spyder to work with other environments you need to install spyder-kernels in this environment (e.g. pip install spyder-kernels)

Installation using setup.py is now impossible on macOS Big Sur

I have 2 of my projects that install perfectly fine on ubuntu, both require python3.8, both are installable using python setup.py install and I occasionally uninstall / install them from / to virtual environments as well as globally for testing purposes. Recently, all kinds of troubles and error messages started to occur not following any special event, I just woke up one day, and tried to install one of them, I got an error, then I tried the other and got another error. I will include both setup.py scripts and show the steps I follow on macOS Big Sur 11.0 to reproduce the results, given that both install perfectly fine on Ubuntu with the same list of respective dependencies.
Versions:
python: 3.8.6
pip: 20.3.1
macOS: 11.0.1
Project 1:
requirements.txt
imagesize==1.2.0
numpy==1.18.5
pandas==1.1.4
seaborn==0.11.0
tensorflow==2.3.1
matplotlib==3.3.3
lxml==4.6.2
imgaug==0.4.0
tensorflow-addons==0.11.2
opencv-python-headless==4.4.0.46
imagecorruptions==1.1.2
configparser~=5.0.1
scipy==1.5.4
PyQt5==5.15.2
tabulate==0.8.7
ipykernel==5.3.4
setup.py
from setuptools import setup, find_packages
install_requires = [dep.strip() for dep in open('requirements.txt')]
setup(
name='project1',
version='1.0',
packages=find_packages(),
url='url',
license='MIT',
author='author',
author_email='email#domain',
description='description goes here',
install_requires=install_requires,
python_requires='>=3.8',
entry_points={
'console_scripts': [
'some_entry_point',
],
},
)
Notes:
I will truncate logs for character limits.
The attempts below are done within a virtualenv
All kinds of problems do not occur if I just use pip install -r requirements
Installation works perfectly fine on Ubuntu and used to work perfectly fine on macOS even after I upgraded to Big Sur, it just suddenly decided to never work again.
python is installed using brew install python#3.8
Attempt 1:
virtualenv proj1
source proj1/bin/activate
python setup.py install
Result:
Searching for PyQt5==5.15.2
Reading https://pypi.org/simple/PyQt5/
Downloading https://files.pythonhosted.org/packages/28/6c/640e3f5c734c296a7193079a86842a789edb7988dca39eab44579088a1d1/PyQt5-5.15.2.tar.gz#sha256=372b08dc9321d1201e4690182697c5e7ffb2e0770e6b4a45519025134b12e4fc
Best match: PyQt5 5.15.2
Processing PyQt5-5.15.2.tar.gz
error: Couldn't find a setup script in /var/folders/hr/61r_7jcx2r3cnklwrr2zwbqw0000gn/T/easy_install-sl9y34mj/PyQt5-5.15.2.tar.gz
Attempt 2: (I install pyqt using pip)
pip install pyqt5==5.15.2 # success
python setup.py install # fail
Result:
No module named numpy.
Attempt 3: (add setup_requires=['numpy==1.18.5'] to setup())
python setup.py install
Result:
numpy.distutils.system_info.NotFoundError: No lapack/blas resources found. Note: Accelerate is no longer supported.
During handling of the above exception, another exception occurred:
# Traceback goes here
RuntimeError: implement_array_function method already has a docstring
Attempt 4:
I checked this issue and there is nothing helpful yet so I manually install numpy and scipy:
pip install numpy==1.18.5 scipy # success
python setup.py install # fail
Result:
ModuleNotFoundError: No module named 'skbuild'
Attempt 5:
pip install scikit-build==0.11.1 # success
python setup.py install # fail
Result:
error: Setup script exited with Problem with the CMake installation, aborting build. CMake executable is cmake
Attempt 6:
I checked the issue here and accordingly:
pip install cmake # success
python setup.py install
Opencv is being built, I'm not sure why or what triggered the build so I aborted and manually installed opencv using pip:
pip install opencv-python-headless==4.4.0.46 # success
python setup.py install # fail
Result:
Could not find suitable distribution for Requirement.parse('tensorflow-addons==0.11.2')
Attempt 7:
pip install tensorflow-addons==0.11.2 # success
python setup.py install
Again lxml is being built, I don't know how or why, so I manually install it and:
pip install lxml==4.6.2 # success
python setup.py install # fail
Result:
Running matplotlib-3.3.3/setup.py -q bdist_egg --dist-dir /var/folders/hr/61r_7jcx2r3cnklwrr2zwbqw0000gn/T/easy_install-q32mufo3/matplotlib-3.3.3/egg-dist-tmp-_js7sem9
UPDATING build/lib.macosx-11.0-x86_64-3.8/matplotlib/_version.py
set build/lib.macosx-11.0-x86_64-3.8/matplotlib/_version.py to '3.3.3'
error: Setup script exited with error: Failed to download FreeType. Please download one of ['https://downloads.sourceforge.net/project/freetype/freetype2/2.6.1/freetype-2.6.1.tar.gz', 'https://download.savannah.gnu.org/releases/freetype/freetype-2.6.1.tar.gz'] and extract it into build/freetype-2.6.1 at the top-level of the source repository.
Attempt 8:
pip install matplotlib==3.3.3 # success
python setup.py install # fail
Result:
Could not find suitable distribution for Requirement.parse('tensorflow==2.3.1')
Attempt 9:(already more than 75% of the requirements were already installed manually using pip in the previous attempts)
pip install tensorflow==2.3.1 # success
python setup.py install # fail
Result:(pandas fails to install)
RuntimeError: Cannot cythonize without Cython installed.
Attempt 10: (I manually install cython)
pip install cython # success
python setup.py install
Again pandas for some reason is being built, so I manually install it and the list goes on ...
I think after this simple demonstration, I don't even need to replicate the steps of the second project however here are its requirements and I pretty much get a variation of the same problems shown above.
requirements.txt for project2
oauth2client==4.1.3
gcloud==0.18.3
pyarrow==2.0.0
requests==2.24.0
pandas==1.1.4
httplib2==0.15.0
TA-Lib==0.4.19
matplotlib==3.3.2
matplotlib fails first, followed by ta-lib (given that I run brew install ta-lib before then) and the list goes on ...

Instead of python setup.py install do python -m pip install ..
Instead of python setup.py develop do python -m pip install --editable ..

When I install eric6, I get the following error:Please install QScintilla2 and its PyQt5/PyQt4 wrapper

When I installed eric6, I was prompted with the following problems:
Python Version: 3.6.5
Found PyQt5
Found pyuic5
Sorry, please install QScintilla2 and its PyQt5/PyQt4 wrapper.
Error: DLL load failed: Cannot find the specified program!
But I have installed QScintilla via "pip install QScintilla", so what should I do next?
My operating system is Windows.

I see this error on today too , I used pip to install this module but all stop or fail, so I just can download 'whl' from pypi.org and 'pip install *.whl'. and re-install sip and PyQt5<5.9.3> before. And fixed. you can try it.

sip-4.19.4-cp36-none-win_amd64
PyQt5-5.9.2-5.9.3-cp35.cp36.cp37-none-win_amd64
QScintilla-2.10.1-5.9.1-cp35.cp36.cp37-none-win_amd64
install sequence: sip -> PyQt5 - QScintilla

Install ROS Kinetic on MacOS under a separate Anaconda environment

I'm installing ROS Kinetic on MacOS Sierra 10.12.6 under a seperate environment under Python Anaconda distribution.
Here is my python version(my seperate python environment in Anaconda is called "ros-kinetic"):
(ros-kinetic) jizhe#jizhedeMacBook-Pro  ~/ros_catkin_ws  python --version
Python 3.6.4 :: Anaconda, Inc.
I followed the guide: http://wiki.ros.org/kinetic/Installation/OSX/Homebrew/Source.
When I execute the "rosdep install" command, I got the following error:
(ros-kinetic) jizhe#jizhedeMacBook-Pro ~/ros_catkin_ws  rosdep install --from-paths src --ignore-src --rosdistro kinetic -y
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
actionlib: No definition of [python-wxtools] for OS [osx]
catkin: No definition of [google-mock] for OS [osx]
roslisp: No definition of [google-mock] for OS [osx]
Obviously, there is something wrong with the package "python-wxtools" and "google-mock".
I've already install the wxpython via "brew install wxpython", and here is the wxpython info:
(ros-kinetic) jizhe#jizhedeMacBook-Pro  ~/ros_catkin_ws  brew info wxpython
wxpython: stable 3.0.2.0 (bottled)
Python bindings for wxWidgets
https://www.wxwidgets.org/
/usr/local/Cellar/wxpython/3.0.2.0_1 (1,108 files, 37.9MB) *
Poured from bottle on 2018-02-24 at 03:56:21
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/wxpython.rb
==> Dependencies
Required: wxmac ✔
==> Caveats
Python modules have been installed and Homebrew's site-packages is not
in your Python sys.path, so you will not be able to import the modules
this formula installed. If you plan to develop with these modules,
please run:
mkdir -p /Users/jizhe/Library/Python/2.7/lib/python/site-packages
echo 'import site; site.addsitedir("/usr/local/lib/python2.7/site-packages")' >> /Users/jizhe/Library/Python/2.7/lib/python/site-packages/homebrew.pth
And I've already run the command:
$ mkdir -p ~/Library/Python/2.7/lib/python/site-packages
$ echo "$(brew --prefix)/lib/python2.7/site-packages" >> ~/Library/Python/2.7/lib/python/site-packages/homebrew.pth
So, what's wrong with the package "google-mock" and "python-wxtools", how can I solve this problem?

Try installing with --skip-keys python-wxtools --skip-keys google-mock. It worked for me.

There is a ticket for this issue. Basically the workaround is to install googletest manually. Not sure what's wrong with python-wxtools.
edit: If you're using ros-install-osx, you can add the following lines to your rosdeps.yaml file:
google-mock:
osx:
homebrew:
packages: []

I found this question from a Google search, just for your information in 2021 installing ROS in conda also on macOS should be much straightforward, see the following links:
https://medium.com/robostack/cross-platform-conda-packages-for-ros-fa1974fd1de3
https://github.com/RoboStack/ros-noetic

How to get Python packages installed as .exe to run on Windows 10; "failed to create process"

After installing Python 3.5.1 on Windows 10 x64, cpplint installed from pip produces the error: failed to create process. There seems to be a possibly related issue with pip related to having a space in the path, which exists in my use case. It seems to be related to how the exe is created. How can this error be remedied? Reinstalling Python and cpplint does not solve the problem.
C:\Users>python -V
Python 3.5.1
C:\Users>pip list
pip (7.1.2)
setuptools (18.2)
C:\Users>pip -V
pip 7.1.2 from c:\users\john hagen\appdata\local\programs\python\python35\lib\site-packages (python 3.5)
C:\Users>pip install cpplint
Collecting cpplint
Using cached cpplint-0.0.6.tar.gz
Installing collected packages: cpplint
Running setup.py install for cpplint
Successfully installed cpplint-0.0.6
C:\Users>where.exe cpplint
C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\Scripts\cpplint.exe
C:\Users>cpplint
failed to create process.

Yep, this is a pip bug.
The workaround is to add quotes around the path in the first line of generated python scripts in the \Scripts\ directory. In the case of cpplint this is 'cpplint-script.py'. In your case the generated first line should something like:
#!c:\users\john hagen\appdata\local\programs\python\python35\python.exe
and should be edited to:
#!"c:\users\john hagen\appdata\local\programs\python\python35\python.exe"