We Keep Coding

MKL Issues installing PyFlashlight

I am attempting to install from a Python setup. I had this working after a month of hell-ish attempts on 18.04, and now need it on 20.04.
The setup.py while I am using is on this repository ( https://github.com/flashlight/flashlight/tree/main/bindings/python ).
I am using Ubuntu 20.04, fresh install sans a few dependencies I know the system needs.
I have Python3.10 installed locally.
During the install at steps
cd bindings/python
python3.10 setup.py install
I hit the error
CMake Error at cmake/FindMKL.cmake:400. MKL library not found. Please specify the library location by appending the root directory of the MKL installation to the environment variable CMAKE_PREFIX_PATH
Which is weird for a variety of reasons.
I did a sudo apt-get install mkl-devel which took, but also
The setup script claims it defaults to MKL off.
Right prior to this error, I do get a warning.
Cmake Warning at cmake/FindMKL.cmake:387. MKL libraries files are found, but MKL header files are not. You can get them by 'pip install mkl-devel' (I did this btw) if using pip. If build fails with header files available on the system, please make sure that CMake will search the directory containing them by setting CMAKE_INCLUDE_PATH
That last part I did not do, primarily because I'm not sure how to tell where pip install mkl-devel would have put those header files.
That said, when I do a find / "*mkl*" I notice files in two primary locations
.h files : /home/myusername/.local/include
.so files: /home/myusername/.local/lib
So I set the following environment variables in my terminal
LIBRARY_PATH=/home/myusername/.local/lib
LD_LIBRARY_PATH=/home/myusername/.local/lib
KENLM_ROOT=/home/myusername/Flashlight/kenlm
USE_MKL=0
CMAKE_INCLUDE_PATH=/home/myusername/.local/include/
CMAKE_PREFIX_PATH=/home/myusername/.local/lib
However when I try to install again, the process still fails with both of the MKL issues above (Warning and Error)
I'm baffled at what I am supposed to be pointing where to get this to succeed at this point.

Can't find Brew installed packages on Mac M1 in Pycharm

I'm having trouble installing packages and using them in Pycharm. I've followed various threads (I'm new to Macs and seem to have tried everything) now I'm stuck.
In this case, I want to use the package xgboost.
I have brew installed, after launching a terminal using Rosetta:
%brew install xgboost
Warning: xgboost 1.3.3 is already installed and up-to-date.
It appears installed OK here:
/opt/homebrew/Cellar/xgboost
I also have Python installed here:
/opt/homebrew/Cellar/python#3.9
But no matter how I configure an Interpreter in Pycharm, I can't seem to get the package recognised.
Where have I gone wrong?

I am very unsure exactly how, but I've got this working.
Following: https://abbasegbeyemi.me/blog/homebrew-python-apple-m1
I changed the order of elements in my path:
export PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/homebrew/bin
then a new interpreter in Pycharm using:
usr/local/Cellar/Python#3.9/3.9.2_2/bin/python3.9
Now I can install packages just using pip in pycharm and it works.
This has been 6 hours of pain - warning to anyone who isn't well versed in macs, setting up an M1 for python dev was a complete nightmare for me.

Docs: https://xgboost.readthedocs.io/en/latest/build.html
Pre-built binary wheel for Python
If you are planning to use Python, consider installing XGBoost from a pre-built binary wheel, available from Python Package Index (PyPI). You may download and install it by running
# Ensure that you are downloading one of the following:
# * xgboost-{version}-py2.py3-none-manylinux1_x86_64.whl
# * xgboost-{version}-py2.py3-none-win_amd64.whl
pip3 install xgboost

Where did my python module install to?

I'm running python 3.6 via anaconda 3, using Visual Studio Code.
I followed instructions like these (Interactive Brokers API install) and downloaded the package to a local directory of mine say: c:\dev\pyib, so now the code is in c:\dev\pyib\IbPy-master
I open that directory in command line and run
python setup.py install
All runs ok.
But then my program, which is in c:\dev\pyib says Module not found. (In my case ibapi). The linter is also showing red.
There is no other python installed on this pc.
Where did the package install to? and how do I check that? What will I find where the package installed itself to that shows me its there?
Or do I have to use a trial-and-error with the linter and sys.path.append()? (I tried that with the directory where the files are downloaded to - to no avail)
I'm trying to set up the PYTHONPATH using the "env" in launch.json from Visual Studio Code, as shown in this unaccepted answer.
Current sys.path:
'c:\\dev\\pyIb',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\python36.zip',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\DLLs',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-
packages',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\Babel-2.5.0-py3.6.egg',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\win32',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\win32\\lib',
'C:\\Users\\user\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\Pythonwin'
I deleted the ib directory and re-ran the install. The last line says: Writing C:\Users\user\AppData\Local\Continuum\anaconda3\Lib\site-pac‌​kages\IbPy2-0.8.0-py‌​3.6.egg-info So is the location of the egg-info the location of my undetected module? The actual folder in the site-packages is called ib.
Or could my problems be because of a difference in Lib vs. lib with the lowercase in the sys.path and the uppercase in the actual directory?
But the real question here is still: HOW DO I KNOW WHERE the package was installed what should I search for?

This answer is specific for anaconda3 Python and packages installed using python setup.py install (which is actually using distutils)
Take a look at anaconda3\Lib\site-packages you should see a directory for the package you installed.
The way to know for sure where your package is, is by doing a pip list then trying to pip uninstall and re-install again using the python setup.py install: Here are the detailed instructions:
When uninstalling, pip will tell you it cannot because it was done via distutils.
You'll get a message like this:
DEPRECATION: Uninstalling a distutils installed project (ibpy2) has been deprecated and will be removed in a future version.
This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
You'll be prompted to continue anyway. If you choose No, then you can find the directory in
C:\Users\<yourusername>\AppData\Local\Continuum\anaconda3\Lib\site-packages
Thanks to Emanuel Mtali for pointing me in the right direction
Some more information:
The problem I had was due to a stupid mistake of mine. I was running setup of a different (but related) package not used anymore. IbPy2 instead of TwsAPI. I was supposed to run the setup.py of the package installed via the latest version of the MSI from IB, and NOT the IbPy2 package. :-(

Python pip install gives “Command ”python setup.py egg_info“ failed with error code 1”

Edit: Yes I know this question already exists, except my question is a bit different and none of the solutions fixed it.
I do most of my Python stuff when I'm at work and not on my personal machine, but I decided to install it on my personal computer as well. I fresh installed python 3.6.1, and created a virtual environment with virtualenv. Then within the virtualenv I tried to pip install urllib (or any module) and I received the error:
(pdbot) C:\Users\user\Documents\pdbot>pip install urllib
Collecting urllib
Using cached urllib-1.21.1.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\setup.py", line 191
s.connect((base64.b64decode(rip), 017620))
^
SyntaxError: invalid token
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\
I read elsewhere that this error had something to do with setuptools not being properly installed. So I ran this to attempt to fix the issue:
easy_install -U setuptools
I ended up receiving an even weirder error next:
(pdbot) C:\Users\zeke\Documents\pdbot>easy_install -U setuptools
Searching for setuptools
Reading https://pypi.python.org/simple/setuptools/
Downloading https://pypi.python.org/packages/a9/23/720c7558ba6ad3e0f5ad01e0d6ea2288b486da32f053c73e259f7c392042/setuptools-36.0.1.zip#md5=430eb106788183eefe9f444a300007f0
Best match: setuptools 36.0.1
Processing setuptools-36.0.1.zip
Writing C:\Users\zeke\AppData\Local\Temp\easy_install-jhg1val_\setuptools-36.0.1\setup.cfg
Running setuptools-36.0.1\setup.py -q bdist_egg --dist-dir C:\Users\zeke\AppData\Local\Temp\easy_install-jhg1val_\setuptools-36.0.1\egg-dist-tmp-8apak7kn
warning: no files found matching '*' under directory 'setuptools\_vendor'
Copying setuptools-36.0.1-py3.6.egg to c:\users\zeke\documents\pdbot\lib\site-packages
Adding setuptools 36.0.1 to easy-install.pth file
Installing easy_install-script.py script to c:\users\zeke\documents\pdbot\Scripts
Installing easy_install.exe script to c:\users\zeke\documents\pdbot\Scripts
error: [WinError 5] Access is denied: 'c:\\users\\zeke\\documents\\pdbot\\Scripts\\easy_install.exe'
This looks like a permissions error, but I ran these both in an administrator command prompt (Windows 10) and got the same result. I am the only user on this computer and I have all admin permissions. Is this virtualenv causing an issue? How do I remedy it?
EDIT: I was able to fix the permissions issue by leveraging the python executable like so:
python -m easy_install -U setuptools
But it didn't fix the python setup.py egg_info issue. I still get this error message when trying to pip install anything:
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-50tn0wlb\urllib\
I have tried both python -m pip install urllib and pip install urllib and neither work.

I had the same problem when trying to install urllib, but after doing a pip search urllib, I discovered that the problem was due to the version of urllib. From the search:
$ pip search urllib
...
> urllib5 (5.0.0) - Just increment the number and create a new lib. Never fix the original one.
At the end, a simple
pip install urllib5
within an elevated shell solved it.

Your problem has to do with permissions. The related/similar tools setup_tools, easy_install, and pip all tend to set a default set of permissions on files and folders they try to create in the package installation folder(s), rather than trying to match access permissions of the location they're installing in.
On Linux systems, where files and folders individually have permissions, this is frequently bypassed with the sudo command. On Windows, the equivalent is to run the installer as an Administrator. Since you're in the console, you have to open a console with Administrator privileges to run the pip command in.
Notable under Windows, the modules installed with pip from an Administrator console are still accessible to all users of the system that have the proper path in the PYTHONPATH system environment variable. Under Linux however, the problem is exacerbated by the fact that the files themselves may not be created with read and execute access for other users and may need to have their permissions manually modified after installation.
WARNING: urllib vs urllib2 vs urllibx
Both other answers claim that the problem is you're not specifying the correct "version" of the module in the call to pip. Neither is correct, as the error clearly indicates an installation folder access permissions violation causing the failure, but they also incorrectly recommended VERY unsafe behavior.
pip install urllib != pip install urllib5 these are two completely different packages.
The documentation for pip (https://packaging.python.org/tutorials/installing-packages/#id17) clearly says the way to specify a module version explicitly is pip install 'urllib==5'.
As part of how the package management engine implemented by pip works, running the command pip install urllib will always try to use the latest version of the urllib package, so you shouldn't need to specify the version unless you have some reason that you need a very specific version of the module.

There are two points to make in order to answer your question:
1. You are lucky you did not install that package!
The package you were trying to install was a maliciously created python package that was designed to look like a real package (in this case urllib3). If you had installed it, the package would have operated as normal except it would have sent some basic information about the system on which you installed the package to a URL (you can see more details on this here). You can read more about this fake package at either of the following links:
https://app.threatconnect.com/auth/incident/incident.xhtml?incident=5256822&owner=Common%20Community (you can sign up for a free account to view this one)
http://www.nbu.gov.sk/skcsirt-sa-20170909-pypi/index.html
Sending basic information about your systems to an unknown source isn't the worst thing you could do, but is certainly something you want to avoid when possible.
2. To properly install a package...
Specifically urllib:
To install urllib, you need to specify the version of the package you would like to install. For example, pip install urllib3.
Any package in general:
As #Elisabete Coelho suggested, you can use the pip search <package-name> feature to view the available packages. This is not perfect, however, as it may list malicious libraries like the one you were trying to install. A good guideline is that you should follow the installation instructions in a package's documentation closely to avoid any unforeseen issues. This is just an unfortunate necessity of living in a world where people make pretend python packages.

Installing (build) matplotlib in mac osx lion

I installed pynum and scipy (on osx Lion with python 2.7), but when I tried to build matplotlib
git clone https://github.com/matplotlib/matplotlib.git
cd matplotlib
python setup.py build
python setup.py install
I've got these errors:
/System/Library/Frameworks/ApplicationServices.framework/Frameworks/ATS.framework/Headers/ATSTypes.h:242: error: declaration for parameter ‘FMFontDirectoryFilter’ but no such parameter
src/_macosx.m:5912: error: expected ‘{’ at end of input
lipo: can't open input file: /var/folders/qw/pr2f7vq91b3c3ngkxrrqplm8zkv09r/T//ccAgEklo.out (No such file or directory)
error: command 'llvm-gcc-4.2' failed with exit status 1
Could somebody please tell me what is the problem?
P.S.
At first I tried to install it in this way:
pip install -e git+https://github.com/matplotlib/matplotlib#egg=matplotlib-dev
but it does not worked for me

I always find this process more painful than it should be, but I've done it a few times now and I believe that these steps should get you set up:
Get Xcode 4.3.2, it's required for some of the later steps.
Download the latest version of python for OSX from python.org
Grab the Scipy superpack.
Uninstall any previous versions of numpy/matplotlib/scipy that you currently have. That includes doing cd /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/ and moving any numpy/matplotlib/scipy directories or eggs into a temp directory.
cd ~/Downloads(or wherever you downloaded the superpack script to) and run sh install_superpack.sh. Answer no to the question are you installing from a repository cloned to this machine or you'll be confused about why the script keeps failing.
That should be it! You should now be able to boot up the python console and import numpy, scipy, matplotlib.