I am trying to install the mysqlclient Python package (https://pypi.python.org/pypi/mysqlclient) into a virtual Python 2.7 environment on Windows 7 (on a local PC, and on the Appveyor CI) and cannot get it done.
I am describing the issues for the local PC with Win 7, but it happens basically the same way on Appveyor.
Packages in my virtualenv when I start:
pip (9.0.1)
setuptools (36.6.0)
wheel (0.30.0)
pip install mysqlclient initially says:
. . .
running build_ext
building '_mysql' extension
error: Microsoft Visual C++ 9.0 is required. Get it from http://aka.ms/vcpython27
Apparently, Python 2.7 requires VC9. After installing it, pip install mysqlclient gets a little further, but misses include files:
. . .
building '_mysql' extension
creating build\temp.win-amd64-2.7
creating build\temp.win-amd64-2.7\Release
C:\Program Files (x86)\Common Files\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Dversion_info=(1,3,12,'final',0) -D__version__=1.3.12 "-IC:\Program Files (x86)\MySQL\MySQL Connector C 6.1\include" -IC:\Python27\include -IC:\Users\Andi\virtualenvs\mysqlclient\PC /Tc_mysql.c /Fobuild\temp.win-amd64-2.7\Release\_mysql.obj /Zl
_mysql.c
_mysql.c(29) : fatal error C1083: Cannot open include file: 'mysql.h': No such file or directory
error: command 'C:\\Program Files (x86)\\Common Files\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\amd64\\cl.exe' failed with exit status 2
In order to get the header files, I installed the "MySQL Connector/C 6.1.10" from https://downloads.mysql.com/archives/get/file/mysql-connector-c-6.1.10-winx64.msi. This is admittedly only the second latest version, but I need to install it on Appveyor from the command line, and did not find out how to download the latest version in an unattended manner.
Also, using the MSI installer installs the package in C:\Program Files\MySQL\MySQL Connector C 6.1 but the build step in pip install expects it in C:\Program Files (x86)\MySQL\MySQL Connector C 6.1, so I softlinked it:
mklink /D "C:\Program Files (x86)\MySQL\MySQL Connector C 6.1" "C:\Program Files\MySQL\MySQL Connector C 6.1"
When looking at the files in that version of the MySQL Connector/C, it has only lib/vs12 and lib/vs14 directories:
lib/vs12/mysqlclient.lib
lib/vs14/mysqlclient.lib
As expected, pip install mysqlclient now gets beyond the compiles, but fails when linking:
. . .
running build_ext
building '_mysql' extension
creating build\temp.win-amd64-2.7
creating build\temp.win-amd64-2.7\Release
C:\Program Files (x86)\Common Files\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Dversion_info=(1,3,12,'final',0) -D__version__=1.3.12 "-IC:\Program Files (x86)\MySQL\MySQL Connector C 6.1\include" -IC:\Python27\include -IC:\Users\Andi\virtualenvs\mysqlclient\PC /Tc_mysql.c /Fobuild\temp.win-amd64-2.7\Release\_mysql.obj /Zl
_mysql.c
_mysql.c(242) : .... several warnings ...
C:\Program Files (x86)\Common Files\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\link.exe /DLL /nologo /INCREMENTAL:NO "/LIBPATH:C:\Program Files (x86)\MySQL\MySQL Connector C 6.1\lib\vs9" /LIBPATH:C:\Python27\Libs /LIBPATH:C:\Users\Andi\virtualenvs\mysqlclient\libs /LIBPATH:C:\Users\Andi\virtualenvs\mysqlclient\PCbuild\amd64 /LIBPATH:C:\Users\Andi\virtualenvs\mysqlclient\PC\VS9.0\amd64 kernel32.lib advapi32.lib wsock32.lib mysqlclient.lib /EXPORT:init_mysql build\temp.win-amd64-2.7\Release\_mysql.obj /OUT:build\lib.win-amd64-2.7\_mysql.pyd /IMPLIB:build\temp.win-amd64-2.7\Release\_mysql.lib /MANIFESTFILE:build\temp.win-amd64-2.7\Release\_mysql.pyd.manifest /MANIFEST
LINK : fatal error LNK1181: cannot open input file 'mysqlclient.lib'
error: command 'C:\\Program Files (x86)\\Common Files\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\amd64\\link.exe' failed with exit status 1181
I did not find a version of the "MySQL Connector/C" package that can still be downloaded and that works with VC9. Using the mysqlclient.lib files in the vs12 or vs14 directories in this vc9 build results in unresolved symbols.
My questions are:
What package(s) do I need to use in order to get the right header files (e.g. mysql.h) and libraries (e.g. mysqlclient.lib) for a successful pip install mysqlclient on Windows 7?
More generally, what are the prerequisites for a successful pip install mysqlclient on Windows 7?
download mysqlclient from http://www.lfd.uci.edu/~gohlke/pythonlibs/#mysql-python
choose the version matching your python version, example mysqlclient‑1.4.2‑cp37‑cp37m‑win_amd64.whl, and then:
pip install wheel
pip install mysqlclient‑1.4.2‑cp37‑cp37m‑win_amd64.whl
Incidentally I ran into the same issue in July last year and only now finished an entry on my blog about it.
Basically, you're not going to like it, but you need to re-compile mysqlclient from scratch, because -- as you pointed out -- it's not available for Visual Studio 9 since a long time.
However, doing it right is somewhat tricky, because you need to fix some compilation issues that apparently aren't documented anywhere in Oracle's official documentation. There are also issues with the Python setup.py file, as it doesn't care about the platform architecture and always looks in the x86 directory. That's why you soft-linked the directory. There is an open issue on Github mentioning this.
So basically, using the VC++ for Python 2.7 you already installed:
Download the Oracle MySQL C Connector source code.
Download and install Microsoft Visual C++ Compiler for Python 2.7.
Use CMake to generate NMake Makefiles for the MySQL C Connector source code. Pay attention to environment variables, see this Github issue comment.
Compile the MySQL C Connector mysqlclient library using nmake mysqlclient.
Install the MySQL C Connector binary and add the compiled mysqlclient.
Run pip install mysqlclient again.
Step 4 is tricky, because Microsoft don't supply C99 stdint.h, inttypes.h and stdlib.h is missing lldiv_t. You can use this StackOverflow post to get stdint.h and inttypes.h and then manually define lldiv_t in decimal.h as:
typedef struct {
long long quot;
long long rem;
} lldiv_t;
You can also copy these headers from a newer Visual Studio, like 2013 and 2015. After that, as you had already figured out, you need to put the compiled mysqlclient.lib into C:\Program Files (x86)\MySQL\MySQL Connector C 6.1\lib\vs9.
Then issue another pip install mysqlclient and you're all done.
I am trying to install Scrapy via the project interpreter but I end with the error. Later I tried installing using pip install Scrapy but I end up with the same error. The error details are:
running build_ext
generating cffi module 'build\\temp.win-amd64-2.7\\Release\\_padding.c'
already up-to-date
generating cffi module 'build\\temp.win-amd64-2.7\\Release\\_constant_time.c'
already up-to-date
generating cffi module 'build\\temp.win-amd64-2.7\\Release\\_openssl.c'
already up-to-date
building '_openssl' extension
C:\Users\xxxx\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\Python27\include "-IC:\aaaa\PC" /Tcbuild\temp.win-amd64-2.7\Release\_openssl.c /Fobuild\temp.win-amd64-2.7\Release\build\temp.win-amd64-2.7\Release\_openssl.obj
_openssl.c
build\temp.win-amd64-2.7\Release\_openssl.c(434) : fatal error C1083: Cannot open include file: 'openssl/opensslv.h': No such file or directory
error: command 'C:\\Users\\xxxx\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\amd64\\cl.exe' failed with exit status 2
You could be missing Microsoft C++ for Python 2, since the file that cannot be opened is from it.
The download is here: https://www.microsoft.com/en-us/download/details.aspx?id=44266
I am using python 2.7 and I am trying to pip install imposm.parser.
The first thing I did was to install protobuf with pip install protobuf. That installed version 2.6.1 in C:\Python27\Lib\site-packages\google. Then I went here to download protoc-2.6.1-build2-windows-x86_32.exe which I renamed to protoc.exe and placed in C:\Python27\Lib\site-packages\google folder.
I then installed Microsoft Visual C++ 9.0 from here
Finally, I executed pip install imposm.parser and this is the error I get:
running build_ext
building 'imposm.parser.pbf.OSMPBF' extension
creating build\temp.win-amd64-2.7
creating build\temp.win-amd64-2.7\Release
creating build\temp.win-amd64-2.7\Release\imposm
creating build\temp.win-amd64-2.7\Release\imposm\parser
creating build\temp.win-amd64-2.7\Release\imposm\parser\pbf
C:\Users\Shiro\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -C:\Python27\include -IC:\Python27\PC /Tpimposm/parser/pbf/osm.cc Fobuild\temp.win-amd64-2.7\Release\imposm/parser/pbf/osm.obj
osm.cc
C:\Users\Shiro\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Include\xlocale(342) : warning C4530: C++ exception handler used, but unwind semantics are not enabled. Specify /EHsc
c:\users\shiro\appdata\local\temp\pip-build-jel4jl\imposm.parser\imposm\parser\pbf\osm.pb.h(9) : fatal error C1083: Cannot open include file: 'google/protobuf/stubs/common.h': No such file or directory
error: command '"C:\Users\Shiro\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe"' failed with exit status 2
Edit: The header header file saying that is missing is from protobuf which I just installed with pip install protobuf. The file missing is right here in the github google/protobuf/stubs/common.h shouldn't it exist on my system since I installed with pip ? I don't understand...
I am trying to build talib for 64 bit Python. I have a 64 bit Python installation (Using Anaconda 64 bit distribution). I installed Microsoft SDK and set the environment to x64 using the following command
"C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.cmd" /x64
but still no success. I have been struggling with this for past 2 days. Please help.
Setting SDK environment relative to C:\Program Files\Microsoft SDKs\Windows\v7.1
\.
Targeting Windows 7 x64 Debug
C:\ABC\Python\pkgs\ta-lib\ta-lib-TA_Lib-0.4.7\ta-lib-TA_Lib-0.4.7>C:\Anaconda64\
python.exe setup.py build --compiler msvc
running build
running build_py
running build_ext
skipping 'talib\common.c' Cython extension (up-to-date)
building 'talib.common' extension
C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\Bin\amd64\cl.exe /c /nolo
go /Ox /MD /W3 /GS- /DNDEBUG -IC:\Anaconda64\lib\site-packages\numpy\core\includ
e -Ic:\ta-lib\c\include -IC:\Anaconda64\include -IC:\Anaconda64\PC /Tctalib\comm
on.c /Fobuild\temp.win-amd64-2.7\Release\talib\common.obj
common.c
C:\Anaconda64\include\pyconfig.h(68) : fatal error C1083: Cannot open include fi
le: 'io.h': No such file or directory
error: command 'cl.exe' failed with exit status 2
C:\ABC\Python\pkgs\ta-lib\ta-lib-TA_Lib-0.4.7\ta-lib-TA_Lib-0.4.7>
Download TA_Lib-0.4.10-cp27-cp27m-win_amd64.whl from http://www.lfd.uci.edu/~gohlke/pythonlibs/#ta-lib.
And use command
pip install TA_Lib-0.4.10-cp27-cp27m-win_amd64.whl
I am trying to install the latest version of pymssql (1.0.2) in my Windows 2003 x64.
However, I am constantly getting this error:
running install
running build
running build_py
running build_ext
building '_mssql' extension
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG "-Ic:\Program Files\Microsoft SQL Server\80\Tool
s\DevTools\Include" -Ic:\mssql7\DevTools\Include -Id:\DEVEL\pymssql-DEVTOOLS\INCLUDE -Ic:\Python27\include -Ic:\Python27\PC /Tcmssqldbmodule.c /Fobuil
d\temp.win32-2.7\Release\mssqldbmodule.obj
mssqldbmodule.c
mssqldbmodule.c(50) : fatal error C1083: Cannot open include file: 'sqlfront.h': No such file or directory
error: command '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe"' failed with exit status 2
Any ideas? Isn't sqlfront.h included in the installation of visual studio 2k8 (or any visual studios)
Thank you!
I had the same problem with Visual Studio 2008, Windows 7 64 and Python 2.7 (32 bits).
These were the steps I followed to solved it:
Install Visual Studio 2008 (32 bits).
Install Visual Studio 6.
Install Python 2.7 (32 bits)
Download pymssql (the same package you downloaded)
Try to compile the source code (the same you did when you got the error, you will get it again).
At the same directory where setup.py is, you should find a file named mssqldbmodule.c, open it.
Change the lines #include <sqlfront.h> and #include <sqldb.h> To #include "sqlfront.h" and #include "sqldb.h" So the compiler will look for sqlfront.h and sqldb.h files locally.
Copy the files sqlfront.h and sqldb.h from: C:\Program Files (x86)\Microsoft Visual Studio\VC98\Include (Visual Studio 6 installation) To you local directory (where setup.py is).
Copy the file NTWDBLIB.LIB from C:\Program Files (x86)\Microsoft Visual Studio\VC98\Lib
To your local directory.
Run again setup.py install
This way you will compile the library using the deprecated NTWDBLIB which is the one used by pymssql.
The directory <pymssql root>\freetds\vs2008_32\ (or <pymssql root>\freetds\vs2008_64\ if you use 64bits python) has not been set for header files and libs yet I think.
About freetds, please check the "Building FreeTDS" part of README_building_and_developing.rst.