So this problem started out when I tried building an extension with Cython using the following setup.py script.
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [
Extension("program1",["program1.pyx"],),])
When I try build it using:
python setup.py build I get the following error:
error: Unable to find vcvarsall.bat
I am using the Microsoft Visual C++ Compiler Package for Python 2.7. After a bit a research, some other SO questions led me to believe that my PATH variables weren't properly set.
I set it as follows:
VS90COMNTOOLS=C:\...\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\;
which is where my vcvarsall.bat file is located. This didn't solve my problem as I was still getting the same error about vcvarsall.bat.
I know there's already a bunch of posts about this particular error already but I still haven't found a solution.
I found this discussion specifically about my problem however I don't understand the following steps:
1) Enter MSVC for Python command prompt (how do I do this?)
2) SET DISTUTILS_USE_SDK=1
3) SET MSSdk=1
4) python.exe setup.py ...
Finally: further down in the discussion Steve proposes to replace the
from distutils.core import setup
with
from setuptools import setup
This worked! However now I have a problem, whenever I try to run something that uses distutils setup instead of setuptools, it raises the same "cannot find vcvarsall.bat" problem.
For example, I tried using the IPython magic command
%%cython_inline
I've been at this for a couple days now and all I want is to properly fix the issue for good, it's been very frustrating.
A patch was made recently which is supposed to fix the issue with finding vcvarsall.bat. You can see that here. Do I just run the script/functions inside it? It didn't seem to change anything in my distutils\msvc9compiler.py file.
I hope someone can point me in the right direction here.
There has been further discussions on the python bug tracker here. It's not an ideal fix but Mark Lawrence suggests a rather simple hack that works. Here's what was suggested:
My work around for this was to edit the distutils msvc9compiler.py file.
I commented out the line vcvarsall = find_vcvarsall(version) in the function
query_vcvarsall and hard coded vcvarsall = r'C:\Program Files\Microsoft
Visual Studio 9.0\VC\vcvarsall.bat'. This assumes that you've got a copy of
Visual C++!:) HTH.
The msvc9compiler.py file is located in C:\Python27\Lib\distutils (or somewhere else if you installed Python in a different location than the default). The only change I made is pointing the vcvarsall.bat file to my specific MSVC compiler.
In my case, since I used the Microsoft Visual C++ Compiler Package for Python 2.7 to compile Python, I changed the variable as follows:
vcvarsall =
r'C:\Users\..\AppData\Local\Programs\Common\Microsoft\Visual C++ for
Python\9.0\vcvarsall.bat' #where .. is my user account name
From this message on the Python bug tracker
The correct solution to this issue now is to install the Microsoft supplied "Visual C++ Compiler for Python 2.7" package (available as a free download from MSDN). That has all the components needed to build 32 and 64-bit extensions. Your setup.py needs to use setuptools (as there is a setuptools extension to recognise this compiler package) but otherwise the process should be seamless.
Related
Short Description:
I'm trying to build an example cython script, but when I run the python setup.py build_ext --inplace command, I get an error saying that I need MS Visual C++ version 14.0 or greater. I've tried a lot of the things on related SO threads and other forums but to no avail in resolving the issue.
Longer Description:
The specific cython script:
test.pyx:
cpdef int test(int n):
cdef int sum_ = 0, i = 0
while i < n:
sum_ += i
i += 1
return sum_
setup.py:
# from setuptools import setup
from distutils.core import setup
from Cython.Build import cythonize
setup(
name = "test",
ext_modules = cythonize('test.pyx'), # accepts a glob pattern
)
I'm on python 3.10.0 and cython 0.29.30 and am using Windows 10
And here is the error that I get:
C:\Users\LENOVO PC\PycharmProjects\MyProject\cython_src>py setup.py build_ext --inplace
Compiling test.pyx because it changed.
[1/1] Cythonizing test.pyx
C:\Users\LENOVO PC\AppData\Local\Programs\Python\Python310\lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\LENOVO PC\PycharmProjects\MyProject\cython_src\test.pyx
tree = Parsing.p_module(s, pxd, full_module_name)
running build_ext
building 'test' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
C:\Users\LENOVO PC\PycharmProjects\MyProject\cython_src>
I've tried numerous different things:
Visited the link in the error and downloaded and installed the build tools
Installed multiple versions of Visual Studio (2022, 2019, 2017) CE and Build Tools
Uninstalled all of the above and reinstalling MSVC 2019 CE and Build Tools from scratch
Browsed through a lot of other related SO threads about this error and none of the solutions presented in them have worked for me so far, they have broadly included:
Building the script from the developer console
Updating setuptools
Installing numerous different components in MSVC
Installing numerous vc redistributables
But none of these have worked for me unfortunately, and I keep getting the same error.
I personally think the cause might be related to missing registry keys, or missing path variables, because the MSVC tools are definitely installed on my machine, but the setup script is unable to find them, but I do not know how to find out for sure.
Some additional info that might be relevant(?):
I've used Cython on the same machine before, and it used to work just fine, I had Visual Studio 2019 at this time. At some point though, I uninstalled it and upgraded to Visual Studio 2022 because I was learning C++ and wanted to use a newer C++ standard. Oddly enough, when I did this, the IDE that I use for C++ (CLion) stopped detecting the MSVC toolchain as well, and I never got it to correctly detect it again (I've been using WSL toolchain on CLion since)
Recently when I tried to use Cython again and got this error, and did a lot of digging, I realised that the two incidents might be related, so I thought that it may be worth mentioning here.
Both the main python issue and the secondary CLion thing that I mentioned were resolved with this one solution (the issues were connected after all!)
Clear the registry key that is mentioned in this SO thread:
https://stackoverflow.com/a/64389979/15379178
This error had nothing to do with python (sort of) or msvc, in short, an anaconda installation had left an invalid path in my cmd's auto run regkey and it was causing a "The system cannot find the path specified" error and despite it being unrelated to python or msvc it was causing the build to fail. I am so glad I finally figured this out!
I had the same issue as you (Cython: error while building extension).
I installed the latest Visual C++ but it didn't work,
and I tried to follow your solution, but there was no registry key that is mentioned from the link in my registry. (I guess it's because I didn't have Visual Studio installed before.)
Finally I installed MS C++ Build Tools which is mentioned in the error message, and it worked for me. (The reason I didn't consider this as the first option was because the installation required a large space.)
I know it didn't work for your case, but I'm leaving it for those like me who will search this post with the same error.
I am currently running a source code in python which uses METIS. I wanted to run it using Jetbrains PyCharm, conda interpreter and these are installed on windows 10. Although I have installed METIS using conda, I could not enable shared libraries and I faced this error:
ModuleNotFoundError: No module named 'metis'
When I tried to run it without conda I faced the following error although I have added the metis.dll location to the environment variables.
RuntimeError: Could not load METIS dll
Could anyone please help me about it?
This question is old, but I ran into the same problem and after going down the rabbit's hole for.. too long... I have an answer that worked for me.
First, you should get the python metis wrapper using pip: pip install metis.
Second, You must install conda-metis, which you can find here. Although pip calls the metis python wrapper metis, it just the metis wrapper and does not have metis itself.
Place the files in conda-metis-master in some file path.
The installation requires some fixes. Make sure you have a recent version of Visual Studios (I used 2017). For me, VS had a problem running the instructions in BUILD-WINDOWS.txt, which landed me on this thread:
Why does MSBuild look in C:\ for Microsoft.Cpp.Default.props instead of c:\Program Files (x86)\MSBuild? ( error MSB4019)
Specifically, the answer For Visual Studio 2017 and 2019 on Windows 10 was what I went with. Run:
set VCTargetPaths=C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets
in command prompt with full permissions to set the environment variable.
Also go to Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSBuild\ in regedit and change in 4.0 the variable MSBuildOverrideTasksPath and in ToolsVersion\4.0\ the variable MSBuildToolPath to C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin.
For the python metis, we need a .dll, not .lib[1], so we must also look closer at the CMakeLists.txt (see the Linux version of this discussion here). We add on line 19: set(METIS_LIBRARY_TYPE SHARED).
You should follow the BUILD-WINDOWS.txt instructions, but run .\vsgen -G "Visual Studio 15 2017 Win64" inside your conda-metis file path in command prompt with full permissions instead of using Visual Studio 10.
Wait! We need another fix before following the instructions further on using VS to build the the library. Following what was said in:
rint() issue after creating VS Project using CMake
we have to edit the file path_to_your_metis_dir\GKlib\gk_arch.h by removing the line: #define rint(x) ((idx_t)((x)+0.5)). (Or the fix listed on this github exchange.)
Then go to path_to_your_metis_dir\build\windows\.
Open METIS.sln in Visual Studios, go to the top to Build and from the scroll-down go to Build Solution. Afterwards, the .dll file will be in path_to_your_metis_dir\build\windows\libmetis\Release\.
Last, we run in command prompt:
set METIS_DLL=path_to_your_metis_dir\build\windows\libmetis\Release\metis.dll
After this, python metis no longer failed on the step from metis import * for me.
[1] If we could use .lib, then conda install -c conda-forge metis would work to get this file, but setting METIS_DLL to the .lib file leads to a windows error.
After getting the error described by #XiaojianChen , this is how I solved it (VS Studio 2022). Continuing after the above fixes, to generate a .dll file after opening up the VS solution file, go to the Solution Explorer, right click on the metis project file -> properties.
In General Properties, change the configuration type to Dynamic Library (.dll)
Screenshot of General Properties window
Under Advanced Properties, change the Target File Extension from .lib to .dll
Screenshot of Advanced Properties window
Since we only need the metis.dll file, right click the metis project file in the Solution Explorer and click build. The .dll can be found in ..\build\windows\libmetis\Release.
I have got a cython script example1.pyx with various functions. I am trying to use one of these functions in my python script. To do this I am simply trying to import the function from the cython module as follows:
from example1 import myfunction1
I am using Spyder to run my python script. When I run it I get the following error
ModuleNotFoundError: No module named 'example1'
for some reason, .pyx extension scripts are not recognised as modules that can be imported. I can successfully import other python scripts, but no luck with cython .pyx scripts. Turns out I wasnot compiling the python script. So I did so using the following script:
from setuptools import setup, Extension
import numpy
from Cython.Distutils import build_ext as _build_ext
list_pyx = ['cydtw', 'cygak', 'cycc', 'soft_dtw_fast']
ext = [Extension(
'%s' % s, ['%s.pyx' % s], include_dirs=[numpy.get_include()]
) for s in list_pyx]
setup(
include_dirs=[numpy.get_include()],
ext_modules=ext,
cmdclass={'build_ext': _build_ext},
)
when I ran this, I got the error saying I need Visual C++ 14.0 compiler. Looking at documentation in https://wiki.python.org/moin/WindowsCompilers, Visual Studio 15 which I have installed already should have Visual C++ 14.0 so that was confusing. So I updated my setuptools. Got rid of that error, but turns out I it cant find a file path C;\program files\Microsoft SDKs\Windows\v8.1\lib. which I can see it doesn't. So that's where I am stuck now. Any clues?
Managed to resolve the issue. Didn't have the Visual C++ compiler installed on Visual Studio 2015.
To check, Open up Visual Studio 15.
Look at Templates under Visual C++.
I could see Visual C++ 2015 Tools for Windows Desktop which needed installing.
This includes tools and libraries for Windows Desktop, which includes the compiler, the Universal CRT and the windows 8.1 SDK. Ran the installation. It will ask you close visual studio in the meantime.
Was able to compile the cython scripts as well as call the functions in the script in my python project.
What I would say you have to be careful about your compiler version and python version matching up. There can be compatibility issues.
Details on Visual Studio Build Tools are available here:
https://landinghub.visualstudio.com/visual-cpp-build-tools
For installing non-pure python packages or compile a cython or pyrex file, the details in terms of compatibility are here:
https://wiki.python.org/moin/WindowsCompilers
I'm trying to build a package from source by executing python setup.py py2exe
This is the section of code from setup.py, I suppose would be relevant:
if sys.platform == "win32": # For py2exe.
import matplotlib
sys.path.append("C:\\Program Files\\Microsoft Visual Studio 9.0\\VC\\redist\\x86\\Microsoft.VC90.CRT")
base_path = ""
data_files = [("Microsoft.VC90.CRT", glob.glob(r"C:\Program Files\Microsoft Visual Studio 9.0\VC\redist\x86\Microsoft.VC90.CRT\*.*")),
Error it shows:
*** finding dlls needed ***
error: MSVCP90.dll: No such file or directory
But I've installed "Microsoft Visual C++ 2008 Redistributable Package". I'm running 32-bit python on 64-bit Windows 8. I'm trying to build a 32-bit binaries.
Also there is no folder like this: "C:\Program Files\Microsoft Visual Studio 9.0\VC\redist\". This is what my computer contains:
EDIT:
On searching for msvcp90.dll on my C:\ drive I found that they are installed in weird paths like this:
I would recommend ignoring the dependency outright. Add MSVCP90.dll to the list of dll_excludes given as an option to py2exe. Users will have to install the Microsoft Visual C++ 2008 redistributable. An example:
setup(
options = {
"py2exe":{
...
"dll_excludes": ["MSVCP90.dll", "HID.DLL", "w9xpopen.exe"],
...
}
},
console = [{'script': 'program.py'}]
)
(new answer, since the other answer describes an alternate solution)
You can take the files from the WinSxS directory and copy them to the C:\Program Files\Microsoft Visual Studio 9.0\VC\redist\x86\Microsoft.VC90.CRT directory (normally created by Visual Studio, which you don't have). Copy them to get the following structure:
+-Microsoft.VC90.CRT
| |
| +-Microsoft.VC90.CRT.manifest
| +-msvcm90.dll
| +-msvcp90.dll
| +-msvcr90.dll
Then, you should be able to run the setup program (still excluding msvcp90.dll, as in the other answer), and it should successfully find the files under Microsoft.VC90.CRT and copy them as data files to your bundle.
See the py2exe tutorial for more information.
I used to have a huge number of problems with complication on Windows, like the issue you're facing as well as installing packages like Cython with pip install cython.
The solution that worked best for me after two weeks of pain was downloading and running the unofficial MinGW GCC binary for Windows provided here. You might want to try giving that a shot and seeing if it helps.
If you do do it, you might want to uninstall MinGW if you have it already. I don't know if that's strictly necessary, but I always did it just in case. I did have it installed side-by-side with Cygwin without any problems.
I think it has something to do with the spaces in the directory. You should try using .rstrip(). For example, put this:
directory='C:\Program Files\Microsoft Visual Studio 9.0\VC\redist\x86\Microsoft.VC90.CRT'
directory=directory.rstrip()
You can then use the variable directory like you would have used the actual path.
This should make python able to recognize the directory where it wouldn't be able to decipher it before.
Um, I'm stumped and really stuck (+_+) here after googling many sites. I'm using mingw32-gcc.exe from Dev-c++ compiler and currently following instructions from this site.
Every time I try here's the result:
C:\Python26>python c:\python26\python-mcrypt-1.1\setup.py install
running install
running build
running build_ext
error: don't know how to compile C/C++ code on platform 'nt' with 'mingw32-gcc.exe' compiler
I've edited the distuils.cfg file in my c:\python26\lib\distuils\distuils.cfg with this parameter
[build]
compiler = mingw32-gcc.exe
Any ideas would be appreciated!
You installed python outside of a compilation environment valids one include cygwin or mingw32 (wich is used by Dev-C++).
To use the command-line mode, you need to find the your compiler installation and find bat script which launch the shell with the proper parameters (This is needed for bith cygwin and mingw).
The alternative is to use Microsft solution for that problem.