Spyder does not recognise cython modules when importing in a script - python

I have got a cython script example1.pyx with various functions. I am trying to use one of these functions in my python script. To do this I am simply trying to import the function from the cython module as follows:
from example1 import myfunction1
I am using Spyder to run my python script. When I run it I get the following error
ModuleNotFoundError: No module named 'example1'
for some reason, .pyx extension scripts are not recognised as modules that can be imported. I can successfully import other python scripts, but no luck with cython .pyx scripts. Turns out I wasnot compiling the python script. So I did so using the following script:
from setuptools import setup, Extension
import numpy
from Cython.Distutils import build_ext as _build_ext
list_pyx = ['cydtw', 'cygak', 'cycc', 'soft_dtw_fast']
ext = [Extension(
'%s' % s, ['%s.pyx' % s], include_dirs=[numpy.get_include()]
) for s in list_pyx]
setup(
include_dirs=[numpy.get_include()],
ext_modules=ext,
cmdclass={'build_ext': _build_ext},
)
when I ran this, I got the error saying I need Visual C++ 14.0 compiler. Looking at documentation in https://wiki.python.org/moin/WindowsCompilers, Visual Studio 15 which I have installed already should have Visual C++ 14.0 so that was confusing. So I updated my setuptools. Got rid of that error, but turns out I it cant find a file path C;\program files\Microsoft SDKs\Windows\v8.1\lib. which I can see it doesn't. So that's where I am stuck now. Any clues?

Managed to resolve the issue. Didn't have the Visual C++ compiler installed on Visual Studio 2015.
To check, Open up Visual Studio 15.
Look at Templates under Visual C++.
I could see Visual C++ 2015 Tools for Windows Desktop which needed installing.
This includes tools and libraries for Windows Desktop, which includes the compiler, the Universal CRT and the windows 8.1 SDK. Ran the installation. It will ask you close visual studio in the meantime.
Was able to compile the cython scripts as well as call the functions in the script in my python project.
What I would say you have to be careful about your compiler version and python version matching up. There can be compatibility issues.
Details on Visual Studio Build Tools are available here:
https://landinghub.visualstudio.com/visual-cpp-build-tools
For installing non-pure python packages or compile a cython or pyrex file, the details in terms of compatibility are here:
https://wiki.python.org/moin/WindowsCompilers

Related

Cython: error while building extension: Microsoft Visual C++ 14.0 or greater is required

Short Description:
I'm trying to build an example cython script, but when I run the python setup.py build_ext --inplace command, I get an error saying that I need MS Visual C++ version 14.0 or greater. I've tried a lot of the things on related SO threads and other forums but to no avail in resolving the issue.
Longer Description:
The specific cython script:
test.pyx:
cpdef int test(int n):
cdef int sum_ = 0, i = 0
while i < n:
sum_ += i
i += 1
return sum_
setup.py:
# from setuptools import setup
from distutils.core import setup
from Cython.Build import cythonize
setup(
name = "test",
ext_modules = cythonize('test.pyx'), # accepts a glob pattern
)
I'm on python 3.10.0 and cython 0.29.30 and am using Windows 10
And here is the error that I get:
C:\Users\LENOVO PC\PycharmProjects\MyProject\cython_src>py setup.py build_ext --inplace
Compiling test.pyx because it changed.
[1/1] Cythonizing test.pyx
C:\Users\LENOVO PC\AppData\Local\Programs\Python\Python310\lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\LENOVO PC\PycharmProjects\MyProject\cython_src\test.pyx
tree = Parsing.p_module(s, pxd, full_module_name)
running build_ext
building 'test' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
C:\Users\LENOVO PC\PycharmProjects\MyProject\cython_src>
I've tried numerous different things:
Visited the link in the error and downloaded and installed the build tools
Installed multiple versions of Visual Studio (2022, 2019, 2017) CE and Build Tools
Uninstalled all of the above and reinstalling MSVC 2019 CE and Build Tools from scratch
Browsed through a lot of other related SO threads about this error and none of the solutions presented in them have worked for me so far, they have broadly included:
Building the script from the developer console
Updating setuptools
Installing numerous different components in MSVC
Installing numerous vc redistributables
But none of these have worked for me unfortunately, and I keep getting the same error.
I personally think the cause might be related to missing registry keys, or missing path variables, because the MSVC tools are definitely installed on my machine, but the setup script is unable to find them, but I do not know how to find out for sure.
Some additional info that might be relevant(?):
I've used Cython on the same machine before, and it used to work just fine, I had Visual Studio 2019 at this time. At some point though, I uninstalled it and upgraded to Visual Studio 2022 because I was learning C++ and wanted to use a newer C++ standard. Oddly enough, when I did this, the IDE that I use for C++ (CLion) stopped detecting the MSVC toolchain as well, and I never got it to correctly detect it again (I've been using WSL toolchain on CLion since)
Recently when I tried to use Cython again and got this error, and did a lot of digging, I realised that the two incidents might be related, so I thought that it may be worth mentioning here.
Both the main python issue and the secondary CLion thing that I mentioned were resolved with this one solution (the issues were connected after all!)
Clear the registry key that is mentioned in this SO thread:
https://stackoverflow.com/a/64389979/15379178
This error had nothing to do with python (sort of) or msvc, in short, an anaconda installation had left an invalid path in my cmd's auto run regkey and it was causing a "The system cannot find the path specified" error and despite it being unrelated to python or msvc it was causing the build to fail. I am so glad I finally figured this out!
I had the same issue as you (Cython: error while building extension).
I installed the latest Visual C++ but it didn't work,
and I tried to follow your solution, but there was no registry key that is mentioned from the link in my registry. (I guess it's because I didn't have Visual Studio installed before.)
Finally I installed MS C++ Build Tools which is mentioned in the error message, and it worked for me. (The reason I didn't consider this as the first option was because the installation required a large space.)
I know it didn't work for your case, but I'm leaving it for those like me who will search this post with the same error.

install METIS library for python3 on windows7

I want to install METIS for python3.6 on windows7.I did the following steps:
1-Download the source (tar.gz or zip file) from
https://pypi.python.org/pypi/networkx-metis/
2-Unpack and change directory to the source directory (it should have the
setup.py on top level).
3-Run
python setup.py build
to build
in this step, I encountered the following error:
error: Microsoft Visual C++ 14.0 is required.
i installed "Visual Studio Tools 2017", Then I run the code again. I encountered the following error:
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio
14.0\\VC\\BIN\\cl.exe' failed with exit status 2
I would be very grateful if you answer my questions.
Although what I used was Windows 10, perhaps the same steps as I laid out here may work for you.
First, you should get the python metis wrapper using pip: pip install metis.
Second, you must install conda-metis, which you can find here. Note, it does not require you to install Anaconda to work.
Place the files in conda-metis-master in some file path.
The installation requires some fixes. Make sure you have a recent version of Visual Studios (I used 2017). For me, VS had a problem running the instructions in BUILD-WINDOWS.txt, which landed me on this thread:
Why does MSBuild look in C:\ for Microsoft.Cpp.Default.props instead of c:\Program Files (x86)\MSBuild? ( error MSB4019)
Specifically, the answer For Visual Studio 2017 and 2019 on Windows 10 was what I went with. It could potentially be different for you since you are on Windows 7.
For the python metis package, we need to generate a metis.dll, not a metis.lib file, so we must also look closer at the CMakeLists.txt (see the Linux version of this discussion here). We add on line 19: set(METIS_LIBRARY_TYPE SHARED).
You should follow the BUILD-WINDOWS.txt instructions, but run .\vsgen -G "Visual Studio 15 2017 Win64" inside your conda-metis file path in command prompt with full permissions instead of using Visual Studio 10.
Before moving on to using the files that were generated by the above command in path_to_your_metis_dir\build\windows\, we need another fix. Following what was said in:
rint() issue after creating VS Project using CMake
we have to edit the file path_to_your_metis_dir\GKlib\gk_arch.h by removing the line: #define rint(x) ((idx_t)((x)+0.5)). (Or the fix listed on this github exchange.)
Then go to path_to_your_metis_dir\build\windows\.
Open METIS.sln in Visual Studios, go to the top to Build and from the scroll-down go to Build Solution. Afterwards, the .dll file will be in path_to_your_metis_dir\build\windows\libmetis\Release\.
Last, we run in command prompt:
set METIS_DLL=path_to_your_metis_dir\build\windows\libmetis\Release\metis.dll
After this, python metis no longer failed when I would import it into my code with the step from metis import *.

Problems finding vcvarsall.bat when using distutils

So this problem started out when I tried building an extension with Cython using the following setup.py script.
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext': build_ext},
ext_modules = [
Extension("program1",["program1.pyx"],),])
When I try build it using:
python setup.py build I get the following error:
error: Unable to find vcvarsall.bat
I am using the Microsoft Visual C++ Compiler Package for Python 2.7. After a bit a research, some other SO questions led me to believe that my PATH variables weren't properly set.
I set it as follows:
VS90COMNTOOLS=C:\...\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\;
which is where my vcvarsall.bat file is located. This didn't solve my problem as I was still getting the same error about vcvarsall.bat.
I know there's already a bunch of posts about this particular error already but I still haven't found a solution.
I found this discussion specifically about my problem however I don't understand the following steps:
1) Enter MSVC for Python command prompt (how do I do this?)
2) SET DISTUTILS_USE_SDK=1
3) SET MSSdk=1
4) python.exe setup.py ...
Finally: further down in the discussion Steve proposes to replace the
from distutils.core import setup
with
from setuptools import setup
This worked! However now I have a problem, whenever I try to run something that uses distutils setup instead of setuptools, it raises the same "cannot find vcvarsall.bat" problem.
For example, I tried using the IPython magic command
%%cython_inline
I've been at this for a couple days now and all I want is to properly fix the issue for good, it's been very frustrating.
A patch was made recently which is supposed to fix the issue with finding vcvarsall.bat. You can see that here. Do I just run the script/functions inside it? It didn't seem to change anything in my distutils\msvc9compiler.py file.
I hope someone can point me in the right direction here.
There has been further discussions on the python bug tracker here. It's not an ideal fix but Mark Lawrence suggests a rather simple hack that works. Here's what was suggested:
My work around for this was to edit the distutils msvc9compiler.py file.
I commented out the line vcvarsall = find_vcvarsall(version) in the function
query_vcvarsall and hard coded vcvarsall = r'C:\Program Files\Microsoft
Visual Studio 9.0\VC\vcvarsall.bat'. This assumes that you've got a copy of
Visual C++!:) HTH.
The msvc9compiler.py file is located in C:\Python27\Lib\distutils (or somewhere else if you installed Python in a different location than the default). The only change I made is pointing the vcvarsall.bat file to my specific MSVC compiler.
In my case, since I used the Microsoft Visual C++ Compiler Package for Python 2.7 to compile Python, I changed the variable as follows:
vcvarsall =
r'C:\Users\..\AppData\Local\Programs\Common\Microsoft\Visual C++ for
Python\9.0\vcvarsall.bat' #where .. is my user account name
From this message on the Python bug tracker
The correct solution to this issue now is to install the Microsoft supplied "Visual C++ Compiler for Python 2.7" package (available as a free download from MSDN). That has all the components needed to build 32 and 64-bit extensions. Your setup.py needs to use setuptools (as there is a setuptools extension to recognise this compiler package) but otherwise the process should be seamless.

Cython with python 3.3

I have been using python 3.3
This is an old problem as I searched, and this is what I did:
helloworld.pyx
print("Hello world!")
Then, in ipython, I did:
import pyximport; pyximport.install()
import helloworld
It says:
ImportError: Building module helloworld failed: ["ValueError:
['path']\n"]
The same problem did not happen with python 2.7
I googled this: https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows
and realized that I have to install Windows SDK for Windows 7 and .NET Framework 4. As it comes with the VC++2010 Redistributables, I did not install the redistributables alone again. I thought I had everything ready, but the import error still remains.
Could anyone please help me solve it?
Thank you!
-Shawn
install Windows 7 SDK, then open cmd run:
"C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.Cmd" /Release /x64
then run:
python setup.py install
I was having the same issue and the same environment (win7 64bit, python-3.3.3 64bit).
I have
installed the Windows 7 SDK as described on the wiki cython wiki as you did,
applied the patch (msvccompiler9_33.diff) from this python bug,
fixed the above patch by defining a missing variable as described here
Installed MS Visual C++ Express 2010 (seems to be required)
Updated the Windows 7 SDK to include the "Windows Headers and Libraries", "Tools" and of course the "Visual C++ Compilers" the MSVC++2010 redistributable.
And now I can compile and import the helloworld.py just fine.

Error: "MSVCP90.dll: No such file or directory" even though Microsoft Visual C++ 2008 Redistributable Package is installed

I'm trying to build a package from source by executing python setup.py py2exe
This is the section of code from setup.py, I suppose would be relevant:
if sys.platform == "win32": # For py2exe.
import matplotlib
sys.path.append("C:\\Program Files\\Microsoft Visual Studio 9.0\\VC\\redist\\x86\\Microsoft.VC90.CRT")
base_path = ""
data_files = [("Microsoft.VC90.CRT", glob.glob(r"C:\Program Files\Microsoft Visual Studio 9.0\VC\redist\x86\Microsoft.VC90.CRT\*.*")),
Error it shows:
*** finding dlls needed ***
error: MSVCP90.dll: No such file or directory
But I've installed "Microsoft Visual C++ 2008 Redistributable Package". I'm running 32-bit python on 64-bit Windows 8. I'm trying to build a 32-bit binaries.
Also there is no folder like this: "C:\Program Files\Microsoft Visual Studio 9.0\VC\redist\". This is what my computer contains:
EDIT:
On searching for msvcp90.dll on my C:\ drive I found that they are installed in weird paths like this:
I would recommend ignoring the dependency outright. Add MSVCP90.dll to the list of dll_excludes given as an option to py2exe. Users will have to install the Microsoft Visual C++ 2008 redistributable. An example:
setup(
options = {
"py2exe":{
...
"dll_excludes": ["MSVCP90.dll", "HID.DLL", "w9xpopen.exe"],
...
}
},
console = [{'script': 'program.py'}]
)
(new answer, since the other answer describes an alternate solution)
You can take the files from the WinSxS directory and copy them to the C:\Program Files\Microsoft Visual Studio 9.0\VC\redist\x86\Microsoft.VC90.CRT directory (normally created by Visual Studio, which you don't have). Copy them to get the following structure:
+-Microsoft.VC90.CRT
| |
| +-Microsoft.VC90.CRT.manifest
| +-msvcm90.dll
| +-msvcp90.dll
| +-msvcr90.dll
Then, you should be able to run the setup program (still excluding msvcp90.dll, as in the other answer), and it should successfully find the files under Microsoft.VC90.CRT and copy them as data files to your bundle.
See the py2exe tutorial for more information.
I used to have a huge number of problems with complication on Windows, like the issue you're facing as well as installing packages like Cython with pip install cython.
The solution that worked best for me after two weeks of pain was downloading and running the unofficial MinGW GCC binary for Windows provided here. You might want to try giving that a shot and seeing if it helps.
If you do do it, you might want to uninstall MinGW if you have it already. I don't know if that's strictly necessary, but I always did it just in case. I did have it installed side-by-side with Cygwin without any problems.
I think it has something to do with the spaces in the directory. You should try using .rstrip(). For example, put this:
directory='C:\Program Files\Microsoft Visual Studio 9.0\VC\redist\x86\Microsoft.VC90.CRT'
directory=directory.rstrip()
You can then use the variable directory like you would have used the actual path.
This should make python able to recognize the directory where it wouldn't be able to decipher it before.

Categories