We Keep Coding

How do I build a C Python Extension for Windows using Cygwin64?

To preface: my code works as I expect when compiling and running on Linux. However, this library needs to be compiled for use on a Windows machine. I looked in to a couple different options, and decided that using Cygwin to compile for Windows seemed to be the correct choice. I'm using a setup.py file with the distutils.core library and compiling using python setup.py install. When compiling on Windows in Cygwin, it fails to find pthread.h, arpa/inet.h, netinet/in.h, and sys/socket.h. I was under the impression that Cygwin came prepackaged with these headers, which is why I chose to use it. The alternative to Cygwin is putting preprocessor commands everywhere and using Windows specific libraries such as winsock2.h, which I want to avoid if at all possible. Is it possible to compile for Windows using Cygwin? If so, what have I done wrong to cause Cygwin to not recognize these headers?

You need to install the proper headers
$ cygcheck -p usr/include/pthread.h
Found 9 matches for usr/include/pthread.h
cygwin-devel-3.0.7-1 - cygwin-devel: Core development files
..
cygwin-devel-3.1.6-1 - cygwin-devel: Core development files
...
so install the cygwin-devel package
To check all the shared libraries needed by the built dll, you can use cygcheck
$ cygcheck /usr/lib/python3.8/site-packages/Cython/Compiler/FlowControl.cpython-38-x86_64-cygwin.dll
D:\cygwin64\lib\python3.8\site-packages\Cython\Compiler\FlowControl.cpython-38-x86_64-cygwin.
dll
D:\cygwin64\bin\cygwin1.dll
C:\WINDOWS\system32\KERNEL32.dll
C:\WINDOWS\system32\ntdll.dll
C:\WINDOWS\system32\KERNELBASE.dll
D:\cygwin64\bin\libpython3.8.dll
D:\cygwin64\bin\cygintl-8.dll
D:\cygwin64\bin\cygiconv-2.dll
D:\cygwin64\bin\cyggcc_s-seh-1.dll
As was built with Cygwin Python, you need also to transfer the cygwin python...

Most important, I think, is to follow the instructions in the Python help or on the Python doc web site for "Extending and Embedding the Python Interpreter" for the version you are building the extension for. For windows, the build instructions identify the build environment used to create the binary package that you download from python.org, usually something like VS2013 or VS2017. (As an aside, I think the Community editions have everything you need, and I don't think you actually have to use the Visual Studio GUI when you build using nmake from the CMD.EXE terminal.)
To build in Cygwin for use in a Windows version of Python, you may need to install and then use the x86_64-w64-mingw32-gcc, etc., cygwin packages to cross-compile non-cygwin (i.e. pure windows) executables and DLLs from Cygwin.
Binary extensions must be built using the source tree for a specific Python major.minor version, and bitness. For windows, you will need to build multiple versions of the extension, one for each major.minor, bitness version of Python that will import it, e.g. 3.6, 3.7, 3.8, 3.9, 32-bit, 64-bit. The extension code may not require changes between versions, but it still needs to be compiled with the right compiler and linked against exactly the same shared libraries (in this case .DLL files) as used by the Python executable. For instance, it must use exactly the same version of Microsoft's C run time library DLL as the Python executable does. This is a bit more sensitive and restrictive than on Linux, where you can rebuild the python executable and your extension with the same toolchain from your distro more easily.

Installing python package without a C++ compiler

I'm trying to get the HDBSCAN package to run on a Windows 7 machine with no C++ compiler. Installing a compiler is not an option, unfortunately.
I read that some packages have precompiled wheel files that require no compiler to install. The installation notes state "Binary wheels for a number of platforms are available thanks to the work of Ryan Helinski". However, there is no mention of where those can be found. My questions are then:
How do I obtain the .whl file for the HDBSCAN package?
Is it possible to simply compile+install on another machine and copy it? If so, what should the machine on which I compile have in common with the one on which the code must run? Can I do it on a Windows 10 machine, or does it have to be Win7 as well, do the same Windows updates need to have been run on both, etc?

Looks like there is no pre-built wheel distribution published by the maintainers of the project themselves on PyPI.
As already mentioned by other contributors, one could get such wheel distributions from a third party such as Christoph Gohlke's "Unofficial Windows Binaries for Python Extension Packages".
It is also of course possible to build such wheels yourself on one machine and then reuse it on another. As far as I know in the case of Windows, both machines need to have the same Python interpreter (minor) version as well as the same bitness (both 32 bits or both 64 bits). The exact version of the operating system should not matter (from Windows 10 to Windows 7 or the other way around should work).

I ended up getting this to work by compiling on another machine and copying the package from it. It was critical that the required packages were of the same version on both machines, so I simply set up a new conda environment which had the same package versions as the target machine, then intalled hdbscan with pip there, and copied.
I was worried about Windows version compatibility, but this worked even though I installed on Windows 10 and moved to a machine running Win7.

here is the site you can download the wheel file :https://www.lfd.uci.edu/~gohlke/pythonlibs/
for python 3.5>
or PyPi for older versions
run python -m pip install thefiledownloadedforyourpythonversion.whl
For the second question , yes you can but is rather complicated and you should avoid it when you can :)

I just checked. Python 3.8 is coded in C. You need a C compiler, not a C++ one. MINGW is one, and TinyCC Win32 is another one (a small one, producing quickly slow executables). Look also into this list of free C or C++ compilers.
A possibility (which could take several days of work) might be to use some WSL or some Linux emulator such as JSLinux (it runs in a web browser). Then you could (painfully) build a cross compiler (starting first with a tinycc-win32, then compiling an old cross GCC 3 compiler, then compiling with that a newer C++ GCC, etc....)
But the reality is that you should not be allowed to do this. Your real issue is non technical: why are you not allowed to install then use a C++ compiler, such as MinGW? Get permission (and resources) to install some....
Alternatively, consider installing some Linux distribution (ensure you are allowed to do so). Most of them have a recent Python and GCC...
Is it possible to simply compile+install on another machine and copy it?
This is called cross-compilation and is possible in general. The point is to be permitted to do that. You should find relevant cross-compilers for your situation.
If you are allowed to, you could even use a live Linux USB stick....
Some C compilers written in Python do exist.... You could use them to cross-compile tinycc for Win32. Then you should have a C compiler for Win32. You could then compile an old GCC, etc.... Qemu exist for windows. You could run a Linux with a cross-GCC compiler in Qemu.

WinDBG can't find python interpreter for PyKd

I'm trying to load a python extension to WinDBG, using PyKd.
.load pykd succeeds.
When I'm running !py, I get this:
failed to find python interpreter
Things I've tried:
Different versions of python
Using PyKd's dll and pyd files
.reload /f
Different symbol paths
Debugging WinDBG and PyKd, also going through the source, no success
Yes, my environment is set correctly with the Python path.
Couldn't find any mention of that problem in the web.
It had always ran smoothly on different systems and configurations I had, but on my new system I get this weird message.
Details:
Windows 10 Home, version 1607, x64
WinDBG x86, build 10.0.15063.0
Python 2.7.13, but I tried many other versions as well.
Symbols path: cache*c:\symbols;srv*https://msdl.microsoft.com/download/symbols
Dell XPS 15

guy
I installed the latest pykd too, and occur the same problem.
Then I found that I hadn't installed python 2.7 at this virtualbox.
Installing python 2.7 will work.
You have installed it.I suggest you check whether python installation path is in "path" environment variable

if you don't have this register path:HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\xx\PythonPath, then you should install python-2.7(x64) with Path enabled.

I had this issue when having multiple python installations on the same machine. Managed to solve it by manually creating required registry key, as I did not want to reinstall any python versions, nor change the global path/pythonpath variables.
Note that you will need x64 python for windbgx64 and x86 python for windbgx86
Let's say you want to use python 3.7x86 installed at c:\python37_x86, and python 3.6x64 installed at c:\python36_x64
Create keys leading to HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Python\PythonCore\3.7-32\InstallPath there, and set the (Default) to c:\python37_x86\ (the trailing backslash is important!)
Similarly, create keys leading to HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\3.6\InstallPath there, and set the (Default) to c:\python36_x64\
Unfortunately the PYTHONPATH environment variable is still being used, and must match the referenced python version(otherwise you will encounter Fatal Python error: unable to load the file system codec . NOTE: this may only be important if one of the versions is 2.x and the other 3.x. Try making it work without this step
To circumvent the last issue I created .bat files to modify the environment and launch windbg with the correct environment.
For example for launching 32 bit windbg:
set PYTHONPATH=C:\python37_x86\Lib;[...Rest of the PYTHONPATH for this python version...]
start windbgx -debugArch x86 -c ".load c:\dev\tools\pykd\x86\pykd.dll"

WinDBG x86, build 10.0.15063.0
Python 2.7.13, but I tried many other versions as well.
You need python 2.7.13 x86
After installation you can check it visibility for pykd with command pykd.info - it should show path to python. For python 2.7 it is commonly C:\windows\system32\python27.dll

Also had the same issue, in the end it depends on which windbg version you are using, and what python versions you have installed;
If it is windbg x86, you need python 32 bits installed.
If it is windbg x64, you need python 64 bits installed.
I did not have to add any specific version to the PATH, I have 3 versions of python in my system; python2.7x86, python2.7x64 and python3x64 and if I run !pykd.info in windbgx86 it recognizes both x86 pythons and if I run it in sindbgx64 it recognizes the x64 python.
If your !pykd.info does not list any python, you probably have them mixed up (you only have pythonx86 while running windbgx64, or you only have pythonx64 while running windbgx86).

Building 64bit libpython27.a using cygwin, dlltool

I'm trying to build a python extension DLL on a 64bit Win7 machine using cygwin (as cygwin only run as 32bit process, this is actually cross-compiling).
I created libpython27.a myself from python27.dll using dlltool (as explained, for example, here), but the build fail during the linker phase saying
skipping incompatible c:\Python27\libs/libpython27.a when searching for -lpython27
This is exactly the error reported here (where the guy ended up moving to MSVC compiler...).
More info:
- Active Python 2.7.2, win64, x64
- latest version of cygwin, using the /usr/bin/x86_64-w64-mingw32-g++.exe compiler
Does anyone know if this is supported?
Is there way to use dlltool which I miss here?
(I did found here the guidance to use
dlltool --as-flags=--64 -m i386:x86-64 -k -l libpython27.a -d python.def
but when doing so I got "invalid bfd target" error from dlltool)
Thanks!
Update: I believe it can be done because Enthought python contains such a file. I would like to create one for the more common distributions which don't contain it.

The problem is that you are using the 32 bit dlltool. Probably in C:\MinGW\bin instead of C:\MinGW64\bin. You can change your path, or run the 64 bit tool specifically as such:
C:\MinGW64\bin\dlltool -v --dllname python27.dll --def python27.def --output-lib libpython27.a

I'm not sure how helpful you find this, but at the bottom of the page you linked to there's a link to here - Where it says:
Do not use MinGW-w64. As you will notice, the MinGW import library for
Python (e.g. libpython27.a) is omitted from the AMD64 version of
Python. This is deliberate. Do not try to make one using dlltool.
There is no official MinGW-w64 release yet, it is still in "beta" and
considered unstable, although you can get a 64-bit build from e.g.
TDM-GCC. There have also been issues with the mingw runtime
conflicting with the MSVC runtime; this can happen from places you
don't expect, such as inside runtime libraries for g++ or gfortran. To
stay on the safe side, avoid MinGW-w64 for now.

How to enable Python support in gVim on Windows?

I'm trying to get Python support in gVim on Windows. Is there a way to accomplish that?
I'm using:
Windows XP SP3
gVim v. 7.3
Python 2.7.13 (ActivePython through Windows Installer binaries)

I had the same issue, but on Windows 7, and a restart didn't fix it.
I already had gVim 7.3 installed. At the time of writing the current Python version was 3.3, so I installed that. But :has ("python") and :has ("python3") still returned 0.
After much trial and error, I determined that:
If gVim is 32-bit, and it usually is even on 64-bit Windows (you can confirm using the :version command), then you need the 32-bit python installation as well
No restart of Windows 7 is required
The version of python needs to match the version that gVim is compiled for as it looks for a specific DLL name. You can work this out from the :version command in gVim, which gives something like:
Compilation: cl -c /W3 /nologo -I. -Iproto -DHAVE_PATHDEF -DWIN32
-DFEAT_CSCOPE -DFEAT_ NETBEANS_INTG -DFEAT_XPM_W32 -DWINVER=0x0400 -D_WIN32_WINNT=0x0400 /Fo.\ObjGOLYHTR/ / Ox /GL -DNDEBUG /Zl /MT -DFEAT_OLE -DFEAT_MBYTE_IME -DDYNAMIC_IME -DFEAT_GUI_W32 -DDYNAMI C_ICONV -DDYNAMIC_GETTEXT -DFEAT_TCL -DDYNAMIC_TCL
-DDYNAMIC_TCL_DLL=\"tcl83.dll\" -DDYNAM IC_TCL_VER=\"8.3\" -DFEAT_PYTHON -DDYNAMIC_PYTHON -DDYNAMIC_PYTHON_DLL=\"python27.dll\" -D FEAT_PYTHON3 -DDYNAMIC_PYTHON3 -DDYNAMIC_PYTHON3_DLL=\"python31.dll\" -DFEAT_PERL -DDYNAMI C_PERL -DDYNAMIC_PERL_DLL=\"perl512.dll\" -DFEAT_RUBY -DDYNAMIC_RUBY -DDYNAMIC_RUBY_VER=19 1 -DDYNAMIC_RUBY_DLL=\"msvcrt-ruby191.dll\" -DFEAT_BIG /Fd.\ObjGOLYHTR/ /Zi
So the above told me that I don't actually want python 3.3, I need 3.1 (or 2.7). After installing python 3.1, :has ("python") still returns 0, but :has ("python3") now returns 1. That should mean that python based scripts will now work!
I imagine future versions of gVim may be compiled against other versions of python, but using this method should let you work out which version is required.

Usually, python support is built in the official gvim distribution.
You will need to install python though: Python Downloads
to check if vim supports python:
:echo has("python")

I encountered this problem on Windows 7 64-bit. I realized I was using 64-bit Python 2.7.3 and 32-bit vim 7.3-46. I reinstalled both as 32-bit versions and then restarted the computer. Now it works.

If you have installed Python via one of the Windows installers it is probably compiled with Python 2.7 support. You can verify this by running:
:version
It will spit out all the options Vim was compiled with. Yours should say something like
+python/dyn +python3\dyn
This means you have support for python 2.7 and 3.x. If you already have 2.5 it won't work. You will need to upgrade to either 2.7 or 3.x.

Sorry for a late contribution.
The problem is that you can not mix x86 vim with x64 python libs, and all suggested solutions boil down to reinstalling x86 python.
Well, I do not want to reinstall Python, Ruby and who knows what else dependent on those just because vim does not officially provide fair x64 distribution on windows. The good news is that you can still find it well hidden at http://vim.wikia.com/wiki/Where_to_download_Vim . Good luck, and take care of Python library versions.

The accepted answer didn't solve my problem, so I decide to post the solution I found after some efforts.
First, as the accepted answer pointed out, you'll need both gVim compiled with python enabled, and a corresponding python installation. Make sure they are both 32-bit or 64-bit.
I found that the default build from www.vim.org didn't enable python, also it seemed to be 32-bit, which didn't match my python27. What I ended up with was the build "gvim_8.0.0003_x64.zip" from this vim Git repository
I then unzipped it, copied the "vim80" folder into the official vim installation location (created by the 32-bit installation downloaded www.vim.org).
Now vim works with my 64-bit python2.7 .
UPDATE 02/24/2017:
The procedure above failed on another machine where the local python installation is 2.7.9 . My Python version is 2.7.11 when I succeeded.
So, it seems the build from the vim Git repository works for a specific Python version. Try update your Python installation to 2.7.11 if you can. If you have to use an older version of Python, then maybe you need to build the Vim source code on your machine. It's not too hard following the instructions, and use the Visual Studio provided cmd instead of the Windows default cmd.

Add following scripts to your .vimrc
set pythonthreedll = python36.dll
Most distribution of vim in Windows is loading python dynamically, you must tell vim what the dll is, and make sure that python36.dll is in your PATH environment variable.

I had a similar problem. I've been enjoying vim's omni-completion feature for some years,using Windows XP, Python 2.7, gVim 7. Recently I moved to a new PC running Windows 8.1. I installed gVim and the plugins I like, then tried out everything. Omni-completion gave an error, saying I needed the version of vim compiled with Python support. At that stage, I had not yet installed Python. The solution was to install Python then re-install vim. Omni-conpletion now works. Perhaps the order of installation matters.

When I typed :version, it revealed that my Vim was not compiled with Python. Perhaps because I did not have Python (32-bit?) at the time.
I did install 32-bit Python as suggested, but reinstalling Vim seemed necessary.

After reading the above, I can confirm that on Win8.1 it does matter the order you install them (least for me it did). I had 32bit VIM 7.4 installed for a few months, then tried adding Python and couldn't do it. Left Python 2.7.9 installed and uninstalled/reinstalled VIM and now it works.

Download the one called "OLE GUI executable"

After trying all answers in this thread without success, the following worked for me (Win10, Python 2.7 32bit, gvim 7.4 32bit):
Reinvoke the Python Installer, select Change Python
Select the Option Add Python to Path, which is off by default
After the installer is done, restart your machine