I am taking a course in C, and I decided to write a little script in python that would compile the homework questions we are given and run them against test files containing input and expected output.
It was straightforward to compile the code with the gcc or clang compilers from the script. I just needed to run shutil.which(compiler) where compiler is either gcc or clang to get the compiler's path, and then invoke it using the subprocess module to compile the code.
The problem I am facing is handling the Microsoft MSVC compiler for Windows. Some students who use Windows like to use Visual Studio for writing code and I can't blame them for that. I found it quite challenging to invoke the MSVC compiler from a Python script. As opposed to other compilers like gcc or clang, which are binaries located on the system's PATH and thus discoverable using shutil.which, the MSVC compiler isn't usually found directly on the system's PATH. I know that on Windows if you want to invoke the MSVC compiler, mainly accessible from the cl.exe binary, you have to open a special developer command prompt that sets up the environment for cl.exe to work correctly. This means that a simple shutil.which("cl.exe") wouldn't work, and more effort is required to invoke the cl.exe compiler. I read that there are certain batch scripts you can use to set up the MSVC compiler such as vcvarall.bat. However, I couldn't find an easy and portable way to invoke them from a script.
The Microsoft documentation specifies the location of these scripts in the Visual Studio installation directory, but there doesn't seem to be a portable way to access these locations. What if Visual Studio isn't installed in C:\Program Files, but elsewhere? How can I determine the version and edition of Visual Studio programmatically? What if only the VisualC++ tools are installed and not Visual Studio itself? Where would the scripts be located then?
There doesn't seem to be an environment variable or other portable tool that holds this information, so it looks like any attempt to find these scripts will be system dependent and is doomed to fail on a system with a custom installation.
Even if I manage to get these scripts programmatically, the documentation doesn't give a clear example of how to use them. I have no experience with using the MSVC compilers outside Visual Studio, so I don't know how to use these scripts and invoke the cl.exe compiler.
Can anyone find an easy and portable way to invoke the MSVC compilers from a Python script? Visual Studio is an extremely common C/C++ IDE on Windows, so I would like my script to be capable of invoking the MSVC compilers. If you can provide an example of how this could be accomplished, I'd really appreciate it.
Related
there,
I've developed a Python package (https://github.com/pedrosecchi67/LovelacePM) which makes intensive use of f2py to speed up calculations. It was installed and ran smoothly on Ubuntu from day 1 and pip had no trouble using the setup.py script to compile Fortran extensions.
When I first tried using pip to install it on my Windows machine, however, I had an installation error reporting that Visual C++ Compiler 14.0 was not found and had to be installed in order to compile the same modules.
I used to have VS Community installed until about a year ago, but removed most of its suite because I stopped using it and wanted to save disk space - and I'm afraid I might have uninstalled the compiler along with it. I don't know if it was a default Windows feature or a part of VS Community, though.
My question is: does Visual C++ Compiler come installed on Windows by default?
I need to know whether or not my package's users will be able to install it on a Windows machine without previously installing the compiler from the internet.
Visual C++ is part of Visual Studio, not of Windows. It is a separate product line with separate versions, maintained by a separate team. VS Community 2019 is the latest free version of Visual Studio, and still comes with the 14.0 compiler. (as well as 14.1 and 14.2)
The closest you could get was a command-line compiler in the Windows SDK (Software Development Kit). But that has been removed. You can still get the Visual Studio Build Tools without the whole IDE.
Does python require microsoft visual c++ redistributable to run the code ?
I'm using pyinstaller to compile my .py code into exe. In some systems my exe is asking for microsoft visual c++ redistributable package to run.
Does pyinstaller includes microsoft visual c++ redistributable files while making exe ? If no, how can I include those files so that I don't need to install microsoft visual c++ redistributable package into other's system to run my software ?
What are the other alternatives to build a standalone software in python ? I'm reading to use other languages along with python.
I saw electron js and python can be used together to make desktop application. But how will I distribute that application as a standalone exe ?
Python itself does not depend on the presence of MSVC. You can download a portable Python package, and it will run wherever you copy it. Those are the embeddable ones from https://www.python.org/downloads/windows/
But, Python modules with native extension code inside can depend on MSVC on multiple levels:
if the native part comes in binary format (.pyd file on Windows), actually that is a .dll, and it may depend on other .dll-s, depending on how it has been built
if the native part comes as C/C++ source code, it will be built at installation time, typically via a "setup.py", and this procedure will need a C compiler installed on the system
PyInstaller is a different story. First of all, it has a documentation which you may want to read. For example the page What PyInstaller Does and How It Does It clearly gives a direct answer to at least one of your questions:
Bundling to One File
PyInstaller can bundle your script and all its dependencies into a single executable named myscript (myscript.exe in Windows).
There is also a list of packages with known compatibility and known compatibility issues: https://github.com/pyinstaller/pyinstaller/wiki/Supported-Packages, which you may find useful depending on what packages you need.
While it is not a duplicate, this question: How to package a linked DLL and a pyd file into one self contained pyd file? (and another one it links) may be interesting to read.
Of course Python requires MSVC Redistributable, any native Windows program using standard library functions requires it. Obviously, Python uses lots of them and should provide a consistent environment across all extension modules.
However, since Python 3.5 it is bundled with an installer, so there's no need to install it manually. Python installers prior to 3.5 don't include it and I wasn't able to find any clarifications whether it's downloaded during installation or not.
By default Python also enforces extension modules to be compiled with the same (or, since 3.5, compatible) version of MSVC as an interpreter itself. So except for some very rare cases extension modules will also use the same redistributable.
"Embeddable" Python releases referred by #tevemadar are NOT a "portable Python"! Here's what the documentation says about their usage:
It is intended for acting as part of another application, rather than being directly accessed by end-users.
Note: The embedded distribution does not include the Microsoft C Runtime and it is the responsibility of the application installer to provide this. The runtime may have already been installed on a user’s system previously or automatically via Windows Update, and can be detected by finding ucrtbase.dll in the system directory.
But you still don't need them if you use PyInstaller.
To check whether or not redistributable files are included in your .exe file you could probably open it with any archiver software and see it for yourself. My guess is that they can be included at least if Python is installed in a single user mode, as in such case they're installed in the Python directory as well.
Other than that, however, you should really ask your questions separately.
I want to send somebody my compiled fortran extension on a Mac (compiled with f2py and gfortran).
Problem is that it doesn't work on other Macs unless they also instal xcode (2 GB, yikes!) and gfortran. So apparently there are some additional files missing when I just send the compiled extension.
Does anybody know what other files to include or (better) how to compile a fortran extension without needing to send any additional files?
Thanks,
Mark
Well, when you compile a module with f2py, it essentially creates a dynamic library (.so) which uses your system libraries. For instance, on my computer, the linking step is,
gfortran [...] -lpython2.7 -lgfortran -o ./my_f2py_module.so
Therefore, if you want to be able to execute the resulting f2py module on a different computer (assuming the same architecture), libpython2.7.so and libgfortran.so should be available there.
I don't know much about OS/X deployment, but I think you should
use a compiler present by default on Mac (i.e. clang), which should work with f2py. Or alternatively install gfortran on both systems.
make sure you link to the same version of python in both cases
Also use ldd ./my_f2py_module.so to list all the libraries that it is linked to. If some of them cannot be found on the current system, you will also see it with this command.
I am trying to build a python module for android.
While I can set a compiler with export CC=, I am not able to do the same for C++ compilers using export CXX=. It still uses the basic C compiler.
Can anyone help me to figure out how to configure the compilers while building python modules?
Thanks.
Here are some old instructions for Windows how to change the compiler
Create a file called 'distutils.cfg' in "C:\Python26\Lib\distutils".
Open it with a text editor ('notepad distutils.cfg') and fill in the following lines:
[build]
compiler=mingw32
The same should go for other platforms as well when you manage to locate distutils folder.
Howver setup.py / Python packages can do their own stuff and these must be resolved case-by-case basis.
So, I'm just trying to build pywin32 on my Windows 7 machine. When running the setup script, I ran into the common error of "Unable to find vcvarsall.bat". I do have Visual C++ installed, so I was somewhat confused. After a bit of searching around, here as well as other sites, I found that the best solution to my problem would probably be to add the path to the batch file right to my path. However, when looking for that path, I realized that, since I have Visual Studio 2010 installed rather than 2008 or earlier, I have a file called "vcvars32.bat" instead of the usual "vavarsall.bat".
Is it possible to get the setup script to use vcvars32? This is probably ridiculous and may reveal how little I know about batch files, but could I perhaps copy and rename the vcvars32.bat file to vcvarsall.bat, or are they too fundamentally different to work properly?
Or do I just need to install an older version of Visual Studio? I would really rather not if I could avoid it. I have also been having trouble building with MinGW. Are there any files I need to download first?
Thanks in advance,
Ken
Is there a way to get pywin32 to use vcvars32.bat instead of
vcvarsall.bat?
There's no need.
I realized that, since I have Visual Studio 2010 installed rather than
2008 or earlier, I have a file called "vcvars32.bat" instead of the
usual "vcvarsall.bat".
You have both and vcvars32.bat is being used by vcvarsall.bat. For instance, on the 64 bit Windows typical installation paths for these files in Visual Studio 2010 are
C:\Program Files (x86)\Microsoft Visual Studio
10.0\VC\vcvarsall.bat
C:\Program Files (x86)\Microsoft Visual Studio
10.0\VC\bin\vcvars32.bat
Or do I just need to install an older version of Visual Studio?
If you use official Python builds then you have to install an older version - Visual Studio 2008 or 2005 depending on what version of Python you use. That's because these versions of Visual Studio are used to make official Python builds and you have to use the same version of compiler when building C extension modules.
You could write a vcvarsall.bat which simply
"%~dp0\vcvars32.bat"
runs the vcvars32.bat in the same folder.
(I think. I haven't touched Windows in what feels like years… happily ☺)
Installing the binary version of pywin32 is much simpler and doesn't require the installation of any c++ compilers. For some reason it isn't easy to find these on the sourceforge website, but it is definitely there. Here is the link to all of the executables for version 2.14: http://sourceforge.net/projects/pywin32/files/pywin32/Build%20214/