An opensource project currently uses CMake to generate project files for multiple versions of Visual Studio.
I noticed that Node uses python script to do a similar thing but is more elegant since the output of the tool doesn't hardcode directory paths into those project files. However the current node setup only seems to support 2010 and 2012. The node documentation doesn't say what the build tools are just "The build tools distributed with Node run on python."
The batch file included with Node calls python configure, but doing a search on that returns a zillion unrelated results.
I was wondering if this tool/script is
an open source project with website, documentation, etc.
extensible to earlier versions of VS (2003 onwards).
What I am looking to do is produce a bunch of solutions and projects that should just work straight out the box and not be tied to a particular dev's directory layout like CMake does. Also the script allows me to easily update multiple scripts if a file gets added or removed.
It turns out Node uses GYP to generate it's project files. I had a quick play earlier and seems fairly easy to use.
Related
I currently work on a Python project, which I'd like to upload to the Microsoft Store in the future.
As far as I am aware, in order to upload applications to the Microsoft Store, it is necessary that the application will be packed into the MSIX format.
Now the question is - is it possible to pack a Python project into the MSIX format?
I already tried two possible approaches
The first approach
I assumed that it will be much easier to pack an .exe file into an MSIX package. Since .py files require an interpreter in order to run, I managed to freeze the Python project into a standalone .exe runnable file - and it works pretty good.
I found a useful tool made by Microsoft, which is supposed to pack .exe files under the MSIX format. The tool is MSIX Packaging Tool which is available at the Microsoft Store. I did manage to create an .msix file but I can't run in since Windows says that I have to sign the .exe first.
The second approach
I found out that it is possible to pack a project into an MSIX package, by using built-in tools inside Visual Studio 2019. So I managed to move my whole python project into Visual Studio, and follow the next steps in order to pack my project.
The problem is that already in the early stages, when adding the reference to my python project, the next error occurs:
I'd love to know if you have any other possible approaches for packing a Python project into an MSIX package.
I've made this question because I had to go through the whole process of creating my own application using Apple's somewhat lacking documentation, and without the use of py2app. I wanted to create the whole application structure so I know exactly what was inside, as well as create an installer for it. The latter of these is still a mystery, so any additional answers with information on making a custom installer would be appreciated. As far as the actual "bundle" structure goes, however, I think I've managed to get the basics down. See the answer below.
Edit: A tutorial has been linked at the end of this answer on using PyInstaller; I don't know how much it helps as I haven't used it yet, but I have yet to figure out how to make a standalone Python application without the use of a tool like this and it may just be what you're looking for if you wish to distribute your application without relying on users knowing how to navigate their Python installations.
A generic application is really just a directory with a .app extension. So, in order to build your application, just make the folder without the extension first. You can rename it later when you're finished putting it all together. Inside this main folder will be a Contents folder, which will hold everything your application needs. Finally, inside Contents, you will place a few things:
Info.plist
MacOS
Resources
Frameworks
Here you can find some information on how to write your Info.plist file. Basically, this is where you detail information about your application.
Inside the MacOS you want to place your main executable. I'm not sure that it matters how you write it; at first, I just had a shell script that called python3 ./../Resources/MyApp.py. I didn't think this was very neat though, so eventually I called the GUI from a Python script which became my executable (I used Tkinter to build my application's GUI, and I wrote several modules which I will get to later). So now, my executable was a Python script with a shebang pointing to the Python framework in my application's Frameworks folder, and this script just created an instance of my custom Tk() subclass and ran the mainloop. Both methods worked, though, so unless someone points out a reason to choose one method over the other, feel free to pick. The one thing that I believe is necessary, is that you name your executable the SAME as your application (before adding the .app). That, I believe, is the only way that MacOS knows to use that file as your application's executable. Here is a source that describes the bundle structure in more detail; it's not a necessary read unless you really want to get into it.
In order to make your executable run smoothly, you want to make sure you know where your Python installation is. If you're like me, the first thing you tried doing on your new Mac was open up Terminal and type in python3. If this is the case, this prompted you to install the Xcode Command Line tools, which include an installation of Python 3.8.2 (most recent on Xcode 12). Then, this Python installation would be located at /usr/bin/python3, although it's actually using the Python framework located at
/Applications/Xcode.app/Developer/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3
I believe, but am NOT CERTAIN, that you could simply make a copy of this framework and add it to your Frameworks folder in order to make the app portable. Make a copy of the Python3.framework folder, and add it to your app's Frameworks folder. A quick side note to be wary of; Xcode comes packaged with a lot of useful tools. In my current progress, the tool I am most hurting for is the Fortran compiler (that I believe comes as a part of GCC), which comes with Xcode. I need this to build SciPy with pip install scipy. I'm sure this is not the only package that would require tools that Xcode provides, but SciPy is a pretty popular package and I am currently facing this limitation. I think by copying the Python framework you still lose some of the symlinks that point to Xcode tools, so any additional input on this would be great.
In any case, locate the Python framework that you use to develop your programs, and copy it into the Frameworks folder.
Finally, the Resources folder. Here, place any modules that you wrote for your Python app. You also want to put your application's icon file here. Just make sure you indicate the name of the icon file, with extension, in the Info.plist file. Also, make sure that your executable knows how to access any modules you place in here. You can achieve this with
import os
os.chdir('./../Resources')
import MyModules
Finally, make sure that any dependencies your application requires are located in the Python framework site-packages. These will be located in Frameworks/Python3.framework/Versions/3.X.Y/lib/python3.x.y/site-packages/. If you call this specific installation of Python from the command line, you can use path/to/application/python3 -m pip install package and it should place the packages in the correct folder.
P.S. As far as building the installer for this application, there are a few more steps needed before your application is readily downloaded. For instance, I believe you need to use the codesign tool in order to approve your application for MacOS Gatekeeper. This requires having a developer license and manipulating certificates, which I'm not familiar with. You can still distribute the app, but anyone who downloads it will have to bypass the security features manually and it will seem a bit sketchy. If you're ready to build the installer (.pkg) file, take a look at the docs for productbuild; I used it and it works, but I don't yet know how to create custom steps and descriptions in the installer.
Additional resources:
A somewhat more detailed guide to the anatomy of a macOS app
A guide I found, but didn't use, on using codesign to get your app past Gatekeeper
A RealPython tutorial I found on using PyInstaller to build Python-based applications for all platforms
I've downloaded the python 3.6.6 source from here...
https://www.python.org/downloads/release/python-366/
...and followed the instruction on how to build on Windows (run ../PCbuild/build.bat). Python compiles and seems to be working (funny and scary: while fetching externals, it actually downloads python-3.7.0 as a dependency... :/ ). However, it looks like the build is somehow 'in place', and the binaries end up in some sub-folder of the source (../PCbuild/amd64/python.exe). This means I'm left with source and compiled code mixed up instead of some clean/lean and deployable package.
can I somehow provide '--prefix=/target/build/path' to define a target location to build to, like I would on linux?
is there a way of removing all src files/folders and leave only the required files/folders (../lib, ../include, etc...).
Or in general, is there a way of making the build process more behave like on linux?
Thanks for your help,
Max
The build.bat from PCBuild is intended for developers, that is, for testing purposes. What you want is under \Tools\msi\buildrelease.bat. This creates a subdirectory under \PCBuild\ that places all msi, cab and exe files ready for later installation. According to the readme, there doesn't seem to be an option to pack all those files in a single .exe file, like all installers eventually do, but another option is under \Tools\msi\build.bat which does have an option for packing (namely build.bat --pack). "But", the readme does state that the buildrelease.bat should be used for an official release. The advantage of doing so is that Pyhton would be optimized using PGO to your own hardware. I am also trying to compile from source using this method but I am having an issue with a recurring error (and other ones):
PGO run did not succeed (no python36!*.pgc files) and there is no data to merge [E:\RepoGiT\3.6\PCbuild\pythoncore.vcxproj]
so, if you do go this route, and find this, or other errors, please send the bug report to python's bug tracker webpage. And better yet, if you find errors and their solution, please report back here!
I've read the Python documentation chapter explaining how to embed the Python interpreter in a C/C++ application. Also, I've read that you can install Python modules either in a system-wide fashion, or locally to a given user.
But let's suppose my C/C++ application will use some Python modules such as SymPy, Matplotlib, and other related modules. And let's suppose end users of my application won't have any kind of Python installation in their machines.
This means that my application needs to ship with "pseudo-installed" modules, inside its data directories (just like the application has a folder for icons and other resources, it will need to have a directory for Python modules).
Another requirement is that the absolute path of my application installation isn't fixed: the user can "drag" the application bundle to another directory and it will run fine there (it already works this way but that's prior to embedding Python in it, and I wish it continues being this way after embedding Python).
I guess my question could be expressed more concisely as "how can I use Python without installing Python, neither system-wide, nor user-wide?"
There are various ways you could attempt to do this, but none of them are general solutions. From the (docs):
5.5. Embedding Python in C++
It is also possible to embed Python in a C++ program; precisely how this is done will depend on the details of the C++ system used; in general you will need to write the main program in C++, and use the C++ compiler to compile and link your program. There is no need to recompile Python itself using C++.
This is the shortest section in the document, and is roughly equivalent to: 'left as an exercise for the reader`. I do not believe you will find any straight forward solutions.
Use pyinstaller to gather the pieces:
This means that my application needs to ship with "pseudo-installed" modules, inside its data directories (just like the application has a folder for icons and other resources, it will need to have a directory for Python modules).
If I needed to tackle this problem, I would use pyinstaller as a base. (Disclosure: I am an occasional contributer). One of the major functions of pyinstaller is to gather up all of the needed resources for a python program. In onedir mode, all of the things needed to let the program run are gathered into one directory.
You could include this tool into your make system, and have it place all of the needed pieces into your python data directory in your build tree.
I am wondering how to intelligently manage the building and installation for some of our 3rd party C++ dependencies on Linux(Ubuntu). The way I currently have it set up is a git-lfs with all the necessary compressed 3rd party sources. I then use a shell script I wrote to install all the necessary system dependencies and then unzip and build the desired library. This shell script also takes care setting up all the paths so that our source code can easily link to the 3rd party libraries.
Example commands for our script are ./install opencv or ./install everything
However, over the months the script has gotten quite large and breaks sometimes when certain libraries are already installed or other minor issues. Thus I would like to replace it with something a bit more intelligent and useful. I have currently been looking into writing some kind of python script, but just changing the language from shell to python is not that big of an advantage. So I am looking if there are any specific python libraries that can help me with managing these libraries.
I have looked into things like chef and other automated builds stuff, but that is overkill for the small project I am working on.
I was wondering what other people used for this 3rd party management stuff, as sadly C++ does not have anything like pip.
I use jhbuild for this kind of thing (if I understand what you are doing correctly). It came out of the GNOME project (they use it to build the whole desktop from source), but it's easy to customize for any set of projects. The jhbuild packaged in recent Ubuntus works fine.
You write a little XML to describe each project: where to download the sources, what patches to apply, what configure flags to use, what projects it depends on, and so on; then when you enter jhbuild build mything it works out what to build and in what order and gets on with it. It's reasonably smart about changes, so if you edit a source file in one of the projects that makes up your stack, it'll only rebuild the affected parts.
For example, I have this for fftw3, the excellent fast Fourier transform library:
<autotools id="fftw3"
autogen-sh="configure"
autogenargs="--disable-static --enable-shared --disable-threads"
>
<branch
repo="fftw"
module="fftw-3.3.4.tar.gz"
/>
<dependencies>
<dep package="libiconv"/>
</dependencies>
</autotools>
With probably obvious meanings. That's building from a release tarball, but it can build from git as well. It's happy with cmake projects. jhbuild is written in Python, so it's simple to customize. Thanks to GNOME, many common libraries are included.
I actually use it to build Windows binaries. You can set it up to build everything with a cross-compiler, then put it inside Docker. It makes it a one-liner for anyone to be able to build a large and complex application on (almost) any platform. You can use it to do automated nightly builds as well, of course.
There are probably better things around, but it has worked well for me.