I need to present the user a list of Python installations to choose from for executing something. I suppose in Windows I could get this information from registry. Don't know about Linux and Mac.
Any hints? Or maybe you even know a place where I could find Python code for this?
EDIT: it is not important that I find really all interpreters. Finding interpreters from standard locations would be actually fine. Agreed, it's not something too difficult, but I was just hoping that maybe someone has code for this lying around or that I've overlooked a function for that in stdlib.
I'm writing a Python IDE and I want to let user to choose the interpreter for executing the program.
Just do it like other IDEs then and simply supply a dialog where users can add interpreters they want to be able to run the code with.
Eclipse does this for example for Java Runtimes, and it’s perfectly fine to have it like that. Especially for languages like Python where virtual environments are an important thing which each have their own exectutable.
You certainly can come up with a one-time detection that checks some common locations. For Windows, this would obviously be the registry, as the py.exe launcher requires the interpreters to be registered there—at least the system-wide ones. On Unix machines, you could check the common bin/ folders, most prominently /usr/local/bin/ which is the standard location where Python installs itself. You could also check the PATH for Python executables. But all those things should be considered carefully and only offer an initial setup. There are always edge cases where a user didn’t do the “standard thing” where your detection will fail. For example I don’t have my Python interpreters in my path, and a linux server I access I have installed Python into a non-standard folder in my home directory. And finally, just because it looks like Python doesn’t mean it is Python.
Yes, you can do some guesswork to come up with an initial set of interpreters, but really don’t spend too much time on it. In the end, you won’t be able to detect everything perfectly anyway. And you will miss virtual environments—which might be very crucial to the project the user is working on in your IDE.
So instead of wasting time on bad detection, spend more time on creating a manual dialog to register interpreters. You will need that anyway, and a good interface can make it very easy—even for beginners—to use it.
Not sure this is entirely possible, but here's a dirty fix:
from subprocess import *
from time import sleep
for i in range(2, 4):
x = Popen('python' + str(i) + ' --version', shell=True, stdout=PIPE, stdin=PIPE, stderr=STDOUT)
while x.poll() == None:
sleep(0.025)
print('Exit code of ' + str(i) + ' is:',x.poll())
x.stdout.close()
x.stdin.close()
The exit code will tell you if Python2 or Python3 is installed.
You could add a second iterator for python versions 2.4, 2.4, 3.1, 3.2 etc etc. Or just keep them in a list, whichever you prefer for this already dirty fix.
Related
I've got a script that requires python3.7 or above.
I've got multiple systems each with different python versions. all of them have either python3.7 or python3.8 but just "python3" can also be python3.6.
Is there a way to make a shebang line take either python3.7 or python3.8? and fail if none exist of course.
I tried a billion of options and nothing works. I was hoping for something like:
#!/usr/bin/python3.7 || /usr/bin/python3.8
or
#!/usr/bin/python3.7
#!/usr/bin/python3.8
but nada.. I also couldn't google it up which kinda makes me sad but who knows maybe the answer is there somewhere and I'm just using a bizarre jargon
these are not systems I can change defaults on and installing python3.7 on each such setup is sadly not an option (would make life awesome).
The shebang simply needs to point to an actual executable.
That executable can be one which finds a suitable version of the Python interpreter; see for example the py tool which was popular on Windows for a long time but which is now also being made available on real computers. other platforms.
Ultimately, something somewhere needs to be on the user's PATH, which obviously you will not always have control over. If you are targeting unsophisticated users, perhaps look into providing your code in a compiled executable, a prepackaged Docker image, or something vaguely like that.
I'm working on an Inno Setup installer for a Python application for Windows 7, and I have these requirements:
The app shouldn't write anything to the installation directory
It should be able to use .pyc files
The app shouldn't require a specific Python version, so I can't just add a set of .pyc files to the installer
Is there a recommended way of handling this? Like give the user a way to (re)generate the .pyc files? Or is the shorter startup time benefit from the .pyc files usually not worth worrying about?
PYC files aren't guaranteed to be compatible for different python versions. If you don't know that all your customers are running the same python versions, you really don't want to distribute pyc's directly. So, you have to choose between distributing PYCs and supporting multiple python versions.
You could create build process that compiles all your files using py_compile and zips them up into a version-specific package. You can do this with setuptools.; however it will be awkward to do because you'll have to run py_compile in every version you need to support.
If you are basically distributing a closed application and don't want people to have trivial access to your source code, then py2exe is probably a simpler alternative. If your python is supposed to be integrated into the user's python install, then it's probably simpler to just create a zip of your .py files and add a one-line .py stub that imports the zipped package(s) using zipfile
if it makes you feel better, PYC doesn't provide much extra security and it doesn't really boost perf much either :)
If you haven't read PEP 3147, that will probably answer your questions.
I don't mean the solution described in that PEP and implemented as of Python 3.2. That's great if your "multiple Python versions" just means "3.2, 3.3, and probably future 3.x". Or even if it means "2.6+ and 3.1+, but I only really care about 3.2 and 3.3, so if I don't get the pyc speedups for other ones that's OK".
But when I asked your supported versions, you said, "2.7", which means you can't rely on PEP 3147 to solve your problems.
Fortunately, the PEP is full of discussion of earlier attempts to solve the problem, and the pitfalls of each, and there should be more than enough there to figure out what the options are and how to implement them.
The one problem is that the PEP is very linux-centric—mainly because it's primarily linux distros that tried to solve the problem in the past. (Apple also did so, but their solution was (a) pretty much working, and (b) tightly coupled with the whole Mac-specific "framework" thing, so they were mostly ignored…)
So, it largely leaves open the question of "Where should I put the .pyc files on Windows?"
The best choice is probably an app-specific directory under the user's local application data directory. See Known Folders if you can require Vista or later, CSIDL if you can't. Either way, you're looking for the FOLDERID_LocalAppData or CSIDL_LOCAL_APPDATA, which is:
The file system directory that serves as a data repository for local (nonroaming) applications. A typical path is C:\Documents and Settings\username\Local Settings\Application Data.
The point is that it's a place for applications to store data that's separate for each user (and inside that user's profile directory), and also for each machine the user's roaming profile might end up on, which means you can safely put stuff there and know that the user has the permissions to write there without UAC getting involved, and also know (as well as you ever can) that no other user or machine will interfere with what's there.
Within that directory, you create a directory for your program, and put whatever you want there, and as long as you picked a unique name (e.g., My Unique App Name or My Company Name\My App Name or a UUID), you're safe from accidental collision with other programs. (There used to be specific guidelines on this in MSDN, but I can no longer find them.)
So, how do you get to that directory?
The easiest way is to just use the env variable %LOCALAPPDATA%. If you need to deal with older Windows, you can use %USERPROFILE% and tack \Local Settings\Application Data onto the end, which is guaranteed to either be the same, or end up in the same place via junctions.
You can also use pywin32 or ctypes to access the native Windows APIs (since there are at least 3 different APIs for this and at least two ways to access those APIs, I don't want to give all possible ways to write this… but a quick google or SO search for "pywin32 SHGetFolderPath" or "ctypes SHGetKnownFolderPath" or whatever should give you what you need).
Or, there are multiple third-party modules to handle this. The first one both Google and PyPI turned up was winshell.
Re-reading the original question, there's a much simpler answer that probably fits your requirements.
I don't know much about Inno, but most installers give you a way to run an arbitrary command as a post-copy step.
So, you can just use python -m compileall to create the .pyc files for you at install time—while you've still got elevated privileges, so there's no problem with UAC.
In fact, if you look at pywin32, and various other Python packages that come as installer packages, they do exactly this. This is an idiomatic thing to do for installing libraries into the user's Python installation, so I don't see why it wouldn't be considered reasonable for installing an executable that uses the user's Python installation.
Of course if the user later decides to uninstall Python 2.6 and install 2.7, your .pyc files will be hosed… but from your description, it sounds like your entire program will be hosed anyway, and the recommended solution for the user would probably be to uninstall and reinstall anyway, right?
I want to create a GUI application which should work on Windows and Mac. For this I've chosen Python.
The problem is on Mac OS X.
There are 2 tools to generate an ".app" for Mac: py2app and pyinstaller.
py2app is pretty good, but it adds the source code in the package. I
don't want to share the code with the final users.
Pyinstaller generates UNIX executable, so how to run it on Mac? I
created a bundles with this executable, but the resulted ".app" is
not working.
The questions are:
How to configure py2app to include the source code in the
executable, so the final users will not have access to my program?
How to convert UNIX executable to Mac ".app" ?
Is there a way to compile Python code with GCC ?
In Windows it's easy, I created an "exe" file from Python code and
it works. Is it possible to create a single file "app" for Mac ?
P.S. I use two computers (Windows and for Mac), Python 2.7, wxPython, py2exe, py2app and pyinstaller.
Also, I have checked out these sites:
http://svn.pythonmac.org/py2app/py2app/trunk/doc/index.html
http://www.pyinstaller.org/export/develop/project/doc/Manual.html?format=raw
http://www.pyinstaller.org/wiki/Features/MacOsCompatibility
http://www.stackoverflow.com/questions/2933/an-executable-python-app
How to configure py2app to include the source code in the executable,
so the final users will not have access to my program?
Unless you very seriously hack the python interpreter (and include the mangled version) there is no really good way to hide the source from a moderately skilled and determined user. I strongly believe this is true on Windows also. Basically, whether you include true source or bytecode, a pretty clean version of the source can be recovered. More importantly, in my opinion, unless you include the actual source code (as opposed to bytecode, you will introduce a possible dependency on the interpreter version).
How to convert UNIX executable to Mac ".app" ?
What do you mean by a UNIX executable? A Darwin (OS X) binary [which isn't actually UNIX]? That can be done using the kinds of tools you already mentioned, but it must be done carefully to avoid library dependencies.
If all you want it a simple wrapper to put a command-line binary into a window, it's pretty easy to accomplish and the free XCode suite has several examples that would serve (depending on what output
you wan to deliver, if any).
Is there a way to compile Python code with GCC ?
GCC does not compile Python. It's a different language (although there tools in the gcc family rthat support multiple language front-ends, but not Python). There are tools that attempt to translate Python into C, and then you can compile that into a true binary, but this only works for programs that avoid certain types of construct, and the process (and restrictions) need to apply your libraries as well.
One project to allow this is Cython. It works well for some types
of code, mostly numerical code, but it is not trivial to install and
exploit, very especially if you want to produce something that runs on multiple
different computers.
In Windows it's easy, I created an "exe" file from Python code and it
works. Is it possible to create a single file "app" for Mac ?
I would have to say I am skeptical -- very skeptical -- about this. Just like the OS X case, the exe almost certainly has the source code trivially accessible within it.
One fairly easy trick is to encrypt the source code and then decrypt it on the fly, but this
seems to me like more trouble than it's worth.
PyInstaller will automatically create bundles under Mac OSX for windowed executables. When running ypinstaller.py, make sure to pass the option "--windowed".
This feature is documented in the website of pyinstaller
If you're not completely committed to wxPython (and for anyone else looking for a cross platform Python GUI framework), I recommend you check out Kivy. It's cross platform, GPU accelerated, and it will do the app packaging for you. It's easy to jump into, has a well thought-out architecture, and gives you an incredible amount of flexibility in terms of the interface. It's the best way I've found to make a cross platform Python GUI app.
cxFreeze was the choice.
I use it pack my python program to a Mac OS X app. Which works like a charm.
Automator was already mentioned as a quick and simple solution for Pythons scripts that are contained in a single file, but since the Automator UI has so many options, and it is not obvious how to actually do it, I'll provide step-by-step instructions (verified to work on Yosemite):
In Automator select File > New and pick Application as document type.
Next, make sure Actions tab is selected on the left, and then in the search box type run. Among other options you'll see Run Shell Script — doubleclick it, and an editor window will appear in the right panel.
From the Shell dropdown menu select /usr/bin/python.
Paste your Python code into the edit window and then pick File > Save.
By default, the app will be saved under $HOME/Applications and will appear in Spotlight.
If you want to be able to set your own icon and have some fancy features, like task bar icons with a menu, log windows etc, then have a look at Platypus — an open-source app for creating MacOS native bundles.
2: You can't "convert" it, but you can move the executable to App.app/Contents/MacOS/something in a .app file, with CFBundleExecutable set to "something". This would not generally be recommended.
A motivated person could probably reconstruct usable source code from the Python bytecode in your app, so you might reconsider your opposition to py2app. If you don't trust your final users, why are you doing business with them?
Having used py2exe for windows users so they wouldn't have to deal with library versions, I've torn apart the compiled programs, they include the python bytecode files. While you can make it a violation of the license to look inside those, the fact is that if a computer can execute them, I can read them. It is possible to compile python programs with gcc, via a C preprocessor (try looking for 2c.py on google), I don't know if any of them support GCC. Again, you don't gain any security through using them, but you can get a significant speed improvement.
I haven't tried it with big Python projects, but for my own scripts, the easiest way I found was to use Automator
You can interactively create an app project with Run Shell Script action, then paste in your script in its editor, select your shell program (/usr/bin/python), finally save the project. And you have yourself a Mac native app.
Automator can also be driven by AppleScript. So you can pipeline this py-2-app conversion process to your build scripts.
I've never tested a GUI program with it so I don't know if you'll be happy with it. But I'd give it a try since you may wonder how well all the cited 3rd-party python modules/applications are maintained, and how long they are gonna last. Coming bundled with OS X, Automator will likely stay, unless Apple got REALLY tired of it.
cxFreeze is best solution available, first create your program or application using python and than make setup file for your application, and than build the app using build command python setup.py build, according to your requirement you need to make some changes.
The only way is py2app. You have no other way. Sorry.
The research you did seems very solid and you did not miss anything.
I have used just Windows for programming so far. Now, I have an internship starting in two weeks and I will be using just Linux environment with Python programming language. I've installed Ubuntu on my system but have no exposure to shell scripting.
I need some advice on how I can quickly learn to use the Linux terminal quickly. Any books or web resources that you can suggest?
Also, is there a particular IDE that is generally preferred for Python programming on Linux, or is Vim preferred? How can I best prepare myself for the internship ahead?
Thanks for taking the time.
As an intern you'll want to use the tools your mentor is most comfortable with. If you get stuck you'll be able to ask for advice quickly.
Learning your way around either vi, vim, or emacs to start with will help. The basic concepts used in one will transfer to the other. You'll need to be able to open and read files, search through files, edit and save files, and learn how to apply any python formatting helpers correctly.
You should also familiarize yourself with version control if you haven't already. Again any one will do, you need to focus on concepts and etiquette rather than the specific tool.
The goal of the internship (and really your entire time at university) should be used to learn concepts rather than specific tools. If you learn the concepts you'll be well placed to apply those concepts using any tool. You will also "learn how to learn" a new tool, which is really valuable.
Your lack of shell scripting knowledge shouldn't matter in this case, although it won't be hard to learn. I read over some shell tutorials and put them into practice. Try doing everything from the command line, including find (grep), find/replace all (sed), finding files (find), automating things using python scripts etc. Basically, don't cheat. You'll pick up a lot this way. You'll also probably end up wondering how you ever managed with Windows.
What I use depends on the project. I really like Eclipse+PyDev but that's my personal preference, I also use Vim depending on where I am/what I'm doing. Remember you can just type python from the command line and it drops you into the python environment.
I recommend Eclipse + PyDev too. You can get started quickly with this develop environment. I also recommend the website Dive Into Python. It provides you a online free version of Dive Into Python book, which is very easy to read, easy to understand, and very suitable for Python beginners. If you really want a paper book at hand, Learning Python, a.k.a. The Animal Guide, is simply the best.
Learn to understand man(ual) pages.
For almost any old linux command/program there is a man page which usually explains the command in good detail.
So basics for filesystem navigation:
Show directory contents (list)
ls
Show hidden files
ls -a
Show details
ls -l
Change directory
cd /full/path/name
Print current directory
pwd
Delete a file
rm file
Delete a directory (recursive)
rm -r directoryName
Make a directory
mkdir directoryName
Move (or rename) a file
mv /path/to/file /new/path/to/file
Show the man page for mv
man mv
Learning vim might be necessary, depending on your intern environment. I do my Python (and everything that isn't simple text editing) in Eclipse. You should in any case learn enough to open a file, makes some changes and save the changes in Vim.
Keep in mind, Ubuntu is very easy. To make things harder on yourself, use the command line for every conceivable thing. Open programs by typing their names into a terminal. Browse your files with the terminal. Do simple editing with vim. That should provide good practice for the day you need to SSH into a computer in Neverland and download and install a local copy of your favorite interpreter from source in order to set up a cron job to run a script to play a clock noise.
In addition to the great advice already written, I'd suggest you install IPython (Open a terminal with Applications>Accessories>Terminal and type):
sudo apt-get install ipython
Also at the terminal, you can then type ipython to start the Python interpreter.
Unlike the built in python interpreter, ipython gives you tab completion.
For example, if you type the name of an object followed by a period and TAB (e.g. sys.[TAB]), ipython will show you (almost) all of object's attributes.
Type a question mark after an object name (e.g. sys?), and you get documentation on that object.
This is a great way to explore Python.
have no exposure to shell scripting
Good! You've got Python so hopefully there should be no need to resort to writing actual scripts with the shell. It may be more powerful than DOS batch files, but it's just as ugly.
I need some advice on how I can quickly learn to use the Linux terminal quickly.
Something like this?
As well as learning the commands, you'll want to get used to using tab-completion and arrow key command recall (if you don't already do that with the Windows Command Prompt), scrolling with shift-arrows, and so on. Also useful to know the & (perform in background) command suffix, ctrl-C-to-stop, ctrl-Z-to-pause, jobs, and screen.
Incidentally if you will be spending any amount of time in the interactive Python interpreter it is well worth adding tab completion there, too. (This is just as much the case on Windows, but on Win you tend not to get pyreadline by default.)
is there a particular IDE that is generally preferred for Python programming on Linux
Just like on Windows, there are IDEs available if you want them but many people just use a normal text editor. vim is fine if that's what you like. nano is another in-terminal text editor you usually get that's relatively simple. Ubuntu's default desktop-based editor gedit is also fine. It's a matter of personal taste.
(If you are interning at a particular company they might have their own development environment they'd prefer you to use.)
For a Python IDE, I recommend using either IDLE or Eclipse with PyDev.
Keep in mind you can also just use python on the linux command-line. It supports loading code from files, and if you use two command windows then one of them will be your "REPL" where you will be running python and dynamically loading code - and the other window can run your editor.
Regarding linux command line, I cannot recommend any great resources. However, you will be off to a great start if you immerse yourself in this environment and only use linux for the next 2 weeks. Just keep learning, and when you do not know how to do something, read a manpage or google it to find the answer.
for a very beginner intro to the command line, check out: http://en.flossmanuals.net/CommandLineIntro/GettingStarted
As far as a Python editor goes, I personally prefer to use SciTE. It's just a programmer's text editor with syntax highlighting for various languages. I prefer a lightweight editor over a more complicated environment, but if you want a full-fledged IDE you can always try out NetBeans, IDLE, or Komodo (all of which are available in both Windows and Linux).
as for terminall and quick way to understand it's and learn it there are a nice cheat sheets on net like this:
http://fosswire.com/post/2007/8/unixlinux-command-cheat-sheet/
I've got some experience with Bash, which I don't mind, but now that I'm doing a lot of Windows development I'm needing to do basic stuff/write basic scripts using
the Windows command-line language. For some reason said language really irritates me, so I was considering learning Python and using that instead.
Is Python suitable for such things? Moving files around, creating scripts to do things like unzipping a backup and restoring a SQL database, etc.
Python is well suited for these tasks, and I would guess much easier to develop in and debug than Windows batch files.
The question is, I think, how easy and painless it is to ensure that all the computers that you have to run these scripts on, have Python installed.
Summary
Windows: no need to think, use Python.
Unix: quick or run-it-once scripts are for Bash, serious and/or long life time scripts are for Python.
The big talk
In a Windows environment, Python is definitely the best choice since cmd is crappy and PowerShell has not really settled yet. What's more Python can run on several platform so it's a better investment. Finally, Python has a huge set of library so you will almost never hit the "god-I-can't-do-that" wall. This is not true for cmd and PowerShell.
In a Linux environment, this is a bit different. A lot of one liners are shorter, faster, more efficient and often more readable in pure Bash. But if you know your quick and dirty script is going to stay around for a while or will need to be improved, go for Python since it's far easier to maintain and extend and you will be able to do most of the task you can do with GNU tools with the standard library. And if you can't, you can still call the command-line from a Python script.
And of course you can call Python from the shell using -c option:
python -c "for line in open('/etc/fstab') : print line"
Some more literature about Python used for system administration tasks:
The IBM lab point of view.
A nice example to compare bash and python to script report.
The basics.
The must-have book.
Sure, python is a pretty good choice for those tasks (I'm sure many will recommend PowerShell instead).
Here is a fine introduction from that point of view:
http://www.redhatmagazine.com/2008/02/07/python-for-bash-scripters-a-well-kept-secret/
EDIT: About gnud's concern: http://www.portablepython.com/
Are you aware of PowerShell?
Anything is a good replacement for the Batch file system in windows. Perl, Python, Powershell are all good choices.
#BKB definitely has a valid concern. Here's a couple links you'll want to check if you run into any issues that can't be solved with the standard library:
Pywin32 is a package for working with low-level win32 APIs (advanced file system modifications, COM interfaces, etc.)
Tim Golden's Python page: he maintains a WMI wrapper package that builds off of Pywin32, but be sure to also check out his "Win32 How Do I" page for details on how to accomplish typical Windows tasks in Python.
Python is certainly well suited to that. If you're going down that road, you might also want to investigate SCons which is a build system itself built with Python. The cool thing is the build scripts are actually full-blown Python scripts themselves, so you can do anything in the build script that you could otherwise do in Python. It makes make look pretty anemic in comparison.
Upon rereading your question, I should note that SCons is more suited to building software projects than to writing system maintenance scripts. But I wouldn't hesitate to recommend Python to you in any case.
As a follow up, after some experimentation the thing I've found Python most useful for is any situation involving text manipulation (yourStringHere.replace(), regexes for more complex stuff) or testing some basic concept really quickly, which it is excellent for.
For stuff like SQL DB restore scripts I find I still usually just resort to batch files, as it's usually either something short enough that it actually takes more Python code to make the appropriate system calls or I can reuse snippets of code from other people reducing the writing time to just enough to tweak existing code to fit my needs.
As an addendum I would highly recommend IPython as a great interactive shell complete with tab completion and easy docstring access.
I've done a decent amount of scripting in both Linux/Unix and Windows environments, in Python, Perl, batch files, Bash, etc. My advice is that if it's possible, install Cygwin and use Bash (it sounds from your description like installing a scripting language or env isn't a problem?). You'll be more comfortable with that since the transition is minimal.
If that's not an option, then here's my take. Batch files are very kludgy and limited, but make a lot of sense for simple tasks like 'copy some files' or 'restart this service'. Python will be cleaner, easier to maintain, and much more powerful. However, the downside is that either you end up calling external applications from Python with subprocess, popen or similar. Otherwise, you end up writing a bunch more code to do things that are comparatively simple in batch files, like copying a folder full of files. A lot of this depends on what your scripts are doing. Text/string processing is going to be much cleaner in Python, for example.
Lastly, it's probably not an attractive alternative, but you might also consider VBScript as an alternative. I don't enjoy working with it as a language personally, but if portability is any kind of concern then it wins out by virtue of being available out of the box in any copy of Windows. Because of this I've found myself writing scripts that were unwieldy as batch files in VBScript instead, since I can't usually depend on Python or Perl or Bash being available on Windows.
Python, along with Pywin32, would be fine for Windows automation. However, VBScript or JScript used with the Windows Scripting Host works just as well, and requires nothing additional to install.
I've been using a lot of Windows Script Files lately. More powerful than batch scripts, and since it uses Windows scripting, there's nothing to install.
As much as I love python, I don't think it a good choice to replace basic windows batch scripts.
I can't see see someone having to import modules like sys, os or getopt to do basic things you can do with shell like call a program, check environment variable or an argument.
Also, in my experience, goto is much easier to understand to most sysadmins than a function call.