running win32 ipython binaries interactively from cygwin - python

I would really like to run my Win7 ipython interactively inside a proper terminal and shell provided by cygwin (mintty or rxvt would be great).
I have had some success with python, but IPython is not really cooperating. If we run this from rxvt bash prompt:
$ /cygdrive/c/pydir/python.exe -i
I end up with a fairly successful python interactive session. But vanilla python is just not as wonderful as IPython. So then I try this:
$ /cygdrive/c/pydir/ipython.exe
and the program just blocks the bash prompt, and seems to spawn an entirely separate process (which does not produce any windows, or show signs of stdin/stdout).
If instead I use this:
$ /cygdrive/c/pydir/python.exe -i /cygdrive/c/pydir/ipython-script.py
it is exciting at first, but you quickly realize that ipython is not properly engaging with the terminal. There is no readline support, poor cursor control, stdin seems to handle typical python, but there are no [out] prompts (although "prints" do spit out text), simple things like backspace/enter/tab seem completely broken (the cursor is very defiant), there is nothing like an ncurse buffer being maintained (you can just type over the ipy command prompts).
Once I have given up on the session, exiting becomes another problem. I can return to the bash prompt, but I have no standard input. It turns out that there is always a python.exe process hanging that must be killed from the Windows side (and it doesnt release stdin until it dies).
Is there a quick fix or alternate method to run Ipython in this manner? I can do most of my development using the cygwin binaries, but being able to run win32 binaries interactive would help tremendously when debugging/testing win32 specific python libraries.
P.S.:::: I really need tab-completion and clean output. I am trying to piece my way through a bunch of COM interfaces, and the only way I can get anywhere is with an interactive ipy session.
P.S.:::: I am using a 64-bit Cygwin and a 32-bit win32 python. Could this be simple mismatch?

Running win32 IPython.exe executables in cygwin is problematic at best. Best to run the cygwin Ipython version. Depending on the libraries you need, this can work just fine. But if you need to work with win32 compiled libraries, the cygwin Ipython will not work (as far as I know).
Overall, I still struggle for pleasant terminal experiences with ipython in Windows. There is no one solution that "just works". I have 3 different ways of interacting with IPython, depending on the job I need to get done.
Use cmd.exe (or Console.exe) and run the win32 ipython. This is the best way (in my opinion) for a vim+ipython workflow. This is very functional and fast. It will hurt your eyes a little, and still lacks a good "terminal" experience... but it will get the job done if you need to grind through some work. Extra Bonus: Seems like a few extra ipy %magics work better in this workflow.
Use cygwin and the ipython package that comes with cygwin. You lose "native" win32 support, but if you generally only need the standard library, this will look better and feel better (assuming you go all the way with CygwinX and run your favorite terminal).
Use win32 Ipython, and get QTConsole working. If your in Windows and you haven't already tried QTConsole, go get it right now and spend a little time setting it up. It can be extremely fast and pleasant (fonts that don't hurt my eyes!), and its loaded with features. Its biggest drawbacks: it doesn't handle some %magics well (like %edit, so I lose my vim+ipython workflow),it doesn't do system background processes well, and it won't always handle stdin/stdout like you would think. But, if you combine QTConsole with the cmd.exe ipython, and give it a little practice+patience, you can get a pretty decent "console" development workflow.

Related

Executing Python code lines in Cygwin

I am new to Python. I must precise that I don't clearly understand the relation between Cygwin and Python. I've seen tutorials of programmers launching a Python script in Cygwin with the following line:
python "file path"
I think that this line makes python build and run that script. My question is:
is it possible to directly write "print ("Hello World") " in Cygwin ?
By the way, are the three arrows (>>>) used to designate a Cygwin Shell input line?
Many thanks in advance!
Nicola
If you go to the Cygwin site, you can find the answers to all of your questions.
Cygwin provides a collection of tools that give functionality similar to a Linux distro on Windows. Cygwin also provides substantial POSIX API functionality. When programmers launch their python scripts using Cygwin, they are using the tools provided within the Cygwin library.
To avoid spoon-feeding while still answering your question, go into Cygwin and test it for yourself. What happens when you enter that command within Cygwin? Once you see the result, if you have any other questions, comment them on here.

MAKE through Cygwin overloads memory (too many processes)

What I'm trying to do is install SIP 4.14.7 through Cygwin using the make command. I'm running Python version 3.3.2 (with Python added to the PATH) on a Windows 7 x64 SP1 machine with 4GB RAM and an Intel Core 2 Duo. Since what I'm doing is from within the Cygwin terminal, I'll avoid using the Win32 path format.
Following the installation instructions provided with sip-4.14.7.zip, here is what I've done:
Uncompressed the .zip into /c/python33/SIP/
Launched the Cygwin terminal and went to the /cygdrive/c/python33/SIP/ folder
Ran python configure.py (No options since I was fine with the default settings)
Ran make install
As far as I can tell, I followed the instructions as I should have, but obviously I'm not doing something right here.
Here's what happens:SCREENSHOT
The number of make.exe processes go up to about 1800 before Windows gets too low on memory and the whole thing reverses itself until there are no more 'make.exe' processes running as shown here: SCREENSHOT2
I've Googled this and searched around here on stackoverflow.com but couldn't find anything related to this particular issue. It seems that unless using the -j option the MAKE command should only process one job at a time. I've also tried using the -l option thinking it would limit the processes unless enough memory was available, but the results were the same.
I tried to provide as much detail as possible, but if there is any more information that I should post to help diagnose this issue, I'd be glad to provide it. Otherwise, any suggestions here would be much appreciated.
The latest version of Cygwin includes the PyQT4 package (in All->Python within Setup.exe). It's python-pyqt4 and python3-pyqt4. If you are trying to live in Cygwin, I'd install that version into Cygwin and use it. No make required from the looks of it.

What are the differences between ipython and bpython?

What does ipython have that bpython lacks and vice versa? How do the two differ?
If you just want an interactive interpreter, bpython should be fine. Just use it until you miss some feature you liked about IPython.
There are lots of features that IPython offers over bpython:
Special threading options. I like -gthread for experimenting with PyGTK and -pylab for matplotlib.
direct invocation of shell commands. cd in IPython is quite useful.
Full readline library support -- I can use the keyboard shortcuts I am used to.
Module reload facility - You can do a deep reload of a module after you have changed your code. This is very useful for testing and debugging.
Run functions in the background in a separate task with %bg.
A whole parallel programming environment (not really a feature you expect from an interactive Python shell, but IPython offers it).
This list could be nearly arbitrarily continued. And of course there will be lots of features in bpython lacking from IPython, but you did not ask for those.
So just use the one that works for you!
IPython Notebook (since 0.12) is a killer feature.

How do I prepare myself for a summer of working on Python using Linux environment?

I have used just Windows for programming so far. Now, I have an internship starting in two weeks and I will be using just Linux environment with Python programming language. I've installed Ubuntu on my system but have no exposure to shell scripting.
I need some advice on how I can quickly learn to use the Linux terminal quickly. Any books or web resources that you can suggest?
Also, is there a particular IDE that is generally preferred for Python programming on Linux, or is Vim preferred? How can I best prepare myself for the internship ahead?
Thanks for taking the time.
As an intern you'll want to use the tools your mentor is most comfortable with. If you get stuck you'll be able to ask for advice quickly.
Learning your way around either vi, vim, or emacs to start with will help. The basic concepts used in one will transfer to the other. You'll need to be able to open and read files, search through files, edit and save files, and learn how to apply any python formatting helpers correctly.
You should also familiarize yourself with version control if you haven't already. Again any one will do, you need to focus on concepts and etiquette rather than the specific tool.
The goal of the internship (and really your entire time at university) should be used to learn concepts rather than specific tools. If you learn the concepts you'll be well placed to apply those concepts using any tool. You will also "learn how to learn" a new tool, which is really valuable.
Your lack of shell scripting knowledge shouldn't matter in this case, although it won't be hard to learn. I read over some shell tutorials and put them into practice. Try doing everything from the command line, including find (grep), find/replace all (sed), finding files (find), automating things using python scripts etc. Basically, don't cheat. You'll pick up a lot this way. You'll also probably end up wondering how you ever managed with Windows.
What I use depends on the project. I really like Eclipse+PyDev but that's my personal preference, I also use Vim depending on where I am/what I'm doing. Remember you can just type python from the command line and it drops you into the python environment.
I recommend Eclipse + PyDev too. You can get started quickly with this develop environment. I also recommend the website Dive Into Python. It provides you a online free version of Dive Into Python book, which is very easy to read, easy to understand, and very suitable for Python beginners. If you really want a paper book at hand, Learning Python, a.k.a. The Animal Guide, is simply the best.
Learn to understand man(ual) pages.
For almost any old linux command/program there is a man page which usually explains the command in good detail.
So basics for filesystem navigation:
Show directory contents (list)
ls
Show hidden files
ls -a
Show details
ls -l
Change directory
cd /full/path/name
Print current directory
pwd
Delete a file
rm file
Delete a directory (recursive)
rm -r directoryName
Make a directory
mkdir directoryName
Move (or rename) a file
mv /path/to/file /new/path/to/file
Show the man page for mv
man mv
Learning vim might be necessary, depending on your intern environment. I do my Python (and everything that isn't simple text editing) in Eclipse. You should in any case learn enough to open a file, makes some changes and save the changes in Vim.
Keep in mind, Ubuntu is very easy. To make things harder on yourself, use the command line for every conceivable thing. Open programs by typing their names into a terminal. Browse your files with the terminal. Do simple editing with vim. That should provide good practice for the day you need to SSH into a computer in Neverland and download and install a local copy of your favorite interpreter from source in order to set up a cron job to run a script to play a clock noise.
In addition to the great advice already written, I'd suggest you install IPython (Open a terminal with Applications>Accessories>Terminal and type):
sudo apt-get install ipython
Also at the terminal, you can then type ipython to start the Python interpreter.
Unlike the built in python interpreter, ipython gives you tab completion.
For example, if you type the name of an object followed by a period and TAB (e.g. sys.[TAB]), ipython will show you (almost) all of object's attributes.
Type a question mark after an object name (e.g. sys?), and you get documentation on that object.
This is a great way to explore Python.
have no exposure to shell scripting
Good! You've got Python so hopefully there should be no need to resort to writing actual scripts with the shell. It may be more powerful than DOS batch files, but it's just as ugly.
I need some advice on how I can quickly learn to use the Linux terminal quickly.
Something like this?
As well as learning the commands, you'll want to get used to using tab-completion and arrow key command recall (if you don't already do that with the Windows Command Prompt), scrolling with shift-arrows, and so on. Also useful to know the & (perform in background) command suffix, ctrl-C-to-stop, ctrl-Z-to-pause, jobs, and screen.
Incidentally if you will be spending any amount of time in the interactive Python interpreter it is well worth adding tab completion there, too. (This is just as much the case on Windows, but on Win you tend not to get pyreadline by default.)
is there a particular IDE that is generally preferred for Python programming on Linux
Just like on Windows, there are IDEs available if you want them but many people just use a normal text editor. vim is fine if that's what you like. nano is another in-terminal text editor you usually get that's relatively simple. Ubuntu's default desktop-based editor gedit is also fine. It's a matter of personal taste.
(If you are interning at a particular company they might have their own development environment they'd prefer you to use.)
For a Python IDE, I recommend using either IDLE or Eclipse with PyDev.
Keep in mind you can also just use python on the linux command-line. It supports loading code from files, and if you use two command windows then one of them will be your "REPL" where you will be running python and dynamically loading code - and the other window can run your editor.
Regarding linux command line, I cannot recommend any great resources. However, you will be off to a great start if you immerse yourself in this environment and only use linux for the next 2 weeks. Just keep learning, and when you do not know how to do something, read a manpage or google it to find the answer.
for a very beginner intro to the command line, check out: http://en.flossmanuals.net/CommandLineIntro/GettingStarted
As far as a Python editor goes, I personally prefer to use SciTE. It's just a programmer's text editor with syntax highlighting for various languages. I prefer a lightweight editor over a more complicated environment, but if you want a full-fledged IDE you can always try out NetBeans, IDLE, or Komodo (all of which are available in both Windows and Linux).
as for terminall and quick way to understand it's and learn it there are a nice cheat sheets on net like this:
http://fosswire.com/post/2007/8/unixlinux-command-cheat-sheet/

Would Python make a good substitute for the Windows command-line/batch scripts?

I've got some experience with Bash, which I don't mind, but now that I'm doing a lot of Windows development I'm needing to do basic stuff/write basic scripts using
the Windows command-line language. For some reason said language really irritates me, so I was considering learning Python and using that instead.
Is Python suitable for such things? Moving files around, creating scripts to do things like unzipping a backup and restoring a SQL database, etc.
Python is well suited for these tasks, and I would guess much easier to develop in and debug than Windows batch files.
The question is, I think, how easy and painless it is to ensure that all the computers that you have to run these scripts on, have Python installed.
Summary
Windows: no need to think, use Python.
Unix: quick or run-it-once scripts are for Bash, serious and/or long life time scripts are for Python.
The big talk
In a Windows environment, Python is definitely the best choice since cmd is crappy and PowerShell has not really settled yet. What's more Python can run on several platform so it's a better investment. Finally, Python has a huge set of library so you will almost never hit the "god-I-can't-do-that" wall. This is not true for cmd and PowerShell.
In a Linux environment, this is a bit different. A lot of one liners are shorter, faster, more efficient and often more readable in pure Bash. But if you know your quick and dirty script is going to stay around for a while or will need to be improved, go for Python since it's far easier to maintain and extend and you will be able to do most of the task you can do with GNU tools with the standard library. And if you can't, you can still call the command-line from a Python script.
And of course you can call Python from the shell using -c option:
python -c "for line in open('/etc/fstab') : print line"
Some more literature about Python used for system administration tasks:
The IBM lab point of view.
A nice example to compare bash and python to script report.
The basics.
The must-have book.
Sure, python is a pretty good choice for those tasks (I'm sure many will recommend PowerShell instead).
Here is a fine introduction from that point of view:
http://www.redhatmagazine.com/2008/02/07/python-for-bash-scripters-a-well-kept-secret/
EDIT: About gnud's concern: http://www.portablepython.com/
Are you aware of PowerShell?
Anything is a good replacement for the Batch file system in windows. Perl, Python, Powershell are all good choices.
#BKB definitely has a valid concern. Here's a couple links you'll want to check if you run into any issues that can't be solved with the standard library:
Pywin32 is a package for working with low-level win32 APIs (advanced file system modifications, COM interfaces, etc.)
Tim Golden's Python page: he maintains a WMI wrapper package that builds off of Pywin32, but be sure to also check out his "Win32 How Do I" page for details on how to accomplish typical Windows tasks in Python.
Python is certainly well suited to that. If you're going down that road, you might also want to investigate SCons which is a build system itself built with Python. The cool thing is the build scripts are actually full-blown Python scripts themselves, so you can do anything in the build script that you could otherwise do in Python. It makes make look pretty anemic in comparison.
Upon rereading your question, I should note that SCons is more suited to building software projects than to writing system maintenance scripts. But I wouldn't hesitate to recommend Python to you in any case.
As a follow up, after some experimentation the thing I've found Python most useful for is any situation involving text manipulation (yourStringHere.replace(), regexes for more complex stuff) or testing some basic concept really quickly, which it is excellent for.
For stuff like SQL DB restore scripts I find I still usually just resort to batch files, as it's usually either something short enough that it actually takes more Python code to make the appropriate system calls or I can reuse snippets of code from other people reducing the writing time to just enough to tweak existing code to fit my needs.
As an addendum I would highly recommend IPython as a great interactive shell complete with tab completion and easy docstring access.
I've done a decent amount of scripting in both Linux/Unix and Windows environments, in Python, Perl, batch files, Bash, etc. My advice is that if it's possible, install Cygwin and use Bash (it sounds from your description like installing a scripting language or env isn't a problem?). You'll be more comfortable with that since the transition is minimal.
If that's not an option, then here's my take. Batch files are very kludgy and limited, but make a lot of sense for simple tasks like 'copy some files' or 'restart this service'. Python will be cleaner, easier to maintain, and much more powerful. However, the downside is that either you end up calling external applications from Python with subprocess, popen or similar. Otherwise, you end up writing a bunch more code to do things that are comparatively simple in batch files, like copying a folder full of files. A lot of this depends on what your scripts are doing. Text/string processing is going to be much cleaner in Python, for example.
Lastly, it's probably not an attractive alternative, but you might also consider VBScript as an alternative. I don't enjoy working with it as a language personally, but if portability is any kind of concern then it wins out by virtue of being available out of the box in any copy of Windows. Because of this I've found myself writing scripts that were unwieldy as batch files in VBScript instead, since I can't usually depend on Python or Perl or Bash being available on Windows.
Python, along with Pywin32, would be fine for Windows automation. However, VBScript or JScript used with the Windows Scripting Host works just as well, and requires nothing additional to install.
I've been using a lot of Windows Script Files lately. More powerful than batch scripts, and since it uses Windows scripting, there's nothing to install.
As much as I love python, I don't think it a good choice to replace basic windows batch scripts.
I can't see see someone having to import modules like sys, os or getopt to do basic things you can do with shell like call a program, check environment variable or an argument.
Also, in my experience, goto is much easier to understand to most sysadmins than a function call.

Categories