We Keep Coding

Python Code Completion for C++ Libraries (Using Pycharm, VTK)

I'm trying to get back into coding for some math/physics experimentations and found VTK as a powerful tool using python. So I installed Python(x,y) and Pycharm Community edition. But I cannot get the Code Completion for VTK to work. I know this question has been posted quite a lot of times, but I couldn't find any concrete answer.
Here's what I know so far:
In order for Code Completion to work Pycharm constructs Skeletons. (Basically Python files with empty Classes/Methods that match the C++ API and can then be used like any other Python file for code completion.)
If I locate these files they don't appear to be complete and look something like this:
If this is indeed the skeleton (the file is called vtkRenderingPython.py) then shouldn't there be empty function declarations?
The result is that I get code completion for the classnames, but not the functions. For a library this huge that's rather annoying. Is there an easy way to get this working, or is this just a limitation I have to live with? Is there maybe a way to get complete Skeletons and replace the ones I have here? Am I missing the point entirely?

After another couple of hours I tried my luck with the PyDev extension for Eclipse. I didn't think that would work, but to my surprise it did! No settings necessary, it just worked out of the box.
The only drawback is that inherited methods are not shown in code completion. The base class is shown in the documentation window though so you can get the available functions by creating a temporary object of the base class and scrolling through the code completion there.

Debugging IDAPython Scripts outside of IDAPro

I'm kinda new to scripting for IDA - nevertheless, I've written a complex script I need to debug, as it is not working properly.
It is composed of a few different files containing a few different classes.
Writing line-by-line in the commandline is not effective for obvious reasons.
Running a whole script from the File doesn't allow debugging.
Is there a way of using the idc, idautils, idaapi not from within IDA?
I've written the script on PyDev for Eclipse, I'm hoping for a way to run the scripts from within it.
A similar question is, can the api classes I have mentioned work on idb files without IDA having them loaded?
Thanks.

Now I may be wrong for I haven't written any IDA script for long time. But as far as I remember the answer to your first question is no. There is the part that loads the IDA script and prepare the whole environment so you could re implement it and create your own environment, however I would not recommend that.
What I can tell you is to consider running your script from command line if automation is what you are aiming for. IDA python (as well as any other IDA plugin) have a good support for running scripts from command line. For performance you can also run the TUI version of IDA.
There is also a hack for that enables you to launch a new python interpreter in the middle of the IDA script. It is useful for debugging a current state yet you will still need to edit the python file every time to launch the interpreter.
Here is the hack:
import code
all = globals()
all.update(locals())
code.interact(local = all)
Anyway - logs are good and debug prints are OK.
Good luck :)

We've just got a notice from one of our users that the latest version of WingIDE supports debugging of IDAPython scripts. I think there are a couple of other programs using the same approach (import a module to do RPC debugging) that might work.

Looking for a Python editor that will let me collapse functions

I really loved this feature when I used Eclipse for Java programming, but I can't find the same functionality for a Python editor. IDLE and Pyscripter are nice, but they don't help in this area.
Basically, I just want the option to collapse or otherwise hide functions that I don't feel like looking at for a while. Know of anything like this?

In addition to the aforementioned (great) editors, you might want to give PyDev a shot as well.

Geany can do this.

Notepad++ has this feature.

Komodo Edit IDE, for Windows, Mac and Linux, for Python, PHP, Ruby, JavaScript, Perl and Web Dev.

I've used Komodo Edit and Notepad++ in the past but my current preference is Sublime Text Edit 2.
Although not free (and actually quite expensive), it can be used in free mode with only an occasional reminder and no other restrictions.
It is actually written in Python so you get a Python console built in - you can also get other consoles such as JavaScript. It is VERY flexible & has some very good features. It is also has an excellent community with loads of very useful plugins.
It is much lighter on resource usage than Komodo, can use Textmate bundles directly (so gets loads of formatting options for different file types). It is cross-platform and doesn't even need installation on Windows.

Pycharm CE, from Jet Brains, indeed, wonderful. Functions and comments collapse is ready out of the box, as well as edit helpers. Project files and assets organization, integrated python console, powerful debugging tools,... Then, lots of plugins: git integration, tinycode view, extra languages' helpers and highlighters,.... anything you need when coding, but simple and easy to use. There's a Pro (paid) version for those who want even more.
https://www.jetbrains.com/pycharm/download
(This question is more than 10 years old. I got surprised, nobody answered about Pycharm before...)

Which Python IDE can run my script line-by-line?

I wouldn't call myself programmer, but I've started learning Python recently and really enjoy it.
I mainly use it for small tasks so far - scripting, text processing, KML generation and ArcGIS.
From my experience with R (working with excellent Notepad++ and NppToR combo) I usually try to work with my scripts line by line (or region by region) in order to understand what each step of my script is doing.. and to check results on the fly.
My question: is there and IDE (or editor?) for Windows that lets you evaluate single line of Python script?
I have seen quite a lot of discussion regarding IDEs in Python context.. but havent stubled upon this specific question so far.
Thanks for help!

If you like R's layout. I highly recommend trying out Spyder. If you are using windows, try out Python(x,y). It is a package with a few different editors and a lot of common extra modules like scipy and numpy.

The only one I've had success with is Eclipse with Pydev

It's not an IDE, but you can use pdb to debug and step through your Python code. I know Emacs has built in support for it, but not so much about other editors (or IDEs) that will run in Windows.

If you are on Windows, give Pyscripter a try -- it offers comprehensive, step-through debugging, which will let you examine the state of your variables at each step of your code.

PyCharm from JetBrains has a very nice debugger that you can step through code with.
Django and console integration built in.

Rodeo seems to be new contender on the IDE market and the docs indicate that running lines of code is possible. I also have to admit it looks and behaves pretty good so far!

WingIDE, I've been using it successfully for over a year, and very pleased with it.

I use Notepad++ for most of my Windows based Python development and for debugging I use Winpdb. It's a cross platform GUI based debugger. You can actually setup a keyboard shortcut in Notepad++ to launch the debugger on your current script:
To do this go to "Run" -> "Run ..." in the menu and enter the following, making sure the path points to your winpdb_.pyw file:
C:\python26\Scripts\winpdb_.pyw "$(FULL_CURRENT_PATH)"
Then choose "Save..." and pick a shortcut that you wish to use to launch the debugger.
PS: You can also setup a shortcut to execute your python scripts similarly using this string instead:
C:\python26\python.exe "$(FULL_CURRENT_PATH)"

The upcoming RStudio 1.2 is so good that you have to try to write some python with it. 🙌

I would plump for EMACS all round.
If you're looking for a function to run code line by line (or a region if you have one highlighted), try adding this to your .emacs (I'm using python.el and Pymacs):
;; send current line to *Python
(defun my-python-send-region (&optional beg end)
(interactive)
(let ((beg (cond (beg beg)
((region-active-p)
(region-beginning))
(t (line-beginning-position))))
(end (cond (end end)
((region-active-p)
(copy-marker (region-end)))
(t (line-end-position)))))
(python-shell-send-region beg end)))
(add-hook 'python-mode-hook
'(lambda()
(local-set-key [(shift return)] 'my-python-send-region)))
I've bound it to [shift-Return]. This is borrowed from here. There's a similar keybinding for running .R files line by line here. I find both handy.

I like vim-ipython. With it I can <ctrl>+s to run a specific line. Or several lines selected on visual modes. Take a look at this video demo.

Visual Studio and PTVS: http://www.hanselman.com/blog/OneOfMicrosoftsBestKeptSecretsPythonToolsForVisualStudioPTVS.aspx
(There is also a REPL inside VS)

The Pythonwin IDE has a built-in debugger at lets you step through your code, inspect variables, etc.
http://starship.python.net/crew/mhammond/win32/Downloads.html
http://sourceforge.net/projects/pywin32/
The package also includes a bunch of other utility classes and modules that are very useful when writing Python code for Windows (interfacing with COM, etc.).
It's also discussed in the O'Reilly book Python Programming On Win32 by Mark Hammond.

Take the hint: The basic Python Read-Execute-Print-Loop (REPL) must work.
Want Evidence?
Here it is: The IDE's don't offer much of an alternative. If REPL wasn't effective, there's be lots of very cool alternatives. Since REPL is so effective, there are few alternatives.
Note that languages like Java must have a step-by-step debugger because there's no REPL.
Here's the other hint.
If you design your code well, you can import your libraries of functions and classes and exercise them in REPL model. Many, many Python packages are documented by exercising the package at the REPL level and copying the interactions.
The Django documentation -- as one example -- has a lot of interactive sessions that demonstrate how the parts work together at the REPL prompt.
This isn't very GUI. There's little pointing and clicking. But it seems to be effective.

You need to set the keyboard shortcut for "run selection" in
Tools > Preferences > Keyboard shortcuts
Then, select the line and hit the "run selection" shortcut

Light Table was doing that for me, unfortunately it is discontinued:
INLINE EVALUTION No more printing to the console in order to view your
results. Simply evaluate your code and the results will be displayed
inline.

How do I prepare myself for a summer of working on Python using Linux environment?

I have used just Windows for programming so far. Now, I have an internship starting in two weeks and I will be using just Linux environment with Python programming language. I've installed Ubuntu on my system but have no exposure to shell scripting.
I need some advice on how I can quickly learn to use the Linux terminal quickly. Any books or web resources that you can suggest?
Also, is there a particular IDE that is generally preferred for Python programming on Linux, or is Vim preferred? How can I best prepare myself for the internship ahead?
Thanks for taking the time.

As an intern you'll want to use the tools your mentor is most comfortable with. If you get stuck you'll be able to ask for advice quickly.
Learning your way around either vi, vim, or emacs to start with will help. The basic concepts used in one will transfer to the other. You'll need to be able to open and read files, search through files, edit and save files, and learn how to apply any python formatting helpers correctly.
You should also familiarize yourself with version control if you haven't already. Again any one will do, you need to focus on concepts and etiquette rather than the specific tool.
The goal of the internship (and really your entire time at university) should be used to learn concepts rather than specific tools. If you learn the concepts you'll be well placed to apply those concepts using any tool. You will also "learn how to learn" a new tool, which is really valuable.

Your lack of shell scripting knowledge shouldn't matter in this case, although it won't be hard to learn. I read over some shell tutorials and put them into practice. Try doing everything from the command line, including find (grep), find/replace all (sed), finding files (find), automating things using python scripts etc. Basically, don't cheat. You'll pick up a lot this way. You'll also probably end up wondering how you ever managed with Windows.
What I use depends on the project. I really like Eclipse+PyDev but that's my personal preference, I also use Vim depending on where I am/what I'm doing. Remember you can just type python from the command line and it drops you into the python environment.

I recommend Eclipse + PyDev too. You can get started quickly with this develop environment. I also recommend the website Dive Into Python. It provides you a online free version of Dive Into Python book, which is very easy to read, easy to understand, and very suitable for Python beginners. If you really want a paper book at hand, Learning Python, a.k.a. The Animal Guide, is simply the best.

Learn to understand man(ual) pages.
For almost any old linux command/program there is a man page which usually explains the command in good detail.
So basics for filesystem navigation:
Show directory contents (list)
ls
Show hidden files
ls -a
Show details
ls -l
Change directory
cd /full/path/name
Print current directory
pwd
Delete a file
rm file
Delete a directory (recursive)
rm -r directoryName
Make a directory
mkdir directoryName
Move (or rename) a file
mv /path/to/file /new/path/to/file
Show the man page for mv
man mv
Learning vim might be necessary, depending on your intern environment. I do my Python (and everything that isn't simple text editing) in Eclipse. You should in any case learn enough to open a file, makes some changes and save the changes in Vim.
Keep in mind, Ubuntu is very easy. To make things harder on yourself, use the command line for every conceivable thing. Open programs by typing their names into a terminal. Browse your files with the terminal. Do simple editing with vim. That should provide good practice for the day you need to SSH into a computer in Neverland and download and install a local copy of your favorite interpreter from source in order to set up a cron job to run a script to play a clock noise.

In addition to the great advice already written, I'd suggest you install IPython (Open a terminal with Applications>Accessories>Terminal and type):
sudo apt-get install ipython
Also at the terminal, you can then type ipython to start the Python interpreter.
Unlike the built in python interpreter, ipython gives you tab completion.
For example, if you type the name of an object followed by a period and TAB (e.g. sys.[TAB]), ipython will show you (almost) all of object's attributes.
Type a question mark after an object name (e.g. sys?), and you get documentation on that object.
This is a great way to explore Python.

have no exposure to shell scripting
Good! You've got Python so hopefully there should be no need to resort to writing actual scripts with the shell. It may be more powerful than DOS batch files, but it's just as ugly.
I need some advice on how I can quickly learn to use the Linux terminal quickly.
Something like this?
As well as learning the commands, you'll want to get used to using tab-completion and arrow key command recall (if you don't already do that with the Windows Command Prompt), scrolling with shift-arrows, and so on. Also useful to know the & (perform in background) command suffix, ctrl-C-to-stop, ctrl-Z-to-pause, jobs, and screen.
Incidentally if you will be spending any amount of time in the interactive Python interpreter it is well worth adding tab completion there, too. (This is just as much the case on Windows, but on Win you tend not to get pyreadline by default.)
is there a particular IDE that is generally preferred for Python programming on Linux
Just like on Windows, there are IDEs available if you want them but many people just use a normal text editor. vim is fine if that's what you like. nano is another in-terminal text editor you usually get that's relatively simple. Ubuntu's default desktop-based editor gedit is also fine. It's a matter of personal taste.
(If you are interning at a particular company they might have their own development environment they'd prefer you to use.)

For a Python IDE, I recommend using either IDLE or Eclipse with PyDev.
Keep in mind you can also just use python on the linux command-line. It supports loading code from files, and if you use two command windows then one of them will be your "REPL" where you will be running python and dynamically loading code - and the other window can run your editor.
Regarding linux command line, I cannot recommend any great resources. However, you will be off to a great start if you immerse yourself in this environment and only use linux for the next 2 weeks. Just keep learning, and when you do not know how to do something, read a manpage or google it to find the answer.

for a very beginner intro to the command line, check out: http://en.flossmanuals.net/CommandLineIntro/GettingStarted

As far as a Python editor goes, I personally prefer to use SciTE. It's just a programmer's text editor with syntax highlighting for various languages. I prefer a lightweight editor over a more complicated environment, but if you want a full-fledged IDE you can always try out NetBeans, IDLE, or Komodo (all of which are available in both Windows and Linux).

as for terminall and quick way to understand it's and learn it there are a nice cheat sheets on net like this:
http://fosswire.com/post/2007/8/unixlinux-command-cheat-sheet/