We Keep Coding

PDB strategy for debugging Python/Bottle web application hang

I have a multi-threaded Bottle/Python web application that only occasionally hangs. The source can be downloaded here: https://github.com/whbrewer/spc
I've been trying to figure out what is causing the hang, but since it only happens very occasionally and I have not yet figured out how to reproduce it, I try to spend some time whenever it hangs to discover what is causing the problem. I have tried attaching a debugger to the running process using:
gdb /usr/bin/python -p 32489
but this only gives me the libc traceback such as:
0x00007fd2adbdb0fc in __libc_waitpid (pid=32490, stat_loc=stat_loc#entry=0x7ffd8aac545c,
options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:31
31 return INLINE_SYSCALL (wait4, 4, pid, stat_loc, options, NULL);
From this output, I was able to find a similarly asked question which provided some useful advice, but ultimately did not help solve the problem. However, from reading another related informative post, I discovered that the Python debugger PDB can be used to solve this problem. So, my question is, how would I go about implementing PDB in a Python/Bottle application so when it occasionally hangs, I can get a python traceback?
One thing I have noted, is that when the hang happens, it seems the server thread turns into a zombie process, so it shows a defunc python process, while the other threads seem to continue to run. I have tried to use different web servers with Bottle, such as cherrypy and rocket, but both gave the same result.

Attaching GDB to a dying process in linux

I would like to attach gdb to a dying process, because the program runs in production and I need to debug it there, if I open the program with gdb it slows down and the computers are not that great. I tried to catch signals in the application and attach gdb there but it just works if I send them signals myself. When the program stalls (multi-threaded program, and the main thread gets a deadlock or somehow gets stuck (or apparently stuck)), and the user forces it to quit in the Desktop Environment (LXDE), I can't catch no signal. The program is all python with PySide for the graphical interface. Just care about linux.
My idea is to create a kernel driver and try too hook process termination or signals sending in there but since it would be much of a hassle I would like to ask if there is some tool for this kind of thing or some information that I could make use of. Thanks.

There might be a way to do what you want, but if you can't perhaps it would be sufficient to freeze the program and inspect its memory image?
Enable core dump file generation before it starts, and then once the process is hosed, terminate it with kill. Then use gdb to open the core file and analyze what was happening.

Python/Flask: Application is running after closing

I'm working on a simple Flask web application. I use Eclipse/Pydev. When I'm working on the app, I have to restart this app very often because of code changes. And that's the problem. When I run the app, I can see the frame on my localhost, which is good. But when I want to close this app, just click on the red square which should stop applications in Eclipse, sometimes (often), the old version of application keeps running so I can't test the new version. In this case the only thing which helps is to force close every process in Windows Task Manager.
Will you give me any advice how to manage this problem? Thank you in advance.
EDIT: This maybe helps: Many times, I have to run the app twice. Otherwise I can't connect.

I've faced the same problem and solved it. I think it may help.
When we run a flask based site locally it is assigned to a TCP port: 5000 and the Default IP: 127.0.0.1:5000
Sometimes TCP connection remains even after closing the program or terminating the code. So, The idea is kill the TCP connection. You can do it from command-prompt(cmd)
Two Steps to Follow:
1. See the Process ID(PID) for the running TCP connection.
Go to cmd and type:
netstat -ano
Kill The Process By PID. Command for this: taskkill /f /im [PID]. Example is showed bellow.
taskkill /f /im 7332

I've had a very similar thing happen to me. I was using CherryPy rather than Flask, but my solution might still work for you. Oftentimes browsers save webpages locally so that they don't have to re-download them every time the website is visited. This is called caching, and although it's very useful for the average web user, it can be a real pain to app developers. If you're frequently generating new versions of the application, it's possible that your browser is displaying an old version of the app that it has cached instead of the most up to date version. I recommend clearing that cache every time you restart your application, or disabling the cache altogether.

This actually shouldn't happen with the latest versions of PyDev (i.e.: since PyDev 3.4.1: http://pydev.org/history_pydev.html, PyDev should kill all the subprocesses of the main process).
So, can you check which PyDev version are you using?
If you're in the latest version of PyDev, you can use Ctrl+Shift+F9 to terminate/relaunch by default.
But as you're dealing with flask, you should be able to use it to reload automatically on code-changes without doing anything by setting use_reloader=True.
I.e.: I haven't actually tested, but its documentation says that you can set the reload flag for that run(use_reloader=True) -- and PyDev should even be able to debug it (I'll take a better look and improve the PyDev docs on that area later on).

Intermittent "Page Not Found" when using Django through Pydev

I recently started developing Django using Pydev in Eclipse, along with several other people I know, and I keep losing several hours of productivity a day to this error that I can't seem to debug. Essentially, sometimes after starting up my django run configuration, I'll go to the URL http://127.0.0.1:8000/foo and it simply spits me back this response:
<h1>Page not found: /foo</h1>
At first I could reproduce this behavior all the time; any time I had multiple workspaces of eclipse open and shut down my Django server once, the server wouldn't actually shut down, but it would just keep serving the page not founds until I shut down both instances of Eclipse.
Stopping doing that and it seemed to work, but then it intermittently seemed to start back up again. When this happens, it will literally not stop happening until I close eclipse, and the eclipse.exe process will hang (and continue to server Page not found pages) until I manually go into the process and kill it. Sometimes, that won't even fix it, and it'll break itself the first time I try to start Django back up.
I'm on the stable build of PyDev and Eclipse 3.6.1.
Any idea what's going on? This is unbelievable frustrating, especially because I tend to like the IDE with this exception.

Are you by any chance using a run configuration to launch your development server? Because that kept happening to me too. It seems that eclipse isn't cleanly closing django when you stop it from the IDE and it's holding on to the http port. In that sense you're sending requests to a zombie server instance, hence the not founds. I have since resorted to launching django from the command line. Not the prettiest solution, I admit.

have python process run similar to how redis process does (in the background)

I asked this question on superuser, but haven't gotten a response. Maybe here would of been more appropriate.
When I start my redis server with redis-server, even after I close the terminal or logout the process will still be there when I log back in, but my python bottle server python server.py will turn off if I close to terminal or logout. How do I get similar behavior as redis in python.

The easy way is to run the process through screen or tmux.
You could also try doing something with e.g. python-daemon on Unix, or various other approaches for running daemons.