How can I spawn a pdb-like debugger in a Splunk application (meaning: an application made for and ran by Splunk) ?
I have no control over the python process itself, so simply putting import pdb; pdb.set_trace() in the code will just result in the web app crashing.
I guess the ideal solution would be to
either run the python part of Splunk manually, so I have control over it (I tried this, but it didn't work correctly; mongodb daemon wasn't starting, among other things)
use the good old import pdb; pdb.set_trace() breakpoint but attach to the process somehow, so I'm able to manipulate the debugger (I tried gdb, but nothing worked as expected -- perhaps I didn't use it correctly)
One way to debug might be a remote debugger, like remote-pdb.
It behaves similar as pdb. You can set a breakpoint, then configure the interface and a TCP port where the debugger will listen.
from remote_pdb import RemotePdb
RemotePdb('127.0.0.1', 4444).set_trace()
After that you can simply connect to the debugger using telnet telnet 127.0.0.1 4444
More info:
https://pypi.org/project/remote-pdb/
Related
I would like to set some debugging command (like import ipdb; ipdb.set_trace()) that would run debugger in jupyter (I would have to run a HTTP server).
Does anybody know about something like this?
Context: I have a long running tasks that are processed by a scheduler (not interactive mode). I would like to be able to debug such a task while running it the same way.
I need to run code in "detached" (not interactive). And when some
error is detected I would like to run debugger. That's why I've been
thinking about remote debugger/jupyter notebook or whatever. So - by
default there is no debugging session - so I think that PyCharm remote
debugger is not a case.
Contrary to what you might seem to think here, you do not really need to run the code in a "debugging session" to use remote debugging.
Try the following:
Install pydevd in the Python environment for your "detached" code:
pip install pydevd
Within the places in that code, where you would have otherwise used pdb.set_trace, write
import pydevd; pydevd.settrace('your-debugger-hostname-or-ip')
Now whenever your code hits the pydevd.settrace instruction, it will attempt to connect to your debugger server.
You may then launch the debugger server from within Eclipse PyDev or Pycharm, and have the "traced" process connect to you ready for debugging. Read here for more details.
It is, of course, up to you to decide what to do in case of a connection timeout - you can either have your process wait for the debugger forever in a loop, or give up at some point. Here is an example which seems to work for me (ran the service on a remote Linux machine, connected to it via SSH with remote port forwarding, launched the local debug server via Eclipse PyDev under Windows)
import pydevd
import socket
from socket import error
def wait_for_debugger(ex, retries=10):
print("Bam. Connecting to debugger now...")
while True:
try:
pydevd.settrace()
break
except SystemExit:
# pydevd raises a SystemExit on connection failure somewhy
retries -= 1
if not retries: raise ex
print(".. waiting ..")
def main():
print("Hello")
world = 1
try:
raise Exception
except Exception as ex:
wait_for_debugger(ex)
main()
It seems you should start the local debug server before enabling port forwarding, though. Otherwise settrace hangs infinitely, apparently believing it has "connected" when it really hasn't.
There also seems to be a small project named rpcpdb with a similar purpose, however I couldn't get it to work right out of the box so can't comment much (I am convinced that stepping through code in an IDE is way more convenient anyway).
I'm developing a cassandra storage finder for graphite-api.
graphite-api is installed via pip and run via gunicorn so I can't just call the script with a debugger but want to use interactive debugging.
When I import pdb in my storage finder and set a breakpoint, the code will halt there, but how can I connect now to the headless running pdb in the script?
Or is my approach to this debugging problem the wrong one and this has to be done in a completely other way?
pdb gives control over to gunicorn, which is not what you want. Have a look at rpdb or other remote debugging solutions.
I'm writing an Python Bottle application (Python 2.7.2 and Bottle 0.10.9) and developing it in the WingIDE (3.2.8-1) Professional for Linux. This all works well, except when I want to debug the Bottle application. I have it running in standalone mode within WingIDE, but it won't stop at any of my break points in the code, even if I set Bottle.debug(False). Does anyone have any suggestions/ideas about how I can setup Bottle so it will stop on breakpoints within WingIDE?
If you have the reloader set to true bottle starts a subproccess for the actual app. In Wing you need to turn off the reloader, then it should work.
run(reloader=False).
But you will have to restart the app in wing every time you make changes.
Are you debugging under WSGI using wingdbstub.py or launching bottle from the IDE? I'm not that familiar with bottle but a common problem is a web framework's reload mechanism running code in a sub-process that is not debugged. I'm not certain bottle would do that under WSGI, however, but printing the process id at time of importing wingdbstub (or startup if launching from the IDE) and again at the line where the breakpoint is missed would rule this in our out. The "reloader" arg for Bottle.__init__ may be relevant here. If set to True, try setting it to False when debugging under Wing.
Another thing to try is to raise an exception on purpose where the breakpoint is (like "assert 0, 'test exception'" and see if this exception is reported in Wing's debugger in the Exceptions tool and if so whether Wing also manages to open the source code. If bottle is running code in a way that doesn't make it possible to find the source code then this would still stop on the assertion (Wing's debugger stops on all assertions by default even if the host code handles the exception) but it would fail to show the debug file and would put up a message in the status area (at bottle of IDE screen and in the Messages tool) that indicates the file name the debug process specified. Depending on this it may be possible to fix the problem (but would require modifying Bottle if the file name is something like "".
BTW, to insert code that is only run under Wing's debugger us something like this:
import os
if 'WINGDB_ACTIVE' in os.environ:
# code here
If this doesn't help please email support at wingware dot com.
I need a reliable way to check if a Twisted-based server, started via twistd (and a TAC-file), was started successfully. It may fail because some network options are setup wrong. Since I cannot access the twistd log (as it is logged to /dev/null, because I don't need the log-clutter twistd produces), I need to find out if the Server was started successfully within a launch-script which wraps the twistd-call.
The launch-script is a Bash script like this:
#!/usr/bin/bash
twistd \
--pidfile "myservice.pid" \
--logfile "/dev/null" \
--python \
myservice.tac
All I found on the net are some hacks using ps or stuff like that. But I don't like an approach like that, because I think it's not reliable.
So I'm thinking about if there is a way to access the internals of Twisted, and get all currently running Twisted applications? That way I could query the currently running apps for the the name of my Twisted application (as I named it in the TAC-file) to start.
I'm also thinking about not using the twistd executable but implementing a Python-based launch script which includes the twistd-content, like the answer to this question provides, but I don't know if that helps me in getting the status of the server to run.
So my question is just: is there a reliable not-ugly way to tell if a Twisted Server started with twistd was started successfully, when twistd-logging is disabled?
You're explicitly specifying a PID file. twistd will write its PID into that file. You can check the system to see if there is a process with that PID.
You could also re-enable logging with a custom log observer which only logs your startup event and discards all other log messages. Then you can watch the log for the startup event.
Another possibility is to add another server to your application which exposes the internals you mentioned. Then try connecting to that server and looking around to see what you wanted to see (just the fact that the server is running seems like a good indication that the process started up properly, though). If you make it a manhole server then you get the ability to evaluate arbitrary Python code, which lets you inspect any state in the process you want.
You could also just have your application code write out an extra state file that explicitly indicates successful startup. Make sure you delete it before starting the application and you'll have a fine indicator of success vs failure.
It is a rather strange 'bug'.
I have written a cherrypy based server. If I run it this way:
python simple_server.py > out.txt
It works as expected.
Without the the redirection at the end, however, the server will not accept any connection at all.
Anyone has any idea?
I am using python 2.4 on a Win XP professional machine.
Are you running the script in an XP "command window"? Otherwise (if there's neither redirection nor command window available), standard output might simply be closed, which might inhibit the script (or rather its underlying framework).
CherryPy runs in a "development" mode by default, which includes logging startup messages to stdout. If stdout is not available, I would assume the server is not able to start successfully.
You can change this by setting 'log.screen: False' in config (and replacing it with 'log.error_file: "/path/to/error.log"' if you know what's good for you ;) ). Note that the global config entry 'environment: production' will also turn off log.screen.