We Keep Coding

How to add a software watchpoint following a breakpoint on gdb

I am trying to debug a C program which allocates and frees various instances of a particular structure during its lifetime. At some point, one of these instances is getting corrupted.
To debug it, I would like to set watchpoints shortly after these structures are allocated and remove the watchpoints shortly before they are free'd. For that, I wrote a python gdb script (see below) which implements two subclasses of gdb.Breakpoint: BreakpointAlloc() and BreakpointFree(). The former has a stop() method which adds a watchpoint on the allocated structure and the latter has a stop() method which removes the watchpoint. (Watchpoints are kept in a dict indexed by a string containing the address of the allocated instance.)
Due to the large number of instances allocated (over 100), I cannot use hardware watchpoints. When using software watchpoints (by first running gdb.execute("set can-use-hw-watchpoints 0")), however, the program seems to wedge and I can't tell what's happening.
#!/usr/bin/env python
import gdb
wps = {}
def BreakpointAlloc(gdb.Breakpoint):
def stop(self):
ptr = gdb.parse_and_eval("ptr").address
wp = gdb.Breakpoint("*({0})({1})".format(ptr.type, ptr), gdb.BP_WATCHPOINT)
wps["{0}".format(ptr)] = wp
return False
def BreakpointFree(gdb.Breakpoint):
def stop(self):
ptr = gdb.parse_and_eval("ptr").address
wp = wps["{0}".format(ptr)]
wp.delete()
del wps["{0}".format(wp)]
return False
bp_alloc = BreakpointAlloc("prog.c:111")
bp_free = BreakpointFree("prog.c:222")
gdb.execute("set can-use-hw-watchpoints 0")
The documentation for the gdb python API suggests you shouldn't do what I'm doing. I believe that may explain why the program is wedging:
Function: Breakpoint.stop (self)
...
You should not alter the execution state of the inferior (i.e., step, next, etc.), alter the current frame context (i.e., change the current active frame), or alter, add or delete any breakpoint.
Considering the documentation, I have also tried modifying my program to add/del the watchpoints via stop events (see below). When using software watchpoints, the same problem occurs: the program seems to wedge.
#!/usr/bin/env python
import gdb
wps = {}
bp_alloc = gdb.Breakpoint("prog.c:111")
bp_free = gdb.Breakpoint("proc.c:222")
def stopAlloc():
ptr = gdb.parse_and_eval("ptr").address
wp = gdb.Breakpoint("*({0})({1})".format(ptr.type, ptr), gdb.BP_WATCHPOINT)
wps["{0}".format(ptr)] = wp
def stopFree():
ptr = gdb.parse_and_eval("ptr").address
wp = wps["{0}".format(ptr)]
wp.delete()
del wps["{0}".format(ptr)]
def handleStop(stopEvent):
for bp in stopEvent.breakpoints:
if bp == bp_alloc:
stopAlloc()
elif bp == bp_free:
stopFree()
gdb.events.stop(handleStop)
gdb.execute("set can-use-hw-watchpoints 0")
Any ideas on how I could manipulate watchpoints off breakpoints?
(Or any other ideas on how to debug this issue?)

Trying to execute a set of functions in order

I am a Python noob, and I'm trying to make a washing device, programming the interface with Python. For now, the machine should work like that:
Wash
Tell washing is complete
Dry
Tell drying is complete
For this, after times for washing/drying are entered, a button is pressed:
button1 = Button(window.tk, command = lambda:main_process(int(varWashtime.get()), int(varDrytime.get())))
def main_process(wash_seconds,dry_seconds):
wash(wash_seconds)
stop_wash()
dry(dry_seconds)
stop_dry()
return
def wash(seconds):
varWashStarted.set("Washing Started")
Timer(seconds,idle_fnc).start()
return
def stop_wash():
varWashStarted.set("Washing Stopped")
Timer(3,idle_fnc,()).start()
return
def dry(seconds):
varDryStarted.set("Drying Started")
Timer(seconds,idle_fnc,()).start()
return
def stop_dry():
varDryStarted.set("Drying Stopped")
return
def idle_fnc():
pass
return
Here, I used the function idle_fnc to make threading.Timer properly work.
I found out that I can just use Timer to call other functions after each other, but I would prefer to return from a function, then branch to a new one.
My problem is, as I click the button, the whole thing executes without waiting; I instantly see "Washing Stopped" and "Drying Stopped" on the corresponding label, without the delays triggered.
What is the problem?

client not receiving data via TCP socket

I have started network programming using Python and am working on a basic peer-to-peer chat client-server application. I got it working for console, but am facing problem while developing a GUI.
This is the code for my client script. It is sending data to the server but is unable to receive /display the data sent from server, I am at a loss. Please show the error in my code and the solution.
from socket import *
from tkinter import *
host="127.0.0.1"
port=1420
buffer=1024
server=(host,port)
clientsock=socket(AF_INET,SOCK_STREAM)
clientsock.connect(server)
class ipbcc(Frame):
def __init__(self,master):
Frame.__init__(self,master)
self.grid()
self.create()
self.connect()
def write(self,event):
msg=self.e.get()
clientsock.send(msg.encode())
def create(self):
self.pic=PhotoImage(file="logo.gif")
self.label=Label(self,image=self.pic)
self.label.grid(column=0)
self.wall=Text(self,width=70,height=20,wrap=WORD)
self.wall.grid(row = 0, column = 1, columnspan = 2, sticky = W)
self.e=Entry(self,width=50)
self.e.grid(row = 1, column = 1, sticky = W)
self.e.bind('<Return>',self.write)
def add(self,data):
self.wall.insert(END,data)
def connect(self):
def xloop():
while 1:
data=clientsock.recv(buffer).decode()
print(data)
self.add(data)
root=Tk()
root.title("IPBCC v0.1")
app=ipbcc(root)
root.mainloop()
PS: Python Version 3.3 and there is no problem in the server script.

Your connect function defines a function called xloop, but it doesn't call that function, or return it, or store it somewhere for anyone else to call it. You need to call that function for it to do anything.
Of course if you just call it directly inline, it will run forever, meaning you never get back to the event loop, and the UI freezes up and stops responding to the user.
There are two options for this: threading, or polling.
The obvious way to do this is with a background thread. The basic idea is very simple:
def connect(self):
def xloop():
while 1:
data=clientsock.recv(buffer).decode()
print(data)
self.add(data)
self.t = threading.Thread(target=xloop)
self.t.start()
However, there are two problems with this.
First, there's no way to stop the background thread. When you try to exit the program, it will wait for the background thread to stop—which means it will wait forever.
There's an easy solution to that one: if you make it a "daemon thread", it will be summarily killed when the main program exits. This is obviously no good for threads that are doing work that could be corrupted if interrupted in the middle, but in your case that doesn't seem to be a problem. So, just change one line:
self.t = threading.Thread(target=xloop, daemon=True)
Second, that self.add method needs to modify a Tkinter widget. You can't do that from a background thread. Depending on your platform, it may fail silently, raise an exception, or even crash—or, worse, it may work 99% of the time and fail 1%.
So, you need some way to send a message to the main thread, asking it to do the widget modification for you. This is a bit complicated, but Tkinter and Threads explains how to do it.
Alternatively, you could use mtTkinter, which intercepts Tkinter calls in background threads and passes them to the main thread automatically, so you don't have to worry about it.
The other option is to change the blocking xloop function into a nonblocking function that polls for data. The problem is that you want to wait on Tkinter GUI events, but you also want to wait on the socket.
If you could integrate the socket into the main event loop, that would be easy: a new message coming in would be handled just like any other event. Some of the more powerful GUI frameworks like Qt give you ways to do this, but Tkinter does not. A reactor framework like Twisted can tie itself into Tkinter and add it for you (or at least fake nicely). But if you want to stick with your basic design, you have to do it yourself.
So, there are two options:
Give Tkinter full control. Ask it to call your function every, say, 1/20th of a second, and in the function do a non-blocking check. Or maybe loop around non-blocking checks until there's nothing left to read.
Give the socket control. Ask Tkinter to call your function every time it gets a chance, and block for 1/20th of a second checking for data before returning to Tkinter.
Of course 1/20th of a second may not be the right length—for many applications, no answer is really correct. Anyway, here's a simple example:
def poll_socket(self):
r, w, x = select.select([clientsock], [], [], 0)
if r:
data=clientsock.recv(buffer).decode()
print(data)
self.add(data)
self.after(50, self.poll_socket)
def connect(self):
self.after(50, self.poll_socket)

You define xloop, however you never actually call it as far as I can see.
I would suggest you look into using threads - the threading module in the standard library would be one way to go. Then, in your code you will be able to create a thread running the xloop function, without stopping the rest of your code. Alternatively, you could remove the loop from xloop (or indeed just put the code in the function into the connect function) and call it periodically, using widget.after(milliseconds, a_function)
I'd also like to mention that from amodule import * is considered bad practice (although tkinter is one of the exceptions to this rule).

It might help to follow the flow. The "app=ipbcc(root)" step would call "self.connect()" and that has a "def xloop():" that has the step "data=clientsock.recv". But, then somebody needs to invoke xloop(). Who does that? Btw, why do have a function inside a method?
Also, I don't see anybody invoking the "clientsock.send(msg.encode())" via the write() method. I am not familiar with the Tinker part (and what the mainloop() does), so can you please check if there are callers to send() and the recv() call.

How do I restructure this code so Pickle 'Buffer Region' errors do not occur

I was wondering if anyone had any good solutions to the pickling error I am having at the moment. I am trying to set my code up to open several different processes in parallel, each with a fitting process to be display on a matplotlib canvas in real time. Within my main application, I have a button which activates this function:
def process_data(self):
process_list = []
for tab in self.tab_list:
process_list.append(mp.Process(target=process_and_fit, args=(tab,)))
process_list[-1].start()
process_list[-1].join()
return
As you may notice, a 'tab' (PyQt4.QtGui.QTabWidget object) is passed to the function process_and_fit, which I have noticed is not able to be pickled readily (link here) .
However, I am not certain how to change the code to get rid of the frame being passed since it needs to be called in the process_and_fit function indirectly. By indirectly I mean something like this: (psuedo code again)
def process_and_fit(tab): # this just sets up and starts the fitting process
result = lmfit.Minimizer(residual, parameters, fcn_args=(tab,))
result.prepare_fit()
result.leastsq()
def residual(params, tab):
residual_array = Y - model
tab.refreshFigure()
return residual_array
class tab(QtGui.QTabWidget):
def __init__(self, parent, spectra):
# stuff to initialize the tab widget and hold all of the matplotlib lines and canvases
# This just refreshes the GUI stuff everytime that the parameters are fit in the least squares method
def refreshFigure(self):
self.line.set_data(self.spectra.X, self.spectra.model)
self.plot.draw_artist(self.line)
self.plot.figure.canvas.blit(self.plot.bbox)
Does anyone know how to get around this pickling error since the tab associated with a process should have only one set of data associated with it? I looked at Steven Bethard's approach but I really didn't understand where to put the code or how to utilize it. (I am a chemical engineer, not a computer scientist so there's a lot that I don't understand)
Any help is greatly appreciated.
EDIT: I added the links in that I forgot about, as requested.

The main issue is that you can't make UI changes from a separate process from the main UI thread (the one that all of your Qt calls are in). You need to use a mp.Pipe or mp.Queue to communicate back to the main process.
def process_data(self):
for tab in self.tab_list:
consumer, producer = mp.Pipe()
process_list.append(mp.Process(target=process_and_fit, args=(producer,)))
process_list[-1].start()
while (true):
message = consumer.recv() # blocks
if message == 'done':
break
# tab.spectra.X, tab.spectra.model = message
tab.refreshFigure()
process_list[-1].join()
return
def process_and_fit(pipe_conn):
...
pipe_conn.send('done')
def residual(params, pipe_conn):
residual_array = Y - model
pipe_conn.send('refresh') # or replace 'refresh' with (X, model)
return residual_array
One more thing to note: blocking for the consumer.recv() will probably hang the GUI thread. There are plenty of resources to mitigate this, the question "subprocess Popen blocking PyQt GUI" will help, since you should probably switch to QThreads. (Qthread: PySide, PyQt)
The advantage of using QThreads instead of Python threads is that with QThreads, since you're already in Qt's main event loop, you can have asynchronous (non-blocking) callbacks to update the UI.

PyQt & unittest - Testing signal and slots

I have a pyqt application that I'm writing unit tests for, and it relies heavily on signals and slots. To properly test it, I have to check that the correct signals are sent.
What is the best way to do this? I see that the Qt library has a QSignalSpy, but I can't find any reference to this in PyQt. The only option I can think of is to mock emit, e.g.
import testedmodule
def myemit(signal):
....
testedmodule.QObject.emit = myemit
but I'm hoping there is a better way.
Edit:
My module is run as a thread, in that case overriding emit of an instance no longer worked after starting the thread so I updated the code above to reflect this.

You can try connecting a slot to your signal, prepare your test, then call qApp.processEvents() to let the signal propagate. But I don't think it's 100% reliable.
It's a pity that QSignalSpy is not part of PyQt indeed.

This is a more elaborate version of what I suggested myself, not necessarily the best solution for unittest, but I think it will be of interest to others that come across this:
Posted by Carlos Scheidegger on the pyqt mailing list (http://thread.gmane.org/gmane.comp.python.pyqt-pykde/9242/focus=9245)
_oldConnect = QtCore.QObject.connect
_oldDisconnect = QtCore.QObject.disconnect
_oldEmit = QtCore.QObject.emit
def _wrapConnect(callableObject):
"""Returns a wrapped call to the old version of QtCore.QObject.connect"""
#staticmethod
def call(*args):
callableObject(*args)
_oldConnect(*args)
return call
def _wrapDisconnect(callableObject):
"""Returns a wrapped call to the old version of QtCore.QObject.disconnect"""
#staticmethod
def call(*args):
callableObject(*args)
_oldDisconnect(*args)
return call
def enableSignalDebugging(**kwargs):
"""Call this to enable Qt Signal debugging. This will trap all
connect, and disconnect calls."""
f = lambda *args: None
connectCall = kwargs.get('connectCall', f)
disconnectCall = kwargs.get('disconnectCall', f)
emitCall = kwargs.get('emitCall', f)
def printIt(msg):
def call(*args):
print msg, args
return call
QtCore.QObject.connect = _wrapConnect(connectCall)
QtCore.QObject.disconnect = _wrapDisconnect(disconnectCall)
def new_emit(self, *args):
emitCall(self, *args)
_oldEmit(self, *args)
QtCore.QObject.emit = new_emit
just call enableSignalDebugging(emitCall=foo) and spy your signals until
you're sick to your stomach :)

Note QSignalSpy is available as QtTest.QSignalSpy in PyQt5.