I have a ListStore in PyGTK, which has a bunch of rows. There is a background job processing the data represented by the rows, and when it finishes, it needs to update the row. Of course, to do this, it needs to know which row to update, and is thus keeping an iterator to the row around. However, during the background jobs life, the user might remove the row. This is OK — we just replace the stored iterator with "None", and the background job continues along merrily. The problem is that when the row is removed, the iterators don't compare as equal, and nothing gets set to None. In fact, no two iterators, AFAIK, compare equal. The problem, in a minimal example, is this:
>>> store = gtk.ListStore(int)
>>> store.insert(1)
<GtkTreeIter at 0x1d49600>
>>> print store[0].iter == store[0].iter
False
False, yet they're the same iterator! (I'm aware they are different instances, but they represent the same thing, and they define a __eq__ method.) What am I missing here, and how do I keep track of rows in a ListStore for later updating?
Try using the list store's .get_path(iter) method, and compare the resulting paths, instead of comparing the iterators directly.
UPDATE: You can just call set_value with the invalid iter. gtk will give you a warning but will not throw an exception or anything. It probably just checks whether it's a valid iter anyway.
I would approach this differently — here's what I've done in a similar situation:
The underlying data object represented in each row is an instance of a GObject
This GObject subclass has a bunch of properties
When the property changes, it emits the notify::myproperty signal
At the same time:
My ListStore stores these objects, and uses the gtk.TreeViewColumn.set_cell_data_func() method to render each column (see note below)
For each object/row, my object managing the TreeView connects to the notify::myproperty
The function connected to this notify::... signal triggers the row-changed signal on the ListStore
Some code:
def on_myprop_changed(self, iter, prop):
path = self.model.get_path(iter)
self.model.row_changed(path ,iter)
def on_thing_processed(self, thingdata):
# Model is a ListStore
tree_iter = self.model.append((thingdata,))
# You might want to connect to many 'notify::...' signals here,
# or even have your underlying object emit a single signal when
# anything is updated.
hid = thingdata.connect_object('notify::myprop',
self.on_myprop_changed,
tree_iter)
self.hids.add((thingdata, hid))
I keep the hids in a list so I can disconnect them when the table is cleared. If you let individual rows get removed, you'll probably need to store them in a map (path -> hid, or object -> hid).
Note: You need to remember that set_cell_data_func causes the row to re-check its information every time there's a redraw, so the underlying function should just be a lookup function, not an intensive computation. Practically speaking, because of this you could get away with not doing the "connect-to-signal/emit-row-changed" procedure, but personally I feel better knowing that there won't be any edge cases.
Related
I am using LLDB Python scripting support to add custom Variable Formatting for a complex C++ class type in XCode.
This is working well for simple situations, but I have hit a wall when I need to call a method which uses a pass-by-reference parameter, which it populates with results. This would require me to create a variable to pass here, but I can't find a way to do this?
I have tried using the target's CreateValueFromData method, as below, but this doesn't seem to work.
import lldb
def MyClass(valobj, internal_dict):
class2_type = valobj.target.FindFirstType('class2')
process = valobj.process
class2Data = [0]
data = lldb.SBData.CreateDataFromUInt32Array(process.GetByteOrder(), process.GetAddressByteSize(), class2Data)
valobj.target.CreateValueFromData("testClass2", data, class2_type)
valobj.EvaluateExpression("getType(testClass2)")
class2Val = valobj.frame.FindVariable("testClass2")
if not class2Val.error.success:
return class2Val.error.description
return class2Val.GetValueAsUnsigned()
Is there some way to be able to achieve what I'm trying to do?
SBValue names are just labels for the SBValue, they aren't guaranteed to exist as symbols in the target. For instance if the value you are formatting is an ivar of some other object, it's name will be the ivar name... And lldb does not inject new SBValue's names into the symbol table - that would end up causing lots of name collisions. So they don't exist in the namespace the expression evaluator queries when looking up names.
If the variable you are formatting is a pointer, you can get the pointer value and cons up an expression that casts the pointer value to the appropriate type for your getType function, and pass that to your function. If the value is not a pointer, you can still use SBValue.AddressOf to get the memory location of the value. If the value exists only in lldb (AddressOf will return an invalid address) then you would have to push it to the target with SBProcess.AllocateMemory/WriteMemory, but that should only happen if you have another data formatter that makes these objects out of whole cloth for its own purposes.
It's better not to call functions in formatters if you can help it. But if you really must call a function in your data formatter, you should to do that judiciously.
They can cause performance problems (if you have an array of 100 elements of this type, your formatter will require 100 function calls in the target to render the array... That's 200 context switches between your process and the debugger, plus a bunch of memory reads and writes) for every step operation.
Also, since you can't ensure that the data in your value is correct (it might represent a variable that has not been initialized yet, or already deallocated) you either need to have your function handle bad data, or at least be prepared for the expression to crash. lldb can clean up the stack and suppress the exception from crashes, but it can't undo any side-effects the expression might have had before crashing.
For instance, if the function you called took some lock before crashing that it was expecting to release on the way out, your formatter will damage the state of the program. So you have to be careful what you call...
And by default, EvaluateExpression will allow all threads to run so that expressions don't deadlock against a lock held by another thread. You probably don't want that to happen, since that means looking at the locals of one thread will "change" the state of another thread. So you really should only call functions you are sure don't take locks. And use the version of EvaluateExpression that takes an SBExpressionOption, in which you set the SBExpressionOptions.StopOthers to True, and SetTryAllThreads to False.
The title is pretty self-explanatory. I'm doing something like:
gen = obj #some generator instance running
frame = obj.gi_frame
prevframe = frame.f_back
But I always get None for prevframe. Why is this the case. Also, is there some workaround for this?
CONTEXT: I'm trying to write a simple call stack method to determine what called a particular function. I'm using twisted manhole and telnetting into a running process, where I then execute these commands but I can't seem to access the previous frames.
To the best of my knowledge, this is both intentional and cannot be worked around. The code in cpython responsible for it is here, which indicates that the reference to the previous frame is broken as soon as the generator yields (or excepts out) in order to prevent issues with reference counting. It also appears that the intended behavior is that the generator's previous frame is swapped out every time it's entered, so while it's not running, the notion of "the parent frame" doesn't make much sense.
The correct way to do this, at least in the post-mortem context, is to use traceback objects, which have their frame lists linked in the reverse order, tb_next instead of f_back.
I currently need to partially create a Python object and be able to update it for some time. Although, I must not be able to update it once I used the object as a dictionary key.
Of course there is the solution of marking the fields as private, which is mostly a warning for the programmer, and I will actually go for that solution.
But I stumbled on another solution and I want to know if this could be a good idea, or if it could simply go horribly wrong. Here it is:
class Foo():
def __init__(self, bar):
self._bar = bar
self._has_been_hashed = False
def __hash__(self):
self._has_been_hashed = True
return self._bar.__hash__()
def __eq__(self, other):
return self._bar == other._bar
def __copy__(self):
return Foo(self._bar)
def set_bar(self, bar):
if self.has_been_hashed:
raise FooIsNowImmutable
else:
self._bar = bar
Some testing proved it to work as desired, I can no longer use set_bar once I, say, used my object as a dictionary key.
What do you think? Is it a good idea? Will it turn against me? Is there an easier way? And is it somehow a bad practice?
Doing it that way is a bit fragile, since you never know when something might be used as a dictionary key, or when its hash might be called for some other reason. An object isn't supposed to "know" whether it's being used as a dictionary key. It will be confusing to have code that may raise an exception just because some other code somewhere else put the object in a dictionary.
Following the Python philosophy of "explicit is better than implicit", it would be safer to just give your object a method called .finalize() or .lock() or something, which would set a flag indicating the object is immutable. You could also reverse the exception-raising logic, so that __hash__ raises an exception if the object is not yet locked (rather than mutation raising an exception if the object has been hashed).
You would then call .lock() when you're ready to make the object immutable. It makes more sense to explicitly set it immutable when you're done with whatever mutating you need to do, rather than implicitly assuming that as soon as you use it in a dictionary, you're done mutating it.
You can do that, but I'm not sure I'd recommend it. Why do you need it in a dictionary?
It requires a lot more awareness of the state of the object... think a file object. Would you put one in a dictionary? It has to be opened for a lot of the functions to work, and once it's closed, you can't do them anymore. The user has to be aware in the surrounding code which state the object is in.
For files, that makes sense - after all, you don't normally hold files open across large parts of your program, or if you do, they have very defined init and close codes; something similar has to make sense for your object. Especially if you have some APIs that take the object, but expect an immutable version, and others that take the same object, but expect to change it...
I have used the lock method before, and it works well for complex, read-only objects that you want to initialize once and then make sure no one is messing with. E.G. you load a copy of a (say, English) dictionary from disk... it has to be mutable while you are populating it, but you don't want anyone to accidentally modify it, so locking it is a great idea. I would only use it if it was a one-time lock though - something you are locking and unlocking seems like a recipe for disaster.
There are two solutions IMHO if you just want to create a version you can use in hashable places. First is to explicitly create an immutable copy when you put it in a dictionary - tuple and frozenset are examples of this sort of behaviour... if you want to put a list in a dict, you can't, but you can create a tuple from it first, and that can be hashed. Create a frozen version of your object, then it's very clear by looking at the object type whether it's expected to be mutable or immutable, and so cases where it was used incorrectly are easily seen.
Second, if you really want it to be hashable, but need it to be mutable... that's actually legal, but implemented a little different. It goes back to the idea of hashing... hashing is used both for optimized lookups, and equality.
The first is to ensure you can get objects back... you put something in a dictionary, and it hashes to a value of 4 - goes in slot 4. Then you modify it. Then you go to look it up again, and now it hashes to 9 - there's nothing in slot 9, or worse, a different object, and you're broken.
Second is equality - for things like sets, I need to know if my object is already in there. I can hash, but if you know anything about hashing, you still need to check equality to check for hash collisions.
That doesn't preclude supporting __hash__ and being mutable, but it's unusual. You need to decide for your item what makes it the same, even though it's mutable. What you need to do then is give each object a unique id. Technically, you may be able to get away with id(self), but something like the uuid module is probably a better possibility. The UUID4 (or technically, the hash of the UUID4) is what determines both the hash and equality; two objects that contain the same UUID4 should be the exact same object; two objects that have the exact same data but a different UUID4 would be different object.
I'm making a Windows application using wx.grid.Grid that will handle very large Microsoft Excel documents. Currently it opens a file with 17 columns and 12 000+ rows quite quickly, and I can scroll around smoothly. (This is a virtual table that operates on data from a custom table class.)
Anyway, the problem starts when I use custom grid cell attribute objects, e.g:
grid.SetAttr(row, col, SomeGridCellAttr('#FF0000'))
Once every cell in a row has a custom grid cell attribute, the performance of the grid just goes 99.9% into the toilet. Instead of smooth scrolling around I'm lucky if I get 1 redraw per 3 seconds. I fixed most of these instances by using grid.SetColAttr instead, which restored performance to it's previous smoothness, but there is one case where that won't work. The application iterates over every cell in a column (12 000 cells), performs some processing on the data and applies a custom grid cell attribute based on the results. Once that is done, the grid becomes a sluggish nightmare to work with.
Is there any way of removing this gruesome performance hit and retain the custom cell attributes? I suspect the answer is quite simple to somebody who knows the inner workings of the grid and it's cell attributes.
Thanks.
Setting a cell attribute will add a new GridCellAttr to the list of the GridCellAttrProvider.
As the list grows, looking up a specific attribute for a cell (by iterating through the list and comparing the coordinates) gets slower and slower.
You can try to speed it up by implementing your own PyGridTableBase.SetAttr and GetAttr (using a dict for example):
EDIT: Updated code to allow overwriting attributes and emulate the default implementations attribute ownership.
class MyTable(wx.grid.PyGridTableBase):
atts = {}
def Hash(self,row,col):
#FIXME: assumes a constant number of rows and rows > cols
return col + row * self.GetNumberRows()
def SetAttr(self,attr,row,col):
HASH = self.Hash(row, col)
if HASH in self.atts:
# decrement usage count of existing attr
self.atts[HASH].DecRef()
#assign new attribute
self.atts[HASH] = attr
def GetAttr(self,row,col,kind):
HASH = self.Hash(row, col)
if HASH in self.atts:
attr = self.atts[HASH]
attr.IncRef() # increment reference count
return attr
return None
To allow setting entire rows and columns, you'll also have to implement:
def SetRowAttr(self,attr,row):
for col in range(self.GetNumberCols()):
attr.IncRef() # increment reference count for SetAttr
self.SetAttr(attr,row,col)
attr.DecRef() # attr passed to SetRowAttr no longer needed
def SetColAttr(self,attr,col):
for row in range(self.GetNumberRows()):
attr.IncRef()
self.SetAttr(attr,row,col)
attr.DecRef()
NOTE: when passing a GridCellAttr to Set*Attr(), the default implementation will take ownership of the attribute.
To re-use the same attribute (stored in a class variable for example), you have to either Clone() it or increment its usage count (IncRef())
before passing it to a Set*Attr() method (cloning may increase memory consumption).
The above example lacks proper removal of attributes: SetAttr() could check for a None attr and decrement the ref count at the coordinates specified, then delete the entry from the dict.
SetCol/RowAttr() can be optimized by adding dicts for row and col, similar to SetAttr(). GetAttr() could then check for an existing entry in the row and col dict and merge/override the attribute(s) with the one from the cell dict (that's the principle used by the default implementation). For proper cleanup of the dict(s), call DecRef on every entry before .clear().
Alternatively, you could derive from wx.grid.GridCellAttrProvider and attach it with PyGridTableBase.SetAttrProvider(). This would prevent direct access to the table, however.
Let's say I have a bpy.types.Object containing a bpy.types.Mesh data field; how can I apply one of the modifiers associated with the object, in order to obtain a NEW bpy.types.Mesh, possibly contained within a NEW bpy.types.Object, thus leaving the original scene unchaged?
I'm interested in applying the EdgeSplit modifier right before exporting vertex data to my custom format; the reason why I want to do this is to have Blender automatically and transparently duplicate the vertices shared by two faces with very different orientations.
I suppose you're using the 2.6 API.
bpy.ops.object.modifier_apply (modifier='EdgeSplit')
...applies to the currently active object its Edge Split modifier. Note that it's object.modifier_apply (...)
You can use
bpy.context.scene.objects.active = my_object
to set the active object. Note that it's objects.active.
Also, check the modifier_apply docs. Lot's of stuff you can only do with bpy.ops.*.
EDIT: Just saw you need a new (presumably temporary) mesh object. Just do
bpy.ops.object.duplicate()
after you set the active object and the new active object then becomes the duplicate (it retains any added modifier; if it was an object named 'Cube', it duplicates it, makes it active and names it 'Cube.001') to which you can then apply the modifier. Hope this was clear enough :)
EDIT: Note, that bpy.ops.object.duplicate() uses not active object, but selected. To ensure the correct object is selected and duplicated do this
bpy.ops.object.select_all(action = 'DESELECT')
object.select = True
There is another way, which seems better suited for custom exporters: Call the to_mesh method on the object you want to export. It gives you a copy of the object's mesh with all the modifiers applied. Use it like this:
mesh = your_object.to_mesh(scene = bpy.context.scene, apply_modifiers = True, settings = 'PREVIEW')
Then use the returned mesh to write any data you need into your custom format. The original object (including it's data) will stay unchanged and the returned mesh can be discarded after the export is finished.
Check the Blender Python API Docs for more info.
There is one possible issue with this method. I'm not sure you can use it to apply only one specific modifier, if you have more than one defined. It seems to apply all of them, so it might not be useful in your case.