We Keep Coding

Pygtk object uninitialized

Hi during the creation of multiple objects in pygtk application, calling some of these object return that they are uninitialised when passing there contents to signals (noting the value are showen correctly in the application)
sections = config.sections()
for section in sections:
box= gtk.Table (3,len(config.options(section)),False)
box.set_col_spacings(2)
box.set_row_spacings(2)
box.show()
label = gtk.Label(section)
label.show()
notebook.append_page (box,label)
for i,option in enumerate(config.options(section)):
optionlabel = gtk.Label(option)
optionvalue = gtk.Entry ()
optionvalue.set_text(config.get(section,option))
--> optionvalue.connect("activate", enter_callback,optionvalue, label, optionlabel)
box.attach(optionlabel,0,1,i,i+1,xoptions=gtk.SHRINK|gtk.FILL,yoptions=gtk.SHRINK)
box.attach(optionvalue,1,2,i,i+1,yoptions=gtk.SHRINK)
box.show_all()
at first I thought that the variables are not in scope but they were ( I've tested several objects and found them working the last 3 lines
Linuxcnc ini.py:70: GtkWarning: IA__gtk_entry_get_text: assertion "GTK_IS_ENTRY (entry) failed
print (widget.get_text())
None
<gtk.Label object at 0x9f44a54 (uninitialized at 0x0)>
<gtk.Label object at 0x9f44c0c (uninitialized at 0x0)>
0
<gtk.Label object at 0x9f4c8ec (GtkLabel at 0xa1a3350)>
<gtk.Label object at 0x9f4dc0c (GtkLabel at 0xa1a3450)>

For this one I think you may need to look in the Gtk system library itself.
(The source code that is used to produce ".so", ".dll" or ".dylib" file, that has the routines that your Python program is accessing).
You can look a the contents of such binary files with commands like "strings", "nm", and "objdump".
But based the nature of the error, it looks like the fix will be done in the source code of the library itself, not the Python program.
Sorry I cannot give more detail than that.

Python import (Could be python or Lego Mindstorms libs)

I'll start this by saying I'm not the most familiar with python, and this issue could be a more general python thing that I don't get (i.e. a glaringly obvious duplicate).
In the python bindings for the ev3, a motor is referenced like this:
# hardware.py #
import ev3dev.ev3 as ev3
motor = ev3.LargeMotor('outA')
motor.connected
Where 'outA' is the output port on the robot that the motor is connected to.
If I then do:
$:python hardware.py
I get no issues and I can use the motor normally. However, if I write a new file
# do_something.py #
from hardware import *
I get an error:
Exception TypeError: "'NoneType' object is not callable" in <bound method LargeMotor.__del__ of <ev3dev.core.LargeMotor object at 0xb67d2fd0>> ignored
Does anyone know why this is happening? Is it a python thing or an ev3 thing?
My reason for wanting to import in this way is so that I can do all of the hardware setup in one file (a sizeable chunk of code) and then import this to the files that actually make the robot perform tasks.
I know that NoneType is the type of None in python, I just don't know why a direct compile works but an import doesn't.
1st Edit:
Ok, so I compiled it as:
$:python hardware.py do_something.py
$:python do_something.py
And this gave no errors.
However, upon request, I've added more code, hardware.py is the same:
# do_something.py #
from hardware import *
counter = 0
while True:
if (counter >= 1000):
break
motor.run_direct(duty_cycle_sp = 20)
counter += 1
I.e. run the motor at a cycle speed of 20 until we've been through a thousand loop iterations.This works, and runs until the loop breaks and the script ends. The same NoneType error is then given and the motor continues to run even though the script has finished. The behaviour is the same with a KeyboardInterrupt. There is no traceback given, just that error message.

First of all, python is a language at which it's codes are formed from words, and on the other hand the Lego Mindstorms "language" is formed of simple blocks. That said, logically the two languages cannot be mixed together and have nothing in common. And with having vast experience with both, I never found any thing in common between them.

Why is multiprocessing copying my data if I don't touch it?

I was tracking down an out of memory bug, and was horrified to find that python's multiprocessing appears to copy large arrays, even if I have no intention of using them.
Why is python (on Linux) doing this, I thought copy-on-write would protect me from any extra copying? I imagine that whenever I reference the object some kind of trap is invoked and only then is the copy made.
Is the correct way to solve this problem for an arbitrary data type, like a 30 gigabyte custom dictionary to use a Monitor? Is there some way to build Python so that it doesn't have this nonsense?
import numpy as np
import psutil
from multiprocessing import Process
mem=psutil.virtual_memory()
large_amount=int(0.75*mem.available)
def florp():
print("florp")
def bigdata():
return np.ones(large_amount,dtype=np.int8)
if __name__=='__main__':
foo=bigdata()#Allocated 0.75 of the ram, no problems
p=Process(target=florp)
p.start()#Out of memory because bigdata is copied?
print("Wow")
p.join()
Running:
[ebuild R ] dev-lang/python-3.4.1:3.4::gentoo USE="gdbm ipv6 ncurses readline ssl threads xml -build -examples -hardened -sqlite -tk -wininst" 0 KiB

I'd expect this behavior -- when you pass code to Python to compile, anything that's not guarded behind a function or object is immediately execed for evaluation.
In your case, bigdata=np.ones(large_amount,dtype=np.int8) has to be evaluated -- unless your actual code has different behavior, florp() not being called has nothing to do with it.
To see an immediate example:
>>> f = 0/0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ZeroDivisionError: integer division or modulo by zero
>>> def f():
... return 0/0
...
>>>
To apply this to your code, put bigdata=np.ones(large_amount,dtype=np.int8) behind a function and call it as your need it, otherwise, Python is trying to be hepful by having that variable available to you at runtime.
If bigdata doesn't change, you could write a function that gets or sets it on an object that you keep around for the duration of the process.
edit: Coffee just started working. When you make a new process, Python will need to copy all objects into that new process for access. You can avoid this by using threads or by a mechanism that will allow you to share memory between processes such as shared memory maps or shared ctypes

The problem was that by default Linux checks for the worst case memory usage, which can indeed exceed memory capacity. This is true even if the python language doesn't exposure the variables. You need to turn off "overcommit" system wide, to achieve the expected COW behavior.
sysctl `vm.overcommit_memory=2'
See https://www.kernel.org/doc/Documentation/vm/overcommit-accounting

python drag drop init issues

I'm confused as to how I would implement a drag and drop ability for a window and then have the url appear in the textbox.
I've updated where I am stuck at
class controller(NSWindow):
#File to encode or decode
form_file = IBOutlet()
mainWindow = IBOutlet()
#drag and drop ability
def awakeFromNib(self):
self.registerForDraggedTypes_([NSFilenamesPboardType, None])
print 'registerd drag type'
def draggingEntered_(self, sender):
print 'dragging entered doctor who'
pboard = sender.draggingPasteboard()
types = pboard.types()
opType = NSDragOperationNone
if NSFilenamesPboardType in types:
opType = NSDragOperationCopy
return opType
def performDragOperation_(self,sender):
print 'preform drag operation'
pboard = sender.draggingPasteboard()
successful = False
if NSFilenamesPboardType in pboard.types():
print 'my actions finally working'
fileAStr = pboard.propertyListForType_(NSFilenamesPboardType)[0]
print type(fileAStr.encode('utf-8'))
successful = True
print self.form_file
return successful
I can drop the file but I am unable to refrence the form_file outlet from inside of the performDragOperation function. As you can see I am attempting to print it but it returns a NoneType error.
(reason '<type 'exceptions.TypeError'>: 'NoneType' object is not callable') was raised during a dragging session

I believe your problem here was that something earlier in the Responder Chain was handling -[draggingEntered:] and rejecting the drag, before your window could get to it.
For a typical AppKit app, before getting to the window, an action message goes to the First Responder from the NIB, and anything hooked onto the back of it, then the innermost view and its delegate, then all of its ancestor views and their delegates. So if, for example, you have a text edit view that handles drag messages, and you drag over that view, the window won't see it.
Anyway, there are lots of ways to debug this, but the simplest one is to just have each method iterate the nextResponder() chain from self, printing (or logging.logging or NSLogging) the result. Then you can see who you're blocking.
Since there were a bunch of other issues we talked about in the comments, I'm not sure if this was the one that actually solved your problem. But one thing in particular to bring up:
I don't think PyObjC was part of the problem here. When the Cocoa runtime sends an ObjC message like -[draggingEntered:] to a PyObjC object, the PyObjC runtime handles that by looking for a draggingEntered_ method and converting it magically. (Well, I say magic, but it's simple science. Also, because it's sonic, it doesn't do wood.) You need #IBAction in the same places an ObjC program would need (IBAction), which are documented pretty well in the Cocoa documentation.
Meanwhile, one general-purpose debugging tip for PyObjC code (or just about any other event-loop-based or otherwise non-linear code). When you get an error like this:
(reason '<type 'exceptions.TypeError'>: 'NoneType' object is not callable') was raised during a dragging session
It's hard to figure out what exactly went wrong. But you can handle the exception inside the function that raised, and get all the information you want. You can wrap a try/except around each line to figure out which line raised, you can print the whole traceback instead of just the summary, etc.

Python mmap ctypes - read only

I think I have the opposite problem as described here. I have one process writing data to a log, and I want a second process to read it, but I don't want the 2nd process to be able to modify the contents. This is potentially a large file, and I need random access, so I'm using python's mmap module.
If I create the mmap as read/write (for the 2nd process), I have no problem creating ctypes object as a "view" of the mmap object using from_buffer. From a cursory look at the c-code, it looks like this is a cast, not a copy, which is what I want. However, this breaks if I make the mmap ACCESS_READ, throwing an exception that from_buffer requires write privileges.
I think I want to use ctypes from_address() method instead, which doesn't appear to need write access. I'm probably missing something simple, but I'm not sure how to get the address of the location within an mmap. I know I can use ACCESS_COPY (so write operations show up in memory, but aren't persisted to disk), but I'd rather keep things read only.
Any suggestions?

I ran into a similar issue (unable to setup a readonly mmap) but I was using only the python mmap module. Python mmap 'Permission denied' on Linux
I'm not sure it is of any help to you since you don't want the mmap to be private?

Ok, from looking at the mmap .c code, I don't believe it supports this use case. Also, I found that the performance pretty much sucks - for my use case. I'd be curious what kind performance others see, but I found that it took about 40 sec to walk through a binary file of 500 MB in Python. This is creating a mmap, then turning the location into a ctype object with from_buffer(), and using the ctypes object to decipher the size of the object so I could step to the next object. I tried doing the same thing directly in c++ from msvc. Obviously here I could cast directly into an object of the correct type, and it was fast - less than a second (this is with a core 2 quad and ssd).
I did find that I could get a pointer with the following
firstHeader = CEL_HEADER.from_buffer(map, 0) #CEL_HEADER is a ctypes Structure
pHeader = pointer(firstHeader)
#Now I can use pHeader[ind] to get a CEL_HEADER object
#at an arbitrary point in the file
This doesn't get around the original problem - the mmap isn't read-only, since I still need to use from_buffer for the first call. In this config, it still took around 40 sec to process the whole file, so it looks like the conversion from a pointer into ctypes structs is killing the performance. That's just a guess, but I don't see a lot of value in tracking it down further.
I'm not sure my plan will help anyone else, but I'm going to try to create a c module specific to my needs based on the mmap code. I think I can use the fast c-code handling to index the binary file, then expose only small parts of the file at a time through calls into ctypes/python objects. Wish me luck.
Also, as a side note, Python 2.7.2 was released today (6/12/11), and one of the changes is an update to the mmap code so that you can use a python long to set the file offset. This lets you use mmap for files over 4GB on 32-bit systems. See Issue #4681 here

Ran into this same problem, we needed the from_buffer interface and wanted read only access. From the python docs https://docs.python.org/3/library/mmap.html "Assignment to an ACCESS_COPY memory map affects memory but does not update the underlying file."
If it's acceptable for you to use an anonymous file backing you can use ACCESS_COPY
An example: open two cmd.exe or terminals and in one terminal:
mm_file_write = mmap.mmap(-1, 4096, access=mmap.ACCESS_WRITE, tagname="shmem")
mm_file_read = mmap.mmap(-1, 4096, access=mmap.ACCESS_COPY, tagname="shmem")
write = ctypes.c_int.from_buffer(mm_file_write)
read = ctypes.c_int.from_buffer(mm_file_read)
try:
while True:
value = int(input('enter an integer using mm_file_write: '))
write.value = value
print('updated value')
value = int(input('enter an integer using mm_file_read: '))
#read.value assignment doesnt update anonymous backed file
read.value = value
print('updated value')
except KeyboardInterrupt:
print('got exit event')
In the other terminal do:
mm_file = mmap.mmap(-1, 4096, access=mmap.ACCESS_WRITE, tagname="shmem")
i = None
try:
while True:
new_i = struct.unpack('i', mm_file[:4])
if i != new_i:
print('i: {} => {}'.format(i, new_i))
i = new_i
time.sleep(0.1)
except KeyboardInterrupt:
print('Stopped . . .')
And you will see that the second process does not receive updates when the first process writes using ACCESS_COPY