I'm trying to write a simple GUI front end for Plurk using pyplurk.
I have successfully got it to create the API connection, log in, and retrieve and display a list of friends. Now I'm trying to retrieve and display a list of Plurks.
pyplurk provides a GetNewPlurks function as follows:
def GetNewPlurks(self, since):
'''Get new plurks since the specified time.
Args:
since: [datetime.datetime] the timestamp criterion.
Returns:
A PlurkPostList object or None.
'''
offset = jsonizer.conv_datetime(since)
status_code, result = self._CallAPI('/Polling/getPlurks', offset=offset)
return None if status_code != 200 else \
PlurkPostList(result['plurks'], result['plurk_users'].values())
As you can see this returns a PlurkPostList, which in turn is defined as follows:
class PlurkPostList:
'''A list of plurks and the set of users that posted them.'''
def __init__(self, plurk_json_list, user_json_list=[]):
self._plurks = [PlurkPost(p) for p in plurk_json_list]
self._users = [PlurkUser(u) for u in user_json_list]
def __iter__(self):
return self._plurks
def GetUsers(self):
return self._users
def __eq__(self, other):
if other.__class__ != PlurkPostList: return False
if self._plurks != other._plurks: return False
if self._users != other._users: return False
return True
Now I expected to be able to do something like this:
api = plurk_api_urllib2.PlurkAPI(open('api.key').read().strip(), debug_level=1)
plurkproxy = PlurkProxy(api, json.loads)
user = plurkproxy.Login('my_user', 'my_pass')
ps = plurkproxy.GetNewPlurks(datetime.datetime(2009, 12, 12, 0, 0, 0))
print ps
for p in ps:
print str(p)
When I run this, what I actually get is:
<plurk.PlurkPostList instance at 0x01E8D738>
from the "print ps", then:
for p in ps:
TypeError: __iter__ returned non-iterator of type 'list'
I don't understand - surely a list is iterable? Where am I going wrong - how do I access the Plurks in the PlurkPostList?
When you define your own __iter__ method, you should realize that that __iter__ method should return an iterator, not an iterable. You are returning a list, not an iterator to a list, so it fails. You can fix it by doing return iter(self._plurks), for example.
If you wanted to do something a little more complex, like process each item in self._plurks as it's being iterated over, the usual trick is to make your __iter__ method be a generator. That way, the returnvalue of the call to __iter__ is the generator, which is an iterator:
def __iter__(self):
for item in self._plurks:
yield process(item)
The __iter__ method should return an object which implements the next() method.
A list does not have a next() method, but it has an __iter__ method, which returns a listiterator object. The listiterator object has a next() method.
You should write:
def __iter__(self):
return iter(self._plurks)
As an alternative, you can also define the next() function and have __iter__() return self. See Build a Basic Python Iterator for a nice example.
Related
I created an iterator to increment the figure number in various plotting function calls:
figndx=itertools.count()
I then proceed to call these throughout my code, passing next(figndx) as an argument to increment the value I use for the figure number: - for ex:
an.plotimg(ref_frame,next(figndx),'Ref Frame')
an.plotimg(new_frame,next(figndx),'New Frame')
etc...
After some particular function call, I want to read back the figndx value and store it in a variable for later use. However, when I look at figndx , it returns count(7), for example. How do I extract the '7' from this?
I've tried :
figndx
figndx.__iter__()
and I can't find anything else in the 'suggested' methods (when I type the dot (.)) that will get the actual iterator value. Can this be done?
`
Just wrap a count object
class MyCount:
def __init__(self, *args, **kwargs):
self._c = itertools.count(*args, **kwargs)
self._current = next(self._c)
def __next__(self):
current = self._current
self._current = next(self._c)
return current
def __iter__(self):
return self
def peek(self):
return self._current
You can create yourself a peeker, using itertools.tee, and encapsulate the peek:
from itertools import count, tee
def peek(iterator):
iterator, peeker = tee(iterator)
return iterator, next(peeker)
Then you can call it like
figndx = count(1)
next(figndx)
next(figndx)
figndx, next_value = peek(figndx)
next_value
# 3
I'm receiving an unknown number of records for background processing from generators. If there is a more important job, I have to stop to release the process.
The main process is best described as:
def main():
generator_source = generator_for_test_data() # 1. contact server to get data.
uw = UploadWrapper(generator_source) # 2. wrap the data.
while not interrupt(): # 3. check for interrupts.
row = next(uw)
if row is None:
return
print(long_running_job(row)) # 4. do the work.
Is there a way to get to __next__ without having to plug __iter__?
Having two steps - (1) make an iterator, then (2) iterate over it, just seems clumsy.
There are many cases where I'd prefer to submit a function to a function manager (mapreduce style), but in this case I need an instantiated class with some settings. Registering a single function can therefor only work if that function alone is __next__
class UploadWrapper(object):
def __init__(self, generator):
self.generator = generator
self._iterator = None
def __iter__(self):
for page in self.generator:
yield from page.data
def __next__(self):
if self._iterator is None: # ugly bit.
self._iterator = self.__iter__() #
try:
return next(self._iterator)
except StopIteration:
return None
Q: Is there a simpler way?
Working sample added for completeness:
import time
import random
class Page(object):
def __init__(self, data):
self.data = data
def generator_for_test_data():
for t in range(10):
page = Page(data=[(t, i) for i in range(100, 110)])
yield page
def long_running_job(row):
time.sleep(random.randint(1,10)/100)
assert len(row) == 2
assert row[0] in range(10)
assert row[1] in range(100, 110)
return row
def interrupt(): # interrupt check
if random.randint(1,50) == 1:
print("INTERRUPT SIGNAL!")
return True
return False
class UploadWrapper(object):
def __init__(self, generator):
self.generator = generator
self._iterator = None
def __iter__(self):
for ft in self.generator:
yield from ft.data
def __next__(self):
if self._iterator is None:
self._iterator = self.__iter__()
try:
return next(self._iterator)
except StopIteration:
return None
def main():
gen = generator_for_test_data()
uw = UploadWrapper(gen)
while not interrupt(): # check for job interrupt.
row = next(uw)
if row is None:
return
print(long_running_job(row))
if __name__ == "__main__":
main()
Your UploadWrapper seems overtly complex, there is more than a single simpler solution.
My first thought is to ditch the class altogether and just use a function instead:
def uploadwrapper(page_gen):
for page in page_gen:
yield from page.data
Just replace uw = UploadWrapper(gen) with uw = uploadwrapper(gen), and that'll work.
If you insist on the class, you can just get rid of the __next__() and replace uw = UploadWrapper(gen) with uw = iter(UploadWrapper(gen)), and it'll work.
In either case, you must also catch the StopIteration in the caller. __next__() is supposed to raise StopIteration when it's done, and not return None, like yours does. Otherwise, it won't work with things expecting a well-behaving iterator, eg. for loops.
I think you might have some misconceptions about how it all is supposed to fit together, so I'll try my best to explain how it's supposed to work, to the best of my knowledge:
The point of __iter__() is that if you have eg. a list, you can get multiple independent iterators by calling iter(). When you have a for loop, you're essentially first getting an iterator with iter() and then calling next() on it on every loop iteration. If you have two nested loops that use the same list, the iterators and their positions are still separate so there's no conflict. __iter__() is supposed to return an iterator for the container it's on, or if it's called on an iterator, it's supposed to just return self. In that sense, it's kind of wrong for UploadWrapper not to return self in __iter__(), since it wraps a generator and so can't really give independent iterators. As for why leaving out __next__() works, it's because when you define a generator (ie. use yield in a function), the generator has an __iter__() (that returns self, as it should) and __next__() that does what you'd expect. In your original code, you're not really using __iter__() at all for what it's supposed to be used: the code works even if you rename it to something else! This is because you never call iter() on the instance, and just directly call next().
If you wanted to do it "properly" as a class, I think something like this might suffice:
class UploadWrapper(object):
def __init__(self, generator):
self.generator = generator
self.subgen = iter(next(generator).data)
def __iter__(self):
return self
def __next__(self):
while True:
try:
return next(self.subgen)
except StopIteration:
self.subgen = iter(next(self.generator).data)
How do I get otherImages to return the string in it so that I can replace a word within it when called from 'narrow' method?
def otherImages(self):
self.wfile.write(bytes("<div id='menu_button'><a href='/narrow'><img src='../images/menu_button.png'></a></div>", "utf8"))
#^word I want to replace
def contentList(self, skip_name=''): # All content methods in list
methods = [self.title, self.containerDIV, self.heading, self.stopSection, self.offlineSection, self.onlineSection, self.endDIV, self.otherImages]
for m in methods:
if m.__name__ != skip_name:
m()
def narrow(self):
try:
self.reply()
self.contentList('onlineSection') # removed onlineSection
for words in self.otherImages():
words.replace("narrow", "index")
self.otherImages doesn't return anything! When a function does not return an explicit value in python, it returns None. You cannot iterate over None.
Here are the changes I made which solves my problem. It allows me to edit the string when called from the 'narrow' method.
def otherImages(self):
return["<div id='menu_button'><a href='/narrow'><img src='../images/menu_button.png'></a></div>"]
def narrow(self):
try:
self.reply()
self.contentList('onlineSection') # removed onlineSectionv
for words in self.otherImages():
words = words.replace("/narrow", "/")
self.wfile.write(bytes(words, "utf8"))
return
i'm trying define a function that return a list when i specify an object, and it returns a list of all the objects in the scene with *_control when i don't specify anything..
that's my function but it doesn't work....
i'm working with maya then..
from maya import cmds
def correct_value(selection):
if not isinstance(selection, list):
selection = [selection]
objs = selection
return objs
if not selection :
objs = cmds.ls ('*_control')
return objs
when i don't specify anything it returns an error :
Error: line 1: TypeError: file line 1: correct_value()
takes exactly 1 argument (0 given)
what's wrong ??
def correct_value(selection=None):
if selection is None: # note that You should check this before
# You wil check whether it is list or not
objs = cmds.ls ('*_control')
return objs
if not isinstance(selection, list):
selection = [selection]
objs = selection
return objs
Well, you wrote your function with a required argument. Therefore, you have to pass the argument. You can write it so the argument is optional by specifying the value that will be used when nothing is passed:
def correct_value(selection=None):
etc.
If you want a parameter to be optional, you need to provide a default value:
def correct_value(selection=None):
# do something
if selection is None:
#do something else
To handle a default parameter even if it might be None
def correct_value(*args):
if not args:
objs = cmds.ls ('*_control')
return objs
elif len(args) == 1:
selection = args
objs = selection
return objs
else:
raise TypeError # ...
Here's a really useful pair of patterns for this kind of stuff:
# always return a list from scene queries (maya will often return 'none'
def get_items_named_foo():
return cmds.ls("foo") or [] # this makes sure that you've always got a list, even if it's empty
# always use the variable *args method to pass in lists
def do_something(*args):
for item in args:
do_something(item) # args will always be a tuple, so you can iterate over it
# this lets you do stuff like this without lots of boring argument checks:
do_something (*get_items_named_foo())
If you use both of these tricks consistently, you can transparently handle cases where your maya queries have returned None instead of a list
As an aside, you can mimic the default maya behaviour (where passing no arguments uses the current selection) like this:
def work_on_list_or_selected(*args):
args = args or cmds.ls(sl=True) or []
for item in args:
do_something (item)
I use Multiprocessing in Python in order to do several requests to a database (and other stuff):
po = multiprocessing.Pool()
for element in setOfElements:
results.append(po.apply_async(myDBRequestModule, (element, other stuff...)))
po.close()
po.join()
for r in results:
newSet.add(r.get())
myDBRequestModule returns an object I defined, made of a list and two numbers. I redefined the hash function, in order to define what I mean by equality in my sets of these objects:
class myObject:
def __init__(self, aList, aNumber, anotherNumber):
self.list = aList
self.number1 = aNumber
self.number2 = anotherNumber
def __hash__(self):
# turn elements of list into a string, in order to hash the string
hash_text = ""
for element in self.list:
hash_text += str(element.x.id) # I use the ID of the element of my list...
return hash(hash_text)
def __eq__(self, other):
self_hash_text = ""
other_hash_text = ""
for element in self.list:
self_hash_text += str(element.x.id)
for element in other.listDest:
other_hash_text += str(element.x.id)
return self_hash_text == other_hash_text
And in most cases it works as it should. Twice, for no known reason and in exactly the same context, I had a bug:
newSet.add(r.get())
File "/usr/lib/python2.6/multiprocessing/pool.py", line 422, in get
raise self._value
TypeError: 'str' object does not support item assignment
It comes from the get method (last line):
def get(self, timeout=None):
self.wait(timeout)
if not self._ready:
raise TimeoutError
if self._success:
return self._value
else:
raise self._value
Since I had this mistake only once and it disappeared, I decided to give up earlier, but it created a second problem recently, and I really don't know how to fight this bug.
In particular, it's difficult for me to tell why it happens almost never, and usually works perfectly fine.
multiprocessing is not the issue here.
You have not given us the right code to diagnose the issue. At some point you have assigned a caught exception to self._value. That is where the error is occurring. Look at everywhere that self._value is assigned and you will be on your way to finding this error.