So I'm working on a chemistry project for fun, and I have a function that initializes a list from a text file. What I want to do s make it so the function replaces itself with a list. So here's my first attempt at it which randomly will or won't work and I don't know why:
def periodicTable():
global periodicTable
tableAtoms = open('/Users/username/Dropbox/Python/Chem Project/atoms.csv','r')
listAtoms = tableAtoms.readlines()
tableAtoms.close()
del listAtoms[0]
atoms = []
for atom in listAtoms:
atom = atom.split(',')
atoms.append(Atom(*atom))
periodicTable = atoms
It gets called in in this way:
def findAtomBySymbol(symbol):
try:
periodicTable()
except:
pass
for atom in periodicTable:
if atom.symbol == symbol:
return atom
return None
Is there a way to make this work?
Don't do that. The correct thing to do would be using a decorator that ensures the function is only executed once and caches the return value:
def cachedfunction(f):
cache = []
def deco(*args, **kwargs):
if cache:
return cache[0]
result = f(*args, **kwargs)
cache.append(result)
return result
return deco
#cachedfunction
def periodicTable():
#etc
That said, there's nothing stopping you from replacing the function itself after it has been called, so your approach should generally work. I think the reason it doesn't is because an exception is thrown before you assign the result to periodicTable and thus it never gets replaced. Try removing the try/except block or replacing the blanket except with except TypeError to see what exactly happens.
This is very bad practice.
What would be better is to have your function remember if it has already loaded the table:
def periodicTable(_table=[]):
if _table:
return _table
tableAtoms = open('/Users/username/Dropbox/Python/Chem Project/atoms.csv','r')
listAtoms = tableAtoms.readlines()
tableAtoms.close()
del listAtoms[0]
atoms = []
for atom in listAtoms:
atom = atom.split(',')
atoms.append(Atom(*atom))
_table[:] = atoms
The first two lines check to see if the table has already been loaded, and if it has it simply returns it.
Related
in fucntion getLink(urls), I have return (cloud,parent,children)
in main function, I have (cloud,parent,children) = getLink(urls) and I got error of this line: TypeError: 'NoneType' object is not iterable
parent and children are all list of http links. since, it is not able to paste them here, parent is a list contains about 30 links; children is a list contains about 30 items, each item is about 10-100 links which is divide by ",".
cloud is a list contain about 100 words, like that: ['official store', 'Java Applets Centre', 'About Google', 'Web History'.....]
I didnot know why I get an error. Is there anything wrong in passing parameter? Or because the list take too much space?
#crawler url: read webpage and return a list of url and a list of its name
def crawler(url):
try:
m = urllib.request.urlopen(url)
msg = m.read()
....
return (list(set(list(links))),list(set(list(titles))) )
except Exception:
print("url wrong!")
#this is the function has gone wrong: it throw an exception here, also the error I mentioned, also it will end while before len(parent) reach 100.
def getLink(urls):
try:
newUrl=[]
parent = []
children =[]
cloud =[]
i=0
while len(parent)<=100:
url = urls[i]
if url in parent:
i += 1
continue
(links, titles) = crawler(url)
parent.append(url)
children.append(",".join(links))
cloud = cloud + titles
newUrl= newUrl+links
print ("links: ",links)
i += 1
if i == len(urls):
urls = list(set(newUrl))
newUrl = []
i = 0
return (cloud,parent,children)
except Exception:
print("can not get links")
def readfile(file):
#not related, this function will return a list of url
def main():
file='sampleinput.txt'
urls=readfile(file)
(cloud,parent,children) = getLink(urls)
if __name__=='__main__':
main()
There might be a way that your function ends without reaching the explicit return statement.
Look at the following example code.
def get_values(x):
if x:
return 'foo', 'bar'
x, y = get_values(1)
x, y = get_values(0)
When the function is called with 0 as parameter the return is skipped and the function will return None.
You could add an explicit return as the last line of your function. In the example given in this answer it would look like this.
def get_values(x):
if x:
return 'foo', 'bar'
return None, None
Update after seing the code
When the exception is triggered in get_link you just print something and return from the function. You have no return statement, so Python will return None. The calling function now tries to expand None into three values and that fails.
Change your exception handling to return a tuple with three values like you do it when everything is fine. Using None for each value is a good idea for it shows you, that something went wrong. Additionally I wouldn't print anything in the function. Don't mix business logic and input/output.
except Exception:
return None, None, None
Then in your main function use the following:
cloud, parent, children = getLink(urls)
if cloud is None:
print("can not get links")
else:
# do some more work
The race-condition-free way of updating a variable in redis is:
r = redis.Redis()
with r.pipeline() as p:
while 1:
try:
p.watch(KEY)
val = p.get(KEY)
newval = int(val) + 42
p.multi()
p.set(KEY, newval)
p.execute() # raises WatchError if anyone else changed KEY
break
except redis.WatchError:
continue # retry
this is significantly more complex than the straight forward version (which contains a race-condition):
r = redis.Redis()
val = r.get(KEY)
newval = int(val) + 42
r.set(KEY, newval)
so I thought a context manager would make this easier to work with, however, I'm having problems...
My initial idea was
with update(KEY) as val:
newval = val + 42
somehow return newval to the contextmanager...?
there wasn't an obvious way to do the last line, so I tried::
#contextmanager
def update(key, cn=None):
"""Usage::
with update(KEY) as (p, val):
newval = int(val) + 42
p.set(KEY, newval)
"""
r = cn or redis.Redis()
with r.pipeline() as p:
while 1:
try:
p.watch(key) # --> immediate mode
val = p.get(key)
p.multi() # --> back to buffered mode
yield (p, val)
p.execute() # raises WatchError if anyone has changed `key`
break # success, break out of while loop
except redis.WatchError:
pass # someone else got there before us, retry.
which works great as long as I don't catch a WatchError, then I get
File "c:\python27\Lib\contextlib.py", line 28, in __exit__
raise RuntimeError("generator didn't stop")
RuntimeError: generator didn't stop
what am I doing wrong?
I think the problem is that you yield multiple times (when the task is repeated) but a context manager is only entered once (the yield is just a syntactic sugar for the __enter__ method). So as soon as the yield can be executed multiple times, you have a problem.
I’m not prefectly sure how to solve this in a good way, and I can’t test it either, so I’m only giving some suggestions.
First of all, I would avoid yielding the rather internal p; you should yield some object that is specifically made for the update process. For example something like this:
with update(KEY) as updater:
updater.value = int(updater.original) + 42
Of course this still doesn’t solve the multiple yields, and you cannot yield that object earlier as you won’t have the original value at that point either. So instead, we could specify a delegate responsible for the value updating instead.
with update(KEY) as updater:
updater.process = lambda value: value + 42
This would store a function inside the yielded object which you can then use inside the context manager to keep trying to update the value until it succeeded. And you can yield that updater from the context manager early, before entering the while loop.
Of course, if you have made it this far, there isn’t actually any need for a context manager left. Instead, you can just make a function:
update(key, lambda value: value + 42)
Consider the following:
def funcA():
some process = dynamicVar
if dynamicVar == 1:
return dynamicVar
else:
print "no dynamicVar"
def main():
outcome = funcA()
If the 'some process' part results in a 1, the var dynamicVar is passed back as outcome to the main func. If dynamicVar is anything but 1, the routine fails as no arguments are being return.
I could wrap the outcome as a list:
def funcA():
outcomeList = []
some process = dynamicVar
if dynamicVar == 1:
outcomeList.append(dynamicVar)
return outcomeList
else:
print "no dynamicVar"
return outcomeList
def main():
outcome = funcA()
if outcome != []:
do something using dynamicVar
else:
do something else!
or maybe as a dictionary item. Each of the 2 solutions I can think of involve another set of processing in the main / requesting func.
Is this the 'correct' way to handle this eventuality? or is there a better way?
What is the proper way of dealing with this. I was particularly thinking about trying to catch try: / except: errors, so in that example the uses are reversed, so something along the lines of:
def funcA():
some process = dynamicVar
if dynamicVar == 1:
return
else:
outcome = "no dynamicVar"
return outcome
def main():
try:
funcA()
except:
outcome = funcA.dynamicVar
In Python, all function that do not return a value will implicitly return None. So you can just check if outcome is not None in main().
I believe when you write a function, it's return value should be clear and expected. You should return what you say you will return. That being said, you can use None as a meaningful return value to indicate that the operation failed or produced no results:
def doSomething():
"""
doSomething will return a string value
If there is no value available, None will be returned
"""
if check_something():
return "a string"
# this is being explicit. If you did not do this,
# None would still be returned. But it is nice
# to be verbose so it reads properly with intent.
return None
Or you can make sure to always return a default of the same type:
def doSomething():
"""
doSomething will return a string value
If there is no value available, and empty string
will be returned
"""
if check_something():
return "a string"
return ""
This handles the case with a bunch of complex conditional tests that eventually just fall through:
def doSomething():
if foo:
if bar:
if biz:
return "value"
return ""
I use Multiprocessing in Python in order to do several requests to a database (and other stuff):
po = multiprocessing.Pool()
for element in setOfElements:
results.append(po.apply_async(myDBRequestModule, (element, other stuff...)))
po.close()
po.join()
for r in results:
newSet.add(r.get())
myDBRequestModule returns an object I defined, made of a list and two numbers. I redefined the hash function, in order to define what I mean by equality in my sets of these objects:
class myObject:
def __init__(self, aList, aNumber, anotherNumber):
self.list = aList
self.number1 = aNumber
self.number2 = anotherNumber
def __hash__(self):
# turn elements of list into a string, in order to hash the string
hash_text = ""
for element in self.list:
hash_text += str(element.x.id) # I use the ID of the element of my list...
return hash(hash_text)
def __eq__(self, other):
self_hash_text = ""
other_hash_text = ""
for element in self.list:
self_hash_text += str(element.x.id)
for element in other.listDest:
other_hash_text += str(element.x.id)
return self_hash_text == other_hash_text
And in most cases it works as it should. Twice, for no known reason and in exactly the same context, I had a bug:
newSet.add(r.get())
File "/usr/lib/python2.6/multiprocessing/pool.py", line 422, in get
raise self._value
TypeError: 'str' object does not support item assignment
It comes from the get method (last line):
def get(self, timeout=None):
self.wait(timeout)
if not self._ready:
raise TimeoutError
if self._success:
return self._value
else:
raise self._value
Since I had this mistake only once and it disappeared, I decided to give up earlier, but it created a second problem recently, and I really don't know how to fight this bug.
In particular, it's difficult for me to tell why it happens almost never, and usually works perfectly fine.
multiprocessing is not the issue here.
You have not given us the right code to diagnose the issue. At some point you have assigned a caught exception to self._value. That is where the error is occurring. Look at everywhere that self._value is assigned and you will be on your way to finding this error.
I have something like this using BeautifulSoup:
for line in lines:
code = l.find('span', {'class':'boldHeader'}).text
coded = l.find('div', {'class':'Description'}).text
definition = l.find('ul', {'class':'definitions'}).text
print code, coded, def
However, not all elements exist at all times. I can enclose this in a try except so that it does not break the program execution like this:
for line in lines:
try:
code = l.find('span', {'class':'boldHeader'}).text
coded = l.find('div', {'class':'Description'}).text
definition = l.find('ul', {'class':'definitions'}).text
print code, coded, def
except:
pass
But how I execute the statements in a greedy fashion? For instance, if there are only two elements available code and coded, I just want to get those and continue with the execution. As of now, even if code and coded exist, if def does not exist, the print command is never executed.
One way of doing this is to put a try...except for every statement like this:
for line in lines:
try:
code = l.find('span', {'class':'boldHeader'}).text
except:
pass
try:
coded = l.find('div', {'class':'Description'}).text
except:
pass
try:
definition = l.find('ul', {'class':'definitions'}).text
except:
pass
print code, coded, def
But this is an ugly approach and I want something cleaner. Any suggestions?
How about capture the "ugly" code in a function, and just call the function as needed:
def get_txt(l,tag,classname):
try:
txt=l.find(tag, {'class':classname}).text
except AttributeError:
txt=None
return txt
for line in lines:
code = get_txt(l,'span','boldHeader')
coded = get_txt(l,'div','Description')
defn = get_txt(l,'ul','definitions')
print code, coded, defn
PS. I changed def to defn because def is a Python keyword. Using it as a variable name raises a SyntaxError.
PPS. It's not a good practice to use bare exceptions:
try:
....
except:
...
because it almost always captures more that you intend. Much better to be explicit about what you want to catch:
try:
...
except AttributeError as err:
...
First of all, you can test for None instead of catching an exception. l.find should return None if it doesn't find your item. Exceptions should be reserved for errors and really extraordinary situations.
Second thing you can do is to create an array of all HTML elements you want to check and then have a nested for loop. Since it's been a while since I've used python, I will outline the code and then (hopefully) edit the answer when I test it.
Something like:
elementsToCheck = [
[ 'span', {'class':'boldHeader'} ],
[ 'div', {'class':'Description'} ],
[ 'ul', {'class':'definitions'} ]]
concatenated = ''
for line in lines:
for something in elementsToCheck
element = l.find(something[0], something[1])
if element is not None
concatenated += element.text
print concatenated
Obviously the code above won't work, but you should get the idea. :)