We Keep Coding

Why does a result() from concurrent.futures.as_completed() occasionally return None?

I've been writing a fairly complex series of modules to help test our poorly documented networking gear, this one focused on trying the various passwords used across the company. I've been using the concurrent.futures module to speed things along by testing many devices in parallel but am running in to a problem where occasionally a result comes back as None and it's a mystery to me as why. It revolves around this chunk of code:
def process_device(dev):
check = CheckPass(creds, inv_list)
dev = switch.Switch(**dev)
online, net_device = check.device_online(dev)
if online == True:
conn, net_device = check.try_passwords(dev, return_conn=True)
if conn != False and conn != "none":
if conn != True:
hostname,model,serial_num,version,date = net_device.get_id_details(conn)
net_device.hostname = hostname
net_device.model = model
net_device.serial_num = serial_num
net_device.version = version
net_device.date = date
conn.disconnect()
conf_dev = check.confirm_device(net_device)
check.dev_confirmed.append(conf_dev)
return dev.hostname, dev.ip
with concurrent.futures.ThreadPoolExecutor(max_workers=120) as executor:
threads = {executor.submit(process_device, dev): dev for dev in inv_list}
for future in concurrent.futures.as_completed(threads):
name, ip = future.result()
At unpredictable intervals future.result() will have a result of None and I can't reliably reproduce it. The error occurs with different devices every time, I've checked the logs and fed it an inventory list containing just the device that it had processed and running that device by itself will be fine. I have tried using fewer workers and more. Clearly I'm not understanding something about how future.result() works. dev.hostname and dev.ip always have values so process_device() should always return them (barring unhandled exceptions, which haven't occurred) yet I always end up with TypeError: Cannot unpack non-iterable NoneType object referencing the name, ip = future.results() command.

It is not problem with future but with your function process_device() which sometimes can return None.
When online is False or when if conn != False and conn != "none": gives False then process_device() will run default return None
def process_device(dev):
# ... your code ...
return None # default behavior
You should filter results
result = future.result()
if result:
name, ip = result

Try/Exception giving multiple results in Python

I have this program that should take the user input (an application name installed) and then it will open the application with subprocess.Popen() . All posibles application names are in a dictionary as keys, and the value is the path to the application .exe file. I'm using a try/Exception to see if there will be a KeyError (if the input name doens't exist in the dictionary).
So the programm should take the user input, see if the application name is in the dictionary, and then open the application. If the name isn't on the dictionary, it will give an error message and then ask for a name again.
But when I enter a non-existent name n times, it will run the finally block n times too. How to solve this?
import subprocess
apps ={
"vlc":"app_path\\vlc.exe",
"sublime":"app_path\\sublime_text.exe",
"chrome":"app_path\\chrome.exe",
"opera":"app_path\\launcher.exe",
"torrent":"app_path.\\qbittorrent.exe"
}
def test():
answer = str(input("What program do you want to run: ")).lower()
try:
print(apps[answer])
except KeyError:
print(f"{answer.capitalize()} application unknown. Try again with a valid application name")
print("===============================================================")
test()
except:
print("==============================")
print("Unknown error.")
else:
subprocess.Popen(apps[answer])
finally:
print("===========================================")
print("FINISHED")
test()

You're using recursion here, so obviously the finally block will run as many times as you enter call the function.
The code within finally is always executed regardless if the try block raises an error or not.
W3Schools has good examples for this: https://www.w3schools.com/python/python_try_except.asp
Instead of using try/except, you can simply use a while loop and apps.get(answer, 'ERROR') or if answer in apps to check if the entered input is in the apps dictionary. The following solution may be optimal.
def test():
while True:
answer = input("What program do you want to run: ").lower()
if answer in apps:
try:
res = subprocess.Popen(apps[answer]).communicate()
print("===========================================")
print("FINISHED")
return res
except:
print("==============================")
print("Unknown error.")
else:
print(f"{answer.capitalize()} application unknown. Try again with a valid application name")
print("===============================================================")
test()

Loop to check if a variable has changed in Python

I have just learned the basics of Python, and I am trying to make a few projects so that I can increase my knowledge of the programming language.
Since I am rather paranoid, I created a script that uses PycURL to fetch my current IP address every x seconds, for VPN security. Here is my code[EDITED]:
import requests
enterIP = str(input("What is your current IP address?"))
def getIP():
while True:
try:
result = requests.get("http://ipinfo.io/ip")
print(result.text)
except KeyboardInterrupt:
print("\nProccess terminated by user")
return result.text
def checkIP():
while True:
if enterIP == result.text:
pass
else:
print("IP has changed!")
getIP()
checkIP()
Now I would like to expand the idea, so that the script asks the user to enter their current IP, saves that octet as a string, then uses a loop to keep running it against the PycURL function to make sure that their IP hasn't changed? The only problem is that I am completely stumped, I cannot come up with a function that would take the output of PycURL and compare it to a string. How could I achieve that?

As #holdenweb explained, you do not need pycurl for such a simple task, but nevertheless, here is a working example:
import pycurl
import time
from StringIO import StringIO
def get_ip():
buffer = StringIO()
c = pycurl.Curl()
c.setopt(pycurl.URL, "http://ipinfo.io/ip")
c.setopt(c.WRITEDATA, buffer)
c.perform()
c.close()
return buffer.getvalue()
def main():
initial = get_ip()
print 'Initial IP: %s' % initial
try:
while True:
current = get_ip()
if current != initial:
print 'IP has changed to: %s' % current
time.sleep(300)
except KeyboardInterrupt:
print("\nProccess terminated by user")
if __name__ == '__main__':
main()
As you can see I moved the logic of getting the IP to separate function: get_ip and added few missing things, like catching the buffer to a string and returning it. Otherwise it is pretty much the same as the first example in pycurl quickstart
The main function is called below, when the script is accessed directly (not by import).
First off it calls the get_ip to get initial IP and then runs the while loop which checks if the IP has changed and lets you know if so.
EDIT:
Since you changed your question, here is your new code in a working example:
import requests
def getIP():
result = requests.get("http://ipinfo.io/ip")
return result.text
def checkIP():
initial = getIP()
print("Initial IP: {}".format(initial))
while True:
current = getIP()
if initial == current:
pass
else:
print("IP has changed!")
checkIP()
As I mentioned in the comments above, you do not need two loops. One is enough. You don't even need two functions, but better do. One for getting the data and one for the loop. In the later, first get initial value and then run the loop, inside which you check if value has changed or not.

It seems, from reading the pycurl documentation, like you would find it easier to solve this problem using the requests library. Curl is more to do with file transfer, so the library expects you to provide a file-like object into which it writes the contents. This would greatly complicate your logic.
requests allows you to access the text of the server's response directly:
>>> import requests
>>> result = requests.get("http://ipinfo.io/ip")
>>> result.text
'151.231.192.8\n'
As #PeterWood suggested, a function would be more appropriate than a class for this - or if the script is going to run continuously, just a simple loop as the body of the program.

Enthought canopy validation error

I need help!!
I've been struggling with this problem and I cannot seem to solve it.
def itemName():
flag = True
while flag == True:
try:
name = input('What would You like to name it? \n')
Input = str(Input)
print(name)
if name.upper() == ('BOAT') or name.upper() == ('CASUALTY'):
flag = False
else:
raise Exception
except:
print('Boat or Casualty only')
return name
name = itemName()
print(name)
This code will not pass when run. This works in the normal python IDLE but not canopy. Also the print statement only outputs if the input is an integer. I'm stumped, any ideas?

This is python 3 code. Canopy uses Python 2.7
The input function differs between these versions.
In python 2.7, the equivalent is raw_input

PySVN - Determine if a repository exists

I'm writing a small script that manages several SVN repositories. Users pass through the ID of the repository they want to change (the root of the repos are of the form https://www.mydomain.com/).
I need to check if the given repo actually exists. I've tried using Client.list to see if I can find any files, like so:
client = pysvn.Client()
client.list("https://.../<username>/")
But if the repo does not exist then the script hangs on the list line. From digging through the tracebacks it looks like pysvn is actually hanging on the login credentials callback (client.callback_get_login - which I have implemented but omitted, it does not fail if the repo exists).
Can you suggest how I can determine if a repo exists or not using pysvn?
Cheers,
Pete

I couldn't reproduce your hanging in credentials callback problem, so it might need an expanded description of the problem. I'm running pysvn 1.7.2 on Ubuntu 10.04, Python 2.6.6.
When I try to list a non-existent remote repository with client.list() it raises an exception. You could also use client.info2() to check for existence of a remote repository:
head_rev = pysvn.Revision(pysvn.opt_revision_kind.head)
bad_repo = 'https://.../xyz_i_dont_exist'
good_repo = 'https://.../real_project'
for url in (bad_repo, good_repo):
try:
info = client.info2(url, revision=head_rev, recurse=False)
print url, 'exists.'
except pysvn._pysvn_2_6.ClientError, ex:
if 'non-existent' in ex.args[0]:
print url, 'does not exist'
else:
print url, 'error:', ex.args[0]

Peter,
My team and I have experienced the same challenge. Samplebias, try providing a callback_get_login function but set your callback_server_ssl_trust_prompt to return (True, trust_dict['failures'], True). IFF subversion has not cached your server certificate trust settings, then you may find the info2() (or Peter's list() command) hangs (it's not actually hanging, it just takes intermittently much longer time to return). Oddly, when you CTRL-C the interpreter in these scenarios, you'll get indication that it hung on the login callback, not the server_cert verification. Play around with your ~/.subversion/auth settings (in particular the svn.simple and svn.ssl.server directories) and you'll see different amounts of 'hang time'. Look at pysvn.Client.callback_cancel if you need to handle situations which truly never return.
Considering: http://pysvn.tigris.org/docs/pysvn_prog_ref.html#pysvn_client_callback_ssl_server_trust_prompt you need to decide what your desired behavior is. Do you want ONLY to allow those connections for which you already have a cached trust answer? Or, do you want to ALWAYS accept regardless of server certificate verification (WARNING: this could (obviously) have negative security implications). Consider the following suggestion:
import pysvn
URL1 = "https://exists.your.org/svn/repos/dev/trunk/current"
URL2 = "https://doesntexit.your.org/svn/repos/dev/trunk/current"
URL3 = "https://exists.your.org/svn/repos/dev/trunk/youDontHavePermissionsBranch"
ALWAYS = "ALWAYS"
NEVER = "NEVER"
DESIRED_BEHAVIOR = ALWAYS
def ssl_server_certificate_trust_prompt(trust_dict):
if DESIRED_BEHAVIOR == NEVER:
return (False, 0, False)
elif DESIRED_BEHAVIOR == ALWAYS:
return (True, trust_dict['failures'], True)
raise Exception, "Unsupported behavior"
def testURL(url):
try:
c.info2(url)
return True
except pysvn.ClientError, ce:
if ('non-existant' in ce.args[0]) or ('Host not found' in ce.args[0]):
return False
else:
raise ce
c = pysvn.Client()
c.callback_ssl_server_trust_prompt = lambda t: (False, t['failures'], True)
c.callback_get_login = lambda x, y, z: (True, "uname", "pw", False)
if not testURL(URL1): print "Test1 failed."
if testURL(URL2): print "Test2 failed."
try:
testURL(URL3)
print "Test3 failed."
except: pass
In actuality, you probably don't want to get as fancy as I have with the return values. I do think it was important to consider a potential 403 returned by the server and the "Host not found" scenario separately.