I am using Python 2.7 in order to perform a simple task of launching a browser, verify the header, and close the browser
#Launch the browser # google
new = 0
url = "http://www.google.com/"
webbrowser.open(url, new=new)
#Check for the header
conn = httplib.HTTPConnection("www.google.com")
conn.request("HEAD", "/")
r1 = conn.getresponse()
#Close the browser
os.system("taskkill /im iexplore.exe")
This just runs on an infinite loop in order to verify continuous connectivity. Ping check isn't sufficient for the amount of traffic I need, or I would use that.
My problem is that if I do lose connectivity, the script freezes and I get addressinfo errors. How do I ignore this, or recognize it, kill the browser and keep the script running?
Sorry, if I'm not doing this right...it is my first post.
I don't think you actually need a browser here at all.
Meanwhile, the way you ignore or recognize errors is with a try statement. So:
while True:
try:
conn = httplib.HTTPConnection("www.google.com")
conn.request("HEAD", "/")
r1 = conn.getresponse()
if not my_verify_response_func(r1):
print('Headers are wrong!')
except Exception as e:
print('Failed to check headers with {}'.format(e))
time.sleep(60) # I doubt you want to run as fast as possible
Related
I am trying to automate saving each tab in my open browser (Google Chrome). This code used to work before when it operated from the command prompt, but now I am making a GUI to use the functions and I get this error.
here is the function that uses Application.connect().
# saves all current tabs to clipboard and closes browser
def save_and_close(self):
global URLS
# list that contains new URLS for storage
updated_data = []
# account for any internet-related disconnections
try:
app = Application(backend='uia').connect(path = self.browser_location)
find_windows(title = "Google Chrome")
while True:
try:
keyboard.send_keys("{F6}^c")
time.sleep(1)
updated_data.append(clipboard.GetData())
time.sleep(3)
keyboard.send_keys("^w")
time.sleep(3)
except KeyboardInterrupt:
keyboard.send_keys("^") # undo the keyboard control key
break
# updates database with new URLS
with open("bmanager.json", "w") as dw:
json.dump(updated_data, dw)
# reloads database for next possible usage
URLS = json.loads(open("bmanager.json").read())
except Exception as e:
print(e)
print ("[ERROR]: Client is not connected to the internet")
Here is the error:
Process [browser location] not found!
Note that [browser location] is just a placeholder for the location of my browser's .exe file
I figured out that I have to register chrome using the location and pywinauto's register function before connecting to it.
So im trying to execute this code:
liner = 0
for eachLine in content:
print(content[liner].rstrip())
raw=str(content[liner].rstrip())
print("Your site:"+raw)
Sitecheck=requests.get(raw)
time.sleep(5)
var=Sitecheck.text.find('oht945t945iutjkfgiutrhguih4w5t45u9ghdgdirfgh')
time.sleep(5)
print(raw)
liner += 1
I would expect this to run through the first print up to the liner variable and then go back up, however something else seems to happen:
https://google.com
Your site:https://google.com
https://google.com
https://youtube.com
Your site:https://youtube.com
https://youtube.com
https://x.com
Your site:https://x.com
This happens before the get requests. And the get requests later just get timed out:
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',))
I tried adding time.sleep(5) in my code for it to run smoother however this failed to yield results
Why don't you use Python's exception handling, to catch failed connections?
import requests
#list with websites
content = ["https://google.com", "https://stackoverflow.com/", "https://bbc.co.uk/", "https://this.site.doesnt.exi.st"]
#get list index and element
for liner, eachLine in enumerate(content):
#not sure, why this line exists, probably necessary for your content list
raw = str(eachLine.rstrip())
#try to get a connection and give feedback, if successful
try:
Sitecheck = requests.get(raw)
print("Tested site #{0}: site {1} responded".format(liner, raw))
except:
print("Tested site #{0}: site {1} seems to be down".format(liner, raw))
Mind you that there are more elaborate ways like scrapy or beautifulsoup in Python to retrieve web content. But I think that your question is more a conceptual than a practical one.
I have some raspberry pi running some python code. Once and a while my devices will fail to check in. The rest of the python code continues to run perfectly but the code here quits. I am not sure why? If the devices can't check in they should reboot but they don't. Other threads in the python file continue to run correctly.
class reportStatus(Thread):
def run(self):
checkInCount = 0
while 1:
try:
if checkInCount < 50:
payload = {'d':device,'k':cKey}
resp = requests.post(url+'c', json=payload)
if resp.status_code == 200:
checkInCount = 0
time.sleep(1800) #1800
else:
checkInCount += 1
time.sleep(300) # 2.5 min
else:
os.system("sudo reboot")
except:
try:
checkInCount += 1
time.sleep(300)
except:
pass
The devices can run for days and weeks and will check in perfectly every 30 minutes, then out of the blue they will stop. My linux computers are in read-only and the computer continue to work and run correctly. My issue is in this thread. I think they might fail to get a response and this line could be the issue
resp = requests.post(url+'c', json=payload)
I am not sure how to solve this, any help or suggestions would be greatly appreciated.
Thank you
A bare except:pass is a very bad idea.
A much better approach would be to, at the very minimum, log any exceptions:
import traceback
while True:
try:
time.sleep(60)
except:
with open("exceptions.log", "a") as log:
log.write("%s: Exception occurred:\n" % datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'))
traceback.print_exc(file=log)
Then, when you get an exception, you get a log:
2016-12-20 13:28:55: Exception occurred:
Traceback (most recent call last):
File "./sleepy.py", line 8, in <module>
time.sleep(60)
KeyboardInterrupt
It is also possible that your code is hanging on sudo reboot or requests.post. You could add additional logging to troubleshoot which issue you have, although given you've seen it do reboots, I suspect it's requests.post, in which case you need to add a timeout (from the linked answer):
import requests
import eventlet
eventlet.monkey_patch()
#...
resp = None
with eventlet.Timeout(10):
resp = requests.post(url+'c', json=payload)
if resp:
# your code
Your code basically ignores all exceptions. This is considered a bad thing in Python.
The only reason I can think of for the behavior that you're seeing is that after checkInCount reaches 50, the sudo reboot raises an exception which is then ignored by your program, keeping this thread stuck in the infinite loop.
If you want to see what really happens, add print or loggging.info statements to all the different branches of your code.
Alternatively, remove the blanket try-except clause or replace it by something specific, e.g. except requests.exceptions.RequestException
Because of the answers given I was able to come up with a solution. I realized requests has a built in time out function. The timeout will never happen if a timeout is not specified as a parameter.
here is my solution:
resp = requests.post(url+'c', json=payload, timeout=45)
You can tell Requests to stop waiting for a response after a given
number of seconds with the timeout parameter. Nearly all production
code should use this parameter in nearly all requests. Failure to do
so can cause your program to hang indefinitely
The answers provided by TemporalWolf and other helped me alot. Thank you to all that helped.
I'm using requests to get a URL, such as:
while True:
try:
rv = requests.get(url, timeout=1)
doSth(rv)
except socket.timeout as e:
print e
except Exception as e:
print e
After it runs for a while, it quits working. No exception or any error, just like it suspended. I then stop the process by typing Ctrl+C from the console. It shows that the process is waiting for data:
.............
httplib_response = conn.getresponse(buffering=True) #httplib.py
response.begin() #httplib.py
version, status, reason = self._read_status() #httplib.py
line = self.fp.readline(_MAXLINE + 1) #httplib.py
data = self._sock.recv(self._rbufsize) #socket.py
KeyboardInterrupt
Why is this happening? Is there a solution?
It appears that the server you're sending your request to is throttling you - that is, it's sending bytes with less than 1 second between each package (thus not triggering your timeout parameter), but slow enough for it to appear to be stuck.
The only fix for this I can think of is to reduce the timeout parameter, unless you can fix this throttling issue with the Server provider.
Do keep in mind that you'll need to consider latency when setting the timeout parameter, otherwise your connection will be dropped too quickly and might not work at all.
The default requests doesn't not set a timeout for connection or read.
If for some reason, the server cannot get back to the client within the time, the client will stuck at connecting or read, mostly the read for the response.
The quick resolution is to set a timeout value in the requests object, the approach is well described here: http://docs.python-requests.org/en/master/user/advanced/#timeouts
(Thanks to the guys.)
If this resolves the issue, please kindly mark this a resolution. Thanks.
I made a script to download wallpapers as a learning exercise to better familiarize myself with Python/Threading. Everything works well unless there is an exception trying to request a URL. This is the function I hit the exception (not a method of the same class, if that matters).
def open_url(url):
"""Opens URL and returns html"""
try:
response = urllib2.urlopen(url)
link = response.geturl()
html = response.read()
response.close()
return(html)
except urllib2.URLError, e:
if hasattr(e, 'reason'):
logging.debug('failed to reach a server.')
logging.debug('Reason: %s', e.reason)
logging.debug(url)
return None
elif hasattr(e, 'code'):
logging.debug('The server couldn\'t fulfill the request.')
logging.debug('Code: %s', e.reason)
logging.debug(url)
return None
else:
logging.debug('Shit fucked up2')
return None
At the end of my script:
main_thread = threading.currentThread()
for thread in threading.enumerate():
if thread is main_thread: continue
while thread.isAlive():
thread.join(2)
break
From my current understanding (which may be wrong) if the thread is not completed it's task within 2 seconds of reaching this it should time out. Instead it will stick in the last while. If I take that out it will just hang once the script is done executing.
Also, I decided it was time to man up and leave Notepad++ for a real IDE with debugging tools so I downloaded Wing. I'm a big fan of Wing, but the script doesn't hang there... What do you all use to write Python?
There is no thread interruption in Python and no way to cancel a thread. It can only finish execution by itself. The join method only waits 2 seconds or until termination, it does not kill anything. You need to implement timeout mechanism in the thread itself.
I hit the books and figured out enough to correct the issue I was having. I was able to remove that code that was near the end of my script completely. I corrected this issue by spawning the thread pool differently.
for i in range(queue.qsize()):
td = ThreadDownload(queue)
td.start()
queue.join()
I also was not using a try: for queue.get() during the thread's execution.
try:
img_url = self.queue.get()
...
except Queue.Empty:
...