Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I'd like to have a utility running to periodically check our websites to make sure they're up and responding. Python is my preferred quick utility environment.
I know I can ping the server with urllib2 or something, but I really want to test that all the resources are there and available as well (CSS, JS, images, etc). Something like what a browser does when it loads a page -- fetch the HTML, then fetch the resources required, and check for any 400 or 500 errors.
Is there some simple way to do this in Python? I could probably use regex to try to grab the resource URLs from the HTML, but I don't want to worry about whether I'm doing it wrong.
Is there a tool or trick that will do the hard work, or will I have to parse the HTML myself? Or am I going about this the wrong way?
For availability monitoring I'd recommend a 3rd party service like newrelic.com or site24x7.com.
If you want to roll your own (which isn't so hard if you have only basic needs) just use an HTML parser and iterate over the DOM to request your linked resources. Just don't use regexes.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I've tried searching the Internet for something that might do what I want, but I don't think I'm using the right search terms. I'm sure something like this must exist.
My use case is this: I have a debit card, which spends money from my account directly. I usually try to keep this account manually at a certain balance, which means I have to log in to the bank website every few days and transfer money from my savings account into the debit account. Since it's always the same procedure - log in, read balance, transfer money to take balance back to threshold - I thought there surely must be some way to automate this. Every time I've tried to do something in Python, I've found a library for it :P so I figured it must be possible.
So, my question: Does there exist any way to use Python (or any language I suppose) to virtually view a web page as though it was in Firefox, and to then interact with that web page? I'd need to be able to do some sort of string searching I suppose, in order to find the required fields and buttons. It will be for a bank website, so I can't just load a HTML file statically and scan through it, it has to pretend it is an actual web browser (at least I imagine it would).
Look for module Selenium - doc: http://selenium-python.readthedocs.org/
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I am building a python script to run nightly, part of which involves invoking a drupal bulk operation to export an XML file. Since the process takes a number of hours, and the next step is to automatically import it to another source, I would like to perform some level of integrity checking.
My first thought would be to simply make sure that the XML is not malformed. I don't particularly want to start inspecting the data, I just want to make sure it's not truncated.
What process might I use to perform this malformed check. Is there an applicable XML library built into Python? I would prefer to keep the script as portable as possible, so if possible a built-in solution would be the most preferable.
Thanks for any advice.
If you want use python, you can consider using element tree
Load your .xml and try to parse. Any Exception means XML is malformed.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I'm using Python 3.6 while I have to fill in a form. Unfortunately, mechanize doesn't work on Python 3.
What do you suggest as an alternative to mechanize?
SeleniumRC with selenium.py is an alternative (and one of the few workable options if the pages you need to scrape have an important, "structural" role for Javascript operations, esp. AJAX-y ones, since Mechanize doesn't execute the Javascript on the pages it's scraping).
For scraping and form handling you can use lxml.html (it doesn't automate fetching and cookies though).
scrapy is a library specifically for scraping.
I've been successful with Splinter, a solution built on top of Selenium - while providing more pythonic API.
I've used twill for a lot of my testing needs. It works as a stand-alone language for "web browsing" or as a library from Python. It actually uses Mechanize under the hood, so I'm not sure if it will meet your needs -- are you encountering problems intrinsic to Mechanize, or would you benefit from a high level layer?
try zope.browser
http://pypi.python.org/pypi?:action=display&name=zope.testbrowser
scrapelib is another option : https://github.com/sunlightlabs/scrapelib
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I'm currently supporting a Python web app with increasingly complicated user/role/permission management requirements. Currently, we are rolling our own user, groups, permissions, etc. code and supporting database.
I'd like to find something like ASP.NET membership that can help manage user authentication and authorization, rather than risk security issues in continuing to create an increasingly complicated custom solution. Are there any similar projects out there worth taking a look at?
If you are looking for off site user authentication you might want to consider openid. People have added openid support to cherrypy.
If you are looking for more user management type code. I guess it depends on exactally what you are doing but others have done user management before, why not leverage off them. Skeletonz is a CMS written on top of cherrypy. If you are not wed to cherrypy you might also want to consider Pinax. It's built on Django with the idea of reusing work others have done so you don't have to do it again.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I'm writting a small python script notify me when certain condition met. I used smtplib which does the emailing for me, but I also want the script to call my cell phone as well.
I can't find a free library for phone callings. Does anyone know any?
Make the calls using Skype, and use the Skype4Py API.
If you want other suggestions, please specify how you want to make the call (modem? Some software bridge? What?).
Also, might I suggest that you send an SMS instead of placing a call? You can do that via Skype too, btw.
Twilio can make calls through their API. Pay as you go. Worked well for wakeup calls for me.
I've used Skype4Py very successfully. Keep in mind though it does require Skype to be installed and costs the standard rate for SkypeOut.