Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
Fabulous people at SO,
I need to process a (bulky) set of data thru various steps; each step can involve mapReduce (using Disco), general distributed processing (using Celery) or some simple processing on the server. I am searching for a workflow engine/library/framework that can help manage such a workflow.
I have looked into numerous options and spiff workflow seems to be the most flexible but it doesn't seem to support actions on state transition and it's single threaded (so I am not sure how parallel gateways can be handled).
Please advise on the approach/tools to manage such workflow/jobs. If there is a framework that comes with a monitoring tool (preferably web based or can be integrated with Pyramid) then even better.
Thanks in advance
For pipelining batch data processing tasks, we use a solution based on great Spotify's Luigi framework. It's central scheduler scheduling and monitoring tool is a web server based on Tornado.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I'd like to have a utility running to periodically check our websites to make sure they're up and responding. Python is my preferred quick utility environment.
I know I can ping the server with urllib2 or something, but I really want to test that all the resources are there and available as well (CSS, JS, images, etc). Something like what a browser does when it loads a page -- fetch the HTML, then fetch the resources required, and check for any 400 or 500 errors.
Is there some simple way to do this in Python? I could probably use regex to try to grab the resource URLs from the HTML, but I don't want to worry about whether I'm doing it wrong.
Is there a tool or trick that will do the hard work, or will I have to parse the HTML myself? Or am I going about this the wrong way?
For availability monitoring I'd recommend a 3rd party service like newrelic.com or site24x7.com.
If you want to roll your own (which isn't so hard if you have only basic needs) just use an HTML parser and iterate over the DOM to request your linked resources. Just don't use regexes.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
This isn't a technical question, however after hours of scouring google I have not found an viable examples or resources to learn the Wokkel framework. I've extrapolated what I can from the source code itself, and the unit tests but still do not have a great understanding of how to implement subprotocols.
I'm working with a jabber server using almost all custom stanzas. If anyone has any resources or examples they could lend, that would be awesome.
Thanks
I'm not sure what kind of "resources" you're looking for, but there are lots of examples out there:
http://wokkel.ik.nu/wiki/XMPPClients
An XMPP Echo Bot with Twisted and Wokkel and Echo Bot Part 2: Making a Component
Twisted Wokkel Bot
Twisted / Wokkel XMPP Client Example
XMPP-Ping Examples
There are also numerous resources linked in How do you create a simple Google Talk Client using the Twisted Words Python library?
If you need more help or want feedback from specialists, the relevant mailing list is Twisted-Jabber.
In addition, Wokkel is closely related to Twisted Words, so check out materials related to that project, especially http://twistedmatrix.com/documents/current/words/.
I've started adding Sphinx-based prose documentation, along with the existing example scripts and put this up on http://wokkel.ik.nu/documentation. It also includes an API Reference generated using pydoctor.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I'm looking hard but I cannot find any XMPP server in python with the following features:
using epoll, just like http://www.gevent.org/
supporting BOSH
modular design
use little RAM/CPU for up to 1000 users
more important than the previous requirement: the CPU/RAM usage must be predictable
Prosody looks quite good feature-wise, but I don't know how many users it can support simultaneously and how it is performance-wise.
Could someone give me an idea?
For a rough idea of how Prosody is performance-wise, see this post on their ML. https://groups.google.com/d/topic/prosody-users/SlXpfwJfgY4/discussion
xmpp.org uses Prosody, any other questions? :P
btw, if you want to toy with it a little, you can always run prosody using luajit (didn't test that myself, but I'm fairly sure it would work). Expect at least 2-4x faster execution.
Look # ejabberd too.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I'm currently supporting a Python web app with increasingly complicated user/role/permission management requirements. Currently, we are rolling our own user, groups, permissions, etc. code and supporting database.
I'd like to find something like ASP.NET membership that can help manage user authentication and authorization, rather than risk security issues in continuing to create an increasingly complicated custom solution. Are there any similar projects out there worth taking a look at?
If you are looking for off site user authentication you might want to consider openid. People have added openid support to cherrypy.
If you are looking for more user management type code. I guess it depends on exactally what you are doing but others have done user management before, why not leverage off them. Skeletonz is a CMS written on top of cherrypy. If you are not wed to cherrypy you might also want to consider Pinax. It's built on Django with the idea of reusing work others have done so you don't have to do it again.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I'm writting a small python script notify me when certain condition met. I used smtplib which does the emailing for me, but I also want the script to call my cell phone as well.
I can't find a free library for phone callings. Does anyone know any?
Make the calls using Skype, and use the Skype4Py API.
If you want other suggestions, please specify how you want to make the call (modem? Some software bridge? What?).
Also, might I suggest that you send an SMS instead of placing a call? You can do that via Skype too, btw.
Twilio can make calls through their API. Pay as you go. Worked well for wakeup calls for me.
I've used Skype4Py very successfully. Keep in mind though it does require Skype to be installed and costs the standard rate for SkypeOut.