We Keep Coding

Automate webpage tasks without having to have a browser open?

I know about tampermonkey/greasemonkey and have used it a fair bit, but now my task is to write a program that runs in the background and automates mundane tasks (clicking buttons, typing into input fields etc.) on a specific webpage. Running a browser in the background takes too much RAM and processing power, so I'm looking for an alternative.
So far I've found selenium, but after a bit of research it looks like that it requires to have a browser open at all times as well (or maybe not? the documentation isn't that good). I thought about python scripts too, but I don't have any experience with those nor have I any idea if they can handle anything that's not basic html. If they can, does anyone know of a good tutorial for python scripts? I have used that language a few years ago, so I shouldn't really have a problem with python itself.
If python scripts aren't ideal either, is there a (preferably somewhat simple) way I could achieve what I want?

It depends on whether you want a script that interacts with a web UI, or a script that automates web requests. Do you really need to click buttons and type into input fields? Presumably, the data from those buttons and input fields is eventually sent to a web server. You could skip the entire UI and just make the requests directly. You don't need a browser for that and python is fine for doing these types of things (you don't even need selenium, you can just use requests)
On the other hand, if you're trying to test out the UI of a web page, or you actually need to interact with the web UI for some other reason, then yeah, you'll need an application (like a web browser) that's capable of rendering the UI so you can interact with it.

Writing for both PySide and web2py

Background: I have a desktop app that I've written in Python27x that uses wxPython for the UI and requests with xml.etree.ElementTree to retrieve open data from a RESTful service and present the data in meaningful ways to the user.
I am currently re-writing my app using PySide. I am fairly certain that I can present my data-candy in HTML5 and I would eventually like to provide a web-app using web2py and JavaScript.
The PySide and web2py versions both need to support:
aquiring a lock on a Throttle object that I've made to handle the fair-usage policy of the service.
presenting the analyzed data in tables with links or buttons for retrieving or donwloading more related data.
presenting lists of related data and highlighting groups of items on mouse-over of any member of the group.
presenting text documents and providing automatic searching / highlighting of keywords / part-of-speech tagging using NLTK.
providing basic graphs and charts of various stats on the data.
Question: Given what I've told you about my app, and assuming web2py and HTML5 can meet my user-interface / presentation needs, what steps can I take / widgets should I use in making my PySide desktop app to maximize re-use when porting over to web2py? What should I be sure to avoid when writing the desktop version?

You need to separate your data and data-processing from the user interface. Qt (and thus PySide) has a very strong focus on this Model-View approach (see e.g. http://qt-project.org/doc/qt-4.8/modelview.html), and provides models to organize your data, and views to present the data. Within Qt, this approach allows to easily use multiple views on the same data sets, without having to worry about how to get the data in the view.
Admittedly, the Qt models take some time to get used to, but the aforementioned tutorial should give you some pointers and references to get you started. In your case, I would go for the following approach:
Find/extend a suitable Qt model to manage your data
Use this data with standard/custom views in your PySide application
Develop web2py-based view to present the data in your webapp
I'm not familiar with web2py, so I can't assess how hard/easy this last step would be. However, I can recommend to invest some time (if you have it) in getting to know the Qt Model-View framework, as it can save you huge amounts of time in the future (at least, in my experience).

In my opinion, you may be over-thinking this.
Basically you will have two methods of presenting data to the end-user;
1) Via a GUI
2) Via your HTML5-candy
Surely the limiting factor will be whatever limitations (if any), web2py/HTML5 place on presenting the data?
If I were writing such an app, which used both PySide and web2py to present the same data but using their respective methods (gui, web), I'd probably want to abstract the data to be presented in such a way that you can feed the same stream into either the GUI function(s) or the Web function(s), with each taking that in and using their respective methods to present the data.
Another alternative I can think of, is embedding a web view in your GUI presentation. Perhaps you can then dispense with a lot of concern of what widgets to use/avoid, merely by using your html5-candy in the embedded web view?

How would I go about uploading a python based file converter/parser to the web?

I have written a python script that takes a text file and reorganizes the information in the file. It takes out certain pieces of information and moves others. So basically it converts it. I want the simplest possible web interface that people can visit. On this interface there is a button to upload a your file then the interface runs this script on it and puts the output file on a new page or the same one for viewing or download. What would be the easiest way to do this? Should I look into Django? I feel like this is so basic that I almost don't need a web framework. I am basically just looking for some good keywords to google for this or other resources. What would this tool be called in web-lingo? Would a framework (django, ect.) help me out that much or is there a way to just basically have an html upload form that calls the python script?

While you could use raw WSGI (the newer way) or Python's cgi module (perhaps more traditional), you might want to look into some minimal web frameworks built on top of WSGI. Here are a few:
Flask
Bottle
CherryPy
web.py
They're all pretty simple to set up and get started with.
Pick one and figure out how to do whatever you need.

Utility Mill makes this kind of utility packaging very easy.

A good Python web framework for existing console program?

I hope I'm asking on the right StackExchange site.
I've written a console program in Python onto which I'd like to put a web interface, but I'm having a hard time deciding what web framework to choose. I don't need much, but I'd like to avoid unnecessary work in trying to use it. I don't have a need for a database (for now), so that's not important to me at all.
I've looked at Django, Web2py, bottle.py, and web.py.
Django and Web2py seem to be great if I were starting out from scratch, but I'm not, and seems a little difficult to integrate into existing code.
bottle.py and web.py almost seem like they could work out, but they're so basic, I'm hoping there's something else out there that wouldn't require so much in the way of templating as these seem to do.
I don't simply want to make a carbon copy of the console interface put into a browser, but rather customize it for a web interface, so I'm not necessarily looking for anything that would simply wrap a console application into a web interface (although that would be interesting too.)

That's a sort of hard problem... Personally I don't see web.py as all that 'basic' as you put it. It should be really easy to wrap your code in some classes with GET and POST functions and be done.
Also, Django can be 'minified' as it were: How do I write a single-file Django application? is a whole conversation about this.
I would say, what is too 'basic' for you? You mentioned 'templating', but how would something magically template for you? There are open source templates for web apps, things like twitter bootstrap come to mind, that kind of give you a ready-made template for your next web app. Also YUI, and dojo do similar sorts of things (tho have a much different focus, since they are full blown JS frameworks).
That said, there is a brand new project called 'shovel' (here): https://github.com/seomoz/shovel
I haven't used it yet, but it seems to do the wrapping of commands into a web interface for you. which you said would be 'interesting'.
Personally I use web.py for all my web stuff.

I suggest Django. I've used Django both for simple mostly static sites and for sites with a lot of forms and I can't say Django imposes any restrictions or forces you to write hundreds LoC even for simple things. Instead you get nice auto generated administrative interface, built-in ORM, internationalization tools and many other things. Thereby, you have great opportunities to grow functionality of your app. In addition it has such vital thing as up-to-date documentation for every module.
Tutorial takes few hours and gives enough information to start developing full-blown sites.

Thanks the continuation which is implemented in the Nagare framework, you can develop a Web application like a console or desktop UI application: put the console code in a component.Task, then create some components for each interaction, i.e. some views that show the data that you print in your console application and receive some user input back. Then, the Nagare framework takes care of the rest: no need to declare URLS, to pass the context from a page to next...

PyQT can be handy if you are looking to implement it, to quote from the RiverBank PyQT website listed below:
"The QtWebKit module implements a web browser engine based on the WebKit open source browser engine used by Apple's Safari. It allows the methods and properties of Python objects to be published and appear as JavaScript objects to scripts embedded in HTML pages."
Source: http://www.riverbankcomputing.co.uk/software/pyqt/intro
Also, do not give up hope if that does not do the trick, as there is also "Pyjamas" which is very handy! Here is a brief description of it:
"Pyjamas is a Rich Internet Application (RIA) Development Platform for both Web and Desktop.
It contains a Python-to-Javascript compiler, an AJAX framework and a Widget Set API. Pyjamas started life as a Python port of Google Web Toolkit, the Java-to-Javascript compiler. Read the FAQ and the list of features."
source: http://pyjs.org
found via: google.com
I would say that when you are always looking to see what the best to use is, ask your question in google, and look on multiple sites and compare the top results of multiple sites to your question, also, it really depends on what you need and what your strong hand plays better in.

rich web client vs thin web client

I have one design decision to make.
In my web(ajax) application we need to decide where should we put user interface logic ?
Should It be completely loaded via javascript ( pure single page ) . and Only data comes and go.
or
Should server send some format (XML) which translated via javascript to dynamically create rich user interface. ( semi-ajax ). so some data and ui comes and go.
Which option is better ? ( speed, ease of development, platform independence )
Thanks.

IMO, it depends mostly on what kind of application it is. Is it used more like a desktop application? Then single page might work well. Having an Ajax-client to a large extent has the same drawbacks as using frames but that isn't a big problem in desktop-style applications.
Your second option works better if it more like a traditional website with many different pages with dfferent content, Then you want to have separate URL's to that different content. But then making an Ajax application might not give you all that much in the first place. Having some bits of Ajax on the page might be useful, but loading all the data with Ajax might not add anything to your app, except making it slower.

I faced similar dilemma few months back. As Lennart (above) says it makes sense to go for pyjamas or similar library if your app is more desktopish.
Further one of the biggest advantage of pyjamas provide is logically well separated backend and frontend code. IMO that is very important.
If your app is not like a desktop app (like ours was), then multipage offers more advantages, such as single change wont breaks entire app, easier to maintain etc. You might want to consider can have your app server serve json and other web server serves static content and js. Js would request json app server for data. That way we managed to keep out frontend and backend separate. Further we chose mootools as js lib over pyjamas. Ofcourse it is upto your taste and need of application. We did use python template server side templates but at compile time not at runtime like usual approach. This needed to change our thinking a little but the offered many advantages.
I end up telling you my story but I thought it's relevant and hope that helps.

The biggest influence is whether you are concerned about initial page load time. If you don't mind having all the UI there at page load, your app can be more responsive by just shuttling data instead of UI. If you want faster load and don't mind larger AJAX requests, sending some UI markup isn't bad. If you have the server power to pre-render UI with data and send the fully-ready marked-up data to the user, their browser will perform more quickly, and initial page-load should be fast.
Which course you choose should depend on the task at hand. Not all requests need be handled the same way.

Which option is better ? ( speed, ease of development, platform independence )
Platform independence, if you mean cross-browser compatibility, is a HUGE reason to use pyjamas because the python code includes a sane override infrastructure which handles everything for you. No more JS compatibility classes.
Anyway Pyjamas is all about loading the client app and then using json-rpc for the data only. That's because it's faster (once the app loaded up), easier to separate server and client, easier to maintain since all the UI code is in widgets in one place.
I've seen stuff like DokuWiki which use a php script to serve up javascript and my first thought was "WHY?" but it works pretty well I guess. It probably makes sense if you mostly have static pages with the occasional bit of JS for decoration.