In general terms, is there a particular method that can be used to prevent the author of a plugin/widget from exploiting the app i.e. for malicious use?
For example, WordPress. Does it follow a particular architecture or set of design patterns?
I could enforce that the author gives me the data and I decide how to render it but that would be limiting. Can this be done without significantly limiting developers? Both front end and backend? (JavaScript and python in my case)
I am creating a dashboard where developers can create widgets, so they can write a plugin which consists of a front end widget (anything from a simple list to Google maps) and a back end script which handles wiring such as API requests.
Related
I am relative new to developing web applications.
I would like your comments and suggestions of improvement for the following architectural considerations.
I have developed an expert system ES using CLIPS. Now I am planning to provide this to variety of users from our company as a web application. Before I start with going into greater details I am currently thinking about which technologies should be involved.
The goal is that the user of the web app is facing a chat-like animation which guides him to the final result while he or she provides more and more input to the ES.
After conducting some research on my own I came up with the following idea
In the backend I use PyCLIPS as an Interface between Python and CLIPS
Then I use DJANGO for integrating my python code into the web page dynamically altering the chat between user and ES.
There is one thing which is particularly still troubling me a lot: How shall I manage many concurrent users? Shall I use one ES with every user having an individual set of facts or shall every user have his or her own instance of the ES?
Do you have any other high level approaches for this problem which could be superior to this one?
I am looking forward to your experience and input regarding this matter.
Best
I'd suggest running the expert system in a stateless mode. Each time the user makes changes, you submit all data from the web page to the expert system and then retrieve the results to display on the web page. Doing it that way scales better if you have multiple users and makes it easier to implement undo logic if the user wants to change a response. There's an example showing how to do this with a CGI application at http://www.clipsrules.net/?q=Downloads/CLIPSCGI.
It is usually a good idea to split your Expert System into separate "shards".
It keeps the rule base simpler (as you don't need to distinguish to which user a fact is referring to) and allows you to scale horizontally when more users will be added.
If running one ES per user sounds overkill, you can decrease the granularity by sharding based on, for example, the first letter of the user surname or Id.
When designing similar solutions I tend to de-couple the frontend application with the ES by using a queueing system. This allows you to modify the cluster layout without the need to change the public APIs.
| Flask | ----> | RabbitMQ | ----> | ES Worker |
In case you want to change your sharding strategy, you can simply re-configure the broker queues layout without affecting the client/end-user.
I have a DB table which refers to itself. For example let's say I have a list of employees, some of them are other employees' bosses. Of course I have SQLAlchemy objects of this table.
I want to visualize the connections with a graph - each employee is a node, and a line is connecting each employee to his boss.
Is there a way to do it using flask? Or any other python or web framework?
You could render this client-side using a JavaScript graphing library like D3 or Google Charts. Retrieve and process your records using SQLAlchemy and Python, structure the data as per your graphing library's specifications, set up your element with a little JS, send your data over, and then render it in HTML. You'll get a pretty, interactive, and mutable graph of your choosing.
Still, this is a Python question. If you're allergic to JavaScript, you could use a Python library, such as the ones recommended here. A lot of those are more geared towards generating static plots, though. That's fine if you want to embed an image into your page or prepare it for a presentation, but it isn't very lively.
If you want to go interactive in the browser, but really don't want to touch any JS, you could conceivably use a Python wrapper for one of the aforementioned graphing libraries and let the wrapper write the JS for you. You're going to run into some JS at one point or another, whether you're the one generating it or not. Why not have the satisfaction and flexibility of doing it yourself? :)
I am writing a webapp and I would like to start charging my users. What are the recommended billing platforms for a python/Django webapp?
I would like something that keeps track of my users' purchase history, can elegantly handle subscription purchases, a la carte items, coupon codes, and refunds, makes it straightforward to generate invoices/receipts, and can easily integrate with most payment processors. Extra points if it comes with a fancy admin interface.
I found this django-billing project, are there any others? Also, do you rely on your payment processor to handle these tasks or do you do all of them yourself?
Note: I am not asking what payment processors to use, but rather what middleware/libraries one should run on their webapp itself.
The koalixcrm is perhaps something you could start with.
It offers some of your required functionality. Still it is in a prealpha stage but it already provides PDF export for Invoices and Quotes, there is already one included plugin for subscriptions.
also try the demo
As i am the developer of the koalixcrm im very interested to work with you - perhaps we can merge our projects.
It's not really clear why Django Community hasn't come up a with complete billing system or at least a generic one to start working on.
There's many packages that can be used for getting an idea how to implement such platform:
https://www.djangopackages.com/grids/g/payment-processing/
At the company I work for, we are building a cluster of web applications for collaboration. Things like accounting, billing, CRM etc.
We are using a RESTfull technique:
For database we use CouchDB
Different applications communicate with one another and with the database via http.
Besides, we have a single sign on solution, so that when you login in one application, you are automatically logged to the other.
For all apps we use Python (Pylons).
Now we need to add instant messaging to the stack.
We need to support both web and desktop clients. But just being able to chat is not enough.
We need to be able to achieve all of the following (and more similar things).
When somebody gets assigned to a task, they must receive a message. I guess this is possible with some system daemon.
There must be an option to automatically group people in groups by lots of different properties. For example, there must be groups divided both by geographical location, by company division, by job type (all the programers from different cities and different company divisions must form a group), so that one can send mass messages to a group of choice.
Rooms should be automatically created and destroyed. For example when several people visit the same invoice, a room for them must be automatically created (and they must auto-join). And when all leave the invoice, the room must be destroyed.
Authentication and authorization from our applications.
I can implement this using custom solutions like hookbox http://hookbox.org/docs/intro.html
but then I'll have lots of problems in supporting desktop clients.
I have no former experience with instant messaging. I've been reading about this lately. I've been looking mostly at things like ejabberd. But it has been a hard time and I can't find whether what I want is possible at all.
So I'd be happy if people with experience in this field could help me with some advice, articles, tales of what is possible etc.
Like frx suggested above, the StropheJS folks have an excellent book about web+xmpp coding but since you mentioned you have no experience in this type of coding I would suggest talking to some folks who have :) It will save you time in the long run - not that I'm saying don't try to implement what frx outlines, it could be a fun project :)
I know of one group who has implemented something similar and chatting with them would help solidify what you have in mind: http://andyet.net/ (I'm not affiliated with them at all except for the fact that the XMPP dev community is small and we tend to know each other :)
All goals could be achieved with ejabberd, strophe and little server side scripting
When someone gets assigned to task, server side script could easily authenticate to xmpp server and send message stanza to assigned JID. That its trivial task.
To group different people in groups, it is easily can be done from web chat app if those user properties are stored somewhere. Just join them in particular multi user chat room after authentication.
Ejabberd has option to automatically create and destroy rooms.
Ejabberd has various authorization methods including database and script auth
You could take look at StropheJS library, they have great book (paperback) released. Really recommend to read this book http://professionalxmpp.com/
I'm a firm believer of the heretic thought of tight coupling between the backend and frontend: I want existing, implied knowledge about a backend to be automatically made use of when generating user interfaces. E.g., if a VARCHAR column has a maximum with of 20 characters, there GUIs should automatically constrain the user from typing more than 20 characters in a related form field.
And I have strong antipathy to ORMs which want to define my database tables, or are based on some hack where every table needs to have extra numeric ID columns because of the ORM.
I've looked a bit into Python database frameworks and I think I can conclude the SQLAlchemy fits best to my mentality.
Now, I need to find a web application framework which fits naturally with SQLAlchemy (or an equivalent) and perhaps even with my appetite for coupling. With "web application framework", I mean products/project such as Pyhons, Django, TurboGears, web2py, etc.
E.g., it should ideally be able to:
automatically select a suitable form widget for data entering a given column if told to do so; e.g., if the column has a foreign key to a column with 10 different values, widget should display the 10 possible values as a dropdown
auto-generate javascript form validation code which gives the end-user quick error feedback if a string is entered into a field which is about to end up in an INTEGER column, etc
auto-generate a calendar widget for data which will end up in a DATE column
hint NOT NULL constraints as javascript which complains about empty or whitespace-only data in a related input field
generate javascript validation code which matches relevant (simple) CHECK-constraints
make it easy to avoid SQL injection, by using prepared statements and/or validation of externally derived data
make it easy to avoid cross site scripting by automatically escape outgoing strings when appropriate
make use of constraint names to generate somewhat user friendly error messages in case a constrataint is violated
All this should happen dynamically, so table adjustments are automatically reflected on the frontend - probably with a caching mechanism, so that all the model introspection wouldn't kill performance. In other words, I don't want to repeat my model definition in an XML file (or alike) when it has already been carefully been defined in my database.
Does such a framework exist for Python (or for any language, for that matter)? If not: Which of the several Python web application frameworks will be least in the way if I were to add parts of the above features myself?
web2py does most of what you ask:
Based on a field type and its validators it will render the field with the appropriate widget. You can override with
db.table.field.widget=...
and use a third party widget.
web2py has js to blocks the user from entering a non-integer in a integer field or a non-double in a double field. time, date and datetime fields have their own pickers. These js validation work with (not instead) of server side validation.
There is IS_EMPTY_OR(...) validator.
The DAL prevents SQL injections since everthing is escaped when goes in the DB.
web2py prevents XSS because in {{=variable}}, 'variable' is escaped unless specified otherwise {{=XML(variable)}} or {{=XML(variable,sanitize=True)}}
Error messages are arguments of validators for example
db.table.field.requires=IS_NOT_EMPTY(error_message=T('hey! write something in here'))
T is for internationalization.
You should have a look at django and especially its newforms and admin modules. The newforms module provides a nice possibility to do server side validation with automated generation of error messages/pages for the user. Adding ajax validation is also possible
TurboGears currently uses SQLObject by default but you can use it with SQLAlchemy. They are saying that the next major release of TurboGears (1.1) will use SQLAlchemy by default.
I know that you specificity ask for a framework but I thought I would let you know about what I get up to here. I have just undergone converting my company's web application from a custom in-house ORM layer into sqlAlchemy so I am far from an expert but something that occurred to me was that sqlAlchemy has types for all of the attributes it maps from the database so why not use that to help output the right html onto the page. So we use sqlAlchemy for the back end and Cheetah templates for the front end but everything in between is basically our own still.
We have never managed to find a framework that does exactly what we want without compromise and prefer to get all the bits that work right for us and write the glue our selves.
Step 1. For each data type sqlAlchemy.types.INTEGER etc. Add an extra function toHtml (or many maybe toHTMLReadOnly, toHTMLAdminEdit whatever) and just have that return the template for the html, now you don't even have to care what data type your displaying if you just want to spit out a whole table you can just do (as a cheetah template or what ever your templating engine is).
Step 2
<table>
<tr>
#for $field in $dbObject.c:
<th>$field.name</th>
#end for
</tr>
<tr>
#for $field in dbObject.c:
<td>$field.type.toHtml($field.name, $field.value)</td>
#end for
</tr>
</table>
Using this basic method and stretching pythons introspection to its potential, in an afternoon I managed to make create read update and delete code for our whole admin section of out database, not yet with the polish of django but more then good enough for my needs.
Step 3 Discovered the need for a third step just on Friday, wanted to upload files which as you know needs more then just the varchar data types default text box. No sweat, I just overrode the rows class in my table definition from VARCHAR to FilePath(VARCHAR) where the only difference was FilePath had a different toHtml method. Worked flawlessly.
All that said, if there is a shrink wrapped one out there that does just what you want, use that.
Disclaimer: This code was written from memory after midnight and probably wont produce a functioning web page.
I believe that Django models does not support composite primary keys (see documentation). But perhaps you can use SQLAlchemy in Django? A google search indicates that you can. I have not used Django, so I don't know.
I suggest you take a look at:
ToscaWidgets
DBSprockets, including DBMechanic
Catwalk. Catwalk is an application for TurboGears 1.0 that uses SQLObject, not SQLAlchemy. Also check out this blog post and screencast.
FastData. Also uses SQLObject.
formalchemy
Rum
I do not have any deep knowledge of any of the projects above. I am just in the process of trying to add something similar to one of my own applications as what the original question mentions. The above list is simply a list of interesting projects that I have stumbled across.
As to web application frameworks for Python, I recommend TurboGears 2. Not that I have any experience with any of the other frameworks, I just like TurboGears...
If the original question's author finds a solution that works well, please update or answer this thread.