I have a set of URL's in my django application that trigger certain actions or processes. This would be similar to cron jobs. I have a script that polls one or more of these URLS at some regular inverval and I'm interested in adding a layer of security.
I'd like to set up an account for the script and require authentication before the proccesses would execute. I've been reading around in the Django user authentication documentation, along with python's urllib2 library and I'm just a bit lost. I have some ideas of how this might be done, but I dont' have a lot of experience in security like this.
Any suggested reading materials?
I have a script that polls one or more of these URLS at some regular inverval and I'm interested in adding a layer of security.
Have you considered using Celery? Celery works seamlessly with Django. This will let you periodically run jobs using the same authentication mechanism as the rest of the project. You can also make things a bit more uniform by avoiding urllib2.
Related
So, I am currently working on a django project hosted at pythonanywhere, which includes a feature for notifications, while also receiving data externally from sensors through AWS. I have been thinking of the best practice in order to implement this.
I currently have a simple implementation which is a view that checks all notifications and does the actions as needed if required, with an always-on task (which simply means a script that is running independently) sending a REST request to the server every minute.
Server side:
views.py:
def checkNotifications(request):
notificationsObject = notifications.objects.order_by('thing').values_list('thing').distinct()
thingsList = list(notificationsObject)
for thing in thingsList:
valuesDic = returnAllField(thing)
thingNotifications = notifications.objects.filter(thing=thing)
#Do stuff for each notification
urls:
path('notifications/',views.checkNotifications,name="checkNotification")
and the client just sents a GET request to my URL/notifications/. which works.
Now, while researching I saw some other options such as the ones discussed here with django background tasks and/or celery:
How to initialize repeating tasks using Django Background Tasks?
Celery task best practices in Django/Python
as well as some other options.
My question is: Is there a benefit to moving from my first implementation to this one? The only benefit I can see directly is avoid abuse from another service trying to hit my URl to check notifications too often, but I can/have a required authentication to avoid that. And, is there a certain "best practice" with regards to this, considering that I am checking with this repeating task quite so often, it almost feels like there should be a more proper/cleaner solution. For one, I am not sure if running a repeating task is the best option with pythonanywhere.
(https://help.pythonanywhere.com/pages/AsyncInWebApps/ suggests using always-on tasks, but it also mentions django background tasks)
Thank you
To use Django background tasks on PythonAnywhere you need to run it using an always-on task, so it is not an alternative, but just the other use of always-on tasks.
You can also access your Django code in your always-on task directly with some kind of long-running management command, so you do not need to hit your web app with a special request.
I have a server which runs flask with python.
Now I want to make an application which can do various tasks like uploading files, updating redis database and various other things.
Now ofcourse this could be done using html pages but since the operation could involve lots of files realtime input of data and other things it might be better to make an application and manage the server from that point rather than webpages.
do you suggest using webpages anyway or would you make an application for it?
and if I make an application should I use http or not?
sorry if this is a uninformed question but I would like to learn the best methods
You might want to look into Flask-Script. It allows you to run various commands related to your flask application easily. It also allows you to easily add your own commands to it. This way you will be able to keep your administrative code still within the Flask app, but not necessarily have it accessible via a web page.
I am currently working on a complex web interface and backend, that will need to address several issues.
Scalablility
multiple deployments of varying load demands
Very structured authorization groups
Different views for different user groups
admin panel
user/content management
Large managed database
current
long term stored data (histories)
Data Updates
Polling
Ex. Search queries, static pages/files, report generation per request
Pushing (likely websockets)
Ex. Real-time notifications
Varying protocols
Ex. HTTP, SSL, Websockets
I would like to use Python, because I have grown to really enjoy the language, and I am considering some combo of Django and Twisted.
I have some experience with Django, which I love for its MVT style of application programming, its authorization models, its admin panel, and its database API. However, it is not so strong in some of the data requirements that I need, in particular, the real-time aspects.
Now, I have not really used Twisted before, but I have seen many interesting things to it. In particular the async aspects, and the ability to run many protocols.
The problems in getting the two to work together are obvious in that Django is a blocking server and Twisted is designed to be non-blocking. I have seen some topics stating using the two together is possible and have had success with it. It also seems possible to run both and proxy them to accept different urls, but getting the authentication over the two may become tricky?
Having said all of that, I would like to ask if I am on the right track for implementing this system, as well as suggestions on how to use the two together, alternatives, or if I should just kick one out (at this point, I guess it'd have to be Django, because the real time stuff is necessary). I should mention that I have written some of the preliminary data models and views in Django already.
I am quite experienced on the client side of things (JS,CSS,HTML), but I am not so savvy in the server side of things. Any input would be helpful, thanks.
You can definitely use Twisted with Django. Several projects have used the two together to good effect. twistd web --wsgi provides a basic way to get it set up, and there's a great example with more bells and whistles, like static content by Alex Clemesha on github.
I'd like to write a simple web content filter with flexible filtering rules that are written in Python. The filter is to be used as a forward proxy.
Now, I have trouble choosing the right tools for this. What do you think would be a good set of tools? So far, I've been considering Apache HTTP server with mod_proxy and mod_python or mod_wsgi, but I got stuck with the setup (mod_python is poorly documentated, IMO).
Btw, I am aware of and have experience with existing content filters such as squid and dansguardian. I am trying to write my own because the filtering capabilities of these content filters aren't sophisticated enough for my case.
You can use django middleware to intercept HTTP request/response traffic before it reaches your application (which might be in this case your graphical interface to fine tune your filter and/or database handling for storing your configurations or preset rules).
My initial imagination for your application, is that you will have a web interface for easy configuration and tuning for your system, store those configurations and rules in the database. In the middleware, put code logic that will read the configurations and rules form the database and apply them on the outgoing/incoming traffic.
I much prefer this model than doing this in django's application itself (views).
You can also put all sorts of logging and monitoring in your middleware script, and don't forget to enable it of course to make it functional :-).
I'm writing a syndication client, with the aim being to have a client for devices, and a web site that has the same functionality. I shall develop the website using Django - this is already decided; the client shall be written in python with both a CLI and a PyQt4 GUI. I have been writing the clinet first, and it's fairly database-heavy, as everything is cached to enable it to be read while offline.
It struck me today that it would make sense to use Django models for my application, to reduce the repetition of effort between the client and the website. My question is how easy it is to seperate this, and how much of Django I will need in my client to use Django's models. AFAIK I should not need to run the server, but what else is needed? I had an idea of generating the same html for my client as the website, but showing it withing Qt widgets rather than serving pages for a browser.
Has anyone tried this sort of thing before? I'm starting on this already, but it would be good to get a warning of potential dead-ends or things that will create a maintainance nightmare...
Read up on standalone Django scripts and you'll be on your path to victory. Basically all you're really doing is referencing the Django settings.py (which Django expects) and then using models without web views or urls.
If all you're really interested in is using Django's ORM to manage your models and database interaction, you might want to consider using SQLAlchemy instead.
You'll still have to run the Django app as a web server, but you can restrict it to serve to only localhost or something. And sure, you can use QtWebKit as the client.