Noob here with Django. I have the following folder structure for a Django app, which is inside the main project folder.
my_app/
__init__.py
admin.py
apps.py
migrations/
__init__.py
models.py
tests.py
views.py
I have a command line python script I wrote to fetch a JSON file and parse it to displays very specific information. it's using requests library for JSON and data parsing.
My question is how do I integrate my script into Django app. specifically how to bring the logic of it and to place under which file? My thinking is to create another file and import them into views. and pass them into render function - this maybe not the right and Django way, but kinda stuck there. Oh and I don't use any DB, the script uses a text file and writes to it as well.
I place a folder called services in my Django app and for each non-django stuff I add a folder. But that is basically just convenience, Django is not posing restrictions on you here.
Since you are including your script in the views, it is supposed that:
Your JSON file does not change frequently
It is not an issue to have multiple requests if your Django app is spawned many times (e.g. if you use uwsgi or gunicorn)
If such is the case, any Pythonic solution will be fine.
If 1 does not apply, it may be the case that you should implement your own middleware that, upon request,
fetches the JSON file if it is expired (e.g. if the last fetch happened long time ago)
adds the JSON file content in the request: this way you do not have to disclosure where you are keeping your JSON file
If you have multiple instances, you can do more or less the same as 1, but you might decide to store the JSON value in a separate storage, together with its expiration. You might also configure uwsgi/guincorn to fetch the JSON file at startup: this way you will not fetch the JSON file multiple times at startup. Then your middleware will do the work to keep it up-to-date, if necessary.
Related
I am quite beginner to Django and I am confronted with this question.
provided I have the following structure:
django_app/
django_app/
__init__.py
asgi.py
settings.py
urls.py
views.py
wsgi.py
loaddata_app/
>migrations
__init__.py
admin.py
apps.py
models.py
test.py
urls.py
views.py
>static
>templates
loaddata.html
My loaddata app is a super easy app that contains a text field to enter (paste) text.
The app should send the text to the server where a series of methods are applied to extract information out of the text and generate output data that will be presented to back to the user.
The first question is: If you have a bunch of methods in various files, where do you actually "store" those methods. And where are those methods supposed to be imported (if they need at all to be imported)?
What happens if those methods take (so to say) 10 minutes to run? is the connection still maintained with the browser of the user in the meanwhile?
So looks the simple app:
NOTE:
The question is partially answered here. But it is not entirely clear where those utils are imported and run in order to process the data.
Django: where to store functions common to all apps
Assuming that the view class for your particular URL which does this job of getting input text and then processing and showing output is in the loaddata_app, and you have a bunch of functions common that you require.
You can keep the function in a python package at the root level of your project. You can name the package as utils or utilities or even as common. You can then create a file common_functions.py to put all your functions in.
So your project structure would look something like this.
django_app/
django_app/
__init__.py
asgi.py
settings.py
urls.py
views.py
wsgi.py
loaddata_app/
>migrations
__init__.py
admin.py
apps.py
models.py
test.py
urls.py
views.py
>static
>templates
loaddata.html
>utils
__init__.py
common_functions.py
You can then import these functions (eg: func1, funct2) in your loaddata_app/views.py as from utils.common_functions import func1, func2
Hope I have answered your doubt.
You can also check how to create a python package in this link
Now, to answer your second question about the time, by default when you are running a django application using the development server, ie, python manage.py runserver and the request - response takes 10 mins, the connection will be live and it will not timeout.
But, say, if you deployed this application in Heroku or behind any application gateway (from AWS or Azure, etc) then, there will be a timeout and the connection will not persist (ie, it will be terminated). By default for heroku, the timeout is 30 seconds. In such cases, you should use something like Celery to handle large tasks as background processes and still send some response to the user's browser or client.
I have JSON data coming through various external API's into my Django project. I have two apps, one called 'products' and other 'extract'. Through product app, have created the database and rendered the views required. Objective of extract app is to parse the JSON data and create/update the fields in the Django database. I have this code saved in extract/views.py as shown below -
import json
import urllib2
from products.models import Product
url = " .........."
.............
for i in data[results]:
Product.objects.get_or_create (...........)
The issue I am having is this code is working well in python shell. It is extracting all the JSON data and updating the database. But, when I run the script on the command line outside of Python shell, it is giving error - 'Import error: No module named products.models'.
Django requires you to run python manage.py shell in order for the product app's models and Django internals to be initialized.
Check out this blog post by Stavros Korokithakis to convert your code into a standalone script.
I believe the error is with the directory set up. Make sure all your migrations are done as well as the initial folder has the paths to your apps. Possible you need to include the paths to each app in the other's settings.
My django application has a file uploader which uploads to a specific location in my local system.
It redirects to a new html page which shows successful message after upload is done.
Once the file is uploaded I need to do some processing of csv files.
I have a python code which does the processing.
Now my question is, where do I put the python file in the django project and how do i call it to be run once the upload is done?
Any help is appreciable
Thanks in advance
You can place it anywhere you like, it's just Python. Maybe in a csv_processing.py if it fits in a single module, or as a completely independent library if it's more. Django doesn't have an opinion on this.
The best way to run it is by doing it asynchronously using Celery.
Make sure the file is within a python package, you do this by adding init.py to the directory, documentation here. In accordance to Django convention; you would place the file within the app that needs to use it, or within another app you would name utils, documentation here.
Question:
I need the file to be run completely after the application uploads the file.
Answer:
new = Storage()
new.file = request.FILES['file']
new.save()
Now we have the database id. (When file object saved into database it emits the id).
originalObj = Storage.objects.get(pk=new.id)
Now you can import the csv file and do modification here.
I am using Django to develop an API using an algorithm I wrote.
When someone requests a url, my urls.py calls a function in views.py which serves
page that returns a JSON string.
If my algorithm is in my views.py file, or in another file on my server, would it be possible for a user to view the contents of this file, and then see my algorithm?
In other words, when using Django, which files will never be served to a user, and which files will be?
Is there any way I can stop someone from viewing my algorithm if it's in a .py file? Other than Chmodding the file or encrypting the code?
Thank you for your time.
Django only serves the responses that you explicitly create and return from your views. There is no general ability to request files from it.
Make sure your source code isn't in a directory that your web server is configured to serve from, and make sure your settings.py value for DEBUG is False, and you should be fine. Oh, and just in case - don't try to use the Django development server in production.
As long as nobody has shell access to your server, people will never see more than the actual HTML output of your page. .py files are not shown to the user that has requested an url in the browser.
I plan to build my project in Django framework. However, I noticed that all Django packages have models.py file. Now, let say I have a set of general purpose functions that I share between several apps in the project and I plan to put these functions definitions in a separate package (or app for that matter?). So, should I create an app "general" and copy-paste these functions into the models.py file? Or can I just create a general.py file in the "general" app directory and leave models.py empty? What is the "Django" way to do that?
Thanks.
models.py file is used to define the structure of database. So you should leave it for defining your database entries. You can make an app named generals and put general.py in that app, and from there you can use it by calling it in any app.
I usually create a utils.py file under my main app that is created from the django-admin.py when starting the project.
I plan to put these functions definitions in a separate package (or app for that matter?)
Before you decide to make this an app (and if you do decide to make it an app), I recommend you take a look at James Bennet keynote on Developing reusable apps and hist post on laying out an application. From one of his slides:
Should this be its own application?
Is it orthogonal to whatever else I’m doing?
Will I need similar functionality on other sites?
Yes? Then I should break it out into a separate application.
If you're cramming too much functionality in one single general purpose app, it might be better to split your general purpose app into multiple reusable apps.
Going back to your original question, Django is expecting a models.py file in every app. So you must have the file even if it's empty.
Inside your models.py, you should only have the application’s model classes. So, you wouldn't be following a best practice if you put inside models.py some miscellaneous code you want to reuse.
From the laying out an application post I mentioned before:
At the application level, I usually drop in a few more files depending on exactly what the application is going to be using:
If the application defines any custom manipulators, I put them in a file called forms.py instead of in the views file.
If there are multiple custom managers in the app, I put them in a file called managers.py instead of the models file.
If I’m defining any custom context processors, I put them in a file called context_processors.py.
If I’m setting up any custom dispatcher signals, they go in a file called signals.py.
If the application is setting up any syndication feeds, the feed classes go in a file called feeds.py. Similarly, sitemap classes go in sitemaps.py.
Middleware classes go in a file called middleware.py.
Any miscellaneous code which doesn’t clearly go anywhere else goes in a file or module called utils.
All of this does not answer directly your original question:
can I just create a general.py file in the "general" app directory and leave models.py empty?
But I hope this gives you additional information to make a decision that better fits your project and requirements.