We Keep Coding

Django upload excel file, process with pandas, download as csv

I'm running Django on a localhost (Later to run on a LAN) where the idea is I can go onto the webpage, click a button where you're prompted to select an excel file from your computer. Pandas will do work on said excel file and Django/Pandas will make excel file of this Pandas data frame as a download prompt.
I've got Django running and using the code below from the module 'Django-Excel' does the basics of what I want. Excel file in ---> excel file out, no saving the file to a database or anything just keeping it in the memory. However, I cannot find a way to shoehorn Pandas into it. The main problem I have is I'm unsure how to return an excel file using Pandas. I've been using '.to_excel()' in my offline python code, however, haven't found a way to use it in a localhost running Django. I'm sure I'm missing something really simple but I just can't get it.
Maybe if someone could show a simple example of say, uploading an excel, pandas will multiply a column of numbers in the excel by 2, new excel is outputted to read/save.
from django.shortcuts import render, redirect
from django.http import HttpResponseBadRequest, HttpResponse
from _compact import JsonResponse
from django import forms
import django_excel as excel
from polls.models import Question, Choice
class UploadFileForm(forms.Form):
file = forms.FileField()
# Create your views here.
def upload(request):
if request.method == "POST":
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
filehandle = request.FILES['file']
# do pandas here to filehandle/ put filehandle into a function
return excel.make_response(filehandle.get_sheet(), "csv",
file_name="download")
else:
form = UploadFileForm()
return render(
request,
'upload_form.html',
{
'form': form,
'title': 'Excel file upload and download example',
'header': ('Please choose any excel file ' +
'from your cloned repository:')
})
Thank you very much, sorry if this isn't clear It's my first time posting on this website, if you don't understand what I'm asking for I'll elaborate more.

I'll upload my answer for anyone looking at this in the future :-)
def uploads(request):
if request.method == "POST":
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
kappa = request.FILES['file']
# Do work on kappa/excel file here with Pandas
output = io.BytesIO()
writer = pd.ExcelWriter(output, engine='xlsxwriter')
kappa.to_excel(writer, index=False)
writer.save()
output.seek(0)
response = HttpResponse(output,
content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')
response['Content-Disposition'] = 'attachment; filename=%s.xlsx' % 'Download'
return response
else:
form = UploadFileForm()
return render(request, 'upload_form.html', {'form': form})
I abandoned using 'Django-excel' and instead used 'BytesIO' from the io module. This has the bonus of not relying on a third-party module.
you're able to write a Pandas dataframe to io.BytesIO() and then use that in your HttpResponse.

django project multiple user use the same project

I am new to django and web programming. Right now, I have created a django project for configuration form generation. It allows the user to input the values and generate a configuration form once the user got the URL.
Project name: portal, App name: home, input_data.txt: a text file stored the values for the corresponding parameter and it will be read for further processing
It works fine for myself, but multiple users use it at the same time. It doesn't work. what can I do in order to allow multiple users use it at the same time?
forms.py
from django import forms
from .models import FormGen
from .models import BB_Data
class FinalForm(forms.ModelForm):
class Meta:
model=FormGen
field=['full_name','backbone_number']
class BBForm(forms.ModelForm):
class Meta:
model=BB_Data
fields=['backbone_name','IP','numbers']
widget={'numbers':forms.TextInput(attrs={'readonly':'readonly'})}
views.py
from django.shortcuts import render,HttpResponse,redirect
from .forms import FinalForm,BBForm
from django.core.files import File
import re
def replacePara(template,para):
for key,val in para.items():
template=template.replace(key,val)
return template
def create(request):
title="Router Generator"
form=FinalForm(request.POST or None)
context={
"template_title":title,
"form":form,
}
if(form.is_valid()):
form_data={
"HNAME":form.cleaned_data.get("Hostname"),
"ROLE":form.cleaned_data.get("Router_Role"),
}
form.save()
f=open('./input_para.txt','r')
a=f.read();
f.close();
tt=replacePara(a,form_data)
b=open('./input_data.txt','w')
b.write(tt)
b.close()
return redirect('/backbone')
return render(request,"home/form.html",context)
def bb(request):
title="Backbone Information"
if request.method=="POST":
form=BBForm(request.POST)
if form.is_valid():
bb_form={
'BBNAME':form.cleaned_data.get('backbone_name'),
'BBIP':form.cleaned_data.get('IP'),
}
a=int(form.cleaned_data.get('numbers')
datainput=''
if a==1:
f=open('./bb_set.txt','w')
f.write(bb_form)
f.close()
else:
f=open('./bb_set.txt','a')
outBB.write(bb_form)
outBB.close()
form.save()
f=open('./input_data.txt','r')
t=f.read()
tt=int(t.split()[7]) #it get the max number of backbones
if(at<tt):
at=at+1;
bb=BBForm(initial={'numbers':str(at)})
context={
"template_title":title,
"form":bb
}
return render(request,"home/bb.html",context)
else:
# generate the configurate text file #
content=final # content store the configurations
filename="ConfigForm.txt"
response=HttpResponse(content,content_type='text/plain')
response['Content-Disposition']='attachment; filename={0}'.format(filename)
return response
else:
return HttpResponse("something wrong")
else:
form=BBForm(initial={'numbers':"1"})
f=open('./input_data.txt','r')
t=f.read()
tt=int(t.split()[7])
context={
'template_title':title,
'form':form,
'max':tt
}
return render(request,"home/bb.html",context)
input_para: it is the text file stored the name of the parameter
input_data: it is the text file stored the values for correspond parameter
bb_set: it is the text file stored all the backbone information

You're saving data to a single file, with no regards as to users. If I go to your website, I'll be writing to the same input_data.txt, input_para.txt, and bb_set.txt files as anyone else also visiting.
This is not the way to persist user data for a website/service. You should be using a database.
Luckily, you're using Django, which has phenomenal database integration, and it's actually really easy. The hardest part, for you, will be designing the database. You'll want a User model, against which you can record input_data, input_para, and bb_set data against, from what I can tell.
I recommend you follow the Django tutorial, specifically this section, and perhaps also read up on database design, (including normalisation of data, which is more interesting than it sounds).

python django run bash script in server

I would like to create a website-app to run a bash script located in a server. Basically I want this website for:
Upload a file
select some parameters
Run a bash script taking the input file and the parameters
Download the results
I know you can do this with php, javascript... but I have never program in these languages. However I can program in python. I have used pyQT library in python for similar purposes.
Can this be done with django? or should I start learning php & javascript?
I cannot find any tutorial for this specific task in Django.

This can be done in Python using the Django framework.
First create a form including a FileField and the fields for the other parameters:
from django import forms
class UploadFileForm(forms.Form):
my_parameter = forms.CharField(max_length=50)
file = forms.FileField()
Include the UploadFileForm in your view and call your function for handling the uploaded file:
from django.http import HttpResponseRedirect
from django.shortcuts import render
from .forms import UploadFileForm
# Imaginary function to handle an uploaded file.
from somewhere import handle_uploaded_file
def upload_file(request):
if request.method == 'POST':
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
my_parameter = form.cleaned_data['my_parameter']
# Handle the uploaded file
results = handle_uploaded_file(request.FILES['file'], title)
# Clear the form and parse the results
form = UploadFileForm()
return render(request, 'upload.html', {'form': form, 'results': results})
else:
form = UploadFileForm()
return render(request, 'upload.html', {'form': form})
Create the function to handle the uploaded file and call your bash script:
import subprocess
import os
def handle_uploaded_file(f, my_parameter):
file_path = os.path.join('/path/to/destination/', f.name)
# Save the file
with open(file_path, 'wb+') as destination:
for chunk in f.chunks():
destination.write(chunk)
# Call your bash script with the
output = subprocess.check_output(['./my_script.sh',str(file_path),str(my_parameter)], shell=True)
return output
Check out https://docs.djangoproject.com/en/1.10/topics/http/file-uploads/ for more examples and instructions on how the handle file uploads in Django.

Django: Using the FileField

I want to make a application when the user upload the file, so get it to system, and execute the linux command like "xxd".
cSite---cSite---settings.py
| |__etc...
|
--myapp---
Here is my file upload codes.
views.py :
from django.shortcuts import render_to_response
from django.template import RequestContext
from django.http import HttpResponseRedirect
from django.core.urlresolvers import reverse
from myapp.models import Document
from myapp.forms import DocumentForm
def list(request):
# Handle file upload
if request.method == 'POST':
form = DocumentForm(request.POST, request.FILES)
if form.is_valid():
newdoc = Document(docfile = request.FILES['docfile'])
newdoc.save()
# Redirect to the document list after POST
return HttpResponseRedirect(reverse('myapp.views.list'))
else:
form = DocumentForm() # A empty, unbound form
# Load documents for the list page
documents = Document.objects.all()
# Render list page with the documents and the form
return render_to_response(
'list.html',
{'documents': documents, 'form': form},
context_instance=RequestContext(request)
)
models.py :
from django.db import models
class Document(models.Model):
docfile = models.FileField(upload_to='documents/%Y/%m/%d')
As you know, this code is widespreaded simple file upload form.
Now, here is my trouble.
Want to use the salt when I save the file
(Can I use the "Keeping Original Filename for FileField in Django"? and if I can, how to apply this?
I don't know what is representing the file path variable in FileField on ./views.py/ It always contain the error. (The command must be executed at the right after same time of file uploaded in user view.)

Does file uploading in Django have a delay before the user can see the uploaded files?

I have a typical models.ModelForm that has a single field: a FileField. However, when the user uploads a file using the form and the page reloads, the file is not listed. The user has to reload the page manually for it to appear as the "current file."
This is my view code:
submission = Submission.objects.filter(user=request.user, problem=problem).first()
if request.POST:
submission_form = SubmissionForm(request.POST, request.FILES, instance=submission)
if submission_form.is_valid():
submission_form.save()
else:
submission_form = SubmissionForm(instance=submission)
return render_to_response('view-problem.html',
dictionary={'problem': problem, 'submission_form': submission_form},
context_instance=RequestContext(request))
My guess is that the file is still being processed while the view is loaded, so the user doesn't immediately see it. How do I fix this?

You have to create a new form after saving the old form.ModelForm.save() returns a model instance, you can use this instance to initialize the new ModelForm.
Code:
if request.POST:
submission_form = SubmissionForm(request.POST, request.FILES, instance=submission)
if submission_form.is_valid():
submission = submission_form.save()
submission_form = SubmissionForm(instance=submission)