I created a view using django that displays a search engine to type in the name of the product to be scraped and starts the scraping process.
Also, I created a view as a loading page that I want to be displayed when I start the scraping and disappear when the scraping finishes and display the datatable as mentioned in my code.
Here is my view.py :
def home_view(request):
context = {}
context ['form'] = Scraping()
# if user inserted into form and validated the form
if request.method =='POST':
form= Scraping(request.POST)
if form.is_valid():
# get variable (nom product)
subject = form.cleaned_data['subject']
# remove space and replace it with a + sign
sbjt = subject.replace(" ","+")
# call scrapy command line with this variable
os.chdir('C:/Users/aicha/Desktop/new_version/django_project/aliScrap/scraper/codes/aliscraper/')
os.system("scrapy crawl aliproduct -a product=" + sbjt)
# get results from database
client = MongoClient("mongodb://localhost:27017/")
db = client["aliexpress"]
col = db["listproducts"]
products = col.find()
context = {'products' : products}
return render(request,'datatable.html', context)
# default page
return render(request,'index.html', context)
Page loading view :
def loading_view(request):
return render(request,'loading.html')
Knowing that I have already prepared the html of my loading page.
The issue is that I don't know how to integrate my loading page after starting the scraping.
I think that it should appear after the execution of this line of code :
os.system("scrapy crawl aliproduct -a product=" + sbjt)
and disappear before this line of code :
return render(request,'datatable.html', context)
I will be so grateful if you help me fixing that problem because I am pretty new using django.
Thank you in advance !
Related
I am creating a webapp with Python/Flask. I am using blueprints.
Say I search for a book and end up at the URL /search?book=somebook&author=someauthor via a GET request. On that page I have buttons for each result which will save that result to the user's saved books. This would be a POST request. However, I want to return the user to the same search page with the same URL params.
So the flow is:
User submits a search and ends up on /search?book=somebook&author=someauthor
User clicks subscribe on one of the results. A POST saves the book to the user's saved books.
User ends up on /search?book=somebook&author=someauthor again and the search result page is repopulated.
I think, incorrectly, I want something like this:
#search_bp.route('/search', methods=["GET", "POST"])
def search():
if request.method == "POST":
# save book to user's saved books
# somehow end up back on the same page from here
elif request.method == "GET":
# use request.args to populate page with results
return render_template("search.html", books=books)
In my mind I want to return redirect(url_for("search_bp.search")) but somehow get the request.args back into the URL. Is my only choice to hardcode the URL, i.e. concatenate a string url = "/search?book=" + request.args.get("book") + "&author=" + request.args.get("author") so that I can return redirect(url)?
You can pass values/variables to flask.url_for, example:
book = request.args.get('book')
author = request.args.get('author')
my_url = url_for('search_bp.search', book=book, author=author)
Additional values/parameters passed to url_for will be added to the URL as GET parameters, then you can do return redirect(my_url).
https://flask.palletsprojects.com/en/1.1.x/api/#flask.url_for
I'm new to django, and I'm trying to access the following web page by clicking on "dataset celiac" and acces so to "celiac.html". On the html side, here is my code which corresponds to this part of the page where I click on "dataset celiac":
<div class="visit">dataset celiac</div>
Then, on the views.py side, here is the code corresponding to the view which is supposed to return the html page to celiac.html:
def CeliacView(request):
# if this is a POST request we need to process the form data
if request.method == 'POST':
# create a form instance and populate it with data from the request:
form = forms.CeliacForm(request.POST)
# check whether it's valid:
if form.is_valid():
# process the data in form.cleaned_data as required
columns = form.cleaned_data['columns']
algo = form.cleaned_data['algo']
if algo == "Logistic regression" and columns == 'CDR3_IMGT':
return render(request, 'data/cdr3.html', locals())
else:
tableData = get_table_data_to_representation()
context = {'form': forms.CeliacForm(), 'tableData': tableData}
return render(request, 'data/celiac.html', context)
And regarding the urls.py file here is my code:
app_name = 'data'
urlpatterns = [
path('datasets/', views.SetView, name='datasets'),
path('celiac/', views.CeliacView, name='celiac'),
]
Finally, here is what django shows me when I click on celiac dataset: Page not found (404)
Can someone tell me what could be the problem here please? why django does not find the correct url ?
Your href shouldn't be the filename, put the path where your url points you towards. For making it more dynamic, I'd suggest using the url tag. Like so:
<div class="visit">dataset celiac</div>
So i'm trying to build something, so that users would be able to report something on site. Here's the model,
class Report(models.Model):
reporting_url = models.URLField()
message = models.TextField()
I created Form for this Model including 'message' field only because 'reporting_url' is something it needs to populate by itself depending upon the specific page from where user has clicked "Report" button.
def report(request):
url_report = ???
if request.method == 'POST':
form = ReportForm(request.POST or None)
if form.is_valid():
new_form = form.save(commit=False)
new_form.reporting_url = url_report
new_form.save()
I was wondering How can I pass the specific url to 'reporting_url' field in form depending on the Page from where user has clicked "Report" button? (Much like s we see on social Networks).
Am I doing this correctly, Or is there a better way for doing this?
Please help me with this code. Thanks in Advance!
If there is a report button on that specific page then I believe you could write custom context processor.
More info: Django: get URL of current page, including parameters, in a template
https://docs.djangoproject.com/en/1.11/ref/templates/api/
Or maybe just write it directly in the views.py in your function and set
url_report = request.get_full_path()
I think you can use the form on the same page of the URL and use:
url_report = request.get_full_path()
in the view, to get the current URL.
Else if you want to create a separate view for the reporting form. You can use
url_report = request.META.get('HTTP_REFERER')
to get the previous or refering URL which led the user to that page.
request.META.get('HTTP_REFERER') will return None if it come from a different website.
I am trying to create a web app that users will select a file, get data processed and then redirected to a confirmation page with a url parameter(?status=1) How can I move from the processing view to the confirmation view and then display the confirmation page?
I have put my code below which is not working in my views and urls py files
# views.py
def marketdata_processing(request):
if request.method == 'POST':
uploadform = forms.MyUploadForm(request.POST, request.FILES)
if uploadform.is_valid():
newdoc = models.UploadDocument(docfile=request.FILES['uploadfile'])
filename = request.FILES['uploadfile'].name
newdoc.save()
selecteddate = newdoc.getselecteddate() # Model method calculated
fileid = newdoc.pk
request.session['fileid'] = fileid
request.session['selecteddate'] = selecteddate
return redirect(reverse('views.confirm_input_details'))
else:
uploadform = forms.MyUploadForm()
# Render list page with the documents and the form
return render_to_response(
'mytemplate.html',
{'uploadform': uploadform},
context_instance=RequestContext(request)
)
def confirm_input_details(request):
fileid = request.session['fileid']
selecteddate = request.session['selecteddate']
msg = 'Proceed to import data for %s? \nFileID Being Imported is %s ' % (
selecteddate, fileid,)
url = reverse('views.confirm_input_details', kwargs={'status': 1})
return HttpResponseRedirect(url)
# urls.py
urlpatterns = [
url(r'', views.marketdata_processing, name='myapp/mytemplate.html'),
url(r'^\?status=(?P<status>[0-9]+)$',
views.confirm_input_details, name='myapp/myconfirmpage.html'),
]
There are a couple of issues I feel exist in your code:
Indentation is a very important thing in python code. Please make sure the indentation of your if-else statements in the 'marketdata_processing' view are in correct order. Beware, in your code:
return render_to_response('mytemplate.html',
{'uploadform': uploadform},
context_instance=RequestContext(request))
will always get executed as it is outside the else statement. You might want to indent it under the else (if it makes sense)- like so:
else:
uploadform = forms.MyUploadForm()
return render_to_response('mytemplate.html',
{'uploadform': uploadform},
context_instance=RequestContext(request))
Instead of 'redirect(reverse())' try 'HttpResponseRedirect()' in the processing view to call the confirm-page view, like so:
from django.http import HttpResponseRedirect
return HttpResponseRedirect('/?status=1') --> relative URL shown
place the relative/absolute url (both would work) above.
Finally, render your confirm-page template in the 'confirm_input_details' view with the context parameters, like so:
def confirm_input_details(request):
fileid = request.session['fileid']
selecteddate = request.session['selecteddate']
msg = 'Proceed to import data for %s? \nFileID Being Imported is %s ' % (selecteddate, fileid)
return render_to_response('confirmpage_template.html'),
{'fileid': fileid,
'selecteddate': selecteddate,
'msg': msg}, context_instance=RequestContext(request))
** P.S: Stick to neat basics and it will work. You happened to call your confirm_input_details view from within the same view itself. As far as I think, that would probably take you into an infinite loop. Simple concept of any view is:
take input:request, parameters
--> process logic with parameters
--> render a specific template or call another url where a template is being rendered.
You can't expect a page to show up without the rendering a template.
I am trying to move from a web page to another when the user submits a POST.
The problem is that the url doesn't change when I submit the POST and the view function that corresponds to the new page doesn't fire but the template loads and it show only the inputs I hard coded inside it without those I pass in the view function of course.
The Code:
In the urls file:
url(r'^addcross/phase1', views.new_cross_phase_1, name='new_cross_phase_1'),
url(r'^addcross/phase2', views.new_cross_phase_2, name='new_cross_phase_2'),
The view function of the 1st page:
def new_cross_phase_1(request):
if request.method == 'POST':
# my code here
return render_to_response('New/New_Cross_Phase_2.html', {'crosses_names': crosses_names, 'creation_methods' : creation_methods},
context_instance=RequestContext(request))
The view function of the 2nd page:
def new_cross_phase_2(request):
print('Phase 2')
if request.method == "GET":
return render_to_response('New/New_Cross_Phase_2.html', {'cross_form': new_cross_form},
context_instance=RequestContext(request))
You should be redirecting in view 1, not rendering the template from view 2.
return redirect('new_cross_phase_2')