Has anyone had experience deploying a Django app on Heroku that contains a script within it?
I'm currently building a web app that works as a front end tool to a couple of python bots that I have been creating during the years, as well as for personal useful tools (Scripts). The issue is that I've been trying to deploy my app on Heroku and test the functionality of running my Tinder Bot from within Heroku, but I have not been successful, i'm currently getting errors such as:
"unknown error: Chrome failed to start: exited abnormally (unknown error: DevToolsActivePort file doesn't exist"
Thing's that I've done to try to solve the issue:
1- Setting the chrome arguments as:
chrome_settings = Options()
chrome_settings.add_argument('--headless')
chrome_settings.add_argument('--no-sandbox')
chrome_settings.add_argument('--disable-dev-shm-usage')
As well as setting the directories for chrome binary and chromedriver to:
GOOGLE_CHROME_BIN = '/app/.apt/usr/bin/google-chrome'
path_of_chrome_driver = '/app/.chromedriver/bin/chromedriver'
But I am still getting the same error. Would anyone know whats the proper procedure to create a web app that can contain python scripts in it, and to be deployed on Heroku?
Ps: I have followed the steps to set up (Whitenoise and gunicorn) for heroku deployment.
Requirements:
Django==2.1.1
gunicorn==19.9.0
pytz==2018.5
selenium==3.14.1
urllib3==1.23
whitenoise==4.1
This is the site I am developing: https://bot-tools-collection.herokuapp.com/
Update
Link to the code:
https://gist.github.com/keithlowc/d0b274005ecf9d41b4f087620b487dc5
Related
Novice here.
Trying to deploy my Django app to Heroku and I get an Unable to connect from the Firefox browser (I tried Chrome as well with similar results). I'm having a lot of trouble troubleshooting where exactly the problem is? Heroku local? Gunicorn? Browser Firewall? Django Settings?
What I have done.
Added the django_heroku import to settings.py
Tried to ensure My browser(Firefox) is not blocking pop-ups etc.
Firefox -> settings-> Privacy and security -> permissions-> Uncheck Block pop-up windows
Ensured my app works on with just gunicorn
I run gunicorn project3.wsgi in my terminal (project3 is the directory for my wsgi file) which works
I have the same command in my Procfile, uppercase 'P' (web: gunicorn project3.wsgi).
However when I run the command 'heroku local' The browser opens up on clicking the link, but I get an error. Any assistance would be appreciated.
I am running commands using windows WSL
Even though 0.0.0.0 is valid IP, it is not usable as a real IP address.
You can use either http://localhost:5000 or http://127.0.0.1:5000/
In the Internet Protocol Version 4, the address 0.0.0.0 is a
non-routable meta-address used to designate an invalid, unknown or
non-applicable target. This address is assigned specific meanings in a
number of contexts, such as on clients or on servers.
You can read more about 0.0.0.0 here
I'm attempting to search docker hub for images using the command "docker search", via python code.
here is the piece of code for this action:
client = docker.from_env()
image_list = client.images.search('ubuntu')
running this in the command line works, and also running it via flask locally works.
however, when deploying to heroku, this crashes the site, and the logs say the following:
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
which basically tells me docker isn't installed on the machine running docker.
I found this question:
Running Docker inside a Python app on Heroku?
but it wasn't answered and basically describes what I need.
any idea what I can do in this situation? thanks!
I have using the subprocess package in my django web, when i run local on server or publish using apache, it work without causeing any issue BUT when I host my django web on IIS, this subprocess does not work.
Below is my subprocess code:
files = subprocess.check_output("dir /b " + path, shell=True).decode()
p_pcat=subprocess.Popen(['java', '-cp', str(PARSER_JAR), 'parsePCAT.ParsePCAT', str(pcat_file_name)],stdout=subprocess.PIPE, stderr=subprocess.PIPE,shell=True)
This both function not working when hosting at IIS (version 10.0.14393.0), anyone have idea on this?
Try to assign the administrator user to the iis application pool identity by using an advanced setting.
after assigning administrator user to recycle the application pool and try to access the site again.
if you still face an issue after doing changes enable detailed logging in iis:
1)select site from iis.
2)click error pages.
3)click edit feature setting from the action pane.
access site and check the error.
you could also log error by adding below variable in iis app setting:
https://learn.microsoft.com/en-us/visualstudio/python/configure-web-apps-for-iis-windows?view=vs-2019
I have a node app running on Heroku. I am scraping a website using selenium in python and calling the python script from my node app whenever I need to. I installed PhantomJS on my mac and when I run the app locally (node index.js), everything works just fine.
path_to_phantom = '/Users/govind/Desktop/phantomjs-2.1.1-
macosx/bin/phantomjs'
browser = webdriver.PhantomJS(executable_path = path_to_phantom)
However, nothing seems to work on Heroku. I also added the PhantomJS buildpack to my node app but it just doesn't call the python script. The problem I think is the path to PhantomJS buildpack. What path should I add? Or is there any other aspect I'm missing here?
I managed to use Selenium with PhantomJS in my Python application deployed to Heroku following these steps:
1) Switch to using the Cedar-14 stack on my Heroku application
$ heroku stack:set cedar-14
2) Install a PhantomJS buildpack
$ heroku buildpacks:add https://github.com/stomita/heroku-buildpack-phantomjs
With these changes I could then use Selenium to fetch websites
from selenium import webdriver
browser = webdriver.PhantomJS()
browser.get("http://www.google.com") # This does not throw an exception if it got a 404
html = browser.page_source
print html # If this outputs more than just '<html><head></head><body></body></html>' you know that it worked
hey guys i tried to deploy my scrapy project onto Heroku and everything was deployed. But when i try to execute heroku run scrapy crawl crawlername i get the error
RuntimeError: Could not find firefox in your system PATH. Please specify the firefox binary location or install firefox. i have added these lines to /etc/paths.
/Applications
/Applications/Firefox.app
/Applications/Firefox.app/Contents/MacOS/firefox-bin
/Applications/Firefox.app/Contents/MacOS/firefox
After adding them to etc/paths i am able to run firefox from the terminal by just typing firefox and firefox will open. But when i try to rerun heroku run scrapy crawl crawlername i get the same error.
self.driver = webdriver.Firefox()
self.driver.implicitly_wait(10)
thats inside my scrapy script to open the web browser.
if im in the same directory that i have created the heroku app and i run scrapy crawl crawlername everything works fine and i get my results but when i try to do heroku run scrapy crawl crawlername i get the error regarding firefox shown above. Any help will be greatly appreciated thanks.
As heinst mentions Firefox isn't installed on Heroku. This means that you have to have Heroku not on your local machine but on the Heroku instance which is running your script.
The lines you've added to /etc/paths are of your Mac -- not the Heroku instance you are trying to run your application on.
Alternatively try to use something like this buildpack. This requires some additional tools like Xvbf to have your Firefox an in-memory buffer to let Selenium click the right buttons and so on.
Alternatively: why not using Scrapy without Selenium? Most of the tasks can be done without any browser-interaction but you have to search of course a bit more to find your solution.