I have a django project deployed in Heroku. It uses python-instragram.
I have a 'client secret' from an instragram client that I have.
I use git/github for version control.
This client_secret is imported from an untracked file because I don't want to have it on my public github repo. I do something like this:
from core_keys import core_client_secret
CONFIG = {
'client_id': '83d1b794dfc24f5588378f88be67c586',
'client_secret': core_client_secret,
'redirect_uri': 'http://localhost:8515/oauth_callback'
}
api = client.InstagramAPI(**CONFIG)
I have core_keys.py added to .gitignore:
*/core_keys.py
When I deploy to heroku the app doesn't work obviously because the file that contains the client_secret was not pushed to heroku since it's in .gitignore.
How can I have this file on heroku without the need for a private repo, what approach should I use?
You should store the secrets as config vars in the environment.
Just as a reference, ended up doing this:
On the terminal at my development machine:
heroku config:set INSTAGRAMSECRET=00000FFFFF
On the file where I need the environment var inside of Heroku:
import os
insta_secret = os.environ['INSTAGRAMSECRET']
Related
I have a Flask JSON API application that would read a file in the code repository and display the data in that file. My application file is as follows:
from flask import Flask, jsonify
from utils import load_data
data = load_data(os.environ.get('DATA_FILE_PATH')) # This would be in data/data.txt for example
app = Flask(__name__)
#app.route('/', methods=['GET'])
def home():
return jsonify(data=data)
When I deploy to Heroku, how can I set the environment variable to point to that particular data file? Locally, I would do
$ export DATA_FILE_PATH=/path/to/repo/data/data.txt
I know I can use the Heroku CLI heroku config:set DATA_FILE_PATH=<path> but I'm not sure how to place that path variable since I don't really know the directory structure on the Heroku deployment.
You can add them in the settings tab config vars
Okay, turns out you can go into the Heroku Dyno's bash using the following command (assuming you've already pushed to Heroku):
$ heroku run bash
and figure out what the file system structure looks like. In my particular case, my repo was served under the /app directory. So all I needed to pass into the value of the DATA_FILE_PATH environment variable is /app/data/data.txt.
I am using the Visual Studio 2019 Flask Web Project template. It runs in my local Python environment and is a good start (very much like the standard Asp.net) template. I created a Python Elastic Beanstalk Application to host this. I am attempting to deploy this on AWS Elastic Beanstalk. I created a repository at: https://github.com/jlongo62/Flask-Web-Project
I think one of these files (or a missing file) needs to be named
application.py(it may also need some special content). I included a
directory listing.
It appears that I need to zip this and upload it
through the portal/cli. I am not sure what zip should look like, but
I suspect requirements.txt needs to be at the root.). If AWS Toolkit
Extension cannot handle this job, it should be easy to script.
Is there a better template or sample project in GitHub ?
Is the fix something simple ?
\FlaskWebProject1\FlaskWebProject1
\FlaskWebProject1\FlaskWebProject1.pyproj
\FlaskWebProject1\FlaskWebProject1.pyproj.user
\FlaskWebProject1\requirements.txt
\FlaskWebProject1\runserver.py
\FlaskWebProject1\FlaskWebProject1\static
\FlaskWebProject1\FlaskWebProject1\templates
\FlaskWebProject1\FlaskWebProject1\views.py
\FlaskWebProject1\FlaskWebProject1\__init__.py
\FlaskWebProject1\FlaskWebProject1\__pycache__
\FlaskWebProject1\FlaskWebProject1\static\content
\FlaskWebProject1\FlaskWebProject1\static\fonts
\FlaskWebProject1\FlaskWebProject1\static\scripts
\FlaskWebProject1\FlaskWebProject1\static\content\bootstrap.css
\FlaskWebProject1\FlaskWebProject1\static\content\bootstrap.min.css
\FlaskWebProject1\FlaskWebProject1\static\content\site.css
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.eot
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.svg
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.ttf
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.woff
\FlaskWebProject1\FlaskWebProject1\static\scripts\bootstrap.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\bootstrap.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.intellisense.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.min.map
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate-vsdoc.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.unobtrusive.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.unobtrusive.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\modernizr-2.6.2.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\respond.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\respond.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\_references.js
\FlaskWebProject1\FlaskWebProject1\templates\about.html
\FlaskWebProject1\FlaskWebProject1\templates\contact.html
\FlaskWebProject1\FlaskWebProject1\templates\index.html
\FlaskWebProject1\FlaskWebProject1\templates\layout.html
\FlaskWebProject1\FlaskWebProject1\__pycache__\views.cpython-37.pyc
\FlaskWebProject1\FlaskWebProject1\__pycache__\__init__.cpython-37.pyc
Your application works on Python 3.7 running on 64bit Amazon Linux 2/3.0.3 EB environment.
Just change runserver.py into application.py. Also you can slightly modify (port and name) its content, unless you want to customize EB environment to match your application' settings.
application.py
"""
This script runs the FlaskWebProject1 application using a development server.
"""
from os import environ
from FlaskWebProject1 import app as application
if __name__ == '__main__':
HOST = environ.get('SERVER_HOST', 'localhost')
try:
PORT = int(environ.get('SERVER_PORT', '8000'))
except ValueError:
PORT = 8000
application.run(HOST, PORT)
Zip it into app.zip (example name) with content:
application.py
FlaskWebProject1
requirements.txt
Deploy the app.zip.
With thanks to Marcin.
AWS Toolkit for Visual Studio will not support deployment or environment creation. It only applies to .Net environments.
I have updated https://github.com/jlongo62/Flask-Web-Project to reflect these notes
Create AWS Elastic Beanstalk Environment. Use the portal or CLI.
Environments > Create environment (Web server environment) > Select
Settings:
Platform: Python
Platform Branch: Python 3.7
Platform Version: 3.0.3
Add application.py with content (the port 8000 seems to be a key ingredient):
"""
This script runs the FlaskWebProject1 application using a development server.
"""
from os import environ
from FlaskWebProject1 import app as application
if __name__ == '__main__':
HOST = environ.get('SERVER_HOST', 'localhost')
try:
PORT = int(environ.get('SERVER_PORT', '8000'))
except ValueError:
PORT = 8000
application.run(HOST, PORT)
Create a zip with the following root directory. Zip file name does not matter:
application.py
FlaskWebProject1
requirements.txt
Deploy zip using the portal (or cli)
Elastic Beanstalk > Environments > Flaskwebproject1-env
click Upload and deploy
I have a Google App Engine Python project that I haven't touched since 2014, when you worked with a desktop app to upload and deploy it. Google App engine changed a lot in the meantime. According to the information the deploying guide I copy
the file(s) like this:
gsutil cp main.py gs://myproject.appspot.com/
I converted my app.yaml file to a json file using their utility. When I initially deployed the app using this page it failed because there was no deployment information, so I added this:
"deployment": {
"files": {
"main": {
"sourceUrl": "https://storage.googleapis.com/myproject/main.py"
},
}
},
However when I deployed again I got an error because it couldn't find the file with that URL. The gsutil has to use a bucket with gs://something and the file has to exist at https:// something. How can I upload the file to the project and specify the location in the app.json file?
You were probably using what is now called App Engine Standard environment. Deploys are handled with the gcloud app deploy command.
Example:
gcloud app deploy --version [YOUR_VERSION_ID] --no-promote --project [YOUR_PROJECT_ID]
Here is more info for Deploying a Python App
The deploy command uses app.yaml file for configuration info.
After successfully deploying a test app using the steps outlined here:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_Python_flask.html
I tried to deploy my actual flask application which has the following structure:
myApp/
runServer.py
requirements.txt
myApp/
__init__.py
helpers.py
clean.sh
static/
myApp.css
handlers/
__init__.py
views.py
templates/
layout.html
viewOne.html
viewTwo.html
Where views.py contains my url mappings.
I have tried initializing the eb instance in the root directory as well as within the myApp module and git aws.push but I get the following error on the AWS dashboard:
ERROR Your WSGIPath refers to a file that does not exist. and the application does not work (404 for any path).
How can I deploy the above Flask application to elastic beanstalk?
I encountered a similar problem deploying a Flask application to EB, with a similar directory structure, and had to do 2 things:
Update my manage.py to create an object of name application, not app
import os
from application import create_app, db
from flask.ext.script import Manager, Shell
application = create_app(os.getenv('FLASK_CONFIG') or 'default')
manager = Manager(application)
Create .ebextensions/myapp.config, and define the following block to point to manage.py
option_settings:
"aws:elasticbeanstalk:container:python":
WSGIPath: manage.py
"aws:elasticbeanstalk:container:python:staticfiles":
"/static/": "application/static/"
This let Elastic Beanstalk find the application callable correctly.
This is described briefly at the official docs, and is described in more detail in this blog post
EDIT - see project structure below
ProjectRoot
.ebextensions
application.config
application
main
forms.py
views.py
static
templates
tests
manage.py
requirements.txt
config.py
etc, etc
Add the following to .ebextensions/<env-name>.config:
option_settings:
"aws:elasticbeanstalk:container:python":
WSGIPath: myApp/handlers/views.py
Update:
If you don't have .ebextensions directory, please create one for the project. You can find more information of what can be done regarding the container configuration in Customizing and Configuring AWS Elastic Beanstalk Environments guide.
Your WSGIPath refers to a file that does not exist.
This error appears because Beanstalk, by default, looks for application.py. Check at Beanstalk web UI, Configuration > Software Configuration, WSGIPath is mapped to application.py
Update the WSGIPath as shown in the previous replies or rename to application.py file.
As of awsebcli 3.0, you can actually edit your configuration settings to represent your WSGI path via eb config. The config command will then pull (and open it in your default command line text editor, i.e nano) an editable config based on your current configuration settings. You'll then search for WSGI and update it's path that way. After saving the file and exiting, your WSGI path will be updated automatically.
WSGI configuration was painful for me. I did changed WSCI settings using eb config command but it did not work. Below you can fix this in 5 easy steps.
1- Moved app.py function to the root of the directory (where I runned eb init command.
2- Also renamed app.py as application.py and in that initilized application as application = Flask(__name__) not app = Flask(__name__)
3- eb deploy did not worked after this (in the same project) I tried to fix config by using eb config but it was too hairy to sort it out. Delete all .extensions, .gitignore etc from your project.
4- re initialize your project on EB with eb init and follow the prompts. when deployment is done, eb open would launch your webapp (hopefully!)
When I encountered this problem it was because I was using the GUI to upload a zip of my project files. Initially I was zipping the project level directory and uploading that zip to EB.
Then I switched to simply uploading a zip of the project files themselves-ie select all files and send those to a zip-and then the GUI upload utility was able to find my application.py file without a problem because the application.py file was not in a subfolder.
Well, In my case I followed the entire process and conventions but was still getting 404. The problem was my virtual environment. I was ignoring all environment config related folders/files in my .gitignore but not in .ebignore. After creating .ebignore and ignoring all the folders/files which were not related to project code, fixed the issue.
I have my neo4j python application using py2neo.
It is working corretly on local but when i deploy it to heroku it is giving error
py2neo.rest.SocketError
SocketError: gaierror(-2, 'Name or service not known')
I am not sure what is the issue.
what should be the correct issue I am using url for db service as
graph_db = neo4j.GraphDatabaseService("http://xyz.hosted.neo4j.org:7480/db/data/")
Configure the authentication as indicated here:
http://packages.python.org/py2neo/neo4j.html#authentication
Use heroku config to note the username and password in the URL.
For example (obviously modified):
$ heroku config
=== secure-caverns-9214 Config Vars
NEO4J_URL: http://username:password#17d0afad4.hosted.neo4j.org:7755
PATH: bin:node_modules/.bin:/usr/local/bin:/usr/bin:/bin