I asked something similar to this question and I haven't gotten any responses that help. So, I have decided to simplify things as much as I can with the following:
I have developed a python flask application and deployed to a beanstalk worker tier python environment. The issue is I can't figure out how to print or log or write anything anywhere. I need to debug this application and the only way I know how to do that is by printing to either the console or a log file to see exactly what is going on. When I run the application locally I can print to the console, write to files, and log with zero problems, it is just when I deploy it to the beanstalk environment that nothing happens. I have SSHed into the ec2 instance where I have application deployed and searched practically every file and I find that nothing was written by my python script anywhere.
This question probably seems absolutely stupid but can someone please provide me with an example of a python flask application that will run on a beanstalk worker environment that just prints "Hello World" to some file that I can find on the ec2 instance? Please include what should be written the requirements.txt file and any *.config files in the .ebextensions folder.
Thank You
Here is another simple python app that you can try. The one in the blog will work as well but this shows a minimal example of an app that prints messages received from SQS to a file on the EC2 instance.
Your app source folder should have the following files:
application.py
import os
import time
import flask
import json
application = flask.Flask(__name__)
start_time = time.time()
counter_file = '/tmp/worker_role.tmp'
#application.route('/', methods=['GET', 'POST'])
def hello_world():
if flask.request.method == 'POST':
with open(counter_file, 'a') as f:
f.write(flask.request.data + "\n")
return flask.Response(status=200)
if __name__ == '__main__':
application.run(host='0.0.0.0', debug=True)
requirements.txt
Flask==0.9
Werkzeug==0.8.3
.ebextensions/01-login.config
option_settings:
- namespace: aws:autoscaling:launchconfiguration
option_name: EC2KeyName
value: your-key-name
Launch a worker tier 1.1 environment with a Python 2.7 solution stack. I tested with (64bit Amazon Linux 2014.03 v1.0.4 running Python 2.7).
Wait for the environment to go green. After it goes green click on the queue URL as visible in the console. This will take you to the SQS console page. Right click on the queue and click on "Send a message". Then type the following message: {"hello" : "world"}.
SSH to the EC2 instance and open the file /tmp/worker_role.tmp. You should be able to see your message in this file.
Make sure you have IAM policies correctly configured for using Worker Role environments.
For more information on IAM policies refer this answer: https://stackoverflow.com/a/23942498/161628
There is a python+flask on beanstalk example on AWS Application Management Blog:
http://blogs.aws.amazon.com/application-management/post/Tx1Y8QSQRL1KQZC/Elastic-Beanstalk-Video-Tutorial-Worker-Tier
http://blogs.aws.amazon.com/application-management/post/Tx36JL4GPZR4G98/A-Sample-App-For-Startups
For the logging issues, i'd suggest:
Check your /var/log/eb-cfn-init.log (and other log files in this directory), if a .config command is failing you will see which and why there.
In your .config commands, output messages to a different log file so you see exactly where your bootstrap failed in your own file.
Add you application log file to EB Log Snapshots (/opt/elasticbeanstalk/tasks/taillogs.d/) and EB S3 log rotation (/opt/elasticbeanstalk/tasks/publishlogs.d/). See other files in these directories for examples.
Related
I have the following:
#! /usr/bin/python3.7
f=open("python_out.txt",'w',encoding='utf-8')
f.write("OK1")
import socket
import telegram
f.write("OK2")
BOT_TOKEN = "telegram BOT_TOKEN"
CHAT_ID = "chat_id"
bot = telegram.Bot(token=BOT_TOKEN)
host_name = socket.gethostname()
content = 'Machine name: %s is shutting down!' % host_name
bot.send_message(chat_id=CHAT_ID, text=content)
f.write("OK3")
I have checked my environment, I can make this script work through python3 script.py when it is in the instance,It can send notifications and output python_out.txt.
I set this script in shutdown-script
But when I manually clicked the "stop" button, it did not work as expected. startup-script too.
I have read many posts:
Shutdown script not executing on a Google Cloud VM
Reliably executing shutdown scripts in Google Compute Engine
Pro Tip: Use Shutdown Script Detect Preemption on GCP
Of course it also includes official documents:
https://cloud.google.com/compute/docs/shutdownscript
I want to try setting powerbtn.sh，but i can't find /etc/acpi/ in GCP Ubuntu 16.04 LTS
I can't find any more schedule, any ideas?
When you use startup script and shutdown script, the current user that execute is the root user, and the default directory /root/. This directory isn't writable, that's why nothing happens with your code.
Simply write files in writable directory and that's all.
Don't forget that the files that you create are written by the root user and all user can't read and/or write on file wrote by root. Use chmod or chown to change this.
I am using python version 3.8 and IIS 7.0. When I try to host my python web api on the IIS server it encounter with the FastCGI error. I have enable CGI in IIS and also added System.WebServer>>handlers>>Python FastCGI in my web config but still it gives same error. I have also checked the wfastcgi and flask are also successfully added.
You had to put both parameters in double quote separated by |
e.g.
"c:\python39\python.exe" | "c:\python39\Lib\site-packages\wfastcgi.py"
or put the whole path in "" after pasting it.
follow below steps to configure iis flask app in iis:
1)install python
2)after installing python install the wfastcgi. run the command prompt as administrator and run below command:
pip install wfastcgi
wfastcgi-enable
3)below is my flask example:
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "Hello from FastCGI via IIS!"
if __name__ == "__main__":
app.run()
4)enable the cgi feature of iis:
5)open iis create a site.
6)after adding site select the site name and select the handler mapping feature from the middle pane.
Click “Add Module Mapping”
executable path value:
C:\Python37-32\python.exe|C:\Python37-32\Lib\site-packages\wfastcgi.py
Click “Request Restrictions”. Make sure “Invoke handler only if the request is mapped to:” checkbox is unchecked:
Click “Yes” here:
7)now go back and select the application setting feature.
click add from the action pane.
Set the PYTHONPATH variable(which is your site folder path):
And the WSGI_HANDLER (my Flask app is named app.py so the value is app.app — if yours is named site.py it would be site.app or similar):
8)Click OK and browse to your site.
Note: Do not forget to assign the iis_iusrs and iusr permission to the site folder and the python folder.
I am using the Visual Studio 2019 Flask Web Project template. It runs in my local Python environment and is a good start (very much like the standard Asp.net) template. I created a Python Elastic Beanstalk Application to host this. I am attempting to deploy this on AWS Elastic Beanstalk. I created a repository at: https://github.com/jlongo62/Flask-Web-Project
I think one of these files (or a missing file) needs to be named
application.py(it may also need some special content). I included a
directory listing.
It appears that I need to zip this and upload it
through the portal/cli. I am not sure what zip should look like, but
I suspect requirements.txt needs to be at the root.). If AWS Toolkit
Extension cannot handle this job, it should be easy to script.
Is there a better template or sample project in GitHub ?
Is the fix something simple ?
\FlaskWebProject1\FlaskWebProject1
\FlaskWebProject1\FlaskWebProject1.pyproj
\FlaskWebProject1\FlaskWebProject1.pyproj.user
\FlaskWebProject1\requirements.txt
\FlaskWebProject1\runserver.py
\FlaskWebProject1\FlaskWebProject1\static
\FlaskWebProject1\FlaskWebProject1\templates
\FlaskWebProject1\FlaskWebProject1\views.py
\FlaskWebProject1\FlaskWebProject1\__init__.py
\FlaskWebProject1\FlaskWebProject1\__pycache__
\FlaskWebProject1\FlaskWebProject1\static\content
\FlaskWebProject1\FlaskWebProject1\static\fonts
\FlaskWebProject1\FlaskWebProject1\static\scripts
\FlaskWebProject1\FlaskWebProject1\static\content\bootstrap.css
\FlaskWebProject1\FlaskWebProject1\static\content\bootstrap.min.css
\FlaskWebProject1\FlaskWebProject1\static\content\site.css
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.eot
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.svg
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.ttf
\FlaskWebProject1\FlaskWebProject1\static\fonts\glyphicons-halflings-regular.woff
\FlaskWebProject1\FlaskWebProject1\static\scripts\bootstrap.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\bootstrap.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.intellisense.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery-1.10.2.min.map
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate-vsdoc.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.unobtrusive.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\jquery.validate.unobtrusive.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\modernizr-2.6.2.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\respond.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\respond.min.js
\FlaskWebProject1\FlaskWebProject1\static\scripts\_references.js
\FlaskWebProject1\FlaskWebProject1\templates\about.html
\FlaskWebProject1\FlaskWebProject1\templates\contact.html
\FlaskWebProject1\FlaskWebProject1\templates\index.html
\FlaskWebProject1\FlaskWebProject1\templates\layout.html
\FlaskWebProject1\FlaskWebProject1\__pycache__\views.cpython-37.pyc
\FlaskWebProject1\FlaskWebProject1\__pycache__\__init__.cpython-37.pyc
Your application works on Python 3.7 running on 64bit Amazon Linux 2/3.0.3 EB environment.
Just change runserver.py into application.py. Also you can slightly modify (port and name) its content, unless you want to customize EB environment to match your application' settings.
application.py
"""
This script runs the FlaskWebProject1 application using a development server.
"""
from os import environ
from FlaskWebProject1 import app as application
if __name__ == '__main__':
HOST = environ.get('SERVER_HOST', 'localhost')
try:
PORT = int(environ.get('SERVER_PORT', '8000'))
except ValueError:
PORT = 8000
application.run(HOST, PORT)
Zip it into app.zip (example name) with content:
application.py
FlaskWebProject1
requirements.txt
Deploy the app.zip.
With thanks to Marcin.
AWS Toolkit for Visual Studio will not support deployment or environment creation. It only applies to .Net environments.
I have updated https://github.com/jlongo62/Flask-Web-Project to reflect these notes
Create AWS Elastic Beanstalk Environment. Use the portal or CLI.
Environments > Create environment (Web server environment) > Select
Settings:
Platform: Python
Platform Branch: Python 3.7
Platform Version: 3.0.3
Add application.py with content (the port 8000 seems to be a key ingredient):
"""
This script runs the FlaskWebProject1 application using a development server.
"""
from os import environ
from FlaskWebProject1 import app as application
if __name__ == '__main__':
HOST = environ.get('SERVER_HOST', 'localhost')
try:
PORT = int(environ.get('SERVER_PORT', '8000'))
except ValueError:
PORT = 8000
application.run(HOST, PORT)
Create a zip with the following root directory. Zip file name does not matter:
application.py
FlaskWebProject1
requirements.txt
Deploy zip using the portal (or cli)
Elastic Beanstalk > Environments > Flaskwebproject1-env
click Upload and deploy
I want to deploy my flask-restx application on a shared hosting. Since I am beginner in deployment, I followed a video tutorial from youtube.
I did step by step by following this tutorial.
For those who do not want to go through the tutorial, I am writing the steps:
I created an application from the Python cPanel
Initial set up in Cpanel
Then I opened terminal and changed my venv and installed flask by "pip install flask"
Project Structure
filas_folder/
├──public
├──tmp
│ └──restart.txt
├──app.py
└──passenger_wsgi.py
app.py looks like
from flask import Flask
app = Flask(__name__)
#app.route("/")
def main_():
return "flask is running"
#app.route("/user")
def main_2():
return "user is running"
if __name__ == "__main__": app.run()
Restart app from cpanel
passenger.py looks like
import imp
import os
import sys
sys.path.insert(0, os.path.dirname(__file__))
wsgi = imp.load_source('wsgi', 'app.py')
application = wsgi.app
when I open www.example.com
flask is running
But when I open www.example.com/user
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator at webmaster#example.com to inform them of the time this error occurred, and the actions you performed just before this error.
More information about this error may be available in the server error log.
Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument to handle the request.
My system has cloudlinux and uses apache server. This is not the first deployment. Many wordpress and static websites are running on the server.
I opened apache logs at /usr/local/apache/logs/error_log
I get the error "Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer" http://example.com/user"
Add the following to the top of your .htaccess file:
RewriteEngine on
RewriteRule ^http://%{HTTP_HOST}%{REQUEST_URI} [END,NE]
Got this info from: https://stackoverflow.com/a/63971427/10122266
I'm attempting to debug a Flask/Python app running on Google Appengine flexible environment.
However, I see a warning message within the Stackdriver Debug interface in Google Console, and am unable to set any breakpoints.
The warning reads:
Stackdriver Debugger is not set up for the python runtime on GAE Flex
Screenshot of warning
Any thoughts on what I'm doing wrong?
I've:
Enabled the Stackdriver Debugger API (as mentioned here)
Imported and initialised the Debugger (following the instructions here)
Included google-python-cloud-debugger in requirements.txt
main.py (the app entry point defined in app.yaml)
from werkzeug.serving import run_simple
from werkzeug.wsgi import DispatcherMiddleware
from wsgi import api, frontend, manage
try:
import googleclouddebugger
googleclouddebugger.AttachDebugger()
except ImportError:
pass
app = DispatcherMiddleware(frontend.create_app(), {
'/api': api.create_app(),
'/manage': manage.create_app()
})
if __name__ == '__main__':
run_simple('0.0.0.0', 5000, app, use_reloader=True, use_debugger=True)
app.yaml
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT main:app
runtime_config:
python_version: 2
manual_scaling:
instances: 1
resources:
cpu: 1
memory_gb: 0.5
disk_size_gb: 10
env_variables:
SQLALCHEMY_DATABASE_URI: "postgresql+psycopg2://myusername:mypassword!#/instancename?host=/cloudsql/instancename"
beta_settings:
cloud_sql_instances: "instancename"
Update 1
After a comment and noticing a urllib import error, I wondered if the wsgi nature of my application was causing issues. I went back to the documentation, saw a note about Django framework doing something similar and changed the following:
googleclouddebugger.AttachDebugger()
to
googleclouddebugger.enable()
This got rid of the urllib import error, but hasn't resolved the overall issue.
I don't see anything wrong with this configuration, but here are some suggestions:
Once you open the GCP Console Debug page, please make sure you selected the correct version of the app from the top dropdown menu before starting the debugging session. It may be pointing to an old app version that is not debuggable, or the page and the dropdown might need a refresh to display the latest version.
Similarly, please check if your Stackdriver stderr logs contains a line that like this:
2017-12-24 15:14:14.000 PST I1224 23:14:14.600462 12 gcp_hub_client.py:335] Debuggee registered successfully, ID: gcp:1025611681465:7144dac417e43025
This means the debugger is actually setup/working properly, and most likely indicates that the UI is requires a refresh or selecting the right app version from the dropdown.
Make sure you have the google-python-cloud-debugger in the requirements.txt file.
Consider adding a simple print statement in the try/except block for the debugger, as folllows:
...
except ImportError as e:
print 'Failed to import Google Cloud Debugger: %s' % str(e)
pass
Then, check the stderr logs on Stackdriver Logging to see if this printed any exceptions.
I don't know about werkzeug, but you might want to try disabling the default debugger there by passing use_debugger=False, just to avoid possible conflicts.
Try deploying a simpler flask app (e.g., start with the hello world for Flex from here but modify it to use python_version: 2) and see if the debugger works for you there.