We Keep Coding

Sending emails from Google Appengine fails because of missing api proxy

I am trying to send e-mails from a GAE application using this code:
from google.appengine.api.mail import send_mail
send_mail(
"sender#nowhere.com",
["user#example.com"],
"Subject",
"Body",
)
I have configured usage of the apis in app.yaml with:
app_engine_apis: true
And deploy to App Engine is done with gcloud beta app deploy.
However, I get this error:
Traceback (most recent call last):
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 2073, in wsgi_app response = self.full_dispatch_request()
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1518, in full_dispatch_request rv = self.handle_user_exception(e)
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1516, in full_dispatch_request rv = self.dispatch_request()
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1502, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File "/srv/infrastructure/view_modifiers.py", line 12, in view_method response_val = f(*args, **kwargs)
File "/srv/views/orders.py", line 25, in create_order vm.create_order()
File "/srv/viewmodels/orders/order_viewmodel.py", line 74, in create_order self._send_order_email()
File "/srv/viewmodels/orders/order_viewmodel.py", line 54, in _send_order_email send_mail(
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/api/mail.py", line 401, in send_mail message.send(make_sync_call=make_sync_call)
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/api/mail.py", line 1209, in send make_sync_call('mail', self._API_CALL, message, response)
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/api/apiproxy_stub_map.py", line 96, in MakeSyncCall return stubmap.MakeSyncCall(service, call, request, response)
File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/api/apiproxy_stub_map.py", line 348, in MakeSyncCall assert stub, 'No api proxy found for service "%s"' % service AssertionError: No api proxy found for service "mail"
This seems to suggest that even for the default behavior of the mail service, some kind of proxy needs to be configured. However, I cannot find any information about the setup of this proxy.
And, my initial understanding was that setting up a proxy is only needed for unit-testing or local development.

Here is an example that works with fastapi. (answer to anton's comment)
from google.appengine.api import mail
from fastapi.middleware.wsgi import WSGIMiddleware
from google.appengine.api import wrap_wsgi_app
from flask import Flask
app = create_app() # this is just app = FastAPI() and somes middleware but but no relevance here
app_flask = Flask(__name__)
app_flask.wsgi_app = wrap_wsgi_app(app_flask.wsgi_app, use_deferred=True)
def send_approved_mail(sender_address):
# [START send_message]
message = mail.EmailMessage(
sender=sender_address,
subject="Your account has been approved")
message.to = "test#gmail.com"
message.body = """Dear Albert:
Your example.com account has been approved. You can now visit
http://www.example.com/ and sign in using your Google Account to
access new features.
Please let us know if you have any questions.
The example.com Team
"""
message.send()
#app_flask.route("/send_email", methods=['GET'])
def send_email():
send_approved_mail("registred_email#domain.com")
return "message sent"
app.mount("/v1", WSGIMiddleware(app_flask))
here app.yaml
runtime: python39
entrypoint: gunicorn -k uvicorn.workers.UvicornWorker app.main:app
instance_class: F1
app_engine_apis: true
inbound_services:
- mail
- mail_bounce
You will not see flask route on swagger.
and for some reason, i could not sent email with app engine's default service account. I had to register my email ( with same domain) here : https://console.cloud.google.com/appengine/settings/emailsenders?project=your_project

I just tested this and it worked for me (i.e. I received the email I supplied for recipient_email_address in the code below). Note that sender must be a value specified under Email Senders in App Engine settings page
requirements.txt file
Flask
appengine-python-standard>=1.0.0
app.yaml file
runtime: python39
app_engine_apis: true
handlers:
- url: /static
static_dir: static/
- url: /.*
script: auto
main.py
from flask import Flask
from google.appengine.api import wrap_wsgi_app
from google.appengine.api.mail import send_mail
app = Flask(__name__)
app.wsgi_app = wrap_wsgi_app(app.wsgi_app)
#app.route("/")
def sendMail():
send_mail(sender= <authorized_email_sender>,
to= <recipient_email_address>,
subject="Testing Python3 sending mails",
body="""Dear Albert:
Your example.com account has been approved. You can now visit
http://www.example.com/ and sign in using your Google Account to
access new features.
Please let us know if you have any questions.
The example.com Team
""")
return "Mail was sent"

Python Flask - Session Not Being Set with Waitress

I am using waitress on IIS with Flask.
I have a route that sets a session as follows
#gp_clinicals_bp.route("/setting", methods=["POST","GET"])
def gp_clinicals():
session["gp_clinicals_meds_repeat"] = "123456"
return session["gp_clinicals_meds_repeat"]
This returns, unsurprisingly:
123456
I have another route as follows:
#gp_clinicals_bp.route("/testing", methods=["POST","GET"])
def gp_clinicals_test():
return session["gp_clinicals_meds_repeat"]
Now, I get nothing back and the following in the error log
2022-08-02 20:03:51,126 - ERROR - app.py - log_exception - 1455 - Exception on /gp-clinicals/medication-repeat [GET]
Traceback (most recent call last):
File "C:\Python310\lib\site-packages\flask\app.py", line 2077, in wsgi_app
response = self.full_dispatch_request()
File "C:\Python310\lib\site-packages\flask\app.py", line 1525, in full_dispatch_request
rv = self.handle_user_exception(e)
File "C:\Python310\lib\site-packages\flask\app.py", line 1523, in full_dispatch_request
rv = self.dispatch_request()
File "C:\Python310\lib\site-packages\flask\app.py", line 1509, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File "E:\Apps\dev\digital\gp_clinicals\gp_clinicals.py", line 158, in gp_clinicals_medication_repeat
return str(session["gp_clinicals_meds_repeat"])
KeyError: 'gp_clinicals_meds_repeat'
2022-08-02 20:03:51,128 - ERROR - __init__.py - handle_error_500 - 92 - 500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
I'm the only one on the server. Below is a snippet from my initilisation file:
"""Initialize Flask app."""
import logging
import os
from logging.handlers import RotatingFileHandler
from flask import Flask, render_template, flash,request
from flask_session import Session
from flask_assets import Environment
from config import Config
if Config.FLASK_ENV == "development" and Config.DD_SERVICE:
patch_all()
sess = Session()
def init_app():
"""Create Flask application."""
app = Flask(__name__, instance_relative_config=False)
app.config["CACHE_TYPE"] = "null"
app.config['SEND_FILE_MAX_AGE_DEFAULT'] = 0
app.config['TEMPLATES_AUTO_RELOAD'] = True
app.config["SESSION_TYPE"] = "filesystem"
app.config.from_object("config.Config")
assets = Environment()
assets.init_app(app)
sess.init_app(app)
What's going wrong? In the flask_session folder I am seeing files being created
Any advice? I tried using single apostophes instead of quotes but no go there either
The site in IIS is running through a reverse proxy in IIS - could that be doing it?

Restful API with Flask and MySQL database with Raspberry Pi

I am trying to test Restful API with Flask and MySQL database with Raspberry Pi. Found a interesting article Tutorial My Full Code is below,
from flask import Flask, request, jsonify, make_response
from flask_sqlalchemy import SQLAlchemy
from marshmallow import fields
from marshmallow_sqlalchemy import SQLAlchemySchema
import json
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:root#localhost:3306/todo'
db = SQLAlchemy(app)
# Model
class Todo(db.Model):
__tablename__ = "todos"
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(20))
todo_description = db.Column(db.String(100))
def create(self):
db.session.add(self)
db.session.commit()
return self
def __init__(self, title, todo_description):
self.title = title
self.todo_description = todo_description
def __repr__(self):
return f"{self.id}"
db.create_all()
class TodoSchema(SQLAlchemySchema):
class Meta(SQLAlchemySchema.Meta):
model = Todo
sqla_session = db.session
id = fields.Number(dump_only=True)
title = fields.String(required=True)
todo_description = fields.String(required=True)
#app.route('/api/v1/todo', methods=['POST'])
def create_todo():
data = request.get_json()
print(data)
todo_schema = TodoSchema()
todo = todo_schema.load(data)
result = todo_schema.dump(todo.create())
return make_response(jsonify({"todo": result}), 200)
# return 'This works',200
#app.route('/api/v1/todo', methods=['GET'])
def index():
get_todos = Todo.query.all()
todo_schema = TodoSchema(many=True)
todos = todo_schema.dump(get_todos)
return make_response(jsonify({"todos": todos}))
#app.route('/api/v1/todo/<id>', methods=['GET'])
def get_todo_by_id(id):
get_todo = Todo.query.get(id)
todo_schema = TodoSchema()
todo = todo_schema.dump(get_todo)
return make_response(jsonify({"todo": todo}))
But when I send Rest POST request through Postman I am getting following error,
Error:
(venv) root#raspberrypi:~/password_db_test1# flask run -h 0.0.0.0
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
/root/password_db_test1/venv/lib/python3.7/site-packages/flask_sqlalchemy/__init__.py:873: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future. Set it to True or False to suppress this warning.
'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and '
* Running on all addresses.
WARNING: This is a development server. Do not use it in a production deployment.
* Running on http://192.168.8.182:5000/ (Press CTRL+C to quit)
{'title': 'test title', 'todo_description': 'test'}
[2021-09-19 07:58:10,883] ERROR in app: Exception on /api/v1/todo [POST]
Traceback (most recent call last):
File "/root/password_db_test1/venv/lib/python3.7/site-packages/flask/app.py", line 2070, in wsgi_app
response = self.full_dispatch_request()
File "/root/password_db_test1/venv/lib/python3.7/site-packages/flask/app.py", line 1515, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/password_db_test1/venv/lib/python3.7/site-packages/flask/app.py", line 1513, in full_dispatch_request
rv = self.dispatch_request()
File "/root/password_db_test1/venv/lib/python3.7/site-packages/flask/app.py", line 1499, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File "/root/password_db_test1/app.py", line 48, in create_todo
result = todo_schema.dump(todo.create())
AttributeError: 'dict' object has no attribute 'create'
192.168.8.228 - - [19/Sep/2021 07:58:10] "POST /api/v1/todo HTTP/1.1" 500 -
I have done playing with the code and change the below
From:
result = todo_schema.dump(todo.create())
To:
result = todo_schema.dump(todo.create())
Since method is Todo
After that I am getting,
Error:
{'title': 'test title', 'todo_description': 'test'}
[2021-09-19 08:02:29,147] ERROR in app: Exception on /api/v1/todo [POST]
Traceback (most recent call last):
File "/root/password_db_test1/venv/lib/python3.7/site-packages/flask/app.py", line 2070, in wsgi_app
response = self.full_dispatch_request()
File "/root/password_db_test1/venv/lib/python3.7/site-packages/flask/app.py", line 1515, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/password_db_test1/venv/lib/python3.7/site-packages/flask/app.py", line 1513, in full_dispatch_request
rv = self.dispatch_request()
File "/root/password_db_test1/venv/lib/python3.7/site-packages/flask/app.py", line 1499, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
File "/root/password_db_test1/app.py", line 48, in create_todo
result = todo_schema.dump(Todo.create())
TypeError: create() missing 1 required positional argument: 'self'
192.168.8.228 - - [19/Sep/2021 08:02:29] "POST /api/v1/todo HTTP/1.1" 500 -
I changed some codes due to testing and changes in marshmallow_sqlalchemy import ModelSchema to SQLAlchemySchema as per the ModelSchema_Change
My Sample JSON POST Request from Postman is below
{
"title": "test title",
"todo_description": "test"
}
Please help me to sort this issue, I am doing a project to encrypted user date in mariaDB for a project using Raspberry Pi which will be this be a sample.

Unable to deploy Python Flask application in Google App Engine Flex Environment

I have searched in multiple places and have tried multiple ways to find a solution to this. But nothing was helpful, can you please help me provide the solution. It is getting extremely difficult to host a python flask application in google app engine flexible environment. Python code: "getresource.py"
Here is the link from google to deploy Python Flask application
Here is the simple code:
from flask import Flask, jsonify
from pymysql import connections, ProgrammingError, DatabaseError, MySQLError, DataError
from os import environ
DB_HOST = environ.get("DB_HOST")
USER = environ.get("USER")
PWD = environ.get("PASSWORD")
DATABASE = environ.get("DATABASE")
app = Flask(__name__)
dbconn = connections.Connection(
host=DB_HOST,
port=3306,
user=USER,
password=PWD,
db=DATABASE
)
#app.route("/getResource/<id>", methods=['GET'])
def getresource(id):
result = ()
select_sql = "SELECT `id`, `firstName`, `middleName`, `lastName`, `listOfTechWorkedOn`, `certifications`, `projects`, `applicationWorkLoadTypes` FROM `resource` WHERE `id`=%s"
cursor = dbconn.cursor()
try:
cursor.execute(select_sql, (id))
result = cursor.fetchone()
response = {}
response['id'] = result[0]
response['firstName'] = result[1]
response['middleName'] = result[2]
response['lastName'] = result[3]
response['listOfTechWorkedOn'] = result[4]
response['certifications'] = result[5]
response['projects'] = result[6]
response['applicationWorkLoadTypes'] = result[7]
return jsonify(response)
except ProgrammingError as p:
return
except DatabaseError as d:
return d
except MySQLError as m:
return m
except DataError as de:
return de
dbconn.close()
if __name__ == '__main__':
app.run(host='127.0.0.1',port=8080, debug=True)
Here is the app.yaml file
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT getresource:app
service: getresource
runtime_config:
python_version: 3
manual_scaling:
instances: 1
env_variables:
DB_HOST: "1.2.3.4"
USER: "root"
PASSWORD: "abcdefg"
DATABASE: "abcd"
requirements.txt file
Flask==1.0.2
gunicorn==19.9.0
PyMySQL==0.9.2
I am getting the following error: "gcloud app deploy"
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred during deployment.
I have tried multiple ways, like changing the python code, putting google cloud libraries in the requirements.txt, chaning port, service name etc. But nothing seems to be working.
And surprisingly, I am not getting the error trace in stackdriver logs as well. And Google does not have proper answer it seems (at least from all the googling that i have done)
gcloud version
Google Cloud SDK 222.0.0
bq 2.0.36
core 2018.10.19
gsutil 4.34
**
EDIT:
** Thanks for all the response. I tried all the options, but getting the same error with following debug message. I cannot make anything out of it
Thanks for the response. I tried with all the suggested options, but getting the same error. Following is the debug log, and I cannot make anything out of it.
Updating service [getresource] (this may take several minutes)...failed.
DEBUG: (gcloud.app.deploy) Error Response: [13] An internal error occurred during deployment.
Traceback (most recent call last):
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\cli.py", line 841, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\backend.py", line 770, in Run
resources = command_instance.Run(args)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\surface\app\deploy.py", line 90, in Run
parallel_build=False)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 620, in RunDeploy
flex_image_build_option=flex_image_build_option)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 422, in Deploy
extra_config_settings)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\appengine_api_client.py", line 207, in DeployService
poller=done_poller)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\operations_util.py", line 315, in WaitForOperation
sleep_ms=retry_interval)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\util\waiter.py", line 254, in WaitFor
sleep_ms, _StatusUpdate)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\util\waiter.py", line 316, in PollUntilDone
sleep_ms=sleep_ms)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\core\util\retry.py", line 229, in RetryOnResult
if not should_retry(result, state):
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\util\waiter.py", line 310, in _IsNotDone
return not poller.IsDone(operation)
File "C:\Users\M1044921\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\operations_util.py", line 184, in IsDone
encoding.MessageToPyValue(operation.error)))
OperationError: Error Response: [13] An internal error occurred during deployment.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred during deployment.

How to properly configure a Service Account in Google AppEngine Flex w/ Python?

I am using Google AppEngine (GAE) Flexible environment with a custom runtime (python2.7). I am using custom so I can just manage a Dockerfile and have easy parity between my laptop and when deployed in GAE.
I have a python 2.7 Flask app that is doing a query against DataStore. I get the following (when starting python main.py in my local docker):
root#24dcf9b8712d:/home/vmagent/app# curl localhost:5000/things
'/home/vmagent/app/clientid-searchapp.json'
[2018-09-04 14:54:19,969] ERROR in app: Exception on /things [GET]
Traceback (most recent call last):
File "/env/local/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/env/local/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/env/local/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/env/local/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/env/local/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "main.py", line 55, in products
for rec in query.fetch(limit=100):
File "/env/lib/python2.7/site-packages/google/api_core/page_iterator.py", line 199, in _items_iter
for page in self._page_iter(increment=False):
File "/env/lib/python2.7/site-packages/google/api_core/page_iterator.py", line 230, in _page_iter
page = self._next_page()
File "/env/local/lib/python2.7/site-packages/google/cloud/datastore/query.py", line 517, in _next_page
query=query_pb,
File "/env/local/lib/python2.7/site-packages/google/cloud/datastore_v1/gapic/datastore_client.py", line 294, in run_query
request, retry=retry, timeout=timeout, metadata=metadata)
File "/env/lib/python2.7/site-packages/google/api_core/gapic_v1/method.py", line 139, in __call__
return wrapped_func(*args, **kwargs)
File "/env/lib/python2.7/site-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
on_error=on_error,
File "/env/lib/python2.7/site-packages/google/api_core/retry.py", line 177, in retry_target
return target()
File "/env/lib/python2.7/site-packages/google/api_core/timeout.py", line 206, in func_with_timeout
return func(*args, **kwargs)
File "/env/lib/python2.7/site-packages/google/api_core/grpc_helpers.py", line 56, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "/env/local/lib/python2.7/site-packages/six.py", line 737, in raise_from
raise value
PermissionDenied: 403 Missing or insufficient permissions.
127.0.0.1 - - [04/Sep/2018 14:54:19] "GET /things HTTP/1.1" 500 -
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
Snippets from main.py
app = Flask(__name__)
ds = datastore.Client()
<snip>
#app.route("/things")
def things():
global ds
pprint(os.environ['GOOGLE_APPLICATION_CREDENTIALS'])
query = ds.query(kind='Things', order=('thing_name',))
results = list()
for rec in query.fetch(limit=100):
results.append(rec)
return jsonify(results)
Output of gcloud auth list:
root#24dcf9b8712d:/home/vmagent/app# gcloud auth list
Credentialed Accounts
ACTIVE ACCOUNT
* <myapp>#<project-name>.iam.gserviceaccount.com
This is the configured service account. I have over-privileged it with excessive Role membership; I'm pretty certain the service account isn't lacking in permissions. In fact, it has 'DataStore Owner'
In main.py I am outputting:
pprint(os.environ['GOOGLE_APPLICATION_CREDENTIALS'])
And this outputs the path to my client-<myapp>.json file for the service account. It is the same file supplied in the Dockerfile as:
RUN gcloud auth activate-service-account --key-file=clientid-<myapp>.json --project <project-name> --quiet
I get the same results when running the Flask app in my local Docker environment and when I deploy to GAE Flexible. The Flask app has other endpoints that work as they don't call any other services. Only /things uses an API to call DataStore and it receives the above error.
I've been through all the docs. I'm sure it's something basic and obvious but afaict, everything looks right.
UPDATE:
I've tried creating the DataStore client with an explicit pass of project name with no change in result. Also, I've run this command from within the Docker container and this seems like unusual output to me but I don't know that it's incorrect for a Service Account.
root#e02b74cd269d:/home/vmagent/app# gcloud projects list
API [cloudresourcemanager.googleapis.com] not enabled on project
[<project id>]. Would you like to enable and retry (this will take a
few minutes)? (y/N)? y
When I respond w/ Y,:
ERROR: (gcloud.projects.list) PERMISSION_DENIED: Not allowed to get project settings for project <project id>
Output of gcloud config list:
root#d18c83cae166:/home/vmagent/app# gcloud config list
[core]
account = <myapp>#<myproject>.iam.gserviceaccount.com
disable_usage_reporting = False
project = <myproject>
Your active configuration is: [default]