Server error when pyramid app hosted with production.ini - python

I have a pyramid project that works fine when hosted with development.ini, but as soon as I use production.ini I get this error in browser:
Internal Server Error
The server encountered an unexpected internal server error
(generated by waitress)
and this log in terminal:
Starting subprocess with file monitor
Starting server in PID 5912.
serving on http://0.0.0.0:6543
2014-05-26 16:54:23,139 ERROR [waitress][Dummy-2] Exception when serving /
Traceback (most recent call last):
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/waitress-0.8.9-py2.7.egg/waitress/channel.py", line 337, in service
task.service()
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/waitress-0.8.9-py2.7.egg/waitress/task.py", line 173, in service
self.execute()
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/waitress-0.8.9-py2.7.egg/waitress/task.py", line 392, in execute
app_iter = self.channel.server.application(env, start_response)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid/router.py", line 242, in __call__
response = self.invoke_subrequest(request, use_tweens=True)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid/router.py", line 217, in invoke_subrequest
response = handle_request(request)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid/tweens.py", line 21, in excview_tween
response = handler(request)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid_tm-0.7-py2.7.egg/pyramid_tm/__init__.py", line 82, in tm_tween
reraise(*exc_info)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid_tm-0.7-py2.7.egg/pyramid_tm/__init__.py", line 63, in tm_tween
response = handler(request)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid/router.py", line 163, in handle_request
response = view_callable(context, request)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid/config/views.py", line 377, in rendered_view
context)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid/renderers.py", line 418, in render_view
return self.render_to_response(response, system, request=request)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid/renderers.py", line 441, in render_to_response
result = self.render(value, system_values, request=request)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid/renderers.py", line 421, in render
renderer = self.renderer
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid/decorator.py", line 37, in __get__
val = self.wrapped(inst)
File "/home/roman/Critical_ID/CCG_Map/local/lib/python2.7/site-packages/pyramid/renderers.py", line 404, in renderer
'No such renderer factory %s' % str(self.type))
ValueError: No such renderer factory .mak
I copied over my sqlalchemy.url and mako.directories under the [app:main] section in the .ini files, and I can't see what is causing this error.
I start the server with: ../bin/pserve/production.ini --reload
What am I missing?

This is because the latest versions of Pyramid do not require/install Mako's templating engine by default. To fix this error, as documented HERE, you must do two things:
Make sure the pyramid_mako package is installed. One way to do this is by adding "pyramid_mako" to the install_requires section of your package's setup.py file and afterwards rerunning setup.py develop:
setup(
#...
install_requires=[
'pyramid_mako', # new dependency
'pyramid',
#...
],
)
Within the portion of your application which instantiates a Pyramid Configurator (often the main() function in your project's __init__.py file), tell Pyramid to include the pyramid_mako includeme:
config = Configurator(.....)
config.include('pyramid_mako')

Related

Asn1 parsing error when trying to connect to mongo atlas

I am using MongoDB atlas for my discord bot but recently ran into an error. On the hosting (Heroku) everything works without errors, I first updated all the modules, but the error has not disappeared.
I am using motor as a driver to work with MongoDB atlas.
Checked the database connection URL, everything is correct.
Python version 3.10(on Heroku too)
Traceback (most recent call last):
File "D:\Проекты\AkainuBot\main.py", line 23, in <module>
mongo = AsyncIOMotorClient(
File "D:\Python\lib\site-packages\motor\core.py", line 159, in __init__
delegate = self.__delegate_class__(*args, **kwargs)
File "D:\Python\lib\site-packages\pymongo\mongo_client.py", line 718, in __init__
self.__options = options = ClientOptions(
File "D:\Python\lib\site-packages\pymongo\client_options.py", line 165, in __init__
self.__pool_options = _parse_pool_options(options)
File "D:\Python\lib\site-packages\pymongo\client_options.py", line 132, in _parse_pool_options
ssl_context, ssl_match_hostname = _parse_ssl_options(options)
File "D:\Python\lib\site-packages\pymongo\client_options.py", line 98, in _parse_ssl_options
ctx = get_ssl_context(
File "D:\Python\lib\site-packages\pymongo\ssl_support.py", line 159, in get_ssl_context
ctx.load_verify_locations(certifi.where())
File "D:\Python\lib\site-packages\pymongo\pyopenssl_context.py", line 276, in load_verify_locations
self._callback_data.trusted_ca_certs = _load_trusted_ca_certs(cafile)
File "D:\Python\lib\site-packages\pymongo\ocsp_support.py", line 79, in _load_trusted_ca_certs
_load_pem_x509_certificate(cert_data, backend))
File "D:\Python\lib\site-packages\cryptography\x509\base.py", line 436, in load_pem_x509_certificate
return rust_x509.load_pem_x509_certificate(data)
ValueError: error parsing asn1 value: ParseError { kind: InvalidValue, location: ["RawCertificate::tbs_cert", "TbsCertificate::serial"] }
I've solved my problem getting the parsing asn1 value error by adding the ssl_cert_reqs option on the MongoClient (and obviously importing ssl).
dbClient = pymongo.MongoClient(uri, ssl_cert_reqs=ssl.CERT_NONE)

How to properly configure a Service Account in Google AppEngine Flex w/ Python?

I am using Google AppEngine (GAE) Flexible environment with a custom runtime (python2.7). I am using custom so I can just manage a Dockerfile and have easy parity between my laptop and when deployed in GAE.
I have a python 2.7 Flask app that is doing a query against DataStore. I get the following (when starting python main.py in my local docker):
root#24dcf9b8712d:/home/vmagent/app# curl localhost:5000/things
'/home/vmagent/app/clientid-searchapp.json'
[2018-09-04 14:54:19,969] ERROR in app: Exception on /things [GET]
Traceback (most recent call last):
File "/env/local/lib/python2.7/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/env/local/lib/python2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/env/local/lib/python2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/env/local/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/env/local/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "main.py", line 55, in products
for rec in query.fetch(limit=100):
File "/env/lib/python2.7/site-packages/google/api_core/page_iterator.py", line 199, in _items_iter
for page in self._page_iter(increment=False):
File "/env/lib/python2.7/site-packages/google/api_core/page_iterator.py", line 230, in _page_iter
page = self._next_page()
File "/env/local/lib/python2.7/site-packages/google/cloud/datastore/query.py", line 517, in _next_page
query=query_pb,
File "/env/local/lib/python2.7/site-packages/google/cloud/datastore_v1/gapic/datastore_client.py", line 294, in run_query
request, retry=retry, timeout=timeout, metadata=metadata)
File "/env/lib/python2.7/site-packages/google/api_core/gapic_v1/method.py", line 139, in __call__
return wrapped_func(*args, **kwargs)
File "/env/lib/python2.7/site-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
on_error=on_error,
File "/env/lib/python2.7/site-packages/google/api_core/retry.py", line 177, in retry_target
return target()
File "/env/lib/python2.7/site-packages/google/api_core/timeout.py", line 206, in func_with_timeout
return func(*args, **kwargs)
File "/env/lib/python2.7/site-packages/google/api_core/grpc_helpers.py", line 56, in error_remapped_callable
six.raise_from(exceptions.from_grpc_error(exc), exc)
File "/env/local/lib/python2.7/site-packages/six.py", line 737, in raise_from
raise value
PermissionDenied: 403 Missing or insufficient permissions.
127.0.0.1 - - [04/Sep/2018 14:54:19] "GET /things HTTP/1.1" 500 -
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>
Snippets from main.py
app = Flask(__name__)
ds = datastore.Client()
<snip>
#app.route("/things")
def things():
global ds
pprint(os.environ['GOOGLE_APPLICATION_CREDENTIALS'])
query = ds.query(kind='Things', order=('thing_name',))
results = list()
for rec in query.fetch(limit=100):
results.append(rec)
return jsonify(results)
Output of gcloud auth list:
root#24dcf9b8712d:/home/vmagent/app# gcloud auth list
Credentialed Accounts
ACTIVE ACCOUNT
* <myapp>#<project-name>.iam.gserviceaccount.com
This is the configured service account. I have over-privileged it with excessive Role membership; I'm pretty certain the service account isn't lacking in permissions. In fact, it has 'DataStore Owner'
In main.py I am outputting:
pprint(os.environ['GOOGLE_APPLICATION_CREDENTIALS'])
And this outputs the path to my client-<myapp>.json file for the service account. It is the same file supplied in the Dockerfile as:
RUN gcloud auth activate-service-account --key-file=clientid-<myapp>.json --project <project-name> --quiet
I get the same results when running the Flask app in my local Docker environment and when I deploy to GAE Flexible. The Flask app has other endpoints that work as they don't call any other services. Only /things uses an API to call DataStore and it receives the above error.
I've been through all the docs. I'm sure it's something basic and obvious but afaict, everything looks right.
UPDATE:
I've tried creating the DataStore client with an explicit pass of project name with no change in result. Also, I've run this command from within the Docker container and this seems like unusual output to me but I don't know that it's incorrect for a Service Account.
root#e02b74cd269d:/home/vmagent/app# gcloud projects list
API [cloudresourcemanager.googleapis.com] not enabled on project
[<project id>]. Would you like to enable and retry (this will take a
few minutes)? (y/N)? y
When I respond w/ Y,:
ERROR: (gcloud.projects.list) PERMISSION_DENIED: Not allowed to get project settings for project <project id>
Output of gcloud config list:
root#d18c83cae166:/home/vmagent/app# gcloud config list
[core]
account = <myapp>#<myproject>.iam.gserviceaccount.com
disable_usage_reporting = False
project = <myproject>
Your active configuration is: [default]

cant add the twilio-python library to a Google App Engine project

hi I have been trying to add twilio-python library to a google app engine project.As recommended I followed the steps from the link below:http://twilio-python.readthedocs.org/en/latest/appengine.html.
However,I am encountering error messages about which I am a bit clueless.I am providing the log of my app engine launcher
Please have a look and let me know what am I doing wrong.
Heres the log message :
*** Running dev_appserver with the following flags:
--skip_sdk_update_check=yes --port=12081 --admin_port=8005
Python command: /usr/bin/python2.7
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/dev_appserver.py", line 83, in <module>
_run_file(__file__, globals())
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/dev_appserver.py", line 79, in _run_file
execfile(_PATHS.script_file(script_name), globals_)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 1040, in <module>
main()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 1033, in main
dev_server.start(options)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 758, in start
options.config_paths, options.app_id)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/application_configuration.py", line 831, in __init__
module_configuration = ModuleConfiguration(config_path, app_id)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/application_configuration.py", line 127, in __init__
self._config_path)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/application_configuration.py", line 424, in _parse_configuration
config, files = appinfo_includes.ParseAndReturnIncludePaths(f)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/appinfo_includes.py", line 82, in ParseAndReturnIncludePaths
appyaml = appinfo.LoadSingleAppInfo(appinfo_file)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/appinfo.py", line 2214, in LoadSingleAppInfo
raise appinfo_errors.EmptyConfigurationFile()
google.appengine.api.appinfo_errors.EmptyConfigurationFile
App Engine tried to load an empty configuration file, your app.yaml file. Hence it's throwing an EmptyConfigurationFile error. App engine needs to be informed about some basic configuration parameters to start serving your app, however basic it is.
The following is a basic app.yaml config file structured for your example:
application: twilio-demo
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: .*
script: helloworld.app
You can read more on Configuring with app.yaml

Duplicity backup to onedrive client error

I'm trying to make backup of files on my computer in onedrive with duplicity.
I have installed all dependencies, when running duplicity there is the auth link generated which I must open in browser and than in duplicity after giving permissions for app paste the return url.
I do all this steps but duplicity is returning me
Traceback (most recent call last):
File "/usr/bin/duplicity", line 1532, in <module>
with_tempdir(main)
File "/usr/bin/duplicity", line 1526, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1364, in main
action = commandline.ProcessCommandLine(sys.argv[1:])
File "/usr/lib/python2.7/site-packages/duplicity/commandline.py", line 1116, in ProcessCommandLine
backup, local_pathname = set_backend(args[0], args[1])
File "/usr/lib/python2.7/site-packages/duplicity/commandline.py", line 1005, in set_backend
globals.backend = backend.get_backend(bend)
File "/usr/lib/python2.7/site-packages/duplicity/backend.py", line 223, in get_backend
obj = get_backend_object(url_string)
File "/usr/lib/python2.7/site-packages/duplicity/backend.py", line 209, in get_backend_object
return factory(pu)
File "/usr/lib/python2.7/site-packages/duplicity/backends/onedrivebackend.py", line 90, in __init__
self.initialize_oauth2_session()
File "/usr/lib/python2.7/site-packages/duplicity/backends/onedrivebackend.py", line 153, in initialize_oauth2_session
authorization_response=redirected_to)
File "/usr/lib/python2.7/site-packages/requests_oauthlib/oauth2_session.py", line 232, in fetch_token
self._client.parse_request_body_response(r.text, scope=self.scope)
File "/usr/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/clients/base.py", line 409, in parse_request_body_response
self.token = parse_token_response(body, scope=scope)
File "/usr/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 376, in parse_token_response
validate_token_parameters(params)
File "/usr/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 383, in validate_token_parameters
raise_from_error(params.get('error'), params)
File "/usr/lib/python2.7/site-packages/oauthlib/oauth2/rfc6749/errors.py", line 271, in raise_from_error
raise cls(**kwargs)
InvalidClientError: (invalid_client) The client does not exist. If you are the application developer, configure a new application through the application management site at https://manage.dev.live.com/.
It looks like there is no app with ID which duplicity generate auth link with.
But when I go to the link provided by duplicity I see that "Duplicity is asking for permissions".
So should I add my own app and in some way provide its id to duplicity? (I was searching how to do it but without result) or is it a duplicity bug?
All programmatic interaction with Windows Live requires a client ID,
which uniquely identifies your application to Windows Live. Your
application must include the client ID in every request that it sends
to the Messenger Connect API Service.
You have to register your application as shown in this official Windows Live tutorial:
https://msdn.microsoft.com/en-us/library/ff751474.aspx
And then pass your ID to the application to be able to authentificate in Windows Live in execution time when asking to the API.
You can use the code in
https://github.com/fkalis/bash-onedrive-upload
which also provide full support for upload files which size is bigger then 100MB

NotAllowedError when trying to access AppStats

After doing everything from here
http://code.google.com/appengine/docs/python/tools/appstats.html
I get
<class 'google.appengine.api.users.NotAllowedError'>:
Traceback (most recent call last):
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/appstats/ui.py", line 324, in <module>
main()
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/appstats/ui.py", line 320, in main
util.run_bare_wsgi_app(app)
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/util.py", line 116, in run_bare_wsgi_app
result = application(env, _start_response)
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/appstats/ui.py", line 308, in __call__
users.create_login_url(os.getenv('PATH_INFO', '')))])
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/users.py", line 256, in create_login_url
raise NotAllowedError
when trying to access myapp.appspot.com/_ah/stats or just myapp.appspot.com/stats/
Most likely you have created your app with authentication set to only allow users from your Google Apps domain, but you're attempting to access appstats via the appspot.com subdomain. Try again using the URL for your app on your custom domain.

Categories