I'm trying to host a Django app on Google App Engine, so I'm using Django nonrel and following these instructions. Now, trying to get Python social auth working on it, I'm running into two problems.
First, when working with code very similar to the example Django config from Python social, trying to load a url from a running server, I get this:
Traceback (most recent call last):
File "/home/pablo/scripts/google_appengine/google/appengine/tools/dev_appserver.py", line 2989, in _HandleRequest
self._Dispatch(dispatcher, self.rfile, outfile, env_dict)
File "/home/pablo/scripts/google_appengine/google/appengine/tools/dev_appserver.py", line 2832, in _Dispatch
request_file = open(request_file_name, 'wb')
File "/home/pablo/scripts/google_appengine/google/appengine/dev_appserver_import_hook.py", line 605, in __init__
raise IOError('invalid mode: %s' % mode)
IOError: invalid mode: wb
Somewhere, the app is trying to create local files, which App Engine doesn't allow, but I'm confused because this is coming from App Engine's code. Does anyone know where this might be coming from?
Secondly, when I try to access root on the server, I get the following error:
... [many lines elided]
File "/home/pablo/scripts/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 692, in Decorate
return func(self, *args, **kwargs)
File "/home/pablo/scripts/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1642, in FindAndLoadModule
description)
File "/home/pablo/scripts/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 692, in Decorate
return func(self, *args, **kwargs)
File "/home/pablo/scripts/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1589, in LoadModuleRestricted
description)
File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 42, in <module>
raise ImportError, str(msg) + ', please install the python-tk package'
ImportError: No module named _tkinter, please install the python-tk package
Somewhere, Django is trying to use python-tk for Tkinter, but as I understand it, python-tk is a GUI library. How did it get here, and how can I get rid of any code that needs it?
For reference, here's the function getting called for the domain I'm trying in urls.py when accessing the running server (and getting these errors):
def home(request):
"""Home view, displays login mechanism"""
if request.user.is_authenticated():
return redirect('done')
return render_to_response('home.html', {
'plus_id': getattr(settings, 'SOCIAL_AUTH_GOOGLE_PLUS_KEY', None)
}, RequestContext(request))
Any help would be appreciated -- I'm pretty new to Django and Python, and I'd love to move forward from this :D
Python on Google App Engine is behaving a bit different, as you already realised with the local files. Another thing that needs a special treatment is the 3rd party libraries that in order to make them available, they should be handled properly.
In many cases, even if you're going to include these libraries into your GAE app, they might be using something that is not supported on the production so the whole thing is not going to work.
Related
I'v been trying to launch project(example_setup folder):
https://github.com/OTA-Insight/djangosaml2idp/tree/master/example_setup
I can anybody answer to men according with documentation. But it does not working. First problem, as I undesrtand is in date of methadata in SP(idp_metadata.xml)- validUntil="2020-12-27T12:41:18Z"> . It does not valid at the moment, and was changed to future date, as example(validUntil="2030-12-27T12:41:18Z"). But next I got another problem when trying to sign in to SP(localhost:8000) in my browser, I have more problem:
Error during SAML2 authentication
IncorrectlySigned
In attempts to find problem, I found the place where it is occured. In original it iis in tryexcept block, and can't be found easy.
Traceback (most recent call last):
File "/home/dmitriy/projects/djangosaml2idp/example_setup/idp/djangosaml2idp/views.py", line 251, in get
req_info = idp_server.parse_authn_request(request.session['SAMLRequest'], binding)
File "/home/dmitriy/projects/djangosaml2idp/example_setup/idp/venv/lib/python3.8/site-packages/saml2/server.py", line 238, in parse_authn_request
return self._parse_request(enc_request, AuthnRequest,
File "/home/dmitriy/projects/djangosaml2idp/example_setup/idp/venv/lib/python3.8/site-packages/saml2/entity.py", line 1036, in _parse_request
_request = _request.loads(xmlstr, binding, origdoc=enc_request,
File "/home/dmitriy/projects/djangosaml2idp/example_setup/idp/venv/lib/python3.8/site-packages/saml2/request.py", line 110, in loads
return self._loads(xmldata, binding, origdoc, must,
File "/home/dmitriy/projects/djangosaml2idp/example_setup/idp/venv/lib/python3.8/site-packages/saml2/request.py", line 51, in _loads
print(self.signature_check(xmldata, origdoc=origdoc,
File "/home/dmitriy/projects/djangosaml2idp/example_setup/idp/venv/lib/python3.8/site-packages/saml2/sigver.py", line 1662, in correctly_signed_authn_request
return self.correctly_signed_message(decoded_xml, 'authn_request', must, origdoc, only_valid_cert=only_valid_cert)
File "/home/dmitriy/projects/djangosaml2idp/example_setup/idp/venv/lib/python3.8/site-packages/saml2/sigver.py", line 1653, in correctly_signed_message
return self._check_signature(
File "/home/dmitriy/projects/djangosaml2idp/example_setup/idp/venv/lib/python3.8/site-packages/saml2/sigver.py", line 1503, in _check_signature
raise MissingKey(_issuer)
saml2.sigver.MissingKey: http://localhost:8000/saml2/metadata/
Internal Server Error: /idp/login/process/
Some key is missing:
Error during SAML2 authentication
MissingKey
http://localhost:8000/saml2/metadata/
My idp_metada in the SP is like in the [example_setup][1] folder of project, only validUntil has been changed as I said above, user in IDP has been created as superuser, I also tried to create user in the SP, the same as in the IDP, but nothing changed
Can anybody anser to me, what my problem is?
It's no any information in docs. But you need to create SP inside IDP from admin panel. That was resolved my problem.
Im having trouble getting identity-toolkit fully working with Python App Engine Sandbox. The sample provided is for a non GAE Sandbox project.
In the sample project it reads gitkit-server-config.json from file using os.path. But this is not supported in GAE Sandbox. To get around this I am creating a GitkitClient directly using the constructor:
gitkit_instance = gitkitclient.GitkitClient(
client_id="123456opg.apps.googleusercontent.com",
service_account_email="my-project#appspot.gserviceaccount.com",
service_account_key="/path/to/my-p12file.p12",
widget_url="http://localhost:8080/callback",
http=None,
project_id="my-project")
Is this the correct way to create the GitkitClient?
The issue now is when I try to do a password reset when running locally using dev_appserver.py I get the following stack trace:
File "dashboard.py", line 89, in post
oobResult = gitkit_instance.GetOobResult(self.request.POST,self.request.remote_addr)
File "identitytoolkit/gitkitclient.py", line 366, in GetOobResult
param['action'])
File "identitytoolkit/gitkitclient.py", line 435, in _BuildOobLink
code = self.rpc_helper.GetOobCode(param)
File "identitytoolkit/rpchelper.py", line 104, in GetOobCode
response = self._InvokeGitkitApi('getOobConfirmationCode', request)
File "identitytoolkit/rpchelper.py", line 210, in _InvokeGitkitApi
access_token = self._GetAccessToken()
File "identitytoolkit/rpchelper.py", line 231, in _GetAccessToken
'assertion': self._GenerateAssertion(),
File "identitytoolkit/rpchelper.py", line 259, in _GenerateAssertion
crypt.Signer.from_string(self.service_account_key),
File "oauth2client/_pure_python_crypt.py", line 183, in from_string
raise ValueError('No key could be detected.')
ValueError: No key could be detected.
Im assuming this is a problem with the .p12 file? I double checked service_account_key="/path/to/my-p12file.p12" and the file exists. What am I missing here?
FYI to others working on this in the future -
I could not get this working in python. The documentation doesn't make it clear how to get this working in app engine. In addition, dependency issues with PyCrypto made this a gcc and dependency nightmare.
I was however able to get this working in Go and there is a semi-working example online that will work with some modifications highlighted in the issues and pull request pages. Good luck.
I hot upon an requirement where I needed to automatically answer the prompt on remote machine and then I found fexpect after reading different stackoverflow questions. But the moment I include fexpect in my script it breaks the whole script!
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/fabric/main.py", line 743, in main
*args, **kwargs
File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 368, in execute
multiprocessing
File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 264, in _execute
return task.run(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/fabric/tasks.py", line 171, in run
return self.wrapped(*args, **kwargs)
File "/etc/puppet/fabfile.py", line 165, in edit_sudoers
run('echo "Current Permission of the file /etc/sudoers - "`stat -c "%a %n" /etc/sudoers`')
File "/usr/local/lib/python2.7/dist-packages/ilogue/fexpect/api.py", line 15, in run
wrappedCmd = wrapExpectations(cmd)
File "/usr/local/lib/python2.7/dist-packages/ilogue/fexpect/internals.py", line 15, in wrapExpectations
script = createScript(cmd)
File "/usr/local/lib/python2.7/dist-packages/ilogue/fexpect/internals.py", line 39, in createScript
for e in fabric.state.env.expectations:
File "/usr/local/lib/python2.7/dist-packages/fabric/utils.py", line 184, in __getattr__
raise AttributeError(key)
AttributeError: expectations
The moment i write
from ilogue.fexpect import expect, expecting, run fabric stops working with the above error message.
I asked in fabric irc as well but I got to know that this might be because of some version related issues. Has anyone else encountered this error before?
fexpect==0.2.post7
Fabric==1.8.0
Just import fexpect's run as erun and its sudo as esudo.
When you use the fexpect run or sudo functions, you must wrap those calls in a with expecting(prompts): context. This is a known issue in fexpect, although there is a pull request, so it might be fixed by the time posterity reads this.
One solution is to import fexpect's run function with a different name, e.g. erun, and use it only when you need the automatic prompt handling functionality:
from fabric.api import run
from ilogue.fexpect import expect, expecting, run as erun
run(a_cmd) # Native Fabric run - should work fine
prompts = [...]
with expecting(prompts):
erun(a_prompting_cmd) # fexpect run - should with fine inside expecting context
Another thing that isn't explicitly stated in the fexpect documentation is that the pexpect package needs to be installed on the target system.
Yet another fexpect gotcha is that the prompt strings are regular expressions -- the fexpect sample code is misleading about this.
I recently upgraded my GAE Python app to Python 2.7. Since then, I periodically get the following error with the dev server and the dev server serves up a blank page:
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 168, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 206, in _LoadHandler
handler = __import__(path[0])
[...]
File "/Users/joneill/OpenSTV/OpenSTV/trunk/OpaVote-HR/main.py", line 2, in <module>
import views
[...]
File "/Users/joneill/OpenSTV/OpenSTV/trunk/OpaVote-HR/views.py", line 3, in <module>
from pytz.gae import pytz
[...]
File "/Users/joneill/OpenSTV/OpenSTV/trunk/OpaVote-HR/pytz/__init__.py", line 34, in <module>
from pkg_resources import resource_stream
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 662, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1818, in load_module
return self.FindAndLoadModule(submodule, fullname, search_path)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 662, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1690, in FindAndLoadModule
description)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 662, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1615, in LoadModuleRestricted
return source_file.load_module(submodule_fullname)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/dist/py_zipimport.py", line 246, in load_module
submodname, is_package, fullpath, source = self._get_source(fullmodname)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/dist/py_zipimport.py", line 207, in _get_source
source = self.zipfile.read(relpath.replace(os.sep, '/'))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 867, in read
return self.open(name, "r", pwd).read()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/zipfile.py", line 882, in open
zef_file = open(self.filename, 'rb')
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 578, in __init__
raise IOError(errno.EACCES, 'file not accessible', filename)
IOError: [Errno 13] file not accessible: '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg'
INFO 2012-01-21 20:50:44,222 dev_appserver.py:2832] "POST /manage HTTP/1.1" 500 -
Some notes:
This doesn't happen on the production server.
On the dev server, my app will work for a few minutes and then this error happens.
If I stop and restart my app on the dev server, it will work again for a few minutes.
I am using the latest version of gae-pytz and you can see that it fails in an import there.
The [...] that I removed are similar to the stuff you see near the end.
I don't know why setuptools is being invoked at the end.
I'm using a Mac with Lion.
I can use the dev server, but it is really annoying to stop and restart every few minutes. Any ideas how to fix this?
The actual problem from the stack trace, is your code is trying to import setup tools from site-packages, which the dev server won't do.
'/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg'
You will need to include setuptools in you application code base. The fact that it works sometimes, suggests that you code paths through various modules vary, and maybe (depending on what your tesing in dev) different import orders mean setup tools has been imported somewhere else, or is only required at certain points in your code.
Have a look at line 4th line of the stack trace where pytz is imported, the next line is from pkg_resources import resource_stream thats whats triggering the rest of the import issue. I use a fake truncated pkg_resources at the root of my project, that doesn't end up trying to import stuff from setup tools. You can run the dev server in debug import mode which will tell you a lot more
Here is a fake pkg_resources.
"""Package resource API
--------------------
A resource is a logical file contained within a package, or a logical
subdirectory thereof. The package resource API expects resource names
to have their path parts separated with ``/``, *not* whatever the local
path separator is. Do not use os.path operations to manipulate resource
names being passed into the API.
The package resource API is designed to work with normal filesystem packages,
.egg files, and unpacked .egg files. It can also work in a limited way with
.zip files and with custom PEP 302 loaders that support the ``get_data()``
method.
"""
import sys, os, zipimport, time, re, imp, new
try:
frozenset
except NameError:
from sets import ImmutableSet as frozenset
from os import utime #, rename, unlink # capture these to bypass sandboxing
from os import open as os_open
There are probably other/better ways of doing this, but it works for me.
Oh, I would also suggest you use http://code.google.com/p/gae-pytz/ instead of pytz.
Cheers
An alternative answer to the above that I prefer.
The __init__.py file for pytz contains the following lines:
#try:
# from pkg_resources import resource_stream
#except ImportError:
resource_stream = None
I commented out the first three lines and that fixed the problem.
The problem is a bug in the App Engine dev server with Python 2.7. The solution is here:
File not accesible error (setuptools) in logs
Yesterday I setup Apache to serve my Mercurial repositories and got everything working properly. I then tested pushing changes back to this repository and was presented with an error, and now that error pops up for every single operation I attempt - even just a simple GET request of the repositories! Here is the error:
mod_wsgi (pid=1771): Target WSGI script '/var/hg/hgweb.wsgi' cannot be loaded as Python module.
mod_wsgi (pid=1771): Exception occurred processing WSGI script '/var/hg/hgweb.wsgi'.
Traceback (most recent call last):
File "/var/hg/hgweb.wsgi", line 18, in ?
application = hgwebdir(config)
File "/usr/lib64/python2.4/site-packages/mercurial/hgweb/__init__.py", line 15, in hgwebdir
return hgwebdir_mod.hgwebdir(*args, **kwargs)
File "/usr/lib64/python2.4/site-packages/mercurial/hgweb/hgwebdir_mod.py", line 52, in __init__
self.refresh()
File "/usr/lib64/python2.4/site-packages/mercurial/hgweb/hgwebdir_mod.py", line 82, in refresh
self.repos = findrepos(paths)
File "/usr/lib64/python2.4/site-packages/mercurial/hgweb/hgwebdir_mod.py", line 36, in findrepos
for path in util.walkrepos(roothead, followsym=True, recurse=recurse):
File "/usr/lib64/python2.4/site-packages/mercurial/util.py", line 1164, in walkrepos
for hgname in walkrepos(fname, True, seen_dirs):
File "/usr/lib64/python2.4/site-packages/mercurial/util.py", line 1146, in walkrepos
for root, dirs, files in os.walk(path, topdown=True, onerror=errhandler):
File "/usr/lib64/python2.4/os.py", line 276, in walk
onerror(err)
File "/usr/lib64/python2.4/site-packages/mercurial/util.py", line 1127, in errhandler
raise err
OSError: [Errno 13] Permission denied: './dev/fd'
My repository directory is owned by apache, the user running Apache. I dont know why './dev/fd' is being operated on either. I've restarted the server numerous times, recreated the repository directory, but I still get this error no matter what! I dont have access to restart the machine, so that is not an option. But it seems to have gotten in a very bad persistent state, and I dont know how to fix it. Any help is appreciated!
This turned out to be a configuration error on my part, and rather than delete the question I'll post the resolution here in case someone has this problem in the future.
Here was the hgweb.config I was using:
[paths]
/ = /var/hg/repos/*
#[web]
style = gitweb
allow_archive = bz2 gz zip
maxchanges = 200
allow_push = *
push_ssl = false
Two problems here, one is obvious. I had the [web] header commented out, and I assume that many of the options are not valid for the [paths] section. Also, after re-reading the Hg docs again, the push_ssl directive does not belong in the hgweb.config file, but rather in each repository's .hg/hgrc (or the ~/.hgrc of the user that runs apache). After fixing these, things are working perfectly!