When I try to upload a sample csv data to my GAE app through appcfg.py, it shows the below 401 error.
2015-11-04 10:44:41,820 INFO client.py:571 Refreshing due to a 401 (attempt 2/2)
2015-11-04 10:44:41,821 INFO client.py:797 Refreshing access_token
Error 401: --- begin server output ---
You must be logged in as an administrator to access this.
--- end server output ---
Here is the command I tried,
appcfg.py upload_data --application=dev~app --url=http://localhost:8080/_ah/remote_api --filename=data/sample.csv
This is how we do it in order to use custom authentication.
Custom handler in app.yaml
- url: /remoteapi.*
script: remote_api.app
Custom wsgi app in remote_api.py to override CheckIsAdmin
from google.appengine.ext.remote_api import handler
from google.appengine.ext import webapp
import re
MY_SECRET_KEY = 'MAKE UP PASSWORD HERE' # make one up, use the same one in the shell command
cookie_re = re.compile('^"?([^:]+):.*"?$')
class ApiCallHandler(handler.ApiCallHandler):
def CheckIsAdmin(self):
"""Determine if admin access should be granted based on the
auth cookie passed with the request."""
login_cookie = self.request.cookies.get('dev_appserver_login', '')
match = cookie_re.search(login_cookie)
if (match and match.group(1) == MY_SECRET_KEY
and 'X-appcfg-api-version' in self.request.headers):
return True
else:
self.redirect('/_ah/login')
return False
app = webapp.WSGIApplication([('.*', ApiCallHandler)])
From here we script the uploading of data that was exported from our live app. Use the same password that you made up in the python script above.
echo "MAKE UP PASSWORD HERE" | appcfg.py upload_data --email=some#example.org --passin --url=http://localhost:8080/remoteapi --num_threads=4 --kind=WebHook --filename=webhook.data --db_filename=bulkloader-progress-webhook.sql3
WebHook and webhook.data are specific to the Kind that we exported from production.
I had a similar issue, where appcfg.py was not giving me any credentials dialog, so I could not authenticate. I downgraded from GAELauncher 1.27 to 1.26, and the authentication started working again.
Temporary solution: go to https://console.developers.google.com/storage/browser/appengine-sdks/featured/ to get version 1.9.26
Submitted bug report: https://code.google.com/p/google-cloud-sdk/issues/detail?id=340
You cannot use the appcfg.py upload_data command with the development server [edit: as is; see Josh J's answer]. It only works with the remote_api endpoint running on App Engine and authenticated with OAuth2.
An easy way to load data into the dev server's datastore is to create an endpoint that reads a CSV file and creates the appropriate datastore entities, then hit it with the browser. (Be sure to remove the endpoint before deploying the app, or restrict access to the URL with login: admin.)
You must have an oauth token for a google account that is not an admin of that project. Try passing the --no_cookies flag so that it prompts for authentication again.
Maybe this has something to do with it? From the docs
Connecting your app to the local development server
To use the local development server for your app running locally, you
need to do the following:
Set environment variables. Add or modify your app's Datastore
connection code. Setting environment variables
Create an environment variable DATASTORE_HOST and set it to the host
and port on which the local development server is listening. The
default host and port is http://localhost:8080. (Note: If you use the
port and/or host command line arguments to change these defaults, be
sure to adjust DATASTORE_HOST accordingly.) The following bash shell
example shows how to set this variable:
export DATASTORE_HOST=http://localhost:8080 Create an environment
variable named DATASTORE_DATASET and set it to your dataset ID, as
shown in the following bash shell example:
export DATASTORE_DATASET= Note: Both the Python and Java
client libraries look for the environment variables DATASTORE_HOST and
DATASTORE_DATASET.
Link to Docs
https://cloud.google.com/datastore/docs/tools/devserver
I'm looking to send an SMS with the Twilio api, but I'm getting the following error:
"unknown url type: https"
I've recompiled python with Openssl, so my code runs fine from the python interpretor, but whenever I try to run it in one of my django views I get this error. Here is my code from my view:
def send_sms(request):
recipient = '1234567890'
account = twilio.Account(settings.TWILIO_ID, settings.TWILIO_TOKEN)
params = { 'From': settings.TWILIO_NUM, 'To': recipient, 'Body': 'This is a test message.', }
account.request('/%s/Accounts/%s/SMS/Messages' % (settings.TWILIO_API_VERSION, settings.TWILIO_ID), 'POST', params)
Edit- More info (thanks for bringing that up Stefan)
The project is hosted on dreamhost via Passenger wsgi. Django is using the same python install location and interp.
I appreciate any insight anyone may have, thanks!
Looks like it was just user error. My wsgi file was using a different interpreter but the paths were so similar I was just over looking it. Once I fixed that django was using the python version that I compiled with openssl and everything worked fine.
Always check if the tv is plugged in before you take it apart. Thanks stefanw!
According to the documentation, if DEBUG is set to False and something is provided under the ADMINS setting, Django will send an email whenever the code raises a 500 status code. I have the email settings filled out properly (as I can use send_mail fine) but whenever I intentionally put up erroneous code I get my 500.html template but no error email is sent. What could cause Django to not do this?
In my case the cause was missing SERVER_EMAIL setting.
The default for SERVER_EMAIL is root#localhost. But many of email servers including
my email provider do not accept emails from such suspicious addresses. They silently drop the emails.
Changing the sender email address to django#my-domain.example solved the problem. In settings.py:
SERVER_EMAIL = 'django#my-domain.example'
Another possibility for error is trouble with your ADMINS setting. The following setting will cause the sending of mail to admins to fail quietly:
ADMINS = (
('your name', 'me#mydomain.example')
)
What's wrong with that? Well ADMINS needs to be a tuple of tuples, so the above needs to be formatted as
ADMINS = (
('your name', 'me#mydomain.example'),
)
Note the trailing comma. Without the failing comma, the 'to' address on the email will be incorrectly formatted (and then probably discarded silently by your SMTP server).
I had the same situation. I created a new project and app and it worked, so I knew it was my code. I tracked it down to the LOGGING dictionary in settings.py. I had made some changes a few weeks back for logging with Sentry, but for some reason the error just started today. I changed back to the original and got it working:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'mail_admins': {
'level': 'ERROR',
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
Then, I made some changes slowly and got it working with Sentry and emailing the ADMINS as well.
Additionally, the LOGGING configuration gets merged with DEFAULT_LOGGING by default, so it's useful to have a look at the source code of django.utils.log.DEFAULT_LOGGING to understand what else may have an effect on your particular situation.
Make sure your EMAIL_HOST and EMAIL_PORT are set up right in settings.py (these refer to your SMTP server). It might be assuming that you have an SMTP server running on localhost.
To test this locally, run Python's built-in test SMTP server:
python -m smtpd -n -c DebuggingServer localhost:1025
Then set these values in your settings.py
EMAIL_HOST='localhost'
EMAIL_PORT=1025
Trigger a 500 error, and you should see the e-mail appear in the python smtpd terminal window.
My web hosting provider - Webfaction - only allows emails to be sent From an email that has been explicitly created in the administrator panel. Creating one fixed the problem.
Another thing worth noting here is that settings handler500 might bypass the mechanism that sends errors on a 500 if the response from the view doesn't have a status code of 500.
If you have a handler500 set, then in that view respond with something like this.
t = loader.get_template('500.html')
response = HttpResponseServerError(
t.render(RequestContext(request, {'custom_context_var':
'IT BROKE OMG FIRE EVERYONE'})))
response.status_code = 500
return response
Sorry if it is too naive, but in my case the emails were sent but were going directly to the SPAM folder. Before trying more complicated things check your SPAM folder first.
If, for some reason, you set DEBUG_PROPAGATE_EXCEPTIONS to True (it's False by default), email to admin will not work.
Just had the same issue after upgraded to Django 2.1 from Django 1.11. Apparently the ADMINS sections in settings.py has a change. It takes a list of tuples now, rather than the old tuple of tuples. This fixed for me.
##### old #####
ADMINS = (
("Your Name", "your_email#company.example")
)
##### new #####
ADMINS = [
("Your Name", "your_email#company.example")
]
Re: https://docs.djangoproject.com/en/2.1/ref/settings/#admins
Try this
# ./manage shell
>>> from django.core.mail import send_mail
>>> send_mail('Subject here', 'Here is the message.', 'from#example.com',['to#example.com'], fail_silently=False)
With a to#example.com that you actually get email at.
Make sure you have DEBUG = False
This problem annoyed me sufficiently to motivate a post. I provide here the steps I took to resolve this problem (cutting a long story short):
Set-up test page to fail (by re-naming test_template.html)
Check email validations through views for test page in production using send_mail('Hello', 'hello, world', info#xyz.example, [('Name', 'name.name#xyz.example'),], fail_silently=False) where SERVER_EMAIL = info#xyz.example and ADMINS = [('Name', 'name.name#xyz.example'),] in Django settings. In my case, I received the 'hello world' email, but not the Django admin email (which was a pain).
Set-up a simple custom logger to report to a file on the server:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'errors_file': {
'level': 'ERROR',
'class': 'logging.FileHandler',
'filename': 'logs/debug.log',
},
},
'loggers': {
'django': {
'handlers': ['errors_file'],
'level': 'ERROR',
'propagate': True,
},
},
}
In my case, navigating to the test page did not generate output in the debug.log file under the logs directory from my project root directory. This indicates that the logger was failing to reach an ERROR 'level'.
Downgrade the threshold for reporting for the custom logger from ERROR to DEBUG. Now, navigating to the test page should deliver some detail. Inspecting this detail revealed in my case that the default 500 page was re-directed (inadvertedly) to an alternative template file called 500.html. This template file made use of a variable for caching, and as the template was not being called through a view that made the variable available in the context, the cache call failed with a missing key reference. Re-naming 500.html solved my problem.
Although it's been a while, here's my response, so that other people can benefit in the future.
In my case, what was preventing emails to be sent to the ADMINS list, when an error occured, was an application specific setting. I was using django-piston, which provides the setting attributes PISTON_EMAIL_ERRORS and PISTON_DISPLAY_ERRORS. Setting these accordingly, enabled the application server to notify my by mail, whenever piston would crash.
... and then there's the facepalm error, when you've used this in development to prevent emails from going out, and then accidentally copy the setting to production:
# Print emails to console
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
(of course you don't see them being printed to console when using a wsgi server). Removing the setting from production fixed this for me.
And yet another thing that can go wrong (I'll just add it to the list, for those people that end up here despite all the great answers above):
Our django setup used SendGrid as the smtp host and had a single admin email-address defined in the django settings. This worked fine for some time, but at some point, mails stopped arriving.
As it turns out, the mail address ended up in the SendGrid 'Bounced' list for some unknown reason, causing emails to that address to be silently dropped forever after. Removing the address from that list, and whitelisting it, fixed the issue.
If you are using or would want to use SendGrid, use the settings below in production.
Install the package
pip install sendgrid-django
Add these settings in settings.py(production)
DEBUG = False
EMAIL_BACKEND = "sendgrid_backend.SendgridBackend"
SENDGRID_API_KEY = "That you generate in sendgrid account"
ADMINS = (
("Your Name", "your_email#company.example")
)
While likely not ideal, I have found using Gmail as the SMTP host works just fine. There is a useful guide at nathanostgard.com.
Feel free to post your relevant settings.py sections (including EMAIL_*, SERVER_EMAIL, ADMINS (just take out your real email), MANAGERS, and DEBUG) if you want an extra set of eyes to check for typos!
For what it's worth I had this issue and none of these suggestions worked for me. It turns out that my problem was that SERVER_EMAIL was set to an address that the server (Webfaction) didn't recognise. If this site were hosted on Webfaction (as my other sites are), this wouldn't be a problem, but as this was on a different server, the Webfaction servers not only check the authentication of the email being sent, but also the From: value as well.
In my case, it's the include_html in mail_admins.
When I set include_html to True,the email server reject to send my email because it think that my emails are spam.
Everything works just fine when I set include_html to False.
I had the same problem and it turned out the mail server did not have the domain name of the email address. I was trying to send from a registered one (it was a new site for a different part of the business). I used an email address that was already valid under the old domain in SERVER_EMAIL. That resolved my issue.
The below info is given in https://docs.djangoproject.com/en/2.1/howto/error-reporting/#email-reports
EMAIL_HOST = "email host"
EMAIL_HOST_USER = "Email username"
EMAIL_HOST_PASSWORD = "Email Password"
DEBUG = False
ADMINS = (
("Your Name", "your_email#company.example")
)
In order to send email, Django requires a few settings telling it how
to connect to your mail server. At the very least, you’ll need to
specify EMAIL_HOST and possibly EMAIL_HOST_USER and
EMAIL_HOST_PASSWORD, though other settings may be also required
depending on your mail server’s configuration. Consult the Django
settings documentation for a full list of email-related settings.
The setting for server_email has a default of root#localhost but so many email carriers don't accept email from such email_servers and that's why the admin emails is not receiving the emails.
This first answer from this trend helped me
Sending email from DebuggingServer localhost:1025 not working
Or you change your DEFAULT_FROM_EMAIL to something else.