Flask_MongoEngine doesn't work on Webfaction? - python

I'm currently trying to deploy a website built with the Python Flask framework to Webfaction. I've deployed a MongoDB to which I can succesfully connect from the MongoDB interactive interpreter:
[celli#web498 ~]$ mongo localhost:18209/admin
MongoDB shell version: 3.0.2
connecting to: localhost:18209/admin
> db.auth("awesomeusername", "anawesomepassword")
1
When starting the website, it can successfully connect using these credentials in my flask config.py file:
MONGODB_SETTINGS = {
'db': 'tc',
'port': 18209,
'username': 'awesomeusername',
'password': 'anawesomepassword',
}
But when I visit a page which does a query I get the error below. This code works perfectly well on another server, so the mistake lies somewhere in either the setup of my MongoDB instance, or in the connection to it. I also find it strange that setting up the connection works fine, but querying it creates this Authentication failed error.
Any idea how I can fix this? All tips are welcome!
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/flask_login.py", line 758, in decorated_view
return func(*args, **kwargs)
File "/home/celli/webapps/tc/tc/app/views/webviews.py", line 220, in myTickets
userDocs = UserDocument.objects(id__in=[ticket.doc_id for ticket in userTickets])
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/mongoengine/queryset/manager.py", line 37, in __get__
queryset = queryset_class(owner, owner._get_collection())
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/mongoengine/document.py", line 168, in _get_collection
db = cls._get_db()
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/mongoengine/document.py", line 162, in _get_db
return get_db(cls._meta.get("db_alias", DEFAULT_CONNECTION_NAME))
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/mongoengine/connection.py", line 143, in get_db
source=conn_settings['authentication_source'])
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/pymongo/database.py", line 978, in authenticate
self.connection._cache_credentials(self.name, credentials)
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/pymongo/mongo_client.py", line 467, in _cache_credentials
auth.authenticate(credentials, sock_info, self.__simple_command)
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/pymongo/auth.py", line 475, in authenticate
auth_func(credentials[1:], sock_info, cmd_func)
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/pymongo/auth.py", line 450, in _authenticate_default
return _authenticate_scram_sha1(credentials, sock_info, cmd_func)
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/pymongo/auth.py", line 238, in _authenticate_scram_sha1
sasl_start, sasl_continue)
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/pymongo/auth.py", line 185, in _scram_sha1_conversation
res, _ = cmd_func(sock_info, source, sasl_start)
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/pymongo/mongo_client.py", line 703, in __simple_command
helpers._check_command_response(response, None, msg)
File "/home/celli/webapps/tc/venv/lib/python2.7/site-packages/pymongo/helpers.py", line 182, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
OperationFailure: command SON([('saslStart', 1), ('mechanism', 'SCRAM-SHA-1'), ('autoAuthorize', 1), ('payload', Binary('n,,n=awesomeusername,r=NjAxNTEwMjc5MzAy', 0))]) on namespace tc.$cmd failed: Authentication failed.

MongoDB's users are per-database. In your example you show that you connect to the admin database, but the error message is simply telling you that your user is trying to authenticate with the tc database. Probably your user doesn't have a role on tc.
(The other possibility is that tc is not the database you want to connect to; you need to specify the correct database name in your connection params/string).

Related

Django occassionally and temporarily cant connect to database. It is causing requests to it to temporarily timeout and fail

Setup:
AWS RDS db (postgres), Dockerised django backend hosted on elastic beanstalk
Some context:
I have a django backend that accepts form post requests from a separate website, and django saves the form data to the database. The django app I'm creating is replacing existing software that is used for this purpose. I started testing the django app by having the website send the form data to BOTH the existing software and my django app. I noticed that occassionally my django app will not receive a form that the existing software receives. E.g django will finish the day with 25 forms but the existing software will finish with 26. I have request logging setup, but even the logs don't show any records of the 26th form.
To debug what was going wrong, I set up a script command within the django app to send forms to my django app endpoint every minute. It eventually failed:
Exception retrieving data fields
SSL SYSCALL error: EOF detected
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
psycopg2.OperationalError: SSL SYSCALL error: EOF detected
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/code/src/leads/management/commands/bug_finder.py", line 115, in handle
data_field = DataField.objects.get(api_field=field)
File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 431, in get
num = len(clone)
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 262, in __len__
self._fetch_all()
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1324, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 51, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 98, in execute
return super().execute(sql, params)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
django.db.utils.OperationalError: SSL SYSCALL error: EOF detected
[DJANGO] DEBUG 2022-09-08 22:43:09,567 connectionpool urllib3.connectionpool._new_conn:1003: Starting new HTTPS connection (1): url:443
[DJANGO] DEBUG 2022-09-08 22:43:09,730 connectionpool urllib3.connectionpool._make_request:456: https://url:443 "POST /new_lead/5kbLWVNXHQ3U6lJ9trME8hPSf HTTP/1.1" 400 799
So it seems like django is unable to connect with the database to perform queries/saves sometimes.
My local script exception occurred when trying to access datafield models from the database to generate a fake payload for the form:
payload = {}
for field in api_fields:
try:
data_field = DataField.objects.get(api_field=field)
payload[field] = generate_value(data_field)
except DataField.DoesNotExist:
continue
So I guess sometimes when requests are made from the website, django cant connect with the db, it raises this exception, doesnt save the form, and fails to send a response to the website.
When searching the above exception, most answers seem to suggest to relate to persistent db connection setups that have improper timeout settings, which I don't think relates to django request response system.
Is this normal, is it due to AWS going down? Would a solution be to wrap the endpoint's view in a try catch for this exception and try to store the form data in cache until django can connect with the DB again? is there an easy way to test whether django is connected to the db?
Something like a wrapper that effectively does:
try:
view_function()
except psycopg2.OperationalError:
counter = 0
while counter <= 3:
if django.is_connected_to_db:
return view_function()
else:
counter += 1
time.sleep(20)
save_request_to_cache_to_try_later
return response
Seems like I shouldn't have to be doing this though
For your question regarding checking the database connection in Django, you can use this code wherever you want:
from django.db import connections
from django.db.utils import OperationalError
try:
db_conn = connections['default']
except OperationalError:
time.sleep(20)
About the database problem:
Which region are you exactly using on AWS? I have deployed multiple projects on us-east-1 and eu-central-1 and didn't have the problem.
This problem seems unusual, and you should send a ticket to AWS support about that. Anyway, you can try migrating your RDS to the Elastic Beanstalk managed databas. It is basically the same as normal RDS, but the network configuration is easier. Just make sure that RDS is in the same VPC as your instances.

Random NoTransaction in Pyramid

I'm having trouble identifying the source of transaction.interfaces.NoTransaction errors within my Pyramid App. I don't see any patterns to when the error happens, so to me it's quite random.
This app is a (semi-) RESTful API and uses SQLAlchemy and MySQL. I'm currently running within a docker container that connects to an external (bare metal) MySQL instance on the same host OS.
Here's the stack trace for a login attempt within the App. This error happened right after another login attempt that was actually successful.
2020-06-15 03:57:18,982 DEBUG [txn.140501728405248:108][waitress-1] new transaction
2020-06-15 03:57:18,984 INFO [sqlalchemy.engine.base.Engine:730][waitress-1] BEGIN (implicit)
2020-06-15 03:57:18,984 DEBUG [txn.140501728405248:576][waitress-1] abort
2020-06-15 03:57:18,985 ERROR [waitress:357][waitress-1] Exception while serving /auth
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/waitress/channel.py", line 350, in service
task.service()
File "/usr/local/lib/python3.8/site-packages/waitress/task.py", line 171, in service
self.execute()
File "/usr/local/lib/python3.8/site-packages/waitress/task.py", line 441, in execute
app_iter = self.channel.server.application(environ, start_response)
File "/usr/local/lib/python3.8/site-packages/pyramid/router.py", line 270, in __call__
response = self.execution_policy(environ, self)
File "/usr/local/lib/python3.8/site-packages/pyramid_retry/__init__.py", line 127, in retry_policy
response = router.invoke_request(request)
File "/usr/local/lib/python3.8/site-packages/pyramid/router.py", line 249, in invoke_request
response = handle_request(request)
File "/usr/local/lib/python3.8/site-packages/pyramid_tm/__init__.py", line 178, in tm_tween
reraise(*exc_info)
File "/usr/local/lib/python3.8/site-packages/pyramid_tm/compat.py", line 36, in reraise
raise value
File "/usr/local/lib/python3.8/site-packages/pyramid_tm/__init__.py", line 135, in tm_tween
userid = request.authenticated_userid
File "/usr/local/lib/python3.8/site-packages/pyramid/security.py", line 381, in authenticated_userid
return policy.authenticated_userid(self)
File "/opt/REDACTED-api/REDACTED_api/auth/policy.py", line 208, in authenticated_userid
result = self._authenticate(request)
File "/opt/REDACTED-api/REDACTED_api/auth/policy.py", line 199, in _authenticate
session = self._get_session_from_token(token)
File "/opt/REDACTED-api/REDACTED_api/auth/policy.py", line 320, in _get_session_from_token
session = service.get(session_id)
File "/opt/REDACTED-api/REDACTED_api/service/__init__.py", line 122, in get
entity = self.queryset.filter(self.Meta.model.id == entity_id).first()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3375, in first
ret = list(self[0:1])
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3149, in __getitem__
return list(res)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3481, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3502, in _execute_and_instances
conn = self._get_bind_args(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3517, in _get_bind_args
return fn(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3496, in _connection_from_session
conn = self.session.connection(**kw)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1138, in connection
return self._connection_for_bind(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1146, in _connection_for_bind
return self.transaction._connection_for_bind(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 458, in _connection_for_bind
self.session.dispatch.after_begin(self.session, self, conn)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/event/attr.py", line 322, in __call__
fn(*args, **kw)
File "/usr/local/lib/python3.8/site-packages/zope/sqlalchemy/datamanager.py", line 268, in after_begin
join_transaction(
File "/usr/local/lib/python3.8/site-packages/zope/sqlalchemy/datamanager.py", line 233, in join_transaction
DataManager(
File "/usr/local/lib/python3.8/site-packages/zope/sqlalchemy/datamanager.py", line 89, in __init__
transaction_manager.get().join(self)
File "/usr/local/lib/python3.8/site-packages/transaction/_manager.py", line 91, in get
raise NoTransaction()
transaction.interfaces.NoTransaction
The trace shows that the execution eventually reaches my project, but only my custom authentication policy. And it fails right where the database should be queried for the user.
What intrigues me here is the third line on the stack trace. It seems Waitress somehow aborted the transaction it created? Any clue why?
EDIT: Here's the code where that happens: policy.py:320
def _get_session_from_token(self, token) -> UserSession:
try:
session_id, session_secret = self.parse_token(token)
except InvalidToken as e:
raise SessionNotFound(e)
service = AuthService(self.dbsession, None)
try:
session = service.get(session_id) # <---- Service Class called here
except NoResultsFound:
raise SessionNotFound("Invalid session found Request headers. "
"Session id: %s".format(session_id))
if not service.check_session(session, session_secret):
raise SessionNotFound("Session signature does not match")
now = datetime.now(tz=pytz.UTC)
if session.validity < now:
raise SessionNotFound(
"Current session ID {session_id} is expired".format(
session_id=session.id
)
)
return session
And here is an a view on the that service class method:
class AuthService(ModelService):
class Meta:
model = UserSession
queryset = Query(UserSession)
search_fields = []
order_fields = [UserSession.created_at.desc()]
# These below are from the generic ModelClass father class
def __init__(self, dbsession: Session, user_id: str):
self.user_id = user_id
self.dbsession = dbsession
self.Meta.queryset = self.Meta.queryset.with_session(dbsession)
self.logger = logging.getLogger("REDACTED")
#property
def queryset(self):
return self.Meta.queryset
def get(self, entity_id) -> Base:
entity = self.queryset.filter(self.Meta.model.id == entity_id).first()
if not entity:
raise NoResultsFound(f"Could not find requested ID {entity_id}")
As you can see, the there's already some exception treatment. I really don't see what other exception I could try to catch on AuthService.get
I found the solution to be much simpler than tinkering inside Pyramid or SQLAlchemy.
Debugging my Authentication Policy closely, I found out that my it was keeping a sticky reference for the dbsession. It was stored on the first request ever who used it, and never released.
The first request works as expected, the following one fails: My understanding is that the object is still in memory while the app is running, and after the initial transaction is closed. The second request has a new connection, and a new transaction, but the object in memory still points to the previous one, that when used ultimately causes this.
What I don't understand is why the exception didn't happen sometimes. As I mentioned initially, it was seemingly random.
Another thing that I struggled with was in writing a test case to expose the issue. On my tests, the issue never happens because I have (and I've never seen it done differently) a single connection and a single transaction throughout the entire testing session, as opposed of a new connection/transaction per request, so I have not found no way to actually reproduce.
Please let me know if that makes sense, and if you can shed a light on how to expose the bug on a test case.

Pymongo pymongo.errors.ServerSelectionTimeoutError when using example code

I am trying to run very straightforward code to figure out how to use pymongo with the MongoDB Atlas Cloud.
Here is the example code
import pymongo
client = pymongo.MongoClient("mongodb+srv://{myusername}:{mypassword}#cluster0-uywu8.mongodb.net/test?retryWrites=true&w=majority")
db = client.BroadwayMatch
print(db)
collection = db.Artists
print(collection)
print(collection.insert_one({'x': 1}))
BroadwayMatch and Artists are existing databases and collections that I was able to insert to last week, I am not sure what changed. It seems to be connecting to the database and collection successfully, but for some reason is unable to read or write to it. All attributes of collections can be accessed, but all methods result in a ServerSelectionTimeoutError. Here is the output from this snippet
Database(MongoClient(host=['cluster0-shard-00-01-uywu8.mongodb.net:27017', 'cluster0-shard-00-00-uywu8.mongodb.net:27017', 'cluster0-shard-00-02-uywu8.mongodb.net:27017'], document_class=dict, tz_aware=False, connect=True, authsource='admin', replicaset='Cluster0-shard-0', ssl=True, retrywrites=True, w='majority'), 'BroadwayMatch')
Collection(Database(MongoClient(host=['cluster0-shard-00-01-uywu8.mongodb.net:27017', 'cluster0-shard-00-00-uywu8.mongodb.net:27017', 'cluster0-shard-00-02-uywu8.mongodb.net:27017'], document_class=dict, tz_aware=False, connect=True, authsource='admin', replicaset='Cluster0-shard-0', ssl=True, retrywrites=True, w='majority'), 'BroadwayMatch'), 'Artists')
Traceback (most recent call last):
File "C:\Python37\Spotify-Match\mongotest.py", line 10, in <module>
print(collection.insert_one({'x': 1}))
File "C:\Python37\lib\site-packages\pymongo\collection.py", line 700, in insert_one
session=session),
File "C:\Python37\lib\site-packages\pymongo\collection.py", line 614, in _insert
bypass_doc_val, session)
File "C:\Python37\lib\site-packages\pymongo\collection.py", line 602, in _insert_one
acknowledged, _insert_command, session)
File "C:\Python37\lib\site-packages\pymongo\mongo_client.py", line 1279, in _retryable_write
with self._tmp_session(session) as s:
File "C:\Python37\lib\contextlib.py", line 112, in __enter__
return next(self.gen)
File "C:\Python37\lib\site-packages\pymongo\mongo_client.py", line 1611, in _tmp_session
s = self._ensure_session(session)
File "C:\Python37\lib\site-packages\pymongo\mongo_client.py", line 1598, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File "C:\Python37\lib\site-packages\pymongo\mongo_client.py", line 1551, in __start_session
server_session = self._get_server_session()
File "C:\Python37\lib\site-packages\pymongo\mongo_client.py", line 1584, in _get_server_session
return self._topology.get_server_session()
File "C:\Python37\lib\site-packages\pymongo\topology.py", line 434, in get_server_session
None)
File "C:\Python37\lib\site-packages\pymongo\topology.py", line 200, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed
I'm not sure what I am doing wrong, can anyone help?
Reasons you might not be able to connect to an Atlas server:
Your white list has not enable your current IP address
you are using the wrong username and/or password. In your case it looks like your fString is missing an f at the start.
When diagnosing these conditions cutting and pasting the MongoDB Atlas connection string (see below) for the MongoDB shell or MongoDB Compass can often expose username and/or password errors.
Either your mongo server is not exposed otherwise it is not in the default port. Try following:
import pymongo
client = pymongo.MongoClient("mongodb://uname:pass#ip:port/")
db = client['BroadwayMatch']

authendication failed when connecting to mongodb in mlab

I am trying to connect to my mongodb in mlab.com using flask but when I run my flask script I am getting Authentication failed error, please help.
my code:
from flask import Flask, jsonify, request
from flask_pymongo import PyMongo
app = Flask(__name__)
app.config['MONGO_DBNAME'] = 'mydb'
app.config['MONGO_URI'] = 'mongodb://user:pwd#ds157799.mlab.com:57799/mydb'
mongo = PyMongo(app)
#app.route('/framework', methods=['GET'])
def get_all_frameworks():
framework = mongo.db.framework
output = []
for q in framework.find():
output.append({'name' : q['name'], 'language' : q['language']})
return jsonify({'result' : output})
Error:
File "mongo.py", line 12, in <module>
mongo = PyMongo(app)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\flask_pymongo\__init__.py", line 97, in __init__
self.init_app(app, config_prefix)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\flask_pymongo\__init__.py", line 283, in init_app
mechanism=auth_mechanism)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\database.py", line 1167, in authenticate
connect=True)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\mongo_client.py", line 588, in _cache_credentials
sock_info.authenticate(credentials)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\pool.py", line 620, in authenticate
auth.authenticate(credentials, self)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\auth.py", line 486, in authenticate
auth_func(credentials, sock_info)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\auth.py", line 237, in _authenticate_scram_sha1
res = sock_info.command(source, cmd)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\pool.py", line 517, in command
collation=collation)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\network.py", line 125, in command
parse_write_concern_error=parse_write_concern_error)
File "C:\Users\ELCOT\AppData\Local\Programs\Python\Python35\lib\site-packages\pymongo\helpers.py", line 145, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: Authentication failed.
Please help to resolve this.
You can use this as an alternative to sign in to your database. This is tested and works :
connection = pymongo.MongoClient(HOST, PORT)
db = connection[databasename]
db.authenticate(database_user, database_pass)
I created a new user with password for my db, and worked fine.
https://www.reddit.com/r/flask/comments/5ftqvm/how_to_use_pymongo_with_hosted_mongodb_mlab/
This Works fine. For me connecting via URI string from mLab database doesn't work as well.

Tornado authenticating to MongoDB using Motor

I made a tornado app that is supposed to run some queries against a mongodb instance running on another machine. To this purpose I have set up mongodb with authentication and users. I have checked that everything works and that I can authenticate to tornado using the Robo 3T app and a small synchronous script that uses pymongo.
This is how I initialize my tornado application:
class API(tornado.web.Application):
def __init__(self):
settings = dict(
autoreload=True,
compiled_template_cache=False,
static_hash_cache=False,
serve_traceback=True,
cookie_secret="secret",
xsrf_cookies=True,
static_path=os.path.join(os.path.dirname(__file__), "media"),
template_loader=tornado.template.Loader('./templates')
)
mongohost = os.environ.get('MONGOHOST', 'localhost')
mongoport = os.environ.get('MONGOPORT', 27017)
mongouser = os.environ.get('MONGOUSER')
mongopass = os.environ.get('MONGOPASS')
mongodb = os.environ.get('MONGODB')
mongouri = f'mongodb://{mongouser}:{mongopass}#{mongohost}:{mongoport}/{mongodb}'
self.client = motor.motor_tornado.MotorClient(mongouri)
logging.info('connected to mongodb')
self.db = self.client.get_default_database()
logging.info('got mongodb database')
tornado.web.Application.__init__(self, url_patterns, **settings)
def main():
port = 8888
FORMAT = ('%(asctime)s %(levelname) -10s %(name) -30s %(funcName) -35s %(lineno) -5d: %(message)s')
logging.basicConfig(level=logging.DEBUG, format=FORMAT)
tornado.ioloop.IOLoop.configure(TornadoUvloop)
app = API()
app.listen(port)
signal.signal(signal.SIGINT, sig_exit)
logging.info('Tornado server started on port %s' % port)
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
Everything appears to run until one of my handlers that actually performs queries against the database is hit. Code from the handler looks like this:
cursor = self.application.db['events'].find(
find,
projection
).limit(perpage).skip(page*perpage)
buffsize = 0
try:
while (yield cursor.fetch_next):
message = cursor.next_object()
self.write(json.dumps(message, default=json_util.default))
buffsize += 1
if buffsize >= 10:
buffsize = 0
yield self.flush()
yield self.flush()
except Exception:
logging.error('Could not connect to mongodb', exc_info=True)
This code worked just fine before trying to use authentication but now it raises exceptions and stoped working:
017-12-12 13:00:20,718 INFO root __init__ 37 : connected to mongodb
2017-12-12 13:00:20,718 INFO root __init__ 39 : got mongodb database
2017-12-12 13:00:20,723 INFO root main 67 : Tornado server started on port 8888
2017-12-12 13:00:25,226 INFO tornado.general _check_file 198 : /Users/liviu/Documents/Work/api_v2/src/api_tmp/handlers/event_list.py modified; restarting server
2017-12-12 13:00:25,469 INFO root __init__ 37 : connected to mongodb
2017-12-12 13:00:25,469 INFO root __init__ 39 : got mongodb database
2017-12-12 13:00:25,472 INFO root main 67 : Tornado server started on port 8888
2017-12-12 13:00:28,152 INFO root get 266 : now querying database
2017-12-12 13:00:28,214 ERROR root get 355 : Could not connect to mongodb
Traceback (most recent call last):
File "/Users/liviu/Documents/Work/api_v2/src/api_tmp/handlers/event_list.py", line 346, in get
while (yield cursor.fetch_next):
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/tornado/gen.py", line 1055, in run
value = future.result()
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/tornado/concurrent.py", line 238, in result
raise_exc_info(self._exc_info)
File "<string>", line 4, in raise_exc_info
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/cursor.py", line 1055, in _refresh
self.__collation))
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/cursor.py", line 892, in __send_message
**kwargs)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/mongo_client.py", line 950, in _send_message_with_response
exhaust)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/mongo_client.py", line 961, in _reset_on_error
return func(*args, **kwargs)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/server.py", line 99, in send_message_with_response
with self.get_socket(all_credentials, exhaust) as sock_info:
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/server.py", line 168, in get_socket
with self.pool.get_socket(all_credentials, checkout) as sock_info:
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/contextlib.py", line 81, in __enter__
return next(self.gen)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/pool.py", line 852, in get_socket
sock_info.check_auth(all_credentials)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/pool.py", line 570, in check_auth
auth.authenticate(credentials, self)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/auth.py", line 486, in authenticate
auth_func(credentials, sock_info)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/auth.py", line 466, in _authenticate_default
return _authenticate_scram_sha1(credentials, sock_info)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/auth.py", line 209, in _authenticate_scram_sha1
res = sock_info.command(source, cmd)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/pool.py", line 477, in command
collation=collation)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/network.py", line 116, in command
parse_write_concern_error=parse_write_concern_error)
File "/Users/liviu/.venv/api/lib/python3.6/site-packages/pymongo/helpers.py", line 210, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: Authentication failed.
2017-12-12 13:00:28,220 INFO tornado.access log_request 2063: 200 GET /event/list/web?starttime=2017-01-01&endtime=2017-02-03T14:00:00&minlatitude=10&maxlatitude=20&minlongitude=10&maxlongitude=20&minmagnitude=2&maxmagnitude=5&mindepth=10&maxdepth=100 (::1) 67.88ms
I was able to find some simple examples on how to log in and authenticate to MongoDB and as far as I can figure, that is exactly how I am also doing it (https://github.com/mongodb/motor/blob/master/doc/examples/authentication.rst).
Can anybody shed some light on what is going on and how to properly authenticate to MongoDB from an actual working tornado application using Motor?
P.S. I am using Python 3.6, tornado 4.5.2 and motor 1.1.
P.P.S. In the meantime I have discovered that using this as the uri makes it work properly:
mongouri = f'mongodb://{mongouser}:{mongopass}#{mongohost}:{mongoport}/admin'
In the above, I replaced {mongodb} with the "admin" db. After the client gets connected and authenticated on the admin database, I can proceed to get_database(mongodb) and it will work properly. If anyone cares to more clearly articulate what is going on I will accept the answer.
From the mongodb docs :
authSource
Specify the database name associated with the user’s credentials.
authSource defaults to the database specified in the connection
string.
That explains why it works if you specify "admin" as target database.
So in your case you should use:
mongouri = f'mongodb://{mongouser}:{mongopass}#{mongohost}:{mongoport}/{mongodb}?authSource=admin'

Categories