We Keep Coding

djongo, unable to connect to remote db on cloud.mongodb.com

i think below setting will work for the djongo to connect to the remote mongodb on mongodb.com but, the error message shows its still trying to connect to the localhost
DATABASES = {
'default': {
'ENGINE': 'djongo',
'HOST': 'mongodb+srv://<username>:<password>#cluster-name/<dbname>?retryWrites=true&w=majority',
}
below is the error traceback
Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "venv/lib/python3.6/site-packages/django/utils/autoreload.py", line 54, in wrapper
fn(*args, **kwargs)
File "venv/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 120, in inner_run
self.check_migrations()
File "venv/lib/python3.6/site-packages/django/core/management/base.py", line 453, in check_migrations
executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
File "venv/lib/python3.6/site-packages/django/db/migrations/executor.py", line 18, in __init__
self.loader = MigrationLoader(self.connection)
File "venv/lib/python3.6/site-packages/django/db/migrations/loader.py", line 49, in __init__
self.build_graph()
File "venv/lib/python3.6/site-packages/django/db/migrations/loader.py", line 212, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "venv/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 73, in applied_migrations
if self.has_table():
File "venv/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 56, in has_table
return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
File "venv/lib/python3.6/site-packages/django/db/backends/base/introspection.py", line 48, in table_names
return get_names(cursor)
File "venv/lib/python3.6/site-packages/django/db/backends/base/introspection.py", line 43, in get_names
return sorted(ti.name for ti in self.get_table_list(cursor)
File "venv/lib/python3.6/site-packages/djongo/introspection.py", line 47, in get_table_list
for c in cursor.db_conn.list_collection_names()
File "venv/lib/python3.6/site-packages/pymongo/database.py", line 856, in list_collection_names
for result in self.list_collections(session=session, **kwargs)]
File "venv/lib/python3.6/site-packages/pymongo/database.py", line 819, in list_collections
_cmd, read_pref, session)
File "venv/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1454, in _retryable_read
read_pref, session, address=address)
File "venv/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1253, in _select_server
server = topology.select_server(server_selector)
File "venv/lib/python3.6/site-packages/pymongo/topology.py", line 235, in select_server
address))
File "venv/lib/python3.6/site-packages/pymongo/topology.py", line 193, in select_servers
selector, server_timeout, address)
File "venv/lib/python3.6/site-packages/pymongo/topology.py", line 209, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused

I have met your problem before. In my case, I run django & mongodb inside containers. And I found that whatever setting I set inside setting.py, the djongo(they use pymongo behind) just ignore my setting.
For others seeking for the solution, you may want to check your syntax in setting.py which should look like below:
DATABASES = {
'default': {
'ENGINE': 'djongo',
'NAME': 'your-db-name',
'ENFORCE_SCHEMA': False,
'CLIENT': {
'host': 'host-name or ip address',
'port': port_number,
'username': 'db-username',
'password': 'password',
'authSource': 'db-name',
'authMechanism': 'SCRAM-SHA-1'
},
.......
}
Detail from djongo document
You can see the host, user, password fields are all under 'CLIENT' , which is different with the default syntax described by Django. So maybe most people get confused here and stuck at this problem.

Go to settings.py file and use below Database settngs
DATABASES = {
'default': {
'ENGINE': 'djongo',
'ENFORCE_SCHEMA': True
'NAME': 'your-db-name',
'HOST': 'host-name or ip address',
'PORT': port_number,
'USER': 'db-username',
'PASSWORD': 'password',
'AUTH_SOURCE': 'db-name',
'AUTH_MECHANISM': 'SCRAM-SHA-1',
}
Please also check out this article for connection between django and mongo db.
https://medium.com/#ksarthak4ever/how-to-use-django-with-mongodb-40ba36a21124
I personally recommend you please use pycharm. In Pycharm you can test your connection. So that you can get more clear image of whole scenario.
I use remote database in django. you can see in the below picture

I GOT THE SOLUTION
I adopted following approach >
so, behind the scene the djongo uses -> pymongo
and pymongo's default configuration are
class MongoClient(common.BaseObject):
HOST = "localhost" # here HOST has the hardcoded value
PORT = 27017
which lies in following file
venv/lib/python3.6/site-packages/pymongo/mongo_client.py
replace the harcoded value of HOST to something like
HOST = 'mongodb+srv://<username>:<password>#cluster-name/<dbname>?retryWrites=true&w=majority'
ADDITIONALLY
We can set the environment variable if we want
HOST = os.getenv('MONGO_DB_URL')

django.core.exceptions.ImproperlyConfigured: The database name '****.amazonaws.com' (85 characters) is longer than PostgreSQL's limit of 63 characters

I am trying to migrate my Django model to the AWS Lightsail database, but I am getting below error. My database is on AWS Lightsail. it is PostgreSQL. I am deploying my project on AWS so I have set up almost everything except model migration.
AMAZON DATABASE AVAILABLE CONNECTION OPTIONS:
Endpoint-> ************.ap-******-1.***.amazonaws.com,
Port-> 5444,
User name-> db,
Password-> Something
MY SETTINGS.PY FILE DATABASE CONNECTION OPTIONS:
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': '****Endpoint*****',
'USER': 'db',
'PASSWORD': 'Something',
'HOST': '5444'
}
}
Terminal Error:
> `2, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/home/bitnami/portfolio/ManojJha-portfolio/.venv/lib/python3.7/site-packages/django/db/migrations/recorder.py", lin
e 76, in applied_migrations
if self.has_table():
File "/home/bitnami/portfolio/ManojJha-portfolio/.venv/lib/python3.7/site-packages/django/db/migrations/recorder.py", lin
e 56, in has_table
return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
File "/home/bitnami/portfolio/ManojJha-portfolio/.venv/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in
inner
return func(*args, **kwargs)
File "/home/bitnami/portfolio/ManojJha-portfolio/.venv/lib/python3.7/site-packages/django/db/backends/base/base.py", line
260, in cursor
return self._cursor()
File "/home/bitnami/portfolio/ManojJha-portfolio/.venv/lib/python3.7/site-packages/django/db/backends/base/base.py", line
236, in _cursor
self.ensure_connection()
File "/home/bitnami/portfolio/ManojJha-portfolio/.venv/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in
inner
return func(*args, **kwargs)
File "/home/bitnami/portfolio/ManojJha-portfolio/.venv/lib/python3.7/site-packages/django/db/backends/base/base.py", line
220, in ensure_connection
self.connect()
File "/home/bitnami/portfolio/ManojJha-portfolio/.venv/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in
inner
return func(*args, **kwargs)
File "/home/bitnami/portfolio/ManojJha-portfolio/.venv/lib/python3.7/site-packages/django/db/backends/base/base.py", line
196, in connect
conn_params = self.get_connection_params()
File "/home/bitnami/portfolio/ManojJha-portfolio/.venv/lib/python3.7/site-packages/django/db/backends/postgresql/base.py"
, line 165, in get_connection_params
self.ops.max_name_length(),
django.core.exceptions.ImproperlyConfigured: The database name '*****************Database End-point*****************.***.amazonaws.com' (85 characters) is longer than PostgreSQL's limit of 63 characters. Supply a shorter NAME in settings.DATABASES.```
So is there any way to shorten the endpoint name or there is any other solution to handle this error. Please guide me to set up this.
Thanks,
-Manoj Jha

"HOST" should be the hostname, not the port!
"NAME" is the name of the database.
An example from the Django documentation:
'ENGINE': 'django.db.backends.oracle',
'NAME': 'xe',
'USER': 'a_user',
'PASSWORD': 'a_password',
'HOST': 'dbprod01ned.mycompany.com',
'PORT': '1540',

Mongo + Postgres + Django Testing Error - Multiple Databases (Django-MongoDB-Engine)

I have been trying to get my django testing working for an API I am building. I am newer to Python and have been stuck on this for a little while.
Some quick facts:
I am using 2 databases - A postgresql database for my users and a mongo database for everything else.
I am using TastyPie, Django-MongoDB-Engine, Djangotoolbox, Django-Non-rel, Pymongo and psycopg2
I have successfully connected to the databases. When I save an auth_user from the shell it saves to the sql database and the models from my app save to the mongoDB
I have to integrate this with another API so was limited to what my options were for databases and libraries.
My big problem is with testing - I can't get it to work.
When I try to run Django's tests with ./manage.py test, I keep getting this error:
pymongo.errors.OperationFailure: command SON([('authenticate', 1), ('user', u'test'), ('nonce', u'blahblah'), ('key', u'blahblah')]) failed: auth fails
After extensive search, I am still not sure what it is. I know that obviously something is trying to authenticate with my MongoDB and then its not allowing it. I dont mind using this DB to test as it is a testing DB anyways.
Please help! My configuration and error messages are below.
I set up the database settings to include both databases, as so:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'main_db',
'USER': os.environ['RDS_USERNAME'],
'PASSWORD': os.environ['RDS_PASSWORD'],
'HOST': os.environ['RDS_HOSTNAME'],
'PORT': os.environ['RDS_PORT'],
},
'mongo': {
'ENGINE': 'django_mongodb_engine',
'NAME': 'mongo_db',
'USER': os.environ['MONGO_USERNAME'],
'PASSWORD': os.environ['MONGO_PASSWORD'],
'HOST': os.environ['MONGO_HOSTNAME'],
'PORT': os.environ['MONGO_PORT'],
}
}
DATABASE_ROUTERS = ['myproject.database_router.AuthRouter']
As you can see, I created my own custom router:
sql_databases = ['admin', 'auth', 'contenttypes', 'tastypie'
'sessions', 'messages', 'staticfiles']
class AuthRouter(object):
"""
A router to control all database operations on models in the
auth application.
"""
def db_for_read(self, model, **hints):
"""
Attempts to read auth models go to auth_db.
"""
if model._meta.app_label in sql_databases:
return 'default'
return 'mongo'
def db_for_write(self, model, **hints):
"""
Attempts to write auth models go to auth_db.
"""
if model._meta.app_label in sql_databases:
return 'default'
return 'mongo'
def allow_relation(self, obj1, obj2, **hints):
"""
Allow relations only if model does includes auth app.
"""
if obj1._meta.app_label in sql_databases or \
obj2._meta.app_label in sql_databases:
return True
return False
def allow_migrate(self, db, model):
"""
Make sure the auth app only appears in the 'auth_db'
database.
"""
if db == 'default':
return model._meta.app_label in sql_databases
elif model._meta.app_label in sql_databases:
return False
return False
The full trace is here:
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "myproject/djenv/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "myproject/djenv/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "myproject/djenv/lib/python2.7/site-packages/django/core/management/commands/test.py", line 50, in run_from_argv
super(Command, self).run_from_argv(argv)
File "myproject/djenv/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "myproject/djenv/lib/python2.7/site-packages/django/core/management/commands/test.py", line 71, in execute
super(Command, self).execute(*args, **options)
File "myproject/djenv/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "myproject/djenv/lib/python2.7/site-packages/django/core/management/commands/test.py", line 88, in handle
failures = test_runner.run_tests(test_labels)
File "myproject/djenv/lib/python2.7/site-packages/django/test/runner.py", line 145, in run_tests
old_config = self.setup_databases()
File "myproject/djenv/lib/python2.7/site-packages/django/test/runner.py", line 107, in setup_databases
return setup_databases(self.verbosity, self.interactive, **kwargs)
File "myproject/djenv/lib/python2.7/site-packages/django/test/runner.py", line 279, in setup_databases
verbosity, autoclobber=not interactive)
File "myproject/djenv/lib/python2.7/site-packages/django_mongodb_engine/creation.py", line 196, in create_test_db
self.connection._reconnect()
File "myproject/djenv/lib/python2.7/site-packages/django_mongodb_engine/base.py", line 279, in _reconnect
self._connect()
File "myproject/djenv/lib/python2.7/site-packages/django_mongodb_engine/base.py", line 268, in _connect
if not self.database.authenticate(user, password):
File "myproject/djenv/lib/python2.7/site-packages/pymongo/database.py", line 891, in authenticate
self.connection._cache_credentials(self.name, credentials)
File "myproject/djenv/lib/python2.7/site-packages/pymongo/mongo_client.py", line 459, in _cache_credentials
auth.authenticate(credentials, sock_info, self.__simple_command)
File "myproject/djenv/lib/python2.7/site-packages/pymongo/auth.py", line 243, in authenticate
auth_func(credentials[1:], sock_info, cmd_func)
File "myproject/djenv/lib/python2.7/site-packages/pymongo/auth.py", line 222, in _authenticate_mongo_cr
cmd_func(sock_info, source, query)
File "myproject/djenv/lib/python2.7/site-packages/pymongo/mongo_client.py", line 690, in __simple_command
helpers._check_command_response(response, None, msg)
File "myproject/djenv/lib/python2.7/site-packages/pymongo/helpers.py", line 178, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: command SON([('authenticate', 1), ('user', u'test'), ('nonce', u'blahblah'), ('key', u'blahblah')]) failed: auth fails
Any help is appreciated!

Wanted to give a update on how I fixed this - I thought it was a read/write issue (with the auth fails issue), but it was not.
If you see this error, its an authentication issue - Either your username, password, database name or collection name is incorrect.
pymongo.errors.OperationFailure: command SON([('authenticate', 1), ('user', u'test'), ('nonce',
u'blahblah'), ('key', u'blahblah')]) failed: auth fails
Django when setting up a test database adds 'test_' to the beginning of the database name. So my main database, called maindb is then created as 'test_maindb'.
You can see info regarding this at: Test Databases and Django
Ensure that you can write to a separate database that is named as your database name with 'test_' appended to the name.
Also - an alternative solution is to define test database setting in your database settings. This can be done by appending 'TEST_' to any of the database attributes. An example inlcudes:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'maindb',
'USER': os.environ['RDS_USERNAME'],
'PASSWORD': os.environ['RDS_PASSWORD'],
'HOST': os.environ['RDS_HOSTNAME'],
'PORT': os.environ['RDS_PORT'],
'TEST_NAME': 'my_test_sql',
'TEST_USER': 'test_sql_user',
'TEST_PASSWORD': 'password'
},
'mongo': {
'ENGINE': 'django_mongodb_engine',
'NAME': 'mongodb',
'USER': os.environ['MONGO_USERNAME'],
'PASSWORD': os.environ['MONGO_PASSWORD'],
'HOST': os.environ['MONGO_HOSTNAME'],
'PORT': os.environ['MONGO_PORT'],
'TEST_NAME': 'my_test_mongodb',
'TEST_USER': 'test_mongo_user',
'TEST_PASSWORD': 'password'
}
}
Hope it helps! And please give feedback if you see anything additional

Using postgresql for Django under win7?

I am currently playing with django and want to use as my database postgresql:
and these are my configurations in my settings file:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql', # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
'NAME': 'django', # Or path to database file if using sqlite3.
'USER': 'postgres', # Not used with sqlite3.
'PASSWORD': 'postgres', # Not used with sqlite3.
'HOST': '', # Set to empty string for localhost. Not used with sqlite3.
'PORT': '5432', # Set to empty string for default. Not used with sqlite3.
}
}
I also tried it with postgresql_psycopg2 and also installed psycopg2(http://www.stickpeople.com/projects/python/win-psycopg/).
Any ideas whats wrong with the configurations?
UPDATE:
The Error:
> File
> "C:\Python27\lib\site-packages\django\core\management\__init__.py",
> line 382, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv) File "C:\Python27\lib\site-packages\django\core\management\base.py", line
> 196, in run_from_argv
> self.execute(*args, **options.__dict__) File "C:\Python27\lib\site-packages\django\core\management\base.py", line
> 232, in execute
> output = self.handle(*args, **options) File "C:\Python27\lib\site-packages\django\core\management\base.py", line
> 371, in handle
> return self.handle_noargs(**options) File "C:\Python27\lib\site-packages\django\core\management\commands\syncdb.py",
> line 57, in handle_noargs
> cursor = connection.cursor() File "C:\Python27\lib\site-packages\django\db\backends\dummy\base.py", line
> 15, in complain
> raise ImproperlyConfigured("settings.DATABASES is improperly configured. " django.core.exceptions.ImproperlyConfigured:
> settings.DATABASES is improperly configured. Please supply the ENGINE
> value. Check settings documentation for more details.

This is an old question but I am adding an answer for future visitors. You seem to have several errors:
If you already installed psycopg2, you should change the ENGINE name to django.db.backends.postgresql_psycopg2.
In your settings you specified your database NAME to be django, but in your pgAdmin screenshot it is seen that you don't have such a database. Create a database named django using pgAdmin.
The correct setup would be then:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'django',
'USER': 'postgres',
'PASSWORD': 'postgres',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}

AttributeError: 'Settings' object has no attribute 'DATABASE_ENGINE'

Im new to django and python, trying to run a piece of django code on my system but im running into these problems , im running version 2.7 python and v1.4 django
$ python manage.py runserver
Running in development mode.
Running in development mode.
Running in development mode.
Running in development mode.
Validating models...
HACKUING USER MODEL
Unhandled exception in thread started by <bound method Command.inner_run of <django.contrib.staticfiles.management.commands.runserver.Command object at 0x101981e50>>
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 91, in inner_run
self.validate(display_num_errors=True)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/base.py", line 266, in validate
num_errors = get_validation_errors(s, app)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/validation.py", line 30, in get_validation_errors
for (app_name, error) in get_app_errors().items():
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/loading.py", line 158, in get_app_errors
self._populate()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/loading.py", line 64, in _populate
self.load_app(app_name, True)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/loading.py", line 88, in load_app
models = import_module('.models', app_name)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/Users/Kinnovate/Desktop/fsdjango/platformsite/notices/models.py", line 9, in <module>
from common.fields import PickleField
File "/Users/Kinnovate/Desktop/fsdjango/platformsite/common/fields/__init__.py", line 1, in <module>
from pickle import *
File "/Users/Kinnovate/Desktop/fsdjango/platformsite/common/fields/pickle.py", line 27, in <module>
mysql_backend = settings.DATABASE_ENGINE == 'mysql'
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/functional.py", line 186, in inner
return func(self._wrapped, *args)
AttributeError: 'Settings' object has no attribute 'DATABASE_ENGINE'
this is the part of settings.py relevant to the question
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
'NAME': os.path.join(DIRNAME, 'database.sqlite3'), # Or path to database file if using sqlite3.
'USER': '', # Not used with sqlite3.
'PASSWORD': '', # Not used with sqlite3.
'HOST': '', # Set to empty string for localhost. Not used with sqlite3.
'PORT': '', # Set to empty string for default. Not used with sqlite3.
}
}
how do i fix this?

What is /Users/Kinnovate/Desktop/fsdjango/platformsite/common/fields/pickle.py? Is it your code? Then you have an error in it, because you really don't have DATABASE_ENGINE in your seetings. Use settings.DATABASES['default']['ENGINE'] instead.

You are missing something
Because as your backtrace your database must be mysql.
File "/Users/Kinnovate/Desktop/fsdjango/platformsite/common/fields/pickle.py", line 27, in <module>
mysql_backend = settings.DATABASE_ENGINE == 'mysql'
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/functional.py", line 186, in inner
return func(self._wrapped, *args)
Because its enter in mysql_backend = settings.DATABASE_ENGINE == 'mysql' line.
As per your settings its 'ENGINE': 'django.db.backends.sqlite3', so it must be enter in sqlite please check your app because it might be possible that your settings.py might be refer from another place.