I am starting a Django project and I am using gitlab's ci/cd on the shared runners and I use Postgres as the database.
I have this weird problem that it seems like Django is creating the test database with the username "postgres" and I can't find a way to configure it's setting and change it to use the role named "runner". This causes a break in my ci/cd pipeline.
here is my .gitlab-ci.yml:
image: python:3.6.5
services:
- postgres:latest
variables:
POSTGRES_DB: asdproject
POSTGRES_USER: runner
POSTGRES_PASSWORD: asdpassword
test:
script:
- whoami
- export DATABASE_URL=postgres://postgres:#postgres:5432/asdproject
- export PGPASSWORD=$POSTGRES_PASSWORD
- apt-get update -qy
- apt-get install -y python-dev python-pip
- pip install -r requirements.txt
- python manage.py test - settings=asd.gitlab_runner_settings
and my gitlab_runner_settings.py file:
I tried many forms of changing settings.py that were recommended in questions but neither worked.
from asd.with_debug_settings import *
import sys
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'asdproject',
'USER': 'runner',
'PASSWORD': 'asdpassword',
'HOST': 'postgres',
'PORT': '5432',
'TEST': {
'NAME': 'asdtest',
'USER': 'runner'
},
}
}
The error I get while running my pipeline script in gitlab is:
---------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 83, in _execute
return self.cursor.execute(sql)
psycopg2.ProgrammingError: role "postgres" does not exist
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/test.py", line 26, in run_from_argv
super().run_from_argv(argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/test.py", line 56, in handle
failures = test_runner.run_tests(test_labels)
File "/usr/local/lib/python3.6/site-packages/django/test/runner.py", line 607, in run_tests
self.teardown_databases(old_config)
File "/usr/local/lib/python3.6/site-packages/django_heroku/core.py", line 41, in teardown_databases
self._wipe_tables(connection)
File "/usr/local/lib/python3.6/site-packages/django_heroku/core.py", line 33, in _wipe_tables
"""
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 83, in _execute
return self.cursor.execute(sql)
django.db.utils.ProgrammingError: role "postgres" does not exist
ERROR: Job failed: exit code 1
You need to have a host
'HOST': 'postgres',
this is wrong
It should be either 'localhost' or the address of remote server ip
The error occurs because when gitlab-ci runs tests it creates database according to the settings set in you yml file:
variables:
POSTGRES_DB: asdproject
POSTGRES_USER: runner
POSTGRES_PASSWORD: asdpassword
So runner role and asdproject database is created.
But you set your database url as DATABASE_URL=postgres://postgres:#postgres:5432/asdproject
Where postgres user is set.
Please try DATABASE_URL=postgres://runner:asdpassword#postgres:5432/asdproject
Related
I trying to add test cases for the django application but the test pass successfully on the locally but it fails on the Travis CI.
M .travis.yml files looks like this:
dist: bionic
services:
- postgresql
addons:
postgresql: '9.5'
apt:
packages:
- postgresql-9.5
before_script:
- psql -c 'create database fecundity_test;' -U postgres
branches:
only:
- "master"
language: python
python:
- "3.8"
install:
- if [ "$TRAVIS_BRANCH" = "master" ]; then pip install -r requirements/dev.txt; fi
- if [ "$TRAVIS_BRANCH" = "master" ]; then pip install coveralls; fi
script:
- if [ "$TRAVIS_BRANCH" = "master" ]; then ./scripts/lib/run-ci; fi
after_success:
- if [ "$TRAVIS_BRANCH" = "master" ]; then coveralls; fi
And my settings.py looks like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.environ['POSTGRES_DB'],
'USER': os.environ['POSTGRES_USER'],
'PASSWORD': os.environ['POSTGRES_PASSWORD'],
'HOST': os.environ['POSTGRES_HOST'],
'PORT': '',
'ATOMIC_REQUESTS': True,
'TEST': {
'NAME': os.environ['POSTGRES_TEST_DB']
}
},
}
When I run it on travis is get this error log:
$ if [ "$TRAVIS_BRANCH" = "master" ]; then ./scripts/lib/run-ci; fi
Using existing test database for alias 'default' ('fecundity_test')...
/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/postgresql
/base.py:294: RuntimeWarning: Normally Django will use a connection to the 'postgres' database
to avoid running initialization queries against the production database when it's not needed
(for example, when running tests). Django was unable to create a connection to the 'postgres'
database and will use the first PostgreSQL database instead.
warnings.warn(
Traceback (most recent call last):
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/base/base.py", line 220, in ensure_connection
self.connect()
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/base/base.py", line 197, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 185, in get_new_connection
connection = Database.connect(**conn_params)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "db" to address: Temporary failure in name resolution
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "./manage.py", line 21, in <module>
main()
File "./manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/core/management/commands/test.py", line 23, in run_from_argv
super().run_from_argv(argv)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/core/management/base.py", line 328, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
output = self.handle(*args, **options)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/core/management/commands/test.py", line 53, in handle
failures = test_runner.run_tests(test_labels)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/test/runner.py", line 684, in run_tests
old_config = self.setup_databases(aliases=databases)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/test/runner.py", line 604, in setup_databases
return _setup_databases(
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/test/utils.py", line 169, in setup_databases
connection.creation.create_test_db(
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/base/creation.py", line 58, in create_test_db
self._create_test_db(verbosity, autoclobber, keepdb)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/base/creation.py", line 168, in _create_test_db
with self._nodb_connection.cursor() as cursor:
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/base/base.py", line 260, in cursor
return self._cursor()
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/base/base.py", line 236, in _cursor
self.ensure_connection()
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/base/base.py", line 220, in ensure_connection
self.connect()
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/base/base.py", line 220, in ensure_connection
self.connect()
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/base/base.py", line 197, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 185, in get_new_connection
connection = Database.connect(**conn_params)
File "/home/travis/virtualenv/python3.8.1/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: could not translate host name "db" to address: Temporary failure in name resolution
The command "if [ "$TRAVIS_BRANCH" = "master" ]; then ./scripts/lib/run-ci; fi" exited with 1.
I have no idea what is working, And I tried to search on the internet but I could not find any help. And further CI is passing without tests. Thanks in Advance.
After so much effort I figured out that the reason to fail was that, I did not add EVN variables in Travis CI. If You are getting the same error make you have added the variables in the ENV settings of TravisCI.
I had a problem with my vagrant boxes with ports and everything so I did something that I suspect is inhibiting my ability to run ./manage.py migrate. Here's what it looks like when I run vagrant ssh-config
HostName 127.0.0.1
User vagrant
Port 2222
UserKnownHostsFile /dev/null
StrictHostKeyChecking no
PasswordAuthentication no
IdentityFile /Users/brock1hj/projects/sodium/to-vagrant/.vagrant/machines/default/virtualbox/private_key
IdentitiesOnly yes
LogLevel FATAL
Here is the full error:
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
utility.execute()
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/core/management/__init__.py", line 330, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/core/management/base.py", line 390, in run_from_argv
self.execute(*args, **cmd_options)
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute
output = self.handle(*args, **options)
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 93, in handle
executor = MigrationExecutor(connection, self.migration_progress_callback)
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/migrations/executor.py", line 19, in __init__
self.loader = MigrationLoader(self.connection)
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/migrations/loader.py", line 47, in __init__
self.build_graph()
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/migrations/loader.py", line 180, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 59, in applied_migrations
self.ensure_schema()
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 49, in ensure_schema
if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/backends/base/base.py", line 162, in cursor
cursor = self.make_debug_cursor(self._cursor())
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/backends/base/base.py", line 135, in _cursor
self.ensure_connection()
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/backends/base/base.py", line 130, in ensure_connection
self.connect()
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/backends/base/base.py", line 130, in ensure_connection
self.connect()
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/backends/base/base.py", line 119, in connect
self.connection = self.get_new_connection(conn_params)
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 176, in get_new_connection
connection = Database.connect(**conn_params)
File "/Users/brock1hj/envs/sodium/lib/python2.7/site-packages/psycopg2/__init__.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: FATAL: password authentication failed for user "vagrant"
I'm really not sure what's going on and I think somehow the password for the user "vagrant" somehow got changed from the default "vagrant".
I am using Django 1.8.2
Here is my settings.py file for databases:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'vagrant',
'USER': 'vagrant',
'PASSWORD': 'vagrant',
'HOST': '127.0.0.1',
'PORT': '',
}
}
I feel like it is going to try putting "vagrant" as a password but that is no longer the case.
Trying to enter the dbshell, was prompted to enter the password to ensure "vagrant" didn't work and got:
Password for user vagrant:
psql: error: could not connect to server: could not initiate GSSAPI security context: Unspecified GSS failure.
Minor code may provide more information could not initiate GSSAPI security context: Configuration file does
not specify default realm
FATAL: password authentication failed for user "vagrant"
Try:
vagrant ssh
psql
Then follow instructions on https://www.postgresql.org/docs/9.6/sql-alterrole.html, looks like you want ALTER ROLE vagrant PASSWORD 'vagrant';
I'm using Mac 10.13.6 and MySql 5.5 (can't upgrade at this time). I have just created a skeleton Django (v 3.0.2)/Python 3.7 app and configured my database setting like so
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'OPTIONS': {
'sql_mode': 'traditional',
},
'NAME': 'maps_data',
'USER': 'myuser',
'PASSWORD': 'mypassword',
'HOST': 'localhost',
'PORT': '3306',
}
}
I have used pip to install the mysqlclient ...
(env) localhost:maps davea$ pip install mysqlclient
Requirement already satisfied: mysqlclient in /Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages (1.4.6)
However, when I attempt to run the initial migrations created for me, I get these errors I don't understand ...
(env) localhost:maps davea$ python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
Traceback (most recent call last):
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 74, in execute
return self.cursor.execute(query, args)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/MySQLdb/cursors.py", line 209, in execute
res = self._query(query)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/MySQLdb/cursors.py", line 315, in _query
db.query(q)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(6) NOT NULL)' at line 1")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/migrations/recorder.py", line 67, in ensure_schema
editor.create_model(self.Migration)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 324, in create_model
self.execute(sql, params or None)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/backends/base/schema.py", line 142, in execute
cursor.execute(sql, params)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/backends/utils.py", line 100, in execute
return super().execute(sql, params)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/backends/utils.py", line 86, in _execute
return self.cursor.execute(sql, params)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/backends/mysql/base.py", line 74, in execute
return self.cursor.execute(query, args)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/MySQLdb/cursors.py", line 209, in execute
res = self._query(query)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/MySQLdb/cursors.py", line 315, in _query
db.query(q)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/MySQLdb/connections.py", line 239, in query
_mysql.connection.query(self, query)
django.db.utils.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(6) NOT NULL)' at line 1")
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/core/management/base.py", line 328, in run_from_argv
self.execute(*args, **cmd_options)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/core/management/base.py", line 369, in execute
output = self.handle(*args, **options)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/core/management/commands/migrate.py", line 233, in handle
fake_initial=fake_initial,
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/migrations/executor.py", line 91, in migrate
self.recorder.ensure_schema()
File "/Users/davea/Documents/workspace/chicommons/maps/env/lib/python3.7/site-packages/django/db/migrations/recorder.py", line 69, in ensure_schema
raise MigrationSchemaMissing("Unable to create the django_migrations table (%s)" % exc)
django.db.migrations.exceptions.MigrationSchemaMissing: Unable to create the django_migrations table ((1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(6) NOT NULL)' at line 1"))
What are the right versions of Django and mysqlclient to use, or what else do I need to do to configure things properly?
MySQL 5.5 was dropped in Django 2.1. Here's the changelog.
So the official answer would be to upgrade your old version of MySQL.
...That said, I had a migration fail mid-execution in production because of this, and I was able to resolve it using the following process:
Obtain the raw sql that the migration would run with python manage.py sqlmigrate myapp 0015
Where 'myapp' is the name of your app, and '0015' is the migration number you're trying to run.
Adjust the query as required.
In this case; replace occurrences of 'datetime(6)' with 'datetime'
Run the new query manually.
Tell django that the migration has been run manually by calling python manage.py migrate myapp 0015 --fake
I am trying to connect my django project 'mysite' to mysql. I made a user in mysql and granted it all privileges to access the project. These are the changes I made to settings.py :
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mysitedb',
'USER': 'username',
'PASSWORD': 'password',
'HOST': 'localhost',
'PORT': '',
}
}
Now when I try to migrate the database using python3 manage.py makemigrations, I get the following error:
django.db.utils.OperationalError: (2059, "Authentication plugin 'caching_sha2_password' cannot be loaded: /usr/lib/mysql/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory")</code></pre>
The complete stack trace is as follows:
Traceback (most recent call last):
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
self.connect()
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 194, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/mysql/base.py", line 236, in get_new_connection
return Database.connect(**conn_params)
File "/usr/lib/python3/dist-packages/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 204, in __init__
super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (2059, "Authentication plugin 'caching_sha2_password' cannot be loaded: /usr/lib/mysql/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
utility.execute()
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/core/management/__init__.py", line 365, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/core/management/base.py", line 332, in execute
self.check()
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/core/management/base.py", line 364, in check
include_deployment_checks=include_deployment_checks,
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/core/management/base.py", line 351, in _run_checks
return checks.run_checks(**kwargs)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/core/checks/registry.py", line 73, in run_checks
new_errors = check(app_configs=app_configs)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/core/checks/model_checks.py", line 27, in check_all_models
errors.extend(model.check(**kwargs))
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/models/base.py", line 1200, in check
errors.extend(cls._check_fields(**kwargs))
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/models/base.py", line 1272, in _check_fields
errors.extend(field.check(**kwargs))
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/models/fields/__init__.py", line 894, in check
errors = super().check(**kwargs)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/models/fields/__init__.py", line 206, in check
errors.extend(self._check_backend_specific_checks(**kwargs))
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/models/fields/__init__.py", line 303, in _check_backend_specific_checks
return connections[db].validation.check_field(self, **kwargs)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/base/validation.py", line 21, in check_field
field_type = field.db_type(self.connection)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/models/fields/__init__.py", line 648, in db_type
return connection.data_types[self.get_internal_type()] % data
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/utils/functional.py", line 36, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/mysql/base.py", line 133, in data_types
if self.features.supports_microsecond_precision:
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/utils/functional.py", line 36, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/mysql/features.py", line 65, in supports_microsecond_precision
return self.connection.mysql_version >= (5, 6, 4)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/utils/functional.py", line 36, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/mysql/base.py", line 345, in mysql_version
with self.temporary_connection() as cursor:
File "/usr/lib/python3.5/contextlib.py", line 59, in __enter__
return next(self.gen)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 576, in temporary_connection
cursor = self.cursor()
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 255, in cursor
return self._cursor()
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 232, in _cursor
self.ensure_connection()
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
self.connect()
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
self.connect()
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 194, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/prabhjeet/.local/lib/python3.5/site-packages/django/db/backends/mysql/base.py", line 236, in get_new_connection
return Database.connect(**conn_params)
File "/usr/lib/python3/dist-packages/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 204, in __init__
super(Connection, self).__init__(*args, **kwargs2)
django.db.utils.OperationalError: (2059, "Authentication plugin 'caching_sha2_password' cannot be loaded: /usr/lib/mysql/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory")
How can I resolve this error? I am using python 3 on ubuntu 16.04 platform.
Actually, there is no need to downgrade the MYSQL Server. Follow these two steps and it should work:
Step 1: Change MYSQL configuration to use mysql_native_password. Edit one of ini files that mysqld is using. You can see which one my using the command
mysqld --verbose --help
and make sure this line is added -
default-authentication-plugin=mysql_native_password
Step 2: Create an new or alter the existing user. To create a new with mysql_native_password password:
CREATE USER 'username'#'ip_address' IDENTIFIED WITH mysql_native_password BY 'password';
and to alter user
ALTER USER 'username'#'ip_address' IDENTIFIED WITH mysql_native_password BY 'password';
By the way, this step is also pointed out by #Kol_ya
The issue is (probably) your (new) version of MySQL.
Starting with version 8.04 MySQL uses caching_sha2_password as default authentication plugin where previously mysql_native_password has been used (which causes compatibility issues with older services that expect mysql_native_password authentication).
Possible solutions:
Downgrade the MySQL Server to a version below that change or change the authentication plugin (on a user basis)
eg when creating the user:
CREATE USER 'username'#'localhost' IDENTIFIED WITH mysql_native_password BY 'password';
#Kol_ya created new user using CREATE USER 'username'#'localhost' IDENTIFIED WITH mysql_native_password BY 'password'; where username = creation of new username and password = current password to login for root(username created with caching_sha2). This step allowed me to connect to MySql Server 8.0.13 without having caching_sha2 authentication error.
Looked for solution several threads but this one helped just like that. Thanks #kol_ya
Suggestion to people who are having caching_sha2 issue. In server side make sure your user authentication type is standard not caching_sha2_password.
In MySql 8.0 the default authentication plugin is 'caching_sha2_password', which causes authentication plugin issues(error 2059), one can create a new user in the MySql workbench
and setting there Authentication to be standard.
This error is typically gotten when the DB user don't have all access to the database
Run this command:
ALTER USER 'username'#'ip_address' IDENTIFIED WITH mysql_native_password BY 'password';
*Make sure to have your MySQL server running.
I've been through the introductory tutorial for Django successfully using SQLite as per the instructions. However, for my real project I want to use MySQL, so I deleted the tutorial project files and started with a fresh setup. The trouble is that when I run "python manage.py migrate" I get the following MySQL error:
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server
version for the right syntax to use near '%s' at line 1").
I have the following in the site settings database section:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'vocabulator$database',
'USER': 'vocabulator',
'PASSWORD': '<password>',
'HOST': 'mysql.server',
}
}
I connected to this database successfully in Bash using, 'mysql --user=vocabulator --host=mysql.server --password=<password> "vocabulator\$database" ', so it appears I have entered these setting details correctly.
I also followed an instruction on setting up database bindings for Python 3, which also appeared to work successfully. It was on either the Django documentation pages or the PythonAnywhere equivalent, but unfortunately I cannot locate the page reference again.
I haven't written any MySQL queries myself yet, so any incorrect ones that are being made must be coming from manage.py, so what could be the cause? The complete error trace is quoted below:
Operations to perform: Synchronize unmigrated apps: staticfiles, messages Apply all migrations: contenttypes, sessions, admin, auth Synchronizing apps without migrations: Creating tables...
Running deferred SQL... Installing custom SQL... Running migrations: Rendering model states... DONE Applying contenttypes.0001_initial...Traceback (most recent call last): File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/MySQLdb/cursors.py", line 184, in execute
self.errorhandler(self, exc, value) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/MySQLdb/connections.py", line 37, in defaulterrorhandler
raise errorvalue File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/MySQLdb/cursors.py", line 171, in execute
r = self._query(query) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/MySQLdb/cursors.py", line 330, in _query
rowcount = self._do_query(q) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/MySQLdb/cursors.py", line 294, in _do_query
db.query(q)
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s' at line 1")
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
utility.execute() File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/core/management/__init__.py", line 330, in execute
self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/core/management/base.py", line 393, in run_from_argv
self.execute(*args, **cmd_options) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/core/management/base.py", line 444, in execute
output = self.handle(*args, **options) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/core/management/commands/migrate.py", line 222, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/migrations/executor.py", line 110, in migrate
self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/migrations/executor.py", line 148, in apply_migration
state = migration.apply(state, schema_editor) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/migrations/migration.py", line 115, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/migrations/operations/models.py", line 59, in database_forwards
schema_editor.create_model(model) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/backends/base/schema.py", line 289, in create_model
self.deferred_sql.extend(self._model_indexes_sql(model)) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/backends/mysql/schema.py", line 55, in _model_indexes_sql
self.connection.cursor(), model._meta.db_table File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/backends/mysql/introspection.py", line 142, in get_storage_engine
"WHERE table_name = %s", [table_name]) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/utils.py", line 97, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/utils/six.py", line 658, in reraise
raise value.with_traceback(tb) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/MySQLdb/cursors.py", line 184, in execute
self.errorhandler(self, exc, value) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/MySQLdb/connections.py", line 37, in defaulterrorhandler
raise errorvalue File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/MySQLdb/cursors.py", line 171, in execute
r = self._query(query) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/MySQLdb/cursors.py", line 330, in _query
rowcount = self._do_query(q) File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-packages/MySQLdb/cursors.py", line 294, in _do_query
db.query(q) django.db.utils.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server v ersion for the right syntax to use near '%s' at line 1") Exception ignored in: <bound method Cursor.__del__ of <MySQLdb.curso rs.Cursor object at 0x7feec394b940>> Traceback (most recent call last): File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-p ackages/MySQLdb/cursors.py", line 67, in __del__ File "/home/vocabulator/.virtualenvs/django18/lib/python3.4/site-p ackages/MySQLdb/cursors.py", line 73, in close ReferenceError: weakly-referenced object no longer exists (django18)03:29 ~/mysite $ mysql --user=vocabulator --host=mysql.server --p mysql: ambiguous option '--p' (pager, plugin_dir)
As I said in my comment above, I found the page which had instructed me on how to install drivers. It is this one:
https://www.pythonanywhere.com/wiki/UsingMySQL
However, this Wiki page is a little dated now and it instructed me to install the MySQL connector version 2.0.1. The current version is now 2.0.4. The old version had a bug which meant it would not operate with Django 1.8, as stated here:
http://bugs.mysql.com/bug.php?id=76752
By version 2.0.4 this bug appears to have been corrected.
The line in the PythonAnywhere Wiki page referenced above should change to:
pip3.4 install https://dev.mysql.com/get/Downloads/Connector-Python/mysql-connector-python-2.0.4.tar.gz
Once I ran this, the migration appears to have worked successfully on my Django installation.
i have try this,and works...
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'database',
'USER': 'vocabulator',
'PASSWORD': '<password>',
'HOST': 'mysql.server',
}
}