as the title says, I'm restricted in my development environment. I cannot create extra databases for testing purposes and only create local files.
Using the environ package for django, the configuration of the database is
DATABASES = {
'default': env.db('DATABASE_URL', default='postgres://name#localhost'),
}
As found in this thread, all I need is a "TEST" key in this configuration. It doesn't have one per default so I've bypassed this by writing
db = env.db('DATABASE_URL', default='postgres://name#localhost')
db["TEST"] = {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
This doesn't work however. python manage.py test gives me the following output:
Creating test database for alias 'default'...
/usr/lib/python2.7/dist-packages/django/db/backends/postgresql/base.py:267: RuntimeWarning: Normally Django will use a connection to the 'postgres' database to avoid running initialization queries against the production database when it's not needed (for example, when running tests). Django was unable to create a connection to the 'postgres' database and will use the default database instead.
RuntimeWarning
Got an error creating the test database: permission denied to create database
Type 'yes' if you would like to try deleting the test database '/home/<closed environment>/<project>/config/db.sqlite3', or 'no' to cancel:
It still tries to create a database inside an environment in which I do not have the power to alter my own privileges, something that of course fails. I did specify a testing database as a local file but it somehow doesn't use that? Am I overseeing something or am I doing something completely wrong?
After toying around I've found a workaround.
It's not great but it works :\
is_test = "manage.py" in sys.argv and "test" in sys.argv
if is_test:
db = {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
else:
db = env.db('DATABASE_URL', default='postgres://name#localhost')
Not gr8, not terrible.
Related
I want to use the simple SQLite database on my local environment and use a Postgresql database in production. How can I configure the settings file to know which database to use based on the value of DEBUG?
There are several options available:
Below is a very cheap solution. Django always selects the database called 'default'. You can assign it conditionally in settings.py:
DATABASES = {
'dev': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
},
'production': {
'ENGINE': 'django.db.backends.postgresql',
# ...
},
}
DATABASES['default'] = DATABASES['dev' if DEBUG else 'production']
You can implement an alternate settings module called settings_dev.py. Configure database there and use the environment variable DJANGO_SETTINGS_MODULE to point to yourapp.settings_dev.
Implementing a custom database router. This is almost certainly overkill for many use-cases. See the Django documentation on multiple database support.
We are a small team, trying to work with Django with a restricted access to a futurely unmanaged PostgreSQL database (i.e: only views and stored procedures ; no access to any tables) for security reasons.
We tried to give (within Postgre) the user external_test the rights to create any tables inside his own schema on external, and to use the following settings (settings.py):
...
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
},
'external': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgre_db',
'USER': 'external_user',
'PASSWORD': 'password',
'HOST': 'integration.project.net',
'PORT': '5432',
'TEST': {
'NAME': 'test_db',
'USER': 'external_test',
'PASSWORD': 'password',
...
Using simple passing unit tests (project/app/tests/test_views.py):
...
class InternalTest(TestCase):
database = ['default']
def test_something(self):
pass
class StoredProcedureTest(TestCase):
databases = ['external']
def test_one_procedure(self):
with connections["external"].cursor() as cursor:
cursor.callproc("some_procedure", [42, ])
pass
...
If we try the first one with ./manage.py test app.tests.test_views.InternalTest
→ ok
If we try the other one with ./manage.py test app.tests.test_views.StoredProcedureTest
→ circular dependency issue
(ImproperlyConfigured: Circular dependency in TEST[DEPENDENCIES])
probably because it's skipping the configuration of default
If we try both tests with ./manage.py test app.tests.test_views:
→ permission denied
Creating test database for alias 'default'...
Creating test database for alias 'external'...
Got an error creating the test database: permission denied to create database
(Django try to create a database test_db as the user external_user)
We don't really get what Django is trying to do and how to properly configure it.
If we give the rights to external_user to create their own databases:
the database test_db is created by external_user
the schema of default (sqlite) is created in test_db of external (postgre)
the schema of external (postgre) is not created in test_db
Questions
Is django able to handle this ?
What are we doing wrong ?
What is the point of specifying a user external_user for TEST if in the end django is using the normal user external_user ?
Why does it write the schema of default in test_db ? Is there a way to create only models of some apps in it ?
Why isn't it able to create the schema of external in test_db ?
I hope it was described enough. Thank you in advance for your responses =)
I have been working on a Django app locally and it's now time to deploy it to a staging machine.
My Postgres database on the staging machine has a different username and password to my local machine.
I have got the Django app running okay on the remote machine, except that the database has not been initialised.
I assume that I should do this with migrate, so I try running:
$ python manage.py migrate
But I see the following error:
django.db.utils.OperationalError: FATAL: no pg_hba.conf entry
for host "127.0.0.1", user "mylocalusername", database "mylocaldbname"
It's failing because it doesn't allow me to log in with mylocalusername.
I assume that mylocalusername etc must be coming from the migrations files? Certainly the local username isn't set anywhere else on the staging machine, either in my settings file, or on the actual database itself.
How can I set up this database on the staging server?
I guess one way would be to delete everything in migrations and create a new local migration. Is that what I need to do?
I thought migrations were supposed to checked into source code, though, so I'd rather not delete all of them. Also, I want to carry on working on locally and updating my staging and production machines, so I need to find a sustainable way of doing this.
"mylocalusername" comes from settings.py file.
It should looks like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
'NAME': 'mylocaldbname', # Or path to database file if using sqlite3.
# The following settings are not used with sqlite3:
'USER': 'mylocalusername',
'PASSWORD': 'password',
'HOST': 'localhost', # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP.
'PORT': '', # Set to empty string for default.
}
}
You can change it or create a valid user in your postgres database.
I am just starting to use MySQL as the database for my project. Previously I had been using SQLite.
I am wondering how to specify the location for the MySQL database the same way I was able to for SQLite. Currently it saves automatically to /usr/local/mysql/data by default. But I believe this will cause issues when I try to upload it to my production envroment.
My old SQLite settings
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'database/db.sqlite3'),
}
}
This saved the database into a database folder in my project which I was able to upload and chown both to www-data. This seems like a simple solution which I would like to replicate with MySQL
My new MySQL settings
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
#'NAME': 'django_db',
'NAME': os.path.join(BASE_DIR, 'database/django_db'),
'USER': 'root',
'PASSWORD': 'myPassword',
'HOST': 'localhost',
'PORT': '3306',
}
}
However when I try to syncdb with this I get the below error
django.db.utils.OperationalError: (1059, "Identifier name
'/users/user/workspace/bias_experiment/src/database/django_db'
is too long") (bias_experiment)localhost:src user$
I also tried to create the database within my project with
mysql> CREATE DATABASE /Users/user/workspace/bias_experiment/src/database/django_db;
But this gave me the error
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '/Users/brendan/Dropbox/workspace/bias_experiment/src/database/django_db' at line 1
mysql>
So can I create the DB with in my project and link to it in the same way as I did previously with SQLite? And If not what is the correct way/location to upload it to and link to it?
Any help is greatly appreciated
MySQL is not a file-based database: you don't give it a file path. In fact the settings file itself is quite clear about what you need to put in the NAME attribute, ie (not surprisingly) the name of the database itself. This is the same as you need for CREATE DATABASE, but again you don't give that a path: just a name.
You cannot access mysql files directly. You must create a database (give it a name, not a path) and access it.
SQL> CREATE DATABASE django_db
I'm trying to get a postgres database talking to a django installation.
I've followed the steps details here: http://blog.iiilx.com/programming/how-to-install-postgres-on-ubuntu-for-django/
However, when I use syncdb to have django update my postgres database, I receive the following error:
connection_factory=connection_factory, async=async)
django.db.utils.OperationalError: FATAL: database "/home/flowcore/django_db"
does not exist
django_db is the name of the database and it DOES exist but of course it doesn't exist at /home/flowcore/django_db (that is not where postgres stores data!)
My databases dict in settings.py looks like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.path.join(BASE_DIR, 'django_db'),
'USER': 'django_login',
'PASSWORD': 'mypasswordhere', #obviously i've replaced this with my actual password
'HOST': 'localhost',
}
}
Do I have to specific an absolute path to my postgres database file and, if so, where are these files stored?
Well, for some reason you have put the full path as the NAME parameter, so it's not surprising that Django is using that. There's no reason to do that, and that tutorial you link to doesn't suggest it. NAME is the database name itself, which as you say is just "django_db".