I'm developing a website with Django 3 (in a docker container) using postgres sql as the backend; i.e. in the project settings file I have:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': 'postgres',
'HOST': 'db',
'PORT': 5432
}
}
I've populated the backend database and can browse the data using the admin. However, Id like to connect to the database via the command line so I can more easily test queries. I have tried connecting to the database the normal way from the command line:
sudo -u postgres psql
postgres=# \c postgres
The problem is that there is no data found:
postgres=# \dt
Did not find any relations.
Since I'm new to docker I thought to try connecting other ways as well; specifically, based on another post I tried:
sudo docker run -d -p 5432 -t postgres/postgresql /bin/su postgres -c '/usr/lib/postgresql/10/bin/postgres -D /var/lib/postgresql/10/main -c config_file=/etc/postgresql/10/main/postgresql.conf'
This throws an error:
pull access denied for psql/postgresql, repository does not exist or may require 'docker login'
Again, I'd like to connect to the database via the command line so I can more easily test queries. Perhaps Im on the right track but assistance would be appreciated.
It is a bad idea to use postgres for the database name, as there is a postgres database used for maintenance by default by PostgreSQL itself. I'd recommend calling the database something like my_project, then creating a service account user my_project_user, and assign a password:
sudo -u postgres psql
postgres=# CREATE USER my_project_user WITH PASSWORD 'blahblah';
postgres=# CREATE DATABASE my_project WITH OWNER my_project_user;
postgres=# \q
Update your Django DATABASES["default"] settings accordingly, and run migrations. After running migrations, you can access your Django database using the following management command:
python manage.py dbshell
You may be able to issue the command above with your current setup, but you may run into problems using postgres as your database name. Good luck!
UPDATE, my docker-compose.yml file had the following service:
db:
image: postgres:11
volumes:
- postgres_data:/var/lib/postgresql/data/
I am able to connect to the Django backend using the following command:
sudo docker-compose exec db psql -U postgres
postgres=# \c postgres
As suggested by #FlipperPA I will start useing a different username and database name
This command below runs(opens) the command-line client for PostgreSQL so that you can test queries:
python manage.py dbshell
Related
I have to get the names of Postgres databases in Python. How do I do that? I need to be able to put these names into a list.
you can use pg_dumpall and psql as superuser
pg_dumpall -h localhost -U postgres -f pg_dumpall.sql
I have a Django app that runs locally with no problems.
Now, I want to get a docker image of my app but when I try to build it, it gives me the following error:
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
I am new on Django and Docker development and i was looking for similar questions but none answer solved my problem.
I'll show you my Dockerfile, I have copied the Postgres commands from another project that does work:
FROM python:3.8
RUN apt-get update
RUN mkdir /project
WORKDIR /project
RUN apt-get install -y vim
COPY requirements.txt /project/
RUN pip install -r requirements.txt
COPY . /project/
# Install postgresql
RUN apt install -y postgresql postgresql-contrib
RUN service postgresql start
# Run the rest of the commands as the ``postgres`` user created by the ``postgres-9.3`` package when it was ``apt-get installed``
USER postgres
# Create a PostgreSQL role
RUN /etc/init.d/postgresql start &&\
psql --command "CREATE USER admin WITH SUPERUSER PASSWORD 'passwd';" &&\
createdb -O admin plataforma
USER root
# setup postgresql
RUN sed -i "/^#listen_addresses/i listen_addresses='*'" /etc/postgresql/13/main/postgresql.conf
RUN sed -i "/^# DO NOT DISABLE\!/i # Allow access from any IP address" /etc/postgresql/13/main/pg_hba.conf
RUN sed -i "/^# DO NOT DISABLE\!/i host all all 0.0.0.0/0 md5\n\n\n" /etc/postgresql/13/main/pg_hba.conf
# running commands of my app
RUN python manage.py makemigrations accounts
RUN python manage.py sqlmigrate accounts 0001
RUN python manage.py migrate
RUN python manage.py inicializar
# Expose some ports
EXPOSE 22 5432 8080 8009 8000
# volumes
VOLUME ["/var/lib/postgresql/12/main"]
# Default command
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
Here is my requirements.txt file:
#requirements.txt
Django==3.2.8
djangorestframework==3.9.1
gunicorn==19.9.0
pandas==1.3.3
path==16.2.0
six==1.14.0
Pillow==7.0.0
psycopg2>=2.8
django-environ==0.8.1
Here is my settings.py file (the databases fragment):
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': env('POSTGRESQL_NAME'),
'USER': env('POSTGRESQL_USER'),
'PASSWORD': env('POSTGRESQL_PASS'),
'HOST': env('POSTGRESQL_HOST'),
'PORT': env('POSTGRESQL_PORT'),
}
}
And here is my .env file:
POSTGRESQL_NAME=plataforma
POSTGRESQL_USER=admin
POSTGRESQL_PASS=passwd
POSTGRESQL_HOST=localhost
POSTGRESQL_PORT=5432
DEBUG=True
Some people told me that I could make a docker-compose file so I erased the postgres install commands from Dockerfile and made this:
version: "3.3"
services:
db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
ports:
- "5432:5432"
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
environment:
- POSTGRES_NAME=plataforma
- POSTGRES_USER=admin
- POSTGRES_PASSWORD=passwd
depends_on:
- db
But this doesn't work either.
Footnote: If I use the Django's default database (SQLite) and (obviously) erase postgress commands from Dockerfile, I can build the Docker image without problems, and, again, this app with postgres works good if I run it locally. So, something is happening with Docker + postgres and I don't know what to do.
Somebody can help me? Thank you!
Edit: I erased the migrations commands from Dockerfile and replaced environ var of POSTGRESQL_HOST to db and when I run $ sudo docker-compose run web python manage.py runserver . the image is created but the container not, and when I try to run a container with that image i get the following error:
django.db.utils.OperationalError: could not translate host name "db" to address: Name or service not known
Sorry my english, I still learning it too.
EDIT AGAIN: I finally solved this issue in this question
Thanks for the help!
While creating a superuser your command is
CREATE USER admin WITH SUPERUSER PASSWORD 'administrador'
And in your env file you are using passwd as password.
Change your env from
POSTGRESQL_PASS=passwd
To this
POSTGRESQL_PASS=administrador
In my project, I have a routers.py with different router classes. Now, I am making a new app. I have created my models.py. I have also registered the app in the INSTALLED_APPS in settings.py. Then, I ran validate. Everything is fine. When I syncd thoughb, Django does not install the tables. I tried using
python manage.py sqlall <app_name> | psql <database>
Then, I get an error message saying:
psql: FATAL: password authentication failed for user <user name>
I noticed that the role does not exist in postgres. So, I created the role with login privilege createdb and password. Then, I get a different error message:
psql: FATAL: password authentication failed for user <user name>
close failed in file object destructor:
Error in sys.excepthook:
Original exception was:
And, it does not provide the original exception.
Any help is much appreciated.
It looks like the Django application is unable to log onto the DB.
In django's settings.py make sure you have the proper DB credentials setup:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'HOST': get_env_variable("DJANGO_DB_HOST"),
'NAME': get_env_variable("DJANGO_DB_NAME"),
'USER': get_env_variable("DJANGO_DB_USER"),
'PASSWORD': get_env_variable("DJANGO_DB_PWD"),
'PORT': '',
},
...
}
As you can see my credentials are grabbed from environment variables. You can hardcode them in for test purposes.
Then in the DB (mine is postgresql) create the user/grant it the correct privileges, for example:
ssh root#dbhost
su - postgres
createdb dbname
psql
GRANT ALL PRIVILEGES ON DATABASE dbname TO dbuser
That should do.
I recommend the following steps as well, if they are missing from your setup:
In your app's admin.py register your models with Django's admin:
# Register your models here.
admin.site.register(Model1)
...
admin.site.register(ModelN)
Then, assuming you have created the project already, run:
python manage.py migrate
(it's the syncdb equivalent. Read the docs about migrations).
If that command does not ask for the admin superuser then create your administrative user (i.e. the user who can manipulate the models through django's admin interface) with:
python manage.py createsuperuser
Fire up Django
python manage.py runserver 0.0.0.0:8000
and see what happens whan you visit your site and admin at
localhost:8000/
localhost:8000/admin
Please pardon me if you know all those things already. That is what I normally do in my dev environment (I also use virtualenv, of course).
I have a Django app (Python 3.4, Django 1.7) on PythonAnywhere, along with a MySQL database.
The database is working fine on the deployed app.
However, I cannot get to connect it to the app on my local machine.
The following error is thrown when I run python manage.py runserver:
django.db.utils.InterfaceError: (2003, "2003: Can't connect to MySQL server on 'mysql.server:3306' (8 nodename nor servname provided, or not known)", None)
These are the attributes I use:
DATABASES = {
'default': {
'ENGINE': 'mysql.connector.django',
'NAME': '<username>$<database_name>',
'USER': '<username>',
'PASSWORD': '<databse_password>',
'HOST': 'pythonanywhere.com'
}
}
I have also tried mysql.server and mysql.server.pythonanywhere.com as HOST without any more luck.
I think It's not possible to connect directly to your mysqlserver instance from remote, for security reason, the port 3306 is blocked.
They suggest to connect through SSH Tunnel, follow this link.I don't know If you can do an ssh tunnelling within Django, You should probably write a custom configuration. It's simpler to install an SSH Tunnel software on your PC and then connect your Django App to localhost on a port You have to choose.
Bye
As per PythonAnyWhere documentation :
Open a terminal and run below command.
ssh -L 3333:username.mysql.pythonanywhere-services.com:3306 username#ssh.pythonanywhere.com
provide your PAW account login password
replace username with your username.
Open another terminal and run below command.
mysql -h 127.0.0.1 --port 3333 -u username -p
provide your mysql password. Available in settings file.
keep terminal 1 open as long as you are working on terminal 2.
Accessing PythonAnyWhere MySQL from outside
Only Paid account have permission to access remoteserver for mysql database
I am new to this so a silly question
I am trying to make a demo website using Django for that I need a database.. Have downloaded and installed MySQL Workbench for the same. But I don't know how to setup this.
Thank you in advance :)
I tried googling stuff but didn't find any exact solution for the same.
Please help
I am a mac user. I have luckily overcome the issue with connecting Django to mysql workbench. I assume that you have already installed Django package created your project directory e.g. mysite.
Initially after installation of MySQL workbench i have created a database : create database djo;
Go to mysite/settings.py and edit following piece of block.
NOTE: Keep Engine name "django.db.backends.mysql" while using MySQL server.
and STOP the other Django MySQL service which might be running.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
'NAME': 'djo', # Or path to database file if using sqlite3.
# The following settings are not used with sqlite3:
'USER': 'root',
'PASSWORD': '****', # Replace **** with your set password.
'HOST': '127.0.0.1', # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP.
'PORT': '3306', # Set to empty string for default.
}
}
now run manage.py to sync your database :
$ python mysite/manage.py syncdb
bash-3.2$ python manage.py syncdb
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_site
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (leave blank to use 'ambershe'): root
Email address: ambershe#netapp.com
/Users/ambershe/Library/Containers/com.bitnami.django/Data/app/python/lib/python2.7/getpass.py:83: GetPassWarning: Can not control echo on the terminal.
passwd = fallback_getpass(prompt, stream)
Warning: Password input may be echoed.
Password: ****
Warning: Password input may be echoed.
Password (again): ****
Superuser created successfully.
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)