We Keep Coding

Setting up a Postgres DB for Django with Fly.io

I'm a novice trying to spin up my first webapp with a combination of Fly.io, Django, and a postgres DB but I'm having some trouble and can't find an answer in walkthroughs or Q&A.
I've set up a simple "Hello world" Django app (models.py is empty so far) and I'm trying to get all the components up and running before I build it out any further.
I've successfully deployed my app on Fly.io with no errors
I've created a postgres cluster on Fly.io using the instructions here: https://fly.io/docs/postgres/
I've attached the cluster to my app, which generates a DB and sets an environment variable with the appropriate details (username, password, port, host, dbname)
I've updated my settings.py file:
DATABASES = {}
DATABASES["default"] = dj_database_url.config(conn_max_age=600, ssl_require=True)
I've added to my fly.toml:
[[services]]
internal_port = 5432 # Postgres instance
protocol = "tcp"
# Open port 10000 for plaintext connections.
[[services.ports]]
handlers = []
port = 10000
I've confirmed I can get into the psql shell with flyctl postgres connect -a MYAPP-pg
But unfortunately when I run python manage.py migrate to check that everything is working, I get the following error:
File "<my_path>\venv\lib\site-packages\django\db\backends\base\base.py", line 282, in ensure_connection
self.connect()
File "<my_path>\venv\lib\site-packages\django\utils\asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "<my_path>\venv\lib\site-packages\django\db\backends\base\base.py", line 263, in connect
self.connection = self.get_new_connection(conn_params)
File "<my_path>\venv\lib\site-packages\django\utils\asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "<my_path>\venv\lib\site-packages\django\db\backends\postgresql\base.py", line 215, in get_new_connection
connection = Database.connect(**conn_params)
File "<my_path>\venv\lib\site-packages\psycopg2\__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not translate host name "top2.nearest.of.MYAPP-pg.internal" to address: Unknown host
Any ideas what might be happening? Any help would be very much appreciated!

psycopg2.OperationalError: FATAL: password authentication failed for user "<my UNIX user>"

I am a fairly new to web developement.
First I deployed a static website on my vps (Ubuntu 16.04) without problem and then I tried to add a blog app to it.
It works well locally with PostgreSQL but I can't make it work on my server.
It seems like it tries to connect to Postgres with my Unix user.
Why would my server try to do that?
I did create a database and a owner via the postgres user, matching the login information in settings.py, I was expecting psycopg2 to try to connect to the database using these login informations:
Settings.py + python-decouple:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': config ('NAME'),
'USER': config ('USER'),
'PASSWORD': config ('PASSWORD'),
'HOST': 'localhost',
'PORT': '',
}
}
This is the error message I get each time I try to ./manage.py migrate
'myportfolio' is my Unix user name, the database username is different:
Traceback (most recent call last):
File "/home/myportfolio/lib/python3.5/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
self.connect()
File "/home/myportfolio/lib/python3.5/site-packages/django/db/backends/base/base.py", line 194, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/myportfolio/lib/python3.5/site-packages/django/db/backends/postgresql/base.py", line 168, in get_new_connection
connection = Database.connect(**conn_params)
File "/home/myportfolio/lib/python3.5/site-packages/psycopg2/__init__.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL: password authentication failed for user "myportfolio"
FATAL: password authentication failed for user "myportfolio"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "./manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "/home/myportfolio/lib/python3.5/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
utility.execute()
File "/home/myportfolio/lib/python3.5/site-packages/django/core/management/__init__.py", line 365, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/myportfolio/lib/python3.5/site-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/myportfolio/lib/python3.5/site-packages/django/core/management/base.py", line 335, in execute
output = self.handle(*args, **options)
File "/home/myportfolio/lib/python3.5/site-packages/django/core/management/commands/migrate.py", line 79, in handle
executor = MigrationExecutor(connection, self.migration_progress_callback)
File "/home/myportfolio/lib/python3.5/site-packages/django/db/migrations/executor.py", line 18, in __init__
self.loader = MigrationLoader(self.connection)
File "/home/myportfolio/lib/python3.5/site-packages/django/db/migrations/loader.py", line 49, in __init__
self.build_graph()
File "/home/myportfolio/lib/python3.5/site-packages/django/db/migrations/loader.py", line 206, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/home/myportfolio/lib/python3.5/site-packages/django/db/migrations/recorder.py", line 61, in applied_migrations
if self.has_table():
File "/home/myportfolio/lib/python3.5/site-packages/django/db/migrations/recorder.py", line 44, in has_table
return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
File "/home/myportfolio/lib/python3.5/site-packages/django/db/backends/base/base.py", line 255, in cursor
return self._cursor()
File "/home/myportfolio/lib/python3.5/site-packages/django/db/backends/base/base.py", line 232, in _cursor
self.ensure_connection()
File "/home/myportfolio/lib/python3.5/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
self.connect()
File "/home/myportfolio/lib/python3.5/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/home/myportfolio/lib/python3.5/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
self.connect()
File "/home/myportfolio/lib/python3.5/site-packages/django/db/backends/base/base.py", line 194, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/myportfolio/lib/python3.5/site-packages/django/db/backends/postgresql/base.py", line 168, in get_new_connection
connection = Database.connect(**conn_params)
File "/home/myportfolio/lib/python3.5/site-packages/psycopg2/__init__.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: FATAL: password authentication failed for user "myportfolio"
FATAL: password authentication failed for user "myportfolio"
I tried to:
delete my django code, re install
delete/purge postgres and reinstall
modify pg_hba.conf local to trust
At one point I did create a django superuser called 'myportfolio' as my unix user: could this have create a problem ?

As per the error, it is clear that the failure is when your Application is trying to postgres and the important part to concentrate is Authentication.
Do these steps to first understand and reproduce the issue.
I assume it as a Linux Server and recommend these steps.
Step 1:
$ python3
>>>import psycopg2
>>>psycopg2.connect("dbname=postgres user=postgres host=localhost password=oracle port=5432")
>>>connection object at 0x5f03d2c402d8; dsn: 'host=localhost port=5432 dbname=postgres user=postgres password=xxx', closed: 0
You should get such a message. This is a success message.
When i use a wrong password, i get this error.
>>>psycopg2.connect("dbname=postgres user=postgres host=localhost password=wrongpassword port=5432")
>>>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/dist-packages/psycopg2/__init__.py", line 130, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL: password authentication failed for user "postgres"
FATAL: password authentication failed for user "postgres"
When there is no entry in pg_hba.conf file, i get the following error.
>>> psycopg2.connect("dbname=postgres user=postgres host=localhost password=oracle port=5432 ")
>>> Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/dist-packages/psycopg2/__init__.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL: no pg_hba.conf entry for host "::1", user "postgres", database "postgres", SSL on
FATAL: no pg_hba.conf entry for host "::1", user "postgres", database "postgres", SSL off
So, the issue is with password. Check if your password contains any special characters or spaces. if your password has spaces or special characters, use double quotes as i used below.
>>> psycopg2.connect(dbname="postgres", user="postgres", password="passwords with spaces", host="localhost", port ="5432")
If all is good with the above steps and you got success messages, it is very clear that the issue is with your dsn.
Print the values passed to these variables.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': config ('NAME'),
'USER': config ('USER'),
'PASSWORD': config ('PASSWORD'),
'HOST': 'localhost',
'PORT': '',
}
}
Validate if all the values are being substituted appropriately. You may have the correct password for the user but the dsn is not picking the correct password for the user. See if you can print the dsn and validate if the connection string is perfectly being generated. You will get the fix there.

So I was just stuck on this problem and I thought I'd save whoever comes across this post some time by posting the actual commands. This was done on my raspberry pi.
sudo su - postgres
postgres#raspberrypi:~$ psql
postgres=# CREATE DATABASE websitenamehere
postgres=# CREATE USER mywebsiteuser WITH PASSWORD 'Password';
postgres=# GRANT ALL PRIVILEGES ON DATABASE websitenamehere to mywebsiteuser;
postgres=# \q
Done, you have now created a user.

What is setup as user in config ('USER'). Following the error:
FATAL: password authentication failed for user "myportfolio"
user is myportfolio, so you will need to create that user if it does not exist.

I had something similar. My issue was that I did not set the environment variables correctly so it couldn't connect. Ensure that if you go to Edit Configurations, then Environment Variables, and put in your answers in that column.

This problem might also occur if you have some special characters within your password that Postgres cannot cope with (unless you do some special encoding).

Try something like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}

For me, I had the wrong port. Additional characters.

This solved for me:
from sqlalchemy import create_engine
connection_string_orig = "postgres://user_with_%34_in_the_string:pw#host:port/db"
connection_string = connection_string_orig.replace("%", "%25")
engine = create_engine(connection_string)
print(engine.url) # should be identical to connection_string_orig
engine.connect()
from:
https://www.appsloveworld.com/coding/python3x/7/flask-alchemy-psycopg2-operationalerror-fatal-password-authentication-fail

Python mysql (using pymysql) failing to connect

I am using Python Pymysql to connect to AWS RDS db.
This was working all these days but now I am not able to connect to the db.
However when I am using the same machine and same credentials it does connect through Mysql workbench, but through pycharm this does not work.,
Code :
configParser = ConfigParser.RawConfigParser()
configFilePath = r'docs/credentials.cfg'
configParser.read(configFilePath)
User = configParser.get('aws-config','user')
Pswd = configParser.get('aws-config','pswd')
Host = configParser.get('aws-config','hostdps')
Databs = configParser.get('aws-config','databs')
db = pymysql.connect(host=Host, user=User, password=Pswd, db=Databs, charset='utf8mb4', cursorclass=pymysql.cursors.DictCursor,local_infile=True)
Error :
File "/usr/local/lib/python2.7/dist-packages/pymysql/__init__.py", line 90, in Connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 688, in __init__
self.connect()
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 906, in connect
self._request_authentication()
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1114, in _request_authentication
auth_packet = self._read_packet()
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 961, in _read_packet
packet_header = self._read_bytes(4)
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 995, in _read_bytes
"Lost connection to MySQL server during query (%s)" % (e,))
pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query ([Errno 104] Connection reset by peer)')
How to make sure that this does not happen on production? (prevent such failures in the future).

why u not use connection pool for these?
connection pool can auto manage connection,so can fix this.
U can try any python mysql pool .
pool example
for this this link
or directly use github handy code .PyMysqlPool

Python MySQLdb error

When I run my script it won't connect to mysql database, I can't find my host name from godaddy I just tried mysq.mydomain.com and mydomain.com. the login details for the database are correct.
Traceback (most recent call last):
File "mysql.py", line 6, in <module>
conn = MySQLdb.connect(host="?",user="?",passwd="?",db="?")
File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__
super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (1045, "Access denied for user 'official_autotes'#'104.156.228.189' (using password: YES)")
------------------
my code
from threading import Thread
import urllib
import re
import MySQLdb
conn = MySQLdb.connect(host="?",user="?",passwd="?",db="?")
query = "INSERT INTO tutorial (symbol) values ('AAPL')"
x = conn.cursor()
x.execute(query)
row = x.fetchal()

Grand privilage to the user and ip trying to access the MySQL table using the Python code. To do the same, execute the following commands in the mysql shell.
GRANT <privileges> ON database.* TO 'user'#'ip' IDENTIFIED BY 'password';
flush privileges;

OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '192.168.129.139' (111)") None None

I am trying to create a remote database using mysql on an Ubuntu machine running 12.04.
It has a root user with remote login enabled and no password.I have started the server.
output of
sudo netstat -tap | grep mysql
shows
tcp 0 0 localhost:mysql *:* LISTEN 13246/mysqld
I have created a database called nwtopology using (as mentioned root doesn't have a password yet.)
create database nwtopology
grant all privileges on *.* to root#192.168.129.221
FLUSH PRIVILEGES;
From the client machine that also runs Ubuntu 12.04 I use a python script to connect to the remote mysql database using sqlalchemy.
from pox.core import core
import pox.openflow.libopenflow_01 as of
import re
import datetime
import time
from sqlalchemy import create_engine, ForeignKey
from sqlalchemy import Column, Date, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, backref
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.sql.expression import exists
log = core.getLogger()
engine = create_engine('mysql://root#192.168.129.139/nwtopology', echo=False)
Base = declarative_base()
Session = sessionmaker(bind=engine)
session = Session()
class SourcetoPort(Base):
""""""
__tablename__ = 'source_to_port'
id = Column(Integer, primary_key=True)
port_no = Column(Integer)
src_address = Column(String,index=True)
#-----------------------------------------
def __init__(self, src_address,port_no):
""""""
self.src_address = src_address
self.port_no = port_no
The create_engine() call is failing with the following error.
POX 0.1.0 (betta) / Copyright 2011-2013 James McCauley, et al.
Traceback (most recent call last):
File "/home/karthik/ms_thesis/pox/pox/boot.py", line 89, in do_import2
__import__(name, globals(), locals())
File "/home/karthik/ms_thesis/pox/custom/tutorial.py", line 39, in <module>
Base.metadata.create_all(engine)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/schema.py", line 2515, in create_all
tables=tables)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2230, in _run_visitor
conn = self.contextual_connect(close_with_result=False)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2340, in contextual_connect
self.pool.connect(),
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 210, in connect
return _ConnectionFairy(self).checkout()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 371, in __init__
rec = self._connection_record = pool._do_get()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 697, in _do_get
con = self._create_connection()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 174, in _create_connection
return _ConnectionRecord(self)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 256, in __init__
self.connection = self.__connect()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 316, in __connect
connection = self.__pool._creator()
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line 80, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 280, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in __init__
super(Connection, self).__init__(*args, **kwargs2)
OperationalError: (OperationalError) (2003, "Can't connect to MySQL server on '192.168.129.139' (111)") None None
I cannot figure out why this is happening?Any help is greatly appreciated?

Looks like the mysql server is configured to listen only on localhost.
You can test this by running telnet 192.168.129.139 3306 from your client machine.
Most probable reason - mysqld (=MySQL daemon) is configured to do so.
Please try to follow Configuration step described here:
Edit the /etc/mysql/my.cnf file to configure MySQL to listen for connections from network hosts, change the bind-address directive to the server's IP address. For example, in your case, replace 192.168.0.5 with 192.168.129.139.
From:
bind-address = 192.168.0.5
to:
bind-address = 192.168.129.139
If there is no such entry and you cannot connect, create a new line.
You may also try commenting out the line instead.
After making a change to /etc/mysql/my.cnf the MySQL daemon will need to be restarted:
sudo systemctl restart mysql.service
Then test with telnet or by running your application once again. Also netstat would have second entry for mysqld.