I am trying to connect the application (which is not running on docker)
i am trying to run this docker image with the help of docker compose.
i am using host network mode connecting external services on
host.docker.internal on port 7497
i am trying to call from the docker container from the python code
this docker is not having port config
services:
ibkr-bot-eminisp500:
container_name: ibkr-bot-eminisp500
image: |my-image|
network_mode: host
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
- IBKR_CLIENT_URL_KEY= "host.docker.internal"
- IBKR_PORT_KEY=7497
but i am getting following error. what i am missing
| API connection failed: gaierror(-2, 'Name or service not known')
ibkr-bot-eminisp500 | Traceback (most recent call last):
ibkr-bot-eminisp500 | File "/usr/bin/src/app/main.py", line 8, in <module>
ibkr-bot-eminisp500 | ibkrBot = IBKRBot()
Combining host.docker.internal with network_mode: host doesn't make any sense.
If you're running under Linux, then when using network_mode: host your container is running your host's main network environment. Drop the extra_hosts section from your config because it isn't doing you any good. You can connect to a service on your host using any ip address from any host interface, including 127.0.0.1.
If you are running on anything other than Linux, then network_mode: host is probably never useful (because the Docker "host" is actually a virtual machine running on top of your primary operating system). In this case, drop network_mode: host from your config, and connect using host.docker.internal.
I am working with sqlalchemy and mysql, the process is working fine for mysql installed locally but I am not able to connect it with a mysql docker image. I am using pymysql as a driver. Here is the line of commands that I run and I am getting an error shown below.
Following are the portions of /docker-compose.yml and the python file. Also I have a script that creates a database named "sqlalchemy" in docker mysql which is not shown below.
/docker-compose.yml
services:
db:
build: ./database
restart: always
ports:
- "3309:3306"
environment:
- MYSQL_USER=root
- MYSQL_ROOT_PASSWORD=password
- MYSQL_HOST=db
/sqlalchemy.py
msqldb_uri = 'mysql+pymysql://root:password#db:3309/sqlalchemy'
engine = create_engine(msqldb_uri)
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'db' ([Errno 111] Connection refused)")
If the script is running inside container then you do not use the publish port in the connection but you should use 3306 within localhost or same network. The publish port is for outer world.
msqldb_uri = 'mysql+pymysql://root:password#localhost:3306/sqlalchemy'
engine = create_engine(msqldb_uri)
If the script is runnin in other container that is in the same docker-compose file then you also you do not need the publish port.
msqldb_uri = 'mysql+pymysql://root:password#db:3306/sqlalchemy'
engine = create_engine(msqldb_uri)
if the script in running on host and DB is running inside container then you need connect with the publish port but you should not use container name as a IP.
msqldb_uri = 'mysql+pymysql://root:password#localhost:3309/sqlalchemy'
engine = create_engine(msqldb_uri)
I have app, that used Tornado and tornado-redis. [image "app" in docker images]
I start redis:
docker run --name some-redis -d redis
Then I want to link my app with redis:
docker run --name some-app --link some-redis:redis app
And I have error:
Traceback (most recent call last):
File "./app.py", line 41, in <module>
c.connect()
File "/usr/local/lib/python3.4/site-packages/tornadoredis/client.py", line 333
, in connect
self.connection.connect()
File "/usr/local/lib/python3.4/site-packages/tornadoredis/connection.py", line
79, in connect
raise ConnectionError(str(e))
tornadoredis.exceptions.ConnectionError: [Errno 111] Connection refused
I have tested my code with local tornado and redis, and it works. The problem in
c = tornadoredis.Client()
c.connect()
Why my app cant connet to redis-container? How to fix that? I use standart port 6379.
Thanks!
tornadoredis attempts to use redis on localhost. (See source here)
So you need to inform tornadoredis where redis is running (since to the docker image it is not running on localhost).
For example:
c = tornadoredis.Client(host="<hostname>")
c.connect()
In your specific case, substitute "redis" for "<hostname>".
I am not sure how to fix this issue
I have no idea why I am getting this error when I try to runserver:
Performing system checks...
System check identified no issues (0 silenced).
Unhandled exception in thread started by <function wrapper at 0x1085589b0>
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/django/utils/autoreload.py", line 222, in wrapper
fn(*args, **kwargs)
File "/Library/Python/2.7/site-packages/django/core/management/commands/runserver.py", line 107, in inner_run
self.check_migrations()
File "/Library/Python/2.7/site-packages/django/core/management/commands/runserver.py", line 159, in check_migrations
executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
File "/Library/Python/2.7/site-packages/django/db/migrations/executor.py", line 17, in __init__
self.loader = MigrationLoader(self.connection)
File "/Library/Python/2.7/site-packages/django/db/migrations/loader.py", line 49, in __init__
self.build_graph()
File "/Library/Python/2.7/site-packages/django/db/migrations/loader.py", line 184, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/Library/Python/2.7/site-packages/django/db/migrations/recorder.py", line 59, in applied_migrations
self.ensure_schema()
File "/Library/Python/2.7/site-packages/django/db/migrations/recorder.py", line 49, in ensure_schema
if self.Migration._meta.db_table in self.connection.introspection.get_table_list(self.connection.cursor()):
File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 165, in cursor
cursor = self.make_debug_cursor(self._cursor())
File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 138, in _cursor
self.ensure_connection()
File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 133, in ensure_connection
self.connect()
File "/Library/Python/2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 133, in ensure_connection
self.connect()
File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 122, in connect
self.connection = self.get_new_connection(conn_params)
File "/Library/Python/2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 134, in get_new_connection
return Database.connect(**conn_params)
File "/Library/Python/2.7/site-packages/psycopg2/__init__.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
When I try to connect to postgres:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'beerad',
'USER': 'bli1',
'PASSWORD': '',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
It can be some issues:
PostgreSQL is not running. Check it with sudo service postgresql status
Your PostgresSQl is not running on port 5432. You can check it typing sudo netstat -nl | grep postgres
You have something wrong trying to connect to your db like the username, the password or the databasename. Check that they are what postgres ask for you to connect it and that is the db_name that you want to access to.
Problems with postmaster.pid in postgres. It can happen because of a shutdown unproperly done. It makes to remind a pid alive that doesn't allow your server start. To fix it you have to:
* rm /usr/local/var/postgres/postmaster.pid
* pg_resetxlog -f /usr/local/var/postgres
After this it should run properly if you make the runserver of postgres
Help in Mac OSX: How to start PostgreSQL server on Mac OS X?
Try killing all postgres processes. Im on a MAC and this solution that I've found on ubuntus forum really works.
https://askubuntu.com/questions/547434/how-to-nicely-stop-all-postgres-processes
For Windows
Go to search bar and just write "Open psql" and hit Enter.
Once screen is opened, rerun django project.
In my case, all was set up well and Postgres had the right port, PostgreSQL was running normally, but the 5432 port was being shared with phppgadmin, I could access the phppgadmin that gives me web access to Postgres database server, but my Django application was not working it would return Connection refused error. so I changed the port number on the phppgadmin config file (/etc/phppgadmin/config.inc.php) to 5433 from 5432 and all worked fine.
In MacOS I stopped and restarted postgresql according to the advice given in this StackExchange answer:
https://dba.stackexchange.com/a/171580/182403
brew services stop postgresql
rm /usr/local/var/postgres/postmaster.pid # adjust path accordingly to your install
brew services start postgresql
In project_folder/settings.py under DATABASE - HOST settings you should use local IP (127.0.0.1) not your public IP.
Correct
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'yourdb',
'USER': 'youruser',
'PASSWORD': 'yourpass',
'HOST': '127.0.0.1',
'PORT': '5432'
}
}
Incorrect
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'yourdb',
'USER': 'youruser',
'PASSWORD': 'yourpass',
'HOST': '188.252.196.234 ',
'PORT': '5432'
}
}
I had the same issue. This error occurs when an improper system shutdown has been done. I'm on an M1, and I've installed PostgreSQL via Homebrew.
I tried to start PostgreSQL after receiving an error identical to yours using brew services start postgresql, but I got this error:
Bootstrap failed: 5: Input/output error
Try re-running the command as root for richer errors.
Error: Failure while executing; `/bin/launchctl bootstrap gui/501 /Users/adithraghav/Library/LaunchAgents/homebrew.mxcl.postgresql.plist` exited with 5.
So, I tried to stop the running instance and start it again with brew services restart postgresql.
This line gave me the following output:
Stopping `postgresql`... (might take a while)
==> Successfully stopped `postgresql` (label: homebrew.mxcl.postgresql)
==> Successfully started `postgresql` (label: homebrew.mxcl.postgresql)
I checked the port on which PostgreSQL was running with sudo netstat -nl | grep postgres, but received no output.
So I deleted the existing postmaster.pid file with this:
rm /opt/homebrew/var/postgres/postmaster.pid
NOTE: If you are not on an M1 Mac, you have to run rm /usr/local/var/postgres/postmaster.pid.
Then, I ran pg_resetwal -f /opt/homebrew/var/postgres to reset the write-ahead log. (NOTE: From Postgres 10 and onwards, pg_resetxlog has been renamed to pg_resetwal).
Now, python3 manage.py runserver works with no issues :)
The following command works for me (Windows)-
pg_ctl -D "C:\Program Files\PostgreSQL\11\data" restart
Then run server again-
python manage.py runserver
Restarting the server solved the problem on my side.
In case you are seeing this error while using PostgresSQL from Google Cloud follow all configurations as mentioned in
https://cloud.google.com/python/django/flexible-environment#macos-64-bit
Also separate the HOST configuration as below
DATABASES['default']['HOST'] = '/cloudsql/'
if os.getenv('GAE_INSTANCE'):
pass
else:
DATABASES['default']['HOST'] = '127.0.0.1'
This helps in resolving this error.
Go to aws instance -> security groups -> source -> inbound -> ::0
You might not connecting to the right database. If you are using Docker be sure you are using 5432 as port at the outside of the image.
eg:
db:
image: postgres:12.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- ./postgres/.env
ports:
- 5432:5432
One problem that I found common with Django/Postgres (especially with Docker) is that your Django Web App maybe starting up before your Postgres server starts up. If the other solutions don't work, try to restart your Django App after start up. With Docker, the command would be docker restart <web-app container name>
You have to enable listen addresses if you are using remote database.
cd /etc/postgresql/12.x/main/
open file named postgresql.conf
sudo nano postgresql.conf
add this line to that file
listen_addresses = '*'
then an open a file named pg_hba.conf
sudo nano pg_hba.conf
and add this line to that file
host all all 0.0.0.0/0 md5
restart your server
sudo /etc/init.d/postgresql restart
I am not able connect to PostgreSQL remotely using python and psycopg2:
Here is my code.
>>> import psycopg2
>>> conn_string = "host='localhost' dbname='mydb' user='postgres'"
>>> print "Connecting to database\n ->%s" % (conn_string)
Connecting to database
->host='localhost' dbname='mydb' user='postgres'
>>> conn = psycopg2.connect(conn_string)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/tools/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
psycopg2.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
The password is not set for postgres user.
Generally, I can connect to database by running following method on host.
1. SSH to box
2. su - postgres
3. psql
4. \c mydb
The server runs PostgreSQL 9.1.
You're trying to connect to PostgreSQL on localhost using a script running on your computer, but there's no PostgreSQL server running there.
For this to work, you'd have to ssh to the remote server, then run your Python script there, where the PostgreSQL server is "local" relative to the Python script.
(That's why running psql works - because you're running it on the remote server, where PostgreSQL is "local" relative to psql).
Alternately, you could:
Use an SSH tunnel to forward the PostgreSQL port from the local computer to the remote one; or
Connect directly over TCP/IP to the remote PostgreSQL server using its host name or IP address, after enabling remote connections on the server.
Note that just putting the server's IP address or host name into the connection string instead of localhost will not work unless you also configure the server to accept remote connections. You must set listen_addresses to listen for non-local connections, add any required firewall rules, set pg_hba.conf to permit connections from remote machines, and preferably set up SSL. All this is covered in the Client Authentication chapter of the PostgreSQL user manual.
You'll probably find an SSH tunnel simpler and easier to understand.