We Keep Coding

Connecting to cosmos container from a different python container gives [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed

I have two containers in docker
Python
Cosmos
docker-compose.yml
version: '3'
services:
python:
image: brunneis/python:3.7.3-ubuntu-18.04
container_name: python
tty: true
ports:
- "5000:5000"
depends_on:
- cosmos
volumes:
- .:/app
networks:
- cosmos-network
cosmos:
image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
container_name: cosmos
tty: true
restart: always
ports:
- "8081:8081"
- "8900:8900"
- "8901:8901"
- "8979:8979"
- "10250:10250"
- "10251:10251"
- "10252:10252"
- "10253:10253"
- "10254:10254"
- "10255:10255"
- "10256:10256"
- "10350:10350"
volumes:
- ./cosmos-data:/data
networks:
- cosmos-network
networks:
cosmos-network:
driver: bridge
Now entering to the python container and running this script:
main.py
from azure.cosmos import CosmosClient
CONFIG = {
"ENDPOINT": "https://host.docker.internal:8081",
"PRIMARYKEY": "<KEY>",
}
client = CosmosClient(url=CONFIG["ENDPOINT"], credential=CONFIG["PRIMARYKEY"])
print(client)
gives error:
azure.core.exceptions.ServiceRequestError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1056)
I've tried adding certificate to the python container using this command:
cd ~/usr/local/share/ca-certificates
curl -k https://localhost:8081/_explorer/emulator.pem > emulatorcert.crt
sudo update-ca-certificates
But still facing the same issue....If someone has connected cosmos container from any internal container (Because connecting from the host machine, it's running fine), please enlighten me.

Access mariadb docker-compose container from host-machine

I'm trying to access a mariadb-container from a python script on my host-machine (MacOS).
I tried all network_modes (host, bridge, default), but nothing works.
I was able to connect to the container through phpmyadmin, but only if both containers are in the same docker-compose-network.
Here is my docker-compose.yml with the attempt on network_mode host:
version: '3.9'
services:
mariadb:
image: mariadb:10.9.1-rc
container_name: mariadb
network_mode: bridge
ports:
- 3306:3306
volumes:
- ...
environment:
- MYSQL_ROOT_PASSWORD=mysqlroot
- MYSQL_PASSWORD=mysqlpw
- MYSQL_USER=test
- MYSQL_DATABASE=test1
- TZ=Europe/Berlin
phpmyadmin:
image: phpmyadmin:5.2.0
network_mode: bridge
container_name: pma
# links:
# - mariadb
environment:
- PMA_HOST=mariadb
- PMA_PORT=3306
- TZ=Europe/Berlin
ports:
- 8081:80
Any tips on how I get access to the container through the python mariadb package?
Thanks!

Every thing seems okay, just check the params when trying to connect to the db:
host: 0.0.0.0
port: 3306 (as in the docker-compose)
user: test (as in the docker-compose)
password: mysqlpw (as in the docker-compose)
database: test1 (as in the docker-compose)
example:
db = MySQLdb.connect("0.0.0.0","test","mysqlpw","test1")

Trouble Remotely Connecting Flask App to Selenium Grid using Docker

I am new to both Docker and Selenium grid and am having issues getting my web app to connect to the selenium hub.
docker-compose.yml
version: '3.8'
services:
db:
image: postgres
environment:
POSTGRES_DB: ${POSTGRES_DB}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
ports:
- "${POSTGRES_PORT}:5432"
volumes:
- pgdata:/var/lib/postgresql/data
web:
build:
context: ..
dockerfile: docker/Dockerfile
environment:
FLASK_ENV: ${FLASK_ENV}
FLASK_CONFIG: ${FLASK_CONFIG}
APPLICATION_DB: ${APPLICATION_DB}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_HOSTNAME: "db"
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_PORT: ${POSTGRES_PORT}
command: flask run --host 0.0.0.0
volumes:
- ..:/opt/code
ports:
- "5000:5000"
chrome:
image: selenium/node-chrome:4.0.0-20211013
shm_size: 2gb
depends_on:
- selenium-hub
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_NODE_GRID_URL=http://localhost:4444
ports:
- "6900:5900"
edge:
image: selenium/node-edge:4.0.0-20211013
shm_size: 2gb
depends_on:
- selenium-hub
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_NODE_GRID_URL=http://localhost:4444
ports:
- "6901:5900"
firefox:
image: selenium/node-firefox:4.0.0-20211013
shm_size: 2gb
depends_on:
- selenium-hub
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_NODE_GRID_URL=http://localhost:4444
ports:
- "6902:5900"
selenium-hub:
image: selenium/hub:4.0.0-20211013
container_name: selenium-hub
ports:
- "4442:4442"
- "4443:4443"
- "4444:4444"
volumes:
pgdata:
When running the stack of containers and checking netstat -a, I can see my desktop listening to port 4444 and when I kill the containers its not.
I can also verify that the hub is running and all of my nodes are connecting fine by visiting https//:localhost/4444, however when I run driver = webdriver.Remote(command_executor="http://localhost:4444") from my python flask app (which is running in the container specified as web above) I get the error:
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4444): Max retries
exceeded with url: /session (Caused by NewConnectionError('<urllib3.connection.HTTPConnection
object at 0x7fd4855b7730>: Failed to establish a new connection: [Errno 111] Connection refused'))
I have tried specifying the desired capabilities to match the driver for specific nodes, however I receive this same error regardless.
I am using Selenium's latest build "4.0.0" and as you can see 4.0.0 images for the parts of the grid so I don't think its a compatibility issue.
docker ps
Name Command State Ports
------------------------------------------------------------------------------------------------------------------------
development_chrome_1 /opt/bin/entry_point.sh Up 0.0.0.0:6900->5900/tcp,:::6900->5900/tcp
development_db_1 docker-entrypoint.sh postgres Up 0.0.0.0:5432->5432/tcp,:::5432->5432/tcp
development_edge_1 /opt/bin/entry_point.sh Up 0.0.0.0:6901->5900/tcp,:::6901->5900/tcp
development_firefox_1 /opt/bin/entry_point.sh Up 0.0.0.0:6902->5900/tcp,:::6902->5900/tcp
development_web_1 flask run --host 0.0.0.0 Up 0.0.0.0:5000->5000/tcp,:::5000->5000/tcp
selenium-hub /opt/bin/entry_point.sh Up 0.0.0.0:4442->4442/tcp,:::4442->4442/tcp,
0.0.0.0:4443->4443/tcp,:::4443->4443/tcp,
0.0.0.0:4444->4444/tcp,:::4444->4444/tcp
I feel like I'm fundamentally missing something here, Any thoughts?

I see the mistake now. I was mistakenly attempting to connect to http://localhost:4444 with my client, when I was needing to specify the network name deployed by selenium grid.
Fix
Change this line in your flask_app.py
driver = webdriver.Remote(command_executor="http://localhost:4444")
To:
driver = webdriver.Remote(command_executor="http://container-name:4444")
Where container-name is the selenium hub name set in docker-compose.yml
selenium-hub:
image: selenium/hub:4.0.0-20211013
container_name: selenium-hub
ports:
- "4442:4442"
- "4443:4443"
- "4444:4444"
in my case: "selenium-hub"
Resource on docker networking used: https://docs.docker.com/compose/networking/
Final Thoughts
I guess I got tripped up by the fact that I am still using http://localhost:port to connect to both the grid hub and web container. I guess the difference is where the client request comes from? From outside the docker stack vs within? Anyway, hope this helps someone.

Django server inside docker is causing robot tests to give blank screens?

I have built an environment inside docker compose in order to run robot tests. The environment consists of django web app, postgres and robot framework container. The Problem I have is that I get many blank screens in different tests, while using external Django web app instance which is installed on a virtual machine doesn't have this problem.
The blank screen causes that elements are not found hence so many failures:
JavascriptException: Message: javascript error: Cannot read property 'get' of undefined
(Session info: headless chrome=84.0.4147.89)
I am sure that the problem is with the Django app container itself not robot container since as said above I have tested with the same environment but against different web app which is installed outside Docker, and it worked.
docker-compose.yml:
version: "3.6"
services:
redis:
image: redis:3.2
ports:
- 6379
networks:
local:
ipv4_address: 10.0.0.20
smtpd:
image: mysmtpd:1.0.5
ports:
- 25
networks:
- local
postgres:
image: mypostgres
build:
context: ../dias-postgres/
args:
VERSION: ${POSTGRES_TAG:-12}
hostname: "postgres"
environment:
POSTGRES_DB: ${POSTGRES_USER}
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
networks:
local:
ipv4_address: 10.0.0.100
ports:
- 5432
volumes:
- my-postgres:/var/lib/postgresql/data
app:
image: mypyenv:${PYENV_TAG:-1.1}
tty: true
stdin_open: true
user: ${MY_USER:-jenkins}
networks:
local:
ipv4_address: 10.0.0.50
hostname: "app"
ports:
- 8000
volumes:
- ${WORKSPACE}:/app
environment:
ALLOW_HOST: "*"
PGHOST: postgres
PGUSER: ${POSTGRES_USER}
PGDATABASE: ${POSTGRES_USER}
PGPASSWORD: ${POSTGRES_PASSWORD}
ANONYMIZE: "false"
REDIS_HOST: redis
REDIS_DB: 2
APP_PATH: ${APP_PATH}
APP: ${MANDANT}
TIMER: ${TIMER:-20}
EMAIL_BACKEND: "dias.core.log.mail.SmtpEmailBackend"
EMAIL_HOST: "smtpd"
EMAIL_PORT: "25"
robot:
image: myrobot:${ROBOT_TAG:-1.0.9}
user: ${ROBOT_USER:-jenkins}
networks:
local:
ipv4_address: 10.0.0.70
volumes:
- ${WORKSPACE}:/app
- ${ROBOT_REPORTS_PATH}:/APP_Robot_Reports
environment:
APP_ROBOT: ${APP_ROBOT}
TIMER: ${TIMER:-20}
PGHOST: postgres
PGUSER: ${POSTGRES_USER}
PGDATABASE: ${POSTGRES_USER}
PGPASSWORD: ${POSTGRES_PASSWORD}
THREADS: ${THREADS:-4}
tty: true
stdin_open: true
entrypoint: start-robot
networks:
local:
driver: bridge
ipam:
config:
- subnet: 10.0.0.0/24
volumes:
my-postgres:
external: true
name: my-postgres
I have monitored the app stats and nothing is abnormal during testing. Also, manually tested the app in browser and it looks just good with nothing wrong about it.
Note: There is no mismatch between chromedriver and google chrome version (anyway this doesn't matter since the same robot container has worked with other instance where no Docker is used for the Django app)
Anyone has an idea ?

I didn't focus before that I run pabot with 8 processes while django app was started with 2 celery workers. As soon as I increased celery workers to 4 it worked. Not sure though if this is the actually cause but it made sense to me as well as it worked.
celery -A server -c ${CELERY_CONCURRENCY:-2} worker

'WARNING:: Cannot resolve hostname: [Errno -2] Name or service not known' when trying to connect Zookeeper with Pysolr

I am trying to run a docker-compose that will let me have a Zookeeper ensemble that manages my SolrCloud. Everything runs and from every way I've checked inside the container, my Zookeeper ensemble appears to be up and running. Instead, everytime I try to connect I get an error that the name or service could not be found.
I've tried using different docker-compose.ymls, I've tried changing the name of my containers in docker, I've tried changing up the ports in the connection string, I've tried changing the hostname in the connection string, and I've tried the localhost for the connection string.
solr1:
container_name: solr1
image: solr:5-slim
ports:
- "8981:8983"
environment:
- ZK_HOST=zoo1:2181,zoo2:2181,zoo3:2181
networks:
- solr
depends_on:
- zoo1
- zoo2
- zoo3
volumes:
- data:/var/solr
command: >
sh -c "solr-precreate users"
solr2:
image: solr:5-slim
container_name: solr2
ports:
- "8982:8983"
environment:
- ZK_HOST=zoo1:2181,zoo2:2181,zoo3:2181
networks:
- solr
depends_on:
- zoo1
- zoo2
- zoo3
solr3:
image: solr:5-slim
container_name: solr3
ports:
- "8983:8983"
environment:
- ZK_HOST=zoo1:2181,zoo2:2181,zoo3:2181
ports:
- 8983:8983
networks:
- solr
depends_on:
- zoo1
- zoo2
- zoo3
zoo1:
image: zookeeper:3.4
container_name: zoo1
restart: always
hostname: zoo1
ports:
- 2181:2181
environment:
ZOO_MY_ID: 1
ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
networks:
- solr
zoo2:
image: zookeeper:3.4
container_name: zoo2
restart: always
hostname: zoo2
ports:
- 2182:2181
environment:
ZOO_MY_ID: 2
ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
networks:
- solr
zoo3:
image: zookeeper:3.4
container_name: zoo3
restart: always
hostname: zoo3
ports:
- 2183:2181
environment:
ZOO_MY_ID: 3
ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
networks:
- solr
and then my Python code is
import pysolr
def connect_solrcloud():
zookeeper = pysolr.ZooKeeper("zoo1:2181,zoo2:2181,zoo3:2181")
solr = pysolr.SolrCloud(zookeeper, "users")
solr.ping()
connect_solrcloud()
I would expect that the Zookeeper object is able to connect and then I would be able to access the "users" core I created in my docker container. Instead I get an error saying
WARNING:: Cannot resolve zoo1: [Errno -2] Name or service not known
WARNING:: Cannot resolve zoo2: [Errno -2] Name or service not known
WARNING:: Cannot resolve zoo3: [Errno -2] Name or service not known
I don't know if this is a docker-compose issue or if it's the way I set Zookeeper up. It appears no one else online has a problem here. They either have problems standing Zookeeper up or some issue once it's connected.

Found my issue. My web container did not include
networks:
- solr
so it wasn't able to access zookeeper.