Querying docker server in parallel raises connection error - python - python

In Python3, I am using get.requests() in a parallel loop to query a server that I have running in a docker container (specifically an OSRM server).
I get this error after it successfully queries a few:
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
I think the Docker container has a limit on the number of queries at a time or from a single IP, but I don't know how to fix it. I'm assuming it's in the Docker settings.
I initiate the docker container using: docker run --name osrm-nc -t -i -p 6002:5000 -v /homedirs/tml62/osm:/data osrm/osrm-backend osrm-routed --algorithm mld /data/north-carolina-latest.osrm, so noteably, I'm assigning the ports
I've set the retries so it can repeat and that sometimes fixes it, but it causes delays and sometimes will still break.

Related

Connect to Apache Phoenix Docker Container Using Python

I am running an Apache Phoenix container along with Hbase using the following command,
docker run -it --name phoenix -p 8765:8765 avapno/apache-phoenix
This is done as explained here,
https://hub.docker.com/r/avapno/apache-phoenix
The containers run without a problem.
Now, I want to connect to Phoenix using Python. This is the code I have used to do so,
import phoenixdb
database_url = 'http://localhost:8765/'
conn = phoenixdb.connect(database_url, autocommit=True)
This does not, however, seem to work. I keep getting the following error,
ConnectionError: HTTPConnectionPool(host='localhost', port=8765): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ffb03782080>: Failed to establish a new connection: [Errno 111] Connection refused'))
How exactly do I get this done correctly?
Also, is there a better way to get Hbase + Phoenix running?

How to run marqo on macbook

i have been using marqo library for a week now and it has been working well on my windows machine, but today i am trying to run the same program on my m1 mac but i keep getting error.
i tried reading through the documentation and i saw this:
Marqo does not yet support the docker-in-docker backend configuration for the arm64 architecture. This means that if you have an M series Mac, you will also need to run marqo's backend, marqo-os, locally.
so i followed the instructions in the documentation, which was to run these two commands on seperate terminal
docker rm -f marqo-os; docker run -p 9200:9200 -p 9600:9600 -e "discovery.type=single-node" marqoai/marqo-os:0.0.2-arm
docker rm -f marqo; docker run --name marqo --privileged \
-p 8882:8882 --add-host host.docker.internal:host-gateway \
-e "OPENSEARCH_URL=https://localhost:9200" \
marqoai/marqo:0.0.3
which i did but i am still getting the same error:
marqo.errors.BackendCommunicationError: MarqoWebError: BackendCommunicationError Error message: Error communicating with OpenSearch backend: HTTPConnectionPool(host='localhost', port=8882): Max retries exceeded with url: /indexes/my-first-index/search?&device=cpu (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1042deb90>: Failed to establish a new connection: [Errno 61] Connection refused'))
i was also having the same issue.Try uninstalling docker, restarting your machine and reinstall docker, then run the two commands again hopefully it should work.

Can't send any request inside docker container

I am developing a Flask application with Docker setup. My Os is Linux Mint. Suddenly can't send any request from my application. It's showing error like this:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.kickbox.io', port=443):
Max retries exceeded with url: /v2/verify?email=harun1393%40gmail.com&timeout=6000 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f71b5e89c90>:
Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
ping google.com is working fine from my OS but showing Temporary failure in name resolution inside docker container.
docker exec -it web /bin/bash
root#b72e895be44a:/web# ping google.com
ping: google.com: Temporary failure in name resolution
/etc/resolve.conf from OS:
nameserver 192.168.0.1
/ect/resolve.conf inside docker container
nameserver 127.0.0.11
options ndots:0
cat /etc/docker/daemon.json
{
"dns": ["192.168.0.1", "8.8.8.8"]
}
I used host network in docker-compose.yml, but result is same.
website:
# network_mode: host
build:
context: .
network: host
container_name: web
I have tried a lot, but couldn't solve my problem.

How to link selenium standalone docker container properly with host networking?

I'm running two docker containers as follows:
one is for selenium/standalone-chrome
$ docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chrome
and the main container uses host networking to connect to MySQL which is running on localhost
$ docker run --rm --network="host" $(IMAGE_REPO)
but when I tried to initiate the driver in the main container
from selenium import webdriver
driver = webdriver.Remote(command_executor='http://127.0.0.1:4444/wd/hub',
desired_capabilities=webdriver.DesiredCapabilities.CHROME)
I got error message which is either
urllib3.exceptions.ProtocolError: ('Connection aborted.', BadStatusLine("''",))
or
urllib3.exceptions.ProtocolError: ('Connection aborted.', error(104, 'Connection reset by peer'))
did I forget anything / any parameters when trying to establish the containers? or the value passing to command_executor should not be 'http://127.0.0.1:4444/wd/hub'?
I also tried to use docker bridge networking, but then I have no idea how to connect to MySQL server on localhost
SQLALCHEMY_URI = '{driver}://{user}:{pwd}#{host}/{db}?charset=utf8' \
.format(
driver='mysql+pymysql',
host='127.0.0.1:3306',
user='root',
pwd='0000',
db='default'
)
Update the following libraries in your Dockerfile:
Set Selenium library version to 3.3.1 (selenium==3.3.1).
i.e: Make use of a Selenium image, whose version is 3.3.1.
Use selenium/node-chrome:3.3.1 instead of selenium/node-chrome:3.141.59-gold

docker-py: Connection reset by peer

I am attempting to create a docker image via docker-py and I am using the following code:
import docker
import os
docker_client = docker.from_env()
path = os.path.dirname(os.path.abspath(__file__)) + "/container"
docker_client.images.build(path=path, tag='container-tag')
The last line throws this error:
requests.exceptions.ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))
The user I'm using is a member of the docker group and I have checked the permissions of the socket file.
Since docker_client.containers.list() works for you, your issue is not with docker connection. It is that the build is failing for some reason.
Run journalctl -f -n10 in another terminal and then run your code and see if you can find what is going wrong. That will give your clear exception happening when you call
docker_client.images.build(path=path, tag='container-tag')

Categories