I have a locally single-node hosted hadoop. my name and datanode are same.
I'm trying to create a file using python library.
self.hdfs = PyWebHdfsClient(host='192.168.231.130', port='9870', user_name='kush',
base_uri_pattern="http://192.168.231.130:9870/webhdfs/v1/", timeout=1)
if not self.hdfs.exists_file_dir(path):
self.hdfs.make_dir(path)
error-->self.hdfs.create_file("{}/results_{}.csv".format(path, name),
'word,negative,neutral,positive,compound\n')
the file exist and make_dir work correctly. But my create file keeps throwing an error
The exception I got is this:
requests.exceptions.ConnectionError: HTTPConnectionPool(host='kush', port=9864): Max retries exceeded with url: /webhdfs/v1/user/kush/data/results_4104.csv?op=CREATE&user.name=kush&namenoderpcaddress=192.168.231.130:9000&createflag=&createparent=true&overwrite=false (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001F8C3FB1C40>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))
I have already allowed firewall for 9000 , 9870 & 9864.
Thanks in advance. Help will be greatly appreciated
apparently i just needed to add 'kush' to /etc/hosts but one issue was my hostname and machine name was same
so i changed my machine name and then added it to my /etc/hosts both windows and linux
and voila!
Related
This question already has an answer here:
Python requests in Docker Compose containers
(1 answer)
Closed last month.
I want to be able to connect to a third-party .NET API I am running as a docker image within a container. When I connect to the ip address and port on my browser it works though when I try and run a get request in python it does not. I am not sure what I am doing wrong.
The Django python application is an image within the container using 0.0.0.0:8000.
This is the docker-compose.yml file for a .NET image I am running from within a container:
mt4rest:
image: mt4rest:latest
ports:
- "0.0.0.0:5000:80"
Here are the ports as per docker inspect
"Ports": {
"443/tcp": null,
"80/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "5000"
}
]
},
When I run
http://0.0.0.0:5000/Ping
or
http://127.0.0.1:5000/Ping
in my browser, I get OK though when I run the same endpoints in Python:
response = requests.get('http://0.0.0.0:5000/Ping')
or
response = requests.get('http://127.0.0.1:5000/Ping')
I receive the following error:
requests.exceptions.ConnectionError: HTTPConnectionPool(host='0.0.0.0', port=5000):
Max retries exceeded with url: /Ping (Caused by
NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe7aa5af640>:
Failed to establish a new connection: [Errno 111] Connection refused'))
or
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000):
Max retries exceeded with url: /Ping (Caused by
NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6322e2b640>:
Failed to establish a new connection: [Errno 111] Connection refused'))
I understand this to mean that the address for the request is not found.
I am not sure what I am doing wrong.
Lets call your host as HOST_BASE.
You are having two containers, one is exposing 5000 on host. Its HOST_BASE:5000.
Other is your python-app container.
Now, you are running another python-app in container on host HOST_BASE. And try to hit: http://0.0.0.0:5000, which is not accessible. Note: its 0.0.0.0 is localhost of the container itself.
You want to connect to HOST_BASE:5000, which is not localhost or 0.0.0.0
You need to configure networking between your two containers. Or, for simplicity, use --network host while running both of your container.
Hope that helps.
There is an acceptable answer for this question on stackoverflow which solves the problem:
Python requests in Docker Compose containers
I'm trying to run some code from this website but I don't understand why I get this error:
qbittorrentapi.exceptions.APIConnectionError: Failed to connect to qBittorrent. Connection Error: ConnectionError(MaxRetryError("HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /api/v2/auth/login (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001FA519F5840>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))"))
The code in question:
import qbittorrentapi
# instantiate a Client using the appropriate WebUI configuration
qbt_client = qbittorrentapi.Client(
host='localhost',
port=8080,
username='admin',
password='adminadmin',
)
# the Client will automatically acquire/maintain a logged-in state
# in line with any request. therefore, this is not strictly necessary;
# however, you may want to test the provided login credentials.
try:
qbt_client.auth_log_in()
except qbittorrentapi.LoginFailed as e:
print(e)
# display qBittorrent info
print(f'qBittorrent: {qbt_client.app.version}')
print(f'qBittorrent Web API: {qbt_client.app.web_api_version}')
for k,v in qbt_client.app.build_info.items(): print(f'{k}: {v}')
# retrieve and show all torrents
for torrent in qbt_client.torrents_info():
print(f'{torrent.hash[-6:]}: {torrent.name} ({torrent.state})')
# pause all torrents
qbt_client.torrents.pause.all()
I'd really appreciate some help with this, thanks ahead :)
So I have pytests (made using requests) and I run them while my server is running, to test if they work. They work fine but currently, the base URL is hardcoded to port number 3000 since my server is running on port 3000. How can I make sure these pytests run even if my port number is not hardcoded, i.e. if my server is running on port 4080 then the tests will succeed without me having to hardcode the URLs.
I tried doing this:
URL = f"http://127.0.0.1:{sys.argv[1]}/user"
since I take in the first argument as my port number when running my server. This does not work though. Any other suggestions for a beginner? Thanks
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /user (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused'))
Everytime I make an external request (including to google.com) I get this response:
HTTPConnectionPool(host='EXTERNALHOSTSITE', port=8080): Max retries exceeded with url: EXTERNALHOSTPARAMS (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x105d8d6d0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known',))
It does seem that your server cannot resolve the hostname into IP, this is probably not Django nor Python problem but your server network setup issue.
Try to reach the same URL with ping tool / wget/curl or troubleshoot DNS with nslookup.
I'm running a local server with my Flask Python program. I am able to GET request the server with curl:
curl http://localhost:5000/ -d "input_data=something" -X GET
and everything goes well.
However, if I try to call it from another Python program this way:
import requests
r=requests.get('http://localhost:5000', json={"input_data": "something"})
I get an error:
HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fe3b510a310>: Failed to establish a new connection: [Errno 111] Connection refused',))
What is the problem here?
Besides, is there a way to insert the server program into a standard Linux pipeline? I mean something along this logic (of course, I know that this concretely doesn't work):
cat "something" | http://localhost:5000 > output