I'm using requests to send a post to an API with proxies however I'm trying to make the console output look clean, not spamming with errors that the proxy 'fails to connect'. Is there a way to hide errors like this one?:
HTTPSConnectionPool(host='test.com', port=443): Max retries exceeded with url: /v1/ (Caused by ProxyError('Cannot connect to proxy.', OSError(0, 'Error')))
I set a timeout for as well, nothing seems to be working and my console output is being spammed with that
Related
I'm currently working with an Imgur Bot using Python and requests lib with a rotating proxy. I run multiple instances of this bot, but sometimes, some of them got this error:
HTTPSConnectionPool(host='api.imgur.com', port=443): Max retries exceeded with url: /3/credits (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response')))
Here is how the session is defined:
session = requests.Session()
session.proxies = {"http": cre["Proxy"]}
I have no idea why it crashes since only a part instances I run got this error. The others work well.
Thanks, mates!
Maybe your proxies are wrong configurated?
I think you run into some form of "requesting too much too fast".
Try to delay some requests (like 3 seconds).
I am using the following code. I have removed the actual 'username' and 'password' and 'proxy_address' objects for privacy reasons.
auth = HTTPProxyAuth('username', 'password')
r = requests.get(hyperlink, proxies={'http': proxy_address, 'https': proxy_address},
auth=auth, timeout=5)
I get the following error:
HTTPSConnectionPool(host='bafybeihpjhkeuiq3k6nqa3fkgeigeri7iebtrsuyuey5y6vy36n345xmbi.ipfs.dweb.link', port=443): Max retries exceeded with url: /45 (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required')))
What should I do to resolve this issue? I have tried looking into the documentation for python Requests but I haven't been able to find a solution for my implementation. Thanks!
So, I was sending a request using the requests library in Python 3.9.1. The problem is, when I tried to use an HTTP proxy it gave me this error:
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x000002B08D6BC9A0>: Failed to establish a new connection:
[WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond')))
This my code, I would appreciate any help.:
import requests
for proxy in open('proxies.txt','r').readlines():
proxies = {
'http': f'http://{proxy}',
'https': f'http://{proxy}'
}
e = requests.get('https://google.com/robots.txt',proxies=proxies)
open('uwu.txt','a').write(e.text)
print(e.text)
I am pretty sure it is not problem with my proxies as they are really good private proxies with 100 gigs of bandwidth. (from zenum.io).
I have a CherryPy server, which is processing requests using a thread pool of, say, 10 threads.
If I send many requests in parallel (~200 processes constantly sending), I start seeing in my logs (client side) 3 types of errors:
HTTPConnectionPool(host='localhost', port=8080):
Max retries exceeded with url: /my-url
(Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1095da780>:
Failed to establish a new connection: [Errno 61] Connection refused'))
BrokenPipeError(32, 'Broken pipe'))
and
('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))
Why 3 different types, btw?
I suppose I see these errors because there were too many requests sent.
I adjusted server.socket_queue_size to match the number of parallel requests I send, and it started working fine. My server configuration looks like this:
cherrypy.config.update({
'server.socket_port': 8080,
'server.socket_host': 'localhost',
'server.thread_pool': 10,
'server.socket_queue_size': 200
})
However, I'm struggling to find the default value of socket_queue_size set in CherryPy. How much is it? cherrypy.config['server.socket_queue_size'] gives me nothing unsell I set this value myself.
So, what is the default value? How do I determine reasonable socket_queue_size?
I would like to surpass SSL verification when I send post request with the python requests module, but it gave me an error even when I added parameter verify=False:
My request:
response = session.request(rest_body["method"], url, data=data,
headers=headers, proxies=PROXIES,
cookies=req_cookies, verify=False)
Error (new lines added):
HTTPSConnectionPool(host='', port=443): Max retries exceeded with url: /oauth/token
(Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object
at 0x7f174234e5f8>: Failed to establish a new connection: [Errno 111]
Connection refused',))
What does this error mean, and how can I avoid it?