urllib3 request time out using inside ThreadPoolExecutor in flask app - python

I have a python script that works perfectly as a standalone module.
I am trying to use the output of that script in my flask app by initiating that module as a background job using concurrent.futures import ThreadPoolExecutor but unfortunately urllib3 gets failed with the following error only either I use ThreadPoolExecutor or Redis process for background task.
error with request HTTPSConnectionPool(host='www.reddit.com', port=443): Max retries exceeded with url: /api/v1/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fe8f9490a50>: Failed to establish a new connection: [Errno 2] Lookup timed out'))
That script is using praw library that further use urllib3 inside it. Is there any way I can run that module just like it works fine using the main thread?

Related

How to run a Python 3 script in OWASP ZAP?

I'm using ZAP to run a scan of a website from the command line, using the form-based authentication script found in the ZAP API Documentation.
java -jar ./zap-2.11.1.jar -script ./auth_script.py
However, it looks like ZAP uses Jython 2.7 instead of Python 3, so running the script this way doesn't work.
I've also tried running the script directly (i.e. python3 auth_script.py), but it throws the following error:
requests.exceptions.ProxyError: HTTPConnectionPool(host='127.0.0.1', port=8080): Max retries exceeded with url: http://zap/JSON/context/action/includeInContext/?contextName=context&regex=https%3A%2F%2Fwebsite.com%2F&apikey=9qFbZD4udTzFVYo0u5UzkZX9iuzbdcJDRAquTfRk
(Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response')))
Has anyone used this form-based authentication script before? How did you get it to work?

Connection error with `requests` on localhost with Docker

I'm trying to develop a client-server application but I'm running into some issues. Both my services are developed locally in a separate docker container.
The client fetches data from the api using requests python library. It works in production but locally, I can't use:
requests.get("http://localhost:PORT/ENDPOINT")
It keeps giving me back:
requests.exceptions.ConnectionError: HTTPConnectionPool(host='0.0.0.0', port=4000): Max retries exceeded with url: /data (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f352ccd42b0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Is there something I'm missing?
After further research, I found my answer here. Basically, when using Docker locally, you simply need to run:
requests.get("http://host.docker.internal:PORT/ENDPOINT")
instead of localhost

How to fix requests.get() failure on gitlab rest api?

I am using python reqests.get() to invoke one of gitlab api's in my Python script running on Windows. It works fine on my dev machine, but it fails with below error message when I run the script in TeamCity job runner.
HTTPSConnectionPool(host='gitlab.myservice.com', port=443): Max retries exceeded with url: /api/v4/projects/1111/pipeline_schedules/111?private_token=******* (Caused by SSLError(SSLError(1, '_ssl.c:503: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure'),))
What is very strange is that very similar script that calls Artifactory rest api runs just fine in the same script even when it runs in TeamCity.
Can anybody shed light on this mystery?
Update
The infra is using Python 2.7.16. When I tested with Python 3.x it worked fine. However, the org is not ready to move to 3.x due to dependencies they cannot solve yet.

Python SSL Problem with PIP, Requests and Other

I've got a problem with a SSLError that appeared since last week.
I've used Python on my machine for a few years without any problem, but now whenever i try to use a library that connects to the web, a SSLError is thrown.
I've tried other solutions to make PIP and Requests work while avoiding the certificate check, but now i need to make it work to use an Azure library.
I know it's not a problem of the Wifi connection i'm using because it works fine on other machines. Could it be something i've installed on the machine? Maybe a VPN? Is there a way to check what is "blocking" the connection?
This is an example of the error when using the Azure Iot Hub library:
ClientRequestError: Error occurred in request., SSLError: HTTPSConnectionPool(host='iothubstreamdemo.azure-devices.net', port=443): Max retries exceeded with url: /devices?api-version=2020-03-01 (Caused by SSLError(SSLError(0, 'unknown error (_ssl.c:3622)'),))
And this is while using requests:
SSLError: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(0, 'unknown error (_ssl.c:3622)'),))
Thanks in advance.

Stop StanfordCoreNLP from Connecting to StanfordCoreNLP server

Whenever I create StanfordCoreNLP for a parsing task,
StanfordCoreNLP('stanford-corenlp-full-2020-04-20', lang='en')
I get this logging
Initializing native server...
java -Xmx4g -cp "/stanford-corenlp-full-2020-04-20/*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port 9032
Server shell PID: 11991
The server is available.
Can StanfordCoreNLP work in python offline without connecting to the server?
Is there a command to stop the logging to screen in stanfordCoreNLP?
On applying
On applying the parser, self.nlp_src.parse(sentence), I got another logging of the form below:
{'properties': "{'annotators': 'pos,parse', 'outputFormat': 'json'}", 'pipelineLanguage': 'en'}
Is there a way to stop the above logging too?
Lastly, I got this error in the process of using the parser in the stanford coreNLP as explained above.
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=9032): Max retries exceeded with url: /?properties=%7B%27annotators%27%3A+%27pos%2Cparse%27%2C+%27outputFormat%27%3A+%27json%27%7D&pipelineLanguage=en (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe04121d6d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
What is the cause of this error? What do I do to prevent this?
Hi if you'd like to use Stanford CoreNLP in Python I'd advise using our official Python package Stanza, available here: https://stanfordnlp.github.io/stanza/
If you want to access Stanford CoreNLP in Python, Stanza will need to launch a Java server process on your local machine. This does not require connecting to the internet or any external network. The Java server process is running on your local machine and the Python process is communicating with it.
There is a detailed write up about how to launch a Java server in Python here: https://stanfordnlp.github.io/stanza/corenlp_client.html#overview
Stanza itself has part-of-speech, dependency parsing, and named entity recognition that is completely native Python and does not involve a server of any sort. More details here: https://stanfordnlp.github.io/stanza/installation_usage.html

Categories