Python requests.get() from inside a request will not complete - python

When I execute the line from inside a request:
page = requests.get("http://localhost:5000/some/page/")
with DEBUG logging turned on, the output is:
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:5000
send: 'GET /some/page/ HTTP/1.1\r\nHost: localhost:5000\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.22.0\r\n\r\n'
and execution will not progress past this point. Am I missing a step somewhere?
Update:
This is happening for requests that themselves contain a requests.get(). So, I think flask is trying to do two different things at once and the dev server is unable to handle that. That also explains why this seems to work on my staging server.
I've tried running Flask with threaded=True, but that didn't make a difference. Any ideas on a fix for the purposes of local dev and testing?
Update2:
Wrapping in with app.test_client() as c: and using c.get() works for localhost, but fails in staging

Related

"Error during WebSocket handshake" when hosted on Heroku

I'm making a chat server on Django and atm trying to implement websockets.
It works fine locally but whenever I launch it on Heroku, websocket is unreachable.
Here's my client script:
var loc = window.location
var ws_start = 'ws://'
if (loc.protocol == 'https:'){
ws_start = 'wss://'
}
var endpoint = ws_start + loc.host + loc.pathname
var socket = new WebSocket(endpoint)
Full error code:
WebSocket connection to 'wss://my-app.herokuapp.com/chat' failed: Error during WebSocket handshake: Unexpected response code: 500
I've seen a variety of similar questions already, and they are either left unanswered or delve into dealing with SSL certificate. There's one answer that would potentially save me (and other folks) alot of frustration if anyone was to confirm it's true. It's quite old and there's no feedback after it was posted: https://stackoverflow.com/a/45173822/7446564.
Thanks to Heroku logs, I was able to get the actual error message:
Django daphne asgi: Django can only handle ASGI/HTTP connections, not websocket
This answer helped me fix it: https://stackoverflow.com/a/59909118/7446564
So in conclusion: if you have an error launching websockets on Django, make sure your .asgi file is properly set up. I'll also attach my Procfile below since setting it up first time was also a little journey and I hope it might be helpful as well:
web: daphne my-app.asgi:application --port $PORT --bind 0.0.0.0 -v2

How to enable CORS in AppRTC?

I have deployed my own apprtc server with collider & turn server configured in Google App engine locally with virtualbox. Everything is working properly but I want to use this apprtc server in my another project that is in anoter IP. So, Apprtc IP and my project IP is different. Now, when I included apprtc.debug.js & appwindow.js files to my project it cannot initialize loadingParams and says this error message "Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://<IP OF APPRTC>:8080/params. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing)".
However, entering http://<IP OF APPRTC>:8080/params in the browser gives me correct response but not in the project called in appwindow.js .After a bit of googling I added below lines in sendUrlRequest function :
xhr.setRequestHeader('Access-Control-Allow-Origin', '*');
xhr.setRequestHeader('Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept');
Now it says "CORS request did not succeed".
How can I make it work? Do I have to enable cors in server side also, if so where and how to do that? Please help me, I'm totally lost here...

503 Reponse when trying to use python request on local website

I'm trying to scrape my own site from my local server. But when I use python requests on it, it gives me a response 503. Other ordinary sites on the web work. Any reason/solution for this?
import requests
url = 'http://127.0.0.1:8080/full_report/a1uE0000002vu2jIAA/'
r = requests.get(url)
print r
prints out
<Response [503]>
After further investigation, I've found a similar problem to mine.
Python requests 503 erros when trying to access localhost:8000
However, I don't think he's solved it yet. I can access the local website via the web browser but can't access using the requests.get function. I'm also using Django to host the server.
python manage.py runserver 8080
When I use:
curl -vvv http://127.0.0.1:8080
* Rebuilt URL to: http://127.0.0.1:8080/
* Trying 10.37.135.39...
* Connected to proxy.kdc.[company-name].com (10.37.135.39) port 8099 (#0)
* Proxy auth using Basic with user '[company-id]'
> GET http://127.0.0.1:8080/ HTTP/1.1
> Host: 127.0.0.1:8080
> Proxy-Authorization: Basic Y2FhNTc2OnJ2YTkxQ29kZQ==
> User-Agent: curl/7.49.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: BlueCoat-Security-Appliance
< Location:http://10.118.216.201
< Connection: Close
<
<HTML>
<HEAD><TITLE>Redirection</TITLE></HEAD>
<BODY><H1>Redirect</H1></BODY>
* Closing connection 0
I cannot request a local url using python requests because the company's network software won't allow it. This is a dead end and other avenues must be pursued.
EDIT: Working Solution
>>> import requests
>>> session = requests.Session()
>>> session.trust_env = False
>>> r = session.get("http://127.0.0.1:8080")
>>> r
<Response [200]>
Maybe you should disable your proxies in your requests.
import requests
proxies = {
"http": None,
"https": None,
}
requests.get("http://127.0.0.1:8080/myfunction", proxies=proxies)
ref:
https://stackoverflow.com/a/35470245/8011839
https://2.python-requests.org//en/master/user/advanced/#proxies
HTTP Error 503 means:
The Web server (running the Web site) is currently unable to handle the HTTP request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. Some servers in this state may also simply refuse the socket connection, in which case a different error may be generated because the socket creation timed out.
You may do following things:
Check you are able to open URL in the browser
If URL is opening, then check the domain in your code, it might be incorrect.
If in browser also it is not opening, your site may be overloaded or server resources are full to perform request
The most common cause of a 503 error is that a proxy host of some form is unable to communicate with the back end. For example, if you have Varnish trying to handle a request but Apache is down.
In your case, you have Django running on port 8080. (That's what the 8080 means). When you try to get content from 127.0.0.1, though, you're going to the default HTTP port (80). This means that your default server (Apache maybe? NginX?) is trying to find a host to serve 127.0.0.1 and can't find one.
You have two choices. Either you can update your server's configuration, or you can include the port in the URL.
url = 'http://127.0.0.1:8080/full_report/a1uE0000002vu2jIAA/'

WSO2 API POST Python Web Service - Empty or No Payload to server

I am not able to send request payload to my POST service from WSO2.
On rest console, my service is working.
From WSO2 server I am able to do curl to my server with successful response.
here is my API configuration
Payload to send:
{"query":"Hi I am a POST query parameter"}
My server is receiving {} as request payload. It expect RAW body in JSON (as above) in payload. I have tried all combinations for Parameter Type, but still not able to send payload to my server from WSO2.
How can I do this?
EDIT 1
I have tried all possible ways of sending data including following.
Am I doing something wrong here???
and
From both I get error that my payload is empty or incorrect!!
Edit 2
I am able to connect with Java based services but not with Python based services.
Do I need any special settings on my python server?
enable wirelogs and check following
payload is coming into the API manager (swagger -> AM )
Payload is going out from api manager (AM -> backend)
Also check the request headers coming in and going out and compare them with the stuff from curl request (successful request)
I am using Flask and I am afraid Flask can not deal with this issue currently.
I could reproduce this issue, the message send to back-end correctly, but Python only handle message until timeout.
Python Flask cannot receive post request from WSO2
The work-round may be using Java or Python get method.
I solved this problem by using apache to proxy this request.
I think this is related with wsgi.
Processing chunked encoded HTTP POST requests in python (or generic CGI under apache)
ProxyPass / http://localhost:8001/
ProxyPassReverse / http://localhost:8001/

Using Python requests to GET not working - web client and browser works

I have my web app API running.
If I go to http://127.0.0.1:5000/ via any browser I get the right response.
If I use the Advanced REST Client Chrome app and send a GET request to my app at that address I get the right response.
However this gives me a 503:
import requests
response = requests.get('http://127.0.0.1:5000/')
I read to try this for some reason:
s = requests.Session()
response = s.get('http://127.0.0.1:5000/')
But I still get a 503 response.
Other things I've tried: Not prefixing with http://, not using a port in the URL, running on a different port, trying a different API call like Post, etc.
Thanks.
Is http://127.0.0.1:5000/ your localhost? If so, try 'http://localhost:5000' instead
Just in case someone is struggling with this as well, what finally worked was running the application on my local network ip.
I.e., I just opened up the web app and changed the app.run(debug=True) line to app.run(host="my.ip.address", debug = True).
I'm guessing the requests library perhaps was trying to protect me from a localhost attack? Or our corporate proxy or firewall was preventing communication from unknown apps to the 127 address. I had set NO_PROXY to include the 127.0.0.1 address, so I don't think that was the problem. In the end I'm not really sure why it is working now, but I'm glad that it is.

Categories