Troubleshooting "Connection reset by peer" - python

I have a web api (Flask) and a mobile app (Xamarin.Forms) that uses it. Everything had been working well for a few months until a week ago. Suddenly mobile app clients started throwing "Connection reset by peer" when trying to access the web api. The exact exception message is:
Read error: ssl=0xd6269d18: I/O error during system call, Connection reset by peer
It seems to happen randomly - sometimes everything works well, sometimes not.
On the clients' side, requests are made using the System.Net.Http.HttpClient:
public async Task<string> GetClientData(string token)
{
HttpResponseMessage response =
await httpClient.GetAsync(server_url + Resx.Urls.get_route + "?token=" + token);
return await response.Content.ReadAsStringAsync();
}
The server is hosted on Heroku. Even when clients throw exceptions, the Heroku logs show that the requests were correctly handled (status 200).
An example route of my server:
#app.route('/get', methods=['GET'])
def get():
clients = get_users() # database call
token = request.args.get('token')
for c in clients:
if(c.token == token):
return c.to_json()
abort(400) # no client with such token
I wrote a short python script that tries to use the api the same way my mobile app does, and it seems that the problem does not occur there.
Where should I look for the solution? Is it more likely that something is wrong with the Flask server, or is it a problem with the mobile app?

Related

Android app written in kotlin with the volley library fails to call my backend, which was written in python with django

so as the post says, i'm making an app, for which i have a server/rest api written with the django framework. When i try to call the api using volley, i fail to get any response, and i don't see it show up on my server's dashboard
The server is running on my local machine, and the app is running on an emulator, since i'm using android studio
i'd like to send a request and display the response in a textview on the app, for now thats all i need to continue onto the next part
What ends up happening instead is that it seems to not even hit my server, the app displays the text i set for if the request fails, adn it doesn't show up on the dashboard of my server
here's basically all the code in my mobile app
val textView = findViewById<TextView>(R.id.text)
val queue = Volley.newRequestQueue(this)
val url = "http://127.0.0.1:8000//index"//i also tried to use 192.168.1.2 as the host ip. the port is always 8000
val stringRequest = StringRequest(Request.Method.GET, url,
Response.Listener<String> { response ->
// Display the first 500 characters of the response string.
textView.text = "Response is: ${response.substring(0, 500)}"
},
Response.ErrorListener { textView.text = "That didn't work!" })
queue.add(stringRequest)
Please check the logs. They should show you a security error because you use HTTP clear text traffic.
With the default settings, apps aren't allowed to transmit cleartest HTTP traffic.
You must either add an SSL certificate to your backend or allow HTTP traffic.
Check out this StackOverflow thread to see how to enable clear text web requests.

GoneException when calling post_to_connection on AWS lambda and API gateway

I want to send a message to a websocket client when it connects to the server on AWS lambda and API gateway. Currently, I use wscat as a client. Since the response 'connected' is not shown on the wscat console when I connect to the server, I added post_to_connection to send a message 'hello world' to the client. However, it raises GoneException.
An error occurred (GoneException) when calling the PostToConnection
operation
How can I solve this problem and send some message to wscat when connecting to the server?
My python code is below. I use Python 3.8.5.
import os
import boto3
import botocore
dynamodb = boto3.resource('dynamodb')
connections = dynamodb.Table(os.environ['TABLE_NAME'])
def lambda_handler(event, context):
domain_name = event.get('requestContext',{}).get('domainName')
stage = event.get('requestContext',{}).get('stage')
connection_id = event.get('requestContext',{}).get('connectionId')
result = connections.put_item(Item={ 'id': connection_id })
apigw_management = boto3.client('apigatewaymanagementapi',
endpoint_url=F"https://{domain_name}/{stage}")
ret = "hello world";
try:
_ = apigw_management.post_to_connection(ConnectionId=connection_id,
Data=ret)
except botocore.exceptions.ClientError as e:
print(e);
return { 'statusCode': 500,
'body': 'something went wrong' }
return { 'statusCode': 200,
"body": 'connected'};
Self-answer: you cannot post_to_connection to the connection itself in onconnect.
I have found that the GoneException can occur when the client that initiated the websocket has disconnected from the socket and its connectionId can no longer be found. Is there something causing the originating client to disconnect from the socket before it can receive your return message?
My use case is different but I am basically using a DB to check the state of a connection before replying to it, and not using the request context to do that. This error's appearance was reduced by writing connectionIds to DynamoDB on connect, and deleting them from the table upon disconnect events. Messaging now writes to connectionIds in the table instead of the id in the request context. Most messages go through but some errors are still emitted when the client leaves the socket but does not emit a proper disconnect event which leaves orphans in the table. The next step is to enforce item deletes when irregular disconnections occur. Involving a DB may be overkill for your situation, just sharing what helped me make progress on the GoneException error.
We need to post to connection after connecting (i.e. when the routeKey is not $connect)
routeKey = event.get('requestContext', {}).get('routeKey')
print(routeKey) # for debugging
if routeKey != '$connect': # if we have defined multiple route keys we can choose the right one here
apigw_management.post_to_connection(ConnectionId=connection_id, Data=ret)
#nemy's answer is totally true but it doesn't explain the reason. So, I just want to explain...
So, first of all What is GoneException or GoneError 410 ?
A 410 Gone error occurs when a user tries to access an asset which no longer exists on the requested server. In order for a request to return a 410 Gone status, the resource must also have no forwarding address and be considered to be gone permanently.
you can find more details about GoneException in this article.
In here, GoneException has occured; it means that the POST connection we are trying to make, doesn't exist - which fits perfectly in the scenario. Because we still haven't established the connection between Client and Server. The way APIGatewayWebsocketAPIs work is that you request an Endpoint(Route) and that Endpoint will invoke that Lambda Function (In our case it is ConnectionLambdaFunction for $connect Route).
Now, if The Lambda function resolves with statusCode: 200 then and only then the API Gateway will allow the connection to be established. So, basically untill we return statusCode: 200 from our Lambda Function we are not connected and untill then we are totally unknown to server and thats why the Post call that has been made before the return statement itself will throw an error.

Python Flask server getting 'code 400' errors (POST request sent from Telegram-webhook)

Context
I'm currently following this tutorial. - Telegram Bot with Python Tutorial #3: Creating Bot and Webhook | Project
FIRST STEP
I have set-up a Flask server using the following python code:
from flask import Flask
from flask import request
from flask import Response
import json
app = Flask(__name__)
#app.route('/', methods=['POST', 'GET'])
def index():
if request.method == 'POST':
print(request)
message = request.json()
with open('telegram_request.json', 'w', encoding='utf-8') as filename:
json.dump(message, filename, ensure_ascii=False, indent=4)
# prevents telegram from spamming
return Response('Ok', status=200)
else:
return """
<h1> Flask Server </h1>
<h2> Up and running </h2>
"""
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0', port=8443)
SECOND STEP
I port forwarded port 8443 in my router to make the server visible to the outside world (tunneling step in tutorial).
The domain name "myprivatedomain.com:8443" now redirects/refers to the flask server that is set-up.
THIRD STEP
I set-up the Telegram-API webhook correctly, getting the following response code from Telegram:
{"ok":true,"result":true,"description":"Webhook was set"}
NOW
Before sending a message in the Telegram chat: there were no errors.
After sending a message in the chat, the following errors popped up:
code 400, message Bad HTTP/0.9 request type
('RANDOM BYTE VALUES like \x00\x03')
code 400, message Bad request syntax
('RANDOM BYTE VALUES like \x00\x03')
code 400, message Bad request version
('RANDOM BYTE VALUES like \x00\x03')
WHAT I WANT
According to the tutorial, you can write a .json file when Telegram makes a POST request (see example: here). I want to save the message object provided by the Telegram webhook (as showcased in the tutorial video). Using the webhook for getting updates is better than constantly querying the getUpdates() method; that method returns old messages also.
WHAT I'VE TRIED
I've tried to add:
ssl_context='adhoc'
to
app.run(debug=True, host='0.0.0.0', port=8443)
to make the connection HTTPS.
While using this ssl_context, loading the homepage isn't possible either..
PREFERABLE OUTPUT
When the user sends a message inside the Telegram chat --> Python saves a .json file of the message object.
You need to have SSL enabled for this to work. Telegram is trying to initiate an SSL session with your server, but you don't have SSL enabled, so you are seeing the bad request.
ssl_context='adhoc' may work for a test app, but I also have a hunch that telegram requires a VALID SSL certificate, not just an ad-hoc (or self-signed one). Notice the lock to the left of the URL in the video and the lack of a security warning that would be present with an invalid or self-signed certificate.
To make sure SSL is working set the ssl_context to adhoc, start the app, and browse to https://myprivatedomain.com:8443/index. If you can browse to it, then Telegram will also be able to browse to it, after getting a valid certificate, of course.
Next, to get a valid (and free) SSL certificate you can use LetsEncrypt.
Once you have a valid SSL certificate and key file you can pass the ssl_context argument to app.run with a tuple of the path to the certificate file and the path to the key file ("/path/to/fullchain.pem", "/path/to/privkey.pem")
Your full run function should look like this
app.run(debug=True, host='0.0.0.0', port=8443, ssl_context=("/path/to/fullchain.pem", "/path/to/privkey.pem"))
Alternatively you can use Apache or Nginx to secure your site with SSL, and reverse proxy to your bot. Those options would normally be used in a final product, so I understand if you don't want to get into the weeds with them right now, but it is good practice regardless.
Hopefully that helps.

flask proxy for ttyd

I am looking for a method to write a simple proxy in flask for ttyd which is an open-source web terminal(https://github.com/tsl0922/ttyd). The most immediate way is to read client request and relay to ttyd server. However, it fails when the websocket is connecting.
My view function is as follows:
#app.route('/')
#app.route('/auth_token.js')
#app.route('/ws')
def ttyd():
if request.path=='/ws':
url = 'ws://192.168.123.172:7681' + request.path
else:
url = 'http://192.168.123.172:7681' + request.path
method = request.method
data = request.data or request.form or None
cookies = request.cookies
headers = request.headers
with closing(
requests.request(method, url, headers=headers, data=data, cookies=cookies)
) as r:
resp_headers = []
for name, value in r.headers.items():
resp_headers.append((name, value))
return Response(r, status=r.status_code, headers=resp_headers)
As you can see, the view function will handle 3 url requests, the first two succeed with status code 200, the third fails with status code 500. The error code in server side is as follows:
requests.exceptions.InvalidSchema: No connection adapters were found for 'ws://192.168.123.172:7681/ws'
I also check the network in two cases(with/without proxy). The picture 'without proxy' means direct type 'http://192.168.123.172:7681', it succeeds. The picture 'with proxy' means access ttyd server with flask proxy, it fails.
Without proxy
With proxy
Since I am new to flask and websocket, I am confused about the result. The sHTTPe flask proxy can handle any other http request(e.g. access google.com) but fails in WebSocket connection.
Thank you for telling me why and how can I fix it?
According to Websockets in Flask there is a flask-sockets project at https://github.com/heroku-python/flask-sockets to serve a websocket-endpoint in flask. To make the backend websocket connection to the server you can't use requests but websocket-client, see How do I format a websocket request?.
When I had this problem I solved it using the autobahn-python project, see https://github.com/arska/stringreplacingwebsocketproxy/
Cheers,
Aarno

Health check failed when service is still running

I'm using google health check in order to send request to my flask client to make sure my service is alive.
the same route in flask client sends request to two more flask clients to make sure the other two is also alive.
For some reason the request sometimes fails when the service is still running.
I tries to figure out why but there is nothing in my services logs that indicates that something happened and on most cases it works fine.
This is my code:
#GET /health_check//
def get(self):
try:
for service in INTERNAL_SERVICES_HEALTH_CHECKS:
client = getattr(all_clients, service + '_client')
response = client.get('g_health_check')
except Exception, e:
sentry_client.captureMessage('health check failed for '+env+ ' environment. error log:' + repr(e))
return output_json({'I\'m Not fine!':False}, requests.codes.server_error)
return output_json({'I\'m fine!':True}, requests.codes.ok)
If anyone has any suggestions I will be happy to try and fix it.

Categories