We Keep Coding

starting a python Http server from Java

I am trying to start a Http server writing in python through my java code. Initially it seems to work fine, but after 8 requests it closes again. What am I doing wrong?
the server is started like this:
ProcessBuilder builder = new ProcessBuilder("python", "src\\main\\java\\python\\HttpHandler.py");
Process p = builder.start();
and within the python code the server looks like this:
class HttpHandler(BaseHTTPRequestHandler):
def do_GET(self):
#handle get request
def do_POST(self):
#handle post request
with HTTPServer(('', 8000), HttpHandler) as server:
server.serve_forever()

From the comments, the problem was caused by the server process writing to its standard output or standard error streams, and the Java code not reading from these streams. Eventually a buffer filled up and the server process blocked because it couldn't write to the buffer. Overriding the http log function fixed the problem.

websockets closing unexpectedly

Update 13. Apr: dart code works consistently
Background:
My TV (Samsung 2019 RU7000) offers a secure websocket connection with which json packets can be send to remote control it. For example
"method":"ms.remote.control",
"params":{ "Cmd": "Click", "DataOfCmd":"KEY_MUTE" }
can be send to mute the TV.
To enable access, a handshake is done. The client connects to wss://ip:8002/api/v2/samsung.remote.control?name=value where the value of name is a base64 encoded string. After a successful connection, a popup window appears on the TV which needs to be accepted. If the user accepts the request, the TV sends a JSON response containing a token:
"data":{
"clients":[ ... ],
"id":"...",
"token":"28852140"
},
"event":"ms.channel.connect"
This token is used for authenticating connections with the same name by attaching &token=value to the URL. Sending commands before the user accepts the popup is ignored.
What works
Command line approaches with wscat and curl are working. The TV shows a pop-up and sends a response:
$ wscat -n -c https://192.168.0.227:8002/api/v2/channels/samsung.remote.control?name=aW9Ccm9rZXI=
Connected (press CTRL+C to quit)
< {"data":{"clients":[...], "id":"...", "token":"57940060"}, "event":"ms.channel.connect"}
dart - dart.io.WebSocket
The following code triggers the popup and gets a response as well.
WebSocket ws = await WebSocket.connect(url,
compression: CompressionOptions.compressionOff);
ws.pingInterval = Duration(seconds: 10000);
ws.listen(print, onError: print, onDone: () {
print("done");
print(ws.closeCode);
print(ws.closeReason);
});
await Future.delayed(Duration(seconds: 30));
ws.close();
Problem
I want to use python to connect to the websocket, authenticate my session and send remote control commands. This does not work for the following implementations.
python - websocket liris
The popup does not appear and there is no response from the TV. The socket does not close.
from websocket import create_connection
from ssl import CERT_NONE
sock = create_connection(url, sslopt={"cert_reqs": CERT_NONE})
print(sock.recv())
python - websockets aaugustin
The pop-up does not appear as well and it seems the connection is closed by the TV.
from websockets.client import connect, WebSocketClientProtocol
import ssl
import asyncio
async def connect():
async with connect(url, ssl=ssl.CERT_NONE) as websocket:
res = await websocket.recv()
print(res)
asyncio.get_event_loop().run_until_complete(connect())
The wireshark log shows that it's sending an HTTP GET asking for a websocket upgrade. The TV responds by closing the connection with FIN and RST.
Observations
Connections to publicly available websocket servers do not share the same issues I am experiencing.
Many popular remote controls available for python have the two websocket libraries as the implemented solution. None of these were working for me.
Comparing the packets sent by the python implementations and the command line tools do not show a noticeable difference to me.
wscat sends an 80 byte long packet to the TV every four seconds. I suppose this is ping/pong.
How can I investigate further?

What about trying Tornado's websocket implementation?
async def samsung_ws():
ws_req = HTTPRequest("wss://ip:8002/api/v2/samsung.remote.control?name=value", validate_cert=False)
ws = await websocket_connect(ws_req)
ws.write_message("hello")
while True:
msg = await ws.read_message()
if not msg:
break
print(msg)
Depending on how the webserver is built, you might need to request the webpage before connecting to the websocket because in your wscat example I noticed you are using https:// instead of wss://.
Can you post a sample of a dart run (with the url used)?

Um.... I have some experience on websockets connection using python websockets.
In article, you indicate that your websockets connections will automatically disconnected from the server.
I think is the mechanism call "ping-pong" in the websockets module cause this problem.
The mechanism default status is true, that means you will send a ping sign to the websocket server periodically, if the server doesn't send pong back to you, module will consider that server have been shutdown.So you need to do is just set the "ping-pong"
status to False.
async def connect():
async with connect(url, ssl=ssl.CERT_NONE,close_timeout = None,ping_interval = None) as websocket:
res = await websocket.recv()
print(res)
asyncio.get_event_loop().run_until_complete(connect())
This is all my personal opinion, you can try this out.

Why not using a higher level Python module like requests?
You could try the following:
import requests
params = (
('name', 'aW9Ccm9rZXI='),
)
response = requests.get('https://192.168.0.227:8002/api/v2/channels/samsung.remote.control', params=params)

Detect when Websocket is disconnected, with Python Bottle / gevent-websocket

I'm using the gevent-websocket module with Bottle Python framework.
When a client closes the browser, this code
$(window).on('beforeunload', function() { ws.close(); });
helps to close the websocket connection properly.
But if the client's network connection is interrupted, no "close" information can be sent to the server.
Then, often, even 1 minute later, the server still believes the client is connected, and the websocket is still open on the server.
Question: How to detect properly that a websocket is closed because the client is disconnected from network?
Is there a websocket KeepAlive feature available in Python/Bottle/gevent-websocket?
One answer from Web Socket: cannot detect client connection on internet disconnect suggests to use a heartbeat/ping packet every x seconds to tell the server "I'm still alive". The other answer suggests using a setKeepAlive(true). feature. Would this feature be available in gevent-websocket?
Example server code, taken from here:
from bottle import get, template, run
from bottle.ext.websocket import GeventWebSocketServer
from bottle.ext.websocket import websocket
users = set()
#get('/')
def index():
return template('index')
#get('/websocket', apply=[websocket])
def chat(ws):
users.add(ws)
while True:
msg = ws.receive()
if msg is not None:
for u in users:
u.send(msg)
else:
break
users.remove(ws)
run(host='127.0.0.1', port=8080, server=GeventWebSocketServer)

First you need to add a timeout to the receive() method.
with gevent.Timeout(1.0, False):
msg = ws.receive()
Then the loop will not block, if you send even an empty packet and the client doesn't respond, WebsocketError will be thrown and you can close the socket.

In python : How to perform a HTTP request, and connect with a websocket, through the same connection?

I am trying to connect to a website through a program, whose authentification protocol appears to be the following :
Connect to the websocket.
Receive crucial data for logging in.
Make a HTTP request with username/password/some of the datas THROUGH THIS CONNECTION.
This means that if i open another connection, the data linked to this connection, according to the server, will be different.
For exemple, receiving the data through the websocket, and doing a HTTP request with a browser will not work, as the browser is another connection.
By connection, i mean what is created at the start of the program, for exemple :
ws2= websocket.create_connection("ws://sim.smogon.com:8000/showdown/websocket") for the websocket module
h1 = httplib.HTTPConnection('www.cwi.nl') for the http module.
My problem is that i have to create 2 different connections to connect to websocket and make a http request. This means that the http request will not work with the websocket received data.
How to make these 2 connections the same ? Or, simplier said : how to solve the problem ?

How do I send and receive HTTP POST requests in Python?

I have these two Python scripts I'm using to attempt to work out how to send and receive POST requests in Python:
The Client:
import httplib
conn = httplib.HTTPConnection("localhost:8000")
conn.request("POST", "/testurl")
conn.send("clientdata")
response = conn.getresponse()
conn.close()
print(response.read())
The Server:
from BaseHTTPServer import BaseHTTPRequestHandler,HTTPServer
ADDR = "localhost"
PORT = 8000
class RequestHandler(BaseHTTPRequestHandler):
def do_POST(self):
print(self.path)
print(self.rfile.read())
self.send_response(200, "OK")
self.end_headers()
self.wfile.write("serverdata")
httpd = HTTPServer((ADDR, PORT), RequestHandler)
httpd.serve_forever()
The problem is that the server hangs on self.rfile.read() until conn.close() has been called on the client but if conn.close() is called on the client the client cannot receive a response from the server. This creates a situation where one can either get a response from the server or read the POST data but never both. I assume there is something I'm missing here that will fix this problem.
Additional information:
conn.getresponse() causes the client to hang until the response is received from the server. The response doesn't appear to be received until the function on the server has finished execution.

There are a couple of issues with your original example. The first is that if you use the request method, you should include the message body you want to send in that call, rather than calling send separately. The documentation notes send() can be used as an alternative to request:
As an alternative to using the request() method described above, you
can also send your request step by step, by using the four functions
below.
You just want conn.request("POST", "/testurl", "clientdata").
The second issue is the way you're trying to read what's sent to the server. self.rfile.read() attempts to read the entire input stream coming from the client, which means it will block until the stream is closed. The stream won't be closed until connection is closed. What you want to do is read exactly how many bytes were sent from the client, and that's it. How do you know how many bytes that is? The headers, of course:
length = int(self.headers['Content-length'])
print(self.rfile.read(length))
I do highly recommend the python-requests library if you're going to do more than very basic tests. I also recommend using a better HTTP framework/server than BaseHTTPServer for more than very basic tests (flask, bottle, tornado, etc.).

Long time answered but came up during a search so I bring another piece of answer. To prevent the server to keep the stream open (resulting in the response never being sent), you should use self.rfile.read1() instead of self.rfile.read()