I'm creating an API with NodeJS. Every request returns a JSON. Some requests call to a python script and I want to send the JSON generated by python to the user. The code would look like:
child = child_process.spawn(cmd, args);
child.stdout.on('data', (chunk) => {
res.write(chunk);
});
child.on('error', function(err) {
res.status(500).json({...});
});
child.on('close', (code) => {
res.end();
});
The problem with this code is that I can't check if the python output is a JSON. Maybe python writes warnings, error...
What can I do to prevent the user will get something different to JSON?.
EDIT
Right now my code is:
var output = [];
command.stdout.on('data', (chunk) => {
output.push(chunk)
});
command.on('close', (code) => {
var stdout = output.join('');
json_cmd = tryParseJSON(stdout)
if (json_cmd)
res.send(json_cmd)...
});
But, I don't want to load all the stdout in a variable. But if I don't do that, I can check if the stdout is a JSON. Can I force python to print just json?. Using always json.dumps and a global try catch would be enough?
Thanks.
Instead of using spawn, you probably want to use exec, which will wait for the Python process to exit and provide you with its output (which you can subsequently try to parse as JSON):
const exec = require('child_process').exec;
...
exec(cmdline, (err, stdout, stderr) => {
if (err) return res.status(500).json({...});
// Try to parse stdout as JSON:
try {
res.json(JSON.parse(stdout));
} catch(e) {
res.status(500).json({...});
}
});
So you need a streaming JSON verifier in NodeJS. The problem is that if you start streaming the json data over the network, and find an error in the JSON halfways, you cannot rollback the traffic to the http headers and change 200 ok to 500 ise. If you want to check the output first you have to accumulate it in the server before sending.
Your current code would do the same. If an error event comes the client will receive a 200 ok with a half stdout and a json object at the end. (Or the response object throws an error if you try to set the status code after it has been sent, I don't know how is this case handled.)
Related
I have a Django project which is using Angular as frontend. I have a button which on clicking is scanning the tables in the database. I have some print statements views.py file which is printing the scanned results constantly in the IDE console. I want that output in the webpage. I want that live printing of the console output in the frontend. Can any one know how i can achieve this?
You can achieve this by using server sent events. python can push these console logs to the frontend. Not a expert of python so giving a link below to how to send server side events from python to frontend
https://medium.com/code-zen/python-generator-and-html-server-sent-events-3cdf14140e56
In frontend you can listen to url exposed and as soon as server will push any message on this stream frontend can receive it and push it into component's array and can display over ui.
for frontend code, i am giving a minimal example below :-
import { Injectable, NgZone } from "#angular/core";
import { Observable } from "rxjs";
#Injectable({
providedIn: "root"
})
export class SseService {
constructor(private _zone: NgZone) {}
getServerSentEvent(url: string): Observable<any> {
return Observable.create(observer => {
const eventSource = this.getEventSource(url);
eventSource.onmessage = event => {
this._zone.run(() => {
observer.next(event);
});
};
eventSource.onerror = error => {
this._zone.run(() => {
observer.error(error);
});
};
});
}
private getEventSource(url: string): EventSource {
return new EventSource(url);
}
}
you can susbcribe to getServerSentEvent in above method and can continuously receive new messages, which is in your case your console logs.
You can try calling the following function with the information needed to be displayed.
addItem(val:any) {
let node = document.createElement("li")
let textnode = document.createTextNode(val)
node.appendChild(textnode)
document.getElementById("output").appendChild(node)
}
Make sure to have an element with the id="output".
I dumped Python array to a response with json.dumps and now I'm trying to retrieve the data as Javascript list.
#app.route('/get_scales')
#cross_origin()
def get_scales():
classes = inspect.getmembers(sys.modules['mingus.core.scales'], inspect.isclass)
scales = [class_[0] for class_ in classes if ('Error' not in class_[0] and class_[0] != '_Scale')]
return json.dumps(scales)
getScales() {
// create a new XMLHttpRequest
var xhr = new XMLHttpRequest();
// get a callback when the server responds
xhr.addEventListener("load", () => {
// update the state of the component with the result here
console.log(xhr.responseText);
});
// open the request with the verb and the url
xhr.open("GET", "http://127.0.0.1:5000/get_scales");
// send the request
xhr.send();
var formatted_response = JSON.stringify(xhr.responseText);
console.log(JSON.parse(xhr.responseText));
return xhr.responseText;
}
When I made the function in getScales log to console type of xhr.responseText it shows String, but then when trying to parse it with JSON.parse it throws an error. Trying to stringify it first, like above doesn't help either.
I don't know what error it gave, but I think it was actually because xhr.response, wasn't there yet when you tried to use it. This is because XMLrequests function asynchronously, meaning that while the XMLrequest is still waiting for a response, the rest of your code continues executing. Try this instead:
xhr.open('GET', url, false);
the "false" parameter basically says you want your XMLrequest to function synchronously. So the rest of your code will wait for it to finish.
Do keep in mind that your performance may suffer because of this in a lot of situations.
So if you have multiple XMLrequests at a time or Sequentially, you could consider using HTML5 Workers for this.
Or If you don't want your requests to function synchronously (if you can avoid having your XMLrequest function synchronously, you definitely should) you could also try something like this (something like this is defentintely the best option for performance, so if you can use it):
getScales() {
// create a new XMLHttpRequest
var xhr = new XMLHttpRequest();
// get a callback when the server responds
xhr.addEventListener("load", () => {
// update the state of the component with the result here
console.log(xhr.responseText);
var formatted_response = JSON.stringify(xhr.responseText);
console.log(JSON.parse(xhr.responseText));
return xhr.responseText;
});
// open the request with the verb and the url
xhr.open("GET", "http://127.0.0.1:5000/get_scales");
// send the request
xhr.send();
}
I'm writing a simple web app to get a handle on how node child processes work. Basically you enter your name into an angular frontend which passes the name to an express backend in a javascript object. The backend passes the name as an argument to a simple python script as a command line argument using the child_process module. Theres nothing wrong with the front end and the name gets successfully passed to the backend but when I call stdout on the python process it doesn't work. There aren't any error messages.
Heres the express code:
app.post('/api', (req, res, next) => {
console.log(req.body.name);
const spawn = require('child_process').spawn;
const process = spawn('python3', ['./hello.py', req.body.name]);
process.stdout.on('data', (data) => {
console.log(data);
res.status(200).json({greeting: data});
});
});
I put two console.log statements in the code to debug it. The first one prints the data but the second one inside the stdout function isn't called at all. The python script is in the same folder as the app.js express file so I'm pretty sure theres nothing wrong with the file path.
Here's the python script:
import sys
print('hello' + sys.argv[1])
sys.stdout.flush()
When I run it in the command line it works exactly as expected but I included it anyway just in case.
Process.stdout.on will keep on streaming until the end event. The code is wrong because you are actually sending response for every time there is some value in stdout. And you cant set the response header more than once. Try writing the code in below way. Thanks
let output;
Process.stdout.on("data", (data) => {
output += data;
});
Process.on("end", () => {
// send response here
});
close will trigger when your data completes
app.get("/list-account", async (req, res) => {
try {
let array = "";
let response = await child.spawn(
path.join(__dirname, "xcv-alpha-keychain.exe"),
["listaccounts"]
);
await response.stdout.on("data", (data) => {
const stdout = data.toString();
console.log("stdout", stdout);
array += stdout;
// return res.status(200).json({ array });
});
response.stderr.on("data", (data) => {
return res.status(500).send(data.toString());
});
response.on("error", (error) => {
return res.status(500).send({ error });
});
response.on("close", (code) => {
return res.status(200).json({ array, code });
});
} catch (error) {
return res.status(500).send(["a", "b", "c"]);
}
});
Instead of
console.log(data)
use
console.log(data.toString())
So I have node calling a python scrip but I want to get an object back from python.
I'm currently using Python-shell (https://www.npmjs.com/package/python-shell) but the problem is its listening so I can't actually send the data I get from it
shell.on('message', function(message){
ah = message;
console.log(message);
console.log("#");
});
console.log(ah);
var host = {
"hostName":ah
};
console.log(host);
return response.send(200, host);
the last section of the code will execute well before the python script returns anything via print()
(I also can't put the response.send in the listening function because it may send every time the python script prints)
is there another way of doing this?
If you want to return the results of a script to the client, just wrap this call: PythonShell.run in in a router endpoint:
app.get('/somepath', (req, res)=>{
PythonShell.run('my_script.py', options, function (err, results) {
if (err) throw err;
// results is an array consisting of messages collected during execution
res.send('results: %j', results);
});
});
Firstly, I'm very new to the world of web development, so sorry if this question is overly simple. I'm trying to use python to handle AJAX requests. From reading the documentation it seems as though Dojo/request should be able to do this form me, however I've not found any examples to help get this working.
Assuming I've got a Python file (myFuncs.py) with some functions that return JSON data that I want to get from the server. For this call I'm interested in a particular function inside this file:
def sayhello():
return simplejson.dumps({'message':'Hello from python world'})
What is not clear to me is how to call this function using Dojo/request. The documentation suggests something like this:
require(["dojo/dom", "dojo/request", "dojo/json", "dojo/domReady!"],
function(dom, request, JSON){
// Results will be displayed in resultDiv
var resultDiv = dom.byId("resultDiv");
// Request the JSON data from the server
request.get("../myFuncs.py", {
// Parse data from JSON to a JavaScript object
handleAs: "json"
}).then(function(data){
// Display the data sent from the server
resultDiv.innerHTML = data.message
},
function(error){
// Display the error returned
resultDiv.innerHTML = error;
});
}
);
Is this even close to what I'm trying to achieve? I don't understand how to specify which function to call inside myFuncs.py?
What you could also do is to create a small jsonrpc server and use dojo to do a ajax call to that server and get the json data....
for python side you can follow this
jsonrpclib
for dojo you could try something like this..
<script>
require(['dojox/rpc/Service','dojox/rpc/JsonRPC'],
function(Service,JsonRpc)
{
function refreshContent(){
var methodParams = {
envelope: "JSON-RPC-2.0",
transport: "POST",
target: "/jsonrpc",
contentType: "application/json-rpc",
services:{}
};
methodParams.services['myfunction'] = { parameters: [] };
service = new Service(methodParams);
function getjson(){
dojo.xhrGet({
url: "/jsonrpc",
load : function(){
var data_list = [];
service.myfunction().then(
function(data){
dojo.forEach(data, function(dat){
data_list.push(dat);
});
console.log(data_list)
},
function(error) {
console.log(error);
}
);
}
});
}
getjson();
}
refreshContent();
});
});
</script>
I've used this approach with django where i am not creating a different server for the rpc calls but using django's url link to forward the call to my function.. But you can always create a small rpc server to do the same..