We Keep Coding

Issue with simple python API in flask. Trying to create a post method to add json data to a list

I am trying to build a simple flask api to post json data to a list (eventually with be redshift but this is just a simple test program).
I have attached the api code first followed by the code to send data.
I am getting internal server error issues when running the second script.
The code seems very simple though and I cannot figure out what is wrong.
from flask_restful import Api, Resource
from flask import request
app = Flask(__name__)
api = Api(app)
audit_log = []
class audit(Resource):
#def get (self):
#return {"data":"HelloWorld"}
def put (self):
new_item = request.get_json()
audit_log.append(new_item)
return new_item
api.add_resource(audit,"/")
app.run()
import requests
BASE = "HTTP://127.0.0.1:5000/"
response = requests.put(BASE, params = {'auditid' : 'xyz', 'jobname' : 'abc'})
print (response.json())

It seems that you haven't imported the Flask properly
instead of this
from flask import request
use this
from flask import Flask, request
This should work fine...

how to integrate kafka with rest and to pyspark?

i have a rest service written in flask sitting at localhost:5000. It has endpoint function 'parser'.
It parses one website.
from flask import Flask
import requests
from flask import jsonify
from flask import make_response
from flask import request
app = Flask(__name__)
#app.route("/", methods = ['GET'])
def main():
return jsonify('service is waiting, standby..')
#app.route("/parser/<string:website>", methods = ['GET'])
def parse(website):
if website != 'news.com':
return make_response(jsonify({'error': 'only news.com is parsable' }), 404)
else:
result = main_parse()#need to send variable result via kafka to streaming for analysis
return make_response(jsonify("parsing this website...", 200))
if __name__ == "__main__":
app.run(debug=True)
The function main_parse has other stuff in it, but it returns some parsed data -a list of lists.
I never used kafka before, how do I throw this parsed data result via kafka or kafka topic(task?) to pyspark for basic analysis? I m naive.
It should be a pipeline without clicking buttons- once rest api receives correct website name - it does the rest.

You don't need Spark for Kafka in this code, and I would not put Spark in any web server as it is meant for distributed processing (meaning your Spark code wouldn't even be running within the context of Flask, if you did have a Spark cluster).
You can use any Kafka Python library, which can be easily found on Github for examples and usage documentation.
do we actually send entire data to kafka
Sure. Keep in mind; Kafka has a default record limit of 1MB

How to invoke/run a Python script using a web URL?

I have a Python script that pulls data from a 3 rd party API. Currently this Pyhton script is automated on server side.
There are few instances where I have to toggle the script manually for new data updates. For the manual toggle I have to login to the server each time and run it from command line. Is there a way where I can create web url or something similar and just run that URL to make that script run from the browser address bar.

One approach you could take is to use Flask, which is a minimal web framework. Here's an example of how you could use it:
from flask import Flask
from your_script import your_func
app = Flask(__name__)
#app.route('/run')
def run_command():
your_func()
return 'Executed your function!'
if __name__ == '__main__':
app.run(debug=False, port=8080)
If you run this code you'd get a web server running on port 8080 that executes your function when you access the url. Here's a tutorial in the Flask documentation to get you started.

I think the easiest way to do this is by using Flask.
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world():
# your code here
return 'Hello, World!'

Calling Python file in an Ajax Call

So I have established a pretty decent understanding of the simple architecture of an angularjs app, calling $http and posting to a php page, and receiving data back.
What I'm wondering, is how to do the same type of function with python. Is it possible to have python act the same, with self contained script files that accept post data and echo json back?
$username = $_POST['username'];
type variable assignment at the beginning of the script, and:
echo json_encode(response);
type response.
I'm wanting to use Python for some Internal Tools for my company, as it offers better libraries for remotely running powershell scripts (as the tools are all linux hosted) and overall just has libraries that fit my needs. I'm just having a difficult time finding a concise answer to how this could be set up.
---EDIT------
So I set up a quick example using the information below.
the angular:
var app = angular.module("api");
app.controller("MainController", ["$scope","$http",MainController]);
function MainController($scope,$http){
$http.post('/api',{test: "hello"})
.then(function(response){
console.log(response.data);
})
}
The flask:
from flask import Flask, request
import json
app = Flask(__name__)
#app.route('/api', methods=['POST', 'GET'])
def api():
if request.method == 'POST':
request.data
return 'You made it' # Just so I originally could see that the flask page
if __name__ == "__main__":
app.run()
I'm getting a 404 for that URL. If I change the angular to look at 'localhost:5000/api' (where my flask app is running),it gives me the error of "Unsupported URL Type".
I am seeing when I do the first case, it tries to look at http://localhost/api , which is correct! except for the port. Which is why I tried to specify the port.
Any suggestions for a next step?

Use flask.
You could host your app on a flask "server" and return the content you'd like too with a python processing.
http://flask.pocoo.org/
Use the documentation to setup a route where you'll POST your data using jquery or whatever, then on the route you can do your python stuff and return a JSON to your angular app if you need to.
from flask import request
#app.route('/test', methods=['POST', 'GET'])
def test():
if request.method == 'POST':
print request.data['your_field']
return your_json_data

Getting plain text instead of Python script execution

I'm trying to execute this little request with jquery on a page load:
$.ajax({
type: "GET",
url: "/static/query.py",
cache: false,
success: function (data) {
$(body).text(data);
}
});
On a server running nginx, python, in a virtualenv, working with Flask framework and the return is the query.py sourcecode, not the data retrieved from the DB.
query.py is marked as executable and the script has the shebang:
#!/home/gabriel/project/bin
which points to the bin in my virtualenv. I think got the basis covered but yet, the problem persists.
Tips?
More info:
Flask script:
from flask import Flask, render_template
application = Flask(__name__)
#application.route('/')
def init():
return render_template('index.html')
if __name__ == "__main__":
application.run(host='0.0.0.0', debug=True)
uwsgi.py to load the Flask script:
from myapp import application
if __name__ == "__main__":
application.run()
The query.py:
from db import * //Database model, SQLAlchemy.
def something():
data = Kingdom.query.all()
return data
something()

You should be running the actual code that's in query.py inside flask. Do something like:
#application.route("/query"):
def get_data():
.. whatever code you need; currently in query.py, you would probably use sqlalchemy/ flask-sqlalchemy
data = your data (dictionary? list?)
return jsonify(data=data) # flask will turn your data into a proper JSON string to send back as a response to the ajax call.
Don't forget to import jsonify from flask, consult docs here.
Also rename "/static/query.py" to "/query" in example above, or anything else you see fit. You can also send flask parameters via the AJAX call to process in your query; e.g., filtering parameters. Focus the question for further guidance.