We Keep Coding

"list index out of range" error in AzureML inference schema

I've deployed a model using AzureML's inference cluster. I recently found that some of the requests to the model's API endpoint resulted in a 404 HTTP error involving a missing swagger.json file.
So I followed this guide in order to auto-generate the swagger.json file. But now all the requests to the endpoint result in a "list index out of range" error and it's something to do with the input_schema decorator. I just can't seem to pinpoint what the problem is exactly.
Here is a minimal recreation of my scoring script:
from inference_schema.schema_decorators import input_schema, output_schema
from inference_schema.parameter_types.standard_py_parameter_type import StandardPythonParameterType
def inference(args):
# inference logic here
return model_output
def init():
global model
model = get_model()
input_sample = StandardPythonParameterType({
'input_1': 'some text',
'input_2': 'some other text',
'input_3': 'other text'
})
sample_global_parameters = StandardPythonParameterType(1.0)
output_sample = StandardPythonParameterType({
'Results': {
'text': 'some text',
'model_output': [
{
'entity_type': 'date',
'value': '05/04/2022'
}
]
}
})
#input_schema('Inputs', input_sample)
#input_schema('GlobalParameters', sample_global_parameters)
#output_schema(output_sample)
def run(Inputs, GlobalParameters):
try:
return inference(Inputs['input_1'], Inputs['input_2'], Inputs['input_3'])
except Exception as e:
error = str(e)
return error
I've checked out this and this question but it didn't seem to help.
I tried looking at the code on GitHub as well but I still can't triangulate on the exact problem.
I'm calling the API from Postman with the default headers (I'm not adding anything). The request body looks like this:
{
"Inputs": {
"input_1": "some text",
"input_2": "some other text",
"input_3": "different text"
},
"GlobalParameters": 1.0
}
This is the error message from the endpoint logs:
2022-04-05 06:33:22,536 | root | ERROR | Encountered Exception: Traceback (most recent call last):
File "/var/azureml-server/synchronous/routes.py", line 65, in run_scoring
response, time_taken_ms = invoke_user_with_timer(service_input, request_headers)
File "/var/azureml-server/synchronous/routes.py", line 110, in invoke_user_with_timer
result, time_taken_ms = capture_time_taken(user_main.run)(**params)
File "/var/azureml-server/synchronous/routes.py", line 92, in timer
result = func(*args, **kwargs)
File "/var/azureml-app/main.py", line 21, in run
return_obj = driver_module.run(**arguments)
File "/azureml-envs/azureml_e63c7c0baf9bf3d861ce5992975a467b/lib/python3.7/site-packages/inference_schema/schema_decorators.py", line 61, in decorator_input
return user_run(*args, **kwargs)
File "/azureml-envs/azureml_e63c7c0baf9bf3d861ce5992975a467b/lib/python3.7/site-packages/inference_schema/schema_decorators.py", line 55, in decorator_input
args[param_position] = _deserialize_input_argument(args[param_position], param_type, param_name)
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/azureml-envs/azureml_e63c7c0baf9bf3d861ce5992975a467b/lib/python3.7/site-packages/flask/app.py", line 1832, in full_dispatch_request
rv = self.dispatch_request()
File "/azureml-envs/azureml_e63c7c0baf9bf3d861ce5992975a467b/lib/python3.7/site-packages/flask/app.py", line 1818, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/var/azureml-server/synchronous/routes.py", line 44, in score_realtime
return run_scoring(service_input, request.headers, request.environ.get('REQUEST_ID', '00000000-0000-0000-0000-000000000000'))
File "/var/azureml-server/synchronous/routes.py", line 74, in run_scoring
raise RunFunctionException(str(exc))
run_function_exception.RunFunctionException

Try on setting the "GlobalParameters" to any kind of floating number other than 1.0 or try to remove it and execute. Sometimes global parameters will cause the issue.
https://learn.microsoft.com/en-us/answers/questions/746784/azure-ml-studio-error-while-testing-real-time-endp.html

Flask-socketio AttributeError: 'Request' object has no attribute 'sid'

I am getting this error: AttributeError: 'Request' object has no attribute 'sid' when using socketio in flask. Here is the output:
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/Users/user/tennisprogram/application.py", line 34, in create_game_post
join_room(str(gameid))
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/flask_socketio/__init__.py", line 865, in join_room
sid = sid or flask.request.sid
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/werkzeug/local.py", line 347, in __getattr__
return getattr(self._get_current_object(), name)
AttributeError: 'Request' object has no attribute 'sid'
Here is the code I'm using. If you need something other than the backend please let me know and I'd be happy to include it :).
Input:
from flask import Flask, redirect, request, render_template, session, url_for
from flask_socketio import SocketIO, emit, join_room, leave_room, close_room
app = Flask(__name__)
socket = SocketIO(app)
app.config["SECRET_KEY"] = 'secret-key'
games = [None, None, None, None, None, None, None, None, None, None]
class Game:
def __init__(self, player1, player2, id):
self.id = id
self.infodictionary = {"player1_name":player1, "player1_score":["0", "0", "0"], "player2_name":player2, "player2_score":["0", "0", "0"]}
#app.route("/")
def index():
return render_template("index.html")
#app.route("/create_game")
def create_game():
return render_template("create_game.html")
#app.route("/create_game/post", methods=["POST"])
def create_game_post():
if "host" in session:
return "Already hosting a game" #Handle later
for gameid, game in enumerate(games):
if game == None:
game = Game(request.form.get("player1"), request.form.get("player2"), gameid)
games[gameid] = game
session["host"] = gameid
return redirect(url_for('game', id=game.id))
return "No game slot available" #Handle later
#app.route("/game/<id>")
def game(id):
join_room(str(id))
if int(id) == session["host"]:
return render_template("score.html", host=True, game=games[int(id)])
else:
return render_template("score.html", host=False, game=games[int(id)])
#socket.on("host-update")
def update(data):
p1 = data["player1_score"]
p2 = data["player2_score"]
games[int(data["id"])].infodictionary["player1_score"] = p1
games[int(data["id"])].infodictionary["player2_score"] = p2
emit("update", {"player1_score":p1, "player2_score":p2}, room=data["id"])
#Handling join game
'''
#socket.on("joingame")
def join(data):
r = data["room"]
join_room(r)
'''
#app.route("/join_game")
def join_game():
return render_template("join_game.html")
#join_game.html will redirect user to join_game_post. The socket will activate in join_game.html
#app.route("/join_game/join", methods=["POST"])
def join_game_post():
id = request.form.get("id")
return redirect(url_for("game", id=id))
#app.route("/del")
def delete_host():
games[int(session["host"])] = None
del session["host"]

Referring to the answer here about AttributeError: 'Request' object has no attribute 'sid'
I believe that your error maybe caused by this line
*emit("update", {"player1_score":p1, "player2_score":p2}, room=data["id"])
Quoting the reason from the source above
emit() function has a default of emitting to the sender of an originating event. This default only makes sense when you call the function from an event handler. You are calling it from a route, which does not have the Socket.IO context (in particular, request.sid).
Do you know to which user(s) you want to emit an event from this
route? If you want to emit to all your connected clients, add
broadcast=True. If you know the sid of the user you want to address,
then add room=sid. You also need to specify the namespace, so add
namespace='/' or use the proper namespace name.
The error could also be as simple as, user has been disconnected before being routed/redirected.

Python Exception Not Caught?

I set up a try catch in my code, but it appears that my exception was not correct because it did not seem to catch it.
I am using an exception from a module, and perhaps I didn't import it correctly? Here is my code:
import logging
import fhirclient.models.bundle as b
from fhirclient.server import FHIRUnauthorizedException
logging.disable(logging.WARNING)
def get_all_resources(resource, struct, smart):
'''Perform a search on a resource type and get all resources entries from all retunred bundles.\n
This function takes all paginated bundles into consideration.'''
if smart.ready == False:
smart.reauthorize
search = resource.where(struct)
bundle = search.perform(smart.server)
resources = [entry.resource for entry in bundle.entry or []]
next_url = _get_next_url(bundle.link)
while next_url != None:
try:
json_dict = smart.server.request_json(next_url)
except FHIRUnauthorizedException:
smart.reauthorize
continue
bundle = b.Bundle(json_dict)
resources += [entry.resource for entry in bundle.entry or []]
next_url = _get_next_url(bundle.link)
return resources
Now when i ran the code I got the following error:
Traceback (most recent call last):
File "code.py", line 79, in <module>
main()
File "code.py", line 42, in main
reports = get_all_resources(dr.DiagnosticReport, search, smart)
File "somepath/fhir_tools/resource.py", line 23, in get_all_resources
json_dict = smart.server.request_json(next_url)
File "/usr/local/lib/python3.6/dist-packages/fhirclient/server.py", line 153, in request_json
res = self._get(path, headers, nosign)
File "/usr/local/lib/python3.6/dist-packages/fhirclient/server.py", line 181, in _get
self.raise_for_status(res)
File "/usr/local/lib/python3.6/dist-packages/fhirclient/server.py", line 256, in raise_for_status
raise FHIRUnauthorizedException(response)
server.FHIRUnauthorizedException: <Response [401]>
Shouldn't my exception catch this?

Return a json on browser - Flask

I have this function, it returns a result on console:
numbers_to_add = list(range(10000001))
try:
req = request.Request('http://127.0.0.1:5000/total'
, data=bytes(json.dumps(numbers_to_add), 'utf_8')
, headers={'Content-Type': 'application/json'}
, method='POST')
result = json.loads(request.urlopen(req).read(), encoding='utf_8')
print(json.dumps(result, indent=4))
except Exception as ex:
print(ex)
It returns a result on range 10000001
Now , I want to return this on browser request, in a Flask application, I've tried this:
def hardCoded():
numbers_to_add = list(range(10000001))
try:
req = request.Request('http://127.0.0.1:5000/total'
, data=bytes(json.dumps(numbers_to_add), 'utf_8')
, headers={'Content-Type': 'application/json'}
, method='POST')
result = json.loads(request.urlopen(req).read(), encoding='utf_8')
print(json.dumps(result, indent=4))
except Exception as ex:
print(ex)
class rangeNumbers(Resource):
def get(self, range):
return {'data': directSum.hardCoded(range)}
api.add_resource(rangeNumbers, '/range/<range>')
When I query this on my browser, it throws me this:
Traceback (most recent call last):
File "/home/user/.virtualenvs/test_sum/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/home/user/.virtualenvs/test_sum/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/user/.virtualenvs/test_sum/lib/python3.6/site-packages/flask_restful/__init__.py", line 480, in wrapper
resp = resource(*args, **kwargs)
File "/home/user/.virtualenvs/test_sum/lib/python3.6/site-packages/flask/views.py", line 84, in view
return self.dispatch_request(*args, **kwargs)
File "/home/user/.virtualenvs/test_sum/lib/python3.6/site-packages/flask_restful/__init__.py", line 595, in dispatch_request
resp = meth(*args, **kwargs)
File "app.py", line 16, in get
return {'data': directSum.hardCoded()}
TypeError: hardCoded() takes 0 positional arguments but 1 was given
Any ideas?

If range is meant to be the n number to return, in this case, 10000001, then you will want to do this instead:
In your directSum file:
def hardCoded(rng):
numbers_to_add = list(range(rng))
try:
# ... rest of code ...
In your main file:
class rangeNumbers(Resource):
def get(self, rng):
return {'data': directSum.hardCoded(rng)}
Where when you call rangeNumbers().get you do this:
rng_num = rangeNumbers()
rng_num.get(10000001)
Notice I changed your variable range to rng. It's in your best interest to not overshadow the builtin names even within a local scope. Otherwise calling range(range) is going to give you endless pain.

Testing boto3/botocore with pagination

I've been trying to add unit tests to my AWS scripts. I've been using botocore.stub to stub the API calls.
I needed to add pagination to various calls, and I can't seem to find a way to write the tests to include pagination.
Here's an example of the non-paginated test, I'm wondering how I can refactor this test and function to use pagination:
# -*- coding: utf-8 -*-
import unittest
import boto3
from botocore.stub import Stubber
from datetime import datetime
def describe_images(client, repository):
return client.describe_images(repositoryName=repository)
class TestCase(unittest.TestCase):
def setUp(self):
self.client = boto3.client('ecr')
def test_describe_images(self):
describe_images_response = {
'imageDetails': [
{
'registryId': 'string',
'repositoryName': 'string',
'imageDigest': 'string',
'imageTags': [
'string',
],
'imageSizeInBytes': 123,
'imagePushedAt': datetime(2015, 1, 1)
},
],
'nextToken': 'string'
}
stubber = Stubber(self.client)
expected_params = {'repositoryName': 'repo_name'}
stubber.add_response(
'describe_images',
describe_images_response,
expected_params
)
with stubber:
response = describe_images(self.client, 'repo_name')
self.assertEqual(describe_images_response, response)
if __name__ == '__main__':
unittest.main()
If I update the function to include pagination like this:
def describe_images(client, repository):
paginator = client.get_paginator('describe_images')
response_iterator = paginator.paginate(
repositoryName=repository
)
return response_iterator
we seem to be getting somewhere. The test fails as it should as equality has changed:
F
======================================================================
FAIL: test_describe_images (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "desc_imgs_paginated.py", line 47, in test_describe_images
self.assertEqual(describe_images_response, response)
AssertionError: {'imageDetails': [{'registryId': 'string'[178 chars]ing'} != <botocore.paginate.PageIterator object at 0x1058649b0>
----------------------------------------------------------------------
Ran 1 test in 0.075s
FAILED (failures=1)
When I try to iterate over the generator::
def describe_images(client, repository):
paginator = client.get_paginator('describe_images')
response_iterator = paginator.paginate(
repositoryName=repository
)
return [r for r in response_iterator]
I get the following error:
E
======================================================================
ERROR: test_describe_images (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "desc_imgs_paginated.py", line 45, in test_describe_images
response = describe_images(self.client, repo_name)
File "desc_imgs_paginated.py", line 14, in describe_images
return '.join([r for r in response_iterator])
File "desc_imgs_paginated.py", line 14, in <listcomp>
return '.join([r for r in response_iterator])
File "lib/python3.6/site-packages/botocore/paginate.py", line 255, in __iter__
response = self._make_request(current_kwargs)
File "lib/python3.6/site-packages/botocore/paginate.py", line 332, in _make_request
return self._method(**current_kwargs)
File "lib/python3.6/site-packages/botocore/client.py", line 312, in _api_call
return self._make_api_call(operation_name, kwargs)
File "lib/python3.6/site-packages/botocore/client.py", line 579, in _make_api_call
api_params, operation_model, context=request_context)
File "lib/python3.6/site-packages/botocore/client.py", line 631, in _convert_to_request_dict
params=api_params, model=operation_model, context=context)
File "lib/python3.6/site-packages/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "lib/python3.6/site-packages/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "lib/python3.6/site-packages/botocore/stub.py", line 337, in _assert_expected_params
self._assert_expected_call_order(model, params)
File "lib/python3.6/site-packages/botocore/stub.py", line 323, in _assert_expected_call_order
pformat(params)))
botocore.exceptions.StubResponseError: Error getting response stub for operation DescribeImages: Unexpected API Call: called with parameters:
{nextToken: string, repositoryName: repo_name}
----------------------------------------------------------------------
Ran 1 test in 0.051s
FAILED (errors=1)
Am I missing the correct approach to testing this? or is this a bug in boto3/botocore?

It's been a while since this question was asked but since there isn't an answer ..
In your set up you provide a response dictionary as below
describe_images_response = {
'imageDetails': [
{
'registryId': 'string',
'repositoryName': 'string',
'imageDigest': 'string',
'imageTags': [
'string',
],
'imageSizeInBytes': 123,
'imagePushedAt': datetime(2015, 1, 1)
},
],
'nextToken': 'string'
}
The key here is that the first response will include a nextToken value. This will result in a second request from the paginator. So you have to provide an additional response for the stub, ultimately you need to end with a response the does not include a nextToken
Now looking back at you set up, there is only a single add_response call to the stubber
stubber.add_response(
'describe_images',
describe_images_response,
expected_params
)
The net result in that when the paginator makes the second request, there is not response specified in the setup.
This results in the exception, the message on which hopefully now makes more sense
botocore.exceptions.StubResponseError: Error getting response stub for operation DescribeImages: Unexpected API Call: called with parameters:
{nextToken: string, repositoryName: repo_name}
Since the second response hasn't been set up, you get an exception with the request that was unexpected, in this request you can see the specification of the nextToken parameter.