We Keep Coding

Python - list comprehension as a decorator (including self)

I have two functions:
job_status is getting a response from boto3 api.
jobs_detailsis a list comprehension that performs job_status on each element of the input list.
I want to change jobs_details into a decorator of jobs_status but below solutions throws inner() takes 1 positional argument but 2 were given error.
Appreciate any comment/alternative approach to my issue. Thanks!
import boto3
class GlueClient:
def __init__(self):
self.glue_client = boto3.client('glue')
#self.envs = envs
def jobs_list(self):
response = self.glue_client.list_jobs()
result = response["JobNames"]
while "NextToken" in response:
response = self.glue_client.list_jobs(NextToken=response["NextToken"])
result.extend(response["JobNames"])
return [e for e in result if "jobs_xyz" in e]
#WHAT IS CURRENTLY
def job_status(self, job_name):
paginator = self.glue_client.get_paginator('get_job_runs')
response = paginator.paginate(JobName=job_name)
return response
def jobs_details(self, jobs):
return [self.job_status(e) for e in jobs]
#WHAT IS EXPECTED
def pass_by_list_comprehension(func):
def inner(list_of_val):
return [func(value) for value in list_of_val ]
return inner
#pass_by_list_comprehension
def job_status(self, job_name):
paginator = self.glue_client.get_paginator('get_job_runs')
response = paginator.paginate(JobName=job_name)
return response
glue_client = GlueClient()
jobs = glue_client.jobs_list()
jobs_status = glue_client.job_status(jobs)
print(jobs)

You want something like:
import boto3
from typing import Callable
def handle_iterable_input(func):
def inner(self, list_of_val):
return [func(self, value) for value in list_of_val]
return inner
class GlueClient:
def __init__(self):
self.glue_client = boto3.client('glue')
#self.envs = envs
def jobs_list(self):
response = self.glue_client.list_jobs()
result = response["JobNames"]
while "NextToken" in response:
response = self.glue_client.list_jobs(NextToken=response["NextToken"])
result.extend(response["JobNames"])
return [e for e in result if "jobs_xyz" in e]
#handle_iterable_input
def job_status(self, job_name):
paginator = self.glue_client.get_paginator('get_job_runs')
response = paginator.paginate(JobName=job_name)
return response
glue_client = GlueClient()
jobs = glue_client.jobs_list()
jobs_status = glue_client.job_status(jobs)
print(jobs)
This is the most basic way to make your decorator handle methods properly, by explicitly handling the passing of self. Note, it assumes the function being decorated will only take a single argument.

If all you want to do is make job_status iterate through a list of job names instead of operating on just one, something like this should work:
def jobs_status(self, job_names):
paginator = self.glue_client.get_paginator('get_job_runs')
return [paginator.paginate(JobName=job_name) for job_name in job_names]
Using a decorator to change what parameters a method expects seems like a bad idea.
Also, naming your class GlueClient would imply that it is a glue client. The fact that it has an attribute named glue_client makes me suspect you could probably choose a clearer name for one or both of them. (However, I'm not familiar with the package you're using.)

How to use Python decorators to optimize writing different methods which do similar job?

I am a beginner in Python and I need some help on decorators. I am writing a few methods which call other methods that are generated using swagger. Basically all these swagger methods have GET APIs. All I need to do in my code is to call those swagger methods and return the value. I am looking for ways to optimize this instead of writing the same kind of method for each API. I came across decorators which can be used in this case. But my implementation is not giving the desired result
def get_component_info(self, func):
def inner():
data = None
try:
ret = func()
if ret.status == 200:
log.info('ret ' + str(ret))
else:
logging.error('Error: ' + str(ret.text))
except Exception as e:
logging.error(" failed with error " + str(e.reason) +
" and error code " + str(e.status))
finally:
return data
return inner()
def get_firewall_info(self):
return self._monitor.list_firewall_monitors_with_http_info() <-- swagger method
def get_firewall_info_caller(self):
get_firewall_info = self.get_component_info(self.get_firewall_info())
But the above implementation always return None because it never executes self._monitor.list_firewall_monitors_with_http_info(), but the test isn't failing
If you help me fix this, then I can use the same for getting server info, auth info, network info, etc. If decorators can't be used, what else I can use to optimize this..?
Thanks

Decorators are usually functions which take a second function as an argument and then define and return a third function which calls the second function while changing its input and/or output. It looks like you have a better handle on this than I did as a beginner.
def decorator(func):
def inner(*args, **kwargs):
# Do stuff here...
value = func(*args, **kwargs)
# ...or here.
return value
return inner
The only change I would recommend to your decorator is not to call inner and return the result, but to return the function itself. When you make this change you'll have to call the function you are returning now after it is returned.
def get_component_info(self, func):
def inner():
# ...
return inner
def get_firewall_info_caller(self):
# You will now want to call the `inner` function after you get
# it from `get_component_info`.
get_firewall_info = self.get_component_info(...)()
It looks like the core of your bug is that you aren't providing a function to get_component_info; you're providing the result of calling that function. I think that changing the code to not call get_firewall_info should fix your code.
def get_firewall_info_caller(self):
# You don't want to call the function you're providing to a
# decorator, since it's expecting the function not the result.
get_firewall_info = self.get_component_info(self.get_firewall_info)()

I resolved it as shown below. Not sure if it is the right approach. Please correct me
def get_component_info(self, func):
def inner():
data = None
try:
ret = func()
if ret.status == 200:
log.info('ret ' + str(ret))
else:
logging.error('Error: ' + str(ret.text))
except Exception as e:
logging.error(" failed with error " + str(e.reason) +
" and error code " + str(e.status))
finally:
return data
return inner
def get_firewall_info(self):
data = self.get_component_info(self._monitor.list_firewall_monitors_with_http_info)()
return data

Python: Use returned values from a function to another function

With the simple function I am able to get data stored in response variable. With a print statement I can get the data showing correctly.
ec2_client = boto3.client("ec2")
def instance_list(instance_name):
response = ec2_client.describe_instances(
Filters=[
{
'Name': 'tag:Name',
'Values': [ instance_name ]
}
]
)['Reservations']
return response
#print(response)
if __name__ == '__main__':
my_instance_list = instance_list("example-*")
However while trying to import the value of response from the above function to another function, getting error as NameError: name 'response' is not defined
def my_list():
list = instance_list(response)
print(list)
Looks like something unidentified.

you need to pass the varable to the next function, for example
my_list(instance_list())

the basic idea is to use return value of one function to another function
Your function should be this:
def my_list():
# List is equal to the response value of instance list.
list = instance_list("example-*")
print(list)
Easy example
def add_five_to_number(number):
number += 5
return number
def print_number(number):
higher_number = add_five_to_number(number)
print(higher_number)
test_number = 3
print_number(test_number)
# Returns 8

method reverse django error

I'm calling a method using reverse but I have problem argument that I am not able to understand.
My error :
Reverse for 'shopping.views.payment_confirmation' with arguments '(35,)' and keyword arguments '{}' not found.
My url:
url(r'^payment_confirmation/(?P<id>\d+\d{2\})/$', 'payment_confirmation', name='payment_confirmation'),
My view:
def show(request):
...
...
payment.submit(settings.SITE_URL + reverse("shopping.views.payment_confirmation", args=[payment.id]))
My model Payment:
class Payment(models.Model):
...
...
def submit(self, redirect_url):
'''
Sends self as a Payment through PagSeguro API.
Payment instance MUST BE SAVED before calling this method.
'''
if not self.id:
#Temporary to identify which problem caused the crash.
raise ValidationError
#creating a reference if its None
if self.reference is None:
self.reference = configs.PAGSEGURO_REFERENCE_PREFIX + str(self.id)
document = Document()
document.appendChild(self.to_element(document, redirect_url))
response = document2dict(api.submit_payment(document))
try:
self.code = response['checkout']['code']
self.answered_on = datetime.datetime.now()
self.save()
except:
error_str = ""
if type(response["errors"]["error"]) != list:
response["errors"]["error"] = [response["errors"]["error"]]
for error in response["errors"]["error"]:
error_payment = ErrorPayment()
error_payment.code = int(error['code'])
error_payment.payment = self
error_payment.save()
error_str += "[%s: %s]" % (error_payment.code,
error_payment.get_code_display())
raise Exception(error_str)
the error is here payment.submit (settings.SITE_URL + reverse ("shopping.views.payment_confirmation", args = [payment.id]))
I', using this api https://bitbucket.org/leonardosantos/django-pagseguro2/

This line: reverse("shopping.views.payment_confirmation", args=[payment.id]) tells Django to find a url that matches a method called payment_confirmation in the views.py file in the shopping app that will accept a payment ID parameter.
In the error that you shared, the payment.id was 35. The error is saying that either there is no method called payment_confirmation in your shopping.views or the method does not accept a single int as a parameter.
You didn't share a payment_confirmation method in your views file so it seems that that is the issue. You need to add:
def payment_confirmation(payment_id):
#do stuff
to your views.

Global variables in Ironpython

I'm having terrible trouble trying to understand ironpython scoping rules.
With the following script:
global data
// function for call xml-rpc
def CallListDatabases(self):
global synC, synCtx, result, data
self.synCtx = synC.Current
service = XmlRpcService("http://localhost:8000/rpc")
req = XmlRpcRequest(service, 'vocab_list')
req.XmlRpcCallCompleteHandler += self.req_XmlRpcCallCompleteHandler
result = req.Execute(self)
//if call xml-rpc complete then use working rpc
def req_XmlRpcCallCompleteHandler (self, response, userState):
global synCtx, synC, data
word = []
f = response.TryCast(clr.GetClrType(Fault))
if f != None:
self.synCtx.Post(self.SetCallResult, f)
if f.FaultCode == -1:
pass
else:
self.synCtx.Post(self.SetCallResult, response)
// show result with rpc complete
def SetCallResult(self, userState):
global data, result
if userState.GetType() == clr.GetClrType(Fault):
f = userState
if f != None:
print str(f.FaultString)
return
response = userState
result = response.TryCast(clr.GetClrType(Array[str]))
data = result //I want to use value it
print "value: "+data //show value
Problem
print "value: "+data
value: [] <<<======== Not value

First of all, you don't seem to ever be calling any of the functions you have defined. If you are calling the functions, it appears that the return value of response.TryCast(clr.GetClrType(Array[str])) is an empty list. Have you tried printing the value of result within SetCallResult()? I'd bet that it's [].