I'm trying to validate the headers of a flask request and its failing. I'm trying to use the below code to simulate the same and can see that its failing to validate the headers properly even if I miss some of the mandatory headers.
The below code is expected to fail but its passing.
import validictory
from werkzeug.datastructures import EnvironHeaders
obj = EnvironHeaders(environ={})
validictory.validate(obj,{'type': 'object', 'properties': {'test':{'required': True, 'type': 'any'}}})
If I convert the EnvironHeaders as dict then validation is happening properly.
import validictory
from werkzeug.datastructures import EnvironHeaders
obj = EnvironHeaders(environ={})
validictory.validate(dict(obj),{'type': 'object', 'properties': {'test':{'required': True, 'type': 'any'}}})
This properly raises the below error during validation. Any idea on the reason for improper validation happened in the first case?
validictory.validator.RequiredFieldValidationError: Required field 'test' is missing
I was able to find out the reason for this issue by going through the source code of validictory.
It was passing the type validation since EnvironHeaders has both the attributes 'keys' and 'values'.
def validate_type_object(self, val):
return isinstance(val, Mapping) or (hasattr(val, 'keys') and hasattr(val, 'items'))
Property validation is happening only for dict types and the validation is passing since the code doesn't raise any error if the input type is not a dictionary.
def validate_properties(self, x, fieldname, schema, path, properties=None):
''' Validates properties of a JSON object by processing the object's schema recursively '''
value = x.get(fieldname)
if value is not None:
if isinstance(value, dict):
if isinstance(properties, dict):
if self.disallow_unknown_properties or self.remove_unknown_properties:
self._validate_unknown_properties(properties, value, fieldname,
schema.get('patternProperties'))
for property in properties:
self.__validate(property, value, properties.get(property),
path + '.' + property)
else:
raise SchemaError("Properties definition of field '{0}' is not an object"
.format(fieldname))
Note: Validictory has stopped support and hence not going to raise any issue in git repo. Will try using jsonschema package as suggested.
Related
I am using a python library (ccxt) in which one base exchange class is inherited by exchange-specific classes, to provide a unified interface to several exchanges (coinbase, binance etc.).
The function definition for a sub-class might look something like this (not necessarily exactly): def fetch_ledger(self, symbols = None, since = None, params = {}):
The thing is, for e.g. the coinbase class, this method calls another method called prepareAccountRequestWithCurrencyCode(), which raises the exception:
raise ArgumentsRequired(self.id + ' prepareAccountRequestWithCurrencyCode() method requires an account_id(or accountId) parameter OR a currency code argument') if "accountId" or "code" is not provided in the params dict. These arguments are not in the function signature, as they are to be provided in the params dict (e.g. params = {"accountId" : "0x123"}).
I want to know that these arguments are required before I use the method, as I want to implement some automation and GUI-elements which can work across several exchanges (sub-classes). Some of these sub-classes have their own fetch_ledger methods which might not require e.g. the "accountId" argument to be provided in the params dict.
What is a god way to automatically obtain required aguments that are not in the function signature, for all exchanges?
I am providing the relevant ccxt code below since it's open-source:
def fetch_ledger(self, code=None, since=None, limit=None, params={}):
self.load_markets()
currency = None
if code is not None:
currency = self.currency(code)
request = self.prepare_account_request_with_currency_code(code, limit, params) # REQUIRES "accountId" in params
query = self.omit(params, ['account_id', 'accountId'])
response = self.v2PrivateGetAccountsAccountIdTransactions(self.extend(request, query))
return self.parse_ledger(response['data'], currency, since, limit)
def prepare_account_request_with_currency_code(self, code=None, limit=None, params={}):
accountId = self.safe_string_2(params, 'account_id', 'accountId')
if accountId is None:
if code is None:
raise ArgumentsRequired(self.id + ' prepareAccountRequestWithCurrencyCode() method requires an account_id(or accountId) parameter OR a currency code argument')
accountId = self.find_account_id(code)
if accountId is None:
raise ExchangeError(self.id + ' prepareAccountRequestWithCurrencyCode() could not find account id for ' + code)
request = {
'account_id': accountId,
}
if limit is not None:
request['limit'] = limit
return request
I've already thought of a few ways of doing it, such as running the function, catching the exception and dissecting the string to prompt the user for any missing arguments during run-time. I've also thought about making a source code parser, and even making changes to the library code, but I'm currently not sure what is best. I'd prefer to not have to look at the documentation of each unified method for all 100 exchanges and having to do it manually.
I'm wondering if anyone knows of an elegant or best-practice way of obtaining such optionally provided, yet required arguments for such methods (or just for the library I am currently using).
The problem in code is that if a field is missed then it raises error and if I except the error then it will not show anything
import pyshark
from tabulate import tabulate
capture = pyshark.FileCapture('/home/sipl/Downloads/DHCP.cap', display_filter='udp.port eq 67')
# capture2 = pyshark.LiveCapture(interface='wlo2', display_filter='arp')
d = dict()
for packet in capture:
try:
d['mac'] = packet.dhcp.hw_mac_addr
d['hname'] = packet.dhcp.option_hostname
d['vend'] = packet.dhcp.option_vendor_class_id
except AttributeError:
pass
try:
d['srvrid'] = packet.dhcp.option_dhcp_server_id
d['smask'] = packet.dhcp.option_subnet_mask
d['DNS'] = packet.dhcp.option_domain_name_server
d['Domain'] = packet.dhcp.option_domain_name
except AttributeError:
pass
try:
d['ip'] = packet.dhcp.option_requested_ip_address
except AttributeError:
pass
try:
table = {'Mac': [d['mac']], 'IP': [d['ip']], 'host': [d['hname']],'vendor': [d['vend']], 'Server id': [d['srvrid']],
'Sub mask': [d['smask']], 'DNS': [d['dns']], 'Domain': [d['Domain']]}
print(tabulate(table, headers='keys'))
except KeyError:
continue
I want that if a field is missed then it store the incoming fields i got in a packet and show in the table, for empty field it doesn't show anything and leave the field empty in table.
Basically I want that it stores the incoming field and prints in table and didn't raise error for the missed field.
I'm trying it now on fileCapture to check working but i need to do this on liveCapture
If Im understand you correctly, you don't want to get the Attribute Error but put an empty value when field is missing.
You can do it by check for value using getattr function.
so I have no Idea exactly what dhcp and if its missing or always existing and only what comes after can be missing.
But lets says that dhcp always exists and the actual fields you are pointing at can be missed:
Create a function called: get_value_or_none(obj, key, default='') -> str
Now lets implement it using the getattr.
def get_value(obj, key, default='') -> str:
return getattr(obj, key, default=default)
Now replace all the coresponding assignments you made in your code by wrapping the calls with the function calls:
i.e: get_value(packet.dhcp, 'option_domain_name')
That's it, it should work.
PS. If the dhcp is not always present, you will have to do the same with it too.
I did it by using the method:
dictionary.get()
I'm using the Python library connexion in conjunction with my Swagger specification which features the x-nullable attribute in some of the definitions.
This x-nullable attribute is essentially a polyfill for the lack of support for nullable attributes in OAS 2.0. However, most frameworks support it with varying degrees.
connexion does appear to support this attribute in parameters but not responses.
So if you attempt to return a response with null as a value for any of items return in a response with x-nullable and validate_responses set to True it will fail validation and produce Response body does not conform to specification in the response.
How best can I polyfill support for x-nullable in a connexion based Python application such as the ones generate from the swagger-codegen tool?
Managed to specify a custom validator that patches the JSON schema's in the definition of the operation passed to it.
from connexion.decorators.response import ResponseValidator
class CustomResponseValidator(ResponseValidator):
def __init__(self, operation, mimetype):
operation.definitions = { name: self.patch_nullable_in_definition(definition) for name, definition in operation.definitions.items() }
ResponseValidator.__init__(self, operation, mimetype)
def patch_nullable_in_definition(self, definition):
definition['properties'] = { key: property_ if '$ref' in property_ else self.patch_nullable_in_property(property_) for key, property_ in definition['properties'].items() }
return definition
def patch_nullable_in_property(self, property_):
if isinstance(property_, dict) and \
not isinstance(property_, list) and \
'x-nullable' in property_ and \
property_['x-nullable'] and \
'type' in property_:
if not isinstance(property_['type'], list):
property_['type'] = [property_['type']]
if not 'null' in property_['type']:
property_['type'].append('null')
return property_
I am new to Python, Django and the Django Rest Framework - although I am loving the learning curve!
I would like to know what is the standard (most common) way of raising an exception when an API's parameter is not provided ?
Obviously if conditions in the view's body is not the way to go. Are there an decorators that I can pass parameter names to ?
# urls.py
urlpatterns = [
url(r'test', test),
url(r'errand/make', errand.make),
url(r'errand/preview', errand.preview)
]
# views/errand.py
#api_view(['GET'])
#renderer_classes((JSONRenderer, ))
def preview(request):
e = Errand.objects.get(pk=request.GET['errand_id'])
return Response({'data': e.get_preview_data()})
In order for this line
e = Errand.objects.get(pk=request.GET['errand_id'])
To run fine, errand_id needs to be available. How can I check for certain request keys ?
You'll get a TypeError on any function call where a parameter does not have a default value.
As long as you aren't passing in defaults, you'll get an exception, even if the parameter is never used inside the function.
In your example, since you use dict.get, by default if the key is not found it will return None
If you wanted an exception there, you could try directly accessing the key, which would result in a KeyError when it isn't found. Like so:
...
e = Errand.objects.get(pk=request['errand_id'])
...
Otherwise another solution would be to create a schema that represents the desired structure of the request, and validate the request either in the function or using a decorator.
Something like this would be a start:
def errand_id_required(func):
def func_wrapper(request):
if not request.get('errand_id', False):
raise KeyError('errand_id not present in request')
return func(request)
return func_wrapper
#errand_id_required
def preview(request):
e = Errand.objects.get(pk=request.GET['errand_id'])
return Response({'data': e.get_preview_data()})
I would generally only do this if I had a sophisticated way of validating dictionary schemas, otherwise it's very much overkill to write a decorator to check a single key.
You could try the schema library on pypi, and define something like this:
import schema
errand_schema = {
schema.Optional('some_key'): str,
'id': int,
'errand_id': int,
}
Errand = schema.Schema(errand_schema, ignore_extra_keys=True)
and you could use Errand.validate(request) instead of the if.. raise KeyError that I put in the decorator.
But I'll leave that up to you to decide upon...
I'm trying to upsert a record via the salesforce Beatbox python client the upsert operation seems to work fine but I can't quite work out how to specify an externalid as a foreign key:
Attempting to upsert with:
consolidatedToInsert = []
for id,ce in ConsolidatedEbills.items():
consolidatedToInsert.append(
{
'type':'consolidated_ebill__c',
'Account__r':{'type':'Account','ETL_Natural_Key__c':ce['CLASS_REFERENCE']},
'ETL_Natural_Key__c':ce['ISSUE_UNIQUE_ID']
}
)
print consolidatedToInsert[0]
pc.login('USERNAME', 'TOTALLYREALPASSWORD')
ret = pc.upsert('ETL_Natural_Key__c',consolidatedToInsert[0])
print ret
gives the error:
'The external foreign key reference does not reference a valid entity: Account__r'
[{'isCreated': False, 'errors': [{'fields': [], 'message': 'The external foreign key reference does not reference a valid entity: Account__r', 'statusCode': 'INVALID_FIEL
D'}], 'id': '', 'success': False, 'created': False}]
The soap examples and the specificity of the error text seem to indicate that it's possible but I can find little in the documentation about inserting with external ids.
On a closer look I'm not sure if this is possible at all, a totally mangled key to Account__r seems to pass silently as if it's not even being targeted for XML translation, I'd love to be wrong though.
A quick change to pythonclient.py 422:0:
for k,v in field_dict.items():
if v is None:
fieldsToNull.append(k)
field_dict[k] = []
if k.endswith('__r') and isinstance(v,dict):
pass
elif hasattr(v,'__iter__'):
if len(v) == 0:
fieldsToNull.append(k)
else:
field_dict[k] = ";".join(v)
and another to __beatbox.py 375:0
for fn in sObjects.keys():
if (fn != 'type'):
if (isinstance(sObjects[fn],dict)):
self.writeSObjects(s, sObjects[fn], fn)
else:
s.writeStringElement(_sobjectNs, fn, sObjects[fn])
and it works like some dark magic.
Currently Beatbox doesn't support serializing nested dictionaries like this, which is needed for the externalId resolution you're trying to do. (If you look at the generated request, you can see that the nested dictionary is just serialized as a string)).