Do I need to use mocks? - python

I have the function which handles a specific bot event from Slack. Generally speaking, user clicks a button then my server receives and handles payload of this button.
The question is how should I test it? Do I need to mock make_admin and build_admins_message and check that they were called or I need to test real implementations? For example, I can retrieve the user from the database and check that it is actually an admin and also check that build_admins_message returns a dictionary that I expect to receive.
#slack_interactions.on('admin_add')
def handle_admin_add(payload):
team_id = payload['team']['id']
user_id = payload['user']['id']
action_value = payload['actions'][0]['selected_options'][0]['value']
user = SlackUser.objects.find_by_ids(team_id, action_value)
if user and not user.is_bot:
user.make_admin()
return build_admins_message(team_id, user_id)
Currently my tests look like this:
class TestAdminAddHandler(TestCase):
def setUp(self):
team = SlackTeam.objects.create(team_id='TEAMID')
SlackUser.objects.create(team=team, user_id='USERID')
SlackUser.objects.create(team=team, user_id='BOTID', is_bot=True)
SlackUser.objects.create(
team=team, user_id='ADMINID', is_bot_admin=True)
def tearDown(self):
SlackUser.objects.all().delete()
SlackTeam.objects.all().delete()
def test_wrong_callback(self):
payload = {'callback_id': 'wrong_callback'}
message = handle_admin_add(payload)
self.assertIsNone(message)
def test_has_no_user(self):
payload = {
'callback_id': 'admin_add',
'team': {'id': 'TEAMID'},
'user': {'id': 'ADMINID'},
'actions': [{
'selected_options': [{'value': 'BADID'}]
}]
}
message = handle_admin_add(payload)
user = SlackUser.objects.get(user_id='USERID')
self.assertFalse(user.is_bot_admin)
for att in message['attachments']:
self.assertNotIn('BADID', att.get('title', ''))
def test_user_is_bot(self):
payload = {
'callback_id': 'admin_add',
'team': {'id': 'TEAMID'},
'user': {'id': 'ADMINID'},
'actions': [{
'selected_options': [{'value': 'BOTID'}]
}]
}
message = handle_admin_add(payload)
user = SlackUser.objects.get(user_id='BOTID')
self.assertFalse(user.is_bot_admin)
for att in message['attachments']:
self.assertNotIn('BOTID', att.get('title', ''))
def test_add_admin(self):
payload = {
'callback_id': 'admin_add',
'team': {'id': 'TEAMID'},
'user': {'id': 'ADMINID'},
'actions': [{
'selected_options': [{'value': 'USERID'}]
}]
}
message = handle_admin_add(payload)
user = SlackUser.objects.filter(user_id='USERID').first()
self.assertTrue(user.is_bot_admin)
user_in_list = False
for att in message['attachments']:
if 'USERID' in att.get('title', ''):
user_in_list = True
self.assertTrue(user_in_list)

The problem here is two-fold. First, you have to verify that your code would work correctly with a functioning Slack server - as you have deduced, mocks would be a good way to do this, because unit tests should be completely standalone. You can also write mocks to emulate the behaviour of an incorrectly-functioning server.
However, this leaves open the possibility that your mocks don't correctly emulate the behaviour of the Slack server, and therefore your code doesn't work in real life even though it passes unit testing. In order to do that you will need integration tests to verify (as your current test class appears to) that the code works correctly against a Slack server.
When creating mock objects you can even go as far as capturing network traffic from successful transactions and then using that content to generate mock responses by patching lower-level components to generate the appropriate network-level responses to avoid interaction with the server. Since you don't often have the ability to modify production servers for testing convenience, mocking is often the simplest way to verify correct handling of unusual server responses. It all depends how far you want to go.
Unit tests should validate the function of a single component, and should not rely on any external services. Integration tests verify that the code functions correctly in conjunction with other components, and will normally be performed only when the individual components' integrity has been verified.
Testing is a very large subject, so I hope this answers your question.

Related

How to set the 'iss' claim of JWT using Flask-JWT-Extended's create_access_token()

Is there a way to set the iss claim of the JWT that is generated by create_access_token of Flask-JWT-Extended?
I tried to put the iss claim under the parameter 'user_claims' of the create_access_token:
access_token = create_access_token(
identity=data['username'],
expires_delta = timedelta(seconds=JWT_LIFE_SPAN),
user_claims={'iss': ISSUER}
)
However, when I decoded the token using PyJWT from the side of resource server, it looked like this:
{'iat': 1597581227, 'nbf': 1597581227, 'jti': '4e6c9677-d698-421c-91c4-0b2f3a6da4e9', 'exp':
1597583027, 'identity': 'asdf', 'fresh': False, 'type': 'access', 'user_claims': {'iss':
'sample-auth-server'}}
I tried to look for a configuration options from the docs, but I can't find any option to set the iss. There's an iss set, but it is under the user_claims. What I want to accomplish is to set it as one of the registered claims for the JWT.
I think you can make use of encode_access_token api and create an encoded access code with issuer.
Example:
encode_access_token(
identity=data['username'],
issuer=JWT_ISSUER,
expires_delta=timedelta(seconds=JWT_LIFE_SPAN),
...
):
Reference : https://github.com/vimalloc/flask-jwt-extended/blob/5bd8b1ed08ea64d23869a629af3c3c868816b8a8/flask_jwt_extended/tokens.py#L34

Jira API Assignee not populating with "name" anymore, instead needs "accoundId"

I'm working with Python 3.x
Previously, I had a function to create tickets that looks like this
def jira_incident(jira_subject, jira_description):
user = "username"
apikey = 'apikey'
server = 'https://serverName.atlassian.net'
options = {
'server': server,
'verify': False
}
issue_dict = {
'project': {'key': 'project_name'},
'summary': str(jira_subject),
'description': str(jira_description),
'issuetype': {'name': 'Incident'},
'assignee': {'name': my_username},
'priority': {'name': 'Low'},
'customfield_10125':
{'value': 'Application Ops'}
}
jira = JIRA(options, basic_auth=(user, apikey))
new_issue = jira.create_issue(fields=issue_dict)
return new_issue
my_username is a global variable that's used for other things as well.
Anyway, the assignee is no longer working as of about 2 days ago. I did some googling and found that it now needs the accountId instead of the name, I can get this via the web UI by leaving a comment as #'ing someone in a comment. As a temporary solution I've populated a dictionary to reference (and that works), however I'd like to make this more dynamic for future proofing the script.
'assignee': {'accountId': jira_dict[my_username]},
I can't seem to really find any documentation on looking up the accountId from the name, and I figured I'd go ahead and ask the community to see if anyone else has run into/solved this issue.
I was thinking about just writing a new function that performs this query for me, then returns the accountId.
EDIT
I did find this:
import requests
from requests.auth import HTTPBasicAuth
import json
url = "/rest/api/3/user/bulk/migration"
auth = HTTPBasicAuth("email#example.com", "<api_token>")
headers = {
"Accept": "application/json"
}
response = requests.request(
"GET",
url,
headers=headers,
auth=auth
)
print(json.dumps(json.loads(response.text), sort_keys=True, indent=4, separators=(",", ": ")))
However it 404's on me, I add the server address to the beginning of the url, and replace user, with the username in question.
Okay, I found a solution, it's not an elegant solution, but it does exactly what I need it to. So here is the new function:
def jira_account_id_from_username(username):
r = requests.get('https://serverName.atlassian.net/rest/api/3/user?username=' + username, auth=("username",api_key), verify=False)
value = re.search('"accountId":"(.*?)",', str(r.text)).group(1)
return value
I strongly encourage you to not rely on the username anymore. The endpoint you are using is deprecated, see also https://developer.atlassian.com/cloud/jira/platform/deprecation-notice-user-privacy-api-migration-guide/.
The "new" or probably better way is to use the /user/search endpoint as described here: https://developer.atlassian.com/cloud/jira/platform/rest/v3/#api-rest-api-3-user-search-get There you can define a query that is matching against certain properties of a user (displayName or emailAddress), or search for the accountId if you already have it. Therefore, if you are linking users from the cloud to some other "user directory" (or just a scripts where you have stored some usernames), replace it by using either email address or accountId so you can properly link your users.

How to unsubscribe / delete list members using Mailchimp3 in Python?

I am using mailchimp3 in Python. I managed to make batch insertion of users using this function:
client = MailChimp(USERNAME, APIKEY)
def fill_list(list_id, subscribers_data):
data = {'operations': create_subscriptions_data(list_id, subscribers_data)}
client.batches.create(data)
def create_subscriptions_data(list_id, users_data):
return [{'method': 'PUT',
'path': 'lists/{}/members/{}'.format(list_id, str(md5(user['email_address'].encode('utf-8')))),
'body': json.dumps(user)} for user in users_data]
Here is how one user dict looks like:
{"email_address": "user#somemail.com", "status": "subscribed"}
Then I wanted to use similar method to unsubscribe list of users. To achieve that I tried to use the same logic, just to change the user objects. Now, I used this user format:
{"email_address": "user#somemail.com", "status": "unsubscribed"}
It doesn't update the subscribe status. When I deleted all users manually (using the web interface) and tried the same command I successfully created users with "status": "unsubscribed". I am wondering why this approach can't change the status? I tried also using POST instead of PUT, but it didn't work. Any idea what can be the issue?
I used this reference https://devs.mailchimp.com/blog/batch-operations-and-put-in-api-v3-0/ and it mentions that this approach should work fine for updates as well.
Thank you in advance!
The only way to unsubscribe an already subscribed user will be to update with a list id and an MD5 hash of the lowercase version of the list member’s email address.
client.lists.members.update('LIST_ID', 'MD5 HASH', {"status":
"unsubscribed"})
Actually, I was using some wrong functions, so here is the fixed code. I also had some problems with the size of the batches. The maximum batch size is 500, so I did some splits of the data across several batches.
Here is a simple code how the insertion should be done:
client = MailChimp(USERNAME, APIKEY)
def _update_list(list_id: str, members_data: list):
client.lists.update_members(list_id, {'members': members_data, 'update_existing': True})
Each member in members_data has data like this:
mailchimp_user = {
'email_address': user.email,
'status': user.subscription_status,
'merge_fields': {
'FNAME': user.first_name,
'LNAME': user.last_name
},
'interests': {}
}
And here comes the most important function:
def fill_in_multiple_batches(list_id, mailchimp_members):
step_size = 400
for i in range(0, len(mailchimp_members), step_size):
batch_start_idx = i
batch_end_idx = min(i + step_size, len(mailchimp_members))
this_batch_of_members = mailchimp_members[batch_start_idx:batch_end_idx]
client.lists.update_members(list_id, {'members': members_data, 'update_existing': True})
After that, in the main of the script:
if __name__ == '__main__':
fill_in_multiple_batches('your_list_id', your_data_list)

ServiceNow SOAP API close ticket with python

I'm working on connecting a python script to a ServiceNow ticketing environment. Thankfully, ServiceNow has documentation on how to create a ticket from a python script, see documentation here:
http://wiki.servicenow.com/index.php?title=Python_Web_Services_Client_Examples#gsc.tab=0
Here's the script I'm using to create a ticket:
#!/usr/bin/python
from SOAPpy import SOAPProxy
import sys
def createincident(params_dict):
# instance to send to
instance='demo'
# username/password
username='itil'
password='itil'
# proxy - NOTE: ALWAYS use https://INSTANCE.service-now.com, not https://www.service-now.com/INSTANCE for web services URL from now on!
proxy = 'https://%s:%s#%s.service-now.com/incident.do?SOAP' % (username, password, instance)
namespace = 'http://www.service-now.com/'
server = SOAPProxy(proxy, namespace)
# uncomment these for LOTS of debugging output
#server.config.dumpHeadersIn = 1
#server.config.dumpHeadersOut = 1
#server.config.dumpSOAPOut = 1
#server.config.dumpSOAPIn = 1
response = server.insert(impact=int(params_dict['impact']), urgency=int(params_dict['urgency']), priority=int(params_dict['priority']), category=params_dict['category'], location=params_dict['location'], caller_id=params_dict['user'], assignment_group=params_dict['assignment_group'], assigned_to=params_dict['assigned_to'], short_description=params_dict['short_description'], comments=params_dict['comments'])
return response
values = {'impact': '1', 'urgency': '1', 'priority': '1', 'category': 'High',
'location': 'San Diego', 'user': 'fred.luddy#yourcompany.com',
'assignment_group': 'Technical Support', 'assigned_to': 'David Loo',
'short_description': 'An incident created using python, SOAPpy, and web
services.', 'comments': 'This a test making an incident with python.\nIsn\'t
life wonderful?'}
new_incident_sysid=createincident(values)
print "Returned sysid: "+repr(new_incident_sysid)
However, I cannot find any good documentation on the process to resolve the ticket that I just created using the API. When I run the above script, I get the ticket number as well as the sys_id.
Any help would be appreciated.
Thanks.
Apparently thats an old request, but i landed here searching, and apprently 400 other people too, so here is my solution :
Use the server.update(sys_id="...",state="..",...) to modify your record and set the right values to "resolve" it.
For this method only the sys_id parameter is mandatory, it's up to you for the other fields of your form.
P.S. : you can find the API here - https://.service-now.com/incident.do?WSDL

How do you use cookies and HTTP Basic Authentication in CherryPy?

I have a CherryPy web application that requires authentication. I have working HTTP Basic Authentication with a configuration that looks like this:
app_config = {
'/' : {
'tools.sessions.on': True,
'tools.sessions.name': 'zknsrv',
'tools.auth_basic.on': True,
'tools.auth_basic.realm': 'zknsrv',
'tools.auth_basic.checkpassword': checkpassword,
}
}
HTTP auth works great at this point. For example, this will give me the successful login message that I defined inside AuthTest:
curl http://realuser:realpass#localhost/AuthTest/
Since sessions are on, I can save cookies and examine the one that CherryPy sets:
curl --cookie-jar cookie.jar http://realuser:realpass#localhost/AuthTest/
The cookie.jar file will end up looking like this:
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.
localhost FALSE / FALSE 1348640978 zknsrv 821aaad0ba34fd51f77b2452c7ae3c182237deb3
However, I'll get an HTTP 401 Not Authorized failure if I provide this session ID without the username and password, like this:
curl --cookie 'zknsrv=821aaad0ba34fd51f77b2452c7ae3c182237deb3' http://localhost/AuthTest
What am I missing?
Thanks very much for any help.
So, the short answer is you can do this, but you have to write your own CherryPy tool (a before_handler), and you must not enable Basic Authentication in the CherryPy config (that is, you shouldn't do anything like tools.auth.on or tools.auth.basic... etc) - you have to handle HTTP Basic Authentication yourself. The reason for this is that the built-in Basic Authentication stuff is apparently pretty primitive. If you protect something by enabling Basic Authentication like I did above, it will do that authentication check before it checks the session, and your cookies will do nothing.
My solution, in prose
Fortunately, even though CherryPy doesn't have a way to do both built-in, you can still use its built-in session code. You still have to write your own code for handling the Basic Authentication part, but in total this is not so bad, and using the session code is a big win because writing a custom session manager is a good way to introduce security bugs into your webapp.
I ended up being able to take a lot of things from a page on the CherryPy wiki called Simple authentication and access restrictions helpers. That code uses CP sessions, but rather than Basic Auth it uses a special page with a login form that submits ?username=USERNAME&password=PASSWORD. What I did is basically nothing more than changing the provided check_auth function from using the special login page to using the HTTP auth headers.
In general, you need a function you can add as a CherryPy tool - specifically a before_handler. (In the original code, this function was called check_auth(), but I renamed it to protect().) This function first tries to see if the cookies contain a (valid) session ID, and if that fails, it tries to see if there is HTTP auth information in the headers.
You then need a way to require authentication for a given page; I do this with require(), plus some conditions, which are just callables that return True. In my case, those conditions are zkn_admin(), and user_is() functions; if you have more complex needs, you might want to also look at member_of(), any_of(), and all_of() from the original code.
If you do it like that, you already have a way to log in - you just submit a valid session cookie or HTTPBA credentials to any URL you protect with the #require() decorator. All you need now is a way to log out.
(The original code instead has an AuthController class which contains login() and logout(), and you can use the whole AuthController object in your HTTP document tree by just putting auth = AuthController() inside your CherryPy root class, and get to it with a URL of e.g. http://example.com/auth/login and http://example.com/auth/logout. My code doesn't use an authcontroller object, just a few functions.)
Some notes about my code
Caveat: Because I wrote my own parser for HTTP auth headers, it only parses what I told it about, which means just HTTP Basic Auth - not, for example, Digest Auth or anything else. For my application that's fine; for yours, it may not be.
It assumes a few functions defined elsewhere in my code: user_verify() and user_is_admin()
I also use a debugprint() function which only prints output when a DEBUG variable is set, and I've left these calls in for clarity.
You can call it cherrypy.tools.WHATEVER (see the last line); I called it zkauth based on the name of my app. Take care NOT to call it auth, or the name of any other built-in tool, though .
You then have to enable cherrypy.tools.WHATEVER in your CherryPy configuration.
As you can see by all the TODO: messages, this code is still in a state of flux and not 100% tested against edge cases - sorry about that! It will still give you enough of an idea to go on, though, I hope.
My solution, in code
import base64
import re
import cherrypy
SESSION_KEY = '_zkn_username'
def protect(*args, **kwargs):
debugprint("Inside protect()...")
authenticated = False
conditions = cherrypy.request.config.get('auth.require', None)
debugprint("conditions: {}".format(conditions))
if conditions is not None:
# A condition is just a callable that returns true or false
try:
# TODO: I'm not sure if this is actually checking for a valid session?
# or if just any data here would work?
this_session = cherrypy.session[SESSION_KEY]
# check if there is an active session
# sessions are turned on so we just have to know if there is
# something inside of cherrypy.session[SESSION_KEY]:
cherrypy.session.regenerate()
# I can't actually tell if I need to do this myself or what
email = cherrypy.request.login = cherrypy.session[SESSION_KEY]
authenticated = True
debugprint("Authenticated with session: {}, for user: {}".format(
this_session, email))
except KeyError:
# If the session isn't set, it either wasn't present or wasn't valid.
# Now check if the request includes HTTPBA?
# FFR The auth header looks like: "AUTHORIZATION: Basic <base64shit>"
# TODO: cherrypy has got to handle this for me, right?
authheader = cherrypy.request.headers.get('AUTHORIZATION')
debugprint("Authheader: {}".format(authheader))
if authheader:
#b64data = re.sub("Basic ", "", cherrypy.request.headers.get('AUTHORIZATION'))
# TODO: what happens if you get an auth header that doesn't use basic auth?
b64data = re.sub("Basic ", "", authheader)
decodeddata = base64.b64decode(b64data.encode("ASCII"))
# TODO: test how this handles ':' characters in username/passphrase.
email,passphrase = decodeddata.decode().split(":", 1)
if user_verify(email, passphrase):
cherrypy.session.regenerate()
# This line of code is discussed in doc/sessions-and-auth.markdown
cherrypy.session[SESSION_KEY] = cherrypy.request.login = email
authenticated = True
else:
debugprint ("Attempted to log in with HTTBA username {} but failed.".format(
email))
else:
debugprint ("Auth header was not present.")
except:
debugprint ("Client has no valid session and did not provide HTTPBA credentials.")
debugprint ("TODO: ensure that if I have a failure inside the 'except KeyError'"
+ " section above, it doesn't get to this section... I'd want to"
+ " show a different error message if that happened.")
if authenticated:
for condition in conditions:
if not condition():
debugprint ("Authentication succeeded but authorization failed.")
raise cherrypy.HTTPError("403 Forbidden")
else:
raise cherrypy.HTTPError("401 Unauthorized")
cherrypy.tools.zkauth = cherrypy.Tool('before_handler', protect)
def require(*conditions):
"""A decorator that appends conditions to the auth.require config
variable."""
def decorate(f):
if not hasattr(f, '_cp_config'):
f._cp_config = dict()
if 'auth.require' not in f._cp_config:
f._cp_config['auth.require'] = []
f._cp_config['auth.require'].extend(conditions)
return f
return decorate
#### CONDITIONS
#
# Conditions are callables that return True
# if the user fulfills the conditions they define, False otherwise
#
# They can access the current user as cherrypy.request.login
# TODO: test this function with cookies, I want to make sure that cherrypy.request.login is
# set properly so that this function can use it.
def zkn_admin():
return lambda: user_is_admin(cherrypy.request.login)
def user_is(reqd_email):
return lambda: reqd_email == cherrypy.request.login
#### END CONDITIONS
def logout():
email = cherrypy.session.get(SESSION_KEY, None)
cherrypy.session[SESSION_KEY] = cherrypy.request.login = None
return "Logout successful"
Now all you have to do is enable both builtin sessions and your own cherrypy.tools.WHATEVER in your CherryPy configuration. Again, take care not to enable cherrypy.tools.auth. My configuration ended up looking like this:
config_root = {
'/' : {
'tools.zkauth.on': True,
'tools.sessions.on': True,
'tools.sessions.name': 'zknsrv',
}
}

Categories