I am kind of new in python, trying to develop an user interaction with an API.
In this API I have to authenticate first and then with a token received I can do multiple things,
I have a module authfile.py that returns a cookie which is in a variable
url = "https://172.16.1.77:8089/api"
headers = {
'Content-Type': 'application/json;charset=UTF-8',
'Connection': 'close',
}
def auth(self, url, headers):
data = '{"request": {"action":"challenge","user":"apiuser","version":"1.0"}}'
# First send the challenge.
response = requests.post(url, headers=headers, data=data, verify=False)
# Get the challenge into variable.
a = response.json()
b = a['response']
c = b['challenge']
print("This is the challenge number: " + c)
# Getting the challenge with the Password with MD5
e = input("Please enter the Password\n")
user = c+e
h = hashlib.md5(user.encode())
md = h.hexdigest()
print("This is the Token: " + md)
datas = '{"request":{"action":"login", "token":"' + str(md) + '", "url":"' + str(url) + '" , "user": "cdrapi"}}'
#Send the token to get the Cookie
response2 = requests.post(url, headers=headers, data=datas, verify=False)
#Getting the cookie into a variable
global cookie
f = response2.json()
g = f['response']
cookie = g['cookie']
print("This is the Cookie: " +cookie)
return cookie
cookie = auth(url, headers)
So I have another file that I use to apply some changes, it will need to be in a different module since the applychanges() functions will be called many times
applychanes.py:
from authfile import url, headers, cookie
import requests
def changes():
while True:
q = (input("Are you sure you would like to apply changes?\nPlease enter only Y for Yes and N for No\n").lower())
if q.lower() not in ("y", "n"):
print("This is not a valid entry")
if q == "y":
postta = '{"request":{"action":"applyChanges", "cookie":"' + str(cookie) + '"}}'
format_to_json = requests.post(url, headers=headers, data=postta, verify=False)
jsonresponse = format_to_json.json()
# Get the response in a variable
presponse = jsonresponse['response']
# Get the status code into a variable
codestatus = jsonresponse['status']
# Print the reponse
print(presponse)
print(codestatus)
print("The changes has been applied")
break
if q == "n":
print("NO changes has been made")
break
else:
continue
return q
When I call the Cookie, the auth file is excecuted again changing the Cookie and the changes will not be applied since the new cookie means a new session**
How do I call a variable preventing to run the function?
On the Main program the auth MUST be excecuted before anything else.
when I call the cookie, the authfile is executed again
If I read you right here, you mean than when you do import authfile or from authfile import cookie, authfile gets evaluated again, and the cookie changes.
This is impossible, unless you are using reload (or a reload-like trick, like del sys.modules["authfile"]. Python will only ever evaluate an imported module once: after that it just binds a new name to it.* Thus if your debugging is premised on the idea that importing the same file multiple times is causing the file to be evaluated multiple times, it's based on a misconception, and something else is going wrong.
On the other hand, I could easily be misreading you here! Do clarify if I'm wrong; I'll check back on this tomorrow [and delete this sentence either way].
In any case, this is probably an instance of a bigger problem: modules are not really for state: that's more what classes are for. Modules just insulate namespaces, but that can have problems if you rely on module-level variables to hold state. So if you really do have a problem with state changing stored in a module, either make a class, to be the 'single source of truth' with respect to this state:
class Authentication:
def __init__(self, url, headers):
# put your auth code here
self._cookie = ...
#property
def cookie(self):
return self._cookie
With appropriate methods to refresh the cookie when needed, or possibly with a dict of cookies if you have multiple realms.
If you don't want to do that and you still think something else is modifying your cookie, replace all instances of cookie with a get_cookie() function defined as:
_cookie = auth(url, headers)
def get_cookie():
return _cookie
That will throw an error when somebody tries to write to your cookie.
Note that all this assumes that cookie is an immuteable type. A possible cause of your problem is cookie being e.g. a dict or list, and something else modifying that---which will modify the original object.
Lastly, if your problem is simply that you want to define cookie in authfile but only generate it later, wrap it in a function or method on a class (personally I would do this---classes are for state, modules for namespace), and call the method later. Something like:
class Authentication:
def __init__(self, url, headers):
self._url = url
self._headers = headers
#staticmethod
def authenticate(url, headers):
# your auth fn
#property
def cookie(self):
if not self._cookie:
self._cookie = self.authenticate(self._url, self._headers)
return self._cookie
auth = Authentication(url, headers)
This will generate the cookie the first time it's accessed. So import as:
from authfile import auth
...
with auth.cookie as cookie:
...
*Alright, I don't actually know what happens in the case of circular imports, whether Python 'evaluates' the same file multiple times before bailing. But it bails, so the question is moot.
Related
I have a function like this:
def get_some_data(api_url, **kwargs)
# some logic on generating headers
# some more logic
response = requests.get(api_url, headers, params)
return response
I need to create a fake/mock "api_url", which, when made request to, would generate a valid response.
I understand how to mock the response:
def mock_response(data):
response = requests.Response()
response.status_code = 200
response._content = json.dumps(data)
return response
But i need to make the test call like this:
def test_get_some_data(api_url: some_magic_url_path_that_will_return_mock_response):
Any ideas on how to create an url path returning a response within the scope of the test (only standard Django, Python, pytest, unittest) would be very much appreciated
The documentation is very well written and more than clear on how to mock whatever you want. But, let say you have a service that makes the 3rd party API call:
def foo(url, params):
# some logic on generating headers
# some more logic
response = requests.get(url, headers, params)
return response
In your test you want to mock the return value of this service.
#patch("path_to_service.foo")
def test_api_call_response(self, mock_response):
mock_response.return_value = # Whatever the return value you want it to be
# Here you call the service as usual
response = foo(..., ...)
# Assert your response
There are some operations that needs to be done before running some routes. For example :
check if we recognise the user,
check the language,
check the location,
set variables in the navbar (here after named header) of the html
and so on, then make decisions based on the outcome and lastly run the requested route.
I find it hard to use the respose.set_cookie("cookie_name", actual_cookie) inside a decorator. It seems flask has a "make_response" object that works well (see here on stack overflow issue 34543157 : Python Flask - Setting a cookie using a decorator), but I find it difficult to reproduce the same thing with bottle.
any how here is my attempt that is not working :
#python3
#/decorator_cookie.py
from bottle import request, response, redirect
from other_module import datamodel, db_pointer, secret_value #custom_module
import json
cookie_value = None
surfer_email_exist_in_db = None
header = None
db_pointer = instanciation_of_a_db_connexion_to_tables
surfer = db_pointer.get(request.get_cookie('surfer')) if db_pointer.get(request.get_cookie('surfer')) != None else "empty"
def set_header(func):
def header_manager():
global cookie_value, surfer_email_exist_in_db, header, db_pointer
cookie_value = True #for stack-overflow question convenience
surfer_email_exist_in_db = True #for stack-overflow question convenience
if not all([cookie_value, surfer_email_exist_in_db]):
redirect('/login')
else:
header = json.dumps(db_pointer.get('header_fr'))
response.set_cookie("header", header, secret = secret_value, path = "/", httponly = True)
return func()
return header_manager
and the main file where the routing goes to
#python3
#/main.py
from bottle import route, request
from decorator_cookie import set_header
from other_module secret_value
#route('/lets_try')
#set_header
def lets_try():
header = request.get_cookie('header', secret = secret_value)
print(header) #here I get None
return template('lets_try.tpl', headers = header)
I also tried set the cookie like that :
make_response = response(func).set_cookie("header", header, secret = secret_value, path = "/", httponly = True)
But got an error :)
Here is the response doc : Response documentation
Do you have any clues ?
Thanks
There is no issue with your code, what you are missing is understanding is understanding
Request 1 [By Browser/No Cookies] -> Request has No cookies -> Response you add cookie header
Request 2 [By Browser/Header Cookies] -> Request has Header cookies -> Response
So for your first request Request.get_cookie will return None but for your second request it will actually return the value
The code below sets up _request if the HTTP method is GET, then it has an ifstatement for handling PUT POST and PATCH.
I'm trying to have one single request setup statement for all method types.
Is this possible? It appears to me that there is no way to add data to a prepared request, and if this is true then perhaps I'm stuck with needing two different ways of setting up a request, one way for GET and one way for PUT, PATCH and POST.
def fetch_from_api(self):
s = Session()
headers = { "Authorization" : REST_API_AUTHORIZATION_HEADER}
_request = Request(self.method, self.url_for_api, headers=headers)
if self.method in ['POST', 'PATCH', 'PUT']:
headers['content-type'] = 'application/x-www-form-urlencoded'
_request = Request(self.method, self.url_for_api, headers=headers, data=self.postdata)
prepped = _request.prepare()
self.api_response = s.send(prepped)
The question is a little old and hopefully #DukeDougal already has a solution. Maybe this will help others, though.
The first thing I notice in the example is that a Request object is created near the beginning of the method. Then, if the method is "POST", "PATCH", or "PUT", the Request constructor is called again to get another object. In that case, the first object is gone. It was created unnecessarily.
When a data= argument isn't given to the Request constructor, it's the same as specifying data=None. Take advantage of that and call the constructor only once, then a data value won't need to be added to an existing Request (or PreparedRequest) object:
def fetch_from_api(self):
s = Session()
headers = {'Authorization': REST_API_AUTHORIZATION_HEADER}
data = None # Assume no data until method is checked
if self.method in ['POST', 'PATCH', 'PUT']:
headers['content-type'] = 'application/x-www-form-urlencoded'
data = self.postdata # Add the data
# Now headers and data are ready, get a Request object
_request = Request(self.method, self.url_for_api, headers=headers, data=data)
prepped = _request.prepare()
self.api_response = s.send(prepped)
If you look at the requests.Request model, it looks like you can set the data attribute if needed:
some_request = Request(method, url, headers=headers)
if # ...we decide we need to add data:
some_request.data = data
Looking at the model, it appears that this would work, because when you prepare the request later on, it looks at the instance's data attribute.
EDIT:
But reading your question a bit more closely, it looks like you want to add data to a prepared_request. I guess you could create your own prepared_request and pass the data in specifically when you call the prepare method, but I don't see how that helps? It seems like you want to just branch and maybe add data or maybe not?
Anyway, the above seems it could potentially simplify your code slightly to the following:
def fetch_from_api(self):
s = Session()
headers = { "Authorization" : REST_API_AUTHORIZATION_HEADER}
_request = Request(self.method, self.url_for_api, headers=headers)
if self.method in ['POST', 'PATCH', 'PUT']:
headers['content-type'] = 'application/x-www-form-urlencoded'
_request.data = self.postdata
prepped = _request.prepare()
self.api_response = s.send(prepped)
(But that doesn't look much simpler to me. What are we trying to achieve? Also, it seems weird to have a method called fetch_from_api that could also be POSTing or PUTing data. As a dev, I would not be expecting that to be the case from the name.)
In the past, I've done stuff like this as a result of having to sign requests: I have to create them in one place and then hand them off to a class that knows how to create signatures, which then hands them back. In other words, you can certainly edit requests before preparing and sending them on their way.
Anyway, I haven't tried any of this, but it's similar to some things I've done in the past with requests, so it looks legit, but I would be concerned about what you are attempting to achieve and whether or not things are being crammed together which maybe should not be.
I am using HTTPforHumans, requests module.
import requests
def pv_request(url, methods, data=None, headers=None, type=None):
try:
if 'POST' in methods:
return requests.post(url=url, headers=headers, data=data).json()
elif 'GET' in methods:
return requests.get(url=url, headers=headers, data=data).json()
elif 'PUT' in methods:
if type == 'music':
return requests.put(url=url, headers=headers, data=data).json()
elif type == 'image':
return requests.put(url=url, headers=headers, data=open(data, 'rb')).json()
except requests.exceptions.ConnectionError:
return None
Might not be in the lines of what you are looking for, but here is my all-in-on purpose request handler.
I created a chatbot that connects to a server and can read messages, now I'm at the point where I need to send messages, requiring request payload (according to the Network tab in Developer tools on google chrome). My opener consists of nothing but the following:
import urllib
import urllib2
from cookielib import CookieJar
self.cj = CookieJar()
self.opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(self.cj))
To stay and connected and read messages, I do the following, I do the following:
def connect(self,settings,xhr):
xhr_polling = self.get_code(xhr)
data = self.opener.open("http://chat2-1.wikia.com:80/socket.io/1/xhr-polling/" + xhr_polling + "?name=HairyBot&key=" +
settings['chatkey'] + "&roomId=" + str(settings['room']) + "&t=" + timestamp())
return data.read()
Settings consisting of the roomId and chatkey. The timestamp function creates a timestamp in accordance to what the servers needs (which isn't necessary to know for this question). Back to the question though, how can a payload be added to the opener to send a message to the chat?
As a suggestion, I recommend use the Requests library. It makes this stuff really simple:
import requests
session = requests.session() # For connection pooling
def connect(self,settings,xhr):
xhr_polling = self.get_code(xhr)
request = session.get('http://chat2-1.wikia.com:80/socket.io/1/xhr-polling/' + xhr_polling, params={
'name': 'HairyBot',
'key': settings['chatkey'],
'roomId': settings['room'],
't': timestamp()
})
return request.text
If you want to send a POST request instead, just change get to post and add some data:
def connect(self,settings,xhr):
xhr_polling = self.get_code(xhr)
request = session.post('http://chat2-1.wikia.com:80/socket.io/1/xhr-polling/' + xhr_polling, params={
'name': 'HairyBot',
'key': settings['chatkey'],
'roomId': settings['room'],
't': timestamp()
}, data={
'key': 'value'
})
return request.text
I'm not sure what you mean by "a payload", but presumably it's just another form variable named payload. If so, you send it the same way you do any other form variable, and you're already sending a bunch—roomId, t, etc.
One way of sending form variables is by URL-encoding them, tacking them onto the query string, and sending a GET request. That's what you're doing now. (It would be better to use proper urllib methods instead of hacking it together with string concatenation, but the end result is the same.)
The other way is by sending a POST body. The urllib2 documentation explains how to do this, and there are plenty of good examples online, but basically all you have to do is call urllib.urlencode() on your name-value pairs, then pass the result as the second argument (or as a keyword argument named data) to the open call.
In other words, something like this:
data = self.opener.open("http://chat2-1.wikia.com:80/socket.io/1/xhr-polling/" + xhr_polling,
urllib.urlencode(("name", "HairyBot"),
("key", settings['chatkey']),
("roomId", str(settings['room']),
("key", settings['chatkey']),
("t", timestamp()),
("payload", payload)))
Or, if you prefer, most servers will allow you to send some parameters on the query string and others in the POST data, so you can leave your existing code alone and just make one change:
data = self.opener.open("http://chat2-1.wikia.com:80/socket.io/1/xhr-polling/" + xhr_polling + "?name=HairyBot&key=" +
settings['chatkey'] + "&roomId=" + str(settings['room']) + "&t=" + timestamp(),
urllib.urlencode(("payload", payload)))
There's a lot of stuff out there on urllib2 and POST calls, but I'm stuck on a problem.
I'm trying to do a simple POST call to a service:
url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe',
'age' : '10'})
content = urllib2.urlopen(url=url, data=data).read()
print content
I can see the server logs and it says that I'm doing GET calls, when I'm sending the data
argument to urlopen.
The library is raising an 404 error (not found), which is correct for a GET call, POST calls are processed well (I'm also trying with a POST within a HTML form).
Do it in stages, and modify the object, like this:
# make a string with the request type in it:
method = "POST"
# create a handler. you can specify different handlers here (file uploads etc)
# but we go for the default
handler = urllib2.HTTPHandler()
# create an openerdirector instance
opener = urllib2.build_opener(handler)
# build a request
data = urllib.urlencode(dictionary_of_POST_fields_or_None)
request = urllib2.Request(url, data=data)
# add any other information you want
request.add_header("Content-Type",'application/json')
# overload the get method function with a small anonymous function...
request.get_method = lambda: method
# try it; don't forget to catch the result
try:
connection = opener.open(request)
except urllib2.HTTPError,e:
connection = e
# check. Substitute with appropriate HTTP code.
if connection.code == 200:
data = connection.read()
else:
# handle the error case. connection.read() will still contain data
# if any was returned, but it probably won't be of any use
This way allows you to extend to making PUT, DELETE, HEAD and OPTIONS requests too, simply by substituting the value of method or even wrapping it up in a function. Depending on what you're trying to do, you may also need a different HTTP handler, e.g. for multi file upload.
This may have been answered before: Python URLLib / URLLib2 POST.
Your server is likely performing a 302 redirect from http://myserver/post_service to http://myserver/post_service/. When the 302 redirect is performed, the request changes from POST to GET (see Issue 1401). Try changing url to http://myserver/post_service/.
Have a read of the urllib Missing Manual. Pulled from there is the following simple example of a POST request.
url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe', 'age' : '10'})
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
print response.read()
As suggested by #Michael Kent do consider requests, it's great.
EDIT: This said, I do not know why passing data to urlopen() does not result in a POST request; It should. I suspect your server is redirecting, or misbehaving.
The requests module may ease your pain.
url = 'http://myserver/post_service'
data = dict(name='joe', age='10')
r = requests.post(url, data=data, allow_redirects=True)
print r.content
it should be sending a POST if you provide a data parameter (like you are doing):
from the docs:
"the HTTP request will be a POST instead of a GET when the data parameter is provided"
so.. add some debug output to see what's up from the client side.
you can modify your code to this and try again:
import urllib
import urllib2
url = 'http://myserver/post_service'
opener = urllib2.build_opener(urllib2.HTTPHandler(debuglevel=1))
data = urllib.urlencode({'name' : 'joe',
'age' : '10'})
content = opener.open(url, data=data).read()
Try this instead:
url = 'http://myserver/post_service'
data = urllib.urlencode({'name' : 'joe',
'age' : '10'})
req = urllib2.Request(url=url,data=data)
content = urllib2.urlopen(req).read()
print content
url="https://myserver/post_service"
data["name"] = "joe"
data["age"] = "20"
data_encoded = urllib2.urlencode(data)
print urllib2.urlopen(url + "?" + data_encoded).read()
May be this can help