We Keep Coding

I'm getting an invallid syntax error with my class

I've been learning python for a while now but I really want to start using oop but I'm having trouble understanding it, please can you tell me where I'm going wrong with my class.
class Savecookies():
driver = webdriver.Firefox()
def __init__(self, site, url):
self.site = site
self.url = url
def twitter(driver, self.site, self.url):
if __name__=='__main__':
cooks = Savecookies('twitter', 'https://twitter.com/')
My error:
File "twitter_test2.py", line 26
def twitter(driver, self.site, self.url):
^
SyntaxError: invalid syntax

def twitter(driver, self.site, self.url):
What’s that?
First of all, methods need a body. Otherwise they are incomplete. The simplest body would be to just do pass (i.e. do nothing). But you probably want to add actual stuff in there.
Second, your arguments make no sense at all. The first argument of a method is self, and then you specify which other arguments you want the method to accept. And argument names need to be valid variables, so you cannot have a dot in there. And if you want the method to access self.site and self.url, you can just do that without needing to pass it to the function (since you have access to self). In your case, you already have the site and url from the Savecookies object, so you probably want something like this:
def twitter(self, driver):
# Do something useful here
print(self.site, self.url)
print(driver)

in case twitter is an instance method
change
def twitter(driver, self.site, self.url):
to
def twitter(self):
#now do something with the params
print(self.driver, self.site, self.url)
Basically, let it access the site and url instance attributes set by __init__
and let it access the driver class attribute set in the class
Both types of attributes can be reached through self., there is no need to pass it again as a parameter.

Why can I call all these methods that aren't explicitly defined in a class?

So I am working with an API wrapper in python for vk, Europe's Facebook equivalent. The documentation on the vk site has all the API calls that can be used, and the wrapper is able to call them correctly. For example, to get a user's information, you would call api.users.get(id) to get a user by id. My question is this: how can the wrapper correctly handle such a call when neither users or a corresponding users.get() method is defined inside the api object?
I know it involves the __getattr__() and __call__() methods, but I can't find any good documentation on how to use them in this way.
EDIT
the api object is instantiated via api = vk.API(id, email, password)

Let's walk through this together, shall we?
api
To execute api.users.get(), Python first has to know api. And due to your instantiation, it does know it: It's a local variable holding an instance of APISession.
api.users
Then, it has to know api.users. Python first looks at the members of the api instance, at the members of its class (APISession) and at the members of that class' super-classes (only object in the case of APISession). Failing to find a member called users in any of these places, it looks for a member function called __getattr__ in those same places. It will find it right on the instance, because APISession has an (instance) member function of this name.
Python then calls it with 'users' (the name of the so-far missing member) and uses the function's return value as if it were that member. So
api.users
is equivalent to
api.__getattr__('users')
Let's see what that returns.
def __getattr__(self, method_name):
return APIMethod(self, method_name)
Oh. So
api.users # via api.__getattr__('users')
is equivalent to
APIMethod(api, 'users')
creating a new APIMethod instance.
api and 'users' end up as that instance's _api_session and _method_name members. Makes sense, I guess.
api.users.get
Python still hasn't executed our statement. It needs to know api.users.get() to do so. The same game as before repeats, just in the api.users object instead of the api object this time: No member method get() and no member get is found on the APIMethod instance api.users points to, nor on its class or superclasses, so Python turns to the __getattr__ method, which for this class does something peculiar:
def __getattr__(self, method_name):
return APIMethod(self._api_session, self._method_name + '.' + method_name)
A new instance of the same class! Let's plug in the instance members of api.users, and
api.users.get
becomes equivalent to
APIMethod(api, 'users' + '.' + 'get')
So we will have the api object also in api.user.get's _apisession and the string 'users.get' in its _method_name.
api.users.get() (note the ())
So api.users.get is an object. To call it, Python has to pretend it's a function, or more specifically, a method of api.users. It does so, by instead calling api.users.get's __call__ method, which looks like this:
def __call__(self, **method_kwargs):
return self._api_session(self._method_name, **method_kwargs)
Let's work this out:
api.users.get()
# is equivalent to
api.users.get.__call__() # no arguments, because we passed none to `get()`
# will return
api.users.get._api_session(api.users.get._method_name)
# which is
api('users.get')
So now Python is calling the api object as if it were a function. __call__ to the rescue, once more, this time looking like this:
def __call__(self, method_name, **method_kwargs):
response = self.method_request(method_name, **method_kwargs)
response.raise_for_status()
# there are may be 2 dicts in 1 json
# for example: {'error': ...}{'response': ...}
errors = []
error_codes = []
for data in json_iter_parse(response.text):
if 'error' in data:
error_data = data['error']
if error_data['error_code'] == CAPTCHA_IS_NEEDED:
return self.captcha_is_needed(error_data, method_name, **method_kwargs)
error_codes.append(error_data['error_code'])
errors.append(error_data)
if 'response' in data:
for error in errors:
warnings.warn(str(error))
return data['response']
if AUTHORIZATION_FAILED in error_codes: # invalid access token
self.access_token = None
self.get_access_token()
return self(method_name, **method_kwargs)
else:
raise VkAPIMethodError(errors[0])
Now, that's a lot of error handling. For this analysis, I'm only interested in the happy path. I'm only interested in the happy path's result (and how we got there). So lets start at the result.
return data['response']
Where did data come from? It's the first element of response.text interpreted as JSON that does contain a 'response' object. So it seems that from the response object we got, we're extracting the actual response part.
Where did the response object come from? It was returned by
api.method_request('users.get')
Which, for all we care, is a plain normal method call that doesn't require any fancy fallbacks. (Its implementation of course, on some levels, might.)

Assuming the comments are correct, and api is an instance of APISession as defined in this particular commit, then this is a bit of an interesting maze:
So first you want to access api.user. APISession has no such attribute, so it calls __getattr__('user') instead, which is defined as:
def __getattr__(self, method_name):
return APIMethod(self, method_name)
So this constructs an APIMethod(api,'user'). Now you want to call the method get on the APIMethod(api,'user') that is bound to api.users, but an APIMethod doesn't have a get method, so it calls its own __getattr__('get') to figure out what to do:
def __getattr__(self, method_name):
return APIMethod(self._api_session, self._method_name + '.' + method_name)
This returns a APIMethod(api,'users.get') which is then called, invoking the __call__ method of the APIMethod class, which is:
def __call__(self, **method_kwargs):
return self._api_session(self._method_name, **method_kwargs)
So this tries to return api('users.get'), but api is an APISession object, so it invokes the __call__ method of this class, defined as (stripping out the error handling for simplicity):
def __call__(self, method_name, **method_kwargs):
response = self.method_request(method_name, **method_kwargs)
response.raise_for_status()
for data in json_iter_parse(response.text):
if 'response' in data:
return data['response']
So it then calls a method_request('users.get'), which if you follow that method actually does a POST request, and some data comes back as a response, which is then returned.

The users.get() has nothing to do with the api object. As for the users, you are right, if there is no such member defined, then there is certainly some logic inside the __getattr__. So as you can see in the documentation __getattr__ is...
Called when an attribute lookup has not found the attribute in the usual places (i.e. it is not an instance attribute nor is it found in the class tree for self). name is the attribute name.
So exactly, as there is no users defined for the api's class, then the __getattr__ is being called with 'users' passed as the name parameter. Then, most probably dynamically, depending on the passed parameter, an object is being constructed for the users component and returned, which will be responsible to define or handle in similar way the get() method.
To get the whole idea, try the following:
class A(object):
def __init__(self):
super(A, self).__init__()
self.defined_one = 'I am defined inside A'
def __getattr__(self, item):
print('getting attribute {}'.format(item))
return 'I am ' + item
a = A()
>>> print(a.some_item) # this will call __getattr__ as some_item is not defined
getting attribute some_item
I am some_item
>>> print(a.and_another_one) # this will call __getattr__ as and_another_one is not defined
getting attribute and_another_one
I am and_another_one
>>> print(a.defined_one) # this will NOT call __getattr__ as defined_one is defined in A
I am defined inside A

How to patch a function that a Flask view calls

My webapp wants to send a message to AWS SQS with boto and I'd want to mock out sending the actual message and just checking that calling send_message is called. However I do not understand how to use python mock to patch a function that a function being tested calls.
How could I achieve mocking out boto con.send_message as in the pseudo-like code below?
views.py:
#app.route('/test')
def send_msg():
con = boto.sqs.connect_to_region("eu-west-1",aws_access_key_id="asd",aws_secret_access_key="asd")
que = con.get_queue('my_queue')
msg = json.dumps({'data':'asd'})
r=con.send_message(que, msg)
tests.py
class MyTestCase(unittest.TestCase):
def test_test(self):
with patch('views.con.send_message') as sqs_send:
self.test_client.get('/test')
assert(sqs_send.called)

To do this kind of test you need patch connect_to_region(). When this method is patched return a MagicMock() object that you can use to test all your function behavior.
Your test case can be something like this one:
class MyTestCase(unittest.TestCase):
#patch("boto.sqs.connect_to_region", autospec=True)
def test_test(self, mock_connect_to_region):
#grab the mocked connection returned by patched connect_to_region
mock_con = mock_connect_to_region.return_value
#call client
self.test_client.get('/test')
#test connect_to_region call
mock_connect_to_region.assert_called_with("eu-west-1",aws_access_key_id="asd",aws_secret_access_key="asd")
#test get_queue()
mock_con.get_queue.assert_called_with('my_queue')
#finaly test send_message
mock_con.send_message.assert_called_with(mock_con.get_queue.return_value, json.dumps({'data':'asd'}))
Just some notes:
I wrote it in a white box style and check all calls of your view: you can do it more loose and omit some checks; use self.assertTrue(mock_con.send_message.called) if you want just check the call or use mock.ANY as argument if you are not interested in some argument content.
autospec=True is not mandatory but very useful: take a look at autospeccing.
I apologize if code contains some error... I cannot test it now but I hope the idea is clear enough.

Calling another view in Pyramid

My goal: In Pyramid, to call another view-callable, and to get a Response object back without knowing any details about that view-callable.
In my Pyramid application, say I have a view "foo" which is defined using a view_config decorator:
#view_config(route_name="foo",
renderer="foo.jinja2")
def foo_view(request):
return {"whereami" : "foo!"}
Now say that I want to route "bar" to a view that does the same thing for the time being, so it internally calls foo_view and returns its Response:
#view_config(route_name="bar")
def bar_view(request):
return foo_view(request)
...but wait! That doesn't work, since foo_view doesn't return a Response, its renderer does.
So, this will work:
#view_config(route_name="bar",
renderer="foo.jinja2")
def bar_view(request):
return foo_view(request)
as it will apply the same renderer as foo_view did. But this is bad, as I now must repeat myself by copying the renderer value AND having to know the renderer of the view being called.
So, I am going to hope that there is some function available in Pyramid that allows calling another view-callable and getting a Response object back without knowing or caring how it was rendered:
#view_config(route_name="bar")
def bar_view(request):
response = some_function_that_renders_a_view_callable(foo_view, request)
return response
What would some_function_that_renders_a_view_callable be?
pyramid.views.render_view appears to search for a view by name; I don't want to give my views names.
(Note: Returning HTTPFound to cause the client to redirect to the target route is what I am trying avoid. I want to "internally" redirect).

Yep. There is some concerns
doesn't return a Response
predicates/renderer
permissions
request properties associated to old request
Thats why you should not call view from view as function, unless you know what you doing
Pyramid creators did awesome tool for server side redirect - http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/subrequest.html

You can invoking a view with using request.invoke_subrequest:
from wsgiref.simple_server import make_server
from pyramid.config import Configurator
from pyramid.request import Request
def view_one(request):
subreq = Request.blank('/view_two')
response = request.invoke_subrequest(subreq)
return response
def view_two(request):
request.response.body = 'This came from view_two'
return request.response
if __name__ == '__main__':
config = Configurator()
config.add_route('one', '/view_one')
config.add_route('two', '/view_two')
config.add_view(view_one, route_name='one')
config.add_view(view_two, route_name='two')
app = config.make_wsgi_app()
server = make_server('0.0.0.0', 8080, app)
server.serve_forever()`
When /view_one is visted in a browser, the text printed in the
browser pane will be "This came from view_two". The view_one view
used the pyramid.request.Request.invoke_subrequest() API to obtain a
response from another view (view_two) within the same application
when it executed. It did so by constructing a new request that had a
URL that it knew would match the view_two view registration, and
passed that new request along to
pyramid.request.Request.invoke_subrequest(). The view_two view
callable was invoked, and it returned a response. The view_one view
callable then simply returned the response it obtained from the
view_two view callable.

I was struggling with this as well. I have a solution using the render_to_response method, though I'm sure there's a "more correct" way to do it. Until someone posts it, however, here is how I handled this:
from pyramid.renderers import render_to_response
#view_config(route_name="foo", renderer="foo.mak")
def foo_view(request):
return {'stuff':'things', '_renderer':'foo.mak')
def bar_view(request):
values = foo_view(request)
renderer = values['_renderer']
return render_to_response(renderer,values)
(Pyramid 1.3)
This requires a renderer to be used, but by declaring that renderer in the original view's return values, you can retrieve it in another view without knowing what it is. I'm suspecting the need to do this isn't easily findable because there's other, better methods for accomplishing tasks solved by this solution.
Another shortcoming is that it relies on direct import of the view callable. It would be nice if it could be looked up directly by route.

The Pyramid documentation here indicates that leaving the name key word argument out of view_config will cause the view to be registered by the function itself (rather than a string):
Such a registration... implies that the view name will be *my_view*
So, in your case you should be able to use pyramid.view.render_view or pyramid.view.render_view_to_response referencing foo_view directly:
#view_config(route_name="bar")
def bar_view(request):
return pyramid.views.render_view_to_response(None, request, name=foo_view)
Update:
Yep, your right, passing the view function does not work.
It's interesting, but taking your example code and applying the route_name to the config
did not work for me. However, the following example, just giving the view a name sets the route url
and gives the view a name. In this fashion render_view_to_response works as advertised. Naming,
your views may not be what you want, but this configuration accomplishes the same thing as your
example code without added configuration.
#view_config(name="foo")
def foo_view(request):
# returning a response here, in lieu of having
# declared a renderer to delegate to...
return Response('Where am i? `{0[whereami]}'.format({"whereami" : "foo!"}))
#view_config(name="bar")
def bar_view(request):
# handles the response if bar_view has a renderer
return render_view_to_response(None, request, name='foo')
#view_config(name="baz")
def baz_view(request):
# presumably this would not work if foo_view was
# not returning a Response object directly, as it
# skips over the rendering part. I think you would
# have to declare a renderer on this view in that case.
return foo_view(request)
if __name__ == '__main__':
config = Configurator()
config.scan()
app = config.make_wsgi_app()
serve(app, host='127.0.0.1', port='5000')

Not the precise solution you asked for, but a solution to the problem you describe:
Create a view class, of which both foo and bar are methods. Then bar can call self.foo()
Common view_configuration, such as the template name can be applied to the class, and then you can decorate each method with just the view name.
In short, the following should meet your needs, if I understand the problem correctly.
#view_defaults(renderer="foo.jinja2")
class WhereaboutsAreFoo(object):
#view_config(route-name="foo")
def foo_view(self):
return {"whereami" : "foo!"}
#view_config(route-name="bar")
def bar_view(self):
return self.foo_view()

can't you do something like that:
#view_config(name="baz")
def baz_view(request):
return HTTPFound(location=self.request.route_path('foo'))

App Engine (Python) Datastore Precall API Hooks

Background
So let's say I'm making app for GAE, and I want to use API Hooks.
BIG EDIT: In the original version of this question, I described my use case, but some folks correctly pointed out that it was not really suited for API Hooks. Granted! Consider me helped. But now my issue is academic: I still don't know how to use hooks in practice, and I'd like to. I've rewritten my question to make it much more generic.
Code
So I make a model like this:
class Model(db.Model):
user = db.UserProperty(required=True)
def pre_put(self):
# Sets a value, raises an exception, whatever. Use your imagination
And then I create a db_hooks.py:
from google.appengine.api import apiproxy_stub_map
def patch_appengine():
def hook(service, call, request, response):
assert service == 'datastore_v3'
if call == 'Put':
for entity in request.entity_list():
entity.pre_put()
apiproxy_stub_map.apiproxy.GetPreCallHooks().Append('preput',
hook,
'datastore_v3')
Being TDD-addled, I'm making all this using GAEUnit, so in gaeunit.py, just above the main method, I add:
import db_hooks
db_hooks.patch_appengine()
And then I write a test that instantiates and puts a Model.
Question
While patch_appengine() is definitely being called, the hook never is. What am I missing? How do I make the pre_put function actually get called?

Hooks are a little low level for the task at hand. What you probably want is a custom property class. DerivedProperty, from aetycoon, is just the ticket.
Bear in mind, however, that the 'nickname' field of the user object is probably not what you want - per the docs, it's simply the user part of the email field if they're using a gmail account, otherwise it's their full email address. You probably want to let users set their own nicknames, instead.

The issue here is that within the context of the hook() function an entity is not an instance of db.Model as you are expecting.
In this context entity is the protocol buffer class confusingly referred to as entity (entity_pb). Think of it like a JSON representation of your real entity, all the data is there, and you could build a new instance from it, but there is no reference to your memory-resident instance that is waiting for it's callback.
Monkey patching all of the various put/delete methods is the best way to setup Model-level callbacks as far as I know†
Since there doesn't seem to be that many resources on how to do this safely with the newer async calls, here's a BaseModel that implements before_put, after_put, before_delete & after_delete hooks:
class HookedModel(db.Model):
def before_put(self):
logging.error("before put")
def after_put(self):
logging.error("after put")
def before_delete(self):
logging.error("before delete")
def after_delete(self):
logging.error("after delete")
def put(self):
return self.put_async().get_result()
def delete(self):
return self.delete_async().get_result()
def put_async(self):
return db.put_async(self)
def delete_async(self):
return db.delete_async(self)
Inherit your model-classes from HookedModel and override the before_xxx,after_xxx methods as required.
Place the following code somewhere that will get loaded globally in your applicaiton (like main.py if you use a pretty standard looking layout). This is the part that calls our hooks:
def normalize_entities(entities):
if not isinstance(entities, (list, tuple)):
entities = (entities,)
return [e for e in entities if hasattr(e, 'before_put')]
# monkeypatch put_async to call entity.before_put
db_put_async = db.put_async
def db_put_async_hooked(entities, **kwargs):
ents = normalize_entities(entities)
for entity in ents:
entity.before_put()
a = db_put_async(entities, **kwargs)
get_result = a.get_result
def get_result_with_callback():
for entity in ents:
entity.after_put()
return get_result()
a.get_result = get_result_with_callback
return a
db.put_async = db_put_async_hooked
# monkeypatch delete_async to call entity.before_delete
db_delete_async = db.delete_async
def db_delete_async_hooked(entities, **kwargs):
ents = normalize_entities(entities)
for entity in ents:
entity.before_delete()
a = db_delete_async(entities, **kwargs)
get_result = a.get_result
def get_result_with_callback():
for entity in ents:
entity.after_delete()
return get_result()
a.get_result = get_result_with_callback
return a
db.delete_async = db_delete_async_hooked
You can save or destroy your instances via model.put() or any of the db.put(), db.put_async() etc, methods and get the desired effect.
†would love to know if there is an even better solution!?

I don't think that Hooks are really going to solve this problem. The Hooks will only run in the context of your AppEngine application, but the user can change their nickname outside of your application using Google Account settings. If they do that, it won't trigger any logic implement in your hooks.
I think that the real solution to your problem is for your application to manage its own nickname that is independent of the one exposed by the Users entity.