vcr.py store call from twitter api - python

I want to use https://github.com/bear/python-twitter/ and check API requests https://github.com/kevin1024/vcrpy or https://github.com/agriffis/vcrpy-unittest.
From lines 30:
https://github.com/bear/python-twitter/blob/master/twitter/api.py#L30
30: import requests
and later on:
res = requests.post(url='https://api.twitter.com/oauth2/token',
data={'grant_type': 'client_credentials'},
headers=post_headers)
# ... etc ...
Yet when doing something like:
from vcr_unittest import VCRTestCase
import vcr
import twitter
from django.conf import settings
class TwitterRetrievalAndStorageTests(VCRTestCase):
#vcr.use_cassette()
def test_recorded_session(self):
api = twitter.Api(
consumer_key=settings.TWITTER_CONSUMER_KEY,
consumer_secret=settings.TWITTER_CONSUMER_SECRET,
access_token_key=settings.TWITTER_ACCESS_KEY,
access_token_secret=settings.TWITTER_ACCESS_SECRET)
statuses = api.GetUserTimeline(screen_name='nntaleb')
for s in statuses:
print(s)
Not a cassette file is being created. Is there a way to do this with python-twitter?

VCRTestCase is built on top of vcr. It doesn't make much sense to use both simultaneously. If you remove the line #vcr.use_cassette(), you should see a cassette named TwitterRetrievalAndStorageTests.test_recorded_session.yaml (or json) in your cwd. If instead, you don't inherit from VCRTestCase and use #vcr.use_cassette(), the cassette name should be test_recorded_session.yaml and it should be in the same folder as your class.
In general, if you use both, I have noticed that vcr takes precedence over vcrpy-unittest but that is not consistently the case.

Related

python django soaplib response with classmodel issue

I run a soap server in django.
Is it possible to create a soap method that returns a soaplib classmodel instance without <{method name}Response><{method name}Result> tags?
For example, here is a part of my soap server code:
# -*- coding: cp1254 -*-
from soaplib.core.service import rpc, DefinitionBase, soap
from soaplib.core.model.primitive import String, Integer, Boolean
from soaplib.core.model.clazz import Array, ClassModel
from soaplib.core import Application
from soaplib.core.server.wsgi import Application as WSGIApplication
from soaplib.core.model.binary import Attachment
class documentResponse(ClassModel):
__namespace__ = ""
msg = String
hash = String
class MyService(DefinitionBase):
__service_interface__ = "MyService"
__port_types__ = ["MyServicePortType"]
#soap(String, Attachment, String ,_returns=documentResponse,_faults=(MyServiceFaultMessage,) , _port_type="MyServicePortType" )
def sendDocument(self, fileName, binaryData, hash ):
binaryData.file_name = fileName
binaryData.save_to_file()
resp = documentResponse()
resp.msg = "Saved"
resp.hash = hash
return resp
and it responses like that:
<senv:Body>
<tns:sendDocumentResponse>
<tns:sendDocumentResult>
<hash>14a95636ddcf022fa2593c69af1a02f6</hash>
<msg>Saved</msg>
</tns:sendDocumentResult>
</tns:sendDocumentResponse>
</senv:Body>
But i need a response like this:
<senv:Body>
<ns3:documentResponse>
<hash>A694EFB083E81568A66B96FC90EEBACE</hash>
<msg>Saved</msg>
</ns3:documentResponse>
</senv:Body>
What kind of configurations should i make in order to get that second response i mentioned above ?
Thanks in advance.
I haven't used Python's SoapLib yet, but had the same problem while using .NET soap libs. Just for reference, in .NET this is done using the following decorator:
[SoapDocumentMethod(ParameterStyle=SoapParameterStyle.Bare)]
I've looked in the soaplib source, but it seems it doesn't have a similar decorator. The closest thing I've found is the _style property. As seen from the code https://github.com/soaplib/soaplib/blob/master/src/soaplib/core/service.py#L124 - when using
#soap(..., _style='document')
it doesn't append the %sResult tag, but I haven't tested this. Just try it and see if this works in the way you want it.
If it doesn't work, but you still want to get this kind of response, look at Spyne:
http://spyne.io/docs/2.10/reference/decorator.html
It is a fork from soaplib(I think) and has the _soap_body_style='bare' decorator, which I believe is what you want.

GAE Webapp: the cost of importing a bunch of request handlers

My python GAE app's central application file looks like this:
import webapp2
import homepage
import user_auth
import user_confirm
import admin_user
import admin_config
import config
app = webapp2.WSGIApplication([
(user_auth.get_login_url(), user_auth.LoginHandler),
(user_auth.get_logout_url(), user_auth.LogoutHandler),
("/user/confirm", user_confirm.UserConfirmHandler),
("/admin/config", admin_config.AdminConfigHandler),
("/admin/user/add", admin_user.AdminAddUserHandler),
("/admin/user", admin_user.AdminUserHandler),
("/", homepage.HomepageHandler),
], debug=True)
As you can see, I must import a bunch of request handlers, but for each request, only one of them is used, the other imports are just useless!
That's a big waste of memory and performance because those unnecessary imports also import other things on their own. Does Google App Engine have some "caching" mechanism or something that makes these unnecessary imports negligible? I think not.
How can I avoid them? I just haven't found out the way to import 1 Request Handler per request. If I put all the routing to app.yaml, that would work the way I want, but it makes things complex because I must write app = webapp2.WSGIApplication(... for every request handler file and repeat those boring urls twice (both in the python file and in app.yaml).
Found the way here, already built into webapp2
http://webapp-improved.appspot.com/guide/routing.html#lazy-handlers

what is the best way to make my folders invisible / restricted in twistd?

a fews days ago, i tried to learn the python twisted..
and this is how i make my webserver :
from twisted.application import internet, service
from twisted.web import static, server, script
from twisted.web.resource import Resource
import os
class NotFound(Resource):
isLeaf=True
def render(self, request):
return "Sorry... the page you're requesting is not found / forbidden"
class myStaticFile(static.File):
def directoryListing(self):
return self.childNotFound
#root=static.file(os.getcwd()+"/www")
root=myStaticFile(os.getcwd()+"/www")
root.indexNames=['index.py']
root.ignoreExt(".py")
root.processors = {'.py': script.ResourceScript}
root.childNotFound=NotFound()
application = service.Application('web')
sc = service.IServiceCollection(application)
i = internet.TCPServer(8080, server.Site(root))##UndefinedVariable
i.setServiceParent(sc)
in my code, i make an instance class for twisted.web.static.File and override the directoryListing.
so when user try to access my resource folder (http://localhost:8080/resource/ or http://localhost:8080/resource/css), it will return a notFound page.
but he can still open/read the http://localhost:8080/resource/css/style.css.
it works...
what i want to know is.. is this the correct way to do that???
is there another 'perfect' way ?
i was looking for a config that disable directoryListing like root.dirListing=False. but no luck...
Yes, that's a reasonable way to do it. You can also use twisted.web.resource.NoResource or twisted.web.resource.Forbidden instead of defining your own NotFound.

Link generator using django or any python module

I want to generate for my users temporary download link.
Is that ok if i use django to generate link using url patterns?
Could it be correct way to do that. Because can happen that I don't understand some processes how it works. And it will overflow my memory or something else. Some kind of example or tools will be appreciated. Some nginx, apache modules probably?
So, what i wanna to achieve is to make url pattern which depend on user and time. Decript it end return in view a file.
A simple scheme might be to use a hash digest of username and timestamp:
from datetime import datetime
from hashlib import sha1
user = 'bob'
time = datetime.now().isoformat()
plain = user + '\0' + time
token = sha1(plain)
print token.hexdigest()
"1e2c5078bd0de12a79d1a49255a9bff9737aa4a4"
Next you store that token in a memcache with an expiration time. This way any of your webservers can reach it and the token will auto-expire. Finally add a Django url handler for '^download/.+' where the controller just looks up that token in the memcache to determine if the token is valid. You can even store the filename to be downloaded as the token's value in memcache.
Yes it would be ok to allow django to generate the urls. This being exclusive from handling the urls, with urls.py. Typically you don't want django to handle the serving of files see the static file docs[1] about this, so get the notion of using url patterns out of your head.
What you might want to do is generate a random key using a hash, like md5/sha1. Store the file and the key, datetime it's added in the database, create the download directory in a root directory that's available from your webserver like apache or nginx... suggest nginx), Since it's temporary, you'll want to add a cron job that checks if the time since the url was generated has expired, cleans up the file and removes the db entry. This should be a django command for manage.py
Please note this is example code written just for this and not tested! It may not work the way you were planning on achieving this goal, but it works. If you want the dl to be pw protected also, then look into httpbasic auth. you can generate and remove entries on the fly in a httpd.auth file using htpasswd and the subprocess module when you create the link or at registration time.
import hashlib, random, datetime, os, shutil
# model to hold link info. has these fields: key (charfield), filepath (filepathfield)
# datetime (datetimefield), url (charfield), orgpath (filepathfield of the orignal path
# or a foreignkey to the files model.
from models import MyDlLink
# settings.py for the app
from myapp import settings as myapp_settings
# full path and name of file to dl.
def genUrl(filepath):
# create a onetime salt for randomness
salt = ''.join(['{0}'.format(random.randrange(10) for i in range(10)])
key = hashlib('{0}{1}'.format(salt, filepath).hexdigest()
newpath = os.path.join(myapp_settings.DL_ROOT, key)
shutil.copy2(fname, newpath)
newlink = MyDlink()
newlink.key = key
newlink.date = datetime.datetime.now()
newlink.orgpath = filepath
newlink.newpath = newpath
newlink.url = "{0}/{1}/{2}".format(myapp_settings.DL_URL, key, os.path.basename(fname))
newlink.save()
return newlink
# in commands
def check_url_expired():
maxage = datetime.timedelta(days=7)
now = datetime.datetime.now()
for link in MyDlink.objects.all():
if(now - link.date) > maxage:
os.path.remove(link.newpath)
link.delete()
[1] http://docs.djangoproject.com/en/1.2/howto/static-files/
It sounds like you are suggesting using some kind of dynamic url conf.
Why not forget your concerns by simplifying and setting up a single url that captures a large encoded string that depends on user/time?
(r'^download/(?P<encrypted_id>(.*)/$', 'download_file'), # use your own regexp
def download_file(request, encrypted_id):
decrypted = decrypt(encrypted_id)
_file = get_file(decrypted)
return _file
A lot of sites just use a get param too.
www.example.com/download_file/?09248903483o8a908423028a0df8032
If you are concerned about performance, look at the answers in this post: Having Django serve downloadable files
Where the use of the apache x-sendfile module is highlighted.
Another alternative is to simply redirect to the static file served by whatever means from django.

How can I set a custom response header for pylons static (public) files?

How do I add a custom header to files pylons is serving from public?
a) Let your webserver serve files from /public instead of paster and configure it to pass some special headers.
b) Add a special route and serve the files yourself ala
class FilesController(BaseController):
def download(self, path)
fapp = FileApp( path, headers=self.get_headers(path) )
return fapp(request.environ, self.start_response)
c) maybe there is a way to overwrite headers and i just dont know how.
With a recent version of route, you can use the 'Magic path_info' feature, and follow the documentation from here to write your controller so it calls paster.DirectoryApp.
In my project, I wanted to serve any file in the public directory, including subdirs, and ended with this as controller, to be able to override content_type :
import logging
from paste.fileapp import FileApp
from paste.urlparser import StaticURLParser
from pylons import config
from os.path import basename
class ForceDownloadController(StaticURLParser):
def __init__(self, directory=None, root_directory=None, cache_max_age=None):
if not directory:
directory = config['pylons.paths']['static_files']
StaticURLParser.__init__(self, directory, root_directory, cache_max_age)
def make_app(self, filename):
headers = [('Content-Disposition', 'filename=%s' % (basename(filename)))]
return FileApp(filename, headers, content_type='application/octetstream')
In a standard Pylons setup, the public files are served from a StaticUrlParser. This is typically setup in your config/middleware.py:make_app() function
You need to subclass the StaticUrlParser like Antonin ENFRUN describes, though calling it a Controller is confusing because it's doing a different purpose. Add something like the following to the top of the config/middleware.py:
from paste.fileapp import FileApp
from paste.urlparser import StaticURLParser
class HeaderUrlParser(StaticURLParser):
def make_app(self, filename):
headers = # your headers here
return FileApp(filename, headers, content_type='application/octetstream')
then replace StaticUrlParser in config/middleware.py:make_app() with HeaderUrlParser
static_app = StaticURLParser(config['pylons.paths']['static_files'])
becomes
static_app = HeaderURLParser(config['pylons.paths']['static_files'])
A simpler way to use FileApp for streaming, based on the pylons book. The code below assumes your route provides some_file_identifier, but the other two variables are "magic" (see explanation after code).
class MyFileController(BaseController):
def serve(self, environ, start_response, some_file_identifier):
path = self._convert_id_to_path(some_file_identifier)
app = FileApp(path)
return app(environ, start_response)
Pylons automatically gives you the wsgi environ and start_response variables if you have variables of those names in your method signature. You should not need to set or munge headers otherwise, but if you do you can use the abilities built in to FileApp to achieve this.

Categories