I am trying to run the functional tests (using Selenium in python/django) directly from the django views by using management.call_command, in order to allow the user to run the test from the web site. The django view is something like:
class MyView():
def get(self):
output = call_command('test', 'folder.tests.MyTest')
# doing ./manage.py test folder.tests.MyTest
test_result = 'Test result: ' + output
return something_http_with_test_result
What is the best way to do this in order to do not affect the current user data? MyTest is going to create a lot of object in the database but the user must not see them.
Thank you
The best way I found to run it properly is to use os.system :
import os
dir = 'your_absolute_path_to_project'
class MyView:
def some_func_called():
os.system(dir + '/manage.py test >' + dir + '/log.txt')
The project and firefox have to have the same owner and results will be in log.txt.
Related
I need to test a function with different parameters, and the most proper way for this seems to be using the with self.subTest(...) context manager.
However, the function writes something to the db, and it ends up in an inconsistent state. I can delete the things I write, but it would be cleaner if I could recreate the whole db completely. Is there a way to do that?
Not sure how to recreate the database in self.subTest() but I have another technique I am currently using which might be of interest to you. You can use fixtures to create a "snapshot" of your database which will basically be copied in a second database used only for testing purposes. I currently use this method to test code on a big project I'm working on at work.
I'll post some example code to give you an idea of what this will look like in practice, but you might have to do some extra research to tailor the code to your needs (I've added links to guide you).
The process is rather straighforward. You would be creating a copy of your database with only the data needed by using fixtures, which will be stored in a .yaml file and accessed only by your test unit.
Here is what the process would look like:
List item you want to copy to your test database to populate it using fixtures. This will only create a db with the needed data instead of stupidly copying the entire db. It will be stored in a .yaml file.
generate.py
django.setup()
stdout = sys.stdout
conf = [
{
'file': 'myfile.yaml',
'models': [
dict(model='your.model', pks='your, primary, keys'),
dict(model='your.model', pks='your, primary, keys')
]
}
]
for fixture in conf:
print('Processing: %s' % fixture['file'])
with open(fixture['file'], 'w') as f:
sys.stdout = FixtureAnonymiser(f)
for model in fixture['models']:
call_command('dumpdata', model.pop('model'), format='yaml',indent=4, **model)
sys.stdout.flush()
sys.stdout = stdout
In your test unit, import your generated .yaml file as a fixture and your test will automatically use this the data from the fixture to carry out the tests, keeping your main database untouched.
test_class.py
from django.test import TestCase
class classTest(TestCase):
fixtures = ('myfile.yaml',)
def setUp(self):
"""setup tests cases"""
# create the object you want to test here, which will use data from the fixtures
def test_function(self):
self.assertEqual(True,True)
# write your test here
You can read up more here:
Django
YAML
If you have any questions because things are unclear just ask, I'd be happy to help you out.
Maybe my solution will help someone
I used transactions to roll back to the database state that I had at the start of the test.
I use Eric Cousineau's decorator function to parametrizing tests
More about database transactions at django documentation page
import functools
from django.db import transaction
from django.test import TransactionTestCase
from django.contrib.auth import get_user_model
User = get_user_model()
def sub_test(param_list):
"""Decorates a test case to run it as a set of subtests."""
def decorator(f):
#functools.wraps(f)
def wrapped(self):
for param in param_list:
with self.subTest(**param):
f(self, **param)
return wrapped
return decorator
class MyTestCase(TransactionTestCase):
#sub_test([
dict(email="new#user.com", password='12345678'),
dict(email="new#user.com", password='password'),
])
def test_passwords(self, email, password):
# open a transaction
with transaction.atomic():
# Creates a new savepoint. Returns the savepoint ID (sid).
sid = transaction.savepoint()
# create user and check, if there only one with this email in DB
user = User.objects.create(email=email, password=password)
self.assertEqual(User.objects.filter(email=user.email).count(), 1)
# Rolls back the transaction to savepoint sid.
transaction.savepoint_rollback(sid)
I've been meaning to log all the users that visit the site to a file.
Using Flask for the backend.
I have not been able to get python to write to the file. Tried keeping exception handling to catch any errors that might be generated while writing. No exceptions are being raised.
Here is the part of the blueprint that should write to file.
from .UserDataCache import UserDataCache
udc = UserDataCache()
#main.route('/')
def index():
s = Suggestion.query.all()
udc.writeUsertoFile()
return render_template('suggestions.html', suggestions = s)
Here is the UserDataCache class:
from flask import request
from datetime import datetime
class UserDataCache():
def __init__(self):
pass
def writeUsertoFile(self):
try:
with open("userData.txt","a") as f:
f.write(str(datetime.now()) + " " + request.remote_addr + " " + request.url + " " + request.headers.get('User-Agent') + "\n")
except IOError,e:
print e
return
I recommend using an absolute path and verifying the permissions on that file. Something like /tmp/UserData.txt or another absolute path should work. The web server's user is what needs the permission to write to the file (www-data if you're using apache2 with Ubuntu, or check your web server's conf file to verify).
As far as why you're not seeing the exception you're catching, I see you're using print. If you're calling the app using a web browser, you'll need to send the error to something else, like a log file or flash it to the browser, or raise an error so it gets logged in the web server error log.
Is your python file name begins with uppercase? If so, try to modify it into lowercase.
I just came into the same problem and copied the exactly same code into two .py file. The only difference is their file name, one being 'Flask_test.py' and another being 'flask_for_test.py'. It's weird that 'Flask_test.py' works just fine except it cannot write into any file and 'flask_for_test.py' works perfectly.
I don't know whether the format of the file name has an effect on the function of python but using lowercase file name works for me.
By the way, all other solutions I found didn't work.
I am trying to use gdata within a Django app to create a directory in my google drive account. This is the code written within my Django view:
def root(request):
from req_info import email, password
from gdata.docs.service import DocsService
print "Creating folder........"
folder_name = '2015-Q1'
service_client = DocsService(source='spreadsheet create')
service_client.ClientLogin(email, password)
folder = service_client.CreateFolder(folder_name)
Authentication occurs without issue, but that last line of code triggers the following error:
Request Method: GET
Request URL: http://127.0.0.1:8000/
Django Version: 1.7.7
Exception Type: RequestError
Exception Value: {'status': 501, 'body': 'POST method does not support concurrency', 'reason': 'Not Implemented'}
I am using the following software:
Python 2.7.8
Django 1.7.7
PyCharm 4.0.5
gdata 2.0.18
google-api-python-client 1.4.0 (not sure if relevant)
[many other packages that I'm not sure are relevant]
What's frustrating is that the exact same code (see below) functions perfectly when I run it in its own, standalone file (not within a Django view).
from req_info import email, password
from gdata.docs.service import DocsService
print "Creating folder........"
folder_name = '2015-Q1'
service_client = DocsService(source='spreadsheet create')
service_client.ClientLogin(email, password)
folder = service_client.CreateFolder(folder_name)
I run this working code in the same virtual environment and the same PyCharm project as the code that produced the error. I have tried putting the code within a function in a separate file, and then having the Django view call that function, but the error persists.
I would like to get this code working within my Django app.
I don't recall if I got this to work within a Django view, but because Google has since required the use of Oauth 2.0, I had to rework this code anyways. I think the error had something to do with my simultaneous use of two different packages/clients to access Google Drive.
Here is how I ended up creating the folder using the google-api-python-client package:
from google_api import get_drive_service_obj, get_file_key_if_exists, insert_folder
def create_ss():
drive_client, credentials = get_drive_service_obj()
# creating folder if it does not exist
folder = get_file_key_if_exists(drive_client, 'foldername')
if folder: # if folder exists
print 'Folder "' + folder_name + '" already exists.'
else: # if folder doesn't exist
print 'Creating folder........"' + folder_name + '".'
folder = insert_folder(drive_client, folder_name)
After this code, I used a forked version (currently beta) of sheetsync to copy my template spreadsheet and populate the new file with my data. I then had to import sheetsync after the code above to avoid the "concurrency" error. (I could post the code involving sheetsync here too if folks want, but for now, I don't want to get too far off topic.)
How can I add Members folder for my functional tests in plone.app.testing so that it is findable as in real site?
Have have set member area creation flag in my product installation step which I'm testing.
membership.memberareaCreationFlag = 1
I need to get this test working:
class TestMemberFolder(unittest.TestCase):
layer = MY_FUNCTIONAL_TESTING
def setUp(self):
portal = self.portal = self.layer['portal']
def test_members_folder(self):
membership = getToolByName(self.portal, 'portal_membership')
membership.addMember("basicuser", "secret", ["Member"], [])
transaction.commit()
login(self.portal, "basicuser")
# This works just fine, because it was set by my product
self.assertEquals(membership.memberareaCreationFlag, 1,
"memberareaCreationFlag must be 1 when it is enabled")
members_folder = membership.getMembersFolder()
# But this fails
self.assertIsNotNone(members_folder)
# Also we should have the user folder here
self.assertTrue(members_folder.hasObject('basicuser'))
I specifically need Member folder functionality. Just a folder owned by the test user does not cut it.
Also I tried creating new user with acl_users.userFolderAddUser, but that does not help neighter.
The memberareaCreationFlag works just fine in live Plone site.
I finally figured it out.
At first membership.memberareaCreationFlag = 1 is not enough for enabling member folders.
It must be enabled with SecurityControlPanelAdapter in plone.app.controlpanel.security
from plone.app.controlpanel.security import ISecuritySchema
# Fetch the adapter
security_adapter = ISecuritySchema(portal)
security_adapter.set_enable_user_folders(True)
Also the Functional testing fixture does not create the member folder automatically, but is possible to install it manually in your fixture class
class YourPloneFixture(PloneSandboxLayer):
defaultBases = (PLONE_FIXTURE,)
def setUpZope(self, app, configurationContext):
# Required by Products.CMFPlone:plone-content
z2.installProduct(app, 'Products.PythonScripts')
def setUpPloneSite(self, portal):
# Installs all the Plone stuff. Workflows etc.
self.applyProfile(portal, 'Products.CMFPlone:plone')
# Install portal content. Including the Members folder!
self.applyProfile(portal, 'Products.CMFPlone:plone-content')
Finally as Member folders are created uppon user login, but the login helper function in plone.app.testing seem to be too low level for this. Login with zope.testbrowser seems to do the trick
browser = Browser(self.layer['app'])
browser.open(self.portal.absolute_url() + '/login_form')
browser.getControl(name='__ac_name').value = TEST_USER_NAME
browser.getControl(name='__ac_password').value = TEST_USER_PASSWOR
browser.getControl(name='submit').click()
Phew.
self.assert_ isn't a testing method, use something like self.assertTrue, or self.assertIsNotNone.
To add members folders just turn on member folder creation and add a new user.
Something like
def setUpPloneSite(self, portal):
# Install into Plone site using portal_setup
quickInstallProduct(portal, 'Products.DataGridField')
quickInstallProduct(portal, 'Products.ATVocabularyManager')
quickInstallProduct(portal, 'Products.MasterSelectWidget')
if HAVE_LP:
quickInstallProduct(portal, 'Products.LinguaPlone')
applyProfile(portal, 'vs.org:default')
portal.acl_users.userFolderAddUser('god', 'dummy', ['Manager'], [])
setRoles(portal, 'god', ['Manager'])
login(portal, 'god')
is perfectly working for us.
I want to generate for my users temporary download link.
Is that ok if i use django to generate link using url patterns?
Could it be correct way to do that. Because can happen that I don't understand some processes how it works. And it will overflow my memory or something else. Some kind of example or tools will be appreciated. Some nginx, apache modules probably?
So, what i wanna to achieve is to make url pattern which depend on user and time. Decript it end return in view a file.
A simple scheme might be to use a hash digest of username and timestamp:
from datetime import datetime
from hashlib import sha1
user = 'bob'
time = datetime.now().isoformat()
plain = user + '\0' + time
token = sha1(plain)
print token.hexdigest()
"1e2c5078bd0de12a79d1a49255a9bff9737aa4a4"
Next you store that token in a memcache with an expiration time. This way any of your webservers can reach it and the token will auto-expire. Finally add a Django url handler for '^download/.+' where the controller just looks up that token in the memcache to determine if the token is valid. You can even store the filename to be downloaded as the token's value in memcache.
Yes it would be ok to allow django to generate the urls. This being exclusive from handling the urls, with urls.py. Typically you don't want django to handle the serving of files see the static file docs[1] about this, so get the notion of using url patterns out of your head.
What you might want to do is generate a random key using a hash, like md5/sha1. Store the file and the key, datetime it's added in the database, create the download directory in a root directory that's available from your webserver like apache or nginx... suggest nginx), Since it's temporary, you'll want to add a cron job that checks if the time since the url was generated has expired, cleans up the file and removes the db entry. This should be a django command for manage.py
Please note this is example code written just for this and not tested! It may not work the way you were planning on achieving this goal, but it works. If you want the dl to be pw protected also, then look into httpbasic auth. you can generate and remove entries on the fly in a httpd.auth file using htpasswd and the subprocess module when you create the link or at registration time.
import hashlib, random, datetime, os, shutil
# model to hold link info. has these fields: key (charfield), filepath (filepathfield)
# datetime (datetimefield), url (charfield), orgpath (filepathfield of the orignal path
# or a foreignkey to the files model.
from models import MyDlLink
# settings.py for the app
from myapp import settings as myapp_settings
# full path and name of file to dl.
def genUrl(filepath):
# create a onetime salt for randomness
salt = ''.join(['{0}'.format(random.randrange(10) for i in range(10)])
key = hashlib('{0}{1}'.format(salt, filepath).hexdigest()
newpath = os.path.join(myapp_settings.DL_ROOT, key)
shutil.copy2(fname, newpath)
newlink = MyDlink()
newlink.key = key
newlink.date = datetime.datetime.now()
newlink.orgpath = filepath
newlink.newpath = newpath
newlink.url = "{0}/{1}/{2}".format(myapp_settings.DL_URL, key, os.path.basename(fname))
newlink.save()
return newlink
# in commands
def check_url_expired():
maxage = datetime.timedelta(days=7)
now = datetime.datetime.now()
for link in MyDlink.objects.all():
if(now - link.date) > maxage:
os.path.remove(link.newpath)
link.delete()
[1] http://docs.djangoproject.com/en/1.2/howto/static-files/
It sounds like you are suggesting using some kind of dynamic url conf.
Why not forget your concerns by simplifying and setting up a single url that captures a large encoded string that depends on user/time?
(r'^download/(?P<encrypted_id>(.*)/$', 'download_file'), # use your own regexp
def download_file(request, encrypted_id):
decrypted = decrypt(encrypted_id)
_file = get_file(decrypted)
return _file
A lot of sites just use a get param too.
www.example.com/download_file/?09248903483o8a908423028a0df8032
If you are concerned about performance, look at the answers in this post: Having Django serve downloadable files
Where the use of the apache x-sendfile module is highlighted.
Another alternative is to simply redirect to the static file served by whatever means from django.