I need to populate my database with a bunch of dummy entries (around 200+) so that I can test the admin interface I've made and I was wondering if there was a better way to do it. I spent the better part of my day yesterday trying to fill it in by hand (i.e by wrapping stuff like this my_model(title="asdfasdf", field2="laksdj"...) in a bunch of "for x in range(0,200):" loops) and gave up because it didn't work the way I expected it to. I think this is what I need to use, but don't you need to have (existing) data in the database for this to work?
Check this app
https://github.com/aerosol/django-dilla/
Let's say you wrote your blog application (oh yeah, your favorite!) in Django. Unit tests went fine, and everything runs extremely fast, even those ORM-generated ultra-long queries. You've added several categorized posts and it's still stable as a rock. You're quite sure the app is efficient and ready to for live deployment. Right? Wrong.
You can use fixtures for this purpose, and the loaddata management command.
One approach is to do it like this.
Prepare your test database.
Use dumpdata to create JSON export of the database.
Put this in the fixtures directory of your application.
Write your unit tests to load this "fixture": https://docs.djangoproject.com/en/2.2/topics/testing/tools/#django.test.TransactionTestCase.fixtures
Django fixtures provide a mechanism for importing data on syncdb. However, doing this initial data propagation is often easier via Python code. The technique you outline should work, either via syncdb or a management command. For instance, via syncdb, in my_app/management.py:
def init_data(sender, **kwargs):
for i in range(1000):
MyModel(number=i).save()
signals.post_syncdb.connect(init_data)
Or, in a management command in myapp/management/commands/my_command.py:
from django.core.management.base import BaseCommand, CommandError
from models import MyModel
class MyCommand(BaseCommand):
def handle(self, *args, **options):
if len(args) > 0:
raise CommandError('need exactly zero arguments')
for i in range(1000):
MyModel(number=i).save()
You can then export this data to a fixture, or continue importing using the management command. If you choose to continue to use the syncdb signal, you'll want to conditionally run the init_data function to prevent the data getting imported on subsequent syncdb calls. When a fixture isn't sufficient, I personally like to do both: create a management command to import data, but have the first syncdb invocation do the import automatically. That way, deployment is more automated but I can still easily make modifications to the initial data and re-run the import.
I'm not sure why you require any serialization. As long as you have setup your Django settings.py file to point to your test database, populating a test database should be nothing more than saving models.
for x in range(0, 200):
m = my_model(title=random_title(), field2=random_string(), ...)
m.save()
There are better ways to do this, but if you want a quick test set, this is the way to go.
The app recommended by the accepted answer is no longer being maintained however django-seed can be used as a replacement:
https://github.com/brobin/django-seed
I would recommend django-autofixtures to you. I tried both django_seed and django-autofixtures, but django_seed has a lot of issues with unique keys.
django-autofixtures takes care of unique, primary and other db constraints while filling up the database
Related
Basically I have a program which scraps some data from a website, I need to either print it out to a django template or to REST API without using a database. How do I do this without a database?
Your best bet is to
a.) Perform the scraping in views themselves, and pass the info in a context dict to the template
or
b.) Write to a file and have your view pull info from the file.
Django can be run without a database, but it depends on what applications you enable. Some of the default functionality (auth, sites, contenttypes) requires a database. So you'd need to disable those. If you need to use them, you're SOL.
Other functionality (like sessions) usually uses a database, but you can configure it to use a cache or file or something else.
I've taken two approaches in the past:
1) Disable the database completely and disable the applications that require the database:
DATABASES = {}
2) Use a dummy sqlite database just so it works out of box with the default apps without too much tweaking, but don't really use it for anything. I find this method faster and good for setting up quick testing/prototyping.
And to actually get the data from the scraper into your view, you can take a number of approaches. Store the data in a cache, or just write it directly to your context variables, etc.
How would you simulate the running of multiple test servers in a Django unittest?
I have a common code-base that powers two different sites with a separate database and settings.py file. I'm trying to unittest a tool that imports data from one site to another.
The default unittest setup supports multiple sqlite3 databases, so I can confirm the data on one site gets into the other's database, but I'm not sure how to confirm the data in the UI on the other site, since the default client only seems to support a single server/port.
You probably no longer need an answer to this question
I think for your specific case you don't need to have multiple servers, just multiple tests - first testing that the output from the exporting server meets your assumption, and second testing that this assumed output is successfully imported into the database. In Django unit tests the database will be empty at the start of each test
You can generate large datasets in Django using Django Fixtures. Worth noting as you should consider whether you can use their existing tools for the export/import
I have a file with a bunch of data common between several projects. The data needs to be loaded into the Django database. The file doesn't change that much, so loading it once on server start is sufficient. Since the file is shared between multiple projects, I do not have full control over the format, so I cannot convert this into a fixture or something.
I tried loading it in ready(), but then I run into a problem when creating a new database or migrating an existing database, since apparently ready() is called before migrations are complete and I get errors from using models that do not have underlying tables. I tried to set it in class_prepared signal handler, but the loading process uses more than one model, so I cannot really be sure all required model classes are prepared. Also it seems that ready() is not called when running tests, so unit tests fail because the data is missing. What is the right place to do something like this?
It seems that what I am looking for doesn't exist. Django trusts the user to deal with migrations and such and doesn't check the database on load. So there is no place in the system where you can load some data on system start and be sure that you can actually load it. What I ended up doing is loading the data in ready(), but do a sanity check first by doing MyModel.objects.exist() in a try: except: block and returning if there was an exception. This is not ideal, but I haven't found any other way.
I have a situation that requires me to use a list of databases. One database for each tenant. This information is kept in a default database. So my DATABASE_APPS_MAPPING is currently hardcoded with the list of databases that I fetch manually from the default database and save it in the settings.py file.
But every time we upgrade our service, I need to manually copy over the database values into DATABASE_APPS_MAPPING. This is getting to be a pain and I'd like to build this set using code once when django initialises. Maybe have a module dbinit.py
list_of_db=[]
def initialize(defaultDbName):
#Build up list_of_db
Then in settings.py, I can just do :-
import dbinit
.
.
.
DATABASE_APPS_MAPPING=dbinit.list_of_db
My specific question is where do I call dbinit.initialize() ? Is that urls.py, wsgi.py. Thanks.
I believe I have the answer here. Spent a couple of days referring to other stack overflow queries.
The code of dbinit.py needs to be like so:-
list_of_db=[]
initialized=False
def initialize(defaultDbName):
if not initialized:
build_up_list(list_of_db,defaultDbName)#Build the list.
initialize("mydefaultDbContainingInfoOnOtherDBs")
Since any imports by the python interpreter are performed while holding on to the import lock, we are assured that only one thread will be executing the code in the above module's initialize method. So even if the above module is imported multiple times in different threads` execution context, we are assured that the initialization has occurred only once.
I have created a django app. Now i want to test the application's performance with some 5000 data. Is there any method to do it ? Is there any method to randomly enter this data into db and run the application ? I believe there should be some method rather than typing 5000 data manually into db. The database i use is mysql. I am quite new to Python and Django, so please help me to solve this . Thanks in advance.
Check out django-dilla. It helps you to spam your database with random data
There's no real good documentation, but you'll need to install it and add it to your INSTALLED_APPS, then run python manage.py run_dilla from the command line.
To view the options run python manage.py run_dilla --help which let you specify counts and the apps to "spam". Check out the command source for implementation details.
pycheesecake.org provides a good source of testing tools for python.
You might also want to checkout django-autofixture, a nice tool for inserting randomly generated data into the database, and django-mockups, which is a fork from django-autofixture.
Here is a sample usage of django-autofixture:
django-admin.py loadtestdata [options] app.Model:# [app.Model:# ...]
You just supply the app name, model and the number of objects that you want to create.