I am in a situation where I would have to insert multiple records into a postgre db through an ajax call, based on a foreignkey.
Currently I am using db1.db2_set.create(...) for each record, looping over a list of dictionaries.
Is this the best way to do it? It seems like I'm hitting the database for every insert.
I'm pretty sure that django will query database when you call save() method. So if you do something like:
for i in objects:
db1.db2_set.create(i)
db1.save()
It may acess database only once. But still, this might be usefull:
http://djangosnippets.org/snippets/766/
It's a middleware that you can add to see how many querys your django application uses in the every page you are accessing.
Related
I have a huge amount of data in my db.
I cannot use .delete() method cause performance of Django ORM is insufficient in my case.
_raw_delete() method suits me cause I can do it python instead using raw SQL.
But I have problem I have no idead how can I delete relation tables using _raw_delete. They need to be deleted before models cause I have restrict in DB. Any ideas how can I achieve this?
I have found a solution.
You can operate on link model with this:
link_model = MyModel._meta.get_field('my_m2m_field').remote_field.through
qs = link_model.objects.filter(mymodel_id__in=mymodel_ids)
qs._raw_delete(qs.db)
I'm displaying the results from an SQLAlchemy (Flask-SQLAlchemy) query on a particular view. However the sorting/order is only set by what I originally passed into the query ( order_by(desc(SelectedTable.date_changed)) ). I'm trying to now add functionality that each column that is displayed can be selected to order the presentation.
Is there a way to alter the way a returned query object is sorted once it's returned to create this behavior? Or will I need to build custom queries for each possible column that could be sorted by and ascending/descending?
Is there a recipe for implementing something like this? I've tried google, here, the Flask, Flask-SQLAlchemy, and SQLAlchemy docs for something along these lines but haven't seen anything that touches on the subject and beginning to think that I'm going to need to use custom queries or without new queries try some JavaScript in the Jinja Template to achieve this.
Thanks!
I am kind of new to python and django.
I am using bulk_create to insert a lot of rows and as a former DBA I would very much like to see what insert statments are being executed. I know that for querys you can use .query but for insert statments I can't find a command.
Is there something I'm missing or is there no easy way to see it? (A regular print is fine by me.)
The easiest way is to set DEBUG = True and check connection.queries after executing the query. This stores the raw queries and the time each query takes.
from django.db import connection
MyModel.objects.bulk_create(...)
print(connection.queries[-1]['sql'])
There's more information in the docs.
A great tool to make this information easily accessible is the django-debug-toolbar.
Can you take form data and change database schema? Is it a good idea? Is there a downside to many migrations from a 'default' database?
I want users to be able to add / remove tables, columns, and rows. Making schema changes requires migrations, so adding in that functionality would require writing a view that takes form data and inserts it into a function that then uses Flask-Migrate.
If I manage to build this, don't migrations build the required separate scripts and everything that goes along with that each time something is added or removed? Is that practical for something like this, where 10 or 20 new tables might be added to the starting database?
If I allow users to add columns to a table, it will have to modify the table's class. Is that possible, or a safe idea? If not, I'd appreciate it if someone could help me out, and at least get me pointed in the right direction.
In a typical web application, the deployed database does not change its schema at runtime. The schema is only changed during an upgrade, and only the developers make these changes. Operations that users perform on the application can add, remove or modify rows, but not modify the tables or columns themselves.
If you need to offer your users a way to add flexible data structures, then you should design your database schema in a way that this is possible. For example, if you wanted your users to add custom key/value pairs, you could have a table with columns user_id, key_name and value. You may also want to investigate if a schema-less database fits your needs better.
I am doing a massive data conversion for data that will end up in a django managed database. For reasons of efficiency and politics, we need to fill the destination database with manually run mass INSERTS.
I would like to have my Django ORM prepare those statements, so I can write them to a file to be run later.
So I need somthing like this:
50000_or_so_Foos = [...]
sql_str = Foo.objects.bulk_create_sql(50000_or_so_Foos)
with file("pre_preped.sql", 'w') as f:
f.write(sql_str)
Then we will pass pre_preped.sql to another department and they will play it into the database.
Is there a way to do this?
Is this actually going to save us any time?
ADDED Question: Should I be creating a csv for LOADDATA instead?
(I should note that in the real world, we have more than one model and way more than 50000 objects)
I am not sure of any easy way to get the query from bulk_create, because it executes query when it is called, as opposed to somethign like filtering where you can view the querysets query property.
As I was quickly scanning source code, it looks like you can manually build a query using the sql object, the same way django does in bulk_create. https://github.com/django/django/blob/master/django/db/models/query.py#L917 can provide a blueprint on how to do that.