I'm trying to learn Peewee and Bottle by making a book note-taking application.
Say I have the following entities:
Subject
Book
Chapter
Note
Tag
I would like to be able to make Notes for Chapters, Books, and Subjects.
In a DB, you would do:
create table noteable (
noteable_id INT AUTO_INCREMENT PRIMARY KEY
,type VARCHAR(10) NOT NULL CHECK (type in ('SUBJECT','BOOK','CHAPTER','NOTE'))
);
create table subject (
subject_id INT AUTO_INCREMENT PRIMARY KEY
,noteable_id INT UNIQUE REFERENCES noteable (noteable_id)
,...
);
create table book (
book_id INT AUTO_INCREMENT PRIMARY KEY
,subject_id INT NOT NULL REFERENCES subject (subject_id)
,noteable_id INT UNIQUE REFERENCES noteable (noteable_id)
,...
);
create table chapter(
chapter_id INT AUTO_INCREMENT PRIMARY KEY
,book_id INT NOT NULL REFERENCES book (book_id)
,noteable_id INT UNIQUE REFERENCES noteable(noteable_id)
,...
);
create table note(
note_id INT AUTO_INCREMENT PRIMARY KEY
,noteable_id INT UNIQUE REFERENCES noteable(noteable_id)
,...
);
(If you wanted a M:N relationship between note and notable, you would do a note_notable bridge table as well).
You would have before insert triggers on subject, book, and chapter that would insert a row into noteable, retrieve the new row's noteable_id, and use that on the incoming row.
I'm assuming that if you are using an ORM like Peewee you would want to do that in application logic rather than triggers.
How can I implement this model in Peewee?
Here is how I did it. I couldn't find a native way in Peewee to implement inheritance so I just rolled it myslef. If there is a better way, please provide your answer and I'll award it.
import MySQLdb
import peewee
from peewee import *
from datetime import datetime
db = MySQLDatabase('test', user='root',passwd='psswd')
class BaseModel(Model):
class Meta:
database = db
class Noteable(BaseModel):
type = CharField(null = False)
# This will act as the trigger that inserts a row into noteable,
# and retrieves the notable.id to use
class N(BaseModel):
def save(self, *args, **kwargs):
if not self.id:
noteable = Noteable(type=self.__class__.__name__.upper())
noteable.save()
self.noteable = noteable.id
return super(N, self).save(*args, **kwargs)
class Subject(N):
name = CharField(null = False, unique = True)
noteable = ForeignKeyField(Noteable, related_name="noteablesubject", null= False, unique = True)
class Book(N):
name = CharField(null = False, unique = True)
subject = ForeignKeyField(Subject, related_name="books", null = False)
noteable = ForeignKeyField(Noteable, related_name="noteablebook", null= False, unique = True)
class Chapter(N):
name = CharField(null = False)
chapter_number = IntegerField(null = False)
book = ForeignKeyField(Book, related_name="chapters")
noteable = ForeignKeyField(Noteable, related_name="noteablechapter", null= False, unique = True)
class Note(BaseModel):
note = TextField(null = False)
# N.B. unique is not true, as multiple notes can go to the same subject/book/chapter
noteable = ForeignKeyField(Noteable, related_name="notes", null= False)
Note.drop_table(True)
Chapter.drop_table(True)
Book.drop_table(True)
Subject.drop_table(True)
Noteable.drop_table(True)
Noteable.create_table(True)
Subject.create_table(True)
Book.create_table(True)
Chapter.create_table(True)
Note.create_table(True)
s = Subject(name="subject")
s.save()
n = Note(note="subject notes", noteable = s.noteable)
n.save()
n = Note(note="subject notes 2", noteable = s.noteable)
n.save()
b = Book(name="book", subject=s)
b.save()
n = Note(note="book notes", noteable = b.noteable)
n.save()
n = Note(note="book notes 2", noteable = b.noteable)
n.save()
c = Chapter(chapter_number=1, name="chapter", book=b)
c.save()
n = Note(note="chapter notes", noteable=c.noteable)
n.save()
n = Note(note="chapter notes 2", noteable=c.noteable)
n.save()
(if you wished to have a many to many relationship between notes and notable, you would have to define a NoteNotable class with foreign keys and remove the FK from Note)
You can define a helper method to left join whichever class with notes:
def get_notes(clazz, id):
return clazz.select().join(Noteable).join(Note, JOIN_LEFT_OUTER).where(clazz.id = id)
You can iterate over it like:
% for note in chapter.noteable.notes:
% end
Here are the results from a SELECT * FROM NOTABLE;
+----+---------+
| id | type |
+----+---------+
| 1 | SUBJECT |
| 2 | BOOK |
| 3 | CHAPTER |
+----+---------+
Here are the results from a SELECT * FROM NOTE;
+----+-----------------+-------------+
| id | note | noteable_id |
+----+-----------------+-------------+
| 1 | subject notes | 1 |
| 2 | subject notes 2 | 1 |
| 3 | book notes | 2 |
| 4 | book notes 2 | 2 |
| 5 | chapter notes | 3 |
| 6 | chapter notes 2 | 3 |
+----+-----------------+-------------+
Related
after an python3 manager.py inspectdb (mysql) and a look on few helps and tuto, i still have somes mistakes and incorrect results.
models.py
class A(models.Model):
ida = models.AutoField(db_column='idA', primary_key=True)
col1 = #an another column
has_B = models.ManyToManyField(B, related_name='a', through="AHasB", through_fields=('a_ida', 'b_idb'))
#I had add this line after a tuto in django book for the manytomayfield
class B(models.Model):
idb = models.AutoField(db_column='idB', primary_key=True)
col1 = #an another column
class AHasB(models.Model):
a_ida = models.ForeignKey(A)
b_idb = models.ForeignKey(B)
col1 = #an another column
view.py
def myview(request):
for element in b.filter(idb__in=a.values('has_B').distinct()):
print(element)
In my database i have,
A :
ida | col1
1 | ...
2 | ...
3 | ...
B :
idb | col1
1 | ...
AHasB :
a_ida | b_idb
1 | 1
But when i will display result (ida -> idb) like a classic (SELECT idb,ida FROM A, B, AHasB WHERE AHasB.a_ida=A.ida AND AHasB.b_idb=B.idb), i have this ...
1 -> 1
2 -> 1
3 -> 1
And in normal case, i will just have 1 -> 1.
Maybe the model dont fit with my real database in back.
Edit
view.py
def myview(request):
a = A.objects.All()
b = B.objects.All()
for element_a in a.filter("somefilters"):
in_has_b = set(AHasB.objects.values_list('b_idb', flat=True));
print(b.filter(idb__in=in_has_b))
If you want to select all B instances that are referenced / owned by each A instance:
bs_for_each_a = {}
for a in A.objects.all():
bs = AHasB.objects.filter(a_ida=a).values('b_idb')
bs_for_each_a[a] = bs
If you need distinct Bs, you can try to add .distinct() after values('b_idb') but I did not test it. Another way to remove duplicates would be to use values_list('b_idb', flat=True) and pass the result of the query in set():
for a in A.objects.all():
bs = AHasB.objects.filter(a_ida=a).values_list('b_idb', flat=True)
bs_for_each_a[a] = set(bs)
I am implementing search in my project what I want is to concat to column in where clause to get results from table.
Here is what I am doing:
from django.db.models import Q
if 'search[value]' in request.POST and len(request.POST['search[value]']) >= 3:
search_value = request.POST['search[value]'].strip()
q.extend([
Q(id__icontains=request.POST['search[value]']) |
(Q(created_by__first_name=request.POST['search[value]']) & Q(created_for=None)) |
Q(created_for__first_name=request.POST['search[value]']) |
(Q(created_by__last_name=request.POST['search[value]']) & Q(created_for=None)) |
Q(created_for__last_name=request.POST['search[value]']) |
(Q(created_by__email__icontains=search_value) & Q(created_for=None)) |
Q(created_for__email__icontains=search_value) |
Q(ticket_category=request.POST['search[value]']) |
Q(status__icontains=request.POST['search[value]']) |
Q(issue_type__icontains=request.POST['search[value]']) |
Q(title__icontains=request.POST['search[value]']) |
Q(assigned_to__first_name__icontains=request.POST['search[value]']) |
])
Now I want to add another OR condition like:
CONCAT(' ', created_by__first_name, created_by__last_name) like '%'search_value'%'
But when I add this condition to the queryset it becomes AND
where = ["CONCAT_WS(' ', profiles_userprofile.first_name, profiles_userprofile.last_name) like '"+request.POST['search[value]']+"' "]
tickets = Ticket.objects.get_active(u, page_type).filter(*q).extra(where=where).exclude(*exq).order_by(*order_dash)[cur:cur_length]
How do I convert this into an OR condition?
Advanced filters can be solved by
Q() object and
Query expressions like Func(), Value() and F().
The only used trick is a
Custom Lookup "rhs_only" that uses the right-hand-side of the lookup and ignores the left side, because it is easier to use all concatenated fields directly on the right side. A memorable function concat_like encapsulates that all to be easily used in queries.
from django.db.models import F, Func, Lookup, Q, Value
from django.db.models.fields import Field
def concat_like(columns, pattern):
"""Lookup filter: CONCAT_WS(' ', column_0, column_1...) LIKE pattern"""
lhs = '%s__rhs_only' % columns[0]
expr = Func(*(F(x) for x in columns), template="CONCAT_WS(' ', %(expressions)s)")
return Q(**{lhs: Like(expr, Value(pattern))})
class Like(Func):
def as_sql(self, compiler, connection):
arg_sql, arg_params = zip(*[compiler.compile(x) for x in self.source_expressions])
return ("%s LIKE '%s'" % tuple(arg_sql)), arg_params[0] + arg_params[1]
#Field.register_lookup
class RhsOnly(Lookup):
"""Skip the LHS and evaluate the boolean RHS only"""
lookup_name = 'rhs_only'
def as_sql(self, compiler, connection):
return self.process_rhs(compiler, connection)
All boolean expression and related objects are supported by this code. All arguments are correctly escaped.
Example usage:
>>> qs = MyModel.objects.filter(Q(id=1) | concat_like(('first_name', 'surname'), 'searched'))
>>> str(qs.query) # sql output simplified here
"SELECT .. WHERE id=1 OR (CONCAT_WS(' ', first_name, surname) LIKE 'searched')"
Relevant documentation:
https://docs.djangoproject.com/en/1.11/ref/models/expressions/#func-expressions
See also... Value() and F()
https://docs.djangoproject.com/en/1.11/topics/db/aggregation/#aggregations-and-other-queryset-clauses
You can reference annotated fields from inside the filter method. As such, you can filter against two concatenated fields and add it as another OR condition like this:
from django.db.models import F, Func, Value
# Because we added user_full_name as an annotation below,
# we can refer to it in the filters
q.extend([
Q(id__icontains=request.POST['search[value]']) |
(Q(created_by__first_name=request.POST['search[value]']) & Q(created_for=None)) |
Q(created_for__first_name=request.POST['search[value]']) |
(Q(created_by__last_name=request.POST['search[value]']) & Q(created_for=None)) |
Q(created_for__last_name=request.POST['search[value]']) |
(Q(created_by__email__icontains=search_value) & Q(created_for=None)) |
Q(created_for__email__icontains=search_value) |
Q(ticket_category=request.POST['search[value]']) |
Q(status__icontains=request.POST['search[value]']) |
Q(issue_type__icontains=request.POST['search[value]']) |
Q(title__icontains=request.POST['search[value]']) |
Q(assigned_to__first_name__icontains=request.POST['search[value]']) |
Q(user_full_name__icontains=request.POST['search[value]']) # <------
])
# Add the annotation to your queryset
# I'm not actually sure what the related_name or field_name for your user
# profiles are, so I'm pretending that tickets have a profile foreignkey field
# to where the first_name and last_name fields are
user_full_name_expr = Func(Value(' '), F('profile__first_name'), F('profile__last_name'), function='CONCAT_WS')
# The next two lines can be combined as long as the annotation comes first.
tickets = Ticket.objects.annotate(user_full_name=user_full_name_expr)
tickets = tickets.get_active(u, page_type).filter(*q).exclude(*exq).order_by(*order_dash)[cur:cur_length]
For fun, here's a working example based on the User model.
from django.contrib.auth.models import User
from django.db.models import F, Func, Value
User.objects.create(username='john', first_name='John', last_name='Jingleheimer-Schmidt')
User.objects.create(username='mike', first_name='Michael', last_name='Finnigan')
foo = User.objects.annotate(full_name=Func(Value(' '), F('first_name'), F('last_name'), function='CONCAT_WS'))
print(foo.filter(full_name__icontains='john'))
# outputs: [<User: john>]
What you need is, for create search FullText. I recommend use (http://haystacksearch.org/)
See documentation of Django (https://docs.djangoproject.com/en/1.11/ref/contrib/postgres/search/)
I have a model with the following definition
class exam_questions(models.Model):
exam_name=models.ForeignKey(exam,on_delete=models.CASCADE)
question=models.ForeignKey(questions,on_delete=models.CASCADE)
class Meta:
db_table = 'examquestions'
unique_together = (("exam_name", "question"),)
def __str__(self):
return '%s - %s' % (self.exam_name, self.question)
The data on sql table will look like this
+----+----------------+-------------+
| id | exam_name | question |
+----+----------------+-------------+
| 2 | test2 | 29 |
| 3 | test1 | 41 |
| 6 | test2 | 40 |
| 7 | test1 | 42 |
+----+----------------+-------------+
On Django admin I am looking the model objects like the following:
test2-29
test1-41
test2-40
test1-42
Now I want to group questions of same test and want to look them like the below:
test2-29,40
test1-41,42
I tried using normal python string operations, none of them worked on amdin django instead gave me errors.
Is there way for doing this. Any help is greatly appreciated.
Thanks
You can override objects manager in this way :-
class ExamManager(models.Manager):
def get_queryset(self):
return super(ExamManager,self).get_queryset().group_by('exam_name')
class exam_questions(models.Model):
exam_name=models.ForeignKey(exam,on_delete=models.CASCADE)
question=models.ForeignKey(questions,on_delete=models.CASCADE)
objects = ExamManager()
I have a Django view that uses one form multiple times. The form is saving relationship Subgroup id as a foreign key and Student id as a foreign key .
The problem I'm having is when I try to save information to database it only saves the last record.
For example (database model):
1 858 | Pump | Iron
2 78 | Madagaskar| Thomas
And if Im trying to split them into seperate groups, only Madagaskar his data is saved:
id | timestamp | student_Id_id | subgroup_Id_id |
+----+----------------------------+---------------+----------------+
| 62 | 2016-05-06 10:54:49.022000 | 2 | 91 |
The form looks like this:
class ApplicationFormaFull1(MultiModelForm):
form_classes = {
'sub1': FormSubgroup,
'sub2': FormSubgroup,
'stud_sub': FormStudent_in_Subgroup
}
and my view :
sub = form['sub1'].save(commit=False)
sub.student_group = StudentGroup.objects.get(id=element)
sub.number = 1
sub.type = 'Other'
sub.student_count = firstSubgroup
sub.save()
sub1 = form['sub2'].save(commit=False)
sub1.student_group = StudentGroup.objects.get(id=element)
sub1.number = 2
sub1.type = 'Others'
sub1.student_count = secondSubgroup
sub1.save()
if (counter%2==1):
stud_sub = form['stud_sub'].save(commit=True)
stud_sub.subgroup_Id = sub
stud_sub.student_Id = Student.objects.get(id=student)
stud_sub.save()
else:
stud_sub = form['stud_sub'].save(commit=True)
stud_sub.subgroup_Id = sub1
stud_sub.student_Id = Student.objects.get(id=student)
stud_sub.save()
So to sum up, I want that every form would save its information multiple times (dynamically)
Maybe the solution is that I should store information in the list and after all forms are added, save them one by one ?
stud_sub = form['stud_sub'].save(commit=False)
stud_sub.subgroup_Id = sub
stud_sub.student_Id = Student.objects.get(id=student)
list.add(stud_sub)
...
for i in list:
i.save()
Other solution use formset:
ArticleFormSet = formset_factory(ArticleForm, extra=2)
formset = ArticleFormSet(initial=[
{'title': 'Django is now open source',
'pub_date': datetime.date.today(),}
])
However i dont know how to change title, pub_date and to add everyting to formset dynimically.
I have two Python classes Note and Link mapping to PostgresQL tables. Note has a foreign-key reference to Link, while Link points back to the node through a piece of JSON text. Links point to other things besides Notes but that doesn't matter here.
Note
+------+------------------+---------+
| ID | NAME | NOTE_ID |
+------+------------------+---------+
| 1 | Alice | 5 |
| 2 | Bob | 20 |
| 3 | Carla | 6 |
+------+------------------+---------+
Link
+------+--------------+
| ID | CONTENT |
+------+--------------+
| ... | ... |
| 5 | {"t":1} |
| 6 | {"t":3} |
| ... | ... |
| 20 | {"t":2} |
+------+--------------+
Now what I would like is that whenever I create a new Note
note = Note('Danielle')
it would automatically enter the row
(4, 'Danielle', 21)
into Note, AND enter
(21, '{"t":4}')
into Link. Here's what I have tried so far: I create the Note object and THEN try to create the Link in the #events.after_insert event:
class Note(Entity):
name = Field(Unicode)
link = ManyToOne('Link')
. . .
#events.after_insert
def create_link(self):
"""
Create and persist the short link for this note. Must be done
in this after_insert event because the link table has a foreign
key that points back to the note. We need the note to be
already inserted so we can use its id.
"""
self.link = Link.build_link_for_note(self)
elixir.session.flush()
print("NOTE %s GOT LINK %s" % (self, self.link))
In the Link class I have
class Link(Entity):
. . .
#classmethod
def build_link_for_note(cls, note):
return Link(content='{"id": %d}' % note.id)
Both tables have autoincremented primary keys, so no worries there. The error that I get with this code is:
File ".../sqlalchemy/orm/session.py", line 1469, in flush
raise sa_exc.InvalidRequestError("Session is already flushing")
InvalidRequestError: Session is already flushing
I'll buy that. The #after_insert event gets called (I think) after the Note got stored to the database, which happened during the current session flush. And of course if I remove the elixir.session.flush() call, then of course it prints
NOTE <Note id:4 name:Danielle> GOT LINK <id:None content:{"t": 4}>
which again makes sense since I haven't been able to persist the link!
So my question is, how can I, create both a Note and a Link in a single request, so that the mutually dependent ids are available and properly recorded?
P.S. I understand that the schema here is a little unusal, and that I can solve this issue by either (1) spawning a task to create the Link asynchronously or (2) making the Link.content method create the link lazily. These solutions require some concurrency attention, so I am really hoping that a simple, direct SQLAlchemy solution with one session can work.
I'd advise against using Elixir's methods such as "save()" which mis-uses SQLAlchemy's API. Here is the aforementioned approach using standard SQLAlchemy events. Everything is achieved in one flush as well.
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import event
import json
Base = declarative_base()
class Note(Base):
__tablename__ = "note"
id = Column(Integer, primary_key=True)
name = Column(String)
note_id = Column(Integer, ForeignKey('link.id'))
link = relationship("Link")
# if using __init__ here is distasteful to you,
# feel free to use the "init" event illustrated
# below instead
def __init__(self, name):
self.name = name
self.link = Link()
class Link(Base):
__tablename__ = "link"
id = Column(Integer, primary_key=True)
content = Column(String)
# using an event instead of Note.__init__
##event.listens_for(Note, "init")
#def init(target, args, kwargs):
# target.link = Link()
#event.listens_for(Note, "after_insert")
def after_insert(mapper, connection, target):
connection.execute(
Link.__table__.update().\
values(content=json.dumps({"t": target.id}))
)
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e)
note = Note('Danielle')
s.add(note)
s.commit()
note = s.query(Note).first()
assert s.query(Link.content).scalar() == ('{"t": %d}' % note.id)
Since both objects have autogenerated IDs that come from the DB and want to store each other's IDs, you need to save both objects first, then save one of the objects once more, with the updated ID of other object.
So I'd go with removing the flush call and maybe calling save explicitly for each of the objects involved.