DB TABLE
select * from AAA;
id | Name | Class | Grade |
--------------------------------------
1 | john | 1 | A |
2 | Jane | 2 | B |
3 | Joon | 2 | A |
4 | Josh | 3 | C |
|
Code
Django
search_result = AAA.objects.filter(Grade = 'B').count()
print search_result
search_result -> 2
I want to change Grade to Class by VALUE.
Django
target_filter = 'Class'
search_result = AAA.objects.filter(__"target_filter..."__ = '3').count()
search_result -> 1
Q) How can I complete this code? Is it possible?
Maybe you can do it like this:
target_filter = 'Class'
filter_args = {target_filter: 3}
search_result = AAA.objects.filter(**filter_args).count()
you can shorten aeby's example by putting the kwargs in directly
target_filter = 'Class'
search_result = AAA.objects.filter(**{target_filter:3}).count()
It is somehow the same answer I gave here. You can also use the getattr but now with the module object itself. This is either __module__ if it is the same module, or the module you imported.
target_filter = 'Class'
search_result = AAA.objects.filter(getattr(__module__, target_filter) = '3').count()
EDIT: I got it wrong, it is not possible to access the current module via __module__. If the class is declared in the same module as your search, you can use globals()[target_filter] to access it. If your search is from another module, you can do it like this:
import somemodule
...
target_filter = 'Class'
search_result = AAA.objects.filter(getattr(somemodule, target_filter) = '3').count()
Related
I have been playing with PETL and seeing if I could extract multiple xml files and combine them into one.
I have no control over the structure of the XML files, Here are the variations I am seeing and which is giving my trouble.
XML File 1 Example:
<?xml version="1.0" encoding="utf-8"?>
<Export>
<Info>
<Name>John Doe</Name>
<Date>01/01/2021</Date>
</Info>
<App>
<Description></Description>
<Type>Two</Type>
<Details>
<DetailOne>1</DetailOne>
<DetailTwo>2</DetailTwo>
</Details>
<Details>
<DetailOne>10</DetailOne>
<DetailTwo>11</DetailTwo>
</Details>
</App>
</Export>
XML File 2 Example:
<?xml version="1.0" encoding="utf-8"?>
<Export>
<Info>
<Name></Name>
<Date>01/02/2021</Date>
</Info>
<App>
<Description>Sample description here.</Description>
<Type>One</Type>
<Details>
<DetailOne>1</DetailOne>
<DetailTwo>2</DetailTwo>
<DetailOne>3</DetailOne>
<DetailTwo>4</DetailTwo>
</Details>
<Details>
<DetailOne>10</DetailOne>
<DetailTwo>11</DetailTwo>
</Details>
</App>
</Export>
My python code is just scanning the subfolder xmlfiles and then trying to use PETL to parse from there. With the structure of the documents, I am loading three tables so far:
1 to hold the Info name and date
2 to hold the description and type
3 to collect the details
import petl as etl
import os
from lxml import etree
for filename in os.listdir(os.getcwd() + '.\\xmlfiles\\'):
if filename.endswith('.xml'):
# Get the info children
table1 = etl.fromxml((os.getcwd() + '.\\xmlfiles\\' + filename), 'Info', {
'Name': 'Name',
'Date': 'Date'
})
# Get the App children
table2 = etl.fromxml((os.getcwd() + '.\\xmlfiles\\' + filename), 'App', {
'Description': 'Description',
'Type': 'Type'
})
# Get the App Details children
table3 = etl.fromxml((os.getcwd() + '.\\xmlfiles\\' + filename), 'App/Details', {
'DetailOne': 'DetailOne',
'DetailTwo': 'DetailTwo'
})
# concat
c = etl.crossjoin(table1, table2, table3)
# I want the filename added on
result = etl.addfield(c, 'FileName', filename)
print('Results:\n', result)
I concat the three tables because I want the Info and App data on each line with each detail. This works until I get a XML file that has multiples of the DetailOne and DetailTwo elements.
What I am getting as results is:
Results:
+------------+----------+-------------+------+-----------+-----------+----------+
| Date | Name | Description | Type | DetailOne | DetailTwo | FileName |
+============+==========+=============+======+===========+===========+==========+
| 01/01/2021 | John Doe | None | Two | 1 | 2 | one.xml |
+------------+----------+-------------+------+-----------+-----------+----------+
| 01/01/2021 | John Doe | None | Two | 10 | 11 | one.xml |
+------------+----------+-------------+------+-----------+-----------+----------+
Results:
+------------+------+--------------------------+------+------------+------------+----------+
| Date | Name | Description | Type | DetailOne | DetailTwo | FileName |
+============+======+==========================+======+============+============+==========+
| 01/02/2021 | None | Sample description here. | One | ('1', '3') | ('2', '4') | two.xml |
+------------+------+--------------------------+------+------------+------------+----------+
| 01/02/2021 | None | Sample description here. | One | 10 | 11 | two.xml |
+------------+------+--------------------------+------+------------+------------+----------+
The second file showing DetailOne being ('1','3') and DetailTwo being ('2', '4') is not what I want.
What I want is:
+------------+------+--------------------------+------+------------+------------+----------+
| Date | Name | Description | Type | DetailOne | DetailTwo | FileName |
+============+======+==========================+======+============+============+==========+
| 01/02/2021 | None | Sample description here. | One | 1 | 2 | two.xml |
+------------+------+--------------------------+------+------------+------------+----------+
| 01/02/2021 | None | Sample description here. | One | 3 | 4 | two.xml |
+------------+------+--------------------------+------+------------+------------+----------+
| 01/02/2021 | None | Sample description here. | One | 10 | 11 | two.xml |
+------------+------+--------------------------+------+------------+------------+----------+
I believe XPath may be the way to go but after researching:
https://petl.readthedocs.io/en/stable/io.html#xml-files - doesn't go in depth on lxml and petl
some light reading here:
https://www.w3schools.com/xml/xpath_syntax.asp
some more reading here:
https://lxml.de/tutorial.html
Any assistance on this is appreciated!
First, thanks for taking the time to write a good question. I'm happy to spend the time answering it.
I've never used PETL, but I did scan the docs for XML processing. I think your main problem is that the <Details> tag sometimes contains 1 pair of tags, and sometimes multiple pairs. If only there was a way to extract a flat list of the and tag values, without the enclosing tags getting in the way...
Fortunately there is. I used https://www.webtoolkitonline.com/xml-xpath-tester.html and the XPath expression //Details/DetailOne returns the list 1,3,10 when applied to your example XML.
So I suspect that something like this should work:
import petl as etl
import os
from lxml import etree
for filename in os.listdir(os.getcwd() + '.\\xmlfiles\\'):
if filename.endswith('.xml'):
# Get the info children
table1 = etl.fromxml((os.getcwd() + '.\\xmlfiles\\' + filename), 'Info', {
'Name': 'Name',
'Date': 'Date'
})
# Get the App children
table2 = etl.fromxml((os.getcwd() + '.\\xmlfiles\\' + filename), 'App', {
'Description': 'Description',
'Type': 'Type'
})
# Get the App Details children
table3 = etl.fromxml((os.getcwd() + '.\\xmlfiles\\' + filename), '/App', {
'DetailOne': '//DetailOne',
'DetailTwo': '//DetailTwo'
})
# concat
c = etl.crossjoin(table1, table2, table3)
# I want the filename added on
result = etl.addfield(c, 'FileName', filename)
print('Results:\n', result)
The leading // may be redundant. It is XPath syntax for 'at any level in the document'. I don't know how PETL processes the XPath so I'm trying to play safe. I agree btw - the documentation is rather light on details.
I'm trying to use the twint module to get some information from twitter, in particular the bio. The code example works just fine:
import twint
c = twint.Config()
c.Username = "twitter"
twint.run.Lookup(c)
yields
783214 | Twitter | #Twitter | Private: 0 | Verified: 1 | Bio: What’s happening?! | Location: Everywhere | Url: https://about.twitter.com/ | Joined: 20 Feb 2007 6:35 AM | Tweets: 10816 | Following: 140 | Followers: 56328970 | Likes: 5960 | Media: 1932 | Avatar: https://pbs.twimg.com/profile_images/1111729635610382336/_65QFl7B_400x400.png
Thing is, I only need the bio data. According to the site, you can use
c.Format = 'bio: {bio}'
Unfortunately, this yields
CRITICAL:root:twint.get:User:replace() argument 2 must be str, not None
I think this may be due to the following code line (from here):
output += output.replace("{bio}", u.bio)
Where the u.bio value is assigned here:
u.bio = card(ur, "bio")
The card function does the following when our type is "bio":
if _type == "bio":
try:
ret = ur.find("p", "ProfileHeaderCard-bio u-dir").text.replace("\n", " ")
except:
ret = None
I think the problem may lie in the second part, where a value is assigned to u.bio, either not even being called or returning None for some reason. Unfortunately, I do not know how to fix that or call the function.
I've had a similar problem before with a different function, twint.run.Following(c), but was able to solve it by not setting c.User_full = true
Could anyone help me out?
The format should be of the form
c.Format = "{bio}"
If you wanted multiple fields
c.Format = "{bio} | {name}"
I find you get a rate limit of 250 items before a blocker drops down and you need to wait for a few minutes for it to lift.
I have a model with the following definition
class exam_questions(models.Model):
exam_name=models.ForeignKey(exam,on_delete=models.CASCADE)
question=models.ForeignKey(questions,on_delete=models.CASCADE)
class Meta:
db_table = 'examquestions'
unique_together = (("exam_name", "question"),)
def __str__(self):
return '%s - %s' % (self.exam_name, self.question)
The data on sql table will look like this
+----+----------------+-------------+
| id | exam_name | question |
+----+----------------+-------------+
| 2 | test2 | 29 |
| 3 | test1 | 41 |
| 6 | test2 | 40 |
| 7 | test1 | 42 |
+----+----------------+-------------+
On Django admin I am looking the model objects like the following:
test2-29
test1-41
test2-40
test1-42
Now I want to group questions of same test and want to look them like the below:
test2-29,40
test1-41,42
I tried using normal python string operations, none of them worked on amdin django instead gave me errors.
Is there way for doing this. Any help is greatly appreciated.
Thanks
You can override objects manager in this way :-
class ExamManager(models.Manager):
def get_queryset(self):
return super(ExamManager,self).get_queryset().group_by('exam_name')
class exam_questions(models.Model):
exam_name=models.ForeignKey(exam,on_delete=models.CASCADE)
question=models.ForeignKey(questions,on_delete=models.CASCADE)
objects = ExamManager()
I have a Django view that uses one form multiple times. The form is saving relationship Subgroup id as a foreign key and Student id as a foreign key .
The problem I'm having is when I try to save information to database it only saves the last record.
For example (database model):
1 858 | Pump | Iron
2 78 | Madagaskar| Thomas
And if Im trying to split them into seperate groups, only Madagaskar his data is saved:
id | timestamp | student_Id_id | subgroup_Id_id |
+----+----------------------------+---------------+----------------+
| 62 | 2016-05-06 10:54:49.022000 | 2 | 91 |
The form looks like this:
class ApplicationFormaFull1(MultiModelForm):
form_classes = {
'sub1': FormSubgroup,
'sub2': FormSubgroup,
'stud_sub': FormStudent_in_Subgroup
}
and my view :
sub = form['sub1'].save(commit=False)
sub.student_group = StudentGroup.objects.get(id=element)
sub.number = 1
sub.type = 'Other'
sub.student_count = firstSubgroup
sub.save()
sub1 = form['sub2'].save(commit=False)
sub1.student_group = StudentGroup.objects.get(id=element)
sub1.number = 2
sub1.type = 'Others'
sub1.student_count = secondSubgroup
sub1.save()
if (counter%2==1):
stud_sub = form['stud_sub'].save(commit=True)
stud_sub.subgroup_Id = sub
stud_sub.student_Id = Student.objects.get(id=student)
stud_sub.save()
else:
stud_sub = form['stud_sub'].save(commit=True)
stud_sub.subgroup_Id = sub1
stud_sub.student_Id = Student.objects.get(id=student)
stud_sub.save()
So to sum up, I want that every form would save its information multiple times (dynamically)
Maybe the solution is that I should store information in the list and after all forms are added, save them one by one ?
stud_sub = form['stud_sub'].save(commit=False)
stud_sub.subgroup_Id = sub
stud_sub.student_Id = Student.objects.get(id=student)
list.add(stud_sub)
...
for i in list:
i.save()
Other solution use formset:
ArticleFormSet = formset_factory(ArticleForm, extra=2)
formset = ArticleFormSet(initial=[
{'title': 'Django is now open source',
'pub_date': datetime.date.today(),}
])
However i dont know how to change title, pub_date and to add everyting to formset dynimically.
I'm currently trying to parse recursive templates with pyparsing. A template can look like this:
{{Attribute
| name=attr1
| description=First attribute.}}
The template has a name (Attribute) and defines some variables (name = attr1, description = First attribute.). However, there are also templates which can contain zero or more templates:
{{Enum
| name=MyEnum
| description=Just a test enum.
| example=Not given...
| attributes={{Attribute
| name=attr1
| description=First attribute.}}
{{Attribute
| name=attr2
| description=Second attribute.}}}}
To parse these templates I came up with the following:
template = Forward()
lb = '{{'
rb = '}}'
template_name = Word(alphas)
variable = Word(alphas)
value = CharsNotIn('|{}=') | Group(ZeroOrMore(template))
member = Group(Suppress('|') + variable + Suppress('=') + value)
members = Group(OneOrMore(member))
template << Suppress(lb) + Group(template_name + members) + Suppress(rb)
This works quite well, but it does not allow me to use "|{}=" within a value which is problematic if I want to use them. E.g.:
{{Enum
| name=MyEnum
| description=Just a test enum.
| example=<python>x = 1</python>
| attributes=}}
So, how can I change my code so that it allows these characters, too? Unforunately, I have no idea how I can archieve this.
I hope someone can give me some tips!
I found what I was looking for: https://github.com/earwig/mwparserfromhell