We Keep Coding

How to transform an affirmative sentence into a general question using Python Udapi?

I would like to trasnform some pretty simple affirmative sentences into general questions (the language of choise is Spanish). Consider the following example:
Esto es muy difícil. -> Es esto muy difícil?
So I just need to shift the position of subject and predicate (wherever they are).
Normally it can be done with the shift_before_node() method:
pron_node, aux_node = tree.descendants[0], tree.descendants[1]
aux_node.shift_before_node(pron_node)
However, if I want to automate the process (because subject and predicate will not always be in the same position) I need to create a cycle (See The Problem paragraph below) for each node of a tree, where it checks that if node's part of speech (upos) is a PRON or PROPN, and it is followed (not necessarily directly) by a node which is a VERB or AUX, it needs to shift the second node before the first one (like in the example above). But, I dont know how to implement it into cycle. Any suggestions?
Here is my code so far (done in Google Colab). I apologize for excluding some of the console text, otherwise it would be too lengthy:
Request to UDPipe server
import requests
response = requests.get("http://lindat.mff.cuni.cz/services/udpipe/api/models")
info = response.json()
info
for key, data in info["models"].items():
if "spanish" in key:
print(key, data)
params = {"tokenizer": "", "tagger": "", "parser": "", "model": "spanish-gsd-ud-2.6-200830"}
text = "Esto es muy difícil."
params["data"] = text
response = requests.get("http://lindat.mff.cuni.cz/services/udpipe/api/process", params)
json_response = response.json()
parse = json_response["result"]
print(parse)
Output #1 (print (parse)):
# generator = UDPipe 2, https://lindat.mff.cuni.cz/services/udpipe
# udpipe_model = spanish-gsd-ud-2.6-200830
# udpipe_model_licence = CC BY-NC-SA
# newdoc
# newpar
# sent_id = 1
# text = Esto es muy difícil.
1 Esto este PRON _ Number=Sing|PronType=Dem 4 nsubj _ _
2 es ser AUX _ Mood=Ind|Number=Sing|Person=3|Tense=Pres|VerbForm=Fin 4 cop _ _
3 muy mucho ADV _ _ 4 advmod _ _
4 difícil difícil ADJ _ Number=Sing 0 root _ SpaceAfter=No
5 . . PUNCT _ _ 4 punct _ SpaceAfter=No
Udapi Installation:
!pip install --upgrade git+https://github.com/udapi/udapi-python.git
import os
os.environ['PATH'] += ":" + os.path.join(os.environ['HOME'], ".local/bin")
from udapi.block.read.conllu import Conllu
from udapi.core.document import Document
from udapi.block.write.textmodetrees import TextModeTrees
from io import StringIO
Building a tree:
In my understanding a tree is a variable of a built in Udapi class, which is a structured version of a parse variable, and which contains all the information about each word of a sentence - its order (ord), given form (form), initial form (lemma), part of speech (upos) and so on:
tree = Conllu(filehandle=StringIO(parse)).read_tree()
writer = TextModeTrees(attributes="ord,form,lemma,upos,feats,deprel", layout="align")
writer.process_tree(tree)
Output #2 (writer.process_tree(tree)):
# sent_id = 1
# text = Esto es muy difícil.
─┮
│ ╭─╼ 1 Esto este PRON Number=Sing|PronType=Dem nsubj
│ ┢─╼ 2 es ser AUX Mood=Ind|Number=Sing|Person=3|Tense=Pres|VerbForm=Fin cop
│ ┢─╼ 3 muy mucho ADV _ advmod
╰─┾ 4 difícil difícil ADJ Number=Sing root
╰─╼ 5 . . PUNCT _ punct
It is also possible to print out all the dependents for each node of a given tree. As already correctly noted in the comments, tree.descendants consists of a list of nodes:
for node in tree.descendants:
print(f"{node.ord}:{node.form}")
left_children = node.children(preceding_only=True)
if len(left_children) > 0:
print("Left dependents:", end=" ")
for child in left_children:
print(f"{child.ord}:{child.form}", end=" ")
print("")
right_children = node.children(following_only=True)
if len(right_children) > 0:
print("Right dependents:", end=" ")
for child in right_children:
print(f"{child.ord}:{child.form}", end=" ")
print("")
Output #3:
1:Esto
2:es
3:muy
4:difícil
Left dependents: 1:Esto 2:es 3:muy
Right dependents: 5:.
5:.
The problem (beginning of a cycle):
for node in tree.descendants:
if node.upos == "VERB" or node.upos == "AUX":
UPDATE 1
So, I`ve come to the first somewhat complete version of a needed cycle and now it looks like this:
for i, curr_node in enumerate(nodes[1:], 1):
prev_node = nodes[i-1]
if (prev_node.upos == "PRON" or prev_node.upos == "PROPN") and (curr_node.upos == "VERB" or curr_node.upos == "AUX"):
curr_node.shift_before_node(prev_node)
But now I get this error:
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-11-a967bbd730fe> in <module>()
9
10
---> 11 for i, curr_node in enumerate(nodes[1:], 1):
12 prev_node = nodes[i-1]
13 if (prev_node.upos == "PRON" or prev_node.upos == "PROPN") and (curr_node.upos == "VERB" or curr_node.upos == "AUX"):
NameError: name 'nodes' is not defined
UPDATE 2
I tried defining nodes like that:
nodes = tree.descendants
And now my cycle compiles at least, but it still didn't do anything with the structure of a given sentence:
nodes = tree.descendants
for i, curr_node in enumerate(nodes[1:], 1):
prev_node = nodes[i-1]
if (prev_node.upos == "PRON" or prev_node.upos == "PROPN") and (curr_node.upos == "VERB" or curr_node.upos == "AUX"):
curr_node.shift_before_node(prev_node)
Checking the tree:
tree = Conllu(filehandle=StringIO(parse)).read_tree()
writer = TextModeTrees(attributes="ord,form,lemma,upos,feats,deprel", layout="align")
writer.process_tree(tree)
# sent_id = 1
# text = Esto es muy difícil.
─┮
│ ╭─╼ 1 Esto este PRON Number=Sing|PronType=Dem nsubj
│ ┢─╼ 2 es ser AUX Mood=Ind|Number=Sing|Person=3|Tense=Pres|VerbForm=Fin cop
│ ┢─╼ 3 muy mucho ADV _ advmod
╰─┾ 4 difícil difícil ADJ Number=Sing root
╰─╼ 5 . . PUNCT _
Nothing changed.
UPDATE 3
I've also tried to check if the cycle swaps subject and predicate back again (2nd time), making the sentence look like the original one, but I guess it's not the case, becuase even if I comment the break part, flag has increased by 1 only:
nodes = tree.descendants
flag = 1
for i, curr_node in enumerate(nodes[1:], 1):
prev_node = nodes[i-1]
if ((prev_node.upos == "PRON") or (prev_node.upos == "PROPN")) and ((curr_node.upos == "VERB") or (curr_node.upos == "AUX")):
curr_node.shift_before_node(prev_node)
flag = flag + 1
# if flag == 2:
# break
print(flag)
Output
2
HOWEVER, it means, that the condition if ((prev_node.upos == "PRON") or (prev_node.upos == "PROPN")) and ((curr_node.upos == "VERB") or (curr_node.upos == "AUX")) was satisified.

Suppose there is one sentence per line in affirm.txt with affirmative Spanish sentences such as "Esto es muy difícil." or "Tus padres compraron esa casa de la que me hablaste.".
As an alternative to using the UDPipe web service, we can parse the sentences locally (I slightly prefer the es_ancora model over es_gsd):
import udapi
doc = udapi.Document('affirm.txt')
udapi.create_block('udpipe.Base', model_alias='es_ancora').apply_on_document(doc)
To make repeated experiments faster, we can now store the parsed trees to a CoNLL-U file using doc.store_conllu('affirm.conllu') and later load it using doc = udapi.Document('affirm.conllu').
To draw the trees we can use the doc.draw() method (or even tree.draw()), which is a syntactic sugar that uses TextModeTrees() behind the scenes. So to compare the sentences before and after changing the word order, we can use:
print("Original word order:")
doc.draw() # or doc.draw(attributes="ord,form,lemma,deprel,feats,misc")
for tree in doc.trees:
process_tree(tree)
print("Question-like word order:")
doc.draw()
Now comes the main work - to implement the process_tree() subroutine. Note that
We need to change the word order of the main clause only (e.g "Tus padres compraron esa casa."), not any dependent clauses (e.g. "de la que me hablaste".) So we don't want to iterate over all nodes (tree.descendants), we just need to find the main predicate (usually a verb) and its subject.
The subject does not need to be only PRON and PROPN, it can be NOUN or maybe just ADJ if the governing noun is omitted. So it is safer to just ask for deprel=nsubj (handling csubj is beyond the scope of this question).
I don't speak Spanish, but I think the rule is not as simple as moving the verb before the subject (or moving the subject after the verb). At least, we need to distinguish transitive verbs (with objects) and copula constructions. Of course, even the solution below is not perfect. It is rather an example how to use Udapi.
We should handle the nasty details like capitalization and spacing.
def process_tree(tree):
# Find the main predicate and its subject
main_predicate = tree.children[0]
nsubj = next((n for n in main_predicate.children if n.udeprel == 'nsubj'), None)
if not nsubj:
return
# Move the subject
# - after the auxiliary copula verb if present
# - or after the last object if present
# - or after the main predicate (verb)
cop = next((n for n in main_predicate.children if n.udeprel == 'cop'), None)
if cop:
nsubj.shift_after_subtree(cop)
else:
objects = [n for n in main_predicate.children if n.udeprel in ('obj', 'iobj')]
if objects:
nsubj.shift_after_subtree(objects[-1])
else:
nsubj.shift_after_node(verb)
# Fix the capitalization
nsubj_start = nsubj.descendants(add_self=True)[0]
if nsubj_start.lemma[0].islower() and nsubj_start.form[0].isupper():
nsubj_start.form = nsubj_start.form.lower()
tree.descendants[0].form = tree.descendants[0].form.capitalize()
# Add a question mark (instead of fullstop)
dots = [n for n in main_predicate.children if n.form == '.']
if not dots:
dots = [main_predicate.create_child(upos="PUNCT", deprel="punct")]
dots[-1].form = '?'
# Fix spacing
dots[-1].prev_node.misc["SpaceAfter"] = "No"
nsubj_start.prev_node.misc["SpaceAfter"] = ""
# Recompute the string representation of the sentence
tree.text = tree.compute_text()
The solution above uses Udapi as a library. An alternative would be to move the main code into a Udapi block called e.g. MakeQuestions:
from udapi.core.block import Block
class MakeQuestions(Block):
def process_tree(self, tree):
# the rest is same as in the solution above
If we store this block in the current directory in file makequestions.py, we can call it from the command line in many ways:
# parse the affirmative sentences
cat affirm.txt | udapy -s \
read.Sentences \
udpipe.Base model_alias=es_ancora \
> affirm.conllu
# visualize the output with TextModeTrees (-T)
cat affirm.conllu | udapy -T .MakeQuestions | less -R
# store the output in CoNLL-U
udapy -s .MakeQuestions < affirm.conllu > questions.conllu
# show just the plain-text sentences
udapy write.Sentences < questions.conllu > questions.txt
# visualize the differences in HTML
udapy -H \
read.Conllu files=affirm.conllu zone=affirm \
read.Conllu files=questions.conllu zone=questions \
util.MarkDiff gold_zone=affirm attributes=form ignore_parent=1 \
> differences.html

How can i make this script more efficient?(Python)

im currently learning Python(about 3 months expirience now) and wanted to try out the module "Mouse", ive built a simple autoclicker script(not using it in game) but for values above 12(cps) it reaches below targeted cps, i suspect this is because of the if loops in my program, could anybody help me make this more efficient?
Code here:
import mouse
import keyboard
import time
staat = 0
cps = input("Typ de hoeveelheid cps die je wilt in.")
cps = float(cps)
cps = 1 / cps
print("pauzeer de loop door op / te drukken")
while True:
if staat == 1:
mouse.click('left')
if keyboard.is_pressed('/'):
if staat == 0:
staat = 1
else:
staat = 0
time.sleep(cps)
Thanks in advance

Using 1 instead of True is slightly faster. Importing this way is slightly faster and calling without the '.' is also slightly faster. If still not fast enough I could implement multi-threading .
from mouse import click
from keyboard import is_pressed
from time import sleep,perf_counter
staat = 0
cps = input("Typ de hoeveelheid cps die je wilt in.")
cps = float(cps)
cps = 1 / cps
print("pauzeer de loop door op / te drukken")
while 1:
timeing = perf_counter()
if staat:
click('left')
if is_pressed('/'):
if not staat:
staat = 1
else:
staat = 0
sleep(cps - (perf_counter() - timeing)

Pyflakes returning erroneous syntax error

I'm writing a program that generates a random playing card and draws it in the .stdout terminal.
It keeps giving me SyntaxError: invalid syntax when I triple checked my code and everything is correct.
Input:
def drawCard():
#cs and cr are global variables I declared in another function, so please disregard those.
#also, Italic(), Bold(), etc are custom escape methods I made to format my text.
#The indexing on cr and cs is to print the first character in the string (they are strings).
cardlayout = f" ______\n|{cr[0]}| |\n|{Bold(Underline(cs[0]))}{End()}| |\n| {Italic('0' + cr)}{End()} _|\n| |{Bold(cs[0])}{End()}|\n|____{Underline('|' + cr[0])}{End()}|"
if cr == '10':
cardlayout = f" ______\n|{cr}| |\n|{Bold(Underline(cs[0]))}{End()}| |\n| {Italic(cr)}{End()} _|\n| |{Bold(cs[0])}{End()}|\n|{Underline(' |' + cr)}{End()}|"
if cr == 'Ace' or cr == 'Jack' or cr == 'Queen' or cr == 'King':
cardlayout = f" ______\n|{cr[0]}| |\n|{Bold(Underline(cs[0]))}{End()}| |\n| {Italic("a ")}{End()}_|\n| |{Bold(cs[0])}{End()}|\n|____{Underline('|J')}{End()}|"
print(cardlayout)
Output:
File "main.py", line 24
cardlayout = f" ______\n|{cr[0]}| |\n|{Bold(Underline(cs[0]))}{End()}| |\n| {Italic("a ")}{End()}_|\n| |{Bold(cs[0])}{End()}|\n|____{Underline('|J')}{End()}|"
^
SyntaxError: invalid syntax
How is my syntax invalid here?

When using os.startfile file is not found

import os # Tuo os moduuli
print("Valitse suoritettava luokka:"); #Tulosta...
print("1 = Kertolasku");
print("2 = Osamaara");
print("3 = Vertailu");
print("4 = Floydin kolmio");
print("5 = Kertotaulu");
print("6 = Viikonpaiva");
print("7 = Rivit");
v = int(input("Valitse:")) # v = seuraava numero
if v == 1: # jos v on yhtäkuin 1 niin avaa...
os.startfile(r'%USERPROFILE%\Desktop\JAVA\BATCH\x01.bat')
if v == 2:
os.startfile(r'%USERPROFILE%\Desktop\JAVA\BATCH\x02.bat')
if v == 3:
os.startfile(r'%USERPROFILE%\Desktop\JAVA\BATCH\x03.bat')
if v == 4:
os.startfile(r'%USERPROFILE%\Desktop\JAVA\BATCH\x04.bat')
if v == 5:
os.startfile(r'%USERPROFILE%\Desktop\JAVA\BATCH\x05.bat')
if v == 21:
os.startfile(r'%USERPROFILE%\Desktop\JAVA\BATCH\x06.bat') #Secret
if v == 1942:
os.startfile(r'%USERPROFILE%\Desktop\JAVA\BATCH\x07.bat') #Secret
if v == 6:
os.startfile(r'%USERPROFILE%\Desktop\JAVA\BATCH\x08.bat')
if v == 7:
os.startfile(r'%USERPROFILE%\Desktop\JAVA\BATCH\x09.bat')
else: # muutoin tulosta...
print ("VIRHE - Onko syötetty luku 1-5? Onko JAVA kansio sijainnissa %USERPROFILE%\Desktop ?");
print ("VIRHE - Onko Java asennettu? Onko OS = Windows?");
print ("VIRHE - Onko Java:n path variable asetettu oikein?");
input('Paina ENTER sammuttaaksesi...') #Lisää sulkeva komento.
This code returns the following error:
Error message
I have checked that file named x01.bat exists in:%USERPROFILE%\Desktop\JAVA\BATCH
I have tried to replace \ with / but windows doesn't recognize / as an way to access a directory.
I have tried to escape the backslash with another backslash. It only makes the same error with three backslashes instead of 2. (Yes I removed r before ' when I escaped a backslash with a backslash.)
I even tried to escape a backslash with a backslash before a backslash but error is the same with more backslashes.
Verdict: It seems that Python 3.6 on Windows 10 wants to add a backslash before a backslash no matter what I do. Because of that windows doesn't know where the .bat file is.
Any help is appreciated, please share your ideas!

For constructing paths in python, it is generally easier to use os.path.join
So something like
os.startfile(os.path.join(os.environ['USERPROFILE'], 'Desktop', 'JAVA', 'BATCH', 'x01.bat'))

Extracting numbers in text file

I have a text file which came from excel. I dont know how to take five digits after a specific character.
I want to take only five digits after #ACA in a text file.
my text is like:
ERROR_MESSAGE
(((#ACA16018)|(#ACA16019))&(#AQV71767='')&(#AQV71765='2'))?1:((#AQV71765='4')?1:((#AQV71767$'')?(((#AQV71765='1')|(#AQV71765='3'))?1:'Hasar veya Lehe Hukuk seçebilirsiniz'):'Rücu sıra numarasını yazıp Hasar veya Lehe Hukuk seçebilirsiniz'))
Rücu Oranı Girilmesi Zorunludur...'
#ACA17660
#ACA16560
#ACA15623
#ACA17804
BU ALANI BOŞ GEÇEMEZSİNİZ.EKSPER RAPORU GELMEDEN DY YE GERİ GÖNDEREMEZSİNİZ. PERT İHBARI VARSA PERT ÇALINMA OPERASYONU AKTİVİTESİ OLUŞTURULMALIDIR.
(#TSC[T008UNSMAS;FIRM_CODE=2 AND UNIT_TYPE='SG' AND UNIT_NO=#AQV71830]>0)?1:'Girdiğiniz değer fihristte yoktur'
#ACA17602
#ACA17604
#ACA56169
BU ALANI BOŞ GEÇEMEZSİNİZ
#ACA17606
#ACA17608
(#AQV71835='')?'Boş geçilemez':1
Lütfen Gönderilecek Kişinin Mail Adresini Giriniz ! '
LÜTFEN RED NEDENİNİ GİRİNİZ.
EKSİK BİLGİ / BELGE ALANINA GİRMİŞ OLDUĞUNUZ DEĞER YANLIŞ VEYA GEÇERŞİZDİR!!! LÜTFEN KONTROL EDİP TEKRAR DENEYİNİZ.'
BU ALAN BOŞ GEÇİLEMEZ. ÖDEME YAPILMADAN EK ÖDEME SÜRECİNİ BAŞLATAMAZSINIZ.
ONAYLANDI VE REDDEDİLDİ SEÇENEKLERİNİ KULLANAMAZSINIZ
BU ALAN BOŞ GEÇİLEMEZ.EVRAKLARINIZI , VARSA EKSPER RAPORUNU VE MUALLAĞI KONTROL EDİNİZ.
Muallak Tutarını kontrol ediniz.
'OTO BRANŞINDA REDDEDİLDİ NEDENİ SEÇMELİSİNİZ'
'OTODIŞI BRANŞINDA REDDEDİLDİ NEDENİ SEÇMELİSİNİZ'
(#AQV70003$'')?((#TSC[T001HASIHB;FIRM_CODE=#FP10100 AND COMPANY_CODE=2 AND CLAIM_NO=#AQV70003]$0)?1:'Bu dosya sistemde bulunmamaktadır'):'Bu alan boş geçilemez'
(#AQV70503='')?'Bu alan boş geçilemez.':((#ACA18635=1)?1:'Mağdura ait uygun kriterli ödeme kaydı mevcut değildir.')
(#AQV71809=0)?'Boş geçilemez':1
(#FD101AQV71904_AFDS<0)?'Tarih bugünün tarihinden büyük olamaz
I want to take every 5 digits which comes after #ACA, so:
16018, 16019, 17660, etc...

grep -oP '#ACA\K[0-9]{5}' file.txt
#ACA\K will match #ACA but not printed as part of output
[0-9]{5} five digits following #ACA
If variable number of digits are needed, use
grep -oP '#ACA\K[0-9]+' file.txt

If you don't know or don't like regular expressions, you can do this, although the code is a bit longer :
if __name__ == '__main__':
pattern = '#ACA'
filename = 'yourfile.txt'
res = list()
with open(filename, 'rb') as f: # open 'yourfile.txt' in byte-reading mode
for line in f: # for each line in the file
for s in line.split(pattern)[1:]: # split the line on '#ACA'
try:
nb = int(s[:5]) # take the first 5 characters after as an int
res.append(nb) # add it to the list of numbers we found
except (NameError, ValueError): # if conversion fails, that wasn't an int
pass
print res # if you want them in the same order as in the file
print sorted(res) # if you want them in ascending order

This should do it
import re
print(re.findall("#ACA(\d+)",str_var))
If you have the whole text in the variable str_var
Output:
['16018', '16019', '17660', '16560', '15623', '17804', '17602', '17604', '56169', '17606', '17608', '18635']

re.findall(r'#ACA(\d{5})', str_var)

[x[:5] for x in content.split("#ACA")[1:]]

PowerShell solution:
$contet = Get-Content -Raw 'your_file'
$match = [regex]::Matches($contet, '#ACA(\d{5})')
$match | ForEach-Object {
$_.Groups[1].Value
}
Output:
16018
16019
17660
16560
15623
17804
17602
17604
56169
17606
17608
18635