We Keep Coding

How to convert text file into json file?

I am new to python and I want to convert a text file into json file.
Here's how it looks like:
#Q Three of these animals hibernate. Which one does not?
^ Sloth
A Mouse
B Sloth
C Frog
D Snake
#Q What is the literal translation of the Greek word Embioptera, which denotes an order of insects, also known as webspinners?
^ Lively wings
A Small wings
B None of these
C Yarn knitter
D Lively wings
#Q There is a separate species of scorpions which have two tails, with a venomous sting on each tail.
^ False
A True
B False
Contd
.
.
.
.
^ means Answer.
I want it in json format as shown below.
Example:
{
"questionBank": [
{
"question": "Grand Central Terminal, Park Avenue, New York is the worlds",
"a": "largest railway station",
"b": "Longest railway station",
"c": "highest railway station",
"d": "busiest railway station",
"answer": "largest railway station"
},
{
"question": "Eritrea, which became the 182nd member of the UN in 1993, is in the continent of",
"a": "Asia",
"b": "Africa",
"c": "Europe",
"d": "Oceania",
"answer": "Africa"
}, Contd.....
]
}
I came across a few similar posts and here's what I have tried:
dataset = "file.txt"
data = []
with open(dataset) as ds:
for line in ds:
line = line.strip().split(",")
print(line)
To which the output is:
['']
['#Q What part of their body do the insects from order Archaeognatha use to spring up into the air?']
['^ Tail']
['A Antennae']
['B Front legs']
['C Hind legs']
['D Tail']
['']
['#Q What is the literal translation of the Greek word Embioptera', ' which denotes an order of insects', ' also known as webspinners?']
['^ Lively wings']
['A Small wings']
['B None of these']
['C Yarn knitter']
['D Lively wings']
['']
Contd....
The sentences containing commas are separated by python lists. I tried to use .join but didn't get the results I was expecting.
Please let me know how to approach this.

dataset = "text.txt"
question_bank = []
with open(dataset) as ds:
for i, line in enumerate(ds):
line = line.strip("\n")
if len(line) == 0:
question_bank.append(question)
question = {}
elif line.startswith("#Q"):
question = {"question": line}
elif line.startswith("^"):
question['answer'] = line.split(" ")[1]
else:
key, val = line.split(" ", 1)
question[key] = val
question_bank.append(question)
print({"questionBank":question_bank})
#for storing json file to local directory
final_output = {"questionBank":question_bank}
with open("output.json", "w") as outfile:
outfile.write(json.dumps(final_output, indent=4))

Rather than handling the lines one at a time, I went with using a regex pattern approach.
This also more reliable as it will error out if the input data is in a bad format - rather than silently ignoring a grouping which is missing a field.
PATTERN = r"""[#]Q (?P<question>.+)\n\^ (?P<answer>.+)\nA (?P<option_a>.+)\nB (?P<option_b>.+)\n(?:C (?P<option_c>.+)\n)?(?:D (?P<option_d>.+))?"""
def parse_qa_group(qa_group):
"""
Extact question, answer and 2 to 4 options from input string and return as a dict.
"""
# "group" here is a set of question, answer and options.
matches = PATTERN.search(qa_group)
# "group" here is a regex group.
question = matches.group('question')
answer = matches.group('answer')
try:
c = matches.group('option_c')
except IndexError:
c = None
try:
d = matches.group('option_d')
except IndexError:
d = None
results = {
"question": question,
"answer": answer,
"a": matches.group('option_a'),
"b": matches.group('option_b')
}
if c:
results['c'] = c
if d:
results['d'] = d
return results
# Split into groups using the blank line.
qa_groups = question_answer_str.split('\n\n')
# Process each group, building up a list of all results.
all_results = [parse_qa_group(qa_group) for qa_group in qa_groups]
print(json.dumps(all_results, indent=4))
Further details in my gist. Read more on regex Grouping
I left out reading the text and writing a JSON file.

replace trademark symbol (™) when alone

I'm trying to remove trademark symbol (™) but only in the case it's not followed by any other symbol for instance I might have ’ which is a bad encoding of quotation mark (') so I don't want to remove trademark symbol (™) and hence broking the pattern that i'm using to replace xx™ with quotation mark.
dict = {};
chars = {
'\xe2\x84\xa2': '', # ™
'\xe2\x80\x99': "'", # ’
}
def stats_change(char, number):
if dict.has_key(char):
dict[char] = dict[char]+number
else:
dict[char] = number # Add new entry
def replace_chars(match):
char = match.group(0)
stats_change(char,1)
return chars[char]
i, nmatches = re.subn("(\\" + '|\\'.join(chars.keys()) + ")", replace_chars, i)
count_matches += nmatches
Input: foo™ oof
Output: foo oof
Input: o’f oof
Output: o'f oof
Any suggestions ?

Translate unicode characters in a string to ISO 8859-1 Characters

I have the following string:
'\x7B\x22h\x22\x3A\x5B\x7B\x22id\x22\x3A\x22242611\x22,\x22minute\x22\x3A\x222\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.9359999847412109\x22,\x22Y\x22\x3A\x220.534000015258789\x22,\x22xG\x22\x3A\x220.1072189137339592\x22,\x22player\x22\x3A\x22Ari\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x222930\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22Head\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Wanderson\x22,\x22lastAction\x22\x3A\x22Chipped\x22\x7D,\x7B\x22id\x22\x3A\x22242612\x22,\x22minute\x22\x3A\x224\x22,\x22result\x22\x3A\x22SavedShot\x22,\x22X\x22\x3A\x220.8059999847412109\x22,\x22Y\x22\x3A\x220.7069999694824218\x22,\x22xG\x22\x3A\x220.021672379225492477\x22,\x22player\x22\x3A\x22Cristian\x20Ram\x5Cu00edrez\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x225477\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22LeftFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3Anull,\x22lastAction\x22\x3A\x22None\x22\x7D,\x7B\x22id\x22\x3A\x22242613\x22,\x22minute\x22\x3A\x224\x22,\x22result\x22\x3A\x22SavedShot\x22,\x22X\x22\x3A\x220.7780000305175782\x22,\x22Y\x22\x3A\x220.505\x22,\x22xG\x22\x3A\x220.023817993700504303\x22,\x22player\x22\x3A\x22Mauricio\x20Pereyra\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x222922\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Viktor\x20Claesson\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242614\x22,\x22minute\x22\x3A\x2217\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.9330000305175781\x22,\x22Y\x22\x3A\x220.41\x22,\x22xG\x22\x3A\x220.01863950863480568\x22,\x22player\x22\x3A\x22Ari\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x222930\x22,\x22situation\x22\x3A\x22FromCorner\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22Head\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Mauricio\x20Pereyra\x22,\x22lastAction\x22\x3A\x22Aerial\x22\x7D,\x7B\x22id\x22\x3A\x22242617\x22,\x22minute\x22\x3A\x2221\x22,\x22result\x22\x3A\x22SavedShot\x22,\x22X\x22\x3A\x220.710999984741211\x22,\x22Y\x22\x3A\x220.534000015258789\x22,\x22xG\x22\x3A\x220.015956614166498184\x22,\x22player\x22\x3A\x22Ivan\x20Ignatyev\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x226025\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Ari\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242621\x22,\x22minute\x22\x3A\x2231\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.7959999847412109\x22,\x22Y\x22\x3A\x220.4640000152587891\x22,\x22xG\x22\x3A\x220.03898102045059204\x22,\x22player\x22\x3A\x22Viktor\x20Claesson\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x225478\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3Anull,\x22lastAction\x22\x3A\x22None\x22\x7D,\x7B\x22id\x22\x3A\x22242622\x22,\x22minute\x22\x3A\x2236\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.759000015258789\x22,\x22Y\x22\x3A\x220.3509999847412109\x22,\x22xG\x22\x3A\x220.05237437039613724\x22,\x22player\x22\x3A\x22Mauricio\x20Pereyra\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x222922\x22,\x22situation\x22\x3A\x22DirectFreekick\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22LeftFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3Anull,\x22lastAction\x22\x3A\x22Standard\x22\x7D,\x7B\x22id\x22\x3A\x22242624\x22,\x22minute\x22\x3A\x2242\x22,\x22result\x22\x3A\x22BlockedShot\x22,\x22X\x22\x3A\x220.919000015258789\x22,\x22Y\x22\x3A\x220.37\x22,\x22xG\x22\x3A\x220.10843519121408463\x22,\x22player\x22\x3A\x22Sergei\x20Petrov\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x222920\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Viktor\x20Claesson\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242625\x22,\x22minute\x22\x3A\x2248\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.7719999694824219\x22,\x22Y\x22\x3A\x220.385\x22,\x22xG\x22\x3A\x220.023656079545617104\x22,\x22player\x22\x3A\x22Aleksandr\x20Martynovich\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x222790\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Yuri\x20Gazinskiy\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242626\x22,\x22minute\x22\x3A\x2249\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.715999984741211\x22,\x22Y\x22\x3A\x220.4879999923706055\x22,\x22xG\x22\x3A\x220.013118931092321873\x22,\x22player\x22\x3A\x22Yuri\x20Gazinskiy\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x222929\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Viktor\x20Claesson\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242627\x22,\x22minute\x22\x3A\x2254\x22,\x22result\x22\x3A\x22BlockedShot\x22,\x22X\x22\x3A\x220.909000015258789\x22,\x22Y\x22\x3A\x220.3529999923706055\x22,\x22xG\x22\x3A\x220.09400425851345062\x22,\x22player\x22\x3A\x22Magomed\x2DShapi\x20Suleymanov\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x225926\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Viktor\x20Claesson\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242628\x22,\x22minute\x22\x3A\x2254\x22,\x22result\x22\x3A\x22BlockedShot\x22,\x22X\x22\x3A\x220.8859999847412109\x22,\x22Y\x22\x3A\x220.31799999237060544\x22,\x22xG\x22\x3A\x220.061035316437482834\x22,\x22player\x22\x3A\x22Magomed\x2DShapi\x20Suleymanov\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x225926\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22LeftFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Yuri\x20Gazinskiy\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242629\x22,\x22minute\x22\x3A\x2255\x22,\x22result\x22\x3A\x22Goal\x22,\x22X\x22\x3A\x220.9269999694824219\x22,\x22Y\x22\x3A\x220.46\x22,\x22xG\x22\x3A\x220.523554801940918\x22,\x22player\x22\x3A\x22Magomed\x2DShapi\x20Suleymanov\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x225926\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Viktor\x20Claesson\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242630\x22,\x22minute\x22\x3A\x2266\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.915\x22,\x22Y\x22\x3A\x220.5420000076293945\x22,\x22xG\x22\x3A\x220.3631550371646881\x22,\x22player\x22\x3A\x22Christian\x20Cueva\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x226799\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Cristian\x20Ram\x5Cu00edrez\x22,\x22lastAction\x22\x3A\x22Cross\x22\x7D,\x7B\x22id\x22\x3A\x22242631\x22,\x22minute\x22\x3A\x2271\x22,\x22result\x22\x3A\x22BlockedShot\x22,\x22X\x22\x3A\x220.685\x22,\x22Y\x22\x3A\x220.485\x22,\x22xG\x22\x3A\x220.03188558667898178\x22,\x22player\x22\x3A\x22Cristian\x20Ram\x5Cu00edrez\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x225477\x22,\x22situation\x22\x3A\x22DirectFreekick\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22LeftFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3Anull,\x22lastAction\x22\x3A\x22Standard\x22\x7D,\x7B\x22id\x22\x3A\x22242632\x22,\x22minute\x22\x3A\x2272\x22,\x22result\x22\x3A\x22BlockedShot\x22,\x22X\x22\x3A\x220.8909999847412109\x22,\x22Y\x22\x3A\x220.4809999847412109\x22,\x22xG\x22\x3A\x220.09532035887241364\x22,\x22player\x22\x3A\x22Sergei\x20Petrov\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x222920\x22,\x22situation\x22\x3A\x22FromCorner\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3Anull,\x22lastAction\x22\x3A\x22None\x22\x7D,\x7B\x22id\x22\x3A\x22242634\x22,\x22minute\x22\x3A\x2275\x22,\x22result\x22\x3A\x22Goal\x22,\x22X\x22\x3A\x220.794000015258789\x22,\x22Y\x22\x3A\x220.47900001525878905\x22,\x22xG\x22\x3A\x220.05203503370285034\x22,\x22player\x22\x3A\x22Ivan\x20Ignatyev\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x226025\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3Anull,\x22lastAction\x22\x3A\x22None\x22\x7D,\x7B\x22id\x22\x3A\x22242635\x22,\x22minute\x22\x3A\x2277\x22,\x22result\x22\x3A\x22SavedShot\x22,\x22X\x22\x3A\x220.904000015258789\x22,\x22Y\x22\x3A\x220.5\x22,\x22xG\x22\x3A\x220.13627302646636963\x22,\x22player\x22\x3A\x22Viktor\x20Claesson\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x225478\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Christian\x20Cueva\x22,\x22lastAction\x22\x3A\x22Chipped\x22\x7D,\x7B\x22id\x22\x3A\x22242637\x22,\x22minute\x22\x3A\x2281\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.8719999694824219\x22,\x22Y\x22\x3A\x220.6840000152587891\x22,\x22xG\x22\x3A\x220.07149235904216766\x22,\x22player\x22\x3A\x22Cristian\x20Ram\x5Cu00edrez\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x225477\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Mauricio\x20Pereyra\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242638\x22,\x22minute\x22\x3A\x2282\x22,\x22result\x22\x3A\x22SavedShot\x22,\x22X\x22\x3A\x220.895\x22,\x22Y\x22\x3A\x220.6329999923706054\x22,\x22xG\x22\x3A\x220.10822572559118271\x22,\x22player\x22\x3A\x22Ivan\x20Ignatyev\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x226025\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Magomed\x2DShapi\x20Suleymanov\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242639\x22,\x22minute\x22\x3A\x2286\x22,\x22result\x22\x3A\x22BlockedShot\x22,\x22X\x22\x3A\x220.7680000305175781\x22,\x22Y\x22\x3A\x220.6519999694824219\x22,\x22xG\x22\x3A\x220.019864005967974663\x22,\x22player\x22\x3A\x22Viktor\x20Claesson\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x225478\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3Anull,\x22lastAction\x22\x3A\x22None\x22\x7D,\x7B\x22id\x22\x3A\x22242640\x22,\x22minute\x22\x3A\x2287\x22,\x22result\x22\x3A\x22BlockedShot\x22,\x22X\x22\x3A\x220.795\x22,\x22Y\x22\x3A\x220.7330000305175781\x22,\x22xG\x22\x3A\x220.01853240840137005\x22,\x22player\x22\x3A\x22Christian\x20Cueva\x22,\x22h_a\x22\x3A\x22h\x22,\x22player_id\x22\x3A\x226799\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Sergei\x20Petrov\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D\x5D,\x22a\x22\x3A\x5B\x7B\x22id\x22\x3A\x22242615\x22,\x22minute\x22\x3A\x2218\x22,\x22result\x22\x3A\x22SavedShot\x22,\x22X\x22\x3A\x220.89\x22,\x22Y\x22\x3A\x220.4279999923706055\x22,\x22xG\x22\x3A\x220.3485192060470581\x22,\x22player\x22\x3A\x22Andrei\x20Panyukov\x22,\x22h_a\x22\x3A\x22a\x22,\x22player_id\x22\x3A\x226138\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Nikolay\x20Dimitrov\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242616\x22,\x22minute\x22\x3A\x2218\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.8719999694824219\x22,\x22Y\x22\x3A\x220.4\x22,\x22xG\x22\x3A\x220.30197691917419434\x22,\x22player\x22\x3A\x22Nikolay\x20Dimitrov\x22,\x22h_a\x22\x3A\x22a\x22,\x22player_id\x22\x3A\x225496\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3Anull,\x22lastAction\x22\x3A\x22Rebound\x22\x7D,\x7B\x22id\x22\x3A\x22242618\x22,\x22minute\x22\x3A\x2222\x22,\x22result\x22\x3A\x22BlockedShot\x22,\x22X\x22\x3A\x220.86\x22,\x22Y\x22\x3A\x220.5\x22,\x22xG\x22\x3A\x220.06458419561386108\x22,\x22player\x22\x3A\x22Eric\x20Bicfalvi\x22,\x22h_a\x22\x3A\x22a\x22,\x22player_id\x22\x3A\x225483\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22LeftFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Othman\x20El\x20Kabir\x22,\x22lastAction\x22\x3A\x22Cross\x22\x7D,\x7B\x22id\x22\x3A\x22242619\x22,\x22minute\x22\x3A\x2224\x22,\x22result\x22\x3A\x22SavedShot\x22,\x22X\x22\x3A\x220.8030000305175782\x22,\x22Y\x22\x3A\x220.6179999923706054\x22,\x22xG\x22\x3A\x220.035318903625011444\x22,\x22player\x22\x3A\x22Eric\x20Bicfalvi\x22,\x22h_a\x22\x3A\x22a\x22,\x22player_id\x22\x3A\x225483\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22LeftFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3Anull,\x22lastAction\x22\x3A\x22None\x22\x7D,\x7B\x22id\x22\x3A\x22242620\x22,\x22minute\x22\x3A\x2229\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.81\x22,\x22Y\x22\x3A\x220.585\x22,\x22xG\x22\x3A\x220.04518153518438339\x22,\x22player\x22\x3A\x22Othman\x20El\x20Kabir\x22,\x22h_a\x22\x3A\x22a\x22,\x22player_id\x22\x3A\x226587\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22LeftFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Andrei\x20Panyukov\x22,\x22lastAction\x22\x3A\x22HeadPass\x22\x7D,\x7B\x22id\x22\x3A\x22242623\x22,\x22minute\x22\x3A\x2241\x22,\x22result\x22\x3A\x22SavedShot\x22,\x22X\x22\x3A\x220.7580000305175781\x22,\x22Y\x22\x3A\x220.575\x22,\x22xG\x22\x3A\x220.022277826443314552\x22,\x22player\x22\x3A\x22Othman\x20El\x20Kabir\x22,\x22h_a\x22\x3A\x22a\x22,\x22player_id\x22\x3A\x226587\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Nikolay\x20Dimitrov\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242633\x22,\x22minute\x22\x3A\x2274\x22,\x22result\x22\x3A\x22BlockedShot\x22,\x22X\x22\x3A\x220.750999984741211\x22,\x22Y\x22\x3A\x220.35700000762939454\x22,\x22xG\x22\x3A\x220.01709834299981594\x22,\x22player\x22\x3A\x22Nikolay\x20Dimitrov\x22,\x22h_a\x22\x3A\x22a\x22,\x22player_id\x22\x3A\x225496\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22LeftFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Eric\x20Bicfalvi\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D,\x7B\x22id\x22\x3A\x22242636\x22,\x22minute\x22\x3A\x2279\x22,\x22result\x22\x3A\x22BlockedShot\x22,\x22X\x22\x3A\x220.8530000305175781\x22,\x22Y\x22\x3A\x220.3920000076293945\x22,\x22xG\x22\x3A\x220.042678408324718475\x22,\x22player\x22\x3A\x22Yuri\x20Bavin\x22,\x22h_a\x22\x3A\x22a\x22,\x22player_id\x22\x3A\x223085\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3Anull,\x22lastAction\x22\x3A\x22None\x22\x7D,\x7B\x22id\x22\x3A\x22242641\x22,\x22minute\x22\x3A\x2292\x22,\x22result\x22\x3A\x22MissedShots\x22,\x22X\x22\x3A\x220.705999984741211\x22,\x22Y\x22\x3A\x220.534000015258789\x22,\x22xG\x22\x3A\x220.01331254467368126\x22,\x22player\x22\x3A\x22Eric\x20Bicfalvi\x22,\x22h_a\x22\x3A\x22a\x22,\x22player_id\x22\x3A\x225483\x22,\x22situation\x22\x3A\x22OpenPlay\x22,\x22season\x22\x3A\x222018\x22,\x22shotType\x22\x3A\x22RightFoot\x22,\x22match_id\x22\x3A\x229071\x22,\x22h_team\x22\x3A\x22FC\x20Krasnodar\x22,\x22a_team\x22\x3A\x22Ural\x22,\x22h_goals\x22\x3A\x222\x22,\x22a_goals\x22\x3A\x220\x22,\x22date\x22\x3A\x222018\x2D12\x2D02\x2011\x3A00\x3A00\x22,\x22player_assisted\x22\x3A\x22Andrey\x20Egorychev\x22,\x22lastAction\x22\x3A\x22Pass\x22\x7D\x5D\x7D'
I want to transform this string into the following JSON-object:
{"h":[{"id":"242611","minute":"2","result":"MissedShots","X":"0.9359999847412109","Y":"0.534000015258789","xG":"0.1072189137339592","player":"Ari","h_a":"h","player_id":"2930","situation":"OpenPlay","season":"2018","shotType":"Head","match_id":"9071","h_team":"FC Krasnodar","a_team":"Ural","h_goals":"2","a_goals":"0","date":"2018-12-02 11:00:00","player_assisted":"Wanderson","lastAction":"Chipped"},{"id":"242612","minute":"4","result":"SavedShot","X":"0.8059999847412109","Y":"0.7069999694824218","xG":"0.021672379225492477","player":"Cristian Ramirez","h_a":"h","player_id":"5477","situation":"OpenPlay","season":"2018","shotType":"LeftFoot","match_id":"9071","h_team":"FC Krasnodar","a_team":"Ural","h_goals":"2","a_goals":"0","date":"2018-12-02 11:00:00","player_assisted":null,"lastAction":"None"},{"id":"242613","minute":"4","result":"SavedShot","X":"0.7780000305175782","Y":"0.505","xG":"0.023817993700504303","player":"Mauricio Pereyra","h_a":"h","player_id":"2922","situation":"OpenPlay","season":"2018","shotType":"RightFoot","match_id":"9071","h_team":"FC Krasnodar","a_team":"Ural","h_goals":"2","a_goals":"0","date":"2018-12-02 11:00:00","player_assisted":"Viktor Claesson","lastAction":"Pass"},{"id":"242614","minute":"17","result":"MissedShots","X":"0.9330000305175781","Y":"0.41","xG":"0.01863950863480568","player":"Ari","h_a":"h","player_id":"2930","situation":"FromCorner","season":"2018","shotType":"Head","match_id":"9071","h_team":"FC Krasnodar","a_team":"Ural","h_goals":"2","a_goals":"0","date":"2018-12-02 11:00:00","player_assisted":"Mauricio Pereyra","lastAction":"Aerial"},{"id":"242617","minute":"21","result":"SavedShot","X":"0.710999984741211","Y":"0.534000015258789","xG":"0.015956614166498184","player":"Ivan Ignatyev","h_a":"h","player_id":"6025","situation":"OpenPlay","season":"2018","shotType":"RightFoot","match_id":"9071","h_team":"FC Krasnodar","a_team":"Ural","h_goals":"2","a_goals":"0","date":"2018-12-02 11:00:00","player_assisted":"Ari","lastAction":"Pass"},{"id":"242621","minute":"31","result":"MissedShots","X":"0.7959999847412109","Y":"0.4640000152587891","xG":"0.03898102045059204","player":"Viktor Claesson","h_a":"h","player_id":"5478","situation":"OpenPlay","season":"2018","shotType":"RightFoot","match_id":"9071","h_team":"FC Krasnodar","a_team":"Ural","h_goals":"2","a_goals":"0","date":"2018-12-02 11:00:00","player_assisted":null,"lastAction":"None"},{"id":"242622","minute":"36","result":"MissedShots","X":"0.759000015258789","Y":"0.3509999847412109","xG":"0.05237437039613724","player":"Mauricio Pereyra","h_a":"h","player_id":"2922","situation":"DirectFreekick","season":"2018","shotType":"LeftFoot","match_id":"9071","h_team":"FC Krasnodar","a_team":"Ural","h_goals":"2","a_goals":"0","date":"2018-12-02 11:00:00","player_assisted":null,"lastAction":"Standard"},{"id":"242624","minute":"42","result":"BlockedShot","X":"0.919000015258789","Y":"0.37","xG":"0.10843519121408463","player":"Sergei Petrov","h_a":"h","player_id":"2920","situation":"OpenPlay","season":"2018","shotType":"RightFoot","match_id":"9071","h_team":"FC Krasnodar","a_team":"Ural","h_goals":"2","a_goals":"0","date":"2018-12-02 11:00:00","player_assisted":"Viktor Claesson","lastAction":"Pass"},{"id":"242625","minute":"48","result":"MissedShots","X":"0.7719999694824219","Y":"0.385","xG":"0.023656079545617104","player":"Aleksandr Martynovich","h_a":"h","player_id":"2790","situation":"OpenPlay","season":"2018","shotType":"RightFoot","match_id":"9071","h_team":"FC Krasnodar","a_team":"Ural","h_goals":"2","a_goals":"0","date":"2018-12-02 11:00:00","player_assisted":"Yuri Gazinskiy","lastAction":"Pass"},{"id":"242626","minute":"49","result":"MissedShots","X":"0.715999984741211","Y":"0.4879999923706055","xG":"0.013118931092321873","player":"Yuri Gazinskiy","h_a":"h","player_id":"2929","situation":"OpenPlay","season":"2018","shotType":"RightFoot","match_id":"9071","h_team":"FC Krasnodar","a_team":"Ural"}]}
With the help of a table I what each of the following unicode-characters represents:
\x7B = {
\x22 = "
\x3A = :
\x5B = [
\x7D = }
\x5D = ]
As a result, I have written the following script:
string = '\x7B\x22h\x22\x3A\....\\x22\x22\x7D\x5D\x7D'
cleaned_string = string.replace("\x7B","{")
cleaned_string = cleaned_string.replace('\x22', '"')
cleaned_string = cleaned_string.replace("\x3A", ":")
cleaned_string = cleaned_string.replace("\x5B", "[")
cleaned_string = cleaned_string.replace("\x7D", "}")
cleaned_string = cleaned_string.replace("\x5D", "]")
s = json.loads(cleaned_string)
However, The problem arises in the player- and player-assisted -field. Especially, for French or Spanish names there are multiple special characters possible. Currently, my script is only transforming the "standard" characters. In theory, there are of course a lot of different other characters that need to be transformed. How can I do this easily?
Thanks in advance,

Errors in Python AI

I am currently programming an Artificial Intelligence in Python, with some basic code from ELIZA. I will improve on the code once I get it working. My problem is that when I run the program and enter a query to the computer, there is no response. My code is below.
import string
# OSWALD v1.0
switch = [
["I need \(.*\)",
[ "Why do you need %1?",
"Would it REALLY help you to get %1?",
"Are you sure you need %1?"]]
#There is more code with responses.
]
gPats = {
"am" : "are",
"was" : "were",
"i" : "you",
"i'd" : "you would",
"i've" : "you have",
"i'll" : "you will",
"my" : "your",
"are" : "am",
"you've": "I have",
"you'll": "I will",
"your" : "my",
"yours" : "mine",
"you" : "me",
"me" : "you",
}
s = input
gKeys = map(lambda x:regex.compile(x[0]),gPats)
gValues = map(lambda x:x[1],gPats)
print ("Hello, mortal. My name is Oswald. What would you like to talk about?")
while s == input:
try: s = input(">")
def translate(str,dict):
words = string.split(string.lower(str))
keys = dict.keys();
for i in range(0,len(words)):
if words[i] in keys:
words[i] = dict[words[i]]
return print(switch)
def respond(str,keys,values):
for i in range(0,len(keys)):
if input == input:
respnum = whrandom.randint(0,len(values[word])-1)
resp = values[i][respnum]
pos = string.find(resp,'%')
print(string.find(resp,'%'))
while pos > -1:
num = string.atoi(resp[pos+1:pos+2])
resp = resp[:pos] + \
translate(keys[i].group(num),gReflections) + \
resp[pos+2:]
pos = string.find(resp,'%')
if resp[-2:] == '?.': resp = resp[:-2] + '.'
if resp[-2:] == '??': resp = resp[:-2] + '?'
print(string.find(resp,'%'))

Counting lines from a string

I need to create a program which removes punctuation, some specific words, duplicates and return the words left and their respective lines. I also need to keep track of the duplicates. For instance,
Python IDLE
Indexer: type in lines, finish with a . at start of line only
It is a briskly blowing wind that blows
from the north, the North of my youth.
The wind is cold too, colder than the
winds of yesteryear.
.
The index is:
brisk 1
blow 1
wind 1, 3, 4
north 2
youth 2
cold 3
yesteryear 4
The Problem: I need to keep track of the line number of the words left and also their duplicates. I'm not being able to do that.
from string import *
stopWords = [ "a", "i", "it", "am", "at", "on", "in", "to", "too", "very", \
"of", "from", "here", "even", "the", "but", "and", "is", "my", \
"them", "then", "this", "that", "than", "though", "so", "are" ]
endings = [ "es" , "ed" , "er", "ly"]
punctuation = [ ".", "," , ":" , ";" , "!" , "?" , "&" , "'" ]
unindexed_sentence = raw_input("type in lines, finish with a . at start of line only").lower()
#removing duplicates.
def unique_string(l):
ulist = []
ulist2 = []
[ulist.append(x) for x in l if x not in ulist]
[ulist2.append(x)]
global ulist2
return ulist
unindexed_sentence =' '.join(unique_string(unindexed_sentence.split()))
unindexed_sentence1 = split(unindexed_sentence,"\n")
list_unindexed = []
# splitting
i = 0
while i<len(unindexed_sentence1):
list_unindexed += [split(unindexed_sentence1[i])]
i+=1
countline = 0
i = 0
while i < len(list_unindexed):
j = 0
while j < len(list_unindexed[i]):
if list_unindexed[i][j][0] in punctuation:
list_unindexed[i][j] = list_unindexed[i][j][:0]
if list_unindexed[i][j][-1] in punctuation:
list_unindexed[i][j] = list_unindexed[i][j][:-1]
if list_unindexed[i][j][-1] == "s":
list_unindexed[i][j] = list_unindexed[i][j][:-1]
if list_unindexed[i][j][-2:] in endings:
list_unindexed[i][j] = list_unindexed[i][j][:-2]
if list_unindexed[i][j][-3:] == "ing":
list_unindexed[i][j] = list_unindexed[i][j][:-3]
if list_unindexed[i][j] in stopWords:
del list_unindexed[i][j]
else:
j += 1
i += 1
countline += 1
def new_line(n):
split(n,"\n")
count = 1
if n[-1] == "\n":
count += 1
return count
string1 = str(list_unindexed)
string2 = str(string1)
string2 ='\n'.join(unique_string(string2.split()))
print string2

Is it your homework?
Here some tips:
Don't do: from string import *. You don't need it.
Use data.splitlines() to get list of lines
Use enumerate() to get a index, e.g.: for i, line in enumerate(data.splitlines())
Use a dictionary for keeping track of all words. Each value could be a list or a set of line numbers
Don't remove duplicates initially. You can do this using dictionaries or sets.