This program is a basic encoder in python and I want to see if I can make it more efficient without changing the names of the defined variables. Can someone give me some suggestions?
def encode(pattern, filename):
f = open(filename, "rt")
contents = f.read()
f.close()
printNow(contents)
changes = pattern.split("|")
for str in changes:
printNow("Change "+ str[0] + " to " + str[1])
newMsg = ""
for char in contents:
for change in changes:
if char == change [0]:
char = change[1]
newMsg += char
f = open(filename + "encoded", "wt")
f.write(newMsg)
f.close()
f = open(filename + "encoded", "rt")
printNow(f.read())
f.close()
encode("ae|ga|s3", "C:\\Users\\Shaun\\Desktop\\Test.txt")
import string
def encode(pattern, filename):
with open(filename) as f:
contents = f.read()
s = string.maketrans(*[''.join(a) for a in zip(*pattern.split('|'))])
newMsg = contents.translate(s)
with open(filename + 'encoded', 'rt') as f:
f.write(newMsg)
Use str.translate() instead of doing all the replacements the hard way, and do it line-by-line.
First of all you need to consider the option that your algorithm is already good enough. Even if it can be optimized, if your code is part of a bigger program and it only executes during 0.1% of time, for instance, then it will be most probably useless to optimize the code, since the rest of the program will dominate the total execution time.
If you really have a problem in your code, then I would start by analyzing the complexity of your algorithm.
And finally, you could try to find some bottlenecks in your code. For that, I would profile the code with something like python's timeit.
The str.translate() method works well for character substitutions, but here's another fast way I've used that also works for multi-character substitutions:
import re
def encode(pattern, filename):
f = open(filename, "rt")
contents = f.read()
f.close()
printNow(contents)
change_dict = {}
matches = []
changes = pattern.split("|")
for str in changes:
printNow("Change "+ str[0] + " to " + str[1])
change_dict[str[0]] = str[1]
matches.append(str[0])
change_re = re.compile("|".join(re.escape(x) for x in matches))
newMsg = change_re.sub(lambda m: change_dict[m.group(0)], contents)
f = open(filename + "encoded", "wt")
f.write(newMsg)
f.close()
f = open(filename + "encoded", "rt")
printNow(f.read())
f.close()
encode("ae|ga|s3", "C:\\Users\\Shaun\\Desktop\\Test.txt")
Related
How do I do this in python?
badphrases.txt contains
Go away
Don't do that
Stop it
allphrases.txt contains
I don't know why you do that. Go away.
I was wondering what you were doing.
You seem nice
I want allphrases.txt to be clean of the lines in badphrases.txt.
It's trivial in bash
cat badfiles.txt | while read b
do
cat allphrases.txt | grep -v "$b" > tmp
cat tmp > allphrases.txt
done
Oh, you thought I hadn't looked or tried. I searched for over and hour.
Here's my code:
# Files
ttv = "/tmp/tv.dat"
tmp = "/tmp/tempfile"
bad = "/tmp/badshows"
badfiles already exists
...code right here creates ttv
# Function grep_v
def grep_v(f,str):
file = open(f, "r")
for line in file:
if line in str:
return True
return False
t = open(tmp, 'w')
tfile = open(ttv, "r")
for line in tfile:
if not grep_v(bad,line):
t.write(line)
tfile.close
t.close
os.rename(tmp, ttv)
First google how to read a file in python:
you will probably get something like this: How do I read a file line-by-line into a list?
Use this to read both the files in lists
with open('badphrases.txt') as f:
content = f.readlines()
badphrases = [x.strip() for x in content]
with open('allphrases.txt') as f:
content = f.readlines()
allphrases = [x.strip() for x in content]
Now you have both the content in lists.
Iterate over allphrases and check if phrases from badphrases are present in it.
At this point you might consider google :
how to iterate over a list python
how to check if string present in another string python
Take the code from those places and built a brute-force algo like this:
for line in allphrases:
flag = True
for badphrase in badphrases:
if badphrase in line:
flag = False
break
if flag:
print(line)
If you can understand this code then you will notice you need to replace print with output to file:
Now google how to print to file python.
Then think about how to improve the algorithm. All the best.
UPDATE:
#COLDSPEED suggested you can simple google
- how to replace lines in a file in python:
You might get something like this: Search and replace a line in a file in Python
Which also works.
Solution not too bad.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import feedparser, os, re
# Files
h = os.environ['HOME']
ttv = h + "/WEB/Shows/tv.dat"
old = h + "/WEB/Shows/old.dat"
moo = h + "/WEB/Shows/moo.dat"
tmp = h + "/WEB/Shows/tempfile"
bad = h + "/WEB/Shows/badshows"
# Function not_present
def not_present(f,str):
file = open(f, "r")
for line in file:
if str in line:
return False
return True
# Sources (shortened)
sources = ['http://predb.me/?cats=tv&rss=1']
# Grab all the feeds and put them into ttv and old
k = open(old, 'a')
f = open(ttv, 'a')
for h in sources:
d = feedparser.parse(h)
for post in d.entries:
if not_present(old,post.link):
f.write(post.title + "|" + post.link + "\n")
k.write(post.title + "|" + post.link + "\n")
f.close
k.close
# Remove shows without [Ss][0-9] and put them in moo
m = open(moo, 'a')
t = open(tmp, 'w')
file = open(ttv, "r")
for line in file:
if re.search(r's[0-9]', line, re.I) is None:
m.write(line)
# print("moo", line)
else:
t.write(line)
# print("tmp", line)
t.close
m.close
os.rename(tmp, ttv)
# Remove badshows
t = open(tmp, 'w')
with open(bad) as f:
content = f.readlines()
bap = [x.strip() for x in content]
with open(ttv) as f:
content = f.readlines()
all = [x.strip() for x in content]
for line in all:
flag = True
for b in bap:
if b in line:
flag = False
break
if flag:
t.write(line + "\n")
t.close
os.rename(tmp, ttv)
Is there another way of checking if something is first?
I've been using for i,f in enumerate(read_files) where I enumerate a list of files, and use an if statement to check if i==0. I'm curious is there is a different (better, faster, less typed) way to do this?
read_files = glob.glob("post_stats_*.tsv")
with open("result.tsv", "w") as outfile:
for i,f in enumerate(read_files):
with open(f, "r") as infile:
metric_name = (f.strip(".tsv").split("_")[2])
if i == 0:
outfile.write(metric_name.upper() + "\n" + infile.read())
else:
outfile.write("\n" + metric_name.upper() + "\n" + infile.read())
Since it seems the only use of the if is to avoid a blank line at the start of the output file, how about putting the blank line after the file's contents? That will lead to a blank line at the end of the file where it's unlikely to hurt:
read_files = glob.glob("post_stats_*.tsv")
with open("result.tsv", "w") as outfile:
for f in read_files:
with open(f, "r") as infile:
metric_name = (f.strip(".tsv").split("_")[2])
outfile.write(metric_name.upper() + "\n" + infile.read() + "\n")
Im trying to write different things onto a text file in a while loop but it only writes it once. I want to write something to unmigrated.txt
import urllib.request
import json
Txtfile = input("Name of the TXT file: ")
fw = open(Txtfile + ".txt", "r")
red = fw.read()
blue = red.split("\n")
i=0
while i<len(blue):
try:
url = "https://api.mojang.com/users/profiles/minecraft/" + blue[i]
rawdata = urllib.request.urlopen(url)
newrawdata = rawdata.read()
jsondata = json.loads(newrawdata.decode('utf-8'))
results = jsondata['id']
url_uuid = "https://sessionserver.mojang.com/session/minecraft/profile/" + results
rawdata_uuid = urllib.request.urlopen(url_uuid)
newrawdata_uuid = rawdata_uuid.read()
jsondata_uuid = json.loads(newrawdata_uuid.decode('utf-8'))
try:
results = jsondata_uuid['legacy']
print (blue[i] + " is " + "Unmigrated")
wf = open("unmigrated.txt", "w")
wring = wf.write(blue[i] + " is " + "Unmigrated\n")
except:
print(blue[i] + " is " + "Migrated")
except:
print(blue[i] + " is " + "Not-Premium")
i+=1
You keep overwriting opening the file with w inside the loop so you only see the last data that was written to the file, either open the file once outside the loop or open with a to append. Opening once would be the simplest approach, you can also use range instead of your while or better again just iterate over the list:
with open("unmigrated.txt", "w") as f: # with close your file automatically
for ele in blue:
.....
Also wring = wf.write(blue[i] + " is " + "Unmigrated\n") sets wring to None which is what write returns so probably not of any real use.
Lastly using a blank expect is usually never a good idea, catch the specific exceptions you expect and log or at least print when you get an error.
Using the requests library, I would break up your code doing something like:
import requests
def get_json(url):
try:
rawdata = requests.get(url)
return rawdata.json()
except requests.exceptions.RequestException as e:
print(e)
except ValueError as e:
print(e)
return {}
txt_file = input("Name of the TXT file: ")
with open(txt_file + ".txt") as fw, open("unmigrated.txt", "w") as f: # with close your file automatically
for line in map(str.rstrip, fw): # remove newlines
url = "https://api.mojang.com/users/profiles/minecraft/{}".format(line)
results = get_json(url).get("id")
if not results:
continue
url_uuid = "https://sessionserver.mojang.com/session/minecraft/profile/{}".format(results)
results = get_json(url_uuid).get('legacy')
print("{} is Unmigrated".format(line))
f.write("{} is Unmigrated\n".format(line))
I am not sure where 'legacy' fits into the code, that logic I will leave to you. You can also iterate directly over the file object so you can forget about splitting the lines into blue.
try:
with open("filename", "w") as f:
f.write("your content")
But that will overwrite all contents of the file.
Instead, if you want to append to the file use:
with open("filename", "a") as f:
If you choose to not use the with syntax, remember to close the file.
Read more here:
https://docs.python.org/2/library/functions.html#open
can anyone tell me what is wrong with my code?
def count_letters(in_file, out_file):
in_file = open("in_file.txt", "r")
for line in in_file:
for x in sorted(line):
count = x.count()
out_file = open("out_file.txt", "w")
print(x, " ", count)
in_file.close()
out_file.close()
print(out_file)
it's supposed to
Takes two filenames (in_file and out_file)as arguments
Opens and reads the input file specified by in_file, and counts the number of occurrences of each letter (in a case-insensitive manner)
Writes the result in the output file specified by out_file
when I run it, it tells me that "in_file" is not defined, but I have defined it, and made an "in_file.txt."
any help would be appreciated!
You want pass "in_file.txt" as in_file variable and use it like:
in_f=open(in_file,"r")
similarly for out_file otherwise, what is the point of making a function?
But i think your error relates of not having "in_file.txt" in your current working directory, you can check with
import os
print os.getcwd()
this will show what files code can see
in_file = open(in_file, "r") # you must do the way so got an error
try this function:
def count_letters(in_file, out_file):
in_file_fd = open(in_file, "r") # open in file
out_file_fd = open(out_file, "w") # open out file
for line in in_file_fd:
for x in sorted(line):
count = line.count(x)
res = x + " " + str(count) + '\n'
print (res)
out_file_fd.write(res) # write the data into out file
in_file_fd.close() # close in_file
out_file_fd.close() # close out_file
count_letters('test.py', 'out_file')
It works for me, hope helpful.
I've a file entitled 'users.txt' with the following structure; username:info_about_the_user.
Something like this:
users.txt:
mark:stuffabouthim anthony:stuffabouthim peter:stuffabouthim peterpeter:stuffabouthim peterpeterpeter:stuffabouthim peterpeterpeterpeter:stuffabouthim
The following part of the script needs to change a line (change info about an user) but I'm having problems when the string is duplicated (peter - peterpeter) and I dont know how to fix it.
def test():
fn = 'users.txt'
f = open(fn)
output = []
changeuser = 'peterpeter'
userinfo = 'HeIsTall'
for line in f:
if not changeuser+":" in line:
output.append(line)
f.close()
f = open(fn, 'w')
f.writelines(output)
f.close()
f = open("users.txt", "a")
f.write(changeuser + ":" + userinfo+"\n")
f = open("users.txt", "a")
test()
This is the input I have:
Input: users.txt:
mark:stuffabouthim anthony:stuffabouthim peter:stuffabouthim peterpeter:HesAwesome peterpeterpeter:stuffabouthim peterpeterpeterpeter:stuffabouthim
I want to change info about peterpeter and have the following output:
Output I want to have: users.txt:
mark:stuffabouthim anthony:stuff about him peter:stuffabouthim peterpeter:HeIsTall peterpeterpeter:stuffabouthim peterpeterpeterpeter:stuffabouthim
But this is the input I'm having. All the lines behind peterpeter are getting deleted among other things.
mark:stuffabouthim
anthony:stuffabouthim
peter:stuffabouthim
peterpeter:HeIsTall
Can anyone give me a help with the code below to have the desired output? Thanks.
You can have it the easy way with the fileinput module:
import fileinput
def test():
fn = 'users.txt'
changeuser = 'peterpeter'
newinfo = 'HeIsTall'
for line in fileinput.input(fn, inplace=1):
user, oldinfo = line.split(':')
print '%s:%s' % (user, newinfo if user == changeuser else oldinfo.replace('\n', ''))
if __name__ == "__main__":
test()
try this:
def test():
fn = 'users.txt.txt'
f = open(fn)
output = []
changeuser = 'peterpeter'
userinfo = 'HeIsTall'
for line in f:
if line.strip().split(':')[0]!=changeuser:
output.append(line)
else:
output.append(changeuser + ":" + userinfo+"\n")
f.close()
f = open(fn, 'w')
f.writelines(output)
f.close()
test()
output:
mark:stuffabouthim
anthony:stuffabouthim
peter:stuffabouthim
peterpeter:HeIsTall
peterpeterpeter:stuffabouthim
peterpeterpeterpeter:stuffabouthim
You got a logical error in the if-clause, which DELETES all peters*, the only peter remaining is the one you append to the file.
for line in f:
if not changeuser+":" in line: #THAT MEANS ALL PETERS ARE IGNORED!
output.append(line)
It's generaly easier to understand positive clauses then a negation:
for line in f:
if changeuser+":" in line:
output.append('%s:%s\n' %(changeuser,userinfo))
else:
output.append(line)
Good code is easy to read. Try to code like you would try to write a report! That leads automatically to spliting your code into smaller pieces like functions. e.g.:
lines = read_all_lines_from_file(filename)
change_user_info(lines, user, userinfo)
save_lines_to_file(lines, filename)
Your code gets split into smaller pieces and if an error occurs you can pin it down to a few lines of code instead of having to work over several pages. ;-)