so for class we have to start out our problem doing this:
Write a function that takes as its input a filename, and an integer. The file should open the file and read in the first number of lines given as the second argument. (You'll need to have a variable to use as a counter for this part).
It's very basic and I figure a loop is needed but I can't figure out how to incorporate a loop into the question. What I've tried doesn't work and it's been about 3 hours and the best I can come up with is
def filewrite(textfile,line):
infile=open(textfile,'r',encoding='utf-8')
text=infile.readline(line)
print(text)
however that doesn't get me to what I need for the function. It's still early in my intro to python class so basic code is all we have worked with.
There are two basic looping strategies you could use here:
you could count up to n, reading lines as you go
you could read lines from file, keeping track of how many you've read, and stop when you reach a certain number.
def filewrite(textfile, n):
with open(textfile) as infile:
for _ in range(n):
print(infile.readline(), end='')
print()
def filewrite(textfile, n):
with open(textfile) as infile:
counter = 0
for line in infile:
if counter >= n:
break
print(line, end='')
counter += 1
The first is obviously more readable, and since readline will just return an empty string if it runs out of lines, it's safe to use even if the user asks for more lines than the infile has.
Here I'm also using a context manager to make sure the files are closed when I'm done with them.
Here's a version without the stuff you don't recognize
def filewrite(textfile, n):
infile = open(textfile)
count = 0
while count < n:
print(infile.readline())
count += 1
infile.close()
Related
Exporting the data:
num = 0
exportData = open("results_file.txt", "a")
while num < len(runs) - 1:
exportData.write(str(runs[num]) + "\n")
num = num + 1
exportData.close()
Importing the data into the new file:
runs = []
num = 1
count = len(open("results_file.txt").readlines( ))
print(count)
importData = open("results_file.txt", "r")
while num < count:
runs.append(importData.read(num))
print(importData.read(num))
num = num + 1
importData.close()
My goal is to export the array of integers to a file (can be something else than a txt file for all I care) and then to import them at a later date into a new file and use them there as integers (performing mathematical operations on them)
The error that I'm getting (on line 28 I'm trying to use the first number in the array for a mathematical calculation):
line 28, in if runs[num] < 7: TypeError: '<' not supported between instances of 'str' and 'int'
runs = []
num = 1
count = len(open("results_file.txt").readlines( ))
print(count)
importData = open("results_file.txt", "r")
while num < count:
runs.append(int(importData.read(num)))
print(importData.read(num))
num = num + 1
importData.close()
Adding int() returns this error:
ValueError: invalid literal for int() with base 10: '4\n1'
You're not being pythonic, and many of the answers here aren't either. So, let me clean up your code a bit.
from ast import literal_eval
with open("results_file.txt", "a") as exportData:
for run in runs:
exportData.write(str(run) + "\n")
runs = []
with open("results_file.txt", "r") as importData:
runs.extend([literal_eval(x) for x in importData])
I'll break this down line by line:
from ast import literal_eval is the safest way to parse things that are strings into python objects. It's better than using a plain old eval because it won't run arbitrary code. We'll use this function to read the data latter.
with open(...) as ... is the best way to open a file. It contains the file-object within a single scope and catches errors. Look this one up here: Pep 343
for ... in ... For loops that you're using are not pythonoic at all. The pythonic way is to use iterators no need to count lines and declare variables to keep track... the python objects keep track of themselves. (If you need a counter I highly recommend that you look up enumerate() Enumerate() in Python
exportData.write(str(run) + "\n") only change here is that with the pythonic for loop there's no need to index the runs list anymore.
runs = [] I think you know what this is, but I have to declare it out of the with statement because if the with statement throws an error, and you were to catch it, runs will be initialized.
I've already discussed with statements.
runs.extend([literal_eval(x) for x in importData]) Has two things going on. extend appends a list to a list... cool. The more interesting part here is the list comprehension. Here's a tutorial on list comprehensions. As soon as you get comfortable with the for loops, the list comprehension is the next pythonic step. For further pythonic enlightenment, this line could also be replaced with: runs.extend(map(literal_eval, importData))
That's it, 9 lines. Happy hacking.
The error you are experiencing is most likely due to the fact you're trying to add a string to an integer. Try doing
runs = []
num = 1
count = len(open("results_file.txt").readlines( ))
print(count)
importData = open("results_file.txt", "r")
while num < count:
runs.append(int(importData.read(num)))
print(importData.read(num))
num = num + 1
importData.close()
The main function/tool you're looking for is int(). For example:
>>> int('15')
15
>>> int('15') + 5
20
But you also can save yourself some real headaches by coding this differently. For example, you do not need to know the number of lines ahead of time. You can just call .readline() until it returns an empty string. Or even iterate over the file and when it ends, it with exit.
Another tip, and this is just good practice, is to use the context manager for opening files. So instead of
file = open('myfile.txt')
for line in file:
print(line)
you would do:
with open('myfile.txt') as file:
for line in file:
print(line)
The big advantage of the latter is that if will always make sure file is closed properly. This is especially helpful when writing to a file.
So what I am trying to do is to generate a list of number values arranging for example from 1 to 100000.
I have created a script that does that, but it only prints the output to the screen:
1
2
n
100000
What I need is exactly the same thing but appended to a file with a newline following each value:
1
2
n
100000
This is the script that generates the numbers:
def num_gen(min, max):
for i in range(min, max):
print(i)
print('Enter minimum Appoge value. e.g. 1000')
min_value = int(input())
print("Enter maximum Appoge value. e.g. 3000000")
max_value = int(input())
num_gen(min_value, max_value)
This is the script I wrote in a try to append the output to a file:
def num_gen(min, max):
appoge_list = file('appoge_wordlist.txt', 'w')
for i in range(min, max):
appoge_list.write(str(i))
appoge_list('\n')
appoge_list.close()
print('The appoge_list, appoge_wordlist.txt, has been generated. :)')
print('Enter minimum Appoge value. e.g. 1000')
min_value = int(input())
print("Enter maximum Appoge value. e.g. 3000000")
max_value = int(input())
num_gen(min_value, max_value)
Solution: Okay, I have been trying to achieve the above and then tried to oversimplify everything, I let the code explains it all:
Here is the new script, accepts no args:
def num_gen():
for i in range(10000, 2000001):
print(i)
num_gen()
Then run the command: python appoge_generator.py > appoge_wordlist.txt
To run the script from command line you would python appoge_generator.py I just added the > which means append the output of the procedure to the file appoge_wordlist.txt instead of the screen.
Now I have a 1990001 lines, 14,2 MB wordlist generated in 0.9 seconds. I hope some one finds this useful.
It might be faster/more convenient to use Python to write to the file:
with open('appoge_wordlist.txt', 'w') as file:
for i in range(10000, 2000001):
file.write('{}\n'.format(i))
Another option you might consider:
with open('appoge_wordlist.txt', 'wa') as a_file:
a_file.write('\n'.join([str(val) for val in range(10000, 2000001)]))
Basically open the file that you want to write to using a context manager (the with open(...) syntax) in "write-append" mode (the 'wa' argument), create your string of numbers joined by new lines with '\n'.join(...), and finally write that string to the file with a_file.write(...).
If you want this to be callable from the command line and allow you to pass in any desired start and stop values and any filename, I suggest looking at the excellent argparse module.
I'm trying to increase the count of an integer given that an if statement returns true. However, when this program is ran it always prints 0.I want n to increase to 1 the first time the program is ran. To 2 the second time and so on.
I know functions, classes and modules you can use the global command, to go outside it, but this doesn't work with an if statement.
n = 0
print(n)
if True:
n += 1
Based on the comments of the previous answer, do you want something like this:
n = 0
while True:
if True: #Replace True with any other condition you like.
print(n)
n+=1
EDIT:
Based on the comments by OP on this answer, what he wants is for the data to persist or in more precise words the variable n to persist (Or keep it's new modified value) between multiple runs times.
So the code for that goes as(Assuming Python3.x):
try:
file = open('count.txt','r')
n = int(file.read())
file.close()
except IOError:
file = open('count.txt','w')
file.write('1')
file.close()
n = 1
print(n)
n += 1
with open('count.txt','w') as file:
file.write(str(n))
print("Now the variable n persists and is incremented every time.")
#Do what you want to do further, the value of n will increase every time you run the program
NOTE:
There are many methods of object serialization and the above example is one of the simplest, you can use dedicated object serialization modules like pickle and many others.
If you want it to work with if statement only. I think you need to put in a function and make to call itself which we would call it recursion.
def increment():
n=0
if True:
n+=1
print(n)
increment()
increment()
Note: in this solution, it would run infinitely.
Also you can use while loop or for loop as well.
When you rerun a program, all data stored in memory is reset. You need to save the variable somewhere outside of the program, on disk.
for an example see How to increment variable every time script is run in Python?
ps. Nowadays you can simply do += with a bool:
a = 1
b = True
a += b # a will be 2
I have a text file describing an electronic circuit and a few other things done with it. I've built a simple Python code that splits the file into different units which can then be further analyzed if needed.
The syntax of the simulation language defines these units as contained within the following lines:
subckt xxx .....
...
...
ends xxx ...
There is a few of these 'text blocks' and other stuff I'm parsing or leaving out - like comment lines.
To accomplish this, I use the following core:
with open('input') as f:
for l in iter(f):
if 'subckt' not in l:
pass
else:
with open('output') as o:
o.write(l)
for l in iter(f):
if 'ends' in l:
o.write(l)
break
else:
o.write(l)
(can't easily paste the real code, there might be oversights)
The nice thing about it is the fact that iter(f) keeps scanning the file so when I break out of the inner loop as I reached the ends line of a subckt, the outer loop keeps going from that point onward, searching for new occurrences of the token subckt in subsequent lines.
I am looking for suggestions and/or guidance on how to transform the forest of if/then clauses into something more functional, i.e. based on 'pure' functions which just yield values (the file rows or lines) and are then composed as to bring to the final result.
Specifically, I am not sure how to approach the fact that the generator\map\filter should actually yield a different row based on the fact that it has found the subckt token or not.
I can think of a filter of the form:
line = filter(lambda x: 'subckt' in x, iter(f))
but this of course only gives me the lines where that string is present, whereas I would like - from that moment on - yield all lines, until the ends token is found.
Is this something I'd have to handle with recursion? Or maybe itertools.tee?
Seems to me that what I want is to have some form of state, i.e. "you have reached a subckt", but without resorting to a true state variable, which would be against the functional paradigm.
Not sure if this is what you are looking for. blocks(f) is a generator producing the blocks in your file f. Each block is an iterator over the lines between 'subckt' and 'ends'. If you want to include those two lines in the block, you'd have to do some more work in _blocks. But I hope this gives you an idea:
def __block(f):
while 'subckt' not in next(f): pass # raises StopIteration at EOF
return iter(next(iter([])) if 'ends' in l else l.strip() for l in f)
def blocks(f):
while 1: yield __block(f) # StopIteration from __block will stop the generator
f = open('data.txt')
for block in blocks(f):
# process block
for line in block:
# process line
next(iter([])) if is a little hack to terminate a comprehension/generator.
This answer also works, still very keen on hearing comments:
from itertools import takewhile, dropwhile
def start(l): return 'subckt' not in l
def stop(l): return 'ends' not in l
def sub(iter):
while True:
a = list(dropwhile(start,takewhile(stop,iter)))
if len(a):
yield a
else:
return
f = open('file.txt')
for b in sub(f):
#process b
f.close()
Something I couldn't work out yet: enclose the last line (containing ends keyword) in the output.
I am an almost new programmer learning python for a few months. For the last 2 weeks, I had been coding to make a script to search permutations of numbers that make magic squares.
Finally I succeeded in searching the whole 880 4x4 magic square numbers sets within 30 seconds. After that I made some different Perimeter Magic Square program. It finds out more than 10,000,000 permutations so that I want to store them part by part to files. The problem is that my program doesn't use all my processes that while it is working to store some partial data to a file, it stops searching new number sets. I hope I could make one process of my CPU keep searching on and the others store the searched data to files.
The following is of the similar structure to my magic square program.
while True:
print('How many digits do you want? (more than 20): ', end='')
ansr = input()
if ansr.isdigit() and int(ansr) > 20:
ansr = int(ansr)
break
else:
continue
fileNum = 0
itemCount = 0
def fileMaker():
global fileNum, itemCount
tempStr = ''
for i in permutationList:
itemCount += 1
tempStr += str(sum(i[:3])) + ' : ' + str(i) + ' : ' + str(itemCount) + '\n'
fileNum += 1
file = open('{0} Permutations {1:03}.txt'.format(ansr, fileNum), 'w')
file.write(tempStr)
file.close()
numList = [i for i in range(1, ansr+1)]
permutationList = []
itemCount = 0
def makePermutList(numList, ansr):
global permutationList
for i in numList:
numList1 = numList[:]
numList1.remove(i)
for ii in numList1:
numList2 = numList1[:]
numList2.remove(ii)
for iii in numList2:
numList3 = numList2[:]
numList3.remove(iii)
for iiii in numList3:
numList4 = numList3[:]
numList4.remove(iiii)
for v in numList4:
permutationList.append([i, ii, iii, iiii, v])
if len(permutationList) == 200000:
print(permutationList[-1])
fileMaker()
permutationList = []
fileMaker()
makePermutList(numList, ansr)
I added from multiprocessing import Pool at the top. And I replaced two 'fileMaker()' parts at the end with the following.
if __name__ == '__main__':
workers = Pool(processes=2)
workers.map(fileMaker, ())
The result? Oh no. It just works awkwardly. For now, multiprocessing looks too difficult for me.
Anybody, please, teach me something. How should my code be modified?
Well, addressing some things that are bugging me before getting to your asked question.
numList = [i for i in range(1, ansr+1)]
I know list comprehensions are cool, but please just do list(range(1, ansr+1)) if you need the iterable to be a list (which you probably don't need, but I digress).
def makePermutList(numList, ansr):
...
This is quite the hack. Is there a reason you can't use itertools.permutations(numList,n)? It's certainly going to be faster, and friendlier on memory.
Lastly, answering your question: if you are looking to improve i/o performance, the last thing you should do is make it multithreaded. I don't mean you shouldn't do it, I mean that it should literally be the last thing you do. Refactor/improve other things first.
You need to take all of that top-level code that uses globals, apply the backspace key to it, and rewrite functions that pass data around properly. Then you can think about using threads. I would personally use from threading import Thread and manually spawn Threads to do each unit of I/O rather than using multiprocessing.