We Keep Coding

Python: Finding a negative number in a line of text from a file

I have a program that polls a servers current Wi-Fi status every minute, and saves that info to a .txt file. The output is:
*****CURRENT WIFI SIGNAL STRENGTH*****: Link Quality=57/70 Signal level=-53 dBm
The text file contains many of these lines. What I'm trying to accomplish is:
-Find the signal dBm values in all the lines, and append them to an array so I can then I can do other functions such as sort and average. I can't seem to get it working quite right.
Does anyone know how to do this?
Thank you!

I would go through each line in the file and split the line at =, then get the last value, split it at the space, and then get the first value which would yield -53.
strengthValues = []
f = open("input.txt", "r")
fileLines = f.readlines()
for line in fileLines:
lineSplit = line.split('=')
strengthValues.append(lineSplit[-1].split()[0])
print strengthValues
Or list comprehension:
f = open("test.txt", "r")
fileLines = f.readlines()
strengthValues = [line.split('=')[-1].split()[0] for line in fileLines]
print strengthValues

signal_levels = []
try:
with open("file.txt") as fh:
lines = fh.readlines()
except IOError as err:
# error handling
Then you can either make use of the re module:
for line in lines:
matches = re.search(r'Signal level=(-?[0-9]+) dBm$', line)
if matches is None:
# possible error handling
signal_levels.append(int(matches.group(1)))
Or without it (inspired by heinst's answer):
for line in lines:
try:
value = int(line.split('=')[-1].split()[0])
signal_levels.append(value)
except ValueError as err:
# possible error handling

Assuming that the signal level is the only negative number on any line you could use a regular expression with the findall function to search for all negative numbers in the file and return them as a list of strings (based on MC93's answer).
import re
f_in = open("input.txt", "r")
signal_levels = re.findall("-\d+", f_in.read())
Alternatively, you could get a list of ints with a list comprehension.
signal_levels = [int(n) for n in re.findall("-\d+", f_in.read())]

extract float numbers from data file

I'm trying to extract the values (floats) from my datafile. I only want to extract the first value on the line, the second one is the error. (eg. xo # 9.95322254_0.00108217853
means 9.953... is value, 0.0010.. is error)
Here is my code:
import sys
import re
inf = sys.argv[1]
out = sys.argv[2]
f = inf
outf = open(out, 'w')
intensity = []
with open(inf) as f:
pattern = re.compile(r"[^-\d]*([\-]{0,1}\d+\.\d+)[^-\d]*")
for line in f:
f.split("\n")
match = pattern.match(line)
if match:
intensity.append(match.group(0))
for k in range(len(intensity)):
outf.write(intensity[k])
but it doesn't work. The output file is empty.
the lines in data file look like:
xo_Is
xo # 9.95322254`_0.00108217853
SPVII_to_PVII_Peak_type
PVII_m(#, 1.61879`_0.08117)
PVII_h(#, 0.11649`_0.00216)
I # 0.101760618`_0.00190314017
each time the first number is the value I want to extract and the second one is the error.

You were almost there, but your code contains errors preventing it from running. The following works:
pattern = re.compile(r"[^-\d]*(-?\d+\.\d+)[^-\d]*")
with open(inf) as f, open(out, 'w') as outf:
for line in f:
match = pattern.match(line)
if match:
outf.write(match.group(1) + '\n')

I think you should test your pattern on a simple string instead of file. This will show where is the error: in pattern or in code which parsing file. Pattern looks good. Additionally in most languages i know group(0) is all captured data and for your number you need to use group(1)
Are you sure that f.slit('\n') must be inside for?

File operations failing

I am having the following issues in my code below,please provide inputs on where it is going wrong?
change_ignore_base.txt and change_ignore_file.txt are not getting created,where is it going wrong?
I see chagne_ignore has "\r" and "\n" appended,what is the smart way to strip off them and put them in a variable which can later be used to search.
change_ids.txt
206061
150362
147117
147441
143446
200912
change_ignore.txt
150362
147117
147441
143446
200914
Code
import os
import subprocess
from subprocess import check_call
def sync (base_change):
# open a file
with open('change_ignore.txt') as f:
change_ignore = f.readlines()
print "change_ignore"
print change_ignore
with open('change_ids.txt') as f:
lines = f.readlines()
for line in lines:
line=line.strip()
print line
if line <= base_change:
print "IN line<=base_change"
print line
with open("change_ignore_base.txt", "a") as myfile:
myfile.write(line)
if line in change_ignore:
print "IN change_ignore"
print line
with open("change_ignore_file.txt", "a") as myfile:
myfile.write("line")
if line > base_change and line not in change_ignore:
pass
def main ():
base_change=200913
sync(base_change)
if __name__ == '__main__':
main()

Here is a mild adjustment to your program that I believe accomplishes what you want. Key points (as pointed out in the comments) are that you want to compare integers with integers, and that you should avoid opening/closing files multiple times (as was happening with the file appends inside the loop).
import os
import subprocess
from subprocess import check_call
def sync(base_change):
# Generate a list of integers based on your change_ignore file
with open('change_ignore.txt', 'rb') as f:
# Here we make a list of integers based on the file
change_ignore = [int(line.strip()) for line in f]
# Store your hits/misses in lists; that way you do not
# need to continuously open/close files while appending
change_ignore_base = []
change_ignore_file = []
# Now open the file of the IDs
with open('change_ids.txt', 'rb') as f:
# Iterate over the file itself
for line in f:
# Convert the line to an integer (note that this
# implicitly removes the newline characters)
# However we are going to write 'line' to our list,
# which will keep the newline (more on that later)
num = int(line)
print num
# Now we are comparing ints with ints
# I'm assuming the print statements are for debugging,
# so we offset them with some space, making it so that
# any relevant hits are indented under a number
if num <= base_change:
print " IN line<=base_change"
change_ignore_base.append(line)
if num in change_ignore:
print " IN change_ignore"
change_ignore_file.append(line)
if num > base_change and num not in change_ignore:
pass
# Now that you have lists containing the data for your new files,
# write them (they already have newlines appended so writelines works)
# You can use 'with' with two files in this way in Python 2.7+,
# but it goes over 80 characters here so I'm not a huge fan :)
with open('change_ignore_base', 'wb') as b, open('change_ignore_file', 'wb') as f:
b.writelines(change_ignore_base)
f.writelines(change_ignore_file)
def main ():
base_change=200913
sync(base_change)
main()
This should create your files and print the following:
206061
150362
IN line<=base_change
IN change_ignore
147117
IN line<=base_change
IN change_ignore
147441
IN line<=base_change
IN change_ignore
143446
IN line<=base_change
IN change_ignore
200912
IN line<=base_change

Writing to file problems and displaying file contents

I am currently having problems displaying a file correctly once i have written a dictionary to the file. For this program the input file needs to have the format:
ID: Date: Dayskept: ProductName e.g. 1:12/12/2011:12:A
This is fine the first time I read the example file into a dictionary, but once i save the dictionay into a new file and try to open this file i get the output:
1:"date":12/12/2011, "life":12, "name":A
Is there an easy way to format the data in the dictionary before it is written to file?
Thanks for any advice given
def loadProduct(fileName):
global cheeseDictionary
f = open(fileName,"r")
line = f.readline() # Reads line from file
while line:
line = line[:-1]
data = split(line,":") # Splits line when there is a colon
cheeseDictionary[data[0]] = {"date":data[1], "life":data[2], "name":data[3]} # Stores each split item
line = f.readline() # Next line
f.close()
print cheeseDictionary
def saveProduct(fileName):
global cheeseDictionary
f = open(fileName,"w")
pickle.dump(cheeseDictionary, f)
f.close()

Since you have a specific format you want, you're going to need to write code to emit that format. (I don't know what you're trying to do with pickle in there, that produces a binary format that doesn't bear any resemblance to what you say you're getting.)
For instance, you could redefine saveProduct like so:
def saveProduct(fileName, cheeseDictionary):
f = open(fileName, "w")
for i in sorted(cheeseDictionary.keys()):
v = cheeseDictionary[i]
f.write("%s:%s:%s:%s\n" % (i, v["date"], v["life"], v["name"]))

Finding errors in a file

I have a huge file whose contents are generated from running an executable over and over on different input files. The file's pattern is such: -file name followed by an arbitrary amount of text lines. I have to pick up the name of the file when there is an error in reading input data and I am not sure what the best way to do it is. Another problem is that the word error comes up every time anyway in a phrase (Final fitting error was (some numerical value)) which needs to be ignored.
C:\temptest\blahblah1
.. (arbitrary # of text lines)
Final fitting error : (some number) [I have to ignore this]
C:\temptest\blahblah2
.. (arbitrary # of text lines)
Error could not read data !** [I have to pick up blahblah2 and copy the file to another directory, but just logging the name would suffice]
Thanks in advance !

This should do more or less what you need:
f = open("your_file.txt")
file_name = None
for line in f:
if line.startswith(r"C:\"):
file_name = line
elif line.startswith("Error"):
print "Error for file " + file_name
Assumptions:
- File names will start with "C:\", if that isn't true use a regular expression to perform a more accurate match or insert a special character before new files as you mentioned in a comment.
- There will only be one error per file, or printing multiple errors for a file is not a problem. If that is not the case, set some flag when you first print an error for a file and skip all subsequent errors until you find a new file.

So your log file looks like
{filepath}\file1
{
multiple lines
}
Final fitting error : 3.2
{filepath}\file2
{
multiple lines
}
Error could not read data !
and you want a list of all filenames resulting in the 'Error could not read data' message?
import re
import os.path
skipErrs = set("Final fitting error")
saveErrs = set("Error could not read data")
LOOKFOR = re.compile('(' + '|'.join(skipErrs) + '|' + '|'.join(saveErrs) + ')')
class EOF_Exception(Exception): pass
def getLine(f):
t = f.readline()
if t=='':
raise EOF_Exception('found end of file')
else:
return t.strip()
def getFilePath(f):
return os.path.normpath(getLine(f))
errorfiles = []
with open('logfile.txt') as inf:
while True:
try:
filepath = getFilePath(inf)
s = getLine(f)
m = re.match(s)
while not m:
s = getLine(f)
m = re.match(s)
if m.group(1) in saveErrs:
errorfiles.append(filepath)
except EOF_Exception:
break

With special being whatever header you want to append to the file lines:
[line[len(special):].strip() for line in file if line.startswith(special)]
You could use regexes also, but it will be more robust to add your own header, unless you are sure arbitrary lines could not start with a valid file name.

import shutil
f=open("file")
o=open("log","a")
for line in f:
if line.lstrip().startswith("C:"):
filename = line
if "Error" in line or "error" in line:
o.write( filename +"\n")
shutil.move(line,another_directory)
f.close()
o.close()