I have a file with lots of lines that contain numbers seperated by coma , on each line.
Some lines contain numbers that are float numbers ie: ['9.3']
i have been trying to delete those numbers from the list for about 1h~ but no luck. and whenever it tries to use int on a float number it gives error. Im not sure how to remove these floating numbers from the lines
the numbers: https://pastebin.com/7vveTxjW
this is what i've done so far:
with open('planets.txt','r') as f:
lst = []
temp = []
for line in f:
l = line.strip()
l = l.split(',')
for x in l:
if x == '':
l[l.index(x)] = 0
elif x == '\n' or x == '0':
print "removed value", l[l.index(x)]
del l[l.index(x)]
try:
temp.append([int(y) for y in l])
except ValueError:
pass
First off, modifying the list you are iterating over is a bad idea. A better idea might be to construct a new list from the elements that can be converted to an int.
def is_int(element):
try:
int(element)
except ValueError:
return False
return True
with open('planets.txt','r') as f:
lst = []
temp = []
for line in f:
l = line.strip().split(',')
temp.append([int(y) for y in l if is_int(y)])
If you want to include the float values' integral component, you can do:
def is_float(element):
try:
float(element)
except ValueError:
return False
return True
with open('planets.txt','r') as f:
lst = []
temp = []
for line in f:
l = line.strip().split(',')
temp.append([int(float(y)) for y in l if is_float(y)])
looks like youre over complicating it, once you have got all your numbers from the file to a list just run this
numbers=["8","9","10.5","11.1"]
intList=[]
for num in numbers:
try:
int_test=int(num)
intList.append(num)
except ValueError:
pass
just change my numbers list to your list name
You can just match digits followed by a point followed by more digits:
import re
output_list = []
input = open('input.txt', 'r')
for line in input:
if '.' not in line:
output_list.append(line)
else:
output_list.append(re.sub(r"(\d+\.\d+)", '', line))
print("Removed", re.search(r"(\d+\.\d+)", line).group(1))
I would keep the numbers as string and simply check if there is a 'dot' in each 'splitted item' to identify floats.
with open('planets.txt','r') as f:
for line in f:
line = ','.join([item for item in line.strip().split(',') if not '.' in item])
print(line)
# ... write to file ...
When you get a list that contains digits (for instance, ['1','2.3','4.5','1.0']) you can use the following
intDigits = [ i for i in digits if float(i) - int(float(i)) == 0]
Doing int() on a float shouldn't give an error. Maybe your 'float' is actually a string as you're reading from the file, because
int('9.3')
doesn't work but
int(9.3)
does.
Edit:
How about applying this function to every number
def intify(n):
if n == '':
return n
try:
return int(n)
except ValueError:
return int(float(n))
If you just need to remove floats this was the simplest method for me.
mixed = [1,float(1),2,3,4.3,5]
ints = [i for i in mixed if type(i) is not float]
Results in: [1, 2, 3, 5]
Related
f = open("LFx.txt", "r")
numbers=f.read(10)
array=[]
print(numbers)
for num in numbers:
if num%2==0:
array.append(num)
print(array)
I am getting type error everytime when i run this i dont now whats the problem is.Please Help me with this question.
numbers is a string, you need to transform the string into numbers.
suppose that you have the following data :
data = "1, 2, 1, 4"
then you can retrieve the numbers using
numbers = [int(value) for value in data.split(,)]
When you read from file it is in string format.
you need to type cast your data first and then do the operations.
you can try something like this:
f = open("LFx.txt", "r")
numbers=f.read(10)
array=[]
# here convert each number in string format to int
numbers = [int(i) for i in numbers.split() if i.isdigit()]
for num in numbers:
if num%2==0:
array.append(num)
print(array)
Can you provide some more information? How are the numbers stored in the file? line by line? all in one line? Seperated by commas? seperated by new line characters?
Reading the file will give you the stored information as a string, you can not execute mathematical operations between strings and integers.
this is invalid:
f = "1,2,3,4,5" #numbers read from your file
array = []
for num in f:
if num%2==0:
array.append(num)
print(array)
But this prints [2,4]:
f = 1,2,3,4,5
array = []
for num in f:
if num%2==0:
array.append(num)
print(array)
one solution would be:
import os
a = []
o = open("filename.txt","r")
y = o.read()
z = 0
for c in y.split():
a.append(int(c))
#now you can do mathematical operations on the elements in a
array = []
for num in a:
if num%2==0:
array.append(num)
print(array)
I am looking for words in my chosen text files.
def cash_sum(self):
with open(self.infile.name, "r") as myfile:
lines = myfile.readlines()
for line in lines:
if re.search("AMOUNT", line):
x = []
x.append(line[26:29])
print(x)
I have this output:
['100']
['100']
['100']
And I want to add them so I can have sum of all amounts from this file.
Any advices?
Initialize x outside the line-iterating loop.
Also, you don't need to readlines() separately...
def cash_sum(self):
x = []
with open(self.infile.name, "r") as myfile:
for line in myfile:
if re.search("AMOUNT", line):
x.append(line[26:29])
# or cast to int before appending:
# x.append(int(line[26:29]))
return x
This will give you the sum...
def cash_sum(self):
with open(self.infile.name, 'r') as myfile:
lines = myfile.readlines()
totalAmount = 0
for line in lines:
if re.search("AMOUNT", line):
totalAmount += float(line[26:29])
First thing I'd recommend is casting that value to a float. I say float instead of int just in case you have a decimal point. For example, float('100') == 100.0 . Once you have that, you should be able to then add them up as you would normal numbers.
You have redefined x inside your loop, so every time your loop iterates, it resets x to and empty list. Try the following:
def cash_sum(self):
with open(self.infile.name, "r") as myfile:
lines = myfile.readlines()
x = [] # That way x is defined outside the loop
for line in lines:
if re.search("AMOUNT", line):
x.append(line[26:29])
print(x)
# And if you want to add each item in the list...
num = float(0)
for number in x:
num = num + float(number)
return num
I wrote a script that allows me to extract through a loop all the floating numbers and put them in a list, then display this list with all the extracted floating numbers except that in my script only the last number of each list is taken into account, while I would like all the numbers to be displayed. how to do it?
there is my code :
final_result = []
result = []
k = listFps
k = 0
while k < len(listFps):
with open(listFps[k], 'r') as f:
#
statList = f.readlines()
statList = [x.strip() for x in statList]
for line in statList:
if (re.search("=", str(line))):
if (re.search('#IND', str(line))):
print("ok")
else:
result =re.findall("=\s*?(\d+\.\d+|\d+)", str(line))
print (" ca c result " ,result)
numberList = [float(q) for q in result]
print("ca c number list :",numberList)
k+=1
its print me only the last element of my list like this :
[59.889]
[60.874]
etc..
But i actually want a list with all element :
[59.889,60.874....]
Help me please im stuck with it for too long..
Instead of
result = re.findall….
use
result += re.findall…
I am trying to writer unique values to a csv that already has a list of ints inside it.
Currently I have tried to loop through a range of possible numbers then check if those numbers are in the csv. It appears that the checking is not working properly.
def generateUserCode():
with open ('/MyLocation/user_codes.csv') as csvDataFile:
userCodes = csv.reader(csvDataFile)
for x in range(0, 201):
if x not in userCodes:
return x
def writeUserCode(userCode):
with open ('/MyLocation/user_codes.csv', 'a') as csvDataFile:
csvDataFile.write('\n' + str(userCode))
userCode = generateUserCode()
writeUserCode(userCode)
So it should print the first number not in the csv and add the number to the csv. However all it is doing is printing 0 and adding 0 to my csv every time it is run even if 0 is in the csv.
Update:
The csv looks something like this:
3
4
5
35
56
100
There are more values but it is generally the same with no repeats and values between 0 and 200
The problem is with the following line:
if x not in userCodes:
userCodes is not a list it is a csvReader object. Also, you should use
if str(x) not in line:
#use str(x) instead of x
This is the code that works for me:
import csv
def generateUserCode():
with open ('file.csv') as csvDataFile:
csvread = csv.reader(csvDataFile)
userCodes = []
#print(userCodes)
for line in csvread:
try:
userCodes.append(line[0]) # As long as the code is the first
# element in that line, it should work
except:
IndexError # Avoid blank lines
print(userCodes)
for x in range(0, 201):
if str(x) not in userCodes:
return x
def writeUserCode(userCode):
with open ('file.csv', 'a') as csvDataFile:
csvDataFile.write('\n' + str(userCode))
userCode = generateUserCode()
writeUserCode(userCode)
Iterating userCodes shows each item is a list of strings:
for x in userCodes:
print(x)
returns:
['3']
['4']
['5']
['35']
['56']
['100']
So there are a lot of possible fixes, one would be:
def generateUserCode():
with open ('/MyLocation/user_codes.csv') as csvDataFile:
userCodes = csv.reader(csvDataFile)
userCodes = [int(item[0]) for item in userCodes]
for x in range(0, 201):
if x not in userCodes:
return x
It’s tricky to answer without seeing the CSV, but when you read the CSV, all fields are strings. Therefor you need to convert either the userCodes to int or x to string for the comparison to work.
For example:
userCodes = [int(d[0]) for d in csv.reader(csvDataFile)]
for x in range(0, 201):
if x not in userCodes:
return x
You are checking if a str is in an instance of csv.reader. This syntax doesn't work even with a normal file handle:
with open('somefile.txt') as fh:
x = fh.read()
x
'Spatial Reference: 43006\nName: Jones Tract\n424564.620666, 4396443.55267\n425988.30892, 4395630.01652\n426169.09473, 4395426.63249\n426214.291182, 4395268.4449\n\nName: Lewis Tract\n427909.158152, 4393935.14955\n428587.104939, 4393731.76552\n428700.096071, 4393528.38148\n428745.292523, 4393347.59567\n\nName: Adams Tract\n424180.450819, 4393957.74778\n424361.236629, 4393709.16729\n424655.013571, 4393641.37261\n424858.397607, 4393776.96197\n'
# now check if 'e' is in fh
with open('somefile.txt') as fh:
'e' in fh
False
'e' in x
True
Also, your csv file isn't really a csv file, so I'd just use a normal file handle and ignore the csv entirely.
The better approach may be to aggregate your codes in a set and check from there:
def get_codes():
with open('user_codes.csv') as fh:
# return a set to test membership quickly
return {line.strip() for line in fh}
codes = get_codes()
def add_code(code):
if code not in codes:
codes.add(code)
with open('user_codes.csv', 'a') as fh:
fh.write(code)
else:
raise ValueError("Code already exists")
# or do something else
add_code(88)
add_code(88)
# ValueError
To generate a user code automatically, since you are using a range, this becomes relatively easy:
def generate_user_code():
try:
# this returns the first number not in codes
return next(i for i in range(201) if i not in codes)
except StopIteration:
# you've exhausted your range, nothing is left
raise ValueError("No unique codes available")
# and your write method can be changed to
def add_code(code):
with open('user_codes.csv', 'a') as fh:
codes.add(code)
fh.write(code)
codes = get_codes()
user_code = generate_user_code()
add_code(user_code)
You may try to do this:
....
userCodes = csv.reader(csvDataFile)
uc = []
for y in userCodes:
uc += y
for x in range(0, 201):
if str(x) not in uc:
return x
....
This is my code. I want to read the data from "data.txt" and convert it to type float, but there is a problem converting the data. The error says " could not convert string to float: ".
def CrearMatriz():
archi = open("data.txt", "r")
num = archi.readlines()
for lines in num:
nums= [float(x) for x in lines.strip().split(",")]
return nums
Numero = CrearMatriz()
The file contains numbers separated by commas.
The only way I can reproduce this is by having a blank line at the end of the file. So my data file looks like this:
12.34 , 56.78 , 90.12 , 34.56
There's a blank line after it, but I can't display that! So here it is in another format:
od -xc data.txt
0000000 3231 332e 2034 202c 3635 372e 2038 202c
1 2 . 3 4 , 5 6 . 7 8 ,
0000020 3039 312e 2032 202c 3433 352e 0a36 000a
9 0 . 1 2 , 3 4 . 5 6 \n \n
0000037
Note the \n\n near end-of file.
Try this code:
def CrearMatriz():
archi = open("data.txt", "r")
num = archi.readlines()
for line in num:
line=line.strip()
if line:
nums=[float(x) for x in line.split(",")]
return nums
Numero = CrearMatriz()
However, that does not work if you have more than one line of numbers! It will only return the last line of numbers. So this might be better (depending on what you need):
def CrearMatriz():
nums = []
for line in open("data.txt", "r"):
line=line.strip()
if line:
nums.extend([float(x) for x in line.split(",")])
return nums
Numero = CrearMatriz()
If you want a list of lists, then change the extend to append.
I am not saying it's the perfect answer, but I suggest you do your parsing outside of the list comprehension proper and append to nums on success, at least until your formatting fixes everything.
From looking at the errors I am getting, one possibility is that it looks like lines.strip() doesn't account for blank lines in your file.
def CrearMatriz():
archi = open("data.txt", "r")
num = archi.readlines()
for lines in num:
nums= [float(x) for x in lines.strip().split(",")]
return nums
def CrearMatriz2():
with open("data.txt", "r") as archi:
#having it in a separate function allows more robust formatting
#and error handling. append to nums on success
def tofloat(nums, x, lines, cntr):
try:
res = float(x)
assert isinstance(res, float)
nums.append(float(x))
except Exception, e:
msg = "exception %s on field:%s: for line #%s:%s:" % (e, x, cntr,lines)
print (msg)
return
nums = []
num = archi.readlines()
for cntr, lines in enumerate(num):
[tofloat(nums, x, lines, cntr) for x in lines.strip().split(",")]
return nums
data="""
0.12,30.2,30.5,22
3,abc,4
"""
with open("data.txt","w") as f_test:
f_test.write(data)
try:
print "\n\ncalling CrearMatriz"
Numero = CrearMatriz()
print "\nCrearMatriz()=>", Numero
except Exception, e:
print e
try:
print "\n\ncalling CrearMatriz2"
Numero = CrearMatriz2()
print "\nCrearMatriz2()=>", Numero
except Exception, e:
print e
and this gives...
calling CrearMatriz
could not convert string to float:
calling CrearMatriz2
exception could not convert string to float: on field:: for line #0:
:
exception could not convert string to float: abc on field:abc: for line #2:3,abc,4
:
exception could not convert string to float: on field:: for line #3:
:
CrearMatriz2()=> [0.12, 30.2, 30.5, 22.0, 3.0, 4.0]
I'm interested in the type of
lines.strip().split(",")
It seems to me that strip() should work on a string, but lines would be a list of strings. You might need a double comprehension, something like:
[[float(x) for x in line.strip().split(",")] for line in lines]
Without your data, I can't verify this, though.
You might try removing the float(x) and just using
nums= [x for x in lines.strip().split(",")]
print(nums)
to see what the exact values are that float(x) is choking on.