We Keep Coding

Want to append a column in a file without using the Pandas

I have a file say, outfile.txt which looks like below:
1,2,3,4,0,0.95
1,2,4,4,0,0.81
5,6,3,1,0,0.89
7,6,8,8,0,0.77
6,6,4,9,0,0.88
9,9,9,1,0,0.66
4,3,6,9,0,0.85
1,2,6,7,0,0.61
Now I want to append one extra 1 to each row. So the desired output file looks like:
1,2,3,4,0,0.95,1
1,2,4,4,0,0.81,1
5,6,3,1,0,0.89,1
7,6,8,8,0,0.77,1
6,6,4,9,0,0.88,1
9,9,9,1,0,0.66,1
4,3,6,9,0,0.85,1
1,2,6,7,0,0.61,1
How can I do it? Whenever I am googling it to find a solution, I am seeing everywhere this kind of solution is provided using Pandas, But I don't want to use that.

Since your file is in csv format, csv module can help you. If you iterate over the reader object, it gives you a list of the items in each line in the file, then simply .append() what you want.
import csv
with open("outfile.txt") as f:
reader = csv.reader(f)
for line in reader:
line.append("1")
print(",".join(line))
If you have a column like column you can zip it with the reader object and append the corresponding element in the loop:
import csv
column = range(10)
with open("outfile.txt") as f:
reader = csv.reader(f)
for line, n in zip(reader, map(str, column)):
line.append(n)
print(",".join(line))
I printed, you can write it to a new file.

You can read and write files line by line with the csv module. A reader object will iterate the rows of the input file and writer.writerows will consume that iterator. You just need a bit of extra code to add the 1. Using a list generator, this example adds the extra column.
import csv
import os
filename = "outfile.txt"
tmp = filename + ".tmp"
with open(filename, newline="") as infile, open(tmp, "w", newline="") as outfile:
csv.writer(outfile).writerows(row + [1] for row in csv.reader(infile))
os.rename(tmp, filename)

Just, iterate through the file line by line and add ,1 at the end of each line:
with open('outfile.txt', 'r') as input:
with open('outfile_final.txt', 'w') as output:
for line in input:
line = line.rstrip('\n') + ',1'
print(line, file=output)

Python program for writing length of list to file

I have a file list.txt that contains a single list only e.g.
[asd,ask,asp,asq]
The list might be a very long one. I want to create a python program len.py that reads list.txt and writes the length of the within list to the file num.txt. Something like the following:
fin = open("list.txt", "rt")
fout = open("num.txt", "wt")
for list in fin:
fout.write(len(list))
fin.close()
fout.close()
However this does not work. Can someone point out what needs to be changed? Many thanks.

Use:
with open("list.txt") as f1, open("num.txt", "w") as f2:
for line in f1:
line = line.strip('\n[]')
f2.write(str(len(line.split(','))) + '\n')

with open("list.txt") as fin, open("num.txt", "w") as fout:
input_data = fin.readline()
# check if there was any info read from input file
if input_data:
# split string into list on comma character
strs = input_data.replace('[','').split('],')
lists = [map(int, s.replace(']','').split(',')) for s in strs]
print(len(lists))
fout.write(str(len(lists)))
I updated the code to use the with statement from another answer. I also used some code from this answer (How can I convert this string to list of lists?) to (more?) correctly count nested lists.

When python try to read a file using default method it generally treats content of that file as a string. So first responsibility is to type cast string content into appropriate content type for that you can not use default type casting method.
You can use special package by the name ast to type cast the data.
import ast
fin = open("list.txt", "r")
fout = open("num.txt", "w")
for list in fin.readlines():
fout.write(len(ast.literal_eval(list)))
fin.close()
fout.close()

Nested lists in python containing a single string and not single letters

I need to load text from a file which contains several lines, each line contains letters separated by coma, into a 2-dimensional list. When I run this, I get a 2 dimensional list, but the nested lists contain single strings instead of separated values, and I can not iterate over them. how do I solve this?
def read_matrix_file(filename):
matrix = []
with open(filename, 'r') as matrix_letters:
for line in matrix_letters:
line = line.split()
matrix.append(line)
return matrix
result:
[['a,p,p,l,e'], ['a,g,o,d,o'], ['n,n,e,r,t'], ['g,a,T,A,C'], ['m,i,c,s,r'], ['P,o,P,o,P']]
I need each letter in the nested lists to be a single string so I can use them.
thanks in advance

split() function splits on white space by default. You can fix this by passing the string you want to split on. In this case, that would be a comma. The code below should work.
def read_matrix_file(filename):
matrix = []
with open(filename, 'r') as matrix_letters:
for line in matrix_letters:
line = line.split(',')
matrix.append(line)
return matrix

The input format you described conforms to CSV format. Python has a library just for reading CSV files. If you just want to get the job done, you can use this library to do the work for you. Here's an example:
Input(test.csv):
a,string,here
more,strings,here
Code:
>>> import csv
>>> lines = []
>>> with open('test.csv') as file:
... reader = csv.reader(file)
... for row in reader:
... lines.append(row)
...
>>>
Output:
>>> lines
[['a', 'string', 'here'], ['more', 'strings', 'here']]

Using the strip() function will get rid of the new line character as well:
def read_matrix_file(filename):
matrix = []
with open(filename, 'r') as matrix_letters:
for line in matrix_letters:
line = line.split(',')
line[-1] = line[-1].strip()
matrix.append(line)
return matrix

Read in lines from text line as separate arrays

I have a text file containing lines of strings that resemble an array format. I initially had a list of numpy arrays, and read them into the file like this, where each array is about 5 floats:
import numpy as np
parameters = [np.array(...), np.array(...), ...]
with open('params.txt', 'w') as f:
for param in parameters:
f.write(str(param)+'\n')
Now I'd like to read them back out, as a list of separate arrays. I'm having issues with this however -- below is what I'm trying to do:
parameters = []
with open('params.txt', 'r') as f:
for line in f:
parameters.append(np.array(line))
But now when I later try to index elements in these arrays and use list comprehension, like: [params[2] for params in parameters], I get this error: IndexError: too many indices for array.
I have also tried reading them out with line.split(','), but this didn't give me what I wanted and just messed up the formatting further. How can I accomplish this?
The format of my text file:
[242.1383, 131.087, 1590.853, 1306.09, 783.979]
[7917.102, 98.12, 21.43, 13.1383, 6541.33]
[823.74, 51.31, 9622.434, 974.11, 980.177]
...
What I want:
parameters = [np.array([242.1383, 131.087, 1590.853, 1306.09, 783.979]), np.array([7917.102, 98.12, 21.43, 13.1383, 6541.33]), np.array([823.74, 51.31, 9622.434, 974.11, 980.177]), ...]

I figured out a slightly simpler way to accomplish this without having to worry about all the string parsing, using regex:
import re
parameters = []
with open('params.txt', 'r') as f:
for line in f:
set = [float(value) for value in re.findall('\d+\.?\d*', line)]
parameters.append(np.array(set))

are you looking for something like this?
parameters = []
for line in f.readlines():
y = [value for value in line.split()]
parameter.append( y )
would be easier if I knew what the text file looked like if would would show the format of the text file you're trying to read from

open a .json file with multiple dictionaries

I have a problem that I can't solve with python, it is probably very stupid but I didn't manage to find the solution by myself.
I have a .json file where the results of a simulation are stored. The result is stored as a series of dictionaries like
{"F_t_in_max": 709.1800264942982, "F_t_out_max": 3333.1574129603068, "P_elec_max": 0.87088836042046958, "beta_max": 0.38091242406098391, "r0_max": 187.55175182942901, "r1_max": 1354.8636763521174, " speed ": 8}
{"F_t_in_max": 525.61428305710433, "F_t_out_max": 2965.0538075438467, "P_elec_max": 0.80977406754203796, "beta_max": 0.59471606595464666, "r0_max": 241.25371753877008, "r1_max": 688.61786996066826, " speed ": 9}
{"F_t_in_max": 453.71124051199763, "F_t_out_max": 2630.1763649193008, "P_elec_max": 0.64268078173342935, "beta_max": 1.0352896471221695, "r0_max": 249.32706230502498, "r1_max": 709.11415981343885, " speed ": 10}
I would like to open the file and and access the values like to plot "r0_max" as function of "speed" but I can't open unless there is only one dictionary.
I use
with open('./results/rigid_wing_opt.json') as data_file:
data = json.load(data_file)
but When the file contains more than one dictionary I get the error
ValueError: Extra data: line 5 column 1 - line 6 column 1 (char 217 - 431)

If your input data is exactly as provided then you should be able to interpret each individual dictionary using json.load. If each dictionary is on its own line then this should be sufficient:
with open('filename', 'r') as handle:
json_data = [json.loads(line) for line in handle]

I would recommend reading the file line-by-line and convert each line independently to a dictionary.
You can place each line into a list with the following code:
import ast
# Read all lines into a list
with open(fname) as f:
content = f.readlines()
# Convert each list item to a dict
content = [ ast.literal_eval( line ) for line in content ]
Or an even shorter version performing the list comprehension on the same line:
import ast
# Read all lines into a list
with open(fname) as f:
content = [ ast.literal_eval( l ) for l in f.readlines() ]

{...} {...} is not proper json. It is two json objects separated by a space. Unless you can change the format of the input file to correct this, I'd suggest you try something a little different. If the data is a simple as in your example, then you could do something like this:
with open('filename', 'r') as handle:
text_data = handle.read()
text_data = '[' + re.sub(r'\}\s\{', '},{', text_data) + ']'
json_data = json.loads(text_data)
This should work even if your dictionaries are not on separate lines.

That is not valid JSON. You can't have multiple obje at the top level, without surrounding them by a list and inserting commas between them.