We Keep Coding

Counting the command line arguments and removing the not needed one in python

I want to write python code which will be run using the following command :
python3 myProgram.py 4 A B C D stemfile
Where 4 is the number of files and A,B,C,D are 4 files.Then I wanted to generate all the combinations of A,B,C,D except the empty one.(A, B, C, D, AB, AC, AD, BC, BD, CD, ABC, ABD, ACD, BCD, ABCD) But before that it will read the stemfile.names and if stemfile.names has a line | Final Pseudo Deletion Count is 0. Then only it will generate the above 15 combination, else it will say noisy data and will not print the combinations of 3 files and not consider D. So the output will be : (A, B, C, AB, AC, BC, ABC)
So in my code what I did is, I always took D as the last file arguments and ran that loop 1 time less. But it is not always true that D will be the last argument only. It can be like : python3 myProgram.py 4 B D C A stemfile
In this case, in my code the A will not be considered while making the combinations, But whenever that line will not be found in the stemfile.names, I just want to remove D file from the equation. How should I do that?
And later in that code, when the combination is A only it will store the A in a seperate outputfile, whenever it is AB then it stores the union of A,B files in a separate files and so on for all the combinations. Here also if there is noisy data then that D file will not come in any of the outputfile.
One more example, If I give : python3 myProgram.py 3 A D B stemfile
And the stemfile.names doesn't have the line | Final Pseudo Deletion Count is 0. then the output combinations are : A,B,AB and it will create 2 output files only.
Below I am attaching my code:
import sys
import itertools
from itertools import combinations
def union(files):
lines = set()
for file in files:
with open(file) as fin:
lines.update(fin.readlines())
return lines
def main():
number = int(sys.argv[1])
dataset = sys.argv[number+2]
with open(dataset+'.names') as myfile:
if '| Final Pseudo Deletion Count is 0.' in myfile.read():
a_list = sys.argv[2:number+2]
print("All possible combinations:\n")
for L in range(1, len(a_list)+1):
for subset in itertools.combinations(a_list, L):
print(*list(subset), sep=',')
print("...............................")
matrix = [itertools.combinations(a_list, r)
for r in range(1, len(a_list) + 1)]
combinations = [c for combinations in matrix for c in combinations]
for combination in combinations:
filenames = [f'{name}' for name in combination]
output = f'{"".join(combination)}_output'
print(f'Writing union of {filenames} to {output}')
with open(output, 'w') as fout:
fout.writelines(union(filenames))
else:
a_list = sys.argv[2:number+1]
# Here I am reducing a number only
print("Noisy data.\n")
print("So all possible combinations:\n")
for L in range(1, len(a_list)+1):
for subset in itertools.combinations(a_list, L):
print(*list(subset), sep=',')
print("................................")
matrix = [itertools.combinations(a_list, r)
for r in range(1, len(a_list) + 1)]
combinations = [c for combinations in matrix for c in combinations]
for combination in combinations:
filenames = [f'{name}' for name in combination]
output = f'{"".join(combination)}_output'
print(f'Writing union of {filenames} to {output}')
with open(output, 'w') as fout:
fout.writelines(union(filenames))
if __name__ == '__main__':
main()
Please help me out.

I think you should probably break this down into smaller, more specific questions. It seems like there is a lot of detail here that's not focused on the specific problem you're facing. I took a shot at what I think you're asking, however.
I think you're trying to figure out how to remove an item from the command line arguments. If that's the case, there's nothing you can do about what's passed to the program, but you can modify the list of inputs after you parse. I really think you should try reading about the argparse library, as I stated in my comment. I'm not sure if it's exactly what you're looking for, but here's some code using argparse that expects full filenames for each input file. The last argument must be the stemfile.
Once the arguments are parsed, you have list of pathlib.Path objects. You can simply remove the D file from the list.
import argparse
import itertools
import pathlib
NOISY_DATA_LINE = '| Final Pseudo Deletion Count is 0.'
def get_parser():
parser = argparse.ArgumentParser()
parser.add_argument('filenames', type=pathlib.Path, nargs='+')
parser.add_argument('stemfile', type=pathlib.Path)
return parser
def union(files):
lines = set()
for file in files:
with open(file) as fin:
lines.update(fin.readlines())
return lines
def main():
parser = get_parser()
args = parser.parse_args()
stemfile_lines = args.stemfile.read_text().splitlines()
if stemfile_lines[-1] == NOISY_DATA_LINE:
filenames = [p for p in args.filenames if p.stem != 'D']
else:
filenames = args.filenames
matrix = [itertools.combinations(filenames, r) for r in range(1, len(filenames) + 1)]
combinations = [c for combinations in matrix for c in combinations]
print(' '.join([str([p.stem for p in c]) for c in combinations]))
for combination in combinations:
output = f'{"".join([p.stem for p in combination])}_output.txt'
print(f'Writing union of {[p.stem for p in combination]} to {output}')
with open(output, 'w') as fout:
fout.writelines(union(filenames))
if __name__ == '__main__':
main()

How to add digits in output

I have a file which I'm reading using python. In this file, I'm selecting certain numbers which are displayed as a list in the output, I want to add these numbers. Here is the code I'm using:
with open ("C:/xampp/htdocs/Final/uploads/file.dist", 'r') as rf:
g = [rf.replace(' ', '') for rf in rf]
k=[]
for e in g[1::47]:
r=(e[:12])
s=(r[:2])
i.append(s)
m= Counter(i)
for letter in m:
t= m[letter]
print(t)
This gives me output as follows:
80
80
80
80
I want to add these number so that the final output will be 320 (80+80+80+80). I've tried listing method, import math library, but none of them is giving me the required output. Any help will be highly appreciated.

Use += instead of = to add the values of m[letter] to t:
from collections import Counter
with open ("C:/path/file.dist", 'r') as rf:
g = [rf.replace(' ', '') for rf in rf]
i=[]
for e in g[1::47]:
r=(e[:12])
s=(r[:2])
i.append(s)
m = Counter(i)
t = 0
for letter in m:
t += m[letter]
print(t)

How to fix ' 'str' object has no attribute 'tell' ' error in Python when trying to read xpt files?

I am five .xpt data files which I am trying to read in python. I want to ge the data in those files into an array. This is for a machine learning project. I am trying that with this code but am getting an error on the line 'with xport.Reader(fname) as reader: '.
Though I am still not sure if fixing this error would fix the whole code or now. Is there any other method I can use?
# imports to read the xpt file data
import xport
import numpy as np
import os
#read the xpt file data
FN = ["BMX.XPT", "BMX_B.XPT", "BMX_C.XPT", "BMX_D.XPT", "BMX_E.XPT"]
def get_data(fname):
Z={}
H=None
with xport.Reader(fname) as reader:
for row in reader:
if H is None:
H=row.keys()
H.remove("SEQN")
H.sort()
Z[row["SEQN"]] = [row[k] for k in H]
return Z,H
# call get_data method on each file
D,VN =[],[]
for fn in FN:
fn_full = os.path.join("../Data/", fn)
X,H = get_data(fn_full)
s = fn.replace(".XPT", "")
H = [s + ":" + x for x in H]
D.append(X)
VN += H
## The sequence numbers that are in all data sets
KY = set(D[0].keys())
for d in D[1:]:
KY &= set(d.keys())
KY = list(KY)
KY.sort()
def to_float(x):
try:
return float(x)
except ValueError:
return float("nan")
## Merge the data
Z = []
for ky in KY:
z = []
map(z.extend, (d[ky] for d in D))
## equivalent to
## for d in D:
## z.extend(d[ky])
z = [to_float(a) for a in z]
## equivalent to
## map(to_float, z)
Z.append(z)
Z = np.array(Z)

It seems the input to xport.reader should be a file instead of a string. You should open it as a file instead of passing the file name as a parameter.
E.g.
with xport.Reader(open(fname, ‘r’)) as reader:
....

Writing an ASCII file from a 2D numpy array

I have a 2D numpy array. I want to make a ASCII file out of it. Here's the code I am using where a3 is the numpy array
f = open('ASCIIout.asc', 'w')
numpy.savetxt(ASCIIout, a3)
f.write("ncols " + str(ncols) + "\n")
f.write("nrows " + str(nrows) + "\n")
f.write("xllcorner " + str(xllcorner) + "\n")
f.write("yllcorner " + str(yllcorner) + "\n")
f.write("cellsize " + str(cellsize) + "\n")
f.write("NODATA_value " + str(noDATA) + "\n")
f.close()
Now I open it with append option and write the 2D array values:
f_handle = open(ASCIIout, 'a')
numpy.savetxt(f_handle, MyArray, fmt="%.3f")
f_handle.close()
However, I have a couple problems. First, the fmt really does not work I get values like this:
9.999000000000000000e+03
If I JUST use the code below line of code, I get -9999.000 1.345, but then, I haven't attached ncols, nrows, etc to the ASCII file.
numpy.savetxt(f_handle, MyArray, fmt="%.3f")
My data range from 0 to 6. What I really want to get is:
-9999 1.345 -9999 3.21 0.13 -9999
where I do NOT get decimals after -9999 and I do get decimals after real data such as 1.345 I made the ASCII file in R, I am wondering there should be a easy way to do it in Python. Thanks for your help in advance.

If your intention is to have the "stop value" -9999 appear without the decimal point, you can let savetxt() save all the values with the "%.3f" format, and replace the one result you don't want with the one you want:
from StringIO import StringIO
import numpy as np
f = StringIO()
x = np.array(( -9999, 1.345, -9999, 3.21, 0.13, -9999), dtype=float)
np.savetxt(f, x, fmt='%.3f')
f.seek(0)
fs = f.read().replace('-9999.000', '-9999', -1)
f.close()
f = open('ASCIIout.asc', 'w')
f.write("ncols " + str(ncols) + "\n")
f.write("nrows " + str(nrows) + "\n")
f.write("xllcorner " + str(xllcorner) + "\n")
f.write("yllcorner " + str(yllcorner) + "\n")
f.write("cellsize " + str(cellsize) + "\n")
f.write("NODATA_value " + str(noDATA) + "\n")
f.write(fs)
f.close()
If you want all the integer values represented without decimals, .replace('.000', '', -1) would accomplish that.

writing a range output lists into a text file or four array

I have a function and its output is a selection of lists [a,b,c,d] [a,b,c,d] [a,b,c,d] [a,b,c,d]
and I want [a,a,a,a] [b,b,b,b] [c,c,c,c] [d,d,d,d]
def meanarr(image, res=None):
"construct code which runs over a single ccd to get the means"
a = pyfits.getdata(image).MAG_AUTO
q = numpy.mean(a)
s = pyfits.getdata(image).X2WIN_IMAGE
j = numpy.mean(s)
f = pyfits.getdata(image).Y2WIN_IMAGE
z = numpy.mean(f)
g = pyfits.getdata(image).XYWIN_IMAGE
h = abs(numpy.mean(g))
a = [q, j, z, h]
print a
s0 = ''
return res
for arg in sys.argv[1:]:
#print arg
s = meanarr(arg)
This is my function and program how would I get the code to read all of the q's in one list all of the j's z's and h's in their own lists. I know I could separate the function into four different functions but this still doesn't return my results in a list it just outputs them individually.

You might be looking for zip. Try that :
data = [['a','b','c','d'], ['a','b','c','d'], ['a','b','c','d'], ['a','b','c','d']]
print data
print zip(*data)

You can write it this way:
def meanarr(image, res=None):
"costruct code which runs over a single ccd to get the means"
a=pyfits.getdata(image).MAG_AUTO
q=numpy.mean(a)
s=pyfits.getdata(image).X2WIN_IMAGE
j=numpy.mean(s)
f=pyfits.getdata(image).Y2WIN_IMAGE
z=numpy.mean(f)
g=pyfits.getdata(image).XYWIN_IMAGE
h= abs(numpy.mean(g))
a=[q,j,z,h]
return a
data =[meanarr(arg) for arg in sys.argv[1:]]
print zip(*data)