Python writing a .csv file with rows and columns transpose [duplicate] - python
This question already has answers here:
How to transpose a dataset in a csv file?
(7 answers)
Closed 2 years ago.
What I have is a long list of codes that involves reading different files and in the end putting everything into different .csv
This is all my codes
import csv
import os.path
#open files + readlines
with open("C:/Users/Ivan Wong/Desktop/Placement/Lists of targets/Mouse/UCSC to Ensembl.csv", "r") as f:
reader = csv.reader(f, delimiter = ',')
#find files with the name in 1st row
for row in reader:
graph_filename = os.path.join("C:/Python27/Scripts/My scripts/Selenoprotein/NMD targets",row[0]+"_nt_counts.txt.png")
if os.path.exists(graph_filename):
y = row[0]+'_nt_counts.txt'
r = open('C:/Users/Ivan Wong/Desktop/Placement/fp_mesc_nochx/'+y, 'r')
k = r.readlines()
r.close
del k[:1]
k = map(lambda s: s.strip(), k)
interger = map(int, k)
import itertools
#adding the numbers for every 3 rows
def grouper(n, iterable, fillvalue=None):
"grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
args = [iter(iterable)] * n
return itertools.izip_longest(*args, fillvalue=fillvalue)
result = map(sum, grouper(3, interger, 0))
e = row[1]
cDNA = open('C:/Users/Ivan Wong/Desktop/Placement/Downloaded seq/Mouse/cDNA.txt', 'r')
seq = cDNA.readlines()
# get all lines that have a gene name
lineNum = 0;
lineGenes = []
for line in seq:
lineNum = lineNum +1
if '>' in line:
lineGenes.append(str(lineNum))
if '>'+e in line:
lineBegin = lineNum
cDNA.close
# which gene is this
index1 = lineGenes.index(str(lineBegin))
lineEnd = lineGenes[index1+1]
# linebegin and lineEnd now give you, where to look for your sequence, all that
# you have to do is to read the lines between lineBegin and lineEnd in the file
# and make it into a single string.
lineEnd = lineGenes[index1+1]
Lastline = int(lineEnd) -1
# in your code you have already made a list with all the lines (q), first delete
# \n and other symbols, then combine all lines into a big string of nucleotides (like this)
qq = seq[lineBegin:Lastline]
qq = map(lambda s: s.strip(), qq)
string = ''
for i in range(len(qq)):
string = string + qq[i]
# now you want to get a list of triplets, again you can use the for loop:
# first get the length of the string
lenString = len(string);
# this is your list codons
listCodon = []
for i in range(0,lenString/3):
listCodon.append(string[0+i*3:3+i*3])
with open(e+'.csv','wb') as outfile:
outfile.writelines(str(result)+'\n'+str(listCodon))
My problem here is the file produced looks like this:
0 0 0
'GCA' 'CTT' 'GGT'
I want to make it like this:
0 GCA
0 CTT
0 GGT
What can I do in my code to achieve this?
print result:
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 2, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 3, 3, 0, 3, 1, 2, 1, 2, 1, 0, 1, 0, 1, 2, 1, 0, 5, 0, 0, 0, 0, 6, 0, 1, 0, 0, 2, 0, 1, 0, 0, 1, 1, 0, 1, 6, 34, 35, 32, 1, 1, 0, 4, 1, 0, 1, 0, 0, 0, 0, 1, 6, 0, 0, 0, 0, 1, 3, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
print listCodon:
['gtt', 'gaa', 'aca', 'gag', 'aca', 'tgt', 'tct', 'gga', 'gat', 'gag', 'ctg', 'tgg', 'gca', 'gaa', 'gga', 'cag', 'gcc', 'taa', 'gca', 'cag', 'gca', 'gca', 'gag', 'ctt', 'tga', 'tct', 'ctt', 'ggt', 'gat', 'cgg', 'tgg', 'ggg', 'atc', 'cgg', 'tgg', 'cct', 'agc', 'ttg', 'tgc', 'caa', 'gga', 'agc', 'tgc', 'tca', 'gct', 'ggg', 'aaa', 'gaa', 'ggt', 'ggc', 'tgt', 'ggc', 'tga', 'cta', 'tgt', 'gga', 'acc', 'ttc', 'tcc', 'ccg', 'agg', 'cac', 'caa', 'gtg', 'ggg', 'cct', 'tgg', 'tgg', 'cac', 'ctg', 'tgt', 'caa', 'cgt', 'ggg', 'ttg', 'cat', 'acc', 'caa', 'gaa', 'gct', 'gat', 'gca', 'tca', 'ggc', 'tgc', 'act', 'gct', 'ggg', 'ggg', 'cat', 'gat', 'cag', 'aga', 'tgc', 'tca', 'cca', 'cta', 'tgg', 'ctg', 'gga', 'ggt', 'ggc', 'cca', 'gcc', 'tgt', 'cca', 'aca', 'caa', 'ctg', 'gtg', 'aga', 'gag', 'aag', 'ccc', 'ttg', 'ccc', 'tct', 'gca', 'ggt', 'ccc', 'att', 'gaa', 'agg', 'aga', 'ggt', 'ttg', 'ctc', 'tct', 'gcc', 'act', 'cat', 'ctg', 'taa', 'ccg', 'tga', 'gct', 'ttt', 'cca', 'ccc', 'ggc', 'ctc', 'ctc', 'ttt', 'gat', 'ccc', 'aga', 'ata', 'atg', 'act', 'ctg', 'aga', 'ctt', 'ctt', 'atg', 'tat', 'gaa', 'taa', 'atg', 'cct', 'ggg', 'cca', 'aaa', 'acc']
picture on the left is what Marek's code helped me to achieve, I want to make an improvement so it arrange like the picture on the right
You can use zip() to zip together two iterators. So if you have
result = [0, 0, 0, 0, 0]
listCodons = ['gtt', 'gaa', 'aca', 'gag', 'aca']
then you can do
>>> list(zip(result, listCodons))
[(0, 'gtt'), (0, 'gaa'), (0, 'aca'), (0, 'gag'), (0, 'aca')]
or, for your example:
with open(e+'.csv','w') as outfile:
out = csv.writer(outfile)
out.writerows(zip(result, listCodons))
try this:
proper_result = '\n'.join([ '%s %s' % (nr, codon) for nr, codon in zip(result, listCodon) ] )
Edit (codons split into separate columns):
proper_result = '\n'.join(' '.join([str(nr),] + list(codon)) for nr, codon in zip(nrs, cdns))
Edit (comma separated values):
proper_result = '\n'.join('%s, %s' % (nr, codon) for nr, codon in zip(result, listCodon))
Related
How to create a dictionary of permutation values?
I need to assign 0 and 1 as values to keys in the dictionaries: combinations_string_list = [num_list_to_str(i) for i in itertools.product([0, 1], repeat=2)] all_stategy = [] for i in range(16): strategy_table = {x: y for x in combinations_string_list for y in [0, 1]} all_stategy.append(strategy_table) print(all_stategy) I got [{'00': 1, '01': 1, '10': 1, '11': 1}, {'00': 1, '01': 1, '10': 1, '11': 1}, {'00': 1, '01': 1, '10': 1, '11': 1}, ...] but I need [{'00': 0, '01': 0, '10': 0, '11': 0}, {'00': 0, '01': 0, '10': 0, '11': 1}, {'00': 0, '01': 0, '10': 1, '11': 0}, ...] instead. How can I create this kind of value? Thanks!
You can zip the key sequence ["0", "1"] with each element of the cartesian product to produce input to dict: >>> [dict(zip(["0", "1"], x)) for x in product([0,1], repeat=2)] [{'0': 0, '1': 0}, {'0': 0, '1': 1}, {'0': 1, '1': 0}, {'0': 1, '1': 1}] or >>> values=[0,1] >>> [dict(zip(map(str, values), x)) for x in product(values, repeat=2)]
Printing possible substring in set of 3 and matching with dictionary keys in python
I want to print possible substring in set of 3 and assign dictionary values if pattern matches with dictionary.keys() and store them into new dictionary input: dict1={'000': 0, '001': 0, '010': 0, '011': 0, '100': 1, '101': 0, '110': 1, '111': 0} str1=['010110100'] output: sub_string= [010,101,011...] new dict= {'010':0, '101':0, '011':0, '110':1......}
try this: [str1[0][i:i+3] for i in range(len(str1[0])-2) if str1[0][i:i+3] in dict1] {str1[0][i:i+3] : dict1.get(str1[0][i:i+3]) for i in range(len(str1[0])-2) if str1[0][i:i+3] in dict1} # ['010', '101', '011', '110', '101', '010', '100'] # {'010': 0, '101': 0, '011': 0, '110': 1, '100': 1}
You could do like this: dict1={'000': 0, '001': 0, '010': 0, '011': 0, '100': 1, '101': 0, '110': 1, '111': 0} str1= '010110100' sub_string = [] d = {} for i in range(len(str1)-2): temp = str1[i:i+3] sub_string.append(temp) d[temp] = dict1.get(temp) print(sub_string) print(d) ['010', '101', '011', '110', '101', '010', '100'] {'010': 0, '101': 0, '011': 0, '110': 1, '100': 1}
Pandas - Add a column level to multi index
I would like to add a sublevel (L4) in my dataframe, based on a list of values: x = [0.01, 0.01, 0.01, 0.02, 0.02, 0.02] The df.columns returns me this: MultiIndex(levels=[['Foo', 'Bar'], ['A', 'B', 'C'], ['a']], labels=[[0, 0, 0, 1, 1, 1], [0, 1, 2, 0, 1, 2], [0, 0, 0, 0, 0, 0]], names=['L1', 'L2', 'L3']) So far I have tried that: df = pd.concat([df], keys=x, names=['L4'], axis=1).swaplevel(i='L4', j='L1', axis=1).swaplevel(i='L4', j='L2', axis=1).swaplevel(i='L4', j='L3', axis=1) but it doesn't give the good value, it repeats list_levels[0] (0.01). Do you have any idea on how I can do it ? Thanks
Here's a way: cols = pd.MultiIndex(levels=[['Foo', 'Bar'], ['A', 'B', 'C'], ['a']], labels=[[0, 0, 0, 1, 1, 1], [0, 1, 2, 0, 1, 2], [0, 0, 0, 0, 0, 0]], names=['L1', 'L2', 'L3']) pd.DataFrame(columns = cols).T\ .assign(x = [0.01, 0.01, 0.01, 0.02, 0.02, 0.02])\ .set_index('x', append=True).T Output:
You can create a DataFrame with the column index as the Index, and the data being the level you want to add, as set_index(append=True) is only defined for the row Index. Then assign it with df.columns = ... import pandas as pd idx = pd.MultiIndex(levels=[['Foo', 'Bar'], ['A', 'B', 'C'], ['a']], codes=[[0, 0, 0, 1, 1, 1], [0, 1, 2, 0, 1, 2], [0, 0, 0, 0, 0, 0]], names=['L1', 'L2', 'L3']) x = [0.01, 0.01, 0.01, 0.02, 0.02, 0.02] pd.DataFrame(x, index=idx, columns=['L4']).set_index('L4', append=True).index #MultiIndex([('Foo', 'A', 'a', 0.01), # ('Foo', 'B', 'a', 0.01), # ('Foo', 'C', 'a', 0.01), # ('Bar', 'A', 'a', 0.02), # ('Bar', 'B', 'a', 0.02), # ('Bar', 'C', 'a', 0.02)], # names=['L1', 'L2', 'L3', 'L4']) Under the hood set_index just recreates the entire MultiIndex when appending, so a more hands-on approach is arrays = [] for i in range(idx.nlevels): arrays.append(idx.get_level_values(i)) arrays.append(pd.Index(x, name='L4')) # Add the new level new_idx = pd.MultiIndex.from_arrays(arrays) #MultiIndex([('Foo', 'A', 'a', 0.01), # ('Foo', 'B', 'a', 0.01), # ('Foo', 'C', 'a', 0.01), # ('Bar', 'A', 'a', 0.02), # ('Bar', 'B', 'a', 0.02), # ('Bar', 'C', 'a', 0.02)], # names=['L1', 'L2', 'L3', 'L4'])
Replicate a nested for-loop with a dict-comprehension
Given the following code: minimal example of list of lists: c = [['dog', 'Sg', 'Good'], ['cat', 'Pl', 'Okay'], ['dog', 'Pl', 'Bad'], ['dog', 'Sg', 'Good'], ['cat', 'Pl', 'Okay'], ['dog', 'Pl', 'Okay'], ['dog', 'Sg', 'Good'], ['cat', 'Sg', 'Good'], ['dog', 'Pl', 'Bad'], ['dog', 'Sg', 'Good'], ['cat', 'Pl', 'Okay'], ['dog', 'Pl', 'Bad']] create sets of words from c outer_keys = set() inner_keys = set() for x in c: outer_keys.add(x[0]) inner_keys |= set(x[1:]) create dict with for loop Lemma = dict() for i in outer_keys: j_d = dict() for j in inner_keys: j_d[j] = 0 j_d[i] = 0 # this is the line I can't replicate with a comprehension Lemma[i] = j_d for loop result: {'dog': {'Okay': 0, 'Pl': 0, 'Good': 0, 'Bad': 0, 'Sg': 0, 'dog': 0}, 'cat': {'Okay': 0, 'Pl': 0, 'Good': 0, 'Bad': 0, 'Sg': 0, 'cat': 0}} using dict comprehension: This is where I need assistance. I haven't been able to replicate the for-loop with the dict comprehension Lemma = {j: {i: 0 for i in inner_keys} for j in outer_keys} Note: dog should be inside the value of outer dog and the same for cat My dict comprehension result: {'dog': {'Okay': 0, 'Pl': 0, 'Good': 0, 'Bad': 0, 'Sg': 0}, 'cat': {'Okay': 0, 'Pl': 0, 'Good': 0, 'Bad': 0, 'Sg': 0}} Question: How can I replicate the result of the for-loop with the dict comprehension? Order doesn't matter.
You can use dict.fromkeys together with inner_keys | {j}: >>> {j: dict.fromkeys(inner_keys | {j}, 0) for j in outer_keys} {'cat': {'Bad': 0, 'Good': 0, 'Okay': 0, 'Pl': 0, 'Sg': 0, 'cat': 0}, 'dog': {'Bad': 0, 'Good': 0, 'Okay': 0, 'Pl': 0, 'Sg': 0, 'dog': 0}}
Just create a new dict from your inner dict >>> {j: dict({i: 0 for i in inner_keys}, **{j:0}) for j in outer_keys} {'dog': {'Bad': 0, 'Good': 0, 'Okay': 0, 'Sg': 0, 'dog': 0, 'Pl': 0}, 'cat': {'Bad': 0, 'Good': 0, 'Okay': 0, 'Sg': 0, 'Pl': 0, 'cat': 0}}
Class constant dictionary in Python
I'm building a module that has a class variable dictionary: class CodonUsageTable: CODON_DICT={'TTT': 0, 'TTC': 0, 'TTA': 0, 'TTG': 0, 'CTT': 0, 'CTC': 0, 'CTA': 0, 'CTG': 0, 'ATT': 0, 'ATC': 0, 'ATA': 0, 'ATG': 0, 'GTT': 0, 'GTC': 0, 'GTA': 0, 'GTG': 0, 'TAT': 0, 'TAC': 0, 'TAA': 0, 'TAG': 0, 'CAT': 0, 'CAC': 0, 'CAA': 0, 'CAG': 0, 'AAT': 0, 'AAC': 0, 'AAA': 0, 'AAG': 0, 'GAT': 0, 'GAC': 0, 'GAA': 0, 'GAG': 0, 'TCT': 0, 'TCC': 0, 'TCA': 0, 'TCG': 0, 'CCT': 0, 'CCC': 0, 'CCA': 0, 'CCG': 0, 'ACT': 0, 'ACC': 0, 'ACA': 0, 'ACG': 0, 'GCT': 0, 'GCC': 0, 'GCA': 0, 'GCG': 0, 'TGT': 0, 'TGC': 0, 'TGA': 0, 'TGG': 0, 'CGT': 0, 'CGC': 0, 'CGA': 0, 'CGG': 0, 'AGT': 0, 'AGC': 0, 'AGA': 0, 'AGG': 0, #Other code def __init__(self,seqobj): '''Creates codon table for a given Bio.seq object.i The only argument is Bio.Seq object with DNA Currently assumes seq to be DNA, RNA support to be added later''' dnaseq=str(seqobj) self.usage_table=CodonUsageTable.CODON_DICT.deepcopy()#instance of table The last line must make a copy of class dictionary to store instance data in it, but it throws Traceback (most recent call last): File "<stdin>", line 1, in <module> File "./codon_usage.py", line 47, in __init__ self.usage_table=CodonUsageTable.CODON_DICT.deepcopy()#instance of codon usage table NameError: global name 'CODON_DICT' is not defined So does self.CODON_DICT, CODON_DICT or codon_usage.CodonUsageTable.CODON_DICT, when called from __init__. Dictionary is defined: >>>import codon_usage >>> codon_usage.CodonUsageTable.CODON_DICT {'GCT': 0, 'GGA': 0, 'TTA': 0, 'GAT': 0, 'TTC': 0, 'TTG': 0, 'AGT': 0, 'GCG': 0, 'AGG': 0, 'GCC': 0, 'CGA': 0, 'GCA': 0, 'GGC': 0, 'GAG': 0, 'GAA': 0, 'TTT': 0, 'GAC': 0, 'TAT': 0, 'CGC': 0, 'TGT': 0, 'TCA': 0, 'GGG': 0, 'TCC': 0, 'ACG': 0, 'TCG': 0, 'TAG': 0, 'TAC': 0, 'TAA': 0, 'ACA': 0, 'TGG': 0, 'TCT': 0, 'TGA': 0, 'TGC': 0, 'CTG': 0, 'CTC': 0, 'CTA': 0, 'ATG': 0, 'ATA': 0, 'ATC': 0, 'AGA': 0, 'CTT': 0, 'ATT': 0, 'GGT': 0, 'AGC': 0, 'ACT': 0, 'CGT': 0, 'GTT': 0, 'CCT': 0, 'AAG': 0, 'CGG': 0, 'AAC': 0, 'CAT': 0, 'AAA': 0, 'CCC': 0, 'GTC': 0, 'CCA': 0, 'GTA': 0, 'CCG': 0, 'GTG': 0, 'ACC': 0, 'CAA': 0, 'CAC': 0, 'AAT': 0, 'CAG': 0} 'GGT': 0, 'GGC': 0, 'GGA': 0, 'GGG': 0}
The symptoms imply that the story went like this: you wrote the file and saved it; you ran the Python shell; you found that CODON_DICT can't be accessed just like that and fixed that; you tried that call again within the same Python shell and got that error. That happens because Python is still using the old version of the module, which is loaded during the import. Although it shows the line from the new file, since all it has in the memory is the bytecode with metadata and has to refer to the disk when error happens. If you want to pick your latest changes without restarting the shell, run: >>> reload(codon_usage) and try again. (A sidenote: dict has no method deepcopy, that function comes from the module copy. dict.copy is enough here, though).