How to put together datas into a file? - python

I would like to collect different type of datas into a file. Here is a part of the code.
val = str(float(data[-1]))
val_dB = float(val)
val_dB = math.log(val_dB, 10) * 10
myfile = open('../../../MLI_values/mli_value.txt', 'a')
myfile.write(date_ID + " " + val + val_dB + "\n")
myfile.close()
But it gives back an error:
myfile.write(date_ID + " " + val + val_dB + "\n")
TypeError: cannot concatenate 'str' and 'float' objects
How can I solve it to put them together? (into columns) into a file?

Change:
myfile.write(date_ID + " " + val + val_dB + "\n")
to:
myfile.write(date_ID + " " + val + " " + str(val_dB) + "\n")

Related

How to select specific data from CSV files and display all the data in that row

I'm trying to make a code that shows all the details of specific students. The code takes an input of gender and their class and then shows all their data. For example, if I input "Male" and "10A", I want the code to give me all the data from the students who are male and in class 10A. All their information is stored in a CSV file (called details). My code so far is:
file = open("details.csv","rt")
gender_input = input("Input gender of students")
class_input = input("Input class of students")
for line in file:
details_of_gender = line.split(",")
details_of_class = line.split(",")
gender = str(details_of_gender[7])
class1 = str(details_of_class[8])
if gender == "Male":
if class1 == "10A":
print(details[0] + " " + details[1] + " " + details[2] + " " + details[3] + " " + details[4] + " " + details[5])
if class1 == "10B":
print(details[0] + " " + details[1] + " " + details[2] + " " + details[3] + " " + details[4] + " " + details[5])
if gender == "Female":
if class1 == "10A":
print(details[0] + " " + details[1] + " " + details[2] + " " + details[3] + " " + details[4] + " " + details[5])
if class1 == "10B":
print(details[0] + " " + details[1] + " " + details[2] + " " + details[3] + " " + details[4] + " " + details[5])
Assuming that your program isn't working just because you're using a variable details that you haven't defined before, here's a better way of doing this:
import csv
with open("details.csv", newline="") as infile:
reader = csv.reader(infile) # use the csv module to read a CSV file!
for row in reader:
gender = row[7]
class = row[8]
if gender in ("Male", "Female") and class in ("10A", "10B"):
print(" ".join(row[:6])) # join elements 0-5 with spaces

Timestamp into output filename

I would like to insert the timestamp into an output filename of a python script, for example: 20011231_230159_md5_filelist.csv
I am having trouble inserting the code.
This is the end of the script whose output filename needs to have a timestamp:
try:
my_last_data = get_md5(file_full_path) + ", " + get_last_write_time(file_full_path) + ", " + get_size(
file_full_path) + ", " + file_full_path + "\n"
with open("md5_filelist.csv", "a") as my_save_file:
my_save_file.write(my_last_data)
print(str(file_full_path) + " ||| Done")
except:
print("Error On " + str(file_full_path))
This is the timestamping code I am battling with (although not sure if it is the best line for the purpose):
timestr = time.strftime("%Y%m%d_%H%M%S")
I tried inserting in various ways, does not work. Any hints?
Thank you to #pm-2ring (see comments), the solution:
timestr = time.strftime("%Y%m%d_%H%M%S")
(timestr + "_md5_filelist.csv", "a")
in the script:
try:
timestr = time.strftime("%Y%m%d_%H%M%S")
my_last_data = get_md5(file_full_path) + ", " + get_last_write_time(file_full_path) + ", " + get_size(
file_full_path) + ", " + file_full_path + "\n"
with open(timestr + "_md5_filelist.csv", "a") as my_save_file:
my_save_file.write(my_last_data)
print(str(file_full_path) + " ||| Done")
except:
print("Error On " + str(file_full_path))
This is all you need.
try:
my_last_data = get_md5(file_full_path) + ", " + get_last_write_time(file_full_path) + ", " + get_size(
file_full_path) + ", " + file_full_path + "\n"
with open("{}md5_filelist.csv".format(timestr), "a") as my_save_file:
my_save_file.write(my_last_data)
print(str(file_full_path) + " ||| Done")
except:
print("Error On " + str(file_full_path))
You were not using timestr anywhere in you code.

Adding the values of two strings using Python and XML path

It generates an output with wallTime and setupwalltime into a dat file, which has the following format:
24000 4 0
81000 17 0
192000 59 0
648000 250 0
1536000 807 0
3000000 2144 0
6591000 5699 0
I would like to know how to add the two values i.e.(wallTime and setupwalltime) together. Can someone give me a hint? I tried converting to float, but it doesn’t seem to work.
import libxml2
import os.path
from numpy import *
from cfs_utils import *
np=[1,2,3,4,5,6,7,8]
n=[20,30,40,60,80,100,130]
solver=["BiCGSTABL_iluk", "BiCGSTABL_saamg", "BiCGSTABL_ssor" , "CG_iluk", "CG_saamg", "CG_ssor" ]# ,"cholmod", "ilu" ]
file_list=["eval_BiCGSTABL_iluk_default", "eval_BiCGSTABL_saamg_default" , "eval_BiCGSTABL_ssor_default" , "eval_CG_iluk_default","eval_CG_saamg_default", "eval_CG_ssor_default" ] # "simp_cholmod_solver_3D_evaluate", "simp_ilu_solver_3D_evaluate" ]
for cnt_np in np:
i=0
for sol in solver:
#open write_file= "Graphs/" + "Np"+ cnt_np + "/CG_iluk.dat"
#"Graphs/Np1/CG_iluk.dat"
write_file = open("Graphs/"+ "Np"+ str(cnt_np) + "/" + sol + ".dat", "w")
print("Reading " + "Graphs/"+ "Np"+ str(cnt_np) + "/" + sol + ".dat"+ "\n")
#loop through different unknowns
for cnt_n in n:
#open file "cfs_calculations_" + cnt_n +"np"+ cnt_np+ "/" + file_list(i) + "_default.info.xml"
read_file = "cfs_calculations_" +str(cnt_n) +"np"+ str(cnt_np) + "/" + file_list[i] + ".info.xml"
print("File list" + file_list[i] + "vlaue of i " + str(i) + "\n")
print("Reading " + " cfs_calculations_" +str(cnt_n) +"np"+ str(cnt_np) + "/" + file_list[i] + ".info.xml" )
#read wall and cpu time and write
if os.path.exists(read_file):
doc = libxml2.parseFile(read_file)
xml = doc.xpathNewContext()
walltime = xpath(xml, "//cfsInfo/sequenceStep/OLAS/mechanic/solver/summary/solve/timer/#wall")
setupwalltime = xpath(xml, "//cfsInfo/sequenceStep/OLAS/mechanic/solver/summary/setup/timer/#wall")
# cputime = xpath(xml, "//cfsInfo/sequenceStep/OLAS/mechanic/solver/summary/solve/timer/#cpu")
# setupcputime = xpath(xml, "//cfsInfo/sequenceStep/OLAS/mechanic/solver/summary/solve/timer/#cpu")
unknowns = 3*cnt_n*cnt_n*cnt_n
write_file.write(str(unknowns) + "\t" + walltime + "\t" + setupwalltime + "\n")
print("Writing_point" + str(unknowns) + "%f" ,float(setupwalltime ) )
doc.freeDoc()
xml.xpathFreeContext()
write_file.close()
i=i+1
In java you can add strings and floats. What I understand is that you need to add the values and then display them. That would work (stringing the sum)
write_file.write(str(unknowns) + "\f" + str(float(walltime) + float(setupwalltime)) + "\n")
You are trying to add a str to a float. That doesn't work. If you want to use string concatenation, first coerce all of the values to str. Try this:
write_file.write(str(unknowns) + "\t" + str(float(walltime) + float(setupwalltime)) + "\n")
Or, perhaps more readably:
totalwalltime = float(walltime) + float(setupwalltime)
write_file.write("{}\t{}\n".format(unknowns, totalwalltime))

Python - Write Header Once - Loop isn't working properly

I am trying to write an "order file" with a Header and Detail Lines. I am successfully getting the order Header to write to file, but only one detail line seems to get written to the file.
for k, v in atlantic_billing.iteritems():
XHORNO = str(digits + counter)
XHCSNO = k
print XHCSNO
machines = v
line = 1
counter = counter + 1
header_written = False
try :
for machine in machines :
XDORNO = XHORNO
XDORSQ = line
line = line + 1
XDITD1 = ranpak_dict[machine]['MODEL']
XDITD2 = ranpak_dict[machine]['SN']
XDCAVC = ranpak_dict[machine]['TOTAL']
print XDORSQ, XDITD1, XDITD2, XDCAVC
if XDCAVC > 0 :
if header_written == False :
with open(XHORNO + ".txt", 'w') as order:
order.write("H01, " + XHORNO + ", " + XHCSNO + "\n")
order.write("D01," + str(XDORSQ) + ", " + ' EQPRANUSER, ' + XDITD1 + ", " + XDITD2 + ", " + XDCAVC + "\n")
else :
order.write("D01," + str(XDORSQ) + ", " + ' EQPRANUSER, ' + XDITD1 + ", " + XDITD2 + ", " + XDCAVC + "\n")
success.append(machine)
header_written = True
except KeyError, e:
issues.append(machine)
When opening the file, you should use the mode "A" else the file will be overwritten at each loop (again and again):
with open(XHORNO + ".txt", 'a') as order:
...
see https://docs.python.org/3/library/functions.html#open
An other option is to take the withblock over the for block.
You need to open the file only once and not inside the loop
for k, v in atlantic_billing.iteritems():
XHORNO = str(digits + counter)
with open(XHORNO + ".txt", 'w') as order: # <--- here you go
XHCSNO = k
print XHCSNO
machines = v
line = 1
counter = counter + 1
header_written = False
try :
for machine in machines :
XDORNO = XHORNO
XDORSQ = line
line = line + 1
XDITD1 = ranpak_dict[machine]['MODEL']
XDITD2 = ranpak_dict[machine]['SN']
XDCAVC = ranpak_dict[machine]['TOTAL']
print XDORSQ, XDITD1, XDITD2, XDCAVC
if XDCAVC > 0 :
if header_written == False :
order.write("H01, " + XHORNO + ", " + XHCSNO + "\n")
order.write("D01," + str(XDORSQ) + ", " + ' EQPRANUSER, ' + XDITD1 + ", " + XDITD2 + ", " + XDCAVC + "\n")
else :
order.write("D01," + str(XDORSQ) + ", " + ' EQPRANUSER, ' + XDITD1 + ", " + XDITD2 + ", " + XDCAVC + "\n")
success.append(machine)
header_written = True
except KeyError, e:
issues.append(machine)
Try this:
for k, v in atlantic_billing.iteritems():
XHORNO = str(digits + counter)
XHCSNO = k
print XHCSNO
machines = v
line = 1
counter = counter + 1
header_written = False
try :
for machine in machines :
XDORNO = XHORNO
XDORSQ = line
line = line + 1
XDITD1 = ranpak_dict[machine]['MODEL']
XDITD2 = ranpak_dict[machine]['SN']
XDCAVC = ranpak_dict[machine]['TOTAL']
print XDORSQ, XDITD1, XDITD2, XDCAVC
if XDCAVC > 0 :
with open(XHORNO + ".txt", 'a') as order:
if header_written == False :
order.write("H01, " + XHORNO + ", " + XHCSNO + "\n")
order.write("D01," + str(XDORSQ) + ", " + ' EQPRANUSER, ' + XDITD1 + ", " + XDITD2 + ", " + XDCAVC + "\n")
header_written = True
else :
order.write("D01," + str(XDORSQ) + ", " + ' EQPRANUSER, ' + XDITD1 + ", " + XDITD2 + ", " + XDCAVC + "\n")
success.append(machine)
except KeyError, e:
issues.append(machine)

Python 2.7 IndexError: list index out of range, converting file

I have a code for convert Jmeter JTL FILE TO CSV, but when I run the code, I have the following error: IndexError: list index out of range in line 32
This is the code
import sys
import re
import datetime
import time
startTime = time.time()
cnt = 0
cnt2 = 0
failCnt = 0
reCompile = re.compile("\s([^\s]*?)=\"(.*?)\"")
delimiterCharacterOut = ","
def writeCSVLine(line):
x = reCompile.findall(line)
a = dict((row[0], row[1]) for row in x)
try:
a['ts1'] = str(int(int(a['ts'])/1000))
x = str(datetime.datetime.fromtimestamp(float(a['ts1'])))[0:19]
b = a['ts'] + ",\"" + x + "\"," + a['t'] + "," + a['lt'] + ",\"" + a['s'] + "\",\"" + a['lb'] + "\"," + a['rc'] + ",\"" + a['rm'] + "\",\"" + a['tn'] + "\",\"" + a['dt'] + "\"," + a['by'] + ",\"" + a['sc'] + "\"," + a['ec'] + ",\"" + a['ng'] + "\"," + a['na'] + ",\"" + a['hn'] + "\"," + a['in'] + "\n"
except:
return -1
o.write(b)
return 1
print "Splitting JTL file"
try:
runArgv = sys.argv #Save the command line
jtlInfile = str(sys.argv[1]) #Name of JTL input file
cvsOutfile = str(sys.argv[2]) # Name of CVS output file
reFilter = str(sys.argv[3]) # Filter the labels (lb) for the filter
except:
print "Error: Input format: <input file> <output file> <Filter by regular expression>"
raise
try:
f = open(jtlInfile, "r")
o = open(cvsOutfile, "w")
except:
raise
print "Filtering on regular expression : " + reFilter
cmpFilter = re.compile(reFilter)
# o.write("timestamp" + ",\""+ "datetime" + "\n")
o.write("timeStamp" + ",\"" + "datetime" + "\"," + "elapsed" + "," + "Latency" + ",\"" + "success" + "\",\"" + "label" + "\"," + "responseCode" + ",\"" + "responseMessage" + "\",\"" + "threadName"+ "\",\"" + "dataType" + "\"," + "bytes" + ",\"" + "SampleCount" + "\"," + "ErrorCount" + ",\"" + "grpThreads" + "\"," + "allThreads" + ",\"" + "Hostname" + "\"," + "IdleTime" + "\n")
for line in f:
try:
if cmpFilter.search(line):
returnVal = writeCSVLine(line)
if returnVal<0:
failCnt += 1
else:
cnt2 += 1
except:
print 'Error in line : ', cnt, line
raise
cnt += 1
endTime = time.time()
print "Time taken : ", str(endTime-startTime)
print "Lines processed : ", cnt
print "Lines that passed the filter : ", cnt2
print "Lines skipped (error?) : ", failCnt
f.close()
o.close()
Log de CMD
The base tutorial is in : http://balasoftwaretesting.blogspot.com/2012/03/converting-jmeter-jtl-file-to-csv-file.html?spref=bl
From the sys.argv docs, sys.argv is the list of command line arguments passed to a Python script.
Your command line log shows that you ran python JtltoCsv_Jmeter.py, which would result in an empty list for sys.argv. The tutorial provides a jtl file as an argument to JtltoCsv_Jmeter.py:
JtltoCsv_Jmeter.py C:\JtlToCsvConverter\input\sample.jtl
So it looks like maybe an error in copy/paste :)
Looking into the script, you need to pass 3 command line arguments:
Source JTL file
Target CSV file
Regular expression filter
So you need to execute the script like:
JtltoCsv_Jmeter.py example.jtl example.csv "(.*)"
Also there is an option to switch JMeter's results output format to CSV, in order to do so use one of the following steps:
Add jmeter.save.saveservice.output_format=csv line to user.properties file (lives under /bin folder of your JMeter installation)
Pass the property value via -J command line argument as:
jmeter -Jjmeter.save.saveservice.output_format=csv
See Apache JMeter Properties Customization Guide for more information on JMeter properties and ways of passing, setting and overriding them.

Categories