I have the following piece of code that generates plots with gnuplot:
import sys, glob, subprocess, os, time
for file in glob.glob('comb_*.out'):
fNameParts = file[5:].split('.')[0].split('-')
gPlotCmd = []
gPlotCmd = 'unset border; set xl "rotation period #"; set yl "T [K]"\n'
gPlotCmd += 'set key above\n'
gPlotCmd += 'set grid xtics ytics\n'
gPlotCmd += 'set term post eps enh color solid\n'
gPlotCmd += 'set xrange [20:876]\n'
gPlotCmd += 'set output "fig_TestRelax-' + fNameParts[1] + '-' + fNameParts[2] + '-' + fNameParts[3] + '-' + fNameParts[4] + '.eps"\n'
conf = fNameParts[1] + '-' + fNameParts[2] + '-' + fNameParts[3]
gPlotCmd += 'plot "' + file + '" using ($1/36000):($9-$3) w l lw 5 title "OP3-OP1 ' + conf + '", "' + file + '" using ($1/36000):($6-$3) w l lw 3 title "OP2-OP1 ' + conf + '", "' + file + '" using ($1/36000):($9-$6) w l lw 1 title "OP3-OP2 ' + conf + '"'
fw = open('temp.plt','w+')
fw.writelines(gPlotCmd)
subprocess.call(["gnuplot","temp.plt"])
print(file)
In the last loop execution the subprocess.call(["gnuplot","temp.plt"]) does not seem to be executed. At the end of the program, temp.plt exists with data from the last iteration. Also print(file) is executed during the last loop. Also if I plot the temp.plt left after the program I get the last plot (so there is no problem on the side of the data). Only the line subprocess.call(["gnuplot","temp.plt"]) is not executed. I also tried popen and monitor stdout and stderr but both were empty (as in all other iterations.
The checked the problem occurs both on linux and windows and in versions 3.3.5 and 2.7.3.
I conclude that there is something wrong with the script but I do not know what.
#lvc's and yours answer are correct; it is a buffering issue and fw.flush() should fix it. But you don't need the temporary file, you could pass the input commands to gnuplot directly without writing them to disk:
from subprocess import Popen, PIPE
p = Popen('gnuplot', stdin=PIPE)
p.communicate(input=gPlotCmd.encode('ascii'))
One likely error here is that the file temp.plt isn't actually written to disk at the time you run gnuplot. Python doesn't necessarily flush its buffers immediately after a call to writelines. This would mean that when gnuplot is launched from your script, it sees an empty file. It doesn't give an error, because an empty input isn't an error, and it has no way of knowing its expecting anything else. By the time you run it outside of your script, Python has exited and hence can't be holding anything in its own buffers anymore.
Use the with statement to ensure that fw is closed once you're done with it:
with open('temp.plt', 'w') as fw:
fw.writelines(gPlotCmd)
subprocess.call(["gnuplot","temp.plt"])
It seems I figured that out. I am missing fw.close(). The last lines of code should look:
fw = open('temp.plt','w+')
fw.writelines(gPlotCmd)
fw.close()
subprocess.call(["gnuplot","temp.plt"])
print(file)
Now the code produces the intended plots.
Related
I have the code:
import sys
for i in range(0, 20):
for j in range(0, 20):
sys.stdout.write('\x1b[1;32;40m' + ' ' + '\x1b[0m')
sys.stdout.write("\n")
which outputs 400 white squares in a 20x20 grid but also after about 180 squares outputs [1;32;40m in between some of the squares. It doesn't always output in the same place. Why is this happening, and how can it be fixed?
It's probably caused by some buffering bug in the terminal (I guess).
I can't replicate your issue, so here are some suggestions.
Output to sys.stdout is buffered. You could try flushing the output after each line of text ~
sys.stdout.flush()
But a better solution might be to add the bunch of escape codes to a string, then write it in a block ~
import sys
colour_line = '\x1b[1;32;40m' + (' ' * 20) + '\x1b[0m' + '\n'
for i in range(0, 20):
sys.stdout.write(colour_line)
This obviously simplifies the code, but this is getting away from your original design.
It's not ideal, but
time.sleep(0.0005)
or a different adjusted time between each print seems to work
I'm trying to write a script that when I run it, it will write the current time to a file, then I want it to stay open running in the background so that any time I press the Insert key it will write the current time to the file minus 7 minutes.
I have both these functions working as I want, however they only work individually. As in, it will only write the time it is when I run it if I mark out the block that keeps it open searching for a keypress (I'm using pyhook for that). And it will only stay open looking for a keypress if I mark out the part that would write the time it was when I first ran it. Here's my code
import pyHook, pythoncom
from datetime import datetime
#writes the time when it starts bit
t = datetime.now()
h = t.hour
m = t.minute
s = t.second
if h > 12:
h = h - 12
if m < 10:
m = str(m).zfill(2)
file = open('time.txt', 'w')
file.write('Jeremiah got on at ' + str(h) + ':' + str(m))
#stays open and looks for a keypress bit
hookManager = pyHook.HookManager()
def OnKeyboardEvent(event):
keyPressed = event.KeyID
if keyPressed == 45:
t = datetime.now()
h = t.hour
m = t.minute
s = t.second
if h > 12:
h = h - 12
if m < 10:
m = int(str(m).zfill(2))
if m - 7 < 0:
h = h - 1
sub = m - 7
m = 60 + sub
else:
m = m - 7
file = open('time.txt', 'w')
file.write('Jeremiah got on at ' + str(h) + ':' + str(m))
return True
hookManager.KeyDown = OnKeyboardEvent
hookManager.HookKeyboard()
pythoncom.PumpMessages()
If I mark out neither sections, it will clear the text file when it starts and then stay open to look for the keypress. Marking out just the startup time section will keep it open looking for the key properly, and it won't clear the text file. So I'm pretty certain it's some problem that once it starts looking for a key it clears the file, but I'm not exactly sure why, especially since it shouldn't even run any code to do with the text file until the key is pressed, setting off my if statement.
Is it possible to have these two in the same script, or do I have to separate them and call them both with a .bat or something?
Thanks in advance.
(also if you're wondering why in the world I would want a program that would do this, I'm supposed to share the desktop computer with my siblings, and we keep track of how long we've been on by writing down the time we got on. This way I should be able to automate it, as well as have a way to discreetly set my time forward when asked to get off.)
I got it. I just had to close the file after writing to it in the startup section.
file = open('time.txt', 'w')
file.write('Jeremiah got on at ' + str(h) + ':' + str(m))
file.close()
To make things easier you can do this so you don't have to bother about closing the file because it will just close on it's own.
With open ('time.txt', 'w') as file:
file.write('Jeremiah got on at ' + str(h) + ':' + str(m))
I am trying to calculate the sum of size of various files. This is my script:
import os
date = raw_input('Enter date in format YYYYMMDD ')
file1 = 'p_poupe_' + date + '.tar.gz.done'
file2 = 'p_poupw_' + date + '.tar.gz.done'
file3 = 'p_pojk_' + date + '.tar.gz.done'
a1 = os.system('zcat ' + file1 + '|wc --bytes')
a2 = os.system('zcat ' + file2 + '|wc --bytes')
a3 = os.system('zcat ' + file3 + '|wc --bytes')
print a1,a2,a3
sum = a1 + a2 + a3
print sum
But the values are not storing in variable. Can any one tell me what I am doing wrong. How can I modify script so that values are stored in variable and not as a output.
On Unix, the return value is the exit status of the process encoded in
the format specified for wait(). Note that POSIX does not specify the
meaning of the return value of the C system() function, so the return
value of the Python function is system-dependent.
On Windows, the return value is that returned by the system shell
after running command, given by the Windows environment variable
COMSPEC: on command.com systems (Windows 95, 98 and ME) this is always
0; on cmd.exe systems (Windows NT, 2000 and XP) this is the exit
status of the command run; on systems using a non-native shell,
consult your shell documentation.
https://docs.python.org/2/library/os.html#os.system
The problem is that you're using exit-codes rather than stdout data as your "values".
You're probably looking to use subprocess.Popen for instance. Or just simply code the solution manually by opening the files.
Try using https://docs.python.org/3/library/gzip.html
import gzip
def get_fcont_len(fname):
with gzip.open(fname) as f:
return len(f.read())
total = 0
date = raw_input('Enter date in format YYYYMMDD ')
total += get_fcont_len('p_poupe_' + date + '.tar.gz.done')
total += get_fcont_len('p_poupw_' + date + '.tar.gz.done')
total += get_fcont_len('p_pojk_' + date + '.tar.gz.done')
print(total)
os.system return the exit status of the command not the output of the command. To capture the output of a command you should look into the subprocess module.
subprocess.check_output("zcat " + file1 + " | wc --bytes", shell=True)
# Output the size in bytes of file1 with a trailing new line character
However it is probably better to use other python modules/methods to do that as suggested by other as it is preferable to do things directly in Python.
The uncompressed file size is stored in the last 4 bytes of the gzip file. This function will return the size of the uncompressed file, i.e. the "gunzipped" size:
import os
import gzip
import struct
def get_gunzipped_size(filename):
with gzip.open(filename) as f:
_ = f.read(1) # elicit IOError if file is not a gzip file
f.fileobj.seek(-4, os.SEEK_END)
return struct.unpack('<i', f.fileobj.read(4))[0]
On large files this is much faster than reading all of the uncompressed data and counting it's length because the whole file does not need to be decompressed.
Fitting this into your code:
import os
date = raw_input('Enter date in format YYYYMMDD ')
prefixes = ('p_poupe_', 'p_poupw_', 'p_pojk_')
files = ['{}{}.tar.gz.done'.format(prefix, date) for prefix in prefixes]
total_uncompressed = sum(get_gunzipped_size(f) for f in files)
print total_uncompressed
You can capture the output of a command using getoutput function from commands as:
import commands as cm
.
.
.
a1 = cm.getoutput('zcat ' + file1 + '|wc --bytes')
a2 = cm.getoutput('zcat ' + file2 + '|wc --bytes')
a3 = cm.getoutput('zcat ' + file3 + '|wc --bytes')
# Note that the outputs are in string format so you need to convert them to integers or floats
a1, a2, a3 = float(a1), float(a2), float(a3)
print a1,a2,a3
sum = a1 + a2 + a3
print sum
You can use the os module to get the file size. Try this:
import os
import tarfile
tar = tarfile.open("yourFile.tar.gz")
tar.extractall("folderWithExtractedFiles")
print os.path.getsize("folderWithExtractedFiles/yourFileInsideTarGz")
It has been awhile since I have written functions with for loops and writing to files so bare with my ignorance.
This function is given an IP address to read from a text file; pings the IP, searches for the received packets and then appends it to a .csv
My question is: Is there a better or an easier way to write this?
def pingS (IPadd4):
fTmp = "tmp"
os.system ("ping " + IPadd4 + "-n 500 > tmp")
sName = siteNF #sys.argv[1]
scrap = open(fTmp,"r")
nF = file(sName,"a") # appends
nF.write(IPadd4 + ",")
for line in scrap:
if line.startswith(" Packets"):
arrT = line.split(" ")
nF.write(arrT[10]+" \n")
scrap.close()
nF.close()
Note: If you need the full script I can supply that as well.
This in my opinion at least makes what is going on a bit more obvious. The len('Received = ') could obviously be replaced by a constant.
def pingS (IPadd4):
fTmp = "tmp"
os.system ("ping " + IPadd4 + "-n 500 > tmp")
sName = siteNF #sys.argv[1]
scrap = open(fTmp,"r")
nF = file(sName,"a") # appends
ip_string = scrap.read()
recvd = ip_string[ip_string.find('Received = ') + len('Received = ')]
nF.write(IPadd4 + ',' + recvd + '\n')
You could also try looking at the Python csv module for writing to the csv. In this case it's pretty trivial though.
This may not be a direct answer, but you may get some performance increase from using StringIO. I have had some dramatic speedups in IO with this. I'm a bioinformatics guy, so I spend a lot of time shooting large text files out of my code.
http://www.skymind.com/~ocrow/python_string/
I use method 5. Didn't require many changes. There are some fancier methods in there, but they didn't appeal to me as much.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Rename Files in Python
Hey all, I am creating a script in Python to wrap a number of OS commands related to bulk image editing, and most of the tasks are completed, however I am stumped on one task (a seemingly simple one at that).
When I scan pictures on my scanner, I get a filename similar to this:
201105151110_0001A.jpg
where the first part is some sort of a timestamp (which I want to replace), however I wanted to make that an input variable (where I, or other users could just paste that into the command line, if I'm using another scanner where the filename structure could be different. 0001A means the front side of the first photo/document, and I would like to keep that part of the filename.
I have three variables set up:
old_prefix = input(bcolors.PROMPT + "Enter the prefix to replace: " + bcolors.ENDC)
new_prefix = input(bcolors.PROMPT + "Enter the new prefix: " + bcolors.ENDC)
working_directory
working_directory is the variable from another part of the code, which is where the images would be located. The color part is just so I can colorize the output and make it easier to read. There would be anywhere from 1-1000 files in the directory when I work on it.
This script will be run on Linux.
Thank you!
---EDIT---
Sorry for wasting your time guys, it seems I overlooked some information in the question linked here by Kiril Kirov, the code that I came up with, which works is:
elif retouch_option == "06":
print(" ")
old_prefix = input(bcolors.PROMPT + "Enter the prefix to replace: " + bcolors.ENDC)
new_prefix = input(bcolors.PROMPT + "Enter the new prefix.......: " + bcolors.ENDC)
print(bcolors.OUTPUT + " ")
for fname in glob(working_directory + "*.jpg"):
keeper = fname[-9:]
print("Renaming image", keeper)
os.rename(fname, fname.replace(old_prefix, new_prefix))
I think it should be safe, since it is just replacing the old_prefix variable with the new_prefix variable. Is this true? If not I'd definitely appreciate feedback, although this seems to be working fine so far.
something like:
sep = '_'
try:
prefix,keeper = filename.split(sep)
except: # filename does not match desired structure
print "not processed: no '" + sep + "' in '"+ filename + "'"
else: # split succeeded:
if prefix == old_prefix:
filename = new_prefix + sep + keeper
# more processing...
else:
print "prefix doesn't match in '" + filename + "'"