We Keep Coding

How do you use a python variable in popen()?

Im trying to record docker stats for every file in the mydata directory. For example if one of the files is names piano.txt I would like the output file to be piano_stuff.txt. This is what I have so far:
import subprocess
import signal
import os
for file_name in os.listdir('mydata'):
data_txt = "./" + file_name.split(".")[0] + "_stuff.txt"
dockerStats = subprocess.Popen("docker stats --format {{.MemUsage}} >> ${data_txt}", shell=True)
os.killpg(os.getpgid(dockerStats.pid), signal.SIGTERM)

Don't use shell=True. Open the file locally, and pass the file object as the stdout argument. You can also use the --no-stream option to have the command exit after producing one line of output, rather than asynchronously trying to kill the process as soon as possible. (You might get multiple lines of output, or you might get none, depending on when the OS schedules the Docker process to run.)
with open(data_txt, "a") as f:
subprocess.run(["docker", "stats", "--format", "{{.MemUsage}}", "--no-stream"], stdout=f)

How do I run another file in python?

So I know how to write in a file or read a file but how do I RUN another file?
for example in a file I have this:
a = 1
print(a)
How do I run this using another file?

file_path = "<path_to_your_python_file>"
using subprocess standard lib
import subprocess
subprocess.call(["python3", file_path])
or using os standard lib
import os
os.system(f"python3 {file_path}")
or extract python code from the file and run it inside your script:
with open(file_path, "r+", encoding="utf-8") as another_file:
python_code = another_file.read()
# running the code inside the file
exec(python_code)
exec is a function that runs python strings exactly how python interpreter runs python files.
IN ADDITION
if you want to see the output of the python file:
import subprocess
p = subprocess.Popen(
["python3", file_path],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
err, output = p.communicate()
print(err)
print(output)
EXTRA
for people who are using python2:
execfile(file_path)
exec_file documentation

Python checking integrity of gzip archive

Is there a way in Python using gzip or other module to check the integrity of the gzip archive?
Basically is there equivalent in Python to what the following does:
gunzip -t my_archive.gz

Oops, first answer (now deleted) was result of misreading the question.
I'd suggest using the gzip module to read the file and just throw away what you read. You have to decode the entire file in order to check its integrity in any case. https://docs.python.org/2/library/gzip.html
Something like ( Untested code)
import gzip
chunksize=10000000 # 10 Mbytes
ok = True
with gzip.open('file.txt.gz', 'rb') as f:
try:
while f.read(chunksize) != b'':
pass
except:
ok = False
I don't know what exception reading a corrupt zipfile will throw, you might want to find out and then catch only this particular one.

you can use subprocess or os module to execute this command and read the output. something like this
Using os module
import os
output = os.popen('gunzip -t my_archive.gz').read()
Using Subprocess Module
import subprocess
proc = subprocess.Popen(["gunzip", "-t", "my_archive.gz"], stdout=subprocess.PIPE, shell=True)
(out, err) = proc.communicate()

Getting name of extracted file after unzipping

I'm trying to assign the name of the file that is extracted from the zip as a string that I can pass around.
The following is my code so far:
cl = '7z x -ppassowrd Week45_10.zip'
args = shlex.split(cl)
p = subprocess.call(args)
file = open('VENDATA_10', 'r')
VENDATA_10 is the name of the file that is extracted, however, the name of the file extracted from the zip will vary. How do I find the name of the extracted file so that I can open whatever that file is called instead of VENDATA_10?

You could capture the output from the process. 7z will print the filenames that it extracts to the standard output. The output might look like this:
7-Zip 4.44 beta Copyright (c) 1999-2007 Igor Pavlov 2007-01-20
p7zip Version 4.44 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs)
Processing archive: Week45_10.zip
Extracting VENDATA_10
Everything is Ok
So you want to capture the lines that start with "Extracting ". To capture the output you'll need to pipe the output from the process into you program and read it. You can used communicate() from subprocess to get the full output.
I've extended you code a bit to do this:
from subprocess import Popen, PIPE
import shlex
cl = '7z x -ppassowrd Week45_10.zip'
args = shlex.split(cl)
p = Popen(args, stdin=PIPE, stdout=PIPE, stderr=PIPE)
(stdoutdata, stderrdata) = p.communicate()
START_WITH = "Extracting "
N = len(START_WITH)
lines = [line.strip() for line in stdoutdata.split('\n')]
files = [line[N:] for line in lines if line.startswith(START_WITH)]
print files
You should be aware that there might arise problems if the files already exist, because then 7z will prompt the user if it should overwrite the files. If you want to handle that then you need to control the input to the 7z process also. You might want to look at the pexpect module for that.

How to use tempfile.NamedTemporaryFile() in Python

I want to use tempfile.NamedTemporaryFile() to write some contents into it and then open that file. I have written following code:
tf = tempfile.NamedTemporaryFile()
tfName = tf.name
tf.seek(0)
tf.write(contents)
tf.flush()
but I am unable to open this file and see its contents in Notepad or similar application. Is there any way to achieve this? Why can't I do something like:
os.system('start notepad.exe ' + tfName)
at the end.
I don't want to save the file permanently on my system. I just want the contents to be opened as a text in Notepad or similar application and delete the file when I close that application.

This could be one of two reasons:
Firstly, by default the temporary file is deleted as soon as it is closed. To fix this use:
tf = tempfile.NamedTemporaryFile(delete=False)
and then delete the file manually once you've finished viewing it in the other application.
Alternatively, it could be that because the file is still open in Python Windows won't let you open it using another application.
Edit: to answer some questions from the comments:
As of the docs from 2 when using delete=False the file can be removed by using:
tf.close()
os.unlink(tf.name)

You can also use it with a context manager so that the file will be closed/deleted when it goes out of scope. It will also be cleaned up if the code in the context manager raises.
import tempfile
with tempfile.NamedTemporaryFile() as temp:
temp.write('Some data')
temp.flush()
# do something interesting with temp before it is destroyed

Here is a useful context manager for this.
(In my opinion, this functionality should be part of the Python standard library.)
# python2 or python3
import contextlib
import os
#contextlib.contextmanager
def temporary_filename(suffix=None):
"""Context that introduces a temporary file.
Creates a temporary file, yields its name, and upon context exit, deletes it.
(In contrast, tempfile.NamedTemporaryFile() provides a 'file' object and
deletes the file as soon as that file object is closed, so the temporary file
cannot be safely re-opened by another library or process.)
Args:
suffix: desired filename extension (e.g. '.mp4').
Yields:
The name of the temporary file.
"""
import tempfile
try:
f = tempfile.NamedTemporaryFile(suffix=suffix, delete=False)
tmp_name = f.name
f.close()
yield tmp_name
finally:
os.unlink(tmp_name)
# Example:
with temporary_filename() as filename:
os.system('echo Hello >' + filename)
assert 6 <= os.path.getsize(filename) <= 8 # depending on text EOL
assert not os.path.exists(filename)