I have a list of input files and an expected output file, I want to write an auto-grader that does the job of accepting a python program, running it on the input files, and comparing its output to the output file. The approach I have used is to use the os module of python to run the program using os.system('python program.py > actual.out') and then perform a diff between the output and expected.out again using os.system().
The problem which I am currently facing is reading the input from the file because the program which is given is reading from the console. So, how should I redirect the input from a file such that it is readable by sys.stdin in program.py.
import os
def grade(program_py_file_handler,input_dir,output_dir):
#create temporary file for program.py using program_py_file_handler
#one by one read all files from input_dir
#run program.py using os.system generating a temp file
#do diff be temp file and expected file
Is there a better way to perform a diff without using the diff command?
To redirect output from program.py to a file I used python program.py>tem.out. What equivalent should I use to redirect an input file to progam.py such that wherever I have used sys.stdin in program.py it will instead read from the passed file? (Modifying program.py is not an option.)
You can be doing everything using builtin modules in Python 3.3+, since you are effectively spinning up a subprocess and doing a diff on the expected output. Simple minimum example:
check.py
import sys
from subprocess import check_output
from difflib import ndiff
def split(s):
return s.splitlines(keepends=True)
def check(program_name, expected):
output = check_output([sys.executable, program_name]).decode('utf8')
diff = ndiff(split(output), split(expected))
print(''.join(diff), end="")
def main():
check('hello.py', 'Good Morning!\n')
if __name__ == '__main__':
main()
hello.py
print('Good Evening!')
Example run
$ python check.py
- Good Evening!
? ^^^
+ Good Morning!
? ^^^
Modify as you see fit, with other methods/functions in the libraries linked. If you need stdin for the subprocess you probably will need to create Popen object and call communicate, but please read documentations first for future reference.
Related
I have two Python files (main.py and main_test.py). The file main_test.py is executed within main.py. When I do not use a log file this is what gets printed out:
Main file: 17:41:18
Executed file: 17:41:18
Executed file: 17:41:19
Executed file: 17:41:20
When I use a log file and execute main.py>log, then I get the following:
Executed file: 17:41:18
Executed file: 17:41:19
Executed file: 17:41:20
Main file: 17:41:18
Also, when I use python3 main.py | tee log to print out and log the output, it waits and prints out after finishing everything. In addition, the problem of reversing remains.
Questions
How can I fix the reversed print out?
How can I print out results simultaneously in terminal and log them in a correct order?
Python files for replication
main.py
import os
import time
import datetime
import pytz
python_file_name = 'main_test'+'.py'
time_zone = pytz.timezone('US/Eastern') # Eastern-Time-Zone
curr_time = datetime.datetime.now().replace(microsecond=0).astimezone(time_zone).time()
print(f'Main file: {curr_time}')
cwd = os.path.join(os.getcwd(), python_file_name)
os.system(f'python3 {cwd}')
main_test.py
import pytz
import datetime
import time
time_zone = pytz.timezone('US/Eastern') # Eastern-Time-Zone
for i in range(3):
curr_time = datetime.datetime.now().replace(microsecond=0).astimezone(time_zone).time()
print(f'Executed file: {curr_time}')
time.sleep(1)
When you run a script like this:
python main.py>log
The shell redirects output from the script to a file called log. However, if the script launches other scripts in their own subshell (which is what os.system() does), the output of that does not get captured.
What is surprising about your example is that you'd see anything at all when redirecting, since the output should have been redirected and no longer echo - so perhaps there's something you're leaving out here.
Also, tee waits for EOF on standard in, or for some error to occur, so the behaviour you're seeing there makes sense. This is intended behaviour.
Why bother with shells at all though? Why not write a few functions to call, and import the other Python module to call its functions? Or, if you need things to run in parallel (which they didn't in your example), look at multiprocessing.
In direct response to your questions:
"How can I fix the reversed print out?"
Don't use redirection, and write to file directly from the script, or ensure you use the same redirection when calling other scripts from the first (that will get messy), or capture the output from the subprocesses in the subshell and pipe it to the standard out of your main script.
"How can I print out results simultaneously in terminal and log them in a correct order?"
You should probably just do it in the script, otherwise this is not a really a Python question and you should try SuperUser or similar sites to see if there's some way to have tee or similar tools write through live.
In general though, unless you have really strong reasons to have the other functionality running in other shells, you should look at solving your problems in the Python script. And if you can't, use you can use something like Popen or derivatives to capture the subscript's output and do what you need instead of relying on tools that may or may not be available on the host OS running your script.
For the life of me i can't figure this one out.
I have 2 applications build in python, so 2 projects in different folders, is there a command to say in the first application like run file2 from documents/project2/test2.py ?
i tried something like os.system('') and exec() but that only seems to work if its in the same folder. How can i give a command a path like documents/project2 and then for example:
exec(documents/project2 python test2.py) ?
short version:
Is there a command that runs python test2.py while that test2 is in a completely different file/project?
thnx for all feedback!
There's a number of approaches to take.
1 - Import the .py
If the path to the other Python script can be made relative to your project, you can simply import the .py. This will cause all the code at the 'root' level of the script to be executed and makes functions as well as type and variable definitions available to the script importing it.
Of course, this only works if you control how and where everything is installed. It's the most preferable solution, but only works in limited situations.
import ..other_package.myscript
2 - Evaluate the code
You can load the contents of the Python file like any other text file and execute the contents. This is considered more of a security risk, but given the interpreted nature of Python in normal use not that much worse than an import under normal circumstances.
Here's how:
with open('/path/to/myscript.py', 'r') as f:
exec(f.read())
Note that, if you need to pass values to code inside the script, or out of it, you probably want to use files in this case.
I'd consider this the least preferable solution, due to it being a bit inflexible and not very secure, but it's definitely very easy to set up.
3 - Call it like any other external program
From a Python script, you can call any other executable, that includes Python itself with another script.
Here's how:
from subprocess import run
run('python path/to/myscript.py')
This is generally the preferable way to go about it. You can use the command line to interface with the script, and capture the output.
You can also pipe in text with stdin= or capture the output from the script with stdout=, using subprocess.Popen directly.
For example, take this script, called quote.py
import sys
text = sys.stdin.read()
print(f'In the words of the poet:\n"{text}"')
This takes any text from standard in and prints them with some extra text, to standard out like any Python script. You could call it like this:
dir | python quote.py
To use it from another Python script:
from subprocess import Popen, PIPE
s_in = b'something to say\nright here\non three lines'
p = Popen(['python', 'quote.py'], stdin=PIPE, stdout=PIPE)
s_out, _ = p.communicate(s_in)
print('Here is what the script produced:\n\n', s_out.decode())
Try this:
exec(open("FilePath").read())
It should work if you got the file path correct.
Mac example:
exec(open("/Users/saudalfaris/Desktop/Test.py").read())
Windows example:
exec(open("C:\Projects\Python\Test.py").read())
I am revising a script that currently calls other scripts as subprocesses. Instead of doing that, I'm creating functions inside of the main script to perform the tasks previously performed by subprocesses.
One of the subprocesses required that variables be passed to it as you would from the command line.
Here is the code calling the subprocess:
subprocess.call("python cleaner.py < ./results/Temp.csv
>./results/result.csv", shell=True)
os.remove("./results/Temp.csv")
Here is what I'm trying to do:
def cleaner():
#working code that cleans certain characters out of selected
#.csv files.
function("./results/Temp.csv > ./results/result.csv", shell=True)
os.remove("./resluts/Temp.csv")
Ideally I'd like to use the existing code from the subprocess, but I'm open to changing it if that makes solving the problem easier. Here is that code:
from __future__ import print_function
from sys import stdin
print(next(stdin) , end='')
for line in stdin:
toks = [tok.replace("\'",""
).replace("text:u","").replace("number:", "") for tok in
line.split()]
print(' '.join(toks))
The script should clean the specified temp file, copy the cleaned version to a results file, then delete the temp file. Currently it works as a subprocess, but not when I try to run it as a function. I pass the variables incorrectly and it throws this error:
'TypeError: cleaner() takes no arguments (1 given)'
You need to define the arguments as part of the function.
def cleaner(argument1):
#working code that cleans certain characters out of selected
#.csv files.
Read more here.
To start off, I am a beginner in python so I am not even sure if my question makes sense or is even possible.
I have 2 python files app.py. and compare.py. compare.py takes in two arguments (File paths) to run. So for example, when I want to run it, I do python compare.py ./image1.jpg ./image2.jpg. Now the return I get is some text printed to the terminal such as Comparison Done, The distance is 0.544.
Now, I want to run this compare.py from inside app.py and get a string with whatever compare.py would usually output to the terminal. So for example:
result = function('compare.py ./image1.jpg ./image2.jpg') and result will have the required string. Is this possible?
You can use os.popen:
In app.py:
import os
output = os.popen('python compare.py ./image1.jpg ./image2.jpg').readlines()
I got a Python script (test1.py) I need to run with a bat file (render.bat).
Question 1:
First, I had a definition in my test1.py and it always failed to run, nothing happened. Can someone kindly explain why?
import os
def test01 :
os.system('explorer')
and in the bat file:
python c:/test01.py
but as soon as I removed the def it worked. I just want to learn why this happened.
Question 2:
How can I take "render" string from render.bat as a string input for my python script so I can run something like :
import os
def test1(input) :
os.system("explorer " + input)
So the "input" is taken from the .BAT filename?
Functions don't actually do anything unless you call them. Try putting test01() at the end of the script.
%0 will give you the full name of the batch file called, including the .bat. Stripping it will probably be easier in Python than in the batch file.
Question1: Keyword def in python defines a function. However, to use a function you have to explicitly call it, i.e.
import os
def test01(): # do not forget ()
os.system('explorer')
test01() # call the function
1) You have to actually call the functions to achieve your task.
2) %0 refers to the running script. Therefor create a test.bat file like
# echo off
echo %0
Output = test.bat
You can strip the .bat extension from the output.