I am revising a script that currently calls other scripts as subprocesses. Instead of doing that, I'm creating functions inside of the main script to perform the tasks previously performed by subprocesses.
One of the subprocesses required that variables be passed to it as you would from the command line.
Here is the code calling the subprocess:
subprocess.call("python cleaner.py < ./results/Temp.csv
>./results/result.csv", shell=True)
os.remove("./results/Temp.csv")
Here is what I'm trying to do:
def cleaner():
#working code that cleans certain characters out of selected
#.csv files.
function("./results/Temp.csv > ./results/result.csv", shell=True)
os.remove("./resluts/Temp.csv")
Ideally I'd like to use the existing code from the subprocess, but I'm open to changing it if that makes solving the problem easier. Here is that code:
from __future__ import print_function
from sys import stdin
print(next(stdin) , end='')
for line in stdin:
toks = [tok.replace("\'",""
).replace("text:u","").replace("number:", "") for tok in
line.split()]
print(' '.join(toks))
The script should clean the specified temp file, copy the cleaned version to a results file, then delete the temp file. Currently it works as a subprocess, but not when I try to run it as a function. I pass the variables incorrectly and it throws this error:
'TypeError: cleaner() takes no arguments (1 given)'
You need to define the arguments as part of the function.
def cleaner(argument1):
#working code that cleans certain characters out of selected
#.csv files.
Read more here.
Related
I am working to convert a code which is written in a code called " elegant" into another code.
I come by this lines, which i could not understand well, i am quit new to python
import os,sys
def computeOpticsD(quad_name, quad_id,k1):
ex = '/Users/ia/Products/elegant/darwin-x86/elegant'
args = ' compute_optics_d.ele -macro=lattice=fodo,quad_name='+quad_name+',quad_id='+str(quad_id)+',k1='+str(k1)
args += ',dir=data'
os.system(ex + args)
What dose it exactly do ?
os.system allow you to run OS commands as you would do for example in a script or in the terminal directly.
For example, if you run os.system("ls") it will return the list of files and directories in the location
In this case it seems your code fragment is trying to execute the following command in the OS:
/Users/ia/Products/elegant/darwin-x86/elegant compute_optics_d.ele -macro=lattice={fodo}, quad_name={quad_name}, quad_id={quad_id}, k1={k1}, dir=data
Where the elemnts in the key brakets come from your method parameters as defined in: def computeOpticsD(quad_name, quad_id,k1)
I'm working on cloning a Virtual Machine (VM) in vCenter environment using this code. It takes command line arguments for name of the VM, template, datastore, etc. (e.g. $ clone_vm.py -s <host_name> -p < password > -nossl ....)
I have another Python file where I've been able to list the Datastore volumes in descending order of free_storage. I have stored the datastore with maximum available storage in a variable ds_max. (Let's call this ds_info.py)
I would like to use ds_max variable from ds_info.py as a command line argument for datastore command line argument in clone_vm.py.
I tried importing the os module in ds_info.py and running os.system(python clone_vm.py ....arguments...) but it did not take the ds_max variable as an argument.
I'm new to coding and am not confident to change the clone_vm.py to take in the Datastore with maximum free storage.
Thank you for taking the time to read through this.
I suspect there is something wrong in your os.system call, but you don't provide it, so I can't check.
Generally it is a good idea to use the current paradigm, and the received wisdom (TM) is that we use subprocess. See the docs, but the basic pattern is:
from subprocess import run
cmd = ["mycmd", "--arg1", "--arg2", "val_for_arg2"]
run(cmd)
Since this is just a list, you can easily drop arguments into it:
var = "hello"
cmd = ["echo", var]
run(cmd)
However, if your other command is in fact a python script it is more normal to refactor your script so that the main functionality is wrapped in a function, called main by convention:
# script 2
...
def main(arg1, arg2, arg3):
do_the_work
if __name__ == "__main__":
args = get_sys_args() # dummy fn
main(*args)
Then you can simply import script2 from script1 and run the code directly:
# script 1
from script2 import main
args = get_args() # dummy fn
main(*args)
This is 'better' as it doesn't involve spawning a whole new python process just to run python code, and it generally results in neater code. But nothing stops you calling a python script the same way you'd call anything else.
I have a python script which when run, logs information on the terminal, i want to send this logging information to a text file,
To achieve this in the beginning of the file i am inserting
import subprocess
subprocess.call(['script', 'logfile'])
and at the end of the file,i put in,
subprocess.call(['exit'])
The problem with this is when it calls the first commandscript logfile,it terminates the script,
Any suggestions on how i could make this work would be really helpful,Thanks in advance
The problem is that subprocess.call isn't returning until the shell spawned by script exits, at which point your Python script will resume.
The simplest way to do what you want is to call script itself with your Python script as an argument. Instead of
#!/usr/bin/python
import subprocess
subprocess.call(['script', 'logfile'])
# Rest of your Python code
subprocess.call(['exit'])
you will use
#!/usr/bin/python
import os
import sys
if '_underscript' not in os.environ:
os.environ['_underscript'] = "yes"
cmd_args = ['script', 'logfile', 'python'] + sys.argv
os.execvp('script', cmd_args)
# Rest of your Python code
The environment variable prevents your script from entering an infinite loop of re-running itself with script. When you run your Python script, it first checks its environment for a variable that should not yet exist. If it doesn't, it sets that variable, then runs script to re-run the Python script. execvp replaces your script with the call to script; nothing else in the Python script executes. This second time your script runs, the variable _underscript does exist, meaning the if block is skipped and the rest of your script runs as intended.
Seems like that's the expected behaviour of subprocess.call(...). If you want to capture the output of the script to a file, you'll need to open a new file handler in write mode, and tell the subprocess.call where to direct the stdout, which is the terminal output you typically would see.
Try:
import subprocess
f = open('/tmp/mylogfile.log', 'w')
subprocess.call(['/path/to/script'], stdout=f)
f.close()
Then in the terminal you can run tail /tmp/mylogfile.log to confirm.
I'm not sure the last exit call is required for what you're trying to achieve.
You can read more in the python docs, depending which version of Python you're using. https://docs.python.org/2/library/subprocess.html
The file doesn't need to pre-exist. Hope that helps!
I have a list of input files and an expected output file, I want to write an auto-grader that does the job of accepting a python program, running it on the input files, and comparing its output to the output file. The approach I have used is to use the os module of python to run the program using os.system('python program.py > actual.out') and then perform a diff between the output and expected.out again using os.system().
The problem which I am currently facing is reading the input from the file because the program which is given is reading from the console. So, how should I redirect the input from a file such that it is readable by sys.stdin in program.py.
import os
def grade(program_py_file_handler,input_dir,output_dir):
#create temporary file for program.py using program_py_file_handler
#one by one read all files from input_dir
#run program.py using os.system generating a temp file
#do diff be temp file and expected file
Is there a better way to perform a diff without using the diff command?
To redirect output from program.py to a file I used python program.py>tem.out. What equivalent should I use to redirect an input file to progam.py such that wherever I have used sys.stdin in program.py it will instead read from the passed file? (Modifying program.py is not an option.)
You can be doing everything using builtin modules in Python 3.3+, since you are effectively spinning up a subprocess and doing a diff on the expected output. Simple minimum example:
check.py
import sys
from subprocess import check_output
from difflib import ndiff
def split(s):
return s.splitlines(keepends=True)
def check(program_name, expected):
output = check_output([sys.executable, program_name]).decode('utf8')
diff = ndiff(split(output), split(expected))
print(''.join(diff), end="")
def main():
check('hello.py', 'Good Morning!\n')
if __name__ == '__main__':
main()
hello.py
print('Good Evening!')
Example run
$ python check.py
- Good Evening!
? ^^^
+ Good Morning!
? ^^^
Modify as you see fit, with other methods/functions in the libraries linked. If you need stdin for the subprocess you probably will need to create Popen object and call communicate, but please read documentations first for future reference.
I got a Python script (test1.py) I need to run with a bat file (render.bat).
Question 1:
First, I had a definition in my test1.py and it always failed to run, nothing happened. Can someone kindly explain why?
import os
def test01 :
os.system('explorer')
and in the bat file:
python c:/test01.py
but as soon as I removed the def it worked. I just want to learn why this happened.
Question 2:
How can I take "render" string from render.bat as a string input for my python script so I can run something like :
import os
def test1(input) :
os.system("explorer " + input)
So the "input" is taken from the .BAT filename?
Functions don't actually do anything unless you call them. Try putting test01() at the end of the script.
%0 will give you the full name of the batch file called, including the .bat. Stripping it will probably be easier in Python than in the batch file.
Question1: Keyword def in python defines a function. However, to use a function you have to explicitly call it, i.e.
import os
def test01(): # do not forget ()
os.system('explorer')
test01() # call the function
1) You have to actually call the functions to achieve your task.
2) %0 refers to the running script. Therefor create a test.bat file like
# echo off
echo %0
Output = test.bat
You can strip the .bat extension from the output.