Program that gives input to other file and collects output - python

Is there any way that I can create a program where it gives input to another file and and collects its output?
The best that google give me is this. And I tried to recreate (read: copying the code in some unknown manner (read: stabbing in the dark))
And I got this
import time
string="file.py"
process=subprocess.Popen(string,stdin=subprocess.PIPE,stdout=subprocess.PIPE);
process.stdin.write("3 5")
time.sleep(1)
print process.stdout.read()
And it gives me error
File "D:\Pekerjaan non website\IO\reader.py", line 3, in <module>
process=subprocess.Popen(string,stdin=subprocess.PIPE,stdout=subprocess.PIPE);
NameError: name 'subprocess' is not defined
Can anybody tell me how to create that program. (
Note: I have no knowledge in this kind of program I/O or subprocess
Note: It's up to you. Where you will explain this from my code or throw my code away and explain this from zero.
Thank you beforehand.
(PS: If my previous statement is confusing. My point is simple: Can you teach me? :))

subprocess is a stdlib module that you need to import (the same way time is) - so you just need to:
import subprocess
some time before you try to use the functions in it (usually, you want to do this near the top of your code, right underneath your current import time).

In addition to lvc's answer you should consider using Popen.communicate(). something like,
import subprocess
string="file.py"
process=subprocess.Popen(string,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
res=process.communicate("3 5")
print res[0]
In this way you don't have to use that sleep command.it'll Wait for process to terminate and return a tuple as (stdoutdata, stderrdata).

Related

python script that takes command line arguments needs to be called from another python script

I completely understand that I should have written the script right the first time, but the fact is I have a script that generates a data file based upon two values passed to it from the command line- like this:
[sinux1~]: ./sim_gen.py 100 .3
I need to call this script from within another script, iterating over a range of values. I searched around and after navigating through all of the "you shouldn't," I tried :
exec(open("./sim_gen.py 100 .3").read())
And this doesn't seem to work.
Help?
Let's break this down into pieces:
exec(open("./sim_gen.py 100 .3").read())
This is equivalent to:
f = open("./sim_gen.py 100 .3")
contents = f.read()
exec(contents)
That open is the same open you use for, say, reading a text file or a CSV. You're asking for a file named "sim_gen.py 100 .3" in the current directory. Do you have one? Of course not. So the open fails.
The best solution is, as you already know, to rewrite sim_gen.py so that you can import it and call a function and pass the arguments to it.
Failing that, the cleanest answer is probably to just run the Python script as a subprocess:
import subprocess
import sys
subprocess.run([sys.executable, "./sim_gen.py", "100", ".3"])
Notice that this is effectively the same thing you're doing when you run the script from your shell, so if it was OK there, it's almost surely OK here.
If you really need to exec for some reason, you will need to do something really hacky, and temporarily change argv for that script's code:
import sys
_argv = sys.argv
try:
sys.argv = ["./sim_gen.py", "100", ".3"]
with open("./sim_gen.py 100 .3"):
exec(f.read())
finally:
sys.argv = _argv
Although really, unless the point of running this is to silently modify your own module's globals or the like, you (a) almost certainly don't really need exec, and (b) want to pass an explicit globals argument even if you do really need it.

Is there a way to write over the python code after the interpretation?

This may be a dumb question, but I want to add a line at the very start of the code like
print 'previous runtime' time.time()-tic
Is there a way to do it? Or can I somehow get the previous runtime other than keeping a logfile?
I have to disagree with the accepted answer. You can edit Python code during interpretation for future runtime use. The file itself isn't locked during execution and can be edited by itself. Try this simple example named test.py:
def main():
with open('test.py','a') as me:
me.write(' print "test"\n')
if __name__== '__main__':
main()
Every time you run this, it will add an additional "print test" at the end of the file and next runtime will execute the modified code.
Your example is more complex, but I see no reason why you couldn't insert a variable with the last runtime and call it for a print.
me.write('last_runtime = {0}'.format(time.time())
I will say that while this is possible it's probably(read 'almost certainly') not the best approach. It would be much easier to just have a last_run.pickle and pickle dump a datetime object or whatever to save the last runtime (overwriting the previous last runtime). Then have your code reference this reference file.
While you CAN edit your code from your code, the only reason I can imagine for doing it this way is if you 100% need everything for your program in one file (which seems an unlikely requirement). Otherwise best to save your data somewhere and reference it.
No. It wouldn't be possible unless written somewhere. Simple reason is that once the python process ends, GC cleans up everything.

Trouble importing files in Python

I'm just finding out now that when importing a module, it seems to run through ALL the code, instead of just the one function that I want it to go through. I've been trying to find a way around this, but can't seem to get it. Here is what is happening.
#mainfile.py
from elsewhere import something_else
number = 0
def main():
print('What do you want to do? 1 - something else')
donow = input()
if donow == '1':
something_else()
while 1:
main()
#elsewhere.py
print('I dont know why this prints')
def something_else():
from mainfile import number
print('the variable number is',number)
Now, although this code KIND OF works the way I want it to, the first time when I initiate it, it will go to the main menu twice. For example: I start the program, press one, then it asks me what I want to do again. If I press one again, then it will print "the variable number is 0".
Once I get this working, I would like to be importing a lot of variables back and forth. The only issue is,if I add more import statements to "elsewhere.py" I think it will just initiate the program more and more. If I put "from mainfile import number" on line 1 of "elsewhere.py", I think this raises an error. Are there any workarounds to this? Can I make a different file? What if I made a class to store variables, if that is possible? I'm very new to programming, I would appreciate it if answers are easy to read for beginners. Thank you for the help.
As Jan notes, that's what import does. When you run import, it runs all of the code in the module. You might think: no it doesn't! What about the code inside something_else? That doesn't get run! Right, when the def statement is executed it creates a new function, but it doesn't run it. Basically, it saves the code for later.
The solution is that pretty much all interesting code should be in a function. There are a few cases which make sense to put at the top-level, but if in doubt, put it inside a function. In your particular case, you shouldn't be printing at the top level, if you need to print for some reason, put that into a function and call it when you need it. If you care when something happens, put it in a function.
On a second node, don't import your primary script in other scripts. I.e. if your mainfile.py directly, don't import that in other files. You can but it produces confusing results, and its really best to pretend that it doesn't work.
Don't try to import variables back and forth. Down that path lies only misery. You should only be importing things that don't change. Functions, classes, etc. In any other case, you'll have hard time making it do what you want.
If you want to move variables between places, you have other options:
Pass function arguments
Return values from a function
Use classes
I'll leave it is an exercise to the reader to learn how to do those things.
import executes imported code
import simply takes the Python source file and executes it. This is why it prints, because that instruction is in the code and with import all the instructions get exectued.
To prevent execution of part of imported package/module, you shall use the famous:
if __name__ == "__main__":
print("I do not print with `import`")
Note, that this behaviour is not new in Python 3, it works the same way in Python 2.x too.

when is 'commands' preferable to 'popen' subprocess?

I'm apprenticing into system administration without schooling, so sometimes I'm missing what is elementary information to many others.
I'm attempting to give my stdout line another argument before printing, but I'm not sure which process I should use, and I'm a bit fuzzy on the commands for subprocess if that's what I should be using.
My current code is:
f = open('filelist', 'r')
searchterm = f.readline()
f.close()|
#takes line from a separate file and gives it definition so that it may be callable.
import commands
commands.getoutput('print man searchterm')
This is running, but not giving me an ouput to the shell. My more important question is though, am I using the right command to get my preferred process? Should I be using one of the subprocess commands instead? I tried playing around with popen, but I don't understand it fully enough to use it correctly.
Ie, I was running
subprocess.Popen('print man searchterm')
but I know without a doubt that's not how you're supposed to run it. Popen requires more arguments than I have given it, like file location and where to run it (Stdout or stderr). But I was having trouble making these commands work. Would it be something like:
subprocess.Popen(pipe=stdout 'man' 'searchterm')
#am unsure how to give the program my arguments here.
I've been researching everywhere, but it is such a widely used process I seem to be suffering from a surplus of information rather than not enough. Any help would be appreciated, I'm quite new.
Preemptive thanks for any help.
The cannonical way to get data from a separate process is to use subprocess (commands is deprecated)
import subprocess
p = subprocess.Popen(['print','man','searchitem'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdoutdata, stderrdata = p.communicate()
Note that some convenience functions exist for splitting strings into lists of arguments. Most notably is shlex.split which will take a string and split it into a list the same way a shell does. (If nothing is quoted in the string, str.split() works just as well).
commands is deprecated in Python 2.6 and later, and has been removed in Python 3. There's probably no situation where it's preferable in new code, even if you are stuck with Python 2.5 or earlier.
From the docs:
Deprecated since version 2.6: The commands module has been removed in
Python 3. Use the subprocess module instead.
To run man searchterm in a separate process and display the result in the terminal, you could do this:
import subprocess
proc = subprocess.Popen('man searchterm'.split())
proc.communicate()

An interpretation error for a python multithreading and multiprocessing

I want to compare multithreading and multiprocessing python program. But, I got interpretation error:
File "./parallelPython.py", line 23
time fornorm(g,range(100))
^
SyntaxError: invalid syntax
The code is as follows:
#!/usr/bin/python -tt
import numpy as np
import math
def f(x):
print x
y = [1]*10000000
[math.exp(i) for i in y]
def g(x):
print x
y = np.ones(10000000)
np.exp(y)
from handythread import foreach
from processing import Pool
from timings import f,g
def fornorm(f,l):
for i in l:
f(i)
time fornorm(g,range(100))
time fornorm(f,range(10))
time foreach(g,range(100),threads=2)
time foreach(f,range(10),threads=2)
p = Pool(2)
time p.map(g,range(100))
time p.map(f,range(100))
I do not why fornorm() has a problem, it has been defined !!!
thanks
It doesn't say fornorm hasn't been defined, it says you have a syntax error on the line where you're calling fornorm. Syntax errors mean Python can't even understand your code: it's as if I say to you "flrk ask web ski ur lkjq", and then ask you to do what I said. An error about fornorm not being defined would happen much later. As it is, Python can't even tell whether you're asking it to call a function, let alone whether you're calling one that is already defined or not.
It looks like your error is this:
time fornorm(g,range(100))
That looks like you're trying to use the shell command time. Shell commands are not Python, and Python doesn't understand it.
However, your code as pasted into SO also has indentation errors, which should have triggered a syntax error earlier than that line, so I suspect what we can see here is not exactly what you were running.
It looks like an Indentation error here :
def fornorm(f,l):
for i in l:
f(i)
After your def python is expecting an indented block.
By the way, time something is an IPython "magic" function and it won't work in a script file. You should import timeit module and use that instead.
Where are you getting "time" from? That's not a valid python statement. It's not like shell scripting.
If you want to time stuff, use the timeit library:
http://docs.python.org/library/timeit.html

Categories