We Keep Coding

Can't run file with sys.arg[1]

I'm working through https://testdriven.io/developing-an-asynchronous-task-queue-in-python . I've also taken a look at sys.argv[1] meaning in script for clarification on sys.argv
From the former I have:
def save_file(filename, data):
random_str = uuid.uuid4().hex
outfile = f'{filename}_{random_str}.txt'
with open(os.path.join(OUTPUT_DIRECTORY, outfile), 'w') as outfile:
outfile.write(data)
def get_word_counts(filename):
wordcount = collections.Counter()
# get counts
with open(os.path.join(DATA_DIRECTORY, filename), 'r') as f:
for line in f:
wordcount.update(line.split())
for word in set(COMMON_WORDS):
del wordcount[word]
# save file
save_file(filename, json.dumps(dict(wordcount.most_common(20))))
# simulate long-running task
time.sleep(2)
proc = os.getpid()
print(f'Processed {filename} with process id: {proc}')
if __name__ == '__main__':
print(sys.argv, len(sys.argv))
# print(sys.argv[1], len(sys.argv))
get_word_counts(sys.argv[1])
When I run it directly with I get:
$ python tasks.py
['tasks.py'] 1
Traceback (most recent call last):
File "tasks.py", line 46, in <module>
get_word_counts(sys.argv[1])
IndexError: list index out of range
Given that you can see there is only one element in the list, why did the author write the the code in this way?

get_word_counts(sys.argv[1])
Should be
get_word_counts(sys.argv[0])
Indexes start at zero in most languages (including python)

only half of output added into temp file

this is the code I'm working on
remote_conn_pre = paramiko.SSHClient()
remote_conn_pre
remote_conn_pre.connect(ip,
username=username,password=password,look_for_keys=False,allow_agent=False)
remote_conn = remote_conn_pre.invoke_shell()
output = remote_conn.recv(1002)
remote_conn.send("\n")
remote_conn.send("enable\n")
remote_conn.send("show ip int brief\n")
remote_conn.close()
time.sleep(2)
output = remote_conn.recv(65535)
print output
output_cap = tempfile.TemporaryFile(output)
print output_cap
the output I got was:
Traceback (most recent call last):
File "p1.py", line 27, in <module>
output_cap = tempfile.TemporaryFile(output)
File "/usr/lib/python2.7/tempfile.py", line 488, in TemporaryFile
return _os.fdopen(fd, mode, bufsize)
ValueError: mode string must begin with one of 'r', 'w', 'a' or 'U', not '
R1#enable
R1#show ip int brief
Interface IP-Address OK? Method Status
Protocol
FastEthernet0/0 192.168.2.101 YES other up
up '
how can I pass the output I can get from my code into a temporary file?

tempfile.TemporaryFile()'s first parameter is mode, not the data you want to write.

To write to a file
fo = open("filename.txt", wb)
fo.write(output)
fo.close()
mode "wb" writes the file in binary, creating a new file if it doesn't already exist, and overwrites the file if it does exist.

How to implement STDOUT and file write based on parameter input

I have the input file that looks like this (infile.txt):
a x
b y
c z
I want to implement a program that enable user to write to STDOUT or file depending on the command:
python mycode.py infile.txt outfile.txt
Will write to file.
And with this
python mycode.py infile.txt #2nd case
Will write to STDOUT.
I'm stuck with this code:
import sys
import csv
nof_args = len(sys.argv)
infile = sys.argv[1]
print nof_args
outfile = ''
if nof_args == 3:
outfile = sys.argv[2]
# for some reason infile is so large
# so we can't save it to data structure (e.g. list) for further processing
with open(infile, 'rU') as tsvfile:
tabreader = csv.reader(tsvfile, delimiter=' ')
with open(outfile, 'w') as file:
for line in tabreader:
outline = "__".join(line)
# and more processing
if nof_args == 3:
file.write(outline + "\n")
else:
print outline
file.close()
When using 2nd case it produces
Traceback (most recent call last):
File "test.py", line 18, in <module>
with open(outfile, 'w') as file:
IOError: [Errno 2] No such file or directory: ''
What's the better way to implement it?

You can try this:
import sys
if write_to_file:
out = open(file_name, 'w')
else:
out = sys.stdout
# or a one-liner:
# out = open(file_name, 'w') if write_to_file else sys.stdout
for stuff in data:
out.write(stuff)
out.flush() # cannot close stdout
# Python deals with open files automatically
You can also use this instead of out.flush():
try:
out.close()
except AttributeError:
pass
This looks a bit ugly to me, so, flush will be just well.

Passing argument from one python script to another

I've got 3 files in total : clean.txt, origin.py and pump.py
Clean.txt has some lines in it (actually website links. For eg:
www.link1.com
www.link2.com
www.link3.com
www.link4.com
origin.py is the script that's reading lines one by one and I want this script to send the link (one at a time) to pump.py
pump.py is a script which asks me for one input, which is a link.
Now, I want to make it so that I read lines from clean.txt (origin.py is doing this task) and send them to the pump.py one at a time. Like a loop.
Code for Origin.py is :
fo = open("Clean.txt", "r")
nremoval = str(fo.readlines()).replace('\\n','')
for lines in fo :
if __name__ =="__main__":
response = nremoval
p = subprocess.Popen("pump.py", stdin = subprocess.PIPE)
time.sleep(2)
p.stdin.write(bytes(response, 'ascii'))
print("injected string :", response)

origin.py
import subprocess
import time
# open file
with open("Clean.txt", "r") as fo:
# read line by line
for line in fo:
# remove ENTER and SPACES from single line
line = line.strip()
# call "python pump.py"
p = subprocess.Popen(["python","pump.py"], stdin=subprocess.PIPE)
# or "python3 pump.py"
#p = subprocess.Popen(["python3","pump.py"], stdin=subprocess.PIPE)
#time.sleep(2)
# send line to standard input
#p.stdin.write(bytes(line, 'ascii')) # python 3
p.stdin.write(line) # python 2
print("injected string:", line)
pump.py
#line = input() # python 3
line = raw_input() # python 2
print("received:", line)

Read line by line using timer

I am trying to read each line by specific interval using python timer from a txt file
But it read first line only and it shows continuously
my code is
def read():
try:
while True:
fo = open("foo.txt", "r")
threading.Timer(1.0, read).start()
line=fo.readline()
print line
if len(line)==0:
break
except:
pass
read()

The issue is that you again open the file , which starts reading from the first line , and you read that line print it and again continue the loop, this makes the loop an infinite loop.
Also, what you are doing with threading.Timer() is not how you use it, threading.Timer() starts the function read in a new thread after 1 sec, after sometime you would have loads of threads all running the read() function indefinitely.
What you are trying to do can easily be accomplished (without using threading.Timer() ) by using time.sleep() , to make your program sleep for a specific amount of seconds. Example -
def read():
import time
with open("foo.txt", "r") as fo:
for line in fo:
print line
time.sleep(1)
read()
If you really want to use threading.Timer() , then you do not need the while loop and you should pass the file object as an argument to the function, Example -
def read(fo):
line=fo.readline()
print line
if len(line) == 0:
return
t = threading.Timer(1.0, read, args=(fo,))
t.start()
t.join()
And then call the function initially as -
with open("foo.txt", "r") as fo:
read(fo)
Example/Demo -
>>> def a(x):
... if x >50:
... return
... print(x)
... t = threading.Timer(1.0 , a, args=(x+1,))
... t.start()
... t.join()
...
>>> import threading
>>> a(1)
1
2
3
4
5
6
7
8