passing the filename to a python script in command line - python

I want to pass the name of a file to a python script while I'm running it from the command line. I'm trying this clear command:
cat enwiki-latest-pages-articles.xml | python WikiExtractor.py -b 500K -o extracted
however, it gives an error:
'cat' is not recognized as an internal or external command, operable program or batch file.
Thanks in advance.

It seems like you're running the command in Windows. In windows, there's no cat installed unless you installed.
You can use type command instead:
type enwiki-latest-pages-articles.xml | python WikiExtractor.py -b 500K -o extracted

The correct way would be python WikiExtractor.py -b5 -o extracted -f enwiki-latest-pages-articles.xml.
And use sys.argv array of input arguments of python command from sys command.
This may help:
http://www.tutorialspoint.com/python/python_command_line_arguments.htm

Related

linux command pipe with python "-c" flag

I am trying to do a string printing with python -c flag, e.g.
python3 -c "print('Hello World')"
So now I wanna substitute an argument with pipe, e.g. echo "Hello World" | python3 -c "print($1)"
the pipe is to take output from previous command and take it as input to next command, if I am not wrong, this is possible? But I think I got syntax error which I cannot find any source of this
I also bumped into question previously asked, but the solution required python imports and .py file depends on how we run this, I understand but I just wanna get it in a line of command in linux shell
If your input is always single line then you should be able to harness input function for example
echo "Hello World" | python3 -c "print(input().upper())"
would output
HELLO WORLD

Printing .py file output in command line

I am trying to access a python function from the command line, and I would like to write such a command that will print the output in the terminal. The below doesn't work. What could I change?
python -c 'from laser import Laser; laser = Laser();l = laser.embed_sentences("hello", lang = "en").shape == (1, 1024); print(l)'
(base) ~ % python -c 'print("hello, world")'
hello, world
Printing works fine for me when running python through python -c. Are you sure your terminal isn't truncating your output by omitting the last (and in this case, only) line? You could try creating a single line file (no newline at the end) and then running cat [filename] (which is how I sometimes discover that my terminal is doing this)
-c cmd : program passed in as string (terminates option list)
That is the correct flag to be used. This must be a CLI config issue. Or the script is taking longer than you are expecting to run and it appears no output is generated.
Does python -c 'print("hello")' work?

determine file type of a file without extension

I want to use pygmentize to highlight some script files (python/bash/...) without extension. But pygmentize requires me to specify the lexer using -l. It does not automatically identify the file type from the content.
I have the following options at hand, but none of them work now:
use file -b --mime-type. But this command output x-python and x-shellscript instead of python and bash and I don't know the rules
use vim -e -c 'echo &ft|q' the_file. For any file with or without file extension, vim has a mechanism to guess the file type. But it doesn't work. Since the output goes to the vim window and disappears after q.
What can I do?
#Samborski's method works fine in normal case but it does not work in python subprocess.check_output since the pts is not allocated. If you use nvim, you can use this more straightforward way:
HOME=_ nvim --headless -es file <<EOF
call writefile([&ft], "/dev/stdout")
EOF
You can use vim this way:
vim -c ':silent execute ":!echo " . &ft . " > /dev/stdout"' -c ':q!' the_file
It simply constructs command to run in the shell as a string concatenation.

python 2.3 how to run a piped command

I want to run a command like this
grep -w 1 pattern <(tail -f mylogfile.log)
basically from a python script i want to monitor a log file for a specific string and continue with the python script as soon as i found that.
I am using os.system(), but that is hanging. The same command in bash works good.
I have a very old version of python (v2.3) and so don't have sub-process module.
do we have a way to acheive this
In Python 2.3, you need to use subprocess from SVN
import subprocess
import shlex
subprocess.call(shlex.split("/bin/bash -c 'grep -w 1 pattern <(tail -f mylogfile.log)'"))
To be explicit, you need to install it from the SVN link above.
You need to call this with /bin/bash -c due to the shell redirection you're using
EDIT
If you want to solve this with os.system(), just wrap the command in /bin/bash -c since you're using shell redirection...
os.system("/bin/bash -c 'grep -w 1 pattern <(tail -f mylogfile.log)'")
First of all, the command i think you should be using is grep -w -m 1 'pattern' <(tail -f in)
For executing commands in python, use the Popen constructor from the subprocess module. Read more at
http://docs.python.org/library/subprocess.html
If I understand correctly, you want to send the output to python like this -
tail -f mylogfile.log | grep -w 1 pattern | python yourscript.py
i.e., read all updates to the log file, and send matching lines to your script.
To read from standard input, you can use the file-like object: sys.stdin.
So your script would look like
import sys
for line in sys.stdin.readlines():
#process each line here.

Execute bash script from URL using python

Assume I have a file at http://mysite.com/myscript.sh that contains:
#!/bin/bash
echo "Hello $1"
From the command line, I can execute my script (without downloading it) using the following command:
bash <(curl -s http://mysite.com/myscript.sh) World
Now, instead of executing the above command from the command line, I want to execute it from a python script. I tried doing the following:
import os
os.system('bash <(curl -s http://mysite.com/myscript.sh) World')
...but I get the following error:
sh: -c: line 0: syntax error near unexpected token `('
How do I make this execute correctly in python?
Evidently, os.system runs its command through /bin/sh, which usually causes whichever shell it's linked to to drop to a compatibility mode that doesn't include the <(...) construction. You can get around it by either storing the result in a temporary file or using another level of shell. Ugly, but it works.
os.system('bash -c "bash <(curl -s http://mysite.com/myscript.sh) World"')
There is a libcurl for python so you don't have to go the way around to command line behaviour. Here's the function list that should really do it - have never run remote scripts myself though. If you need installing the python binding, the instructions are here.
import curl

Categories