bash: string interpolation - python

I am returning strings from a python module, for consumption within a bash script.
This is what I have so far:
SCRIPT_DIR=/somepath/to/scripts/folder
cd $SCRIPT_DIR/eod
python_cmd='import sys;
sys.path.append("/somepath/to/scripts/folder/utils");
import settings as st;
print " ".join(st.foo())'
# Fix indentation for Python
python_cmd=${python_cmd//' '/''}
my_array=(`python -c "$python_cmd"`)
I want to use the DRY philosophy in the snippet above. However, the string/somepath/to/scripts/folder is repeated in the script. I would like to pass the definition of $SCRIPT_DIR into the python_cmd - however I tried (replacing the path string in python_cmd with $SCRIPT_DIR/utils and it failed with the following error:
Traceback (most recent call last):
File "<string>", line 3, in <module>
ImportError: No module named settings
What am I doing wrong?
Note:
I am running bash 4.1.5

Don't bother to pass the string in at all. Instead of modifying sys.path in the Python text, modify the PYTHONPATH environment variable in the bash script. It's easier, and is the preferred way to affect the import path anyway.
An example of how to set the path:
SCRIPT_DIR=/somepath/to/scripts/folder
cd $SCRIPT_DIR/eod
export PYTHONPATH=$SCRIPT_DIR
python_cmd='import settings as st;
print " ".join(st.foo())'
I also have to say, this seems like an odd way to glue components together, but I don't have enough information to recommend a better way.

Because the python command is enclosed in single quotes, $SCRIPT_DIR is not expanded. Try this:
python_cmd='import sys;
sys.path.append("'$SCRIPT_DIR'");
import settings as st;
print " ".join(st.foo())'
That said, I would go with the answer that modifies PYTHONPATH.

Related

Start Python REPL and execute command [duplicate]

I would like to play around in the python interpreter but with a bunch of imports and object setup completed. Right now I'm launching the interpreter on the command line and doing the setup work every time. Is there any way to launch the command line interpreter with all the initialization work done?
Ex:
# Done automatically.
import foo
import baz
l = [1,2,3,4]
# Launch the interpreter.
launch_interpreter()
>> print l
>> [1,2,3,4]
You can create a script with the code you wish to run automatically, then use python -i to run it. For example, create a script (let's call it script.py) with this:
import foo
import baz
l = [1,2,3,4]
Then run the script
$ python -i script.py
>>> print l
[1, 2, 3, 4]
After the script has completed running, python leaves you in an interactive session with the results of the script still around.
If you really want some things done every time you run python, you can set the environment variable PYTHONSTARTUP to a script which will be run every time you start python. See the documentation on the interactive startup file.
I use PYTHONSTARTUP.
My .bash_profile has a path to my home folder .pyrc, which as the import statements in it.
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSTARTUP
I came across this question when trying to configure a new desk for my research and found that the answers above didn't quite suit my desire: to contain the entire desk configuration within one file (meaning I wouldn't create a separate script.py as suggested by #srgerg).
This is how I ended up achieving my goal:
export PYTHONPATH=$READ_GEN_PATH:$PYTHONPATH
alias prepy="python3 -i -c \"
from naive_short_read_gen import ReadGen
from neblue import neblue\""
In this case neblue is in the CWD (so no path extension is required there), whereas naive_short_read_gen is in an arbitrary directory on my system, which is specified via $READ_GEN_PATH.
You could do this in a single line if necessary: alias prepy=PYTHONPATH=$EXTRA_PATH:$PYTHONPATH python3 -i -c ....
You can use the -s option while starting the command line. The details are given in the documentation here
I think I know what you want to do. You might want to check IPython, because you cannot start the python interpreter without giving the -i option (at least not directly).
This is what I did in my project:
def ipShell():
'''Starts the interactive IPython shell'''
import IPython
from IPython.config.loader import Config
cfg = Config()
cfg.TerminalInteractiveShell.confirm_exit = False
IPython.embed(config=cfg, display_banner=False)
# Then add the following line to start the shell
ipShell()
You need to be careful, though, because the shell will have the namespace of the module that the function ipShell() is defined. If you put the definition in the file you run, then you will be able to access the globals() you want. There could be other workarounds to inject the namespace you want, b̶u̶t̶ ̶y̶o̶u̶ ̶w̶o̶u̶l̶d̶ ̶h̶a̶v̶e̶ ̶t̶o̶ ̶g̶i̶v̶e̶ ̶a̶r̶g̶u̶m̶e̶n̶t̶s̶ ̶t̶o̶ ̶t̶h̶e̶ ̶f̶u̶n̶c̶t̶i̶o̶n̶ ̶i̶n̶ ̶t̶h̶a̶t̶ ̶c̶a̶s̶e̶.
EDIT
The following function defaults to caller's namespace (__main__.__dict__).
def ipShell():
'''Starts the interactive IPython shell
with the namespace __main__.__dict__'''
import IPython
from __main__ import __dict__ as ns
from IPython.config.loader import Config
cfg = Config()
cfg.TerminalInteractiveShell.confirm_exit = False
IPython.embed(config=cfg, user_ns=ns, display_banner=False)
without any extra arguments.

How do I execute a function?

I'm new to the python and i was trying to do my first python function, but unfortunately i faced some problems to get the expected result from this simple function please help me to show the output of that function. the below posted function is written in the python editor
i do not know how to call this function from the python shell to show its result.
python code:
def printme( str ):
"This prints a passed string into this function"
print str;
return;
python shell:
>>> printme("d")
>>> Traceback (most recent call last):
File "<pyshell#11>", line 1, in <module>
printme("d")
NameError: name 'printme' is not defined
$ cd /path/to/your/filename.py
$ python
>>> from filename import printme
>>> printme("hello world!")
You have to load the script as you start the interpreter. From a terminal shell (like bash or zsh):
$ python2 -i script.py
>>> printme("hola")
hola
>>>
On a side note, you don't have to terminate your statements with a semicolon (if they are in their own line), neither have to append a return statement at the end of the function (since indentation and line separation are significative in Python).
If you are using any of the IDEs for python, you could actually run the program in python shell by pressing/typing the Run(F5 equivalent). If that is not the case, read along:
Save the program as test.py (or any other name) in any location of your choice.
Start python shell
>>import sys
>>sys.path
If the directory in which you saved the test.py is present in the output of sys.path, go to step 7
sys.path.append("directory address where you saved the test.py")
>>import test #note .py is removed
>>test.printme("Hello World")
sys.path is the list containing all the directories where python looks for importing modules. By adding (appending) your directory you are ensuring the test.py can be imported as module test. You can then call any functions of test.py as test.fucn()
At step 7 you could have done:
7. >>from test import printme
8. >>printme("Hello again")
If you're using the unix shell:
$ cd C:\yourpath
$ python mypythonfile.py
If you are using the interactive mode, then this:
execfile("C:\\myfolder\\myscript.py")
The long way in interactive mode, but if you prefer to set your default path:
import os
prevPath = os.getcwd() #save the default path
myPath = "C:\myPython\somepath"
os.chdir(myPath) #set your python path
execfile("myscript.py") #executes the file
#os.chdir(prevPath) will restore the default path
Or did i misunderstood your question? If you just want to run a function, it's just as simple as this..
>>> def printme(str):
print str
>>> printme("Hello world!")
Hello world!
>>>
Hope this helps!
My python knowledge is very low... , you question come from this tutorial ,I have all to write as your example on a Linux shell , and i having none problem...
>>> def printme(str):
This print .......................
print str
return
>>> printme('d')
d
how i have Understand , you problem is that you to prove working with idle console and a Linux shell without before your code to save....i think , the examples from shellfly and alKid describe gut , how can you solving your problem...
sorry about my English....

run python command line interpreter with imports loaded automatically

I would like to play around in the python interpreter but with a bunch of imports and object setup completed. Right now I'm launching the interpreter on the command line and doing the setup work every time. Is there any way to launch the command line interpreter with all the initialization work done?
Ex:
# Done automatically.
import foo
import baz
l = [1,2,3,4]
# Launch the interpreter.
launch_interpreter()
>> print l
>> [1,2,3,4]
You can create a script with the code you wish to run automatically, then use python -i to run it. For example, create a script (let's call it script.py) with this:
import foo
import baz
l = [1,2,3,4]
Then run the script
$ python -i script.py
>>> print l
[1, 2, 3, 4]
After the script has completed running, python leaves you in an interactive session with the results of the script still around.
If you really want some things done every time you run python, you can set the environment variable PYTHONSTARTUP to a script which will be run every time you start python. See the documentation on the interactive startup file.
I use PYTHONSTARTUP.
My .bash_profile has a path to my home folder .pyrc, which as the import statements in it.
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSTARTUP
I came across this question when trying to configure a new desk for my research and found that the answers above didn't quite suit my desire: to contain the entire desk configuration within one file (meaning I wouldn't create a separate script.py as suggested by #srgerg).
This is how I ended up achieving my goal:
export PYTHONPATH=$READ_GEN_PATH:$PYTHONPATH
alias prepy="python3 -i -c \"
from naive_short_read_gen import ReadGen
from neblue import neblue\""
In this case neblue is in the CWD (so no path extension is required there), whereas naive_short_read_gen is in an arbitrary directory on my system, which is specified via $READ_GEN_PATH.
You could do this in a single line if necessary: alias prepy=PYTHONPATH=$EXTRA_PATH:$PYTHONPATH python3 -i -c ....
You can use the -s option while starting the command line. The details are given in the documentation here
I think I know what you want to do. You might want to check IPython, because you cannot start the python interpreter without giving the -i option (at least not directly).
This is what I did in my project:
def ipShell():
'''Starts the interactive IPython shell'''
import IPython
from IPython.config.loader import Config
cfg = Config()
cfg.TerminalInteractiveShell.confirm_exit = False
IPython.embed(config=cfg, display_banner=False)
# Then add the following line to start the shell
ipShell()
You need to be careful, though, because the shell will have the namespace of the module that the function ipShell() is defined. If you put the definition in the file you run, then you will be able to access the globals() you want. There could be other workarounds to inject the namespace you want, b̶u̶t̶ ̶y̶o̶u̶ ̶w̶o̶u̶l̶d̶ ̶h̶a̶v̶e̶ ̶t̶o̶ ̶g̶i̶v̶e̶ ̶a̶r̶g̶u̶m̶e̶n̶t̶s̶ ̶t̶o̶ ̶t̶h̶e̶ ̶f̶u̶n̶c̶t̶i̶o̶n̶ ̶i̶n̶ ̶t̶h̶a̶t̶ ̶c̶a̶s̶e̶.
EDIT
The following function defaults to caller's namespace (__main__.__dict__).
def ipShell():
'''Starts the interactive IPython shell
with the namespace __main__.__dict__'''
import IPython
from __main__ import __dict__ as ns
from IPython.config.loader import Config
cfg = Config()
cfg.TerminalInteractiveShell.confirm_exit = False
IPython.embed(config=cfg, user_ns=ns, display_banner=False)
without any extra arguments.

Running fselect using Python

I am trying to run this python script called fselect in windows 7. It can be downloaded from this website: http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/ under this name called Feature selection tool. I am running it on Python 2.7.2.Facing a bit of problem running it..
Typed this first in IDLE:
>>> import pprint
>>> import sys
>>> print pprint.pprint(sys.path)
>>> sys.path.append("C:\Users\HP\Documents\MATLAB\libsvm-3.11\tools")
>>> import fselect
Usage: training_file [testing_file]
Then the problem is when i type the next part:
Tried this:
>>> ./fselect.py TrainVec
SyntaxError: invalid syntax
Next tried this:
>>> fselect.py TrainVec
SyntaxError: invalid syntax
Next tried this:
>>> TrainVec
Traceback (most recent call last):
File "<pyshell#7>", line 1, in <module>
TrainVec
NameError: name 'TrainVec' is not defined
Tried this also:
>>> TrainVec.mat
Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
TrainVec.mat
NameError: name 'TrainVec' is not defined
What is the correct way of typing it? Need some guidance on it...
tried running using cmd but there is an error...
If you are trying to run the fselect.py directly from the command prompt, make sure that python is set into the path variable. For guidance with that, please read http://people.cis.ksu.edu/~schmidt/200f07/setpath.html.
The script will also invoke grid.py. grid.py requires gnuplot to be there. So ensure that grid.py is running properly and if necessary check the paths of the svm_train, svm_test in the script along with that of grid.py.
Hope it will work now.
If it is a tool, you should run it, not import it. And of course you should not try to enter random commands, even if they are valid shell commands, in your Python prompt.
Assuming TrainVec is your data (since you use it in the context of TrainVec.mat it must be a Matlab data file) then run it on the Command Prompt like this:
python fselect.py TrainVec.mat
The example of ./fselect.py is intended for Unix systems. Make sure you run the above command in what ever directory you have saved fselect.py in.
If you need to write your own scripts to leverage this .py file then I refer you here for an example of how to do this.
Like the previous answer said, it looks like you are (incorrectly) trying to run the script from inside the Python interpreter. According to the documentation on the page you link to, it is not a module but a free-standing script and should be run as such:
Usage: ./fselect.py training_file [testing_file]

Problem with reading in parameters with special characters in Python

I have a scripts (a.py) reads in 2 parameters like this:-
#!/usr/bin/env python
import sys
username = sys.argv[1]
password = sys.argv[2]
Problem is, when I call the script with some special characters:-
a.py "Lionel" "my*password"
It gives me this error:-
/swdev/tools/python/current/linux64/bin/python: No match.
Any workaround for this?
Updated-
It has been suspected that this might be a shell issue rather than the script issue.
I thought the same too, until i tried it out on a perl script(a.pl), which works perfectly without any issue:-
#!/usr/bin/env perl
$username = $ARGV[1];
$password = $ARGV[2];
print "$username $password\n";
%a.pl "lionel" "asd*123"
==> lionel asd*123
No problem.
So i guess , this looks to me more like a PYTHON issue.
Geezzz ........
The problem is in the commands you're actually using, which are not the same as the commands you've shown us. Evidence: in Perl, the first two command-line arguments are $ARGV[0] and $ARGV[1] (the command name is $0). The Perl script you showed us wouldn't produce the output you showed us.
"No match" is a shell error message.
Copy-and-paste (don't re-type) the exact contents of your Python script, the exact command line you used to invoke it, and the exact output you got.
Some more things to watch out for:
You're invoking the script as a.py, which implies either that you're copying it to some directory in your $PATH, or that . is in your $PATH. If the latter, that's a bad idea; consider what happens if you cd info a directory that contains a (possibly malicious) command called ls. Putting . at the end of your $PATH is safer than putting it at the beginning, but I still recommend leaving it out altogether and using ./command to invoke commands in the current directory. In any case, for purposes of this exercise, please use ./a.py rather than a.py, just so we can be sure you're not picking up another a.py from elsewhere in your $PATH.
This is a long shot, but check whether you have any files in your current directory with a * character in their names. some_command asd*123 (without quotation marks) will fail if there are no matching files, but not if there happens to be a file whose name is literally "asd*123".
Another thing to try: change your Python script as follows:
#!/usr/bin/env python
print "before import sys"
import sys
print "after import sys"
username = sys.argv[1]
password = sys.argv[2]
This will tell you whether the shell is invoking your script at all.
That error comes from your shell, not from Python. Do you have a shopt -s failglob set in your .bashrc or somewhere?
/swdev/tools/python/current/linux64/bin/python: No match.
I think the problem is that the python env is not set:
Does python run at all on your machine ?

Categories