I'm sorry for asking a duplicate as this and this are very similar but with those answers I don't seem to be able to make my code work.
If have a jupyter notebook cell with:
some_folder = "/some/path/to/files/*.gif"
!for name in {some_folder}; do echo $name; done
The output I get is just {folder}
If I do
some_folder = "/some/path/to/files/*.gif"
!for name in "/some/path/to/files/*.gif"; do echo $name; done # <-- gives no newlines between file names
# or
!for name in /some/path/to/files/*.gif; do echo $name; done # <-- places every filename on its own line
My gif files are printed to screen.
So my question why does it not use my python variable in the for loop?
Because the below, without a for loop, does work:
some_folder = "/some/path/to/files/"
!echo {some_folder}
Follow up question: I actually want my variable to just be the folder and add the wildcard only in the for loop. So something like this:
some_folder = "/some/path/to/files/"
!for name in {some_folder}*.gif; do echo $name; done
For context, later I actually want to rename the files in the for loop and not just print them. The files have an extra dot (not the one from the .gif extension) which I would like to remove.
There's an alternative way to use shell bash in a Jupyter cell with cell magic, see here. It seems to allow what you are trying to do.
If you already ran in a normal cell some_folder = r"/some/path/to/files/*.gif" or some_folder = "/some/path/to/files/*.gif", then you can try in a separate cell:
%%bash -s {some_folder}
for i in {1..5}; do echo $1; done
That said, what you seems to be trying to do with some_folder = "/some/path/to/files/*.gif" isn't going to work as such. If you try to pass "/some/path/to/files/*.gif" from Python to bash, it isn't going to work like passing /some/path/to/files/*.gif directly to bash. Bash isn't passing "/some/path/to/files/*.gif" directly to a command, it expands it and then passes it. There's not going to be an expansion passing from Python. And there's other peculiarities you'll come across. Tar you can pass a Python list of files directly using the bracket notation and it will handle that.
The solutions are to either do more on the Python side or more in the shell side. Python has it's own glob module, see here. You can combine that with working with os.system(). Python has fnamtch that is nice because you can use Unix-like file name matching. Plus there's shutil that allows moving/renaming, see shutil.move(). In Python os.remove(temp_file_name) can delete files. If you aren't working on a Windows machine there's the sh module that makes things nice. See here and here.
Related
I'm trying to launch may main Python script with some arguments listed in a txt file (config.txt).
Because parameters change almost every launch and I dont want to type them every time. They are not literally a "config" but I didn't find the correct file name (that's an other story).
See below:
-param1 1
-param2 2
-verbose
Using Run Configuration of PyCharm.
I would like to finally do something like :
python C:\somewhere\main.py -param1 1 -param2 2 -verbose
Instead of current behavior :python C:\somewhere\main.py config.txt
Which, by the way, is missed understood by the program (obviously).
#32951846
I already tried windows for loops in the section "before launch: activate tools":
$: for /f "delims=" %x in (config.txt) do set ARGS=%ARGS%%x
$: python C:\somewhere\main.py %ARGS%
But it only keep the last line of the config.txt inside ARGS.
#51948712
I also tried to pipe the content of the file into my python main program like:
python C:\somewhere\main.py < config.txt
But it do not work neither.
#syntax-redirection
Am I right that you'd like to see something like https://youtrack.jetbrains.com/issue/PY-5543?
Consider using the following plugin: https://plugins.jetbrains.com/plugin/7861-envfile/
This is not exactly what you were asking for, but you can follow this guideline to store the run configurations in a file and then modify the file, share it or add to git.
The key steps are to tick the box "Store as project file" in PyCharm's "Run/Debug Configurations" window. This will create the new subfolder "runConfigurations" in the ".idea" folder in the project folder.
The folder will contain an xml file with the line
<option name="PARAMETERS" value=""arg1" "arg2"" />
where "arg1" and "arg2" are the arguments which are passed to your script.
I have couple of file with same name, and I wanted to get the latest file
[root#xxx tmp]# ls -t1 abclog*
abclog.1957
abclog.1830
abclog.1799
abclog.1742
I can accomplish that by executing below command.
[root#xxx tmp]# ls -t1 abclog*| head -n 1
abclog.1957
But when I am trying to execute the same in python , getting error :
subprocess.check_output("ls -t1 abclog* | head -n 1",shell=True)
ls: cannot access abclog*: No such file or directory
''
Seems it does not able to recognize '*' as a special parameter. How can I achieve the same ?
As others noted, your code should work. It doesn't work probably because the current directory isn't the one you suppose it is, so abc* is not expanded by the shell (even if shell=True is set), and passed as-is to ls, resulting in a "no such file" error.
You have to pass the absolute path or use cwd= parameter when calling check_output. Another nice pythonic alternative would be to avoid subprocess, and return the most recently modified file using only python code:
most_recent = max(glob.glob(os.path.join("path/to/file","abclog*"),key=os.path.getmtime)
(using max with os.path.getmtime as key and glob.glob to filter the files)
Make sure you execute this in the directory where the files exist. If you just fire up Idle to run this code, you will not be in that directory.
Is there a way to retrieve the path to the interpreter a UNIX shell would use for a given script? (preferably in a Python API or as shell command)?
To be used like this:
$ get_bang ./myscript.py
/usr/bin/python3
Of course I could extract it manually using RE but I'm sure in real world that's more complicated than just handling the first line and I don't want to re-invent the wheel..
The reason I need this is I want to call the script from inside another script and I want to add parameters to the interpreter.
Actually, it isn't more complicated than reading (the first word) of the first line.
Try putting the shebang on the second line (or even just putting a space before the #) and see what happens.
Also see http://www.in-ulm.de/~mascheck/various/shebang/ and http://homepages.cwi.nl/~aeb/std/hashexclam-1.html for more than you've ever wanted to know about the shebang feature.
Many ways - for example:
sed -n '1s/^#!//p' filename
prints for example
/bin/sh
or (if multiword)
/usr/bin/env perl
or nothing, if here isn't shebang
I am trying to make a python script that will open a directory, apply a perl script to every file in that directory and get its out put in either multiple text files or just one.
I currently have:
import shlex, subprocess
arg_str = "perl tilt.pl *.pdb > final.txt"
arg = shlex.split(arg_str)
import os
framespdb = os.listdir("prac_frames")
for frames in framespdb:
subprocess.Popen(arg, stdout=True)
I keep getting *.pdb not found. I am very new to all of this so any help trying to complete this script would help.
*.pdb not found means exactly that - there won't be a *.pdb in whatever directory you're running the script... and as I read the code - I don't see anything to imply it's within 'frames' when it runs the perl script.
you probably need os.chdir(path) before the Popen.
How do I "cd" in Python?
...using a python script to run somewhat dubious syscalls to perl may offend some people but everyone's done it.. aside from that I'd point out:
always specify full paths (this becomes a problem if you will later say, want to run your job automatically from cron or an environment that doesn't have your PATH).
i.e. 'which perl' - put that full path in.
./.pdb would be better but not as good as the fullpath/.pdb (which you could use instead of the os.chdir option).
subprocess.Popen(arg, stdout=True)
does not expand filename wildcards. To handle your *.pdb, use shell=True.
I'm trying to to a ls -l from python, to check for the last modification date of a file.
os.listdir doesn't show the long list format.
subprocess.call shows the format, but actually prints it, and returns 0. I want to be able to put it in a variable. Any ideas ?
Also, I tried
subprocess.call("ls","*.py")
which answers
ls: cannot access *.py: No such file or directory
it works with shell=True, but if someone could explain why it doesn't work without it, I'll appreciate. If you know how to make it work, even better.
It doesn't work without shell=True because the * is a shell expansion character - going from *.py to a list of files ending in .py is a function performed by the shell itself, not ls or python.
If you want to get the output of a command invoked via subprocess, you should use subprocess.check_output() or subprocess.Popen.
ls_output = subprocess.check_output(['ls', '-l'])
With nice formatting:
import subprocess
print(subprocess.check_output(['ls', '-lh']).decode('utf-8'))