We Keep Coding

grep unable to locate string when executing from python

Edit: I use Ubuntu 20.04.2 LTS and python 3.8.5
I got a bigger project where there are unused translation keys. I wrote a shell script to do this job. I exported some of the code to 2 testing files to narrow down on the weird behaviour. I am able to reproduce the bug but I fail to understand why it happens.
if sudo grep -r --include=\*.{js,html,json} --exclude-dir={node_modules,locale} 404_custom_error ../myreactproject/
then
echo 'found'
else
echo 'not found'
fi
If I run it via the terminal, it gets found.
import os
import subprocess
os.system("./testing.sh")
subprocess.call(['sh', 'testing.sh'])
But when I execute it from a python file, it gets a 'not found'.
os.getcmd() -> /home/{name}/projects/translation-cleaner
echo $PWD -> /home/{name}/projects/translation-cleaner
Note: The python file and shell script are in the same directory. The other project is in /home/{name}/projects/myreactproject.
That specific key is being found in ../myreactproject/src/views/Pages/Page404/Page404Admin.js:

I have 2 solutions but I have not tested them before on my previous Windows 10 machine since I migrated now fully to linx:
The proper one:
subprocess.call(['bash', './testing.sh'])
The dirty one:
I had to remove all brackets from my shell script. so my grep command looks like this:
grep -qr --include=*.js --include=*.html --include=*.json --exclude-dir=node_modules --exclude-dir=locale 404_custom_error ../myreactproject/;

How to make GitHub action runners find an executable installed using `pip install --user`?

When using python3 -m pip install --user foo on github action runners, supposing foo package contains an executable bar.
I would like to know where the user bin directory is located on Linux, Windows and MacOS runners in order to add it to GITHUB_PATH.
So a subsequent call like command -v bar, which bar or bar won't fail...
EDIT:
On Linux, using bash, it will be:
run: echo "$HOME/.local/bin" >> $GITHUB_PATH
On Windows, using powershell, it could be:
run: echo "$((Get-Item ~).FullName)/AppData/Roaming/Python/Python37/Scripts" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
On MacOS, using bash, it will be:
run: echo "$HOME/Library/Python/3.9/bin" >> $GITHUB_PATH

Duplicate question:
https://stackoverflow.com/a/62167797
I believe the following should give the expected result
import os
import sysconfig
user_scripts_path = sysconfig.get_path('scripts', f'{os.name}_user')
print(user_scripts_path)
But it might be that pip uses a different logic internally (probably based on distutils), but the results should still be the same.
References
https://docs.python.org/3/library/sysconfig.html#installation-paths
https://docs.python.org/3/library/os.html#os.name

Error in check_call() subprocess, executing 'mv' unix command: "Syntax error: '(' unexpected"

I'm making a python script for Travis CI.
.travis.yml
...
script:
- support/travis-build.py
...
The python file travis-build.py is something like this:
#!/usr/bin/env python
from subprocess import check_call
...
check_call(r"mv !(my_project|cmake-3.0.2-Darwin64-universal) ./my_project/final_folder", shell=True)
...
When Travis building achieves that line, I'm getting an error:
/bin/sh: 1: Syntax error: "(" unexpected
I just tried a lot of different forms to write it, but I get the same result. Any idea?
Thanks in advance!
Edit
My current directory layout:
- my_project/final_folder/
- cmake-3.0.2-Darwin64-universal/
- fileA
- fileB
- fileC
I'm trying with this command to move all the current files fileA, fileB and fileC, excluding my_project and cmake-3.0.2-Darwin64-universal folders into ./my_project/final_folder. If I execute this command on Linux shell, I get my aim but not through check_call() command.
Note: I can't move the files one by one, because there are many others
I don't know which shell Travis are using by default because I don't specify it, I only know that if I write the command in my .travis.yml:
.travis.yml
...
script:
# Here is the previous Travis code
- mv !(my_project|cmake-3.0.2-Darwin64-universal) ./my_project/final_folder
...
It works. But If I use the script, it fails.
I found this command from the following issue:
How to use 'mv' command to move files except those in a specific directory?

You're using the bash feature extglob, to try to exclude the files that you're specifying. You'll need to enable it in order to have it exclude the two entries you're specifying.
The python subprocess module explicitly uses /bin/sh when you use shell=True, which doesn't enable the use of bash features like this by default (it's a compliance thing to make it more like original sh).
If you want to get bash to interpret the command; you have to pass it to bash explicitly, for example using:
subprocess.check_call(["bash", "-O", "extglob", "-c", "mv !(my_project|cmake-3.0.2-Darwin64-universal) ./my_project/final_folder"])
I would not choose to do the job in this manner, though.

Let me try again: in which shell do you expect your syntax !(...) to work? Is it bash? Is it ksh? I have never used it, and a quick search for a corresponding bash feature led nowhere. I suspect your syntax is just wrong, which is what the error message is telling you. In that case, your problem is entirely independent form python and the subprocess module.
If a special shell you have on your system supports this syntax, you need to make sure that Python is using the same shell when invoking your command. It tells you which shell it has been using: /bin/sh. This is usually just a link to the real shell executable. Does it point to the same shell you have tested your command in?
Edit: the SO solution you referenced contains the solution in the comments:
Tip: Note however that using this pattern relies on extglob. You can
enable it using shopt -s extglob (If you want extended globs to be
turned on by default you can add shopt -s extglob to .bashrc)
Just to demonstrate that different shells might deal with your syntax in different ways, first using bash:
$ !(uname)
-bash: !: event not found
And then, using /bin/dash:
$ !(uname)
Linux

The argument to a subprocess.something method must be a list of command line arguments. Use e.g. shlex.split() to make the string be split into correct command line arguments:
import shlex, subprocess
subprocess.check_call( shlex.split("mv !(...)") )
EDIT:
So, the goal is to move files/directories, with the exemption of some file(s)/directory(ies). By playing around with bash, I could get it to work like this:
mv `ls | grep -v -e '\(exclusion1\|exclusion2\)'` my_project
So in your situation that would be:
mv `ls | grep -v -e '\(myproject\|cmake-3.0.2-Darwin64-universal\)'` my_project
This could go into the subprocess.check_call(..., shell=True) and it should do what you expect it to do.

How can I eliminate the need of the command 'python' when running a python file in terminal?

I would like to run python files from terminal without having to use the command $ python. But I would still like to keep the ability of using '$ python to enter the python interpreter. For example if I had a file named 'foo.py', I can use $ foo.py rather than $ python foo.py to run the file.
How can I do this? Would I need to change the bash file or the paths? And is it possible to have both commands available, so I can use both $ foo.py and $ python foo.py?
I am using ubuntu 14.04 LTS and my terminal/shell uses a '.bashrc' file. I have multiple versions of python installed on my computer, but when running a python file I want the default version to be the latest version of 2.7.x. If what I am asking is not possible or not recommended, I want to at least shorten the command $ python to $ py.
Thank you very much for any help!

Yes you can do this.
1) Ensure that the first line in your file looks like this:
#!/usr/bin/env python
2) Ensure that your files have the execute permission bit set, like this:
$ chmod +x foo.py
3) Now, assuming that your $PATH environment variable is set correctly*, you can run foo.py either way:
$ foo.py
$ python foo.py
* By "correctly", I mean, "to include the directory where your python file lives." In the use case you describe, that probably means "to include the current directory". To do that, edit .bashrc. One possible line to put in .bashrc is: PATH="$PATH":.

sharkbyte,
It's easy to insert '#!/usr/bin/env python' at the top of all your Python files. Just run this sed command in the directory where your Python files live:
sed -i '1 i\#! /usr/bin/env python\n' *.py
The -i option tells sed to do an in-place edit of the files, the 1 means operate only on line 1, and the i\ is the insert command. I put a \n at the end of the insertion text to make the modified file look nicer. :)
If you're paranoid of stuffing up, copy some files to an empty directory first & do a test run.
Also, you can tell sed to make a backup of each original file, eg:
sed -i.bak '1 i\#! /usr/bin/env python\n' *.py
for MS style, or
sed -i~ '1 i\#! /usr/bin/env python\n' *.py
for the usual Linux style.

CMD in Windows 7 does not execute command (Python Django)

Ok people at this link for pysec as the technical solution is explained we have some code that you must type to the command prompt (i think because it has a dollar sign in front):
$ cd ~/path/to/pysec && python -c "import sqlite3; sqlite3.connect('edgar.db')"
$ mv ./local-settings-example.py ./local-settings.py
$ mkdir ./pysec/data
However whenever i go to C:\Python27\pysec-master which is the location where the pysec file is stored (according to instructions) and type these commands exactly as i see them i get that the system cannot find the path specified.
Like this
C:\Python27\pysec-master>cd ~/path/to/pysec && python -c
cmd response --> The system cannot find the path specified.
C:\Python27\pysec-master>cd ~/path/to/pysec && python -c "import sqlite3; sqlite3.connect('edgar.db')"
cmd response --> The system cannot find the path specified.
C:\Python27\pysec-master>mv ./local-settings-example.py ./local-settings.py
cmd response --> 'mv' is not recognized as an internal or external command, operable program or batch file.
C:\Python27\pysec-master>mkdir ./pysec/data
cmd response --> The syntax of the command is incorrect.
What seems to be the problem? Don't you have to type these commands in the cmd since they have a dollar sign?

ANSWER FOR THIS QUESTION IS PROVIDED FROM THE COMMENTS UNDER THE QUESTION BY USER Stephan
I decided to put the all together in on place>
cd ~..., mv .` and mkdir .\ looks more like unix syntax than windows-cmd. cd and mkdir work on both platforms, but with different syntax. The cmd-version of mv is move.(ANSWER)
also /path/to/pysec tells you, that you should put in the path to pysec, not the string "\path\to\pysec"(ANSWER)
Can we transform these commands to Windows syntax?(QUESTION)
Should we put the path to pysec like this: C:\Python27\pysec-master i mean the full or absolute as it is called path? Because it that tutorial I can see that the example is trimmed as cd ~/path/to/pysec`(QUESTION)
the tilde (~) has a special meaning in unix. I don't speak unix, but I think it means "Systemdrive". The CMD command would be: cd /d "c:\Python27\pysec-master" (in CMD use \, in unix it's /). Instead of mv use move (ANSWER)
Only the third command does not seem to work mkdir ./pysec/data well I think there muse be something different for windows (QUESTION)
mkdir .\pysec\data ... You remember? "in CMD use \, in unix it's /"(ANSWER)
THANK YOU FOR THE SUPPORT