I am trying to run xargs on multiple files at once:
sh -c 'find . -name "*.py" | xargs pylint'
This will give me a single pylint score for all py files in a repo. However when I try to modify it to do both black and pylint, it loops through each file individually and gives me the pylint score and black diff on each file:
find . -name "*.py" | xargs -I % sh -c 'pylint %; black --check --diff %;'
Any way to pass in the py files in batch rather than each individually?
If you use -I option it runs given command once for each entry in input.
You can do this instead:
find . -name "*.py" |
xargs sh -c 'pylint "$#"; black --check --diff "$#"'
Related
I am running Ubuntu 22.04 with xorg.
I need to find a way to compile microbit python code locally to a firmware hex file. Firstly, I followed the guide here https://microbit-micropython.readthedocs.io/en/latest/devguide/flashfirmware.html.
After a lot of debugging, I got to this point: https://pastebin.com/MGShD31N
However, the file platform.h does exist.
sawntoe#uwubuntu:~/Documents/Assignments/2022/TVP/micropython$ ls /home/sawntoe/Documents/Assignments/2022/TVP/micropython/yotta_modules/mbed-classic/api/platform.h
/home/sawntoe/Documents/Assignments/2022/TVP/micropython/yotta_modules/mbed-classic/api/platform.h
sawntoe#uwubuntu:~/Documents/Assignments/2022/TVP/micropython$
At this point, I gave up on this and tried using Mu editor with the AppImage. However, Mu requires wayland, and I am on xorg.
Does anyone have any idea if this is possible? Thanks.
Mu and the uflash command are able to retrieve your Python code from .hex files. Using uflash you can do the following for example:
uflash my_script.py
I think that you want is somehow possible to do, but its harder than just using their web python editor: https://python.microbit.org/v/2
Peter Till answers the original question. The additional below adds to this answer by showing how to automate the build and load process. I use Debian. The original question states that Ubuntu is used, which is built on Debian.
A script to find and mount the micro:bit
When code is loaded to the micro:bit, the board is dismounted from the system. So each time you have new code to load, you have to remount the board.
I modified a script to find and mount the micro:bit.
#!/bin/bash
BASEPATH="/media/$(whoami)/"
MICRO="MICROBIT"
if [ $# -eq 0 ]
then
echo "no argument supplied, use 'mount' or 'unmount'"
exit 1
fi
if [ $1 == "--help" ]
then
echo "mounts or unmounts a BBC micro:bit"
echo "args: mount - mount the microbit, unmout - unmount the microbit"
fi
# how many MICRO found in udiksctl dump
RESULTS=$(udisksctl dump | grep IdLabel | grep -c -i $MICRO)
case "$RESULTS" in
0 ) echo "no $MICRO found in 'udkisksctl dump'"
exit 0
;;
1 ) DEVICELABEL=$(udisksctl dump | grep IdLabel | grep -i $MICRO | cut -d ":" -f 2 | sed 's/^[ \t]*//')
DEVICE=$(udisksctl dump | grep -i "IdLabel: \+$DEVICELABEL" -B 12 | grep " Device:" | cut -d ":" -f 2 | sed 's/^[ \t]*//')
DEVICEPATH="$BASEPATH""$DEVICELABEL"
echo "found one $MICRO, device: $DEVICE"
if [[ -z $(mount | grep "$DEVICE") ]]
then
echo "$DEVICELABEL was unmounted"
if [ $1 == "mount" ]
then
udisksctl mount -b "$DEVICE"
exit 0
fi
else
echo "$DEVICELABEL was mounted"
if [ $1 == "unmount" ]
then
udisksctl unmount -b "$DEVICE"
exit 0
fi
fi
;;
* ) echo "more than one $MICRO found"
;;
esac
echo "exiting without doing anything"
I alias this script to mm in my .bashrc file.
Automate mounting the micro:bit and flashing the python file
I use the inotifywait command to run mm and to then run uflash to load the .py file I am working on. Each time that the python file is saved, the aliased command mm is run followed by the uflash command.
while inotifywait -e modify <your_file>.py ; do mm && uflash <your_file>.py ; done
Okay, so elaborating on Peter Till's answer.
Firstly, you can use uflash:
uflash path/to/your/code .
Or, you can use microfs:
ufs put path/to/main.py
Working Ubuntu 22.04 host CLI setup with Carlos Atencio's Docker to build your own firmware
After trying to setup the toolchain for a while, I finally decided to Google for a Docker image with the toolchain, and found https://github.com/carlosperate/docker-microbit-toolchain at this commit from Carlos Atencio, a Micro:Bit foundation employee, and that just absolutely worked:
# Get examples.
git clone https://github.com/bbcmicrobit/micropython
cd micropython
git checkout 7fc33d13b31a915cbe90dc5d515c6337b5fa1660
# Get Docker image.
docker pull ghcr.io/carlosperate/microbit-toolchain:latest
# Build setup to be run once.
docker run -v $(pwd):/home --rm ghcr.io/carlosperate/microbit-toolchain:latest yt target bbc-microbit-classic-gcc-nosd#https://github.com/lancaster-university/yotta-target-bbc-microbit-classic-gcc-nosd
docker run -v $(pwd):/home --rm ghcr.io/carlosperate/microbit-toolchain:latest make all
# Build one example.
docker run -v $(pwd):/home --rm ghcr.io/carlosperate/microbit-toolchain:latest \
tools/makecombinedhex.py build/firmware.hex examples/counter.py -o build/counter.hex
# Build all examples.
docker run -v $(pwd):/home --rm ghcr.io/carlosperate/microbit-toolchain:latest \
bash -c 'for f in examples/*; do b="$(basename "$f")"; echo $b; tools/makecombinedhex.py build/firmware.hex "$f" -o "build/${b%.py}.hex"; done'
And you can then flash the example you want to run with:
cp build/counter.hex "/media/$USER/MICROBIT/"
Some further comments at: Generating micropython + python code `.hex` file from the command line for the BBC micro:bit
I have CI/CD Config which required python version to be set to default by pyenv. I want to python2 -V output showed up with only, example, 2.7.18. But, rather than showing 2.7.18, it showing full text Python 2.7.18 .
But, when I use it in python3 python -V, it showed the correct & current python3 version (3.9.0).
I use this code to try showing numbers only : $(python -V | grep -Eo '[0-9]\.[0-9]\.[10-19]').
And to set default with pyenv : pyenv global $(python3 -V | grep -Eo '[0-9]\.[0-9]\.[10-19]') $(python -V | grep -Eo '[0-9]\.[0-9]\.[10-19]')
So pyenv $(python3 version) $(python2 version)
Here is the image :
Image of wrong output
Thanks!
A simple way would be to just replace the string Python with the emtpy string, if it exists.
Here a quick one-liner
python -V 2>&1| sed -e "s/Python//g" | xargs
That would print the python version, redirects stderr to stdout, replaces "Python" with "". Xargs without parameters returns the trimmed input string.
Here are a few more ways to get the version number:
# Print 1 word per line, the select the last word:
python -V 2>&1 | xargs -n1 | tail -n1
# Print the last word:
python -V 2>&1 | perl -lane 'print $F[-1];'
# Print the first stretch of 1 or more { digits or periods }:
python -V 2>&1 | grep -Po '[\d.]+'
We've got a Python application and want to count all the lines of code under a specific directory and its subdirectories.
We don't need to ignore comments but we want to ignore all files containing test cases.
Path to test cases file always has /tests/ in the path (e.g. /python/trieus/persistence/service/tests/batch_service_tests.py).
I used below command to find the count but it did not exclude test files.
find . -name "*.py" -not -path "./tests*" | xargs wc -l | sort
What's the correct syntax here?
You can do:
find . -type d -name tests -prune -o -type f -name '*.py' \
-exec grep -hxc '.*' {} + | paste -sd+ | bc
find . -type d -name tests -prune excludes the tests directory
-type f -name '*.py' matches only .py files
-exec grep -hxc '.*' {} + get individual lines, you can modify the Regex pattern to meet you need here
paste -sd+ formats the output to put in a single line with + in between
bc does the addition on its STDIN data
I would suggest you to go one step at a time to understand the whole thing better.
As a side note, this will not get you the actual LOC, instead would get you the lines count only.
Example from running in an example directory on my system:
% find . -type d -name tests -prune -o -type f -name '*.py' -exec grep -hxc '.*' {} + | paste -sd+ | bc
5594
You can exclude a directory at ANY level by changing the first . to *. so find -name "*.py" -not -path "*/tests/*" would omit files from a directory named "tests" at any level of depth.
So the command will look like this:
find . -name "*.py" -not -path "*/tests/*" | xargs wc -l | sort
I have been following this tutorial to install opencv and python:
https://www.pyimagesearch.com/2015/06/22/install-opencv-3-0-and-python-2-7-on-ubuntu/#comment-441393
The only difference is that I am trying to install opencv 3.3.1 instead of 3.0.0
I'm running on a laptop with Ubuntu 14.04 an i7 and NVIDIA GTX950M
The problem is that when I execute the command ldconfig
$ sudo make install
$ sudo ldconfig
I get the following message:
/sbin/ldconfig.real: /usr/lib/nvidia-384/libEGL.so.1 is not a symbolic link
/sbin/ldconfig.real: /usr/lib32/nvidia-384/libEGL.so.1 is not a symbolic link
So I found a solution to the problem:
Source: https://askubuntu.com/questions/900285/libegl-so-1-is-not-a-symbolic-link #muru #Gerard Tromp
The following is an easy-to-use version of Noisy_Botnet's solution. It
facilitates repeating the process for any update.
Create a shell script, i.e., paste the code bellow a text file, and save it with the .sh extension.
Change the executing permissions of the file i.e., go to the location of the file in the terminal and execute the following command $sudo chmod 744 nameofthefieleyoucreated.sh
execute the following command $sudo ./nameofthefileyoucreated.sh
#! /bin/sh
#
# find the file in /usr/lib
LIBEGL=`find /usr/lib/nvidia* -name libEGL.so.\* | egrep "[0-9][0-9]*\.[0-9][0-9]*$"`
LIBEGL_LINK=`echo $LIBEGL | sed 's/[0-9][0-9]*\.[0-9][0-9]*$/1/'`
printf "\n\nThe following commands will be executed:\n+++++++++++++++++++++++++++++++++++++++\n"
printf "mv $LIBEGL_LINK ${LIBEGL_LINK}.orig\nln -s $LIBEGL $LIBEGL_LINK\n\n"
while true; do
read -p "Do you wish to perform these commands? " yn
case $yn in
[Yy]* ) mv $LIBEGL_LINK ${LIBEGL_LINK}.orig; ln -s $LIBEGL $LIBEGL_LINK ; break;;
[Nn]* ) break;;
* ) echo "Please answer yes or no.";;
esac
done
# find the file in /usr/lib32
LIBEGL=`find /usr/lib32/nvidia* -name libEGL.so.\* | egrep "[0-9][0-9]*\.[0-9][0-9]*$"`
LIBEGL_LINK=`echo $LIBEGL | sed 's/[0-9][0-9]*\.[0-9][0-9]*$/1/'`
printf "\n\nThe following commands will be executed:\n+++++++++++++++++++++++++++++++++++++++\n"
printf "mv $LIBEGL_LINK ${LIBEGL_LINK}.orig\nln -s $LIBEGL $LIBEGL_LINK\n\n"
while true; do
read -p "Do you wish to perform these commands? " yn
case $yn in
[Yy]* ) mv $LIBEGL_LINK ${LIBEGL_LINK}.orig; ln -s $LIBEGL $LIBEGL_LINK ; break;;
[Nn]* ) break;;
* ) echo "Please answer yes or no.";;
esac
done
Running this on osx...
cd ${BUILD_DIR}/mydir && for DIR in $(find ./ '.*[^_].py' | sed 's/\/\//\//g' | awk -F "/" '{print $2}' | sort |uniq | grep -v .py); do
if [ -f $i/requirements.txt ]; then
pip install -r $i/requirements.txt -t $i/
fi
cd ${DIR} && zip -r ${DIR}.zip * > /dev/null && mv ${DIR}.zip ../../ && cd ../
done
cd ../
error:
(env) ➜ sh package_lambdas.sh find: .*[^_].py: No such file or directory
why?
find takes as an argument a list of directories to search. You provided what appears to be regular expression. Because there is no directory named (literally) .*[^_].py, find returns an error.
Below I have revised your script to correct that mistake (if I understand your intention). Because I see so many ill-written shell scripts these days, I've taken the liberty of "traditionalizing" it. Please see if you don't also find it more readable.
Changes:
use #!/bin/sh, guaranteed to be on an Unix-like system. Faster than bash, unless (like OS X) it is bash.
use lower case for variable names to distinguish from system variables (and not hide them).
eschew braces for variables (${var}); they're not needed in the simple case
do not pipe output to /usr/bin/true; route it to dev/null if that's what you mean
rm -f by definition cannot fail; if you meant || true, it's superfluous
put then and do on separate lines, easier to read, and that's how the Bourne shell language was meant to be used
Let && and || serve as line-continuation, so you can see what's happening step by step
Other changes I would suggest:
Use a subshell when changing the working directory temporarily. When it terminates, the working directory is restored automatically (retained by the parent), saving you the cd .. step, and errors.
Use set -e to cause the script to terminate on error. For expected errors, use || true explicitly.
Change grep .py to grep '\.py$', just for good measure.
To avoid Tilting Matchstick Syndrome, use something other than / as a sed substitute delimiter, e.g., sed 's://:/:g'. But sed could be avoided altogether with awk -F '/+' '{print $2}'.
Revised version:
#! /bin/sh
src_dir=lambdas
build_dir=bin
mkdir -p $build_dir/lambdas
rm -rf $build_dir/*.zip
cp -r $src_dir/* $build_dir/lambdas
#
# The sed is a bit complicated to be osx / linux cross compatible :
# ( .//run.sh vs ./run.sh
#
cd $build_dir/lambdas &&
for L in $(find . -exec grep -l '.*[^_].py' {} + |
sed 's/\/\//\//g' |
awk -F "/" '{print $2}' |
sort |
uniq |
grep -v .py)
do
if [ -f $i/requirements.txt ]
then
echo "Installing requirements"
pip install -r $i/requirements.txt -t $i/
fi
cd $L &&
zip -r $L.zip * > /dev/null &&
mv $L.zip ../../ &&
cd ../
done
cd ../
The find(1) manpage says its args are [path ...] [expression], where "expression" consists of "primaries" and "operands" (-flags). '.*[^-].py' doesn't look like any expression, so it's being interpreted as a path, and it's reporting that there is no file named '.*[^-].py' in the working directory.
Perhaps you meant:
find ./ -regex '.*[^-].py'