We Keep Coding

Issue in creating a layer for AWS lambda for psycopg2

I am trying to create a layer for aws lambda for psycopg. I am following the answer to the question - psycopg2 folder zipped into python venv lambda function
Screenshot of the process from above link
However I am getting the below error while executing the docker step.
$ docker run "lambci/lambda:build-python3.8" /bin/sh -c "pip install -r requirements.txt -t /python/lib/python3.8/site-packages/; exit"
**docker: Error response from daemon: failed to create shim task: OCI
runtime create failed: runc create failed: unable to start container
process: exec: "C:/Users/aameen/AppData/Local/Programs/Git/usr/bin/sh":
stat C:/Users/aameen/AppData/Local/Programs/Git/usr/bin/sh: no such file
or directory: unknown.**
I am executing this on the git bash on windows. May be the problem lies there.

Dockerfile RUN command return an exec format error

I am trying to create and image from my Python Flask app, but I am still getting this format error and I have no idea why. (I am running it on a Raspberry with Raspbian)
Step 4/5 : RUN echo "test" ---> Running in ca7b48ab3d4e
standard_init_linux.go:211: exec user process caused "exec format
error"
The command '/bin/sh -c echo "test"' returned a non-zero code: 1
Here is my Dockerfile
#!/usr/bin/env bash
FROM python:3.6.1-alpine
WORKDIR /project
ADD . /project
RUN echo "test"
CMD ["python3","careerdigest.py"]
I think that there is a problem with bash, because it says, that it calls this command with sh, but I do not know, what I am missing.
This echo command will be replaced by RUN pip3 install -r requirements.txt to install dependencies.
Thanks for your help.

Okay, I manage to finally run it. I just replace Python image with this one - https://hub.docker.com/r/balenalib/raspberry-pi-python and now it works.

Activate Anaconda Python environment from makefile

I want to use a makefile to build my project's environment using a makefile and anaconda/miniconda, so I should be able to clone the repo and simply run make myproject
myproject: build
build:
#printf "\nBuilding Python Environment\n"
#conda env create --quiet --force --file environment.yml
#source /home/vagrant/miniconda/bin/activate myproject
If I try this, however, I get the following error
make: source: Command not found
make: *** [source] Error 127
I have searched for a solution, but [this question/answer(How to source a script in a Makefile?) suggests that I cannot use source from within a makefile.
This answer, however, proposes a solution (and received several upvotes) but this doesn't work for me either
( \
source /home/vagrant/miniconda/bin/activate myproject; \
)
/bin/sh: 2: source: not found
make: *** [source] Error 127
I also tried moving the source activate step to a separate bash script, and executing that script from the makefile. That doesn't work, and I assume for the a similar reason, i.e. I am running source from within a shell.
I should add that if I run source activate myproject from the terminal, it works correctly.

I had a similar problem; I wanted to create, or update, a conda environment from a Makefile to be sure my own scripts could use the python from that conda environment.
By default make uses sh to execute commands, and sh doesn't know source (also see this SO answer). I simply set the SHELL to bash and ended up with (relevant part only):
SHELL=/bin/bash
CONDAROOT = /my/path/to/miniconda2
.
.
install: sometarget
source $(CONDAROOT)/bin/activate && conda env create -p conda -f environment.yml && source deactivate
Hope it helps

You should use this, it's functional for me at moment.
report.ipynb : merged.ipynb
( bash -c "source ${HOME}/anaconda3/bin/activate py27; which -a python; \
jupyter nbconvert \
--to notebook \
--ExecutePreprocessor.kernel_name=python2 \
--ExecutePreprocessor.timeout=3000 \
--execute merged.ipynb \
--output=$< $<" )

I had the same problem. Essentially the only solution is stated by 9000. I have a setup shell script inside which I setup the conda environment (source activate python2), then I call the make command. I experimented with setting up the environment from inside Makefile and no success.
I have this line in my makefile:
installpy :
./setuppython2.sh && python setup.py install
The error messages is:
make
./setuppython2.sh && python setup.py install
running install
error: can't create or remove files in install directory
The following error occurred while trying to add or remove files in the
installation directory:
[Errno 13] Permission denied: '/usr/lib/python2.7/site-packages/test-easy-install-29183.write-test'
Essentially, I was able to set up my conda environment to use my local conda that I have write access. But this is not picked up by the make process. I don't understand why the environment set up in my shell script using 'source' is not visible in the make process; the source command is supposed to change the current shell. I just want to share this so that other people don't wast time trying to do this. I know autotoools has a way of working with python. But the make program is probably limited in this respect.
My current solution is a shell script:
cat py2make.sh
#!/bin/sh
# the prefix should be change to the target
# of installation or pwd of the build system
PREFIX=/some/path
CONDA_HOME=$PREFIX/anaconda3
PATH=$CONDA_HOME/bin:$PATH
unset PYTHONPATH
export PREFIX CONDA_HOME PATH
source activate python2
make
This seems to work well for me.
There were a solution for similar situation but it does not seems to work for me:
My modified Makefile segment:
installpy :
( source activate python2; python setup.py install )
Error message after invoking make:
make
( source activate python2; python setup.py install )
/bin/sh: line 0: source: activate: file not found
make: *** [installpy] Error 1
Not sure where am I wrong. If anyone has a better solution please share it.

Starting qsub in parent folder with Python fails

I'm creating a Python script to setup a series of calculations on a remote calculation cluster. However I have run into a problem.
The calculation on the remote server is started with the command:
qsub Run.sh
Run.sh is located in the folder ./sol/
If I enter ./sol/ and run:
python25 -c "import os;os.system(\"qsub Run.sh\")"
Every thing works like it should.
However, If I am located in ./ and run this command:
python25 -c "import os;os.chdir(\"sol\");os.system(\"qsub Run.sh\")"
qsub failes with this error message:
*** error from copy
Host key verification failed.
lost connection
*** end error output
Does anyone know why this is? I use python version 2.5.1 and unix 2.6.18

Python Fabric error when executing git archive command

I'm running Fabric on a windows 7 machine, when I type:
fab production setup
I get the following error:
[localhost] run: git archive --format=tar master > 20110221142115.tar
Warning: local() encountered an error (return code 1) while executing 'git archi
ve --format=tar master > 20110221142115.tar'
the fabric file works fine on my Windows XP machine, but I can't seem to get past this error. I also tried running the "git archive" command from the prompt, it works fine no issues. Any idea why Fabric is puking?

If the same script/command works on Windows XP, then did you check if Windows 7 firewall or any sort-of-protection is not allowing your connection to go through? Try if the following works on interactive prompt:
from fabric.api import local
local('git archive')