I am trying to build a Python package with some wrapped C++ code on Max osx via Travis CI. This is my build config:
{
"os": "osx",
"env": "PYTHON=3.6 CPP=14 CLANG DEBUG=1",
"sudo": false,
"script": [
"python setup.py install",
"py.test"
],
"install": [
"if [ \"$TRAVIS_OS_NAME\" = \"osx\" ]; then\n if [ \"$PY\" = \"3\" ]; then\n brew update && brew upgrade python\n else\n curl -fsSL https://bootstrap.pypa.io/get-pip.py | $PY_CMD - --user\n fi\n fi\nif [[ \"${TRAVIS_OS_NAME}\" == \"osx\" ]]; then\n export CXX=clang++ CC=clang;\n # manually install python on osx\n brew update\n brew install python3\n brew reinstall gcc\n virtualenv venv\n source venv/bin/activate\n pip install -r requirements.txt --upgrade\nfi\n",
"pip install -r requirements.txt --upgrade",
"python --version"
],
"language": "python",
"osx_image": "xcode9"
}
I get the following build error:
2.7 is not installed; attempting download
Downloading archive: https://s3.amazonaws.com/travis-python-archives/binaries/osx/10.12/x86_64/python-2.7.tar.bz2
$ curl -sSf -o python-2.7.tar.bz2 ${archive_url}
curl: (22) The requested URL returned error: 403 Forbidden
Unable to download 2.7 archive. The archive may not exist. Please consider a different version.
I'm not sure what to do about this.
Related
I am trying to synthesize a CDK app (typeScript) which has some python lambda functions.
I am using PythonFunction to use a requirements.txt file to install the external dependencies. I am running vscode on WSL. I am encountering the following error.
Bundling asset Test/test-lambda-stack/test-subscriber-data-validator-poc/Code/Stage...
node:internal/fs/utils:347
throw err;
^
Error: ENOENT: no such file or directory, open '~/.nvm/versions/node/v16.17.0/lib/node_modules/docker/node_modules/highlight.js/styles/cp -rTL /asset-input/ /asset-output && cd /asset-output && python -m pip install -r requirements.txt -t /asset-output.css'
at Object.openSync (node:fs:594:3)
at Object.readFileSync (node:fs:462:35)
at module.exports (~/.nvm/versions/node/v16.17.0/lib/node_modules/docker/src/getColourScheme.js:47:26)
at ~/.nvm/versions/node/v16.17.0/lib/node_modules/docker/src/docker.js:809:47
at FSReqCallback.readFileAfterClose [as oncomplete] (node:internal/fs/read_file_context:68:3)
at FSReqCallback.callbackTrampoline (node:internal/async_hooks:130:17) {
errno: -2,
syscall: 'open',
code: 'ENOENT',
path: '~/.nvm/versions/node/v16.17.0/lib/node_modules/docker/node_modules/highlight.js/styles/cp -rTL /asset-input/ /asset-output && cd /asset-output && python -m pip install -r requirements.txt -t /asset-output.css'
}
Error: Failed to bundle asset Test/test-lambda-stack/test-subscriber-data-validator-poc/Code/Stage, bundle output is located at ~/Code/AWS/CDK/test-dev-poc/cdk.out/asset.6b577fe604573a3b53e635f09f768df3f87ad6651b18e9f628c2a086a525bb49-error: Error: docker exited with status 1
at AssetStaging.bundle (~/Code/AWS/CDK/test-dev-poc/node_modules/aws-cdk-lib/core/lib/asset-staging.js:2:614)
at AssetStaging.stageByBundling (~/Code/AWS/CDK/test-dev-poc/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:4506)
at stageThisAsset (~/Code/AWS/CDK/test-dev-poc/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:1867)
at Cache.obtain (~/Code/AWS/CDK/test-dev-poc/node_modules/aws-cdk-lib/core/lib/private/cache.js:1:242)
at new AssetStaging (~/Code/AWS/CDK/test-dev-poc/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:2262)
at new Asset (~/Code/AWS/CDK/test-dev-poc/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:736)
at AssetCode.bind (~/Code/AWS/CDK/test-dev-poc/node_modules/aws-cdk-lib/aws-lambda/lib/code.js:1:4628)
at new Function (~/Code/AWS/CDK/test-dev-poc/node_modules/aws-cdk-lib/aws-lambda/lib/function.js:1:2803)
at new PythonFunction (~/Code/AWS/CDK/test-dev-poc/node_modules/#aws-cdk/aws-lambda-python-alpha/lib/function.ts:73:5)
at new lambdaInfraStack (~/Code/AWS/CDK/test-dev-poc/lib/serviceInfraStacks/lambda-infra-stack.ts:24:40)
My requirements.txt file looks like this
attrs==22.1.0
jsonschema==4.16.0
pyrsistent==0.18.1
My cdk code is this
new PythonFunction(this,`${appName}-subscriber-data-validator-${stage}`,{
runtime: Runtime.PYTHON_3_9,
entry: join('lambdas/subscriber_data_validator'),
handler: 'lambda_hander',
index: 'subscriber_data_validator.py'
})
Do I need to install anything additional? I have esbuild installed as a devDependency. Having a real hard time getting this work. Any help is appreciated.
i am building a docker image, that will run a flask application.
when i do it locally with no problem i can build the image
my dockerfile:
FROM python:3.7
#RUN apt-get update -y
WORKDIR /app
RUN curl www.google.com
COPY requirements.txt requirements.txt
RUN pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org -r requirements.txt
my jenkinspipeline
pipeline {
agent {
label "linux_machine"
}
stages {
stage('Stage1') {
steps {
//sh 'docker --version'
//sh 'python3 --version'
//sh 'pip3 --version'
checkout([$class: 'GitSCM', branches: [[name: '*/my_branch']], extensions: [], userRemoteConfigs: [[credentialsId: 'credentials_', url: 'https://myrepo.git']]])
}
}
stage('Stage2'){
steps{
sh "docker build --tag tag1 --file path/to/docker_file_in_repo docker_folder_path"
}
}
}
}
i was able to install docker and jenkins locally in my machine and all works fine, but when i put it on the jenkins server with real agents i get:
File "/usr/local/lib/python3.7/site-packages/pip/_internal/network/auth.py", line 256, in handle_401
username, password, save = self._prompt_for_password(parsed.netloc)
File "/usr/local/lib/python3.7/site-packages/pip/_internal/network/auth.py", line 226, in _prompt_for_password
username = ask_input(f"User for {netloc}: ")
File "/usr/local/lib/python3.7/site-packages/pip/_internal/utils/misc.py", line 237, in ask_input
return input(message)
EOFError: EOF when reading a line
Removed build tracker: '/tmp/pip-req-tracker-i4mhh7vg'
The command '/bin/sh -c pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org -r requirements.txt' returned a non-zero code: 2
i try using --no-input but same error,
it seems that is asking for a user and password, why is that?
is the docker using the certification of the agent/host to pass that to the commands of the dockerfile?
any sugestion on how could i make this work?
thanks guys.
Unfortunately, the problem is not clear at all from the message. What happens is that pip gets a 401 Unauthorized from the package index. You have to provide credentials so it can log-in.
You can add --no-input so it doesn't try to ask for a password (where it then fails due to STDIN being unavailable). That doesn't solve the underlying problem of it being unable to connect.
I am trying to launch jupyter lab in VSCode remote server, capsuled by Docker, but got error saying
Unable to start session for kernel Python 3.8.5 64-bit. Select another kernel to launch with.
I set up Dockerfile and .devcontainer.json in workspace dir.
Do I also need docker-compose.yaml file for jupyter lab setting like port forwarding?
Or I can handle and replace docker-compose file by .devcontainer.json file?
Dockerfile:
FROM python:3.8
RUN apt-get update --fix-missing && apt-get upgrade -y
# Set Japanese UTF-8 as locale so Japanese can be used
RUN apt-get install -y locales \
&& locale-gen ja_JP.UTF-8
ENV LANG ja_JP.UTF-8
ENV LANGUAGE ja_JP:ja
ENV LC_ALL ja_JP.UTF-8
# RUN apt-get install zsh -y && \
# chsh -s /usr/bin/zsh
# Install zsh with theme and some plugins
RUN sh -c "$(wget -O- https://raw.githubusercontent.com/deluan/zsh-in-docker/master/zsh-in-docker.sh)" \
-t mrtazz \
-p git -p ssh-agent
RUN pip install jupyterlab
RUN jupyter serverextension enable --py jupyterlab
WORKDIR /app
CMD ["bash"]
.devcontainer.json
{
"name": "Python 3.8",
"build": {
"dockerfile": "Dockerfile",
"context": ".."
},
// Uncomment to use docker-compose
// "dockerComposeFile": "docker-compose.yml",
// "service": "dev",
// Set *default* container specific settings.json values on container create.
"settings": {
"terminal.integrated.shell.linux": "/bin/bash",
"python.pythonPath": "/usr/local/bin/python",
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.formatting.autopep8Path": "/usr/local/py-utils/bin/autopep8",
"python.formatting.blackPath": "/usr/local/py-utils/bin/black",
"python.formatting.yapfPath": "/usr/local/py-utils/bin/yapf",
"python.linting.banditPath": "/usr/local/py-utils/bin/bandit",
"python.linting.flake8Path": "/usr/local/py-utils/bin/flake8",
"python.linting.mypyPath": "/usr/local/py-utils/bin/mypy",
"python.linting.pycodestylePath": "/usr/local/py-utils/bin/pycodestyle",
"python.linting.pydocstylePath": "/usr/local/py-utils/bin/pydocstyle",
"python.linting.pylintPath": "/usr/local/py-utils/bin/pylint"
},
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"ms-python.python",
"teabyii.ayu",
"jeff-hykin.better-dockerfile-syntax",
"coenraads.bracket-pair-colorizer-2",
"file-icons.file-icons",
"emilast.logfilehighlighter",
"zhuangtongfa.material-theme",
"ibm.output-colorizer",
"wayou.vscode-todo-highlight",
"atishay-jain.all-autocomplete",
"amazonwebservices.aws-toolkit-vscode",
"hookyqr.beautify",
"phplasma.csv-to-table",
"alefragnani.bookmarks",
"mrmlnc.vscode-duplicate",
"tombonnike.vscode-status-bar-format-toggle",
"donjayamanne.githistory",
"codezombiech.gitignore",
"eamodio.gitlens",
"zainchen.json",
"ritwickdey.liveserver",
"yzhang.markdown-all-in-one",
"pkief.markdown-checkbox",
"shd101wyy.markdown-preview-enhanced",
"ionutvmi.path-autocomplete",
"esbenp.prettier-vscode",
"diogonolasco.pyinit",
"ms-python.vscode-pylance",
"njpwerner.autodocstring",
"kevinrose.vsc-python-indent",
"mechatroner.rainbow-csv",
"msrvida.vscode-sanddance",
"rafamel.subtle-brackets",
"formulahendry.terminal",
"tyriar.terminal-tabs",
"redhat.vscode-yaml"
],
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [8888],
// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "pip3 install -r requirements.txt",
// Comment out to connect as root instead.
// "remoteUser": "myname",
"shutdownAction": "none"
}
I'm trying to install mxnet on my MacBook Pro. The first step on their website was to install Homebrew by typing:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
in the Terminal.
This gave me the error below.
Conrados-MBP:~ conrados$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
curl: (1) Protocol "https" not supported or disabled in libcurl
How do I fix this?
Hey #Montmons, everything was SUCCESSFUL, but with only ONE error. The error says
Error: Could not link:
/usr/local/share/doc/homebrew
Please delete these paths and run `brew update`.
Below is the output from terminal when I typed
ruby -e "$(/usr/bin/curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Output from terminal:
Conrados-MBP:~ conrados$ ruby -e "$(/usr/bin/curl -fsSL
https://raw.githubusercontent.com/Homebrew/install/master/install)"
==> This script will install:
/usr/local/bin/brew
/usr/local/share/doc/homebrew
/usr/local/share/man/man1/brew.1
/usr/local/share/zsh/site-functions/_brew
/usr/local/etc/bash_completion.d/brew
/usr/local/Homebrew
==> The following new directories will be created:
/usr/local/sbin
==> The Xcode Command Line Tools will be installed.
Press RETURN to continue or any other key to abort
==> /usr/bin/sudo /bin/mkdir -p /usr/local/sbin
Password:
==> /usr/bin/sudo /bin/chmod g+rwx /usr/local/sbin
==> /usr/bin/sudo /bin/chmod 755 /usr/local/share/zsh /usr/local/share/zsh/site-functions
==> /usr/bin/sudo /usr/sbin/chown conrados /usr/local/sbin
==> /usr/bin/sudo /usr/bin/chgrp admin /usr/local/sbin
==> /usr/bin/sudo /bin/mkdir -p /Library/Caches/Homebrew
==> /usr/bin/sudo /bin/chmod g+rwx /Library/Caches/Homebrew
==> /usr/bin/sudo /usr/sbin/chown conrados /Library/Caches/Homebrew
==> Searching online for the Command Line Tools
==> /usr/bin/sudo /usr/bin/touch /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress
==> Installing Command Line Tools (macOS High Sierra version 10.13) for Xcode-9.3
==> /usr/bin/sudo /usr/sbin/softwareupdate -i Command\ Line\ Tools\ (macOS\ High\ Sierra\ version\ 10.13)\ for\ Xcode-9.3
Software Update Tool
Downloading Command Line Tools (macOS High Sierra version 10.13) for Xcode
Downloaded Command Line Tools (macOS High Sierra version 10.13) for Xcode
Installing Command Line Tools (macOS High Sierra version 10.13) for Xcode
Done with Command Line Tools (macOS High Sierra version 10.13) for Xcode
Done.
==> /usr/bin/sudo /bin/rm -f /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress
Password:
==> /usr/bin/sudo /usr/bin/xcode-select --switch /Library/Developer/CommandLineTools
==> Downloading and installing Homebrew...
HEAD is now at 50059990 Merge pull request #4053 from MikeMcQuaid/string-undent-fix
Updated 1 tap (homebrew/core).
==> Cleaning up /Library/Caches/Homebrew...
==> Migrating /Library/Caches/Homebrew to /Users/conrados/Library/Caches/Homebre
==> Deleting /Library/Caches/Homebrew...
==> New Formulae
patchelf
==> Updated Formulae
sqlite ✔ fobis ncmpcpp sqlite-analyzer
afsctool gist nss wildfly-as
apibuilder-cli git-sizer php xrootd
cimg jenkins php#5.6 xtensor
coreos-ct jenkins-lts php#7.0 yaws
dbhash knot php#7.1
di libqalculate sops
fibjs maxwell sqldiff
Error: Could not link:
/usr/local/share/doc/homebrew
Please delete these paths and run `brew update`.
==> Installation successful!
==> Homebrew has enabled anonymous aggregate user behaviour analytics.
Read the analytics documentation (and how to opt-out) here:
https://docs.brew.sh/Analytics.html
==> Next steps:
- Run `brew help` to get started
- Further documentation:
https://docs.brew.sh
Conrados-MBP:~ conrados$
The following answer is copied from here: Apple StackExchange
I am summarising my answer from comments.
In your case it seems that something has installed a version of curl without https support into your local folder. Maybe you tried to compile it yourself (the standard .configure && make && make install procedure installs programs into /usr/local for example.
You can force the command to use system curl by replacing curl by /usr/bin/curl in it, thus use:
ruby -e "$(/usr/bin/curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
When you have Homebrew successfully installed you can re-install the curl in your local folder by using:
brew install curl
There might be some additional steps as you have to overwrite files that are already there. You might need to use the --overwrite option.
I am trying to install python package "M2Crypto" via requirements.txt and I receive the following error message:
/usr/include/openssl/opensslconf.h:36: Error: CPP #error ""This openssl-devel package does not work your architecture?"". Use the -cpperraswarn option to continue swig processing.
error: command 'swig' failed with exit status 1
I tried passing
option_name: SWIG_FEATURES
value: "-cpperraswarn -includeall -I/usr/include/openssl"
But the error persists. Any idea?
The following config file (placed in .ebextensions) works for me:
packages:
yum:
swig: []
container_commands:
01_m2crypto:
command: 'SWIG_FEATURES="-cpperraswarn -includeall -D`uname -m` -I/usr/include/openssl" pip install M2Crypto==0.21.1'
Make sure you don't specify M2Crypto in your requirements.txt though, Elastic Beanstalk will try to install all dependencies before running the container commands.
I have found a solution that gets M2Crypto installed on Beanstalk but it is a bit of hack and it is your responsibility to make sure that it is good for a production environment. I dropped M2Crypto from my project because this issue is ridiculous, try pycrypto if you can.
Based on (I only added python setup.py test):
#!/bin/bash
python -c "import M2Crypto" 2> /dev/null
if [ "$?" == 1 ]
then
cd /tmp/
pip install -d . --use-mirrors M2Crypto==0.21.1
tar xvfz M2Crypto-0.21.1.tar.gz
cd M2Crypto-0.21.1
./fedora_setup.sh build
./fedora_setup.sh install
python setup.py test
fi`
In the environment config file
commands:
m2crypto:
command: scripts/m2crypto.sh
ignoreErrors: True
test: echo '! python -c "import M2Crypto"' | bash
ignoreErrors is NOT a good idea but I just used it to test if the package actually gets installed and seems like it.
Again, this might seem to get the package installed but I am not sure because removing ignoreErrors causes failure. Therefore, I won't mark this as the accepted answer but it was way too much to be a comment.