I have a problem when I was used CI/CD on the gitlab. I always use "python:latest" but it is 2.7.5 version, but I want to use python2.7.15 or python3.7. How can I install it?
-
image: python:latest
services:
- mongo:latest
variables:
MONGO_DB: ipc_alert
cache:
paths:
- ~/.cache/pip/
before_script:
- python -V
- pip install -r req.txt
stages:
- test
test:
stage: test
script:
- echo 'Testing'
On the image you're posting, you have a different problem. You cannot find django on a valid version for your requirements.
About the question itself, if you want to test against multiple versions, you need to create more than one test. For example:
test:
stage: test
script:
- echo 'Testing'
That's will be:
test-python2.7:
stage: test
image: python:2.7
script:
- echo 'Testing'
test-python3.4:
stage: test
image: python:3.4
script:
- echo 'Testing'
test-python3.5:
stage: test
image: python:3.5
script:
- echo 'Testing'
test-python3.6:
stage: test
image: python:3.6
script:
- echo 'Testing'
test-python3.7:
stage: test
image: python:3.7
script:
- echo 'Testing'
test-python.latest:
stage: test
image: python:latest
script:
- echo 'Testing'
However, maybe this doesn't work, because you're using a "Shell executor". If I remember correctly, this runner execute your code against the current machine. You need to install docker and create a new runner who uses these docker. Without it, you cannot test against different environments / versions.
One exception to this, are if you have all python versions you need installed on your machine, and calls each python concrete version. It depends on your environment, but you can check on /usr/bin if you have multiple python versions. On my machine, I have on /usr/bin these ones:
maqui#kanade:~$ python -V
Python 2.7.15+
maqui#kanade:~$ python2.6 -V
Python 2.6.8
maqui#kanade:~$ python2.7 -V
Python 2.7.15+
maqui#kanade:~$ python3.6 -V
Python 3.6.8rc1
maqui#kanade:~$ python3.7 -V
Python 3.7.2rc1
(As you can see, python is an alias for python2.7).
Related
I apologize in advance.I have a task to create CI pipeline in Gitlab for projects on python language with results in SonarQube. I found some gitlab-ci.yml file:
image: image-registry/gitlab/python
before_script:
- cd ..
- git clone https://gitlab-ci-token:${CI_JOB_TOKEN}#gitlab/python-education/junior.git
stages:
- PyLint
pylint:
stage: PyLint
only:
- merge_requests
script:
- cp -R ${CI_PROJECT_NAME}/* junior/project
- cd junior && python3 run.py --monorepo
Is it possible to some code in script to output in SonarQube?
Yes, third party issues are supported with SonarQube. For PyLint, you can set sonar.python.pylint.reportPath in your sonar.properties file with the path of the report(s) for pylint. You must use --output-format=parseable argument to pylint.
When you run sonar scanner, it will collect the report(s) and send it to SonarQube.
I decided to build my pipeline on this plan:
Build stage: Run only if the branch is the main one or one of my build files has been modified. It inherits docker:latest, and builds a test-ready container (pytest, lint) and pushes it to the local registry.
Test stage: always runs, inherits the latest or own branch container from the previous stage. All tests are run in it.
Push to production: it doesn't matter now.
Problems in 2 stage:
I run the ls -la command and I don't see my venv, node_modules folders. I thought GIT_CLEAN_FLAGS would solve my problem. But it didn't help.
How reproduce the problem:
Building image
FROM python:3.7-slim
ARG CI_PROJECT_DIR
WORKDIR $CI_PROJECT_DIR
RUN pip install -r requirements.txt
build:
stage: build
tags:
- build
script:
- docker build --build-arg CI_PROJECT_DIR=$CI_PROJECT_DIR .
Test
lint:
variables:
GIT_CLEAN_FLAGS: none
stage: test
tags:
- test
script:
- pwd
- ls -lah
You don't need to use CI_PROJECT_DIR. Save your code in another directory:
/my-app for example.
And in your second stage use cd /my-app.
Code example of your second stage:
test:
stage: test
tags:
- test
before_script:
- cd /my-app
script:
- pwd
- ls -lah
When tests are launched in GitLab CI, pytest-sugar doesn't show output like in local launching. What the problem can be?
My gitlab config:
image: project.com/path/dir
stages:
- tests
variables:
TESTS_ENVIRORMENT:
value: "--stage my_stage"
description: "Tests launch my_stage as default"
before_script:
- python3 --version
- pip3 install --upgrade pip
- pip3 install -r requirements.txt
api:
stage: tests
script:
- pytest $TESTS_ENVIRORMENT Tests/API/ -v
Local:
GitLab:
It seems that there's a problem with pytest-sugar inside containers. Add --force-sugar option to pytest call, it worked for me
By default docker container do not allocate a pseudo-terminal(tty) as a result not stdout, its simple output from console.
There is not clear solution for that case, mostly needs to do workarounds and try special python libraries.
I'm having trouble implementing an sample program that runs pytest within .gitlab-ci.yml on Windows:
Using Shell executor...
Please find below .gitlab-ci.yml:
# .gitlab-ci.yml
test_sample:
stage: test
tags:
- test_sam
script:
- echo "Testing"
- pytest -s Target\tests
when: manual
CI/CD terminal output:
pytest is not recognized as the name of a cmdlet, function, script file, or operable program.
Python and pytest is already installed on the Windows OS on which the test is running but still the test fails.
I have tried the solution suggested in below thread but it doesn't work:
gitlab-ci.yml: 'script: -pytest cannot find any tests to check'
Could you please suggest how to make this test pass on windows?
If python is recognized, you could replace pytest, as with this similar project, with:
unittests:
script: python -m unittest discover tests -v
core doctests:
script: python -m doctest -v AmpScan/core.py
registration doctests:
script: python -m doctest -v AmpScan/registration.py
align doctests:
script: python -m doctest -v AmpScan/align.py
(The initial switch to pytest failed)
If you want to use pytest, you would need to use a python Docker image in your .gitlab.yml.
See "Setting Up GitLab CI for a Python Application" from Patrick Kennedy.
image: "python:3.7"
before_script:
- python --version
- pip install -r requirements.txt
stages:
- Static Analysis
- Test
...
unit_test:
stage: Test
script:
- pwd
- ls -l
- export PYTHONPATH="$PYTHONPATH:."
- python -c "import sys;print(sys.path)"
- pytest
The below command worked without any issues:
py -m pytest -s Target\tests
I have the following yaml pipeline build file:
pr:
branches:
include:
- master
jobs:
- job: 'Test'
pool:
vmImage: 'Ubuntu-16.04'
strategy:
matrix:
Python36:
python.version: '3.6'
maxParallel: 4
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '$(python.version)'
architecture: 'x64'
env:
POSTGRES: $(POSTGRES)
- script: python -m pip install --upgrade pip && pip install -r requirements.txt
displayName: 'Install dependencies'
- script: |
pip install pytest
pytest tests -s --doctest-modules --junitxml=junit/test-results.xml
displayName: 'pytest'
I set the variable POSTGRES in the pipeline settings as a secret variable. In the python code all environment variables are read with the call
if not os.getenv(var):
raise ValueError(f'Environment variable \'{var}\' is not set')
When the build is executed it will throw exactly the above error for the POSTGRES variable. Are the environment variables not set correctly?
To make the environment variable available in the Python script, you need to define it in the step where it's used:
- script: |
pip install pytest
pytest tests -s --doctest-modules --junitxml=junit/test-results.xml
displayName: 'pytest'
env:
POSTGRES: $(POSTGRES)
I don't know if you still need this but...
If you take a look at the documentation here it says:
Unlike a normal variable, they are not automatically decrypted into
environment variables for scripts. You can explicitly map them in,
though.
So it looks like you were doing it right. Maybe try using a different name for the mapped variable. It could be the name of the initial encrypted variable is confounding the mapping (because it's already a variable it won't remap it).