I have the following yaml pipeline build file:
pr:
branches:
include:
- master
jobs:
- job: 'Test'
pool:
vmImage: 'Ubuntu-16.04'
strategy:
matrix:
Python36:
python.version: '3.6'
maxParallel: 4
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '$(python.version)'
architecture: 'x64'
env:
POSTGRES: $(POSTGRES)
- script: python -m pip install --upgrade pip && pip install -r requirements.txt
displayName: 'Install dependencies'
- script: |
pip install pytest
pytest tests -s --doctest-modules --junitxml=junit/test-results.xml
displayName: 'pytest'
I set the variable POSTGRES in the pipeline settings as a secret variable. In the python code all environment variables are read with the call
if not os.getenv(var):
raise ValueError(f'Environment variable \'{var}\' is not set')
When the build is executed it will throw exactly the above error for the POSTGRES variable. Are the environment variables not set correctly?
To make the environment variable available in the Python script, you need to define it in the step where it's used:
- script: |
pip install pytest
pytest tests -s --doctest-modules --junitxml=junit/test-results.xml
displayName: 'pytest'
env:
POSTGRES: $(POSTGRES)
I don't know if you still need this but...
If you take a look at the documentation here it says:
Unlike a normal variable, they are not automatically decrypted into
environment variables for scripts. You can explicitly map them in,
though.
So it looks like you were doing it right. Maybe try using a different name for the mapped variable. It could be the name of the initial encrypted variable is confounding the mapping (because it's already a variable it won't remap it).
Related
I need to publish the result, but could not and have looked through this site and made research but still having issue to publish test result. Do I need to edit certain file similar to the cypress.json to be able to resolve the issue of Publishing. Both Unit test and the coveratge are not publishing any result and files are not found. The code and also the result with error are highlighted below.
Code :
trigger:
branches:
include:
- #infra-ML-unit-test
variables:
- group: aws_details #just for backup roles approach
- group: snowflake-variables
- group: unit-test-dev
jobs:
- job: 'Unit_Test'
pool:
vmImage: 'windows-latest' # other options: 'macOS-latest', 'windows-latest', 'ubuntu-latest'
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '3.8'
displayName: 'Use Python 3.x'
- task: UsePythonVersion#0
inputs:
versionSpec: '3.8'
- script: |
pip install virtualenv
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
python -m pip install --upgrade pip setuptools sqlalchemy snowflake.sqlalchemy
echo "$(System.DefaultWorkingDirectory)"
echo "$(Build.StagingDirectory)"
pip show pandas
python -m pytest --log-file $SYSTEM_ARTIFACTSDIRECTORY/smoke-qa.log --junitxml=TEST-qa-smoke.xml -rP --excelreport=report.xls --html=pytest_report.html --self-contained-html
#displayName: 'Install python tools'
- job: publish_result
dependsOn: Unit_Test
condition: succeededOrFailed()
steps:
- task: PublishTestResults#2
displayName: 'Publish test result /010.xml'
inputs:
testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit, cTest
testResultsFiles: '**/TEST-*.xml'
testRunTitle: 010
mergeTestResults: true
Pytest result
Unit test and coverage
Is there any file to locate in the repository to update and link with publishing the coverate and test result since the error or warning is mentioning :
**
##[warning]No test result files matching /TEST-*.xml were found.
**
You have two options:
Install the pytest-azurepipelines package before running the tests:
pip install pytest pytest-azurepipelines
Make the PublishTestResults#2 task start the search from $(Pipeline.Workspace):
- task: PublishTestResults#2
displayName: 'Publish test result /010.xml'
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '$(Pipeline.Workspace)/junit/test-*.xml'
For more info see Python: Run tests
When tests are launched in GitLab CI, pytest-sugar doesn't show output like in local launching. What the problem can be?
My gitlab config:
image: project.com/path/dir
stages:
- tests
variables:
TESTS_ENVIRORMENT:
value: "--stage my_stage"
description: "Tests launch my_stage as default"
before_script:
- python3 --version
- pip3 install --upgrade pip
- pip3 install -r requirements.txt
api:
stage: tests
script:
- pytest $TESTS_ENVIRORMENT Tests/API/ -v
Local:
GitLab:
It seems that there's a problem with pytest-sugar inside containers. Add --force-sugar option to pytest call, it worked for me
By default docker container do not allocate a pseudo-terminal(tty) as a result not stdout, its simple output from console.
There is not clear solution for that case, mostly needs to do workarounds and try special python libraries.
I have a simple python dockerized application whose structure is
/src
- server.py
- test_server.py
Dockerfile
requirements.txt
in which the docker base image is Linux-based, and server.py exposes a FastAPI endpoint.
For completeness, server.py looks like this:
from fastapi import FastAPI
from pydantic import BaseModel
class Item(BaseModel):
number: int
app = FastAPI(title="Sum one", description="Get a number, add one to it", version="0.1.0")
#app.post("/compute")
async def compute(input: Item):
return {'result': input.number + 1}
Tests are meant to be done with pytest (following https://fastapi.tiangolo.com/tutorial/testing/) with a test_server.py:
from fastapi.testclient import TestClient
from server import app
import json
client = TestClient(app)
def test_endpoint():
"""test endpoint"""
response = client.post("/compute", json={"number": 1})
values = json.loads(response.text)
assert values["result"] == 2
Dockerfile looks like this:
FROM tiangolo/uvicorn-gunicorn:python3.7
COPY . /app
RUN pip install -r requirements.txt
WORKDIR /app/src
EXPOSE 8000
CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
At the moment, if I want to run the tests on my local machine within the container, one way to do this is
Build the Docker container
Run the container, get its name via docker ps
Run docker exec -it <mycontainer> bash and execute pytest to see the tests passing.
Now, I would like to run tests in Azure DevOps (Server) before pushing the image to my Docker registry and triggering a release pipeline. If this sounds an OK thing to do, what's the proper way to do it?
So far, I hoped that something along the lines of adding a "PyTest" step in the build pipeline would magically work:
I am currently using a Linux agent, and the step fails with
The failure is not surprising, as (I think) the container is not run after being built, and therefore pytest can't run within it either :(
Another way to solve the solve this is to include pytest commands in the Dockerfile and deal with the tests in a release pipeline. However I would like to decouple the testing from the container that is ultimately pushed to the registry and deployed.
Is there a standard way to run pytest within a Docker container in Azure DevOps, and get a graphical report?
Update your azure-pipelines.yml file as follows to run the tests in Azure Pipelines
Method-1 (using docker)
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Docker#2
inputs:
command: 'build'
Dockerfile: '**/Dockerfile'
arguments: '-t fast-api:$(Build.BuildId)'
- script: |
docker run fast-api:$(Build.BuildId) python -m pytest
displayName: 'Run PyTest'
Successfull pipeline screenshot
Method-2 (without docker)
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
strategy:
matrix:
Python37:
python.version: '3.7'
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '$(python.version)'
displayName: 'Use Python $(python.version)'
- script: |
python -m pip install --upgrade pip
pip install -r requirements.txt
displayName: 'Install dependencies'
- script: |
pip install pytest pytest-azurepipelines
python -m pytest
displayName: 'pytest'
BTW, I have one simple FastAPI project, you can reference if your want.
Test your docker script using pytest-azurepipelines:
- script: |
python -m pip install --upgrade pip
pip install pytest pytest-azurepipelines
pip install -r requirements.txt
pip install -e .
displayName: 'Install dependencies'
- script: |
python -m pytest /src/test_server.py
displayName: 'pytest'
Running pytest with the plugin pytest-azurepipelines will let you see your test results in the Azure Pipelines UI.
https://pypi.org/project/pytest-azurepipelines/
You can run your unit tests directly from within your Docker container using pytest-azurepipelines (that you need to install previously in the Docker image):
- script: |
docker run --mount type=bind,source="$(pwd)",target=/results \
--entrypoint /bin/bash my_docker_image \
-c "cd results && pytest"
displayName: 'tests'
continueOnError: true
pytest will create an xml file containing the test results, that will be made available to Azure DevOps pipeline thanks to the --mount flag in the docker run command. Then pytest-azurepipelines will publish directly the results to Azure DevOps.
I am currently running a suite of 37 pytest tests on a python application. I am building under python 3.7.2 and locally all tests passes. On Azure DevOps pipleline I also test for python 3.8. Here is the azure-pipelines.yml:
trigger:
- dev
pool:
vmImage: 'ubuntu-latest'
strategy:
matrix:
Python37:
python.version: '3.7'
Python38:
python.version: '3.8'
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '$(python.version)'
displayName: 'Use Python $(python.version)'
- script: |
python -m pip install --upgrade pip
pip install -r requirements.txt
displayName: 'Install dependencies'
- script: |
pip install pytest pytest-azurepipelines
pytest
displayName: 'pytest'
Under the 3.7 run I have a specific test that fails on assert with an actual calculated value:
self = <unit_tests.test_services_string_jb_router.TestServicesRectilinearRouter object at 0x7f5404e88fd0>
def test_create_block_when_called_calculates_correct_total_string_cable_length(self):
designer = BlockDesigner(CsvStore(self.config), self.router, self.config)
block = designer.create_block("BK1")
result = block.total_cable_length
> assert result == 4943.176
E assert 4932.926 == 4943.176
unit_tests/test_services_string_jb_router.py:28: AssertionError
The test however passes under the 3.8 run.
The fact that the code produces an actual value, just a wrong one, would suggest it gets an incorrect configuration but only under python 3.7 and not python 3.8!.
I run a setup_method at the start of test classes with a str(pathlib.Path(__file__).parent.absolute()) prefix to ensure cross platform compatibility.
Where do I even start looking for this issue?
...and there it passes without my doing anything about it. Assume it was some older config sitting in and uncleared cache somewhere in Azure.
I'm using the unaccent extension for Postgres and followed all the docs to get it to work (installed the extension directly via CREATE EXTENSION and put django.contrib.postgres in INSTALLED_APPS in django settings).
In local environment it's working perfectly however after building and deploying the app to Heroku it looks like it isn't installing django.contrib.postgres. Therefore when i try to use the functionality of my app that queries using unaccent i get the "Unsupported lookup 'unaccent' for CharField" that happens when you don't have django.contrib.postgres in INSTALLED_APPS.
In python shell printing settings.INSTALLED_APPS on local environment shows that django.contrib.postgres is there, but running it on Heroku shows it's missing. Is it unsupported by the buildpack for heroku/python or something or am i missing some config?
I tried to pip install the django-contrib-postgres backport for earlier versions of Django (put it in requirements.txt) to no avail. Python version is 3.6.7 and Django is 2.1.2. Creating the extension using migrations with UnaccentExtension doesn't change anything either and i'm sure it's not Postgres problem because querying directly on the database using unaccent(columnname) works as expected.
Thanks in advance.
Edit: YAML definition for Azure DevOps Pipelines and requirements.txt
requirements.txt
Django==2.1.2
django-cors-middleware==1.3.1
django-heroku==0.3.1
django-oauth-toolkit==1.2.0
djangorestframework==3.9.0
djangorestframework-camel-case==0.2.0
django-contrib-postgres==0.0.1
facepy==1.0.9
factory_boy==2.11.1
flake8==3.5.0
gunicorn==19.8.1
psycopg2-binary==2.7.5
pylint==2.1.1
pytest==3.9.1
pytest-cov==2.6.0
pytest-django==3.4.3
python-dateutil==2.7.5
raven==6.9.0
freezegun==0.3.11
mailchimp3==3.0.4
Build
pool:
name: Hosted VS2017
steps:
- task: UsePythonVersion#0
displayName: 'Use Python 3.6'
inputs:
versionSpec: 3.6
- script: 'pip install -r requirements.txt'
workingDirectory: 'back-end'
displayName: 'Restore dependencies'
- script: 'python manage.py collectstatic'
workingDirectory: 'back-end'
displayName: 'Export static files'
- script: 'flake8 .'
workingDirectory: 'back-end'
displayName: 'Style analysis'
- script: 'pytest --junitxml=junit.xml --cov --cov-report=xml --cov-report=html'
workingDirectory: 'back-end'
displayName: 'Run tests'
- task: PublishTestResults#2
displayName: 'Publish test results'
inputs:
testResultsFiles: 'back-end/junit.xml'
- task: PublishCodeCoverageResults#1
displayName: 'Publish test coverage'
inputs:
codeCoverageTool: Coverage
summaryFileLocation: '$(System.DefaultWorkingDirectory)/back-end/coverage.xml'
reportDirectory: '$(System.DefaultWorkingDirectory)/back-end/htmlcov/'
- task: PublishBuildArtifacts#1
displayName: 'publish artifact'
inputs:
PathtoPublish: 'back-end'
ArtifactName: BackendArtifact
Release (key and names are hidden with *)
steps:
- task: boostingmy.vsts-heroku-tasks.pushu-to-heroku.PushToHeroku#1
displayName: 'Publish on Heroku'
inputs:
ApiKey: '***'
AppName: '***'
PushRoot: '$(System.DefaultWorkingDirectory)/****-Back-end-CI/BackendArtifact'