Pour tout problème contactez-nous par mail : support@froggit.fr | La FAQ :grey_question: | Rejoignez-nous sur le Chat :speech_balloon:

Skip to content
Snippets Groups Projects
Commit c88fe500 authored by Dorian Turba's avatar Dorian Turba
Browse files

add blocking and inputs, sort pypi releases

parent 5b46be3d
No related branches found
No related tags found
1 merge request!1Bump version: 0.4.1 → 0.4.2
Pipeline #61560 passed
......@@ -3,9 +3,7 @@ include:
- project: 'swepy/cicd-templates/pypi-to-docker-missing-version'
ref: $CI_COMMIT_SHA
file: 'templates/pypi-to-docker-missing-version.yml'
pypi-to-docker-missing-version:
stage: test
variables:
PACKAGE_NAME: ruff
IMAGE_NAME: swepy/ruff
inputs:
image_name: swepy/ruff
package_name: ruff
stage: test
......@@ -15,6 +15,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* Update CICD
### Changed
* Variable `REPOSITORIES_API_URL` renamed `CONTAINER_REGISTRY_API_URL`
### Added
* Support [inputs](https://docs.gitlab.com/ci/yaml/inputs/):
* blocking: When no new version to publish, job fails, otherwise consider last release.
* container_registry_api_url: URL of the API to retrieve the container registry.
* image_name: Image name on Container Registry. Should be "<org|user>/<repo>".
* package_index_api_url: URL of the API to retrieve the package index.
* package_name: Name of the package on Package Registry.
* stage: Stage of the job.
* tags: Image Tags to consider as already built. Used to exclude specific versions.
* Variable `BLOCKING`
## [0.4.1] - 2024-07-16
[![Pipeline](https://lab.frogg.it/swepy/cicd-templates/pypi-to-docker-missing-version/badges/0.4.1/pipeline.svg)](https://lab.frogg.it/swepy/cicd-templates/pypi-to-docker-missing-version/-/pipelines?ref=0.4.1)
......
......@@ -12,70 +12,38 @@ used for other jobs to build the image.
## How to use it
### Include the component/template
Add the following to your `.gitlab-ci.yml` file.
#### As a Local component
Recommended if the component is local to the instance:
```yaml
include:
- component: $CI_SERVER_FQDN/swepy/cicd-templates/pypi-to-docker-missing-version/pypi-to-docker-missing-version@0.4.2
- component: $CI_SERVER_FQDN/swepy/cicd-templates/pypi-to-docker-missing-version/pypi-to-docker-missing-version@0.4.2
```
If your GitLab instance version is lower than `16.10`, use `$CI_SERVER_HOST` instead
of `$CI_SERVER_FQDN`. Don't forget to add the port if necessary.
#### As a Remote template
### Inputs
Recommended if the template is not local to the instance:
You can customize the job by overriding specific inputs.
##### Source: lab.frogg.it
```yaml
include:
- remote: 'https://lab.frogg.it/swepy/cicd-templates/pypi-to-docker-missing-version/-/raw/0.4.2/templates/pypi-to-docker-missing-version.yml'
```
##### Mirror: GitLab.com
```yaml
include:
- remote: 'https://gitlab.com/swepy/cicd-templates/pypi-to-docker-missing-version/-/raw/0.4.2/templates/pypi-to-docker-missing-version.yml'
```
#### As a Local template
If the template is local to the instance:
```yaml
include:
- project: 'swepy/cicd-templates/pypi-to-docker-missing-version'
ref: '0.4.2'
file: 'templates/pypi-to-docker-missing-version.yml'
```
### Customize job
You can customize the job by overriding specific keys. For example:
```yaml
pypi_to_docker_missing_version:
variables:
SORT_CMD: grep -E "^[0-9]+(\.[0-9]+)*$" | sort -t. -k1,1n -k2,2n -k3,3n
```
| Name | Description | Default |
|------------------------------|----------------------------------------------------------------------------------|---------------------------------------------------|
| `blocking` | When no new version to publish, job fails, otherwise consider last release. | `false` |
| `container_registry_api_url` | URL of the API to retrieve the container registry. | `https://registry.hub.docker.com/v2/repositories` |
| `image_name` | **Mandatory** Image name on Container Registry. Should be "<org\|user>/\<repo>". | `""` |
| `package_index_api_url` | URL of the API to retrieve the package index. | `https://pypi.org/pypi` |
| `package_name` | Name of the package on Package Registry. | `$CI_PROJECT_NAME` |
| `sort_cmd` | The command to sort the versions. | `sort -V` |
| `stage` | Stage of the job. | `build` |
| `tags` | Image tags to consider as already built. Used to exclude specific versions. | `""` |
## Variables
You can customize the job by overriding the following variables:
| Name | Description | Default |
|-------------------------|----------------------------------------------------------------------------------------|---------------------------------------------------|
| `IMAGE_NAME` | **Mandatory** The name of the image. Should be "<org\|user>/\<repo>". | `""` |
| `PACKAGE_INDEX_API_URL` | The URL of the API to retrieve the package index. | `https://pypi.org/pypi` |
| `PACKAGE_NAME` | The name of the package. | `"$CI_PROJECT_NAME"` |
| `REPOSITORIES_API_URL` | The URL of the API to retrieve the repositories. | `https://registry.hub.docker.com/v2/repositories` |
| `SORT_CMD` | The command to sort the versions. | `sort -V` |
| `TAGS` | The tags of the image to consider as already built. Used to exclude specific versions. | `""` |
You can further customize the job by overriding the following variables:
| Name | Description | Default |
|------------------------------|----------------------------------------------------------------------------------|---------------------------------------------------|
| `BLOCKING` | When no new version to publish, job fails, otherwise consider last release. | `false` |
| `CONTAINER_REGISTRY_API_URL` | URL of the API to retrieve the container registry. | `https://registry.hub.docker.com/v2/repositories` |
| `IMAGE_NAME` | **Mandatory** Image name on Container Registry. Should be "<org\|user>/\<repo>". | `$[[ inputs.image_name ]]` |
| `PACKAGE_INDEX_API_URL` | URL of the API to retrieve the package index. | `$[[ inputs.package_index_api_url ]]` |
| `PACKAGE_NAME` | Name of the package on Package Registry. | `$[[ inputs.package_name ]]` |
| `SORT_CMD` | The command to sort the versions. | `sort -V` |
| `TAGS` | Image tags to consider as already built. Used to exclude specific versions. | `""` |
spec:
inputs:
blocking:
description: When no new version to publish, job fails, otherwise consider last release.
type: boolean
default: false
container_registry_api_url:
description: URL of the API to retrieve the container registry.
default: https://registry.hub.docker.com/v2/repositories
image_name:
description: Image name on Container Registry. Should be "<org|user>/<repo>".
package_index_api_url:
description: URL of the API to retrieve the package index.
default: https://pypi.org/pypi
package_name:
description: Name of the package on Package Registry.
default: $CI_PROJECT_NAME
stage:
description: Stage of the job.
default: build
tags:
description: Image Tags to consider as already built. Used to exclude specific versions.
default: ""
---
pypi-to-docker-missing-version:
stage: build
image: alpine:3.20.1
variables:
IMAGE_NAME: ""
PACKAGE_INDEX_API_URL: https://pypi.org/pypi
PACKAGE_NAME: $CI_PROJECT_NAME
REPOSITORIES_API_URL: https://registry.hub.docker.com/v2/repositories
SORT_CMD: sort -V
TAGS: ""
script:
#!/bin/bash
- apk update
- apk add --no-cache curl jq
stage: $[[ inputs.stage ]]
image: alpine:3.21.3
variables:
BLOCKING: $[[ inputs.blocking ]]
CONTAINER_REGISTRY_API_URL: $[[ inputs.container_registry_api_url ]]
IMAGE_NAME: $[[ inputs.image_name ]]
PACKAGE_INDEX_API_URL: $[[ inputs.package_index_api_url ]]
PACKAGE_NAME: $[[ inputs.package_name ]]
SORT_CMD: sort -V
TAGS: $[[ inputs.tags ]]
script:
#!/bin/bash
- apk update
- apk add --no-cache curl jq
# get docker tags
- echo "Image name=$IMAGE_NAME"
- docker_url="$REPOSITORIES_API_URL/$IMAGE_NAME/tags"
# get docker tags
- echo "Image name=$IMAGE_NAME"
- docker_url="$CONTAINER_REGISTRY_API_URL/$IMAGE_NAME/tags"
# get all tags
- while [ -n "$docker_url" ]; do
- docker_response=$(curl -s "$docker_url")
- if [ "$(echo "$docker_response" | jq -r '.results | length')" -eq 0 ]; then
- break
- fi
- TAGS="$TAGS $(echo "$docker_response" | jq -r '.results[].name')"
- docker_url=$(echo "$docker_response" | jq -r '.next')
- if [ "$docker_url" = "null" ]; then
- docker_url=""
- fi
- done
- echo "tags on docker=$TAGS"
# get all tags
- while [ -n "$docker_url" ]; do
- docker_response=$(curl -s "$docker_url")
- if [ "$(echo "$docker_response" | jq -r '.results | length')" -eq 0 ]; then
- break
- fi
- TAGS="$TAGS $(echo "$docker_response" | jq -r '.results[].name')"
- docker_url=$(echo "$docker_response" | jq -r '.next')
- if [ "$docker_url" = "null" ]; then
- docker_url=""
- fi
- done
- echo "tags on docker=$TAGS"
# pypi
- pypi_url="$PACKAGE_INDEX_API_URL/$PACKAGE_NAME/json"
- pypi_response=$(curl -s "$pypi_url")
- releases=$(echo "$pypi_response" | jq -r '.releases | keys[]')
# pypi
- pypi_url="$PACKAGE_INDEX_API_URL/$PACKAGE_NAME/json"
- pypi_response=$(curl -s "$pypi_url")
- pypi_releases=$(echo "$pypi_response" | jq -r '.releases | keys[]')
- pypi_releases=$(printf "%s\n" $pypi_releases | $SORT_CMD)
- echo "releases on pypi:"
- printf "%s " $releases
- echo "releases on pypi:"
- printf "%s " $pypi_releases
# find missing releases
- missing_releases=""
- eval 'for release in $releases; do found=0; for tag in $TAGS; do if [ "$release" = "$tag" ]; then found=1; break; fi; done; if [ $found -eq 0 ]; then missing_releases="$missing_releases $release"; fi; done'
- missing_releases=$(echo $missing_releases)
- echo "missing tags=$missing_releases"
# find missing releases
- missing_releases=""
- eval 'for release in $pypi_releases; do found=0; for tag in $TAGS; do if [ "$release" = "$tag" ]; then found=1; break; fi; done; if [ $found -eq 0 ]; then missing_releases="$missing_releases $release"; fi; done'
- missing_releases=$(echo $missing_releases)
- echo "missing tags=$missing_releases"
- sorted_releases=$(printf "%s\n" $missing_releases | $SORT_CMD)
- sorted_releases=$(printf "%s\n" $missing_releases | $SORT_CMD)
# print the sorted versions
- echo "Sorted releases:"
- printf "%s " $sorted_releases
# print the sorted versions
- echo "Sorted releases:"
- printf "%s " $sorted_releases
- version_to_publish=$(echo $sorted_releases | awk '{print $1}')
- 'echo "Version to publish: $version_to_publish"'
- version_to_publish=$(echo $sorted_releases | awk '{print $1}')
- 'echo "Version to publish: $version_to_publish"'
- if [ -n "$version_to_publish" ]; then
- echo "Version to publish=$version_to_publish"
- else
- version_to_publish=$(echo $releases | awk '{print $NF}')
- echo "Version to publish=$version_to_publish"
- fi
- if [ -n "$version_to_publish" ]; then
- echo "Version to publish=$version_to_publish"
- elif [ "$BLOCKING" = true ]; then
- printf "Nothing to publish, stop here\n"
- exit 1
- else
- printf "No new publication, publish last"
- version_to_publish=$(echo $pypi_releases | awk '{print $NF}')
- echo "Version to publish=$version_to_publish"
- fi
- echo "$version_to_publish" > version.txt
- cat version.txt
- echo "$version_to_publish" > version.txt
- cat version.txt
artifacts:
name: "version"
paths:
- version.txt
expire_in: 1 day
\ No newline at end of file
artifacts:
name: "version"
paths:
- version.txt
expire_in: 1 day
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment