Draft: Cut necessity to specify docker image

This commit is contained in:
Laurent Indermuehle 2023-03-22 17:59:58 +01:00
commit 5145b3868e
No known key found for this signature in database
GPG key ID: 93FA944C9F34DD09
41 changed files with 504 additions and 476 deletions

View file

@ -21,18 +21,17 @@ For now, the makefile only supports Podman.
- python >= 3.8 and <= 3.10
- make
- podman
- Minimum 15GB of free space on the device storing containers images and volumes. You can use this command to check: `podman system info --format='{{.Store.GraphRoot}}'|xargs findmnt --noheadings --nofsroot --output SOURCE --target|xargs df -h --output=size,used,avail,pcent,target`
- Minimum 2GB of RAM
### Custom ansible-test containers
Our integrations tests use custom containers for ansible-test. Those images definition file are in [https://github.com/community.mysql/test-containers](https://github.com/ansible-collections/community.mysql/tree/main/test-containers) and then pushed to ghcr.io under the ansible-collection namespace: E.G.:
Our integrations tests use custom containers for ansible-test. Those images have their definition file stored in the directory [test-containers](test-containers/). We build and publish the images on ghcr.io under the ansible-collection namespace: E.G.:
`ghcr.io/ansible-collections/community.mysql/test-container-mariadb106-py310-mysqlclient211:latest`.
Look in the link above for a complete list of available containers.
You can also look into `[.github/workflows/ansible-test-plugins.yml](https://github.com/ansible-collections/community.mysql/tree/main/.github/workflows)` to see how those containers are built.
Availables images are listed [here](https://github.com/orgs/ansible-collections/packages).
### Makefile options
@ -45,7 +44,7 @@ The Makefile accept the following options
- "3.8"
- "3.9"
- "3.10"
- Description: This option can be omitted if your system has a version supported by Ansible. You can check with `python -V` and `ls /bin/python*`.
- Description: If `Python -V` shows an unsupported version, use this option and choose one of the version available on your system. Use `ls /usr/bin/python3*|grep -v config` to list them.
- `ansible`
- Mandatory: true
@ -76,26 +75,40 @@ The Makefile accept the following options
- "ghcr.io/ansible-collections/community.mysql/test-container-mariadb106-py310-pymysql102:latest"
- Description: The container image ansible-test will use. You must provide the right container_image that matches the specified `db_engine_version`, `python`, etc...
- `db_engine`
- Mandatory: true
- Choices:
- "mysql"
- "mariadb"
- Description: The name of the database engine to use for the service containers that will host a primary database and two replicas.
- `db_engine_version`
- Mandatory: true
- Choices:
- "mysql:5.7.40"
- "mysql:8.0.31"
- "mariadb:10.4.24"
- "mariadb:10.5.18"
- "mariadb:10.6.11"
- Description: The name of the container to use for the service containers that will host a primary database and two replicas. Either MYSQL or MariaDB. Use ':' as a separator. Do not use short version, like `mysql:8` (don't do that) because our tests expect a full version to filter tests precisely. For instance: `when: db_version is version ('8.0.22', '>')`. You can use any tag available on [hub.docker.com/_/mysql](https://hub.docker.com/_/mysql) and [hub.docker.com/_/mariadb](https://hub.docker.com/_/mariadb) but GitHub Action will only use the versions listed above.
- "5.7.40" <- mysql
- "8.0.31" <- mysql
- "10.4.24" <- mariadb
- "10.5.18" <- mariadb
- "10.6.11" <- mariadb
- Description: The tag of the container to use for the service containers that will host a primary database and two replicas. Do not use short version, like `mysql:8` (don't do that) because our tests expect a full version to filter tests precisely. For instance: `when: db_version is version ('8.0.22', '>')`. You can use any tag available on [hub.docker.com/_/mysql](https://hub.docker.com/_/mysql) and [hub.docker.com/_/mariadb](https://hub.docker.com/_/mariadb) but GitHub Action will only use the versions listed above.
- `connector`
- `connector_name`
- Mandatory: true
- Choices:
- "pymysql==0.7.11" <- Only for MySQL 5.7
- "pymysql==0.9.3"
- "pymysql==1.0.2" <- Not working, need fix
- "mysqlclient==2.0.1"
- "mysqlclient==2.0.3"
- "mysqlclient==2.1.1"
- Description: The name of the python package of the connector along with its version number. Use '==' as a separator.
- "pymysql
- "mysqlclient"
- Description: The python package of the connector to use. This value is used to filter tests meant for other connectors.
- `connector_version`
- Mandatory: true
- Choices:
- "0.7.11" <- Only for MySQL 5.7
- "0.9.3"
- "1.0.2" <- Not working, need fix
- "2.0.1"
- "2.0.3"
- "2.1.1"
- Description: The version of the python package of the connector to use. This value is used to filter tests meant for other connectors.
- `python`
- Mandatory: true
@ -131,17 +144,17 @@ test will recreate those containers so no need to kill it. Add any value to acti
```sh
# Run all targets
make ansible="stable-2.12" db_engine_version="mysql:5.7.40" python="3.8" connector="pymysql==0.7.11" docker_image="ghcr.io/ansible-collections/community.mysql/test-container-my57-py38-pymysql0711:latest"
make ansible="stable-2.12" db_engine_name="mysql" db_engine_version="5.7.40" python="3.8" connector_name="pymysql" connector_version="0.7.11"
# A single target
make ansible="stable-2.14" db_engine_version="mysql:5.7.40" python="3.8" connector="pymysql==0.7.11" docker_image="ghcr.io/ansible-collections/community.mysql/test-container-my57-py38-pymysql0711:latest" target="test_mysql_db"
make ansible="stable-2.14" db_engine_name="mysql" db_engine_version="5.7.40" python="3.8" connector_name="pymysql" connector_version="0.7.11"
# Keep databases and ansible tests containers alives
# A single target and continue on errors
make ansible="stable-2.14" db_engine_version="mysql:8.0.31" python="3.9" connector="mysqlclient==2.0.3" docker_image="ghcr.io/ansible-collections/community.mysql/test-container-my80-py39-mysqlclient203:latest" target="test_mysql_db" keep_containers_alive=1 continue_on_errors=1
make ansible="stable-2.14" db_engine_name="mysql" db_engine_version="8.0.31" python="3.9" connector_name="mysqlclient" connector_version="2.0.3"
# If your system has an usupported version of Python:
make local_python_version="3.8" ansible="stable-2.14" db_engine_version="mariadb:10.6.11" python="3.9" connector="pymysql==0.9.3" docker_image="ghcr.io/ansible-collections/community.mysql/test-container-mariadb103-py39-pymysql093:latest"
make local_python_version="3.8" ansible="stable-2.14" db_engine_name="mariadb" db_engine_version="10.6.11" python="3.9" connector_name="pymysql" connector_version="0.9.3"
```
@ -158,6 +171,8 @@ python run_all_tests.py
### Add a new Python, Connector or Database version
You can look into `[.github/workflows/ansible-test-plugins.yml](https://github.com/ansible-collections/community.mysql/tree/main/.github/workflows)` to see how those containers are built using [build-docker-image.yml](https://github.com/ansible-collections/community.mysql/blob/main/.github/workflows/build-docker-image.yml) and all [docker-image-xxx.yml](https://github.com/ansible-collections/community.mysql/blob/main/.github/workflows/docker-image-mariadb103-py38-mysqlclient201.yml) files.
1. Add a workflow in [.github/workflows/](.github/workflows)
1. Add a new folder in [test-containers](test-containers) containing a new Dockerfile. Your container must contains 3 things:
- Python