[CI] Add 2024 versions to tests (#660)

* Enable mysql_native_password for MySQL 8.2+
* Fix connection to MySQL 8 since Ubuntu 20.04 update
* Cut mysqlclient form the documentation
* Cut tests for Python 3.12 not supported by ansible-test
* Upgrade integration controller to ubuntu2204 by removing python

ansible-test uses python 3.10 if we specify ubuntu2204. Thus we lose the
ability to chose specific version of python to test. But integrations
tests are optional for a collection. And we don't catch a issue with
Python that often (ever ? I don't recall seen one).

This allow us to test MySQL 8.4, so it's a win.

* Cut tests for EoL MariaDB 10.4
* Reduce number of test in the matrix
* Cut support for intermediate LTS
* Fix python command not found with ansible-devel and add the debug

This is puzzling me. Why when using ansible devel the python command
changes? I know ansible-test install python after starting ubuntu22.04
so the way python is install must changes.

* Disable retry-on-error

When reading log we tend to look at the bottom, but doing so we find
often a idempotent error that are nothing to do with the first error.
Disabling this can greatly speedup tests and makes logs more readable.

Plus, now GHA jumps automatically at the latest error message. So with
this modification, we will always jump to the latest real error message.

* Enhance jobs title readability

We can't expand the left column on GHA, so the shorter, the better.
Use Ⓐ instead of Ansible.
This commit is contained in:
Laurent Indermühle 2024-07-19 11:04:13 +02:00 committed by GitHub
commit c503dc5b6b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 201 additions and 180 deletions

View file

@ -19,7 +19,7 @@ For now, the makefile only supports Podman.
### Requirements
- python >= 3.8 and <= 3.10
- python >= 3.8
- make
- podman
- Minimum 15GB of free space on the device storing containers images and volumes. You can use this command to check: `podman system info --format='{{.Store.GraphRoot}}'|xargs findmnt --noheadings --nofsroot --output SOURCE --target|xargs df -h --output=size,used,avail,pcent,target`
@ -41,7 +41,8 @@ The Makefile accept the following options
- "3.8"
- "3.9"
- "3.10"
- Description: If `Python -V` shows an unsupported version, use this option and choose one of the version available on your system. Use `ls /usr/bin/python3*|grep -v config` to list them.
- "3.11" (for stable-2.15+)
- Description: If `Python -V` shows an unsupported version, use this option to select a compatible Python version available on your system. Use `ls /usr/bin/python3*|grep -v config` to list the available versions (You may have to install one). Unsupported versions are those that are too recent for the Ansible version you are using. In such cases, you will see an error message similar to: 'This version of ansible-test cannot be executed with Python version 3.12.3. Supported Python versions are: 3.9, 3.10, 3.11'.
- `ansible`
- Mandatory: true
@ -62,11 +63,10 @@ The Makefile accept the following options
- `db_engine_version`
- Mandatory: true
- Choices:
- "5.7.40" <- mysql
- "8.0.31" <- mysql
- "10.4.24" <- mariadb
- "10.5.18" <- mariadb
- "10.6.11" <- mariadb
- "8.0.38" <- mysql
- "8.4.1" <- mysql (NOT WORKING YET, ansible-test uses Ubuntu 20.04 which is too old to install mysql-community-client 8.4)
- "10.5.25" <- mariadb
- "10.11.8" <- mariadb
- Description: The tag of the container to use for the service containers that will host a primary database and two replicas. Do not use short version, like `mysql:8` (don't do that) because our tests expect a full version to filter tests precisely. For instance: `when: db_version is version ('8.0.22', '>')`. You can use any tag available on [hub.docker.com/_/mysql](https://hub.docker.com/_/mysql) and [hub.docker.com/_/mariadb](https://hub.docker.com/_/mariadb) but GitHub Action will only use the versions listed above.
- `connector_name`
@ -79,22 +79,12 @@ The Makefile accept the following options
- `connector_version`
- Mandatory: true
- Choices:
- "0.7.11" <- pymysql (Only for MySQL 5.7)
- "0.9.3" <- pymysql
- "0.10.1" <- pymysql
- "1.0.2" <- pymysql
- "2.0.1" <- mysqlclient
- "2.0.3" <- mysqlclient
- "2.1.1" <- mysqlclient
- "1.1.1" <- pymysql
- Description: The version of the python package of the connector to use. This value is used to filter tests meant for other connectors.
- `python`
- Mandatory: true
- Choices:
- "3.8"
- "3.9"
- "3.10"
- Description: The python version to use in the controller (ansible-test container).
- `target`
- Mandatory: false
- Choices:
@ -114,30 +104,30 @@ tests will overwrite the 3 databases containers so no need to kill them in advan
- `continue_on_errors`
- Mandatory: false
- Description: Tells ansible-test to retry on errors and also continue on errors. This is the way the GitHub Action's workflow runs the tests. This can be used to catch all errors in a single run, but you'll need to scroll up to find them. Add any value to activate this option: `continue_on_errors=1`
- Description: Tells ansible-test to continue on errors. This is the way the GitHub Action's workflow runs the tests. This can be used to catch all errors in a single run, but you'll need to scroll up to find them. Add any value to activate this option: `continue_on_errors=1`
#### Makefile usage examples:
```sh
# Run all targets
make ansible="stable-2.12" db_engine_name="mysql" db_engine_version="5.7.40" python="3.8" connector_name="pymysql" connector_version="0.7.11"
make ansible="stable-2.16" db_engine_name="mysql" db_engine_version="8.0.31" connector_name="pymysql" connector_version="1.0.2"
# A single target
make ansible="stable-2.14" db_engine_name="mysql" db_engine_version="5.7.40" python="3.8" connector_name="pymysql" connector_version="0.7.11" target="test_mysql_info"
make ansible="stable-2.16" db_engine_name="mysql" db_engine_version="8.0.31" connector_name="pymysql" connector_version="1.0.2" target="test_mysql_info"
# Keep databases and ansible tests containers alives
# A single target and continue on errors
make ansible="stable-2.14" db_engine_name="mysql" db_engine_version="8.0.31" python="3.9" connector_name="mysqlclient" connector_version="2.0.3" target="test_mysql_query" keep_containers_alive=1 continue_on_errors=1
make ansible="stable-2.17" db_engine_name="mysql" db_engine_version="8.0.31" connector_name="mysqlclient" connector_version="2.0.3" target="test_mysql_query" keep_containers_alive=1 continue_on_errors=1
# If your system has an usupported version of Python:
make local_python_version="3.8" ansible="stable-2.14" db_engine_name="mariadb" db_engine_version="10.6.11" python="3.9" connector_name="pymysql" connector_version="0.9.3"
make local_python_version="3.10" ansible="stable-2.17" db_engine_name="mariadb" db_engine_version="10.6.11" connector_name="pymysql" connector_version="1.0.2"
```
### Run all tests
GitHub Action offer a test matrix that run every combination of Python, MySQL, MariaDB and Connector against each other. To reproduce this, this repo provides a script called *run_all_tests.py*.
GitHub Action offer a test matrix that run every combination of MySQL, MariaDB and Connector against each other. To reproduce this, this repo provides a script called *run_all_tests.py*.
Examples:
@ -146,8 +136,8 @@ python run_all_tests.py
```
### Add a new Python, Connector or Database version
### Add a new Connector or Database version
New components version should be added to this file: [.github/workflows/ansible-test-plugins.yml](https://github.com/ansible-collections/community.mysql/tree/main/.github/workflows)
Be careful to not add too much tests. When adding a new version of Python, for instance, only test it agains the latest versions of Ansible and MySQL/MariaDB. When tests are run, you can see that we already start 40 virtual machines!
Be careful to not add too much tests. The matrix creates an exponential number of virtual machines!