community.mysql/TESTING.md

Tests

This collection uses GitHub Actions to run ansible-test to validate its content. Three type of tests are used: Sanity, Integration and Units.

The tests covers plugins and roles (no role available yet, but tests are ready) and can be found here:

• Plugins: .github/workflows/ansible-test-plugins.yml
• Roles: .github/workflows/ansible-test-roles.yml (unused yet)

Everytime you push on your fork or you create a pull request, both workflows runs. You can see the output on the "Actions" tab.

Integration tests

You can use GitHub to run ansible-test either on the community repo or your fork. But sometimes you want to quickly test a single version or a single target. To do that, you can use the Makefile present at the root of this repository.

For now, the makefile only supports Podman.

Requirements

• python >= 3.8 and <= 3.10
• make
• podman
• Minimum 15GB of free space on the device storing containers images and volumes. You can use this command to check: podman system info --format='{{.Store.GraphRoot}}'|xargs findmnt --noheadings --nofsroot --output SOURCE --target|xargs df -h --output=size,used,avail,pcent,target
• Minimum 2GB of RAM

ansible-test environment

Integration tests use the default container from ansible-test. Then required packages for the tests are installed from the setup_controller target located in the tests/integration/targets folder.

Makefile options

The Makefile accept the following options

• local_python_version

  • Mandatory: false
  • Choices:
    • "3.8"
    • "3.9"
    • "3.10"
  • Description: If Python -V shows an unsupported version, use this option and choose one of the version available on your system. Use ls /usr/bin/python3*|grep -v config to list them.

• ansible

  • Mandatory: true
  • Choices:
    • "stable-2.15"
    • "stable-2.16"
    • "stable-2.17"
    • "devel"
  • Description: Version of ansible to install in a venv to run ansible-test

• db_engine_name

  • Mandatory: true
  • Choices:
    • "mysql"
    • "mariadb"
  • Description: The name of the database engine to use for the service containers that will host a primary database and two replicas.

• db_engine_version

  • Mandatory: true
  • Choices:
    • "5.7.40" <- mysql
    • "8.0.31" <- mysql
    • "10.4.24" <- mariadb
    • "10.5.18" <- mariadb
    • "10.6.11" <- mariadb
  • Description: The tag of the container to use for the service containers that will host a primary database and two replicas. Do not use short version, like mysql:8 (don't do that) because our tests expect a full version to filter tests precisely. For instance: when: db_version is version ('8.0.22', '>'). You can use any tag available on hub.docker.com/_/mysql and hub.docker.com/_/mariadb but GitHub Action will only use the versions listed above.

• connector_name

  • Mandatory: true
  • Choices:
    • "pymysql"
    • "mysqlclient"
  • Description: The python package of the connector to use. In addition to selecting the test container, this value is also used for tests filtering: when: connector_name == 'pymysql'.

• connector_version

  • Mandatory: true
  • Choices:
    • "0.7.11" <- pymysql (Only for MySQL 5.7)
    • "0.9.3" <- pymysql
    • "1.0.2" <- pymysql
    • "2.0.1" <- mysqlclient
    • "2.0.3" <- mysqlclient
    • "2.1.1" <- mysqlclient
  • Description: The version of the python package of the connector to use. This value is used to filter tests meant for other connectors.

• python

  • Mandatory: true
  • Choices:
    • "3.8"
    • "3.9"
    • "3.10"
  • Description: The python version to use in the controller (ansible-test container).

• target

  • Mandatory: false
  • Choices:
    • "test_mysql_db"
    • "test_mysql_info"
    • "test_mysql_query"
    • "test_mysql_replication"
    • "test_mysql_role"
    • "test_mysql_user"
    • "test_mysql_variables"
  • Description: If omitted, all test targets will run. But you can limit the tests to a single target to speed up your tests.

• keep_containers_alive

  • Mandatory: false
  • Description: This option keeps all tree databases containers and the ansible-test container alive at the end of tests or in case of failure. This is useful to enter one of the containers with podman exec -it <container-name> bash for debugging. Rerunning the tests will overwrite the 3 databases containers so no need to kill them in advance. But nothing will kill the ansible-test container. You must do that using podman stop and podman rm. Add any value to activate this option: keep_containers_alive=1

• continue_on_errors

  • Mandatory: false
  • Description: Tells ansible-test to retry on errors and also continue on errors. This is the way the GitHub Action's workflow runs the tests. This can be used to catch all errors in a single run, but you'll need to scroll up to find them. Add any value to activate this option: continue_on_errors=1

Makefile usage examples:

# Run all targets
make ansible="stable-2.12" db_engine_name="mysql" db_engine_version="5.7.40" python="3.8" connector_name="pymysql" connector_version="0.7.11"

# A single target
make ansible="stable-2.14" db_engine_name="mysql" db_engine_version="5.7.40" python="3.8" connector_name="pymysql" connector_version="0.7.11" target="test_mysql_info"

# Keep databases and ansible tests containers alives
# A single target and continue on errors
make ansible="stable-2.14" db_engine_name="mysql" db_engine_version="8.0.31" python="3.9" connector_name="mysqlclient" connector_version="2.0.3" target="test_mysql_query" keep_containers_alive=1 continue_on_errors=1

# If your system has an usupported version of Python:
make local_python_version="3.8" ansible="stable-2.14" db_engine_name="mariadb" db_engine_version="10.6.11" python="3.9" connector_name="pymysql" connector_version="0.9.3"

Run all tests

GitHub Action offer a test matrix that run every combination of Python, MySQL, MariaDB and Connector against each other. To reproduce this, this repo provides a script called run_all_tests.py.

Examples:

python run_all_tests.py

Add a new Python, Connector or Database version

New components version should be added to this file: .github/workflows/ansible-test-plugins.yml

Be careful to not add too much tests. When adding a new version of Python, for instance, only test it agains the latest versions of Ansible and MySQL/MariaDB. When tests are run, you can see that we already start 40 virtual machines!