* Fix issues and documentation with integration tests after merge of #490. (#514)
* Fix explanation about containers images
* Add definitive URI to the containers images
* Document that new images must be set as public
* Document makefile options possible values
* Document that any mysql and mariadb tag can be used
* Add computation of docker_image path
* Refactor pre-command to separate commands for cleaner GHA output
* Refactor to use GHA test matrix
* Cut docker_image from documentation since it's now automatic
* Document how to use run_all_test.py to display the test matrix
(cherry picked from commit e2aa655762)
* Add changelog fragment
* Cut tests for MariaDB 10.5 and 10.6 that never worked with stable-1
* Integrations tests : Use containers for more control and verify that versions match expectation (#490)
* Add healthcheck to MariaDB before starting the tests
This prevent the first test to fail because the db isn't ready yet.
* Add % instead of the default 'localhost' since we use remote connection
Previously, everything was on localhost. Now ansible-test is in a
venv and the db is in a container. The db see the IP address from the
podman host (10.88.0.2)
* Add mysql_client to the controller
* Prepare controller with Podman/Docker Network
We use the Podman/Docker network gateway address to communicate between
container. I haven't tested Docker. I would have preferred to use a pod
but only Podman support it and ansible-test only support the
--docker-network option.
* Generalize mysql and mariadb version based on container name
This way we can split db_engine and db_version and simplify tests.
Also this is mandatory to use the matrix.db_engine_version as the
image name for our services containers.
* Fix replication due to usage of gateway_addr instead of localhost
* Refactor setup_mysql into setup_controller
* Fix server_id in GHA
GHA lack a way to pass option to docker's command. Also server_id is
not read as a environment variable. So I'm forced to use a config file.
* Refactor test_mysql_user to work with other host than localhost
* Refactor way tests info are passed from sed to file with lookup
The idea is to avoid modifying test targets from the workflow to prevent
ansible-test to think every tests needs to be run.
* Refactor test to use the db_version from setup_controller
* Add temporary files to .gitignore
* Add back docker healthcheck on services
I saw in the GHA logs that it perform an healtcheck ! So I hope this
will work.
* Refactor the way server_id is set for replicas
The simple way is to add '--server-id 2' after the name of the image of
the container. But GHA doesn't let us do that. The idea of mount a file
from our repo doesn't work because the repo is check out later in the
workflow and I failed to find a pre-job hook. Then I realized that this
MySQL option is dynamic! So we will set that in the test target!
* Add IF NOT EXISTS to prevent misleading error on retry
* Cut python 3.11 not supported by ansible-test yet
* Add option to run only a single target
* Disable replication with channel tests entirely for MySQL 5.7
* Activate Mysql 8 and Mariadb into GitHub Action Workflow
* Document run_all_tests.py
* Fix replication settings
sh don't know 'echo -e', so we use bash instead.
Also, we need to wait for the container to be healthy before trying to
restart it. Otherwise that could corrupt it.
* Add more descriptive tests names
* Use mysql_host var name instead of gateway_addr in tests
* Refactor user@<gateway ip addr> into user@%
* Workaround for plugin role that fails with any MariaDB versions
* Ensure replicas are healthy before rebooting them
* Add a virtualenv for ansible-test used locally
* Cut column-statistics disabling
Thanks to our test-container, we now use the correspond mysql-client.
So to test mysql 5.7 we use mysql-client-5.7 and to test mysql 8 we
use mysql-client-8.
* Fix test matrix
Python version should be quoted, otherwise 3.10 become 3.1
We can skip 2.14 and devel with Python3.8
We can skip devel with Python 3.9
We can skip MariaDB 10.4 with mysql-client-10.6
Add tests for MariaDB 10.6, 10.7 and 10.8
* Fix queries for roles
* Add filter for issues resolved in newer version of mysqlclient
* Add names to tests
* Cut tests for incompatible MySQL 8 and pymysql 0.7.11
* Fix assertion for older mysqlclient than 2.0.1 with mysql (mariadb ok)
* Change docker-image workflow to work on all images using matrix
* Add support for version of mysqlclient
* Fix verify database version
Sometimes, version_full contains trailing information (-log). To prevent
issues it's best to concatenate major and minor version.
* Cut filter for tests now that the right connector is used
* Add clean up in "always" phase of the block
Because our tests use --retry-on-error, and the first thing the test
does is to try to create the database. We must cleanup otherwise if
there is a retry, it will throw a misleading "database already exists"
error.
* Disable tests using pymysql 1.0.2
Many tests are failing but this must be fixed in the plugins in a future
PR.
* Cut test MySQL 8 with incompatible pymysql 0.7.11
It fails to connect with error about cryptography unsupported
* Fix missing cffi package to connect to MySQL 8 using Python 3.9
* Split Docker image workflow to rebuild only changed Dockerfile
My goal is not to save the planet but to make it work. Currently
docker/setup-buildx-action@v2 often fails. You have to rerun the
workflow multiple times until it succeed. When you do that with the
matrix with 15 containers, you never get to the point where they all
built successfully. Having separate workflows makes rerun the failing
build easier.
* Add option to let containers alive at end of testing
* Migrate tests documentations in their own file
* Document usage of continue_on_errors
* Add support for systems with unsupported python set as default
* Add create podman network for system missing it.
We saw that on a Fedora 33 with Podman 3.3.1, an old system. I didn't
find in which release the default network changed and maybe it's
defined in the Linux distribution. So in doubt I always attempt to
create the network.
* Add full path to image to prevent podman asking which registry to use
* Add options to enforce recreate containers even if already exists
* Add deletion of anonymous volumes associated with the container
* Change shebang from python to python3 to avoid confusion with python2
This script is a python3 script.
* Add disk and RAM requirements
* Cut the 3 from python command to follow shebang recommendations
https://docs.ansible.com/ansible-core/devel/dev_guide/testing/sanity/shebang.html
* Reformat file path
Co-authored-by: Jorge Rodriguez (A.K.A. Tiriel) <jorge.rodriguez@futurice.com>
* Move utility task files in their own folder
* Add called workflow file in the GHA hooks
Without this, the containers are not rebuilt when you modify the file
built-docker-image.yml.
* Rollback to github.repository in container image name
This time I think I understood. We publish in the
github.repository_owner's namespace. In my case it's laurent-indermuehle
and in case of upstream it's ansible-collections. A proof of that:
https://github.com/orgs/ansible-collections/packages <- here there is
one attempt I did in february to push my branch to the upstream.
So, our tests containers will be visible to the whole community, not
just community.mysql.
---------
Co-authored-by: Jorge Rodriguez (A.K.A. Tiriel) <jorge.rodriguez@futurice.com>
(cherry picked from commit 6970aef8f6)
* Add changelog fragment
* Disable tests that doesn't work on stable-1
It's shameful to disable tests, but they didn't makes much sense
anyway.
* Fix error message being different on stable-1 than v3
* Disable tests against MariaDB 10.5 and 10.6
Does version have never been tested on stable-1 and many tests fails.
* mysql_info - Add connector_name and connector_version to returned value (#497)
* Add methods to retrieve connector name and version
* Document that mysqlclient is also named MySQLdb
* Document version_added
* Add connector name and version in the returned block
* Cut condition to display any name that is return
In case of MySQLdb is renamed in mysqlclient. In that case, the
integration tests will catch this the day we update the connector
version.
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
(cherry picked from commit a5f3296d73)
* Cut fragment not relevant to the collection usage
* Updated version_added for stable-1
---------
Co-authored-by: Laurent Indermühle <laurent.indermuehle@pm.me>
* Fix mariadb test setup
* Update mysql src URL
* Add changelog fragment
* Update 491_fix_download_url.yaml
Sanity test failed because minor_changes in not an element of a list.
* Fix casing
Co-authored-by: Laurent Indermühle <laurent.indermuehle@pm.me>
(cherry picked from commit 00fa058a18)
Co-authored-by: Jorge Rodriguez (A.K.A. Tiriel) <jorge.rodriguez@futurice.com>
* Sync GHA workflow w/ the collection template (#452)
* Sync GHA workflow w/ the collection template
* Drop the trailing pre-cmd semicolon
* Recover missing `-e` flag of `sed`
* Use relative paths for version configs
* Unquote `env.connector_version_file`
* Use string formatting to fix the substitution problem
(cherry picked from commit 8107530744)
(cherry picked from commit 704a0cea6a)
* Backport mysql_version_parts variable assignation
(cherry picked from commit 79046a88cb)
(cherry picked from commit a59d95501e)
* Add changelog fragment
(cherry picked from commit 26ecd68801)
* Backport flags and variables to differentiate MariaDB from MySQL setup
(cherry picked from commit b4303511d5)
* Backport issue-28 check for tls support
(cherry picked from commit efde607d25)
* Backport tls_requirements simplified and deduplified tests
(cherry picked from commit ac5a339644)
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Is changed (#427)
* Refactor tests to use "is" and "is not" changed
* Refactor tests to use is succeeded or is failed
* Reformat indentation
* Add filter "bool" to prevent issues
(cherry picked from commit 0a68bb270f)
* Fix error message verification
I don't know why this works on main, but in stable-1, the error message
is "invalid privileges string: Invalid privileges specified:
frozenset({'INVALID'})"
* Add filter for test that won't work with mariadb
* Refactor test in their own file instead of main
* Add db_names dict
* Fix registered variable name
* Add missing fact
* Add test databases cleanup
* Cut tests for unsupported db name (%)
* Add missing default vars to tests db_formats
* Backport small diff from main
* Add changes from stable-1 integrations tests (PR 418)
* Refactor to use connectors' info declared in setup_mysql
* Fix 2nd replication stop marked changed by mysqlclient
(cherry picked from commit 61586ae4cc)
* :Fix ci python requirements (#416)
* Add matrix for python and ansible-core versions for sanity tests
* Add python 3.9 to integrations tests
* Add python 3.9 to unit tests
* Reformat sort by python version first
(cherry picked from commit 97318559e5)
* Fix string to int comparison error 1292
* Fix assert expected changed value
* Cut assertion that is incorrect
With both connectors, and only in stable-1 apparently, the test is
always failed.
* Fix bool comparison
* Revert separated tests for both connectors
* Refactor test using connector.name variable
* Refactor filtering of tasks using connector's vars sets during setup
* Fix "command not found" and "database doesn't exists"
* Fix assertion by not running on failing connectors
* Fix missing package when using sha256_password with MySQL 8
* mysql_query: fix false change reports when IF NOT EXISTS clause is used (#322)
* mysql_query: fix false change reports when IF NOT EXISTS clause is used
* Fix
* Fix doc, add fragment
* Improve doc
(cherry picked from commit 647461010d)
* Fix tests
* Clarified error message about missing python modules (#279)
* Clarified error message about missing python modules, and tweak documentation to suggest overriding interpreter.
* Mention mysqlclient as another option
* Correct mysqlclient suggestions from python2 to python3
Co-authored-by: Matthew Exon <git.mexon@spamgourmet.com>
(cherry picked from commit 82baf7508c)
* Copy ignore-2.13.txt to ignore-2.14.txt (#323)
* Copy ignore-2.13.txt to ignore-2.14.txt
* Fix sanity
(cherry picked from commit c16b2428e8)
Co-authored-by: Matthew Exon <github2.mexon@neverbox.com>
* Prepare for distutils.version being removed in Python 3.12 (#267)
* Prepare for distutils.version being removed in Python 3.12
* Update plugins/module_utils/version.py
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit d9e12b85ad)
* Use vendored version of distutils.version (#269)
* Use vendored version of distutils.version
* Correct fragment
* Update plugins/module_utils/version.py
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit 9c575b4762)
* Add ignore-2.13.txt
* mysql_info: fix TypeError failure when there are databases that do not contain tables (#205)
* mysql_info: fix TypeError failure when there are databases that do not contain tables
* Add changelog fragment
(cherry picked from commit a1f419d541)
* Fix sanity errors
* mysql_query: correctly reflect changed status in replace statements.
* Fix the wrong indent.
(cherry picked from commit 9055bb4c8c)
Co-authored-by: Tong He <68936428+unnecessary-username@users.noreply.github.com>
* add mysql_full_version and suffix return variable
add changelog fragment
* rephrase changelog fragment
* Update plugins/modules/mysql_info.py
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
* Add changes as per PR review
* Add tests for new suffix output parameter
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
Co-authored-by: Jorge-Rodriguez <jorge.rodriguez@futurice.com>
* mysql_user: add proper handling of INSERT, UPDATE, REFERENCES on columns
* Add changelog fragment
* fix sanity
* fix CI
* fix sanity
* fix CI
* make the assertion fairer
* Improve
* mysql_user: fix the module is not idempotent when there is SELECT on columns granted
* add changelog fragment
* fix
* Add unit tests for has_select_on_col function
* Add unit tests for sort_column_order function
* Add unit tests for handle_select_on_col function
* Update a comment
* mysql_replication: deprecation of slave related options, adding alternatives
* Add changelog fragment
* Integration tests getslave -> getreplica
* Change the rest of offending choices/comments
* Add announcement about replacing SLAVE to REPLICA in messages
* Deprecate offending values
* mysql_user: fixed encrypted option for MySQL 8.0 and test coverage
The purpose of this change was originally to expand test coverage to
unblock #76, but an issue was detected with the encrypted parameter on
MySQL 8.0 in the process of writing the tests. Additionally,
user_password_update_test.yml had been disabled at some point, so I
opted to replace it with two new files that will focus on the password
and plugin auth paths.
* Updated tests to cover a couple of missing branches
* Skip tests that rely on sha256_password if pymysql < 0.9
* Cover the case where pymysql isn't installed for plugin tests
* Added better plugin auth checking to tests and other minor changes
* Fixed version detection to explicitly handle MariaDB
* Removed unneeded import from previous change
* Remove whitespace that was introduced by change that was removed
* Added unit tests for missing coverage
* mysql_replication: fix crashes caused by deprecated terminology
* Fix unrelated sanity errors
* Tests: mysql 8.0.21 -> 8.022
* Adjust integration tests
* Add version check to the tests
* Add debug statement
* Adjust mysql version
* Fix tests
* Add unit tests
* Add changelog fragment
* Improve code and coverage
* Get rid of extra blank line
* Improve coverage
* Change suggested
* mysql_user: Fixed change detection with append_privs (#69)
Prior to this change, mysql_user with append_privs would attempt to make
a change even if the current privileges were a superset of the new
privileges (shouldn't require any action).
* Fixed unrelated mysql_replication doc causing failures in CI
* Added fragments and check_mode tests
* Expanded priv append tests to cover additional case
* Add changelog fragment
* Add check_hostname option
* Propagate check_hostname option across the collection
* Update documentation fragment
* Propagate test to all other plugins
* Remove stray line
* Give test user privileges to run test operations
* Extend integration tests job matrix
* Add caution note to documentation fragment.
* Update matrix job name
* Rearrange job matrix
* Fix sanity issues
* Fix issue with mysqldb silently failing to update out of range variables
* Fix variable overwrite
* Ignore `check_hostname` when using MySQLdb
* Update plugins/doc_fragments/mysql.py
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
* Update plugins/doc_fragments/mysql.py
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
* Update changelogs/fragments/35-disable-hostname-check.yml
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
* Add test to verify TLS requirements are removed
* Fix cursor parsing
* small fixes
* Refactor TLS tests into their own file
* Fix TLS requirements parsing
* Add TLS connection parameters
* Add check mode tests
* Fix check mode indentation
* Run MySQL commands with the mysql_command variable
* Fix typo
* Restore code lost during cherry pick
* Fix conditionals to accomodate for MySQL v8
* Fix equal operators
* Remove Black formatting in an attempt to make codecov happy
* Remove deprecation notice
* Fix closing bracket
* Remove code duplication
* initial commit
* removed remaining references to community.general
* enabled integration pipeline
* switched from preconfigured replication topology to simple multinode install
* updated version from 1.0.0 to 0.1.0
