From c99c19a489d0c1db85457bc8b7ffbeccf82788dd Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Tue, 27 Feb 2024 10:27:19 +0100 Subject: [PATCH 01/54] README.md: update Communication guide (#617) --- README.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 40264d2..0e0704e 100644 --- a/README.md +++ b/README.md @@ -42,22 +42,25 @@ They also should be subscribed to Ansible's [The Bullhorn newsletter](https://do > The `GitHub Discussions` feature is disabled in this repository. Use the `mysql` tag on the forum in the [Project Discussions](https://forum.ansible.com/new-topic?title=topic%20title&body=topic%20body&category=project&tags=mysql) or [Get Help](https://forum.ansible.com/new-topic?title=topic%20title&body=topic%20body&category=help&tags=mysql) category instead. -We announce releases and important changes through Ansible's [The Bullhorn newsletter](https://eepurl.com/gZmiEP). Be sure you are subscribed. +### Asynchronous channels + +* Join the Ansible forum: + * [MySQL Team](https://forum.ansible.com/g/MySQLTeam): by joining the team you will automatically get subscribed to the posts tagged with [mysql](https://forum.ansible.com/tag/mysql). + * [Get Help](https://forum.ansible.com/c/help/6/none): get help or help others. + * [Posts tagged with 'mysql'](https://forum.ansible.com/tag/mysql): leverage tags to narrow the scope. + * [Social Spaces](https://forum.ansible.com/c/chat/4): gather and interact with fellow enthusiasts. + * [News & Announcements](https://forum.ansible.com/c/news/5/none): track project-wide announcements. + +* The Ansible's [Bullhorn newsletter](https://forum.ansible.com/t/about-the-newsletter-category/166): we use it to announce releases and important changes. + +### Real-time channels -Join [our team](https://forum.ansible.com/g/MySQLTeam) on: -* The Ansible forums: - * [News & Announcements](https://forum.ansible.com/c/news/5/none) - * [Get Help](https://forum.ansible.com/c/help/6/none) - * [Social Spaces](https://forum.ansible.com/c/chat/4) - * [Posts tagged 'mysql'](https://forum.ansible.com/tag/mysql) * Matrix: * `#mysql:ansible.com` [room](https://matrix.to/#/#mysql:ansible.com): questions on how to contribute and use this collection. * `#users:ansible.com` [room](https://matrix.to/#/#users:ansible.com): general use questions and support. * `#ansible-community:ansible.com` [room](https://matrix.to/#/#community:ansible.com): community and collection development questions. * other Matrix rooms; see the [Ansible Communication Guide](https://docs.ansible.com/ansible/devel/community/communication.html) for details. -We take part in the global quarterly [Ansible Contributor Summit](https://github.com/ansible/community/wiki/Contributor-Summit) virtually or in-person. Track [The Bullhorn newsletter](https://eepurl.com/gZmiEP) and join us. - For more information about communication, refer to the [Ansible Communication guide](https://docs.ansible.com/ansible/devel/community/communication.html). ## Governance From bfe2fdc3ff8b94b14574cdade1639ce11877215c Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Thu, 14 Mar 2024 07:19:39 +0100 Subject: [PATCH 02/54] mysql_user: fix ed25512 plugin handling (#619) --- changelogs/fragments/0-mysql_user.yml | 2 ++ plugins/module_utils/user.py | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) create mode 100644 changelogs/fragments/0-mysql_user.yml diff --git a/changelogs/fragments/0-mysql_user.yml b/changelogs/fragments/0-mysql_user.yml new file mode 100644 index 0000000..6b812ab --- /dev/null +++ b/changelogs/fragments/0-mysql_user.yml @@ -0,0 +1,2 @@ +bugfixes: +- mysql_user - add correct ``ed25519`` auth plugin handling (https://github.com/ansible-collections/community.mysql/issues/6). diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index 17ad4b0..f042c85 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -368,7 +368,7 @@ def user_mod(cursor, user, host, host_all, password, encrypted, query_with_args = "ALTER USER %s@%s IDENTIFIED WITH %s AS %s", (user, host, plugin, plugin_hash_string) elif plugin_auth_string: # Mysql and MariaDB differ in naming pam plugin and syntax to set it - if plugin == 'pam': + if plugin in ('pam', 'ed25519'): query_with_args = "ALTER USER %s@%s IDENTIFIED WITH %s USING %s", (user, host, plugin, plugin_auth_string) else: query_with_args = "ALTER USER %s@%s IDENTIFIED WITH %s BY %s", (user, host, plugin, plugin_auth_string) From f105fd9a95581ecf088837b861ae6eb5adcd30f9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Thu, 11 Apr 2024 10:46:43 +0200 Subject: [PATCH 03/54] Add tests for Ansible core 2.17 (devel is 2.18 today) and bump tests dependencies (#623) * Add tests for Ansible core 2.17 (devel is 2.18 today) * Drop tests for Ansible core 2.14 and add 2.17 * Cut duplicate exclude * Add back python 3.8 and 3.9 for stable2.15 * Bump action to prevent deprecation warnings * Cut python 3.9 for devel in roles tests * Attempt to fix GHA line folding * fix typo * Bump ubuntu Latest ansible-test doesn't work with old ubuntu. See here for more info: https://github.com/ansible-collections/collection_template/blob/main/.github/workflows/ansible-test.yml#L83-L91 * fix docker_image var assignation * fix yamllint false positive * Attempt to fix docker_image_multiline assignation * Fix empty var due to scope of each command * Attempt to fix docker_image assignation * fix error "vars should be dict" * Document URL of the repository for the action ansible-test-gh-action * Disable role tests * Document ansible-core version tested * Cut ansible-core 2.14 from testing documentation --- .github/workflows/ansible-test-plugins.yml | 48 +++++++------------ ...t-roles.yml => ansible-test-roles.yml.off} | 26 ++++++---- .github/workflows/build-docker-image.yml | 2 +- ...ker-image-mariadb-py310-mysqlclient211.yml | 2 +- .../docker-image-mariadb-py310-pymysql102.yml | 2 +- ...cker-image-mariadb-py38-mysqlclient201.yml | 2 +- .../docker-image-mariadb-py38-pymysql093.yml | 2 +- ...cker-image-mariadb-py39-mysqlclient203.yml | 2 +- .../docker-image-mariadb-py39-pymysql093.yml | 2 +- .../docker-image-my57-py38-mysqlclient201.yml | 2 +- .../docker-image-my57-py38-pymysql0711.yml | 2 +- .../docker-image-my57-py38-pymysql093.yml | 2 +- ...ocker-image-mysql-py310-mysqlclient211.yml | 2 +- .../docker-image-mysql-py310-pymysql102.yml | 2 +- ...docker-image-mysql-py38-mysqlclient201.yml | 2 +- .../docker-image-mysql-py38-pymysql093.yml | 2 +- ...docker-image-mysql-py39-mysqlclient203.yml | 2 +- .../docker-image-mysql-py39-pymysql093.yml | 2 +- README.md | 2 +- TESTING.md | 4 +- .../tasks/test_tls_requirements.yml | 10 ++-- 21 files changed, 59 insertions(+), 63 deletions(-) rename .github/workflows/{ansible-test-roles.yml => ansible-test-roles.yml.off} (77%) diff --git a/.github/workflows/ansible-test-plugins.yml b/.github/workflows/ansible-test-plugins.yml index 78644bb..77da49e 100644 --- a/.github/workflows/ansible-test-plugins.yml +++ b/.github/workflows/ansible-test-plugins.yml @@ -1,6 +1,6 @@ --- name: Plugins CI -on: +on: # yamllint disable-line rule:truthy push: paths: - 'plugins/**' @@ -18,15 +18,16 @@ on: jobs: sanity: name: "Sanity (Ansible: ${{ matrix.ansible }})" - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 strategy: matrix: ansible: - - stable-2.14 - stable-2.15 - stable-2.16 + - stable-2.17 - devel steps: + # https://github.com/ansible-community/ansible-test-gh-action - name: Perform sanity testing uses: ansible-community/ansible-test-gh-action@release/v1 with: @@ -36,14 +37,14 @@ jobs: integration: name: "Integration (Python: ${{ matrix.python }}, Ansible: ${{ matrix.ansible }}, DB: ${{ matrix.db_engine_name }} ${{ matrix.db_engine_version }}, connector: ${{ matrix.connector_name }} ${{ matrix.connector_version }})" - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 strategy: fail-fast: false matrix: ansible: - - stable-2.14 - stable-2.15 - stable-2.16 + - stable-2.17 - devel db_engine_name: - mysql @@ -111,9 +112,6 @@ jobs: - db_engine_version: 5.7.40 python: '3.10' - - db_engine_version: 5.7.40 - ansible: stable-2.14 - - db_engine_version: 5.7.40 ansible: stable-2.15 @@ -126,9 +124,6 @@ jobs: - db_engine_version: 8.0.31 python: '3.8' - - db_engine_version: 8.0.31 - python: '3.8' - - db_engine_version: 10.4.27 python: '3.10' @@ -174,23 +169,20 @@ jobs: - python: '3.10' connector_version: 2.0.3 - - python: '3.8' - ansible: stable-2.14 - - - python: '3.8' - ansible: stable-2.15 - - python: '3.8' ansible: stable-2.16 + - python: '3.8' + ansible: stable-2.17 + - python: '3.8' ansible: devel - python: '3.9' - ansible: stable-2.15 + ansible: stable-2.16 - python: '3.9' - ansible: stable-2.16 + ansible: stable-2.17 - python: '3.9' ansible: devel @@ -284,16 +276,12 @@ jobs: fi - name: Set docker_image - run: > - docker_image_multiline=(" - ghcr.io/ansible-collections/community.mysql\ + run: |- + echo "docker_image=ghcr.io/ansible-collections/community.mysql\ /test-container-${{ env.db_client }}\ -py${{ env.python_version_flat }}\ -${{ matrix.connector_name }}${{ env.connector_version_flat }}\ - :latest") - - echo "docker_image=$(printf '%s' $docker_image_multiline)" - >> $GITHUB_ENV + :latest" >> $GITHUB_ENV - name: >- Perform integration testing against @@ -332,7 +320,7 @@ jobs: testing-type: integration units: - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 name: Units (Ⓐ${{ matrix.ansible }}) strategy: # As soon as the first unit test fails, @@ -340,20 +328,20 @@ jobs: fail-fast: true matrix: ansible: - - stable-2.14 - stable-2.15 - stable-2.16 + - stable-2.17 - devel python: - 3.8 - 3.9 exclude: - - python: '3.8' - ansible: stable-2.14 - python: '3.8' ansible: stable-2.15 - python: '3.8' ansible: stable-2.16 + - python: '3.8' + ansible: stable-2.17 - python: '3.8' ansible: devel diff --git a/.github/workflows/ansible-test-roles.yml b/.github/workflows/ansible-test-roles.yml.off similarity index 77% rename from .github/workflows/ansible-test-roles.yml rename to .github/workflows/ansible-test-roles.yml.off index da8a805..a11d982 100644 --- a/.github/workflows/ansible-test-roles.yml +++ b/.github/workflows/ansible-test-roles.yml.off @@ -1,6 +1,6 @@ --- name: Roles CI -on: +on: # yamllint disable-line rule:truthy push: paths: - 'roles/**' @@ -15,7 +15,7 @@ on: jobs: molecule: name: "Molecule (Python: ${{ matrix.python }}, Ansible: ${{ matrix.ansible }}, MySQL: ${{ matrix.mysql }})" - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 env: PY_COLORS: 1 ANSIBLE_FORCE_COLOR: 1 @@ -24,26 +24,36 @@ jobs: mysql: - 2.0.12 ansible: - - stable-2.13 - - stable-2.14 - stable-2.15 + - stable-2.16 + - stable-2.17 - devel python: - - 3.8 - - 3.9 + - '3.8' + - '3.9' + - '3.10' exclude: - python: 3.8 + ansible: stable-2.17 + + - python: 3.9 + ansible: stable-2.17 + + - python: 3.8 + ansible: devel + + - python: 3.9 ansible: devel steps: - name: Check out code - uses: actions/checkout@v2 + uses: actions/checkout@v3 with: path: ansible_collections/community/mysql - name: Set up Python ${{ matrix.python }} - uses: actions/setup-python@v2 + uses: actions/setup-python@v4 with: python-version: ${{ matrix.python }} diff --git a/.github/workflows/build-docker-image.yml b/.github/workflows/build-docker-image.yml index fa10268..0edd5ee 100644 --- a/.github/workflows/build-docker-image.yml +++ b/.github/workflows/build-docker-image.yml @@ -1,7 +1,7 @@ --- name: Build Docker Image for ansible-test -on: +on: # yamllint disable-line rule:truthy workflow_call: inputs: registry: diff --git a/.github/workflows/docker-image-mariadb-py310-mysqlclient211.yml b/.github/workflows/docker-image-mariadb-py310-mysqlclient211.yml index be252b7..77286e6 100644 --- a/.github/workflows/docker-image-mariadb-py310-mysqlclient211.yml +++ b/.github/workflows/docker-image-mariadb-py310-mysqlclient211.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mariadb-py310-mysqlclient211 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mariadb-py310-mysqlclient211/**' diff --git a/.github/workflows/docker-image-mariadb-py310-pymysql102.yml b/.github/workflows/docker-image-mariadb-py310-pymysql102.yml index 90fec0e..c7cdfd4 100644 --- a/.github/workflows/docker-image-mariadb-py310-pymysql102.yml +++ b/.github/workflows/docker-image-mariadb-py310-pymysql102.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mariadb-py310-pymysql102 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mariadb-py310-pymysql102/**' diff --git a/.github/workflows/docker-image-mariadb-py38-mysqlclient201.yml b/.github/workflows/docker-image-mariadb-py38-mysqlclient201.yml index c9c04f4..b5b9bb3 100644 --- a/.github/workflows/docker-image-mariadb-py38-mysqlclient201.yml +++ b/.github/workflows/docker-image-mariadb-py38-mysqlclient201.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mariadb-py38-mysqlclient201 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mariadb-py38-mysqlclient201/**' diff --git a/.github/workflows/docker-image-mariadb-py38-pymysql093.yml b/.github/workflows/docker-image-mariadb-py38-pymysql093.yml index 92d0a74..ae6df2e 100644 --- a/.github/workflows/docker-image-mariadb-py38-pymysql093.yml +++ b/.github/workflows/docker-image-mariadb-py38-pymysql093.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mariadb-py38-pymysql093 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mariadb-py38-pymysql093/**' diff --git a/.github/workflows/docker-image-mariadb-py39-mysqlclient203.yml b/.github/workflows/docker-image-mariadb-py39-mysqlclient203.yml index afad5af..4efeef1 100644 --- a/.github/workflows/docker-image-mariadb-py39-mysqlclient203.yml +++ b/.github/workflows/docker-image-mariadb-py39-mysqlclient203.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mariadb-py39-mysqlclient203 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mariadb-py39-mysqlclient203/**' diff --git a/.github/workflows/docker-image-mariadb-py39-pymysql093.yml b/.github/workflows/docker-image-mariadb-py39-pymysql093.yml index 1aa5a04..a3205fb 100644 --- a/.github/workflows/docker-image-mariadb-py39-pymysql093.yml +++ b/.github/workflows/docker-image-mariadb-py39-pymysql093.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mariadb-py39-pymysql093 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mariadb-py39-pymysql093/**' diff --git a/.github/workflows/docker-image-my57-py38-mysqlclient201.yml b/.github/workflows/docker-image-my57-py38-mysqlclient201.yml index 7aaf7e3..b256a47 100644 --- a/.github/workflows/docker-image-my57-py38-mysqlclient201.yml +++ b/.github/workflows/docker-image-my57-py38-mysqlclient201.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI my57-py38-mysqlclient201 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/my57-py38-mysqlclient201/**' diff --git a/.github/workflows/docker-image-my57-py38-pymysql0711.yml b/.github/workflows/docker-image-my57-py38-pymysql0711.yml index 0bc2a9d..0064729 100644 --- a/.github/workflows/docker-image-my57-py38-pymysql0711.yml +++ b/.github/workflows/docker-image-my57-py38-pymysql0711.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI my57-py38-pymysql0711 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/my57-py38-pymysql0711/**' diff --git a/.github/workflows/docker-image-my57-py38-pymysql093.yml b/.github/workflows/docker-image-my57-py38-pymysql093.yml index 462324b..58c7fed 100644 --- a/.github/workflows/docker-image-my57-py38-pymysql093.yml +++ b/.github/workflows/docker-image-my57-py38-pymysql093.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI my57-py38-pymysql093 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/my57-py38-pymysql093/**' diff --git a/.github/workflows/docker-image-mysql-py310-mysqlclient211.yml b/.github/workflows/docker-image-mysql-py310-mysqlclient211.yml index 307aea7..dcb846f 100644 --- a/.github/workflows/docker-image-mysql-py310-mysqlclient211.yml +++ b/.github/workflows/docker-image-mysql-py310-mysqlclient211.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mysql-py310-mysqlclient211 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mysql-py310-mysqlclient211/**' diff --git a/.github/workflows/docker-image-mysql-py310-pymysql102.yml b/.github/workflows/docker-image-mysql-py310-pymysql102.yml index 6f7bf3f..815b923 100644 --- a/.github/workflows/docker-image-mysql-py310-pymysql102.yml +++ b/.github/workflows/docker-image-mysql-py310-pymysql102.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mysql-py310-pymysql102 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mysql-py310-pymysql102/**' diff --git a/.github/workflows/docker-image-mysql-py38-mysqlclient201.yml b/.github/workflows/docker-image-mysql-py38-mysqlclient201.yml index e0da5df..93359a4 100644 --- a/.github/workflows/docker-image-mysql-py38-mysqlclient201.yml +++ b/.github/workflows/docker-image-mysql-py38-mysqlclient201.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mysql-py38-mysqlclient201 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mysql-py38-mysqlclient201/**' diff --git a/.github/workflows/docker-image-mysql-py38-pymysql093.yml b/.github/workflows/docker-image-mysql-py38-pymysql093.yml index 3cc1e0a..ac572ea 100644 --- a/.github/workflows/docker-image-mysql-py38-pymysql093.yml +++ b/.github/workflows/docker-image-mysql-py38-pymysql093.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mysql-py38-pymysql093 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mysql-py38-pymysql093/**' diff --git a/.github/workflows/docker-image-mysql-py39-mysqlclient203.yml b/.github/workflows/docker-image-mysql-py39-mysqlclient203.yml index 0a3a256..b314e57 100644 --- a/.github/workflows/docker-image-mysql-py39-mysqlclient203.yml +++ b/.github/workflows/docker-image-mysql-py39-mysqlclient203.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mysql-py39-mysqlclient203 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mysql-py39-mysqlclient203/**' diff --git a/.github/workflows/docker-image-mysql-py39-pymysql093.yml b/.github/workflows/docker-image-mysql-py39-pymysql093.yml index b974420..55962fb 100644 --- a/.github/workflows/docker-image-mysql-py39-pymysql093.yml +++ b/.github/workflows/docker-image-mysql-py39-pymysql093.yml @@ -1,7 +1,7 @@ --- name: Docker Image CI mysql-py39-pymysql093 -on: +on: # yamllint disable-line rule:truthy push: paths: - 'test-containers/mysql-py39-pymysql093/*' diff --git a/README.md b/README.md index 0e0704e..9853569 100644 --- a/README.md +++ b/README.md @@ -99,9 +99,9 @@ Here is the table for the support timeline: ### ansible-core -- stable-2.14 - stable-2.15 - stable-2.16 +- stable-2.17 - current development version ### Databases diff --git a/TESTING.md b/TESTING.md index 9e0840a..f31db4a 100644 --- a/TESTING.md +++ b/TESTING.md @@ -49,11 +49,9 @@ The Makefile accept the following options - `ansible` - Mandatory: true - Choices: - - "stable-2.12" - - "stable-2.13" - - "stable-2.14" - "stable-2.15" - "stable-2.16" + - "stable-2.17" - "devel" - Description: Version of ansible to install in a venv to run ansible-test diff --git a/tests/integration/targets/test_mysql_user/tasks/test_tls_requirements.yml b/tests/integration/targets/test_mysql_user/tasks/test_tls_requirements.yml index d8c2935..e7c25ce 100644 --- a/tests/integration/targets/test_mysql_user/tasks/test_tls_requirements.yml +++ b/tests/integration/targets/test_mysql_user/tasks/test_tls_requirements.yml @@ -76,14 +76,14 @@ that: - "'SSL' in reqs" vars: - - reqs: "{{((old_result.results[0] is skipped | ternary(new_result, old_result)).results | selectattr('item', 'contains', user_name_1) | first).stdout.split('REQUIRE')[1].split(separator)[0].strip()}}" + reqs: "{{ ((old_result.results[0] is skipped | ternary(new_result, old_result)).results | selectattr('item', 'contains', user_name_1) | first).stdout.split('REQUIRE')[1].split(separator)[0].strip() }}" - name: Tls reqs | Assert user2 TLS requirements assert: that: - "'X509' in reqs" vars: - - reqs: "{{((old_result.results[0] is skipped | ternary(new_result, old_result)).results | selectattr('item', 'contains', user_name_2) | first).stdout.split('REQUIRE')[1].split(separator)[0].strip()}}" + reqs: "{{ ((old_result.results[0] is skipped | ternary(new_result, old_result)).results | selectattr('item', 'contains', user_name_2) | first).stdout.split('REQUIRE')[1].split(separator)[0].strip() }}" - name: Tls reqs | Assert user3 TLS requirements assert: @@ -92,7 +92,7 @@ - "'/CN=org/O=MyDom, Inc./C=US/ST=Oregon/L=Portland' in (reqs | select('contains', 'ISSUER') | first)" - "'ECDHE-ECDSA-AES256-SHA384' in (reqs | select('contains', 'CIPHER') | first)" vars: - - reqs: "{{((old_result.results[0] is skipped | ternary(new_result, old_result)).results | selectattr('item', 'contains', user_name_3) | first).stdout.split('REQUIRE')[1].split(separator)[0].replace(\"' \", \"':\").split(\":\")}}" + reqs: "{{ ((old_result.results[0] is skipped | ternary(new_result, old_result)).results | selectattr('item', 'contains', user_name_3) | first).stdout.split('REQUIRE')[1].split(separator)[0].replace(\"' \", \"':\").split(\":\") }}" # CentOS 6 uses an older version of jinja that does not provide the selectattr filter. when: ansible_distribution != 'CentOS' or ansible_distribution_major_version != '6' @@ -129,7 +129,7 @@ assert: that: "'SSL' in reqs" vars: - - reqs: "{{(old_result is skipped | ternary(new_result, old_result)).stdout.split('REQUIRE')[1].split(separator)[0].strip()}}" + reqs: "{{ (old_result is skipped | ternary(new_result, old_result)).stdout.split('REQUIRE')[1].split(separator)[0].strip() }}" - name: Tls reqs | Modify user with TLS requirements state=present (expect changed=true) mysql_user: @@ -157,7 +157,7 @@ assert: that: "'X509' in reqs" vars: - - reqs: "{{(old_result is skipped | ternary(new_result, old_result)).stdout.split('REQUIRE')[1].split(separator)[0].strip()}}" + reqs: "{{ (old_result is skipped | ternary(new_result, old_result)).stdout.split('REQUIRE')[1].split(separator)[0].strip() }}" - name: Tls reqs | Remove TLS requirements from user (expect changed=true) mysql_user: From 0618ff6c41c0c76c923485d74fa8dd3db7177fd2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Fri, 12 Apr 2024 09:00:43 +0200 Subject: [PATCH 04/54] Fix sanity tests for ansible-core 2.18 (#627) --- tests/sanity/{ignore-2.14.txt => ignore-2.18.txt} | 2 ++ 1 file changed, 2 insertions(+) rename tests/sanity/{ignore-2.14.txt => ignore-2.18.txt} (57%) diff --git a/tests/sanity/ignore-2.14.txt b/tests/sanity/ignore-2.18.txt similarity index 57% rename from tests/sanity/ignore-2.14.txt rename to tests/sanity/ignore-2.18.txt index 90ddba3..55b2904 100644 --- a/tests/sanity/ignore-2.14.txt +++ b/tests/sanity/ignore-2.18.txt @@ -1,2 +1,4 @@ plugins/modules/mysql_db.py validate-modules:use-run-command-not-popen plugins/modules/mysql_user.py validate-modules:undocumented-parameter +plugins/module_utils/mysql.py pylint:unused-import +plugins/module_utils/version.py pylint:unused-import From 47710cfb93fad4f98c5895d5a263fadd1d0cc8c5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Tue, 16 Apr 2024 10:52:24 +0200 Subject: [PATCH 05/54] Enhance support of tls_requires in mysql_user and mysql_info (#628) * fix option name * Add tests for users using SSL * Rewrite get_tls_requires using mysql.user table * Add tls_requires to users_info filter * add more consistant test users * Add tls tests users in cleanup task * Fix tls_requires data structure inconsistencies between modules * Refactor user implementation to host get_tls_requires * fix MySQL tls_requires not removed from user passed as empty * Fix wrong variable used to return a hashed password * Fix sanity * fix unit tests * Add changelog fragment * Add PR URI to the changelog * Add more precise change log * fix documentation using wrong variable as an example * Document example returned value `tls_requires` from users_info filter * Revert changes that will be in a separate PR * Fix sanity --- .../fragments/mysql_user_tls_requires.yml | 6 ++ .../implementations/mariadb/user.py | 45 ++++++++++ .../implementations/mysql/user.py | 46 ++++++++++ plugins/module_utils/user.py | 43 +++------- plugins/modules/mysql_info.py | 22 +++-- plugins/modules/mysql_role.py | 4 +- plugins/modules/mysql_user.py | 3 - .../tasks/filter_users_info.yml | 85 +++++++++++++++++-- tests/unit/plugins/modules/test_mysql_info.py | 14 +-- 9 files changed, 213 insertions(+), 55 deletions(-) create mode 100644 changelogs/fragments/mysql_user_tls_requires.yml diff --git a/changelogs/fragments/mysql_user_tls_requires.yml b/changelogs/fragments/mysql_user_tls_requires.yml new file mode 100644 index 0000000..1fa0c94 --- /dev/null +++ b/changelogs/fragments/mysql_user_tls_requires.yml @@ -0,0 +1,6 @@ +--- +minor_changes: + - mysql_info - Add ``tls_requires`` returned value for the ``users_info`` filter (https://github.com/ansible-collections/community.mysql/pull/628). +bugfixes: + - mysql_user - Fix idempotence when using variables from the ``users_info`` filter of ``mysql_info`` as an input (https://github.com/ansible-collections/community.mysql/pull/628). + - mysql_user - Fix ``tls_requires`` not removing ``SSL`` and ``X509`` when sets as empty (https://github.com/ansible-collections/community.mysql/pull/628). diff --git a/plugins/module_utils/implementations/mariadb/user.py b/plugins/module_utils/implementations/mariadb/user.py index cdc14b2..fa9ecdf 100644 --- a/plugins/module_utils/implementations/mariadb/user.py +++ b/plugins/module_utils/implementations/mariadb/user.py @@ -29,3 +29,48 @@ def server_supports_password_expire(cursor): version = get_server_version(cursor) return LooseVersion(version) >= LooseVersion("10.4.3") + + +def get_tls_requires(cursor, user, host): + """Get user TLS requirements. + Reads directly from mysql.user table allowing for a more + readable code. + + Args: + cursor (cursor): DB driver cursor object. + user (str): User name. + host (str): User host name. + + Returns: Dictionary containing current TLS required + """ + tls_requires = dict() + + query = ('SELECT ssl_type, ssl_cipher, x509_issuer, x509_subject ' + 'FROM mysql.user WHERE User = %s AND Host = %s') + cursor.execute(query, (user, host)) + res = cursor.fetchone() + + # Mysql_info use a DictCursor so we must convert back to a list + # otherwise we get KeyError 0 + if isinstance(res, dict): + res = list(res.values()) + + # When user don't require SSL, res value is: ('', '', '', '') + if not any(res): + return None + + if res[0] == 'ANY': + tls_requires['SSL'] = None + + if res[0] == 'X509': + tls_requires['X509'] = None + + if res[1]: + tls_requires['CIPHER'] = res[1] + + if res[2]: + tls_requires['ISSUER'] = res[2] + + if res[3]: + tls_requires['SUBJECT'] = res[3] + return tls_requires diff --git a/plugins/module_utils/implementations/mysql/user.py b/plugins/module_utils/implementations/mysql/user.py index 4e41c05..700c355 100644 --- a/plugins/module_utils/implementations/mysql/user.py +++ b/plugins/module_utils/implementations/mysql/user.py @@ -8,6 +8,9 @@ __metaclass__ = type from ansible_collections.community.mysql.plugins.module_utils.version import LooseVersion from ansible_collections.community.mysql.plugins.module_utils.mysql import get_server_version +import re +import shlex + def use_old_user_mgmt(cursor): version = get_server_version(cursor) @@ -30,3 +33,46 @@ def server_supports_password_expire(cursor): version = get_server_version(cursor) return LooseVersion(version) >= LooseVersion("5.7") + + +def get_tls_requires(cursor, user, host): + """Get user TLS requirements. + We must use SHOW GRANTS because some tls fileds are encoded. + + Args: + cursor (cursor): DB driver cursor object. + user (str): User name. + host (str): User host name. + + Returns: Dictionary containing current TLS required + """ + if not use_old_user_mgmt(cursor): + query = "SHOW CREATE USER '%s'@'%s'" % (user, host) + else: + query = "SHOW GRANTS for '%s'@'%s'" % (user, host) + + cursor.execute(query) + grants = cursor.fetchone() + + # Mysql_info use a DictCursor so we must convert back to a list + # otherwise we get KeyError 0 + if isinstance(grants, dict): + grants = list(grants.values()) + grants_str = ''.join(grants) + + pattern = r"(?<=\bREQUIRE\b)(.*?)(?=(?:\bPASSWORD\b|$))" + requires_match = re.search(pattern, grants_str) + requires = requires_match.group().strip() if requires_match else "" + + if requires.startswith('NONE'): + return None + + if requires.startswith('SSL'): + return {'SSL': None} + + if requires.startswith('X509'): + return {'X509': None} + + items = iter(shlex.split(requires)) + requires = dict(zip(items, items)) + return requires or None diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index f042c85..d4ae9dd 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -17,6 +17,7 @@ from ansible.module_utils.six import iteritems from ansible_collections.community.mysql.plugins.module_utils.mysql import ( mysql_driver, + get_server_implementation, ) @@ -80,31 +81,6 @@ def do_not_mogrify_requires(query, params, tls_requires): return query, params -def get_tls_requires(cursor, user, host): - if user: - if not impl.use_old_user_mgmt(cursor): - query = "SHOW CREATE USER '%s'@'%s'" % (user, host) - else: - query = "SHOW GRANTS for '%s'@'%s'" % (user, host) - - cursor.execute(query) - require_list = [tuple[0] for tuple in filter(lambda x: "REQUIRE" in x[0], cursor.fetchall())] - require_line = require_list[0] if require_list else "" - pattern = r"(?<=\bREQUIRE\b)(.*?)(?=(?:\bPASSWORD\b|$))" - requires_match = re.search(pattern, require_line) - requires = requires_match.group().strip() if requires_match else "" - if any((requires.startswith(req) for req in ('SSL', 'X509', 'NONE'))): - requires = requires.split()[0] - if requires == 'NONE': - requires = None - else: - import shlex - - items = iter(shlex.split(requires)) - requires = dict(zip(items, items)) - return requires or None - - def get_grants(cursor, user, host): cursor.execute("SHOW GRANTS FOR %s@%s", (user, host)) grants_line = list(filter(lambda x: "ON *.*" in x[0], cursor.fetchall()))[0] @@ -166,6 +142,7 @@ def user_add(cursor, user, host, host_all, password, encrypted, return {'changed': True, 'password_changed': None, 'attributes': attributes} # Determine what user management method server uses + impl = get_user_implementation(cursor) old_user_mgmt = impl.use_old_user_mgmt(cursor) mogrify = do_not_mogrify_requires if old_user_mgmt else mogrify_requires @@ -244,6 +221,7 @@ def user_mod(cursor, user, host, host_all, password, encrypted, grant_option = False # Determine what user management method server uses + impl = get_user_implementation(cursor) old_user_mgmt = impl.use_old_user_mgmt(cursor) if host_all and not role: @@ -499,7 +477,7 @@ def user_mod(cursor, user, host, host_all, password, encrypted, continue # Handle TLS requirements - current_requires = get_tls_requires(cursor, user, host) + current_requires = sanitize_requires(impl.get_tls_requires(cursor, user, host)) if current_requires != tls_requires: msg = "TLS requires updated" if not module.check_mode: @@ -837,6 +815,7 @@ def privileges_grant(cursor, user, host, db_table, priv, tls_requires, maria_rol query.append("TO %s") params = (user) + impl = get_user_implementation(cursor) if tls_requires and impl.use_old_user_mgmt(cursor): query, params = mogrify_requires(" ".join(query), params, tls_requires) query = [query] @@ -973,6 +952,7 @@ def limit_resources(module, cursor, user, host, resource_limits, check_mode): Returns: True, if changed, False otherwise. """ + impl = get_user_implementation(cursor) if not impl.server_supports_alter_user(cursor): module.fail_json(msg="The server version does not match the requirements " "for resource_limits parameter. See module's documentation.") @@ -1108,12 +1088,11 @@ def attributes_get(cursor, user, host): return j if j else None -def get_impl(cursor): - global impl - cursor.execute("SELECT VERSION()") - if 'mariadb' in cursor.fetchone()[0].lower(): +def get_user_implementation(cursor): + db_engine = get_server_implementation(cursor) + if db_engine == 'mariadb': from ansible_collections.community.mysql.plugins.module_utils.implementations.mariadb import user as mariauser - impl = mariauser + return mariauser else: from ansible_collections.community.mysql.plugins.module_utils.implementations.mysql import user as mysqluser - impl = mysqluser + return mysqluser diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index 0be25fa..f30f1a1 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -146,7 +146,7 @@ EXAMPLES = r''' plugin: "{{ item.plugin | default(omit) }}" plugin_auth_string: "{{ item.plugin_auth_string | default(omit) }}" plugin_hash_string: "{{ item.plugin_hash_string | default(omit) }}" - tls_require: "{{ item.tls_require | default(omit) }}" + tls_requires: "{{ item.tls_requires | default(omit) }}" priv: "{{ item.priv | default(omit) }}" resource_limits: "{{ item.resource_limits | default(omit) }}" column_case_sensitive: true @@ -240,7 +240,8 @@ users_info: "host": "host.com", "plugin": "mysql_native_password", "priv": "db1.*:SELECT/db2.*:SELECT", - "resource_limits": { "MAX_USER_CONNECTIONS": 100 } } + "resource_limits": { "MAX_USER_CONNECTIONS": 100 }, + "tls_requires": { "SSL": null } } version_added: '3.8.0' engines: description: Information about the server's storage engines. @@ -300,6 +301,7 @@ from ansible_collections.community.mysql.plugins.module_utils.user import ( privileges_get, get_resource_limits, get_existing_authentication, + get_user_implementation, ) from ansible.module_utils.six import iteritems from ansible.module_utils._text import to_native @@ -327,10 +329,11 @@ class MySQL_Info(object): 5. add info about the new subset with an example to RETURN block """ - def __init__(self, module, cursor, server_implementation): + def __init__(self, module, cursor, server_implementation, user_implementation): self.module = module self.cursor = cursor self.server_implementation = server_implementation + self.user_implementation = user_implementation self.info = { 'version': {}, 'databases': {}, @@ -602,13 +605,17 @@ class MySQL_Info(object): priv_string.remove('*.*:USAGE') resource_limits = get_resource_limits(self.cursor, user, host) - copy_ressource_limits = dict.copy(resource_limits) + + tls_requires = self.user_implementation.get_tls_requires( + self.cursor, user, host) + output_dict = { 'name': user, 'host': host, 'priv': '/'.join(priv_string), 'resource_limits': copy_ressource_limits, + 'tls_requires': tls_requires, } # Prevent returning a resource limit if empty @@ -619,6 +626,10 @@ class MySQL_Info(object): if len(output_dict['resource_limits']) == 0: del output_dict['resource_limits'] + # Prevent returning tls_require if empty + if not tls_requires: + del output_dict['tls_requires'] + authentications = get_existing_authentication(self.cursor, user, host) if authentications: output_dict.update(authentications) @@ -745,11 +756,12 @@ def main(): module.fail_json(msg) server_implementation = get_server_implementation(cursor) + user_implementation = get_user_implementation(cursor) ############################### # Create object and do main job - mysql = MySQL_Info(module, cursor, server_implementation) + mysql = MySQL_Info(module, cursor, server_implementation, user_implementation) module.exit_json(changed=False, connector_name=connector_name, diff --git a/plugins/modules/mysql_role.py b/plugins/modules/mysql_role.py index 3e3462a..65ed894 100644 --- a/plugins/modules/mysql_role.py +++ b/plugins/modules/mysql_role.py @@ -309,7 +309,7 @@ from ansible_collections.community.mysql.plugins.module_utils.mysql import ( ) from ansible_collections.community.mysql.plugins.module_utils.user import ( convert_priv_dict_to_str, - get_impl, + get_user_implementation, get_mode, user_mod, privileges_grant, @@ -1054,7 +1054,7 @@ def main(): # Set defaults changed = False - get_impl(cursor) + impl = get_user_implementation(cursor) if priv is not None: try: diff --git a/plugins/modules/mysql_user.py b/plugins/modules/mysql_user.py index e02b153..fa54c7d 100644 --- a/plugins/modules/mysql_user.py +++ b/plugins/modules/mysql_user.py @@ -401,7 +401,6 @@ from ansible_collections.community.mysql.plugins.module_utils.mysql import ( ) from ansible_collections.community.mysql.plugins.module_utils.user import ( convert_priv_dict_to_str, - get_impl, get_mode, InvalidPrivsError, limit_resources, @@ -528,8 +527,6 @@ def main(): if session_vars: set_session_vars(module, cursor, session_vars) - get_impl(cursor) - if priv is not None: try: mode = get_mode(cursor) diff --git a/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml b/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml index 2c126c1..63ce190 100644 --- a/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml +++ b/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml @@ -47,7 +47,7 @@ state: import target: /root/create_procedure.sql - # Use a query instead of mysql_user, because we want to caches differences + # Use a query instead of mysql_user, because we want to catch differences # at the end and a bug in mysql_user would be invisible to this tests - name: Mysql_info users_info | Prepare common tests users community.mysql.mysql_query: @@ -147,6 +147,69 @@ '*CB3326D5279DE7915FE5D743232165EE887883CA' - GRANT SELECT ON users_info_db.* TO users_info_multi_hosts@'host2' + - >- + CREATE USER users_info_tls_none@'host' + IDENTIFIED WITH mysql_native_password AS + '*CB3326D5279DE7915FE5D743232165EE887883CA' REQUIRE NONE + - GRANT SELECT ON users_info_db.* TO users_info_tls_none@'host' + + - >- + CREATE USER users_info_tls_ssl@'host' + IDENTIFIED WITH mysql_native_password AS + '*CB3326D5279DE7915FE5D743232165EE887883CA' REQUIRE SSL + - GRANT SELECT ON users_info_db.* TO users_info_tls_ssl@'host' + + - >- + CREATE USER users_info_tls_cipher@'host' + IDENTIFIED WITH mysql_native_password AS + '*CB3326D5279DE7915FE5D743232165EE887883CA' + REQUIRE CIPHER 'ECDH-RSA-AES256-SHA384' + - GRANT SELECT ON users_info_db.* TO users_info_tls_cipher@'host' + + - >- + CREATE USER users_info_tls_x509@'host' + IDENTIFIED WITH mysql_native_password AS + '*CB3326D5279DE7915FE5D743232165EE887883CA' REQUIRE X509 + - GRANT SELECT ON users_info_db.* TO users_info_tls_x509@'host' + + - >- + CREATE USER users_info_tls_subject@'host' + IDENTIFIED WITH mysql_native_password AS + '*CB3326D5279DE7915FE5D743232165EE887883CA' + REQUIRE SUBJECT '/CN=Bob/O=MyDom/C=US/ST=Oregon/L=Portland' + - GRANT SELECT ON users_info_db.* TO users_info_tls_subject@'host' + + - >- + CREATE USER users_info_tls_issuer@'host' + IDENTIFIED WITH mysql_native_password AS + '*CB3326D5279DE7915FE5D743232165EE887883CA' + REQUIRE ISSUER '/C=FI/ST=Somewhere/L=City/ + O=CompanyX/CN=Bob/emailAddress=bob@companyx.com' + - GRANT SELECT ON users_info_db.* TO users_info_tls_issuer@'host' + + - >- + CREATE USER users_info_tls_subject_issuer@'host' + IDENTIFIED WITH mysql_native_password AS + '*CB3326D5279DE7915FE5D743232165EE887883CA' + REQUIRE SUBJECT '/CN=Bob/O=MyDom/C=US/ST=Oregon/L=Portland' + AND ISSUER '/C=FI/ST=Somewhere/L=City/ + O=CompanyX/CN=Bob/emailAddress=bob@companyx.com' + - >- + GRANT SELECT ON users_info_db.* + TO users_info_tls_subject_issuer@'host' + + - >- + CREATE USER users_info_tls_sub_issu_ciph@'host' + IDENTIFIED WITH mysql_native_password AS + '*CB3326D5279DE7915FE5D743232165EE887883CA' + REQUIRE SUBJECT '/CN=Bob/O=MyDom/C=US/ST=Oregon/L=Portland' + AND ISSUER '/C=FI/ST=Somewhere/L=City/ + O=CompanyX/CN=Bob/emailAddress=bob@companyx.com' + AND CIPHER 'ECDH-RSA-AES256-SHA384' + - >- + GRANT SELECT ON users_info_db.* + TO users_info_tls_sub_issu_ciph@'host' + - name: Mysql_info users_info | Prepare tests users for MariaDB community.mysql.mysql_user: name: "{{ item.name }}" @@ -154,7 +217,7 @@ plugin: "{{ item.plugin | default(omit) }}" plugin_auth_string: "{{ item.plugin_auth_string | default(omit) }}" plugin_hash_string: "{{ item.plugin_hash_string | default(omit) }}" - tls_require: "{{ item.tls_require | default(omit) }}" + tls_requires: "{{ item.tls_requires | default(omit) }}" priv: "{{ item.priv }}" resource_limits: "{{ item.resource_limits | default(omit) }}" column_case_sensitive: true @@ -174,7 +237,7 @@ plugin: "{{ item.plugin | default(omit) }}" plugin_auth_string: "{{ item.plugin_auth_string | default(omit) }}" plugin_hash_string: "{{ item.plugin_hash_string | default(omit) }}" - tls_require: "{{ item.tls_require | default(omit) }}" + tls_requires: "{{ item.tls_requires | default(omit) }}" priv: "{{ item.priv }}" resource_limits: "{{ item.resource_limits | default(omit) }}" column_case_sensitive: true @@ -196,7 +259,7 @@ plugin: "{{ item.plugin | default(omit) }}" plugin_auth_string: "{{ item.plugin_auth_string | default(omit) }}" plugin_hash_string: "{{ item.plugin_hash_string | default(omit) }}" - tls_require: "{{ item.tls_require | default(omit) }}" + tls_requires: "{{ item.tls_requires | default(omit) }}" priv: "{{ item.priv }}" resource_limits: "{{ item.resource_limits | default(omit) }}" column_case_sensitive: true @@ -227,7 +290,7 @@ plugin: "{{ item.plugin | default(omit) }}" plugin_auth_string: "{{ item.plugin_auth_string | default(omit) }}" plugin_hash_string: "{{ item.plugin_hash_string | default(omit) }}" - tls_require: "{{ item.tls_require | default(omit) }}" + tls_requires: "{{ item.tls_requires | default(omit) }}" priv: "{{ item.priv | default(omit) }}" resource_limits: "{{ item.resource_limits | default(omit) }}" column_case_sensitive: true @@ -237,7 +300,9 @@ label: "{{ item.name }}@{{ item.host }}" register: recreate_users_result failed_when: - - recreate_users_result is changed + - >- + recreate_users_result is changed or + recreate_users_result.msg != 'User unchanged' when: - item.name != 'root' - item.name != 'mysql' @@ -265,6 +330,14 @@ - users_info_usage_only - users_info_columns_uppercase - users_info_multi_hosts + - users_info_tls_none + - users_info_tls_ssl + - users_info_tls_cipher + - users_info_tls_x509 + - users_info_tls_subject + - users_info_tls_issuer + - users_info_tls_subject_issuer + - users_info_tls_sub_issu_ciph - name: Mysql_info users_info | Cleanup databases community.mysql.mysql_db: diff --git a/tests/unit/plugins/modules/test_mysql_info.py b/tests/unit/plugins/modules/test_mysql_info.py index 6aaf66e..0d086f4 100644 --- a/tests/unit/plugins/modules/test_mysql_info.py +++ b/tests/unit/plugins/modules/test_mysql_info.py @@ -14,15 +14,15 @@ from ansible_collections.community.mysql.plugins.modules.mysql_info import MySQL @pytest.mark.parametrize( - 'suffix,cursor_output,server_implementation', + 'suffix,cursor_output,server_implementation,user_implementation', [ - ('mysql', '5.5.1-mysql', 'mysql'), - ('log', '5.7.31-log', 'mysql'), - ('mariadb', '10.5.0-mariadb', 'mariadb'), - ('', '8.0.22', 'mysql'), + ('mysql', '5.5.1-mysql', 'mysql', 'mysql'), + ('log', '5.7.31-log', 'mysql', 'mysql'), + ('mariadb', '10.5.0-mariadb', 'mariadb', 'mariadb'), + ('', '8.0.22', 'mysql', 'mysql'), ] ) -def test_get_info_suffix(suffix, cursor_output, server_implementation): +def test_get_info_suffix(suffix, cursor_output, server_implementation, user_implementation): def __cursor_return_value(input_parameter): if input_parameter == "SHOW GLOBAL VARIABLES": cursor.fetchall.return_value = [{"Variable_name": "version", "Value": cursor_output}] @@ -32,6 +32,6 @@ def test_get_info_suffix(suffix, cursor_output, server_implementation): cursor = MagicMock() cursor.execute.side_effect = __cursor_return_value - info = MySQL_Info(MagicMock(), cursor, server_implementation) + info = MySQL_Info(MagicMock(), cursor, server_implementation, user_implementation) assert info.get_info([], [], False)['version']['suffix'] == suffix From 6ce2f49f96373bc357a71bdcf4ae1412086d8f4b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Thu, 2 May 2024 10:26:04 +0200 Subject: [PATCH 06/54] Improve get replica/primary status (#634) * Fix case where a failed fetchone() still return a dict * Fix test for MariaDB * fix case where a failed fetchone() still return a dict for primary * Add changelog fragment --- .../improve_get_replica_primary_status.yml | 4 ++++ plugins/modules/mysql_replication.py | 20 ++++++++++++------- 2 files changed, 17 insertions(+), 7 deletions(-) create mode 100644 changelogs/fragments/improve_get_replica_primary_status.yml diff --git a/changelogs/fragments/improve_get_replica_primary_status.yml b/changelogs/fragments/improve_get_replica_primary_status.yml new file mode 100644 index 0000000..512d7ef --- /dev/null +++ b/changelogs/fragments/improve_get_replica_primary_status.yml @@ -0,0 +1,4 @@ +--- +minor_changes: + + - mysql_replication - Improve detection of IsReplica and IsPrimary by inspecting the dictionary returned from the SQL query instead of relying on variable types. This ensures compatibility with changes in the connector or the output of SHOW REPLICA STATUS and SHOW MASTER STATUS, allowing for easier maintenance if these change in the future. diff --git a/plugins/modules/mysql_replication.py b/plugins/modules/mysql_replication.py index 934b479..f4f192a 100644 --- a/plugins/modules/mysql_replication.py +++ b/plugins/modules/mysql_replication.py @@ -550,20 +550,26 @@ def main(): if mode == 'getprimary': status = get_primary_status(cursor) - if not isinstance(status, dict): - status = dict(Is_Primary=False, - msg="Server is not configured as mysql primary") - else: + if status and "File" in status and "Position" in status: status['Is_Primary'] = True + else: + status = dict( + Is_Primary=False, + msg="Server is not configured as mysql primary. " + "Meaning: Binary logs are disabled") module.exit_json(queries=executed_queries, **status) elif mode == "getreplica": status = get_replica_status(cursor, connection_name, channel, replica_term) - if not isinstance(status, dict): - status = dict(Is_Replica=False, msg="Server is not configured as mysql replica") - else: + # MySQL 8.0 uses Replica_... + # MariaDB 10.6 uses Slave_... + if status and ( + "Slave_IO_Running" in status or + "Replica_IO_Running" in status): status['Is_Replica'] = True + else: + status = dict(Is_Replica=False, msg="Server is not configured as mysql replica") module.exit_json(queries=executed_queries, **status) From a80b805619f108580ecb09d7d02693316fa3765b Mon Sep 17 00:00:00 2001 From: Dennis Felipe Urtubia <33161939+dennisurtubia@users.noreply.github.com> Date: Tue, 21 May 2024 15:58:05 -0300 Subject: [PATCH 07/54] Adds support for `CHANGE REPLICATION SOURCE TO` statement (#636) * feat: adds support for 'change replication source to' statement --- ...rts_mysql_change_replication_source_to.yml | 3 + plugins/modules/mysql_replication.py | 73 ++++++++++++++++++- .../test_mysql_replication/tasks/main.yml | 5 ++ ...sql_replication_changereplication_mode.yml | 65 +++++++++++++++++ .../tasks/mysql_replication_initial.yml | 2 +- 5 files changed, 146 insertions(+), 2 deletions(-) create mode 100644 changelogs/fragments/supports_mysql_change_replication_source_to.yml create mode 100644 tests/integration/targets/test_mysql_replication/tasks/mysql_replication_changereplication_mode.yml diff --git a/changelogs/fragments/supports_mysql_change_replication_source_to.yml b/changelogs/fragments/supports_mysql_change_replication_source_to.yml new file mode 100644 index 0000000..955d62e --- /dev/null +++ b/changelogs/fragments/supports_mysql_change_replication_source_to.yml @@ -0,0 +1,3 @@ +--- +minor_changes: + - mysql_replication - Adds support for `CHANGE REPLICATION SOURCE TO` statement (https://github.com/ansible-collections/community.mysql/issues/635). diff --git a/plugins/modules/mysql_replication.py b/plugins/modules/mysql_replication.py index f4f192a..23c94c1 100644 --- a/plugins/modules/mysql_replication.py +++ b/plugins/modules/mysql_replication.py @@ -19,11 +19,13 @@ description: author: - Balazs Pocze (@banyek) - Andrew Klychkov (@Andersson007) +- Dennis Urtubia (@dennisurtubia) options: mode: description: - Module operating mode. Could be C(changeprimary) (CHANGE MASTER TO), + C(changereplication) (CHANGE REPLICATION SOURCE TO) - only supported in MySQL 8.0.23 and later, C(getprimary) (SHOW MASTER STATUS), C(getreplica) (SHOW REPLICA STATUS), C(startreplica) (START REPLICA), @@ -34,6 +36,7 @@ options: type: str choices: - changeprimary + - changereplication - getprimary - getreplica - startreplica @@ -229,6 +232,13 @@ EXAMPLES = r''' primary_log_file: mysql-bin.000009 primary_log_pos: 4578 +- name: Change replication source to replica server 192.0.2.1 and use binary log 'mysql-bin.000009' with position 4578 + community.mysql.mysql_replication: + mode: changereplication + primary_host: 192.0.2.1 + primary_log_file: mysql-bin.000009 + primary_log_pos: 4578 + - name: Check replica status using port 3308 community.mysql.mysql_replication: mode: getreplica @@ -438,6 +448,16 @@ def changeprimary(cursor, chm, connection_name='', channel=''): cursor.execute(query) +def changereplication(cursor, chm, channel=''): + query = 'CHANGE REPLICATION SOURCE TO %s' % ','.join(chm) + + if channel: + query += " FOR CHANNEL '%s'" % channel + + executed_queries.append(query) + cursor.execute(query) + + def main(): argument_spec = mysql_common_argument_spec() argument_spec.update( @@ -449,7 +469,8 @@ def main(): 'startreplica', 'resetprimary', 'resetreplica', - 'resetreplicaall']), + 'resetreplicaall', + 'changereplication']), primary_auto_position=dict(type='bool', default=False, aliases=['master_auto_position']), primary_host=dict(type='str', aliases=['master_host']), primary_user=dict(type='str', aliases=['master_user']), @@ -655,6 +676,56 @@ def main(): module.exit_json(msg="Replica reset", changed=True, queries=executed_queries) else: module.exit_json(msg="Replica already reset", changed=False, queries=executed_queries) + elif mode == 'changereplication': + chm = [] + result = {} + if primary_host is not None: + chm.append("SOURCE_HOST='%s'" % primary_host) + if primary_user is not None: + chm.append("SOURCE_USER='%s'" % primary_user) + if primary_password is not None: + chm.append("SOURCE_PASSWORD='%s'" % primary_password) + if primary_port is not None: + chm.append("SOURCE_PORT=%s" % primary_port) + if primary_connect_retry is not None: + chm.append("SOURCE_CONNECT_RETRY=%s" % primary_connect_retry) + if primary_log_file is not None: + chm.append("SOURCE_LOG_FILE='%s'" % primary_log_file) + if primary_log_pos is not None: + chm.append("SOURCE_LOG_POS=%s" % primary_log_pos) + if primary_delay is not None: + chm.append("SOURCE_DELAY=%s" % primary_delay) + if relay_log_file is not None: + chm.append("RELAY_LOG_FILE='%s'" % relay_log_file) + if relay_log_pos is not None: + chm.append("RELAY_LOG_POS=%s" % relay_log_pos) + if primary_ssl is not None: + if primary_ssl: + chm.append("SOURCE_SSL=1") + else: + chm.append("SOURCE_SSL=0") + if primary_ssl_ca is not None: + chm.append("SOURCE_SSL_CA='%s'" % primary_ssl_ca) + if primary_ssl_capath is not None: + chm.append("SOURCE_SSL_CAPATH='%s'" % primary_ssl_capath) + if primary_ssl_cert is not None: + chm.append("SOURCE_SSL_CERT='%s'" % primary_ssl_cert) + if primary_ssl_key is not None: + chm.append("SOURCE_SSL_KEY='%s'" % primary_ssl_key) + if primary_ssl_cipher is not None: + chm.append("SOURCE_SSL_CIPHER='%s'" % primary_ssl_cipher) + if primary_ssl_verify_server_cert: + chm.append("SOURCE_SSL_VERIFY_SERVER_CERT=1") + if primary_auto_position: + chm.append("SOURCE_AUTO_POSITION=1") + try: + changereplication(cursor, chm, channel) + except mysql_driver.Warning as e: + result['warning'] = to_native(e) + except Exception as e: + module.fail_json(msg='%s. Query == CHANGE REPLICATION SOURCE TO %s' % (to_native(e), chm)) + result['changed'] = True + module.exit_json(queries=executed_queries, **result) warnings.simplefilter("ignore") diff --git a/tests/integration/targets/test_mysql_replication/tasks/main.yml b/tests/integration/targets/test_mysql_replication/tasks/main.yml index ab5b4a3..2baa536 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/main.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/main.yml @@ -25,3 +25,8 @@ - import_tasks: mysql_replication_resetprimary_mode.yml - include_tasks: issue-28.yml + +# Tests of changereplication mode: +- import_tasks: mysql_replication_changereplication_mode.yml + when: + - db_engine == 'mysql' diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_changereplication_mode.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_changereplication_mode.yml new file mode 100644 index 0000000..2f593ca --- /dev/null +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_changereplication_mode.yml @@ -0,0 +1,65 @@ +--- + +- vars: + mysql_params: &mysql_params + login_user: '{{ mysql_user }}' + login_password: '{{ mysql_password }}' + login_host: '{{ mysql_host }}' + + block: + # Get primary log file and log pos: + - name: Get primary status + mysql_replication: + <<: *mysql_params + login_port: '{{ mysql_primary_port }}' + mode: getprimary + register: mysql_primary_status + + # Test changereplication mode: + - name: Run replication + mysql_replication: + <<: *mysql_params + login_port: '{{ mysql_replica1_port }}' + mode: changereplication + primary_host: '{{ mysql_host }}' + primary_port: '{{ mysql_primary_port }}' + primary_user: '{{ replication_user }}' + primary_password: '{{ replication_pass }}' + primary_log_file: '{{ mysql_primary_status.File }}' + primary_log_pos: '{{ mysql_primary_status.Position }}' + primary_ssl_ca: '' + primary_ssl: no + register: result + + - name: Assert that changereplication is changed and return expected query + assert: + that: + - result is changed + - result.queries == expected_queries + vars: + expected_queries: ["CHANGE REPLICATION SOURCE TO SOURCE_HOST='{{ mysql_host }}',\ + SOURCE_USER='{{ replication_user }}',SOURCE_PASSWORD='********',\ + SOURCE_PORT={{ mysql_primary_port }},SOURCE_LOG_FILE=\ + '{{ mysql_primary_status.File }}',SOURCE_LOG_POS=\ + {{ mysql_primary_status.Position }},SOURCE_SSL=0,SOURCE_SSL_CA=''"] + + # Test changereplication mode with channel: + - name: Run replication + mysql_replication: + <<: *mysql_params + login_port: '{{ mysql_replica1_port }}' + mode: changereplication + primary_user: '{{ replication_user }}' + primary_password: '{{ replication_pass }}' + channel: '{{ test_channel }}' + + register: with_channel_result_queries + + - name: Assert that changereplication is changed and is called correctly with channel + assert: + that: + - with_channel_result_queries is changed + - with_channel_result_queries.queries == expected_queries + vars: + expected_queries: ["CHANGE REPLICATION SOURCE TO SOURCE_USER='{{ replication_user }}',\ + SOURCE_PASSWORD='********' FOR CHANNEL '{{ test_channel }}'"] diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml index ea7a5ac..e08954b 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml @@ -318,5 +318,5 @@ - name: Assert that stopslave returns expected error message assert: that: - - result.msg == "value of mode must be one of{{ ":" }} getprimary, getreplica, changeprimary, stopreplica, startreplica, resetprimary, resetreplica, resetreplicaall, got{{ ":" }} stopslave" + - result.msg == "value of mode must be one of{{ ":" }} getprimary, getreplica, changeprimary, stopreplica, startreplica, resetprimary, resetreplica, resetreplicaall, changereplication, got{{ ":" }} stopslave" - result is failed From 47610347baa5a23a65f0d3221382a09ee964f0e1 Mon Sep 17 00:00:00 2001 From: Dennis Felipe Urtubia <33161939+dennisurtubia@users.noreply.github.com> Date: Thu, 30 May 2024 12:10:36 -0300 Subject: [PATCH 08/54] Adds support for show binary log status statement (#638) * feat: adds support for show binary log status statement * feat: adds support for mariadb show binlog status statement --- .../get_primary_show_binary_log_status.yml | 4 ++++ plugins/modules/mysql_replication.py | 19 +++++++++++++++---- 2 files changed, 19 insertions(+), 4 deletions(-) create mode 100644 changelogs/fragments/get_primary_show_binary_log_status.yml diff --git a/changelogs/fragments/get_primary_show_binary_log_status.yml b/changelogs/fragments/get_primary_show_binary_log_status.yml new file mode 100644 index 0000000..8757aa1 --- /dev/null +++ b/changelogs/fragments/get_primary_show_binary_log_status.yml @@ -0,0 +1,4 @@ +--- +minor_changes: + + - mysql_replication - Adds support for `SHOW BINARY LOG STATUS` and `SHOW BINLOG STATUS` on getprimary mode. diff --git a/plugins/modules/mysql_replication.py b/plugins/modules/mysql_replication.py index 23c94c1..4f668f2 100644 --- a/plugins/modules/mysql_replication.py +++ b/plugins/modules/mysql_replication.py @@ -297,8 +297,11 @@ queries: import os import warnings +from ansible_collections.community.mysql.plugins.module_utils.version import LooseVersion from ansible.module_utils.basic import AnsibleModule from ansible_collections.community.mysql.plugins.module_utils.mysql import ( + get_server_version, + get_server_implementation, mysql_connect, mysql_driver, mysql_driver_fail_msg, @@ -310,10 +313,18 @@ executed_queries = [] def get_primary_status(cursor): - # TODO: when it's available to change on MySQL's side, - # change MASTER to PRIMARY using the approach from - # get_replica_status() function. Same for other functions. - cursor.execute("SHOW MASTER STATUS") + term = "MASTER" + + version = get_server_version(cursor) + server_implementation = get_server_implementation(cursor) + if server_implementation == "mysql" and LooseVersion(version) >= LooseVersion("8.2.0"): + term = "BINARY LOG" + + if server_implementation == "mariadb" and LooseVersion(version) >= LooseVersion("10.5.2"): + term = "BINLOG" + + cursor.execute("SHOW %s STATUS" % term) + primarystatus = cursor.fetchone() return primarystatus From 6c4dca4bceda609810a5138bc5496a13359bba8d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Robert=20Sil=C3=A9n?= Date: Fri, 31 May 2024 10:14:43 +0300 Subject: [PATCH 09/54] mention MariaDB (#640) * mention MariaDB * mention MariaDB in descriptions and notes * nits * chmod -x --- README.md | 2 +- changelogs/config.yaml | 2 +- galaxy.yml | 2 +- plugins/modules/mysql_db.py | 5 +++-- plugins/modules/mysql_info.py | 5 +++-- plugins/modules/mysql_query.py | 6 ++++-- plugins/modules/mysql_replication.py | 7 ++++--- plugins/modules/mysql_role.py | 5 +++-- plugins/modules/mysql_user.py | 5 +++-- plugins/modules/mysql_variables.py | 7 +++++-- 10 files changed, 28 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index 9853569..07af184 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# MySQL collection for Ansible +# MySQL and MariaDB collection for Ansible [![Plugins CI](https://github.com/ansible-collections/community.mysql/workflows/Plugins%20CI/badge.svg?event=push)](https://github.com/ansible-collections/community.mysql/actions?query=workflow%3A"Plugins+CI") [![Roles CI](https://github.com/ansible-collections/community.mysql/workflows/Roles%20CI/badge.svg?event=push)](https://github.com/ansible-collections/community.mysql/actions?query=workflow%3A"Roles+CI") [![Codecov](https://img.shields.io/codecov/c/github/ansible-collections/community.mysql)](https://codecov.io/gh/ansible-collections/community.mysql) [![Discuss on Matrix at #mysql:ansible.com](https://img.shields.io/matrix/mysql:ansible.com.svg?server_fqdn=ansible-accounts.ems.host&label=Discuss%20on%20Matrix%20at%20%23mysql:ansible.com&logo=matrix)](https://matrix.to/#/#mysql:ansible.com) This collection is a part of the Ansible package. diff --git a/changelogs/config.yaml b/changelogs/config.yaml index 70ab036..40ac5f8 100644 --- a/changelogs/config.yaml +++ b/changelogs/config.yaml @@ -25,5 +25,5 @@ sections: - Bugfixes - - known_issues - Known Issues -title: Community MySQL Collection +title: Community MySQL and MariaDB Collection trivial_section_name: trivial diff --git a/galaxy.yml b/galaxy.yml index dca1e28..512c668 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -5,7 +5,7 @@ version: 3.9.0 readme: README.md authors: - Ansible community -description: MySQL collection for Ansible +description: MySQL and MariaDB collection for Ansible license_file: COPYING tags: - database diff --git a/plugins/modules/mysql_db.py b/plugins/modules/mysql_db.py index 2cb67dc..8742f3c 100644 --- a/plugins/modules/mysql_db.py +++ b/plugins/modules/mysql_db.py @@ -11,9 +11,9 @@ __metaclass__ = type DOCUMENTATION = r''' --- module: mysql_db -short_description: Add or remove MySQL databases from a remote host +short_description: Add or remove MySQL or MariaDB databases from a remote host description: -- Add or remove MySQL databases from a remote host. +- Add or remove MySQL or MariaDB databases from a remote host. options: name: description: @@ -188,6 +188,7 @@ requirements: - mysql (command line binary) - mysqldump (command line binary) notes: + - Compatible with MariaDB or MySQL. - Requires the mysql and mysqldump binaries on the remote host. - This module is B(not idempotent) when I(state) is C(import), and will import the dump file each time if run more than once. diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index f30f1a1..c119b8d 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -11,9 +11,9 @@ __metaclass__ = type DOCUMENTATION = r''' --- module: mysql_info -short_description: Gather information about MySQL servers +short_description: Gather information about MySQL or MariaDB servers description: -- Gathers information about MySQL servers. +- Gathers information about MySQL or MariaDB servers. options: filter: @@ -46,6 +46,7 @@ options: default: false notes: +- Compatible with MariaDB or MySQL. - Calculating the size of a database might be slow, depending on the number and size of tables in it. To avoid this, use I(exclude_fields=db_size). diff --git a/plugins/modules/mysql_query.py b/plugins/modules/mysql_query.py index fd3a8e0..13a07de 100644 --- a/plugins/modules/mysql_query.py +++ b/plugins/modules/mysql_query.py @@ -10,9 +10,9 @@ __metaclass__ = type DOCUMENTATION = r''' --- module: mysql_query -short_description: Run MySQL queries +short_description: Run MySQL or MariaDB queries description: -- Runs arbitrary MySQL queries. +- Runs arbitrary MySQL or MariaDB queries. - Pay attention, the module does not support check mode! All queries will be executed in autocommit mode. - To run SQL queries from a file, use M(community.mysql.mysql_db) module. @@ -56,6 +56,8 @@ attributes: support: none seealso: - module: community.mysql.mysql_db +notes: +- Compatible with MariaDB or MySQL. author: - Andrew Klychkov (@Andersson007) extends_documentation_fragment: diff --git a/plugins/modules/mysql_replication.py b/plugins/modules/mysql_replication.py index 4f668f2..b0caf11 100644 --- a/plugins/modules/mysql_replication.py +++ b/plugins/modules/mysql_replication.py @@ -13,9 +13,9 @@ __metaclass__ = type DOCUMENTATION = r''' --- module: mysql_replication -short_description: Manage MySQL replication +short_description: Manage MySQL or MariaDB replication description: -- Manages MySQL server replication, replica, primary status, get and change primary host. +- Manages MySQL or MariaDB server replication, replica, primary status, get and change primary host. author: - Balazs Pocze (@banyek) - Andrew Klychkov (@Andersson007) @@ -191,7 +191,8 @@ options: version_added: '0.1.0' notes: -- If an empty value for the parameter of string type is needed, use an empty string. + - Compatible with MariaDB or MySQL. + - If an empty value for the parameter of string type is needed, use an empty string. attributes: check_mode: diff --git a/plugins/modules/mysql_role.py b/plugins/modules/mysql_role.py index 65ed894..df8b5fe 100644 --- a/plugins/modules/mysql_role.py +++ b/plugins/modules/mysql_role.py @@ -11,10 +11,10 @@ DOCUMENTATION = r''' --- module: mysql_role -short_description: Adds, removes, or updates a MySQL role +short_description: Adds, removes, or updates a MySQL or MariaDB role description: - - Adds, removes, or updates a MySQL role. + - Adds, removes, or updates a MySQL or MariaDB role. - Roles are supported since MySQL 8.0.0 and MariaDB 10.0.5. version_added: '2.2.0' @@ -132,6 +132,7 @@ options: version_added: '3.8.0' notes: + - Roles are supported since MySQL 8.0.0 and MariaDB 10.0.5. - Pay attention that the module runs C(SET DEFAULT ROLE ALL TO) all the I(members) passed by default when the state has changed. If you want to avoid this behavior, set I(set_default_role_all) to C(no). diff --git a/plugins/modules/mysql_user.py b/plugins/modules/mysql_user.py index fa54c7d..55e34a3 100644 --- a/plugins/modules/mysql_user.py +++ b/plugins/modules/mysql_user.py @@ -11,9 +11,9 @@ __metaclass__ = type DOCUMENTATION = r''' --- module: mysql_user -short_description: Adds or removes a user from a MySQL database +short_description: Adds or removes a user from a MySQL or MariaDB database description: - - Adds or removes a user from a MySQL database. + - Adds or removes a user from a MySQL or MariaDB database. options: name: description: @@ -188,6 +188,7 @@ options: version_added: '3.9.0' notes: + - Compatible with MySQL or MariaDB. - "MySQL server installs with default I(login_user) of C(root) and no password. To secure this user as part of an idempotent playbook, you must create at least two tasks: 1) change the root user's password, without providing any I(login_user)/I(login_password) details, diff --git a/plugins/modules/mysql_variables.py b/plugins/modules/mysql_variables.py index dfe8466..f912a27 100644 --- a/plugins/modules/mysql_variables.py +++ b/plugins/modules/mysql_variables.py @@ -12,9 +12,9 @@ DOCUMENTATION = r''' --- module: mysql_variables -short_description: Manage MySQL global variables +short_description: Manage MySQL or MariaDB global variables description: -- Query / Set MySQL variables. +- Query / Set MySQL or MariaDB variables. author: - Balazs Pocze (@banyek) options: @@ -54,6 +54,9 @@ seealso: description: Complete reference of the MySQL SET command documentation. link: https://dev.mysql.com/doc/refman/8.0/en/set-statement.html +notes: + - Compatible with MariaDB or MySQL. + extends_documentation_fragment: - community.mysql.mysql ''' From 50e7413b88477c333800fc6fa9f8053e493b2469 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Thu, 6 Jun 2024 13:05:31 +0200 Subject: [PATCH 10/54] Fix hashed passwords being returned by get_existing_authentication() via the plugin_auth_string variable instead of plugin_hash_string (#629) * fix returned variable from plugin_auth_string to plugin_hash_string * Refactor to keep plugin_auth_string in addition to plugin_hash_string * Add breaking_changes to the changelog --- .../lie_fix_plugin_hash_string_return.yml | 6 ++ plugins/module_utils/user.py | 14 +++- .../tasks/filter_users_info.yml | 72 +++++-------------- 3 files changed, 36 insertions(+), 56 deletions(-) create mode 100644 changelogs/fragments/lie_fix_plugin_hash_string_return.yml diff --git a/changelogs/fragments/lie_fix_plugin_hash_string_return.yml b/changelogs/fragments/lie_fix_plugin_hash_string_return.yml new file mode 100644 index 0000000..e1a71ea --- /dev/null +++ b/changelogs/fragments/lie_fix_plugin_hash_string_return.yml @@ -0,0 +1,6 @@ +--- +bugfixes: + - mysql_info - Add ``plugin_hash_string`` to ``users_info`` filter's output. The existing ``plugin_auth_string`` contained the hashed password and thus is missleading, it will be removed from community.mysql 4.0.0. (https://github.com/ansible-collections/community.mysql/pull/629). + +breaking_changes: + - mysql_info - The ``users_info`` filter returned variable ``plugin_auth_string`` contains the hashed password and it's misleading, it will be removed from community.mysql 4.0.0. Use the `plugin_hash_string` return value instead (https://github.com/ansible-collections/community.mysql/pull/629). diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index d4ae9dd..25b1734 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -118,11 +118,19 @@ def get_existing_authentication(cursor, user, host): if isinstance(rows, dict): rows = list(rows.values()) + # 'plugin_auth_string' contains the hash string. Must be removed in c.mysql 4.0 + # See https://github.com/ansible-collections/community.mysql/pull/629 if isinstance(rows[0], tuple): - return {'plugin': rows[0][0], 'plugin_auth_string': rows[0][1]} + return {'plugin': rows[0][0], + 'plugin_auth_string': rows[0][1], + 'plugin_hash_string': rows[0][1]} + # 'plugin_auth_string' contains the hash string. Must be removed in c.mysql 4.0 + # See https://github.com/ansible-collections/community.mysql/pull/629 if isinstance(rows[0], dict): - return {'plugin': rows[0].get('plugin'), 'plugin_auth_string': rows[0].get('auth')} + return {'plugin': rows[0].get('plugin'), + 'plugin_auth_string': rows[0].get('auth'), + 'plugin_hash_string': rows[0].get('auth')} return None @@ -152,7 +160,7 @@ def user_add(cursor, user, host, host_all, password, encrypted, existing_auth = get_existing_authentication(cursor, user, host) if existing_auth: plugin = existing_auth['plugin'] - plugin_hash_string = existing_auth['auth_string'] + plugin_hash_string = existing_auth['plugin_hash_string'] password = None used_existing_password = True if password and encrypted: diff --git a/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml b/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml index 63ce190..36508f3 100644 --- a/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml +++ b/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml @@ -211,66 +211,32 @@ TO users_info_tls_sub_issu_ciph@'host' - name: Mysql_info users_info | Prepare tests users for MariaDB - community.mysql.mysql_user: - name: "{{ item.name }}" - host: "users_info.com" - plugin: "{{ item.plugin | default(omit) }}" - plugin_auth_string: "{{ item.plugin_auth_string | default(omit) }}" - plugin_hash_string: "{{ item.plugin_hash_string | default(omit) }}" - tls_requires: "{{ item.tls_requires | default(omit) }}" - priv: "{{ item.priv }}" - resource_limits: "{{ item.resource_limits | default(omit) }}" - column_case_sensitive: true - state: present - loop: - - name: users_info_socket # Only for MariaDB - priv: - '*.*': 'ALL' - plugin: 'unix_socket' + community.mysql.mysql_query: + query: + - >- + CREATE USER users_info_socket@'users_info.com' IDENTIFIED WITH + unix_socket + - GRANT ALL ON *.* to users_info_socket@'users_info.com' when: - db_engine == 'mariadb' - name: Mysql_info users_info | Prepare tests users for MySQL - community.mysql.mysql_user: - name: "{{ item.name }}" - host: "users_info.com" - plugin: "{{ item.plugin | default(omit) }}" - plugin_auth_string: "{{ item.plugin_auth_string | default(omit) }}" - plugin_hash_string: "{{ item.plugin_hash_string | default(omit) }}" - tls_requires: "{{ item.tls_requires | default(omit) }}" - priv: "{{ item.priv }}" - resource_limits: "{{ item.resource_limits | default(omit) }}" - column_case_sensitive: true - state: present - loop: - - name: users_info_sha256 # Only for MySQL - priv: - '*.*': 'ALL' - plugin_auth_string: - '$5$/- + CREATE USER users_info_sha256@'users_info.com' IDENTIFIED WITH + sha256_password BY 'msandbox' + - GRANT ALL ON *.* to users_info_sha256@'users_info.com' when: - db_engine == 'mysql' - name: Mysql_info users_info | Prepare tests users for MySQL 8+ - community.mysql.mysql_user: - name: "{{ item.name }}" - host: "users_info.com" - plugin: "{{ item.plugin | default(omit) }}" - plugin_auth_string: "{{ item.plugin_auth_string | default(omit) }}" - plugin_hash_string: "{{ item.plugin_hash_string | default(omit) }}" - tls_requires: "{{ item.tls_requires | default(omit) }}" - priv: "{{ item.priv }}" - resource_limits: "{{ item.resource_limits | default(omit) }}" - column_case_sensitive: true - state: present - loop: - - name: users_info_caching_sha2 # Only for MySQL 8+ - priv: - '*.*': 'ALL' - plugin_auth_string: - '$A$005$61j/uF%Qb4-=O2xkeO82u2HNkF.lxDq0liO4U3xqi7bDUCbWM6HayRXWn1' - plugin: 'caching_sha2_password' + community.mysql.mysql_query: + query: + - >- + CREATE USER users_info_caching_sha2@'users_info.com' IDENTIFIED WITH + caching_sha2_password BY 'msandbox' + - GRANT ALL ON *.* to users_info_caching_sha2@'users_info.com' when: - db_engine == 'mysql' - db_version is version('8.0', '>=') @@ -283,7 +249,7 @@ - users_info register: result - - name: Recreate users from mysql_info users_info result + - name: Mysql_info users_info | Recreate users from mysql_info result community.mysql.mysql_user: name: "{{ item.name }}" host: "{{ item.host }}" From 0bc3e3d848f8e3714ec2e6a7748ab1b85660e216 Mon Sep 17 00:00:00 2001 From: Matthieu Bourgain Date: Tue, 11 Jun 2024 17:23:05 +0200 Subject: [PATCH 11/54] Add salt parameter to hash generation for sha256 plugins (#631) * add salt parameter to hash generation for sha256 plugin * technomax review modification * no general user test for salt --- .../add_salt_param_to_gen_sha256_hash.yml | 3 + .../implementations/mysql/hash.py | 125 ++++++++++++++++++ plugins/module_utils/user.py | 28 +++- plugins/modules/mysql_role.py | 2 +- plugins/modules/mysql_user.py | 31 ++++- .../tasks/test_user_plugin_auth.yml | 69 ++++++++++ 6 files changed, 251 insertions(+), 7 deletions(-) create mode 100644 changelogs/fragments/add_salt_param_to_gen_sha256_hash.yml create mode 100644 plugins/module_utils/implementations/mysql/hash.py diff --git a/changelogs/fragments/add_salt_param_to_gen_sha256_hash.yml b/changelogs/fragments/add_salt_param_to_gen_sha256_hash.yml new file mode 100644 index 0000000..c49ba1d --- /dev/null +++ b/changelogs/fragments/add_salt_param_to_gen_sha256_hash.yml @@ -0,0 +1,3 @@ +--- +minor_changes: + - mysql_user - Add salt parameter to generate static hash for `caching_sha2_password` and `sha256_password` plugins. diff --git a/plugins/module_utils/implementations/mysql/hash.py b/plugins/module_utils/implementations/mysql/hash.py new file mode 100644 index 0000000..0068a0c --- /dev/null +++ b/plugins/module_utils/implementations/mysql/hash.py @@ -0,0 +1,125 @@ +""" +Generate MySQL sha256 compatible plugins hash for a given password and salt + +based on + * https://www.akkadia.org/drepper/SHA-crypt.txt + * https://crypto.stackexchange.com/questions/77427/whats-the-algorithm-behind-mysqls-sha256-password-hashing-scheme/111174#111174 + * https://github.com/hashcat/hashcat/blob/master/tools/test_modules/m07400.pm +""" + +from __future__ import absolute_import, division, print_function + +__metaclass__ = type + +import hashlib + + +def _to64(v, n): + """Convert a 32-bit integer to a base-64 string""" + i64 = ( + [".", "/"] + + [chr(x) for x in range(48, 58)] + + [chr(x) for x in range(65, 91)] + + [chr(x) for x in range(97, 123)] + ) + result = "" + while n > 0: + n -= 1 + result += i64[v & 0x3F] + v >>= 6 + return result + + +def _hashlib_sha256(data): + """Return SHA-256 digest from hashlib .""" + return hashlib.sha256(data).digest() + + +def _sha256_digest(key, salt, loops): + """Return a SHA-256 digest of the concatenation of the key, the salt, and the key, repeated as necessary.""" + # https://www.akkadia.org/drepper/SHA-crypt.txt + num_bytes = 32 + bytes_key = key.encode() + bytes_salt = salt.encode() + digest_b = _hashlib_sha256(bytes_key + bytes_salt + bytes_key) + + tmp = bytes_key + bytes_salt + for i in range(len(bytes_key), 0, -num_bytes): + tmp += digest_b if i > num_bytes else digest_b[:i] + + i = len(bytes_key) + while i > 0: + tmp += digest_b if (i & 1) != 0 else bytes_key + i >>= 1 + + digest_a = _hashlib_sha256(tmp) + + tmp = b"" + for i in range(len(bytes_key)): + tmp += bytes_key + + digest_dp = _hashlib_sha256(tmp) + + byte_sequence_p = b"" + for i in range(len(bytes_key), 0, -num_bytes): + byte_sequence_p += digest_dp if i > num_bytes else digest_dp[:i] + + tmp = b"" + til = 16 + digest_a[0] + + for i in range(til): + tmp += bytes_salt + + digest_ds = _hashlib_sha256(tmp) + + byte_sequence_s = b"" + for i in range(len(bytes_salt), 0, -num_bytes): + byte_sequence_s += digest_ds if i > num_bytes else digest_ds[:i] + + digest_c = digest_a + + for i in range(loops): + tmp = byte_sequence_p if (i & 1) else digest_c + if i % 3: + tmp += byte_sequence_s + if i % 7: + tmp += byte_sequence_p + tmp += digest_c if (i & 1) else byte_sequence_p + digest_c = _hashlib_sha256(tmp) + + inc1, inc2, mod, end = (10, 21, 30, 0) + + i = 0 + tmp = "" + + while True: + tmp += _to64( + (digest_c[i] << 16) + | (digest_c[(i + inc1) % mod] << 8) + | digest_c[(i + inc1 * 2) % mod], + 4, + ) + i = (i + inc2) % mod + if i == end: + break + + tmp += _to64((digest_c[31] << 8) | digest_c[30], 3) + + return tmp + + +def mysql_sha256_password_hash(password, salt): + """Return a MySQL compatible caching_sha2_password hash in raw format.""" + if len(salt) != 20: + raise ValueError("Salt must be 20 characters long.") + + count = 5 + iteration = 1000 * count + + digest = _sha256_digest(password, salt, iteration) + return "$A${0:>03}${1}{2}".format(count, salt, digest) + + +def mysql_sha256_password_hash_hex(password, salt): + """Return a MySQL compatible caching_sha2_password hash in hex format.""" + return mysql_sha256_password_hash(password, salt).encode().hex().upper() diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index 25b1734..80da47e 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -1,4 +1,6 @@ from __future__ import (absolute_import, division, print_function) + + __metaclass__ = type # This code is part of Ansible, but is an independent component. @@ -19,6 +21,10 @@ from ansible_collections.community.mysql.plugins.module_utils.mysql import ( mysql_driver, get_server_implementation, ) +from ansible_collections.community.mysql.plugins.module_utils.implementations.mysql.hash import ( + mysql_sha256_password_hash, + mysql_sha256_password_hash_hex, +) class InvalidPrivsError(Exception): @@ -135,7 +141,7 @@ def get_existing_authentication(cursor, user, host): def user_add(cursor, user, host, host_all, password, encrypted, - plugin, plugin_hash_string, plugin_auth_string, new_priv, + plugin, plugin_hash_string, plugin_auth_string, salt, new_priv, attributes, tls_requires, reuse_existing_password, module, password_expire, password_expire_interval): # If attributes are set, perform a sanity check to ensure server supports user attributes before creating user @@ -181,6 +187,12 @@ def user_add(cursor, user, host, host_all, password, encrypted, # Mysql and MariaDB differ in naming pam plugin and Syntax to set it if plugin == 'pam': # Used by MariaDB which requires the USING keyword, not BY query_with_args = "CREATE USER %s@%s IDENTIFIED WITH %s USING %s", (user, host, plugin, plugin_auth_string) + elif salt: + if plugin in ['caching_sha2_password', 'sha256_password']: + generated_hash_string = mysql_sha256_password_hash_hex(password=plugin_auth_string, salt=salt) + else: + module.fail_json(msg="salt not handled for %s authentication plugin" % plugin) + query_with_args = ("CREATE USER %s@%s IDENTIFIED WITH %s AS 0x" + generated_hash_string), (user, host, plugin) else: query_with_args = "CREATE USER %s@%s IDENTIFIED WITH %s BY %s", (user, host, plugin, plugin_auth_string) elif plugin: @@ -221,7 +233,7 @@ def is_hash(password): def user_mod(cursor, user, host, host_all, password, encrypted, - plugin, plugin_hash_string, plugin_auth_string, new_priv, + plugin, plugin_hash_string, plugin_auth_string, salt, new_priv, append_privs, subtract_privs, attributes, tls_requires, module, password_expire, password_expire_interval, role=False, maria_role=False): changed = False @@ -342,7 +354,11 @@ def user_mod(cursor, user, host, host_all, password, encrypted, if plugin_hash_string and current_plugin[1] != plugin_hash_string: update = True - if plugin_auth_string and current_plugin[1] != plugin_auth_string: + if salt: + if plugin in ['caching_sha2_password', 'sha256_password']: + if current_plugin[1] != mysql_sha256_password_hash(password=plugin_auth_string, salt=salt): + update = True + elif plugin_auth_string and current_plugin[1] != plugin_auth_string: # this case can cause more updates than expected, # as plugin can hash auth_string in any way it wants # and there's no way to figure it out for @@ -356,6 +372,12 @@ def user_mod(cursor, user, host, host_all, password, encrypted, # Mysql and MariaDB differ in naming pam plugin and syntax to set it if plugin in ('pam', 'ed25519'): query_with_args = "ALTER USER %s@%s IDENTIFIED WITH %s USING %s", (user, host, plugin, plugin_auth_string) + elif salt: + if plugin in ['caching_sha2_password', 'sha256_password']: + generated_hash_string = mysql_sha256_password_hash_hex(password=plugin_auth_string, salt=salt) + else: + module.fail_json(msg="salt not handled for %s authentication plugin" % plugin) + query_with_args = ("ALTER USER %s@%s IDENTIFIED WITH %s AS 0x" + generated_hash_string), (user, host, plugin) else: query_with_args = "ALTER USER %s@%s IDENTIFIED WITH %s BY %s", (user, host, plugin, plugin_auth_string) else: diff --git a/plugins/modules/mysql_role.py b/plugins/modules/mysql_role.py index df8b5fe..032b41e 100644 --- a/plugins/modules/mysql_role.py +++ b/plugins/modules/mysql_role.py @@ -931,7 +931,7 @@ class Role(): if privs: result = user_mod(self.cursor, self.name, self.host, - None, None, None, None, None, None, + None, None, None, None, None, None, None, privs, append_privs, subtract_privs, None, None, self.module, None, None, role=True, maria_role=self.is_mariadb) diff --git a/plugins/modules/mysql_user.py b/plugins/modules/mysql_user.py index 55e34a3..0c7021b 100644 --- a/plugins/modules/mysql_user.py +++ b/plugins/modules/mysql_user.py @@ -139,8 +139,16 @@ options: description: - User's plugin auth_string (``CREATE USER user IDENTIFIED WITH plugin BY plugin_auth_string``). - If I(plugin) is ``pam`` (MariaDB) or ``auth_pam`` (MySQL) an optional I(plugin_auth_string) can be used to choose a specific PAM service. + - You need to define a I(salt) to have idempotence on password change with ``caching_sha2_password`` and ``sha256_password`` plugins. type: str version_added: '0.1.0' + salt: + description: + - Salt used to generate password hash from I(plugin_auth_string). + - Salt length must be 20 characters. + - Salt only support ``caching_sha2_password`` or ``sha256_password`` authentication I(plugin). + type: str + version_added: '3.10.0' resource_limits: description: - Limit the user for certain server resources. Provided since MySQL 5.6 / MariaDB 10.2. @@ -369,6 +377,13 @@ EXAMPLES = r''' priv: '*.*:ALL' state: present +- name: Create user 'bob' authenticated with plugin 'caching_sha2_password' and static salt + community.mysql.mysql_user: + name: bob + plugin: caching_sha2_password + plugin_auth_string: password + salt: 1234567890abcdefghij + - name: Limit bob's resources to 10 queries per hour and 5 connections per hour community.mysql.mysql_user: name: bob @@ -440,6 +455,7 @@ def main(): plugin=dict(default=None, type='str'), plugin_hash_string=dict(default=None, type='str'), plugin_auth_string=dict(default=None, type='str'), + salt=dict(default=None, type='str'), resource_limits=dict(type='dict'), force_context=dict(type='bool', default=False), session_vars=dict(type='dict'), @@ -480,6 +496,7 @@ def main(): plugin = module.params["plugin"] plugin_hash_string = module.params["plugin_hash_string"] plugin_auth_string = module.params["plugin_auth_string"] + salt = module.params["salt"] resource_limits = module.params["resource_limits"] session_vars = module.params["session_vars"] column_case_sensitive = module.params["column_case_sensitive"] @@ -499,6 +516,14 @@ def main(): module.fail_json(msg="password_expire_interval value \ should be positive number") + if salt: + if not plugin_auth_string: + module.fail_json(msg="salt requires plugin_auth_string") + if len(salt) != 20: + module.fail_json(msg="salt must be 20 characters long") + if plugin not in ['caching_sha2_password', 'sha256_password']: + module.fail_json(msg="salt requires caching_sha2_password or sha256_password plugin") + cursor = None try: if check_implicit_admin: @@ -542,13 +567,13 @@ def main(): try: if update_password == "always": result = user_mod(cursor, user, host, host_all, password, encrypted, - plugin, plugin_hash_string, plugin_auth_string, + plugin, plugin_hash_string, plugin_auth_string, salt, priv, append_privs, subtract_privs, attributes, tls_requires, module, password_expire, password_expire_interval) else: result = user_mod(cursor, user, host, host_all, None, encrypted, - None, None, None, + None, None, None, None, priv, append_privs, subtract_privs, attributes, tls_requires, module, password_expire, password_expire_interval) changed = result['changed'] @@ -566,7 +591,7 @@ def main(): priv = None # avoid granting unwanted privileges reuse_existing_password = update_password == 'on_new_username' result = user_add(cursor, user, host, host_all, password, encrypted, - plugin, plugin_hash_string, plugin_auth_string, + plugin, plugin_hash_string, plugin_auth_string, salt, priv, attributes, tls_requires, reuse_existing_password, module, password_expire, password_expire_interval) changed = result['changed'] diff --git a/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml b/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml index d8ff04d..b5ed6c5 100644 --- a/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml +++ b/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml @@ -13,6 +13,7 @@ test_plugin_auth_string: 'Fdt8fd^34ds' test_plugin_new_hash: '*E74368AC90460FA669F6D41BFB7F2A877DB73745' test_plugin_new_auth_string: 'c$K01LsmK7nJnIR4!h' + test_salt: 'TDwqdanU82d0yNtvaabb' test_default_priv_type: 'SELECT' test_default_priv: '*.*:{{ test_default_priv_type }}' @@ -475,3 +476,71 @@ - include_tasks: utils/remove_user.yml vars: user_name: "{{ test_user_name }}" + + # ============================================================ + # Test plugin auth with a salt + # + - name: Plugin auth | Create user with plugin auth and salt + community.mysql.mysql_user: + <<: *mysql_params + name: "{{ test_user_name }}" + host: "%" + plugin: caching_sha2_password + plugin_auth_string: "{{ test_plugin_auth_string }}" + salt: "{{ test_salt }}" + priv: "{{ test_default_priv }}" + + - name: Plugin auth | Connect with user and password + ansible.builtin.command: '{{ mysql_command }} -u {{ test_user_name }} -p{{ test_plugin_auth_string }} -e "SELECT 1"' + + - name: Plugin auth | Alter user with same plugin auth and same salt + community.mysql.mysql_user: + <<: *mysql_params + name: "{{ test_user_name }}" + host: "%" + plugin: caching_sha2_password + plugin_auth_string: "{{ test_plugin_auth_string }}" + salt: "{{ test_salt }}" + priv: "{{ test_default_priv }}" + register: result + failed_when: result is changed + + - name: cleanup user + ansible.builtin.include_tasks: utils/remove_user.yml + vars: + user_name: "{{ test_user_name }}" + + - name: Plugin auth | Create user with too short salt (should fail) + community.mysql.mysql_user: + <<: *mysql_params + name: "{{ test_user_name }}" + host: "%" + plugin: caching_sha2_password + plugin_auth_string: "{{ test_plugin_auth_string }}" + salt: "1234567890az" + priv: "{{ test_default_priv }}" + register: result + failed_when: result is success + + - name: Plugin auth | Create user with salt and no plugin auth string (should fail) + community.mysql.mysql_user: + <<: *mysql_params + name: "{{ test_user_name }}" + host: "%" + plugin: caching_sha2_password + salt: "{{ test_salt }}" + priv: "{{ test_default_priv }}" + register: result + failed_when: result is success + + - name: Plugin auth | Create user with salt and plugin not handled by internal hash generation (should fail) + community.mysql.mysql_user: + <<: *mysql_params + name: "{{ test_user_name }}" + host: "%" + plugin: mysql_native_password + plugin_auth_string: "{{ test_plugin_auth_string }}" + salt: "{{ test_salt }}" + priv: "{{ test_default_priv }}" + register: result + failed_when: result is success From f266ba59c943e9912d65c3e568727b444df49771 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Wed, 19 Jun 2024 10:17:02 +0200 Subject: [PATCH 12/54] mysql_info: add server_engine return value (#649) * mysql_info: add server_engine return value * Incorporate feedback --- changelogs/fragments/1-mysql_info.yml | 2 ++ plugins/modules/mysql_info.py | 7 +++++++ tests/integration/targets/test_mysql_info/tasks/main.yml | 1 + 3 files changed, 10 insertions(+) create mode 100644 changelogs/fragments/1-mysql_info.yml diff --git a/changelogs/fragments/1-mysql_info.yml b/changelogs/fragments/1-mysql_info.yml new file mode 100644 index 0000000..1ab4d2c --- /dev/null +++ b/changelogs/fragments/1-mysql_info.yml @@ -0,0 +1,2 @@ +minor_changes: +- mysql_info - return a database server engine used (https://github.com/ansible-collections/community.mysql/issues/644). diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index c119b8d..6103589 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -162,6 +162,12 @@ EXAMPLES = r''' ''' RETURN = r''' +server_engine: + description: Database server engine. + returned: if not excluded by filter + type: str + sample: 'MariaDB' + version_added: '3.10.0' version: description: Database server version. returned: if not excluded by filter @@ -765,6 +771,7 @@ def main(): mysql = MySQL_Info(module, cursor, server_implementation, user_implementation) module.exit_json(changed=False, + server_engine='MariaDB' if server_implementation == 'mariadb' else 'MySQL', connector_name=connector_name, connector_version=connector_version, **mysql.get_info(filter_, exclude_fields, return_empty_dbs)) diff --git a/tests/integration/targets/test_mysql_info/tasks/main.yml b/tests/integration/targets/test_mysql_info/tasks/main.yml index 5d34da9..93570f2 100644 --- a/tests/integration/targets/test_mysql_info/tasks/main.yml +++ b/tests/integration/targets/test_mysql_info/tasks/main.yml @@ -56,6 +56,7 @@ - result.databases != {} - result.engines != {} - result.users != {} + - result.server_engine == 'MariaDB' or result.server_engine == 'MySQL' - name: mysql_info - Test connector informations display ansible.builtin.import_tasks: From aafe658a85d67cd6c4c23dd0b84acf86ad698da4 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Wed, 19 Jun 2024 10:20:34 +0200 Subject: [PATCH 13/54] Update README.md (#648) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 07af184..2678f31 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ # MySQL and MariaDB collection for Ansible -[![Plugins CI](https://github.com/ansible-collections/community.mysql/workflows/Plugins%20CI/badge.svg?event=push)](https://github.com/ansible-collections/community.mysql/actions?query=workflow%3A"Plugins+CI") [![Roles CI](https://github.com/ansible-collections/community.mysql/workflows/Roles%20CI/badge.svg?event=push)](https://github.com/ansible-collections/community.mysql/actions?query=workflow%3A"Roles+CI") [![Codecov](https://img.shields.io/codecov/c/github/ansible-collections/community.mysql)](https://codecov.io/gh/ansible-collections/community.mysql) [![Discuss on Matrix at #mysql:ansible.com](https://img.shields.io/matrix/mysql:ansible.com.svg?server_fqdn=ansible-accounts.ems.host&label=Discuss%20on%20Matrix%20at%20%23mysql:ansible.com&logo=matrix)](https://matrix.to/#/#mysql:ansible.com) +[![Plugins CI](https://github.com/ansible-collections/community.mysql/workflows/Plugins%20CI/badge.svg?event=push)](https://github.com/ansible-collections/community.mysql/actions?query=workflow%3A"Plugins+CI") [![Codecov](https://img.shields.io/codecov/c/github/ansible-collections/community.mysql)](https://codecov.io/gh/ansible-collections/community.mysql) [![Discuss on Matrix at #mysql:ansible.com](https://img.shields.io/matrix/mysql:ansible.com.svg?server_fqdn=ansible-accounts.ems.host&label=Discuss%20on%20Matrix%20at%20%23mysql:ansible.com&logo=matrix)](https://matrix.to/#/#mysql:ansible.com) This collection is a part of the Ansible package. From 1922e7154e6228100c022d3e7350d12f23eb7d54 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Mon, 24 Jun 2024 09:36:32 +0200 Subject: [PATCH 14/54] [CI] Remove ansible-test custom containers (#650) * Cut tests containers * Cut unused flatten versions * Fix installation of mysqlclient on Ubuntu * Cut unused variables * Fix package missing on Unbuntu 22.04 * Fix variable templating * Fix test for ansible 2.17 and do remove the ignore_errors ignore_errors is bad because it makes searching for real errors difficult. --- .github/workflows/ansible-test-plugins.yml | 32 --------- .github/workflows/build-docker-image.yml | 67 ------------------- ...ker-image-mariadb-py310-mysqlclient211.yml | 21 ------ .../docker-image-mariadb-py310-pymysql102.yml | 21 ------ ...cker-image-mariadb-py38-mysqlclient201.yml | 21 ------ .../docker-image-mariadb-py38-pymysql093.yml | 21 ------ ...cker-image-mariadb-py39-mysqlclient203.yml | 21 ------ .../docker-image-mariadb-py39-pymysql093.yml | 21 ------ .../docker-image-my57-py38-mysqlclient201.yml | 21 ------ .../docker-image-my57-py38-pymysql0711.yml | 21 ------ .../docker-image-my57-py38-pymysql093.yml | 21 ------ ...ocker-image-mysql-py310-mysqlclient211.yml | 21 ------ .../docker-image-mysql-py310-pymysql102.yml | 21 ------ ...docker-image-mysql-py38-mysqlclient201.yml | 21 ------ .../docker-image-mysql-py38-pymysql093.yml | 21 ------ ...docker-image-mysql-py39-mysqlclient203.yml | 21 ------ .../docker-image-mysql-py39-pymysql093.yml | 21 ------ Makefile | 22 +----- TESTING.md | 21 ++---- .../mariadb-py310-mysqlclient211/Dockerfile | 21 ------ .../mariadb-py310-pymysql102/Dockerfile | 15 ----- .../mariadb-py38-mysqlclient201/Dockerfile | 21 ------ .../mariadb-py38-pymysql093/Dockerfile | 15 ----- .../mariadb-py39-mysqlclient203/Dockerfile | 21 ------ .../mariadb-py39-pymysql093/Dockerfile | 15 ----- .../my57-py38-mysqlclient201/Dockerfile | 21 ------ .../my57-py38-pymysql0711/Dockerfile | 21 ------ .../my57-py38-pymysql093/Dockerfile | 15 ----- .../mysql-py310-mysqlclient211/Dockerfile | 21 ------ .../mysql-py310-pymysql102/Dockerfile | 15 ----- .../mysql-py38-mysqlclient201/Dockerfile | 21 ------ .../mysql-py38-pymysql093/Dockerfile | 15 ----- .../mysql-py39-mysqlclient203/Dockerfile | 21 ------ .../mysql-py39-pymysql093/Dockerfile | 16 ----- .../targets/setup_controller/tasks/main.yml | 11 +-- .../setup_controller/tasks/requirements.yml | 20 ++++++ .../setup_controller/tasks/setvars.yml | 14 ++-- .../tasks/config_overrides_defaults.yml | 22 +++--- 38 files changed, 55 insertions(+), 743 deletions(-) delete mode 100644 .github/workflows/build-docker-image.yml delete mode 100644 .github/workflows/docker-image-mariadb-py310-mysqlclient211.yml delete mode 100644 .github/workflows/docker-image-mariadb-py310-pymysql102.yml delete mode 100644 .github/workflows/docker-image-mariadb-py38-mysqlclient201.yml delete mode 100644 .github/workflows/docker-image-mariadb-py38-pymysql093.yml delete mode 100644 .github/workflows/docker-image-mariadb-py39-mysqlclient203.yml delete mode 100644 .github/workflows/docker-image-mariadb-py39-pymysql093.yml delete mode 100644 .github/workflows/docker-image-my57-py38-mysqlclient201.yml delete mode 100644 .github/workflows/docker-image-my57-py38-pymysql0711.yml delete mode 100644 .github/workflows/docker-image-my57-py38-pymysql093.yml delete mode 100644 .github/workflows/docker-image-mysql-py310-mysqlclient211.yml delete mode 100644 .github/workflows/docker-image-mysql-py310-pymysql102.yml delete mode 100644 .github/workflows/docker-image-mysql-py38-mysqlclient201.yml delete mode 100644 .github/workflows/docker-image-mysql-py38-pymysql093.yml delete mode 100644 .github/workflows/docker-image-mysql-py39-mysqlclient203.yml delete mode 100644 .github/workflows/docker-image-mysql-py39-pymysql093.yml delete mode 100644 test-containers/mariadb-py310-mysqlclient211/Dockerfile delete mode 100644 test-containers/mariadb-py310-pymysql102/Dockerfile delete mode 100644 test-containers/mariadb-py38-mysqlclient201/Dockerfile delete mode 100644 test-containers/mariadb-py38-pymysql093/Dockerfile delete mode 100644 test-containers/mariadb-py39-mysqlclient203/Dockerfile delete mode 100644 test-containers/mariadb-py39-pymysql093/Dockerfile delete mode 100644 test-containers/my57-py38-mysqlclient201/Dockerfile delete mode 100644 test-containers/my57-py38-pymysql0711/Dockerfile delete mode 100644 test-containers/my57-py38-pymysql093/Dockerfile delete mode 100644 test-containers/mysql-py310-mysqlclient211/Dockerfile delete mode 100644 test-containers/mysql-py310-pymysql102/Dockerfile delete mode 100644 test-containers/mysql-py38-mysqlclient201/Dockerfile delete mode 100644 test-containers/mysql-py38-pymysql093/Dockerfile delete mode 100644 test-containers/mysql-py39-mysqlclient203/Dockerfile delete mode 100644 test-containers/mysql-py39-pymysql093/Dockerfile create mode 100644 tests/integration/targets/setup_controller/tasks/requirements.yml diff --git a/.github/workflows/ansible-test-plugins.yml b/.github/workflows/ansible-test-plugins.yml index 77da49e..f3f440e 100644 --- a/.github/workflows/ansible-test-plugins.yml +++ b/.github/workflows/ansible-test-plugins.yml @@ -252,37 +252,6 @@ jobs: ${{ job.services.db_primary.id }} | grep healthy && [[ "$SECONDS" -lt 120 ]]; do sleep 1; done - - name: Compute docker_image - Set python_version_flat - run: > - echo "python_version_flat=$(echo ${{ matrix.python }} - | tr -d '.')" >> $GITHUB_ENV - - - name: Compute docker_image - Set connector_version_flat - run: > - echo "connector_version_flat=$(echo ${{ matrix.connector_version }} - |tr -d .)" >> $GITHUB_ENV - - - name: Compute docker_image - Set db_engine_version_flat - run: > - echo "db_engine_version_flat=$(echo ${{ matrix.db_engine_version }} - | awk -F '.' '{print $1 $2}')" >> $GITHUB_ENV - - - name: Compute docker_image - Set db_client - run: > - if [[ ${{ env.db_engine_version_flat }} == 57 ]]; then - echo "db_client=my57" >> $GITHUB_ENV; - else - echo "db_client=$(echo ${{ matrix.db_engine_name }})" >> $GITHUB_ENV; - fi - - - name: Set docker_image - run: |- - echo "docker_image=ghcr.io/ansible-collections/community.mysql\ - /test-container-${{ env.db_client }}\ - -py${{ env.python_version_flat }}\ - -${{ matrix.connector_name }}${{ env.connector_version_flat }}\ - :latest" >> $GITHUB_ENV - - name: >- Perform integration testing against Ansible version ${{ matrix.ansible }} @@ -315,7 +284,6 @@ jobs: echo Setting Ansible version to "${{ matrix.ansible }}"...; echo -n "${{ matrix.ansible }}" > tests/integration/ansible - docker-image: ${{ env.docker_image }} target-python-version: ${{ matrix.python }} testing-type: integration diff --git a/.github/workflows/build-docker-image.yml b/.github/workflows/build-docker-image.yml deleted file mode 100644 index 0edd5ee..0000000 --- a/.github/workflows/build-docker-image.yml +++ /dev/null @@ -1,67 +0,0 @@ ---- -name: Build Docker Image for ansible-test - -on: # yamllint disable-line rule:truthy - workflow_call: - inputs: - registry: - required: true - type: string - image_name: - required: true - type: string - context: - required: true - type: string - -jobs: - - build: - - runs-on: ubuntu-latest - permissions: - contents: read - packages: write - - steps: - # Requirement to use 'context' in docker/build-push-action@v3 - - name: Checkout repository - uses: actions/checkout@v3 - - # https://github.com/docker/login-action - - name: Log into registry ${{ inputs.registry }} - uses: docker/login-action@v2 - with: - registry: ${{ inputs.registry }} - username: ${{ github.actor }} - password: ${{ secrets.GITHUB_TOKEN }} - - # https://github.com/docker/metadata-action - - name: Extract Docker metadata (tags, labels) - id: meta - uses: docker/metadata-action@v4 - with: - images: - "${{ inputs.registry }}\ - /${{ github.repository }}\ - /${{ inputs.image_name }}" - tags: latest - - # Setting up Docker Buildx with docker-container driver is required - # at the moment to be able to use a subdirectory with Git context - # - # https://github.com/docker/setup-buildx-action - - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v2 - - # https://github.com/docker/build-push-action - - name: Build and push Docker image with Buildx - id: build-and-push - uses: docker/build-push-action@v3 - with: - context: ${{ inputs.context }} - push: true - tags: ${{ steps.meta.outputs.tags }} - labels: ${{ steps.meta.outputs.labels }} - cache-from: type=gha - cache-to: type=gha,mode=max diff --git a/.github/workflows/docker-image-mariadb-py310-mysqlclient211.yml b/.github/workflows/docker-image-mariadb-py310-mysqlclient211.yml deleted file mode 100644 index 77286e6..0000000 --- a/.github/workflows/docker-image-mariadb-py310-mysqlclient211.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mariadb-py310-mysqlclient211 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mariadb-py310-mysqlclient211/**' - - '.github/workflows/docker-image-mariadb-py310-mysqlclient211.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mariadb-py310-mysqlclient211 - context: test-containers/mariadb-py310-mysqlclient211 diff --git a/.github/workflows/docker-image-mariadb-py310-pymysql102.yml b/.github/workflows/docker-image-mariadb-py310-pymysql102.yml deleted file mode 100644 index c7cdfd4..0000000 --- a/.github/workflows/docker-image-mariadb-py310-pymysql102.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mariadb-py310-pymysql102 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mariadb-py310-pymysql102/**' - - '.github/workflows/docker-image-mariadb-py310-pymysql102.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mariadb-py310-pymysql102 - context: test-containers/mariadb-py310-pymysql102 diff --git a/.github/workflows/docker-image-mariadb-py38-mysqlclient201.yml b/.github/workflows/docker-image-mariadb-py38-mysqlclient201.yml deleted file mode 100644 index b5b9bb3..0000000 --- a/.github/workflows/docker-image-mariadb-py38-mysqlclient201.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mariadb-py38-mysqlclient201 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mariadb-py38-mysqlclient201/**' - - '.github/workflows/docker-image-mariadb-py38-mysqlclient201.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mariadb-py38-mysqlclient201 - context: test-containers/mariadb-py38-mysqlclient201 diff --git a/.github/workflows/docker-image-mariadb-py38-pymysql093.yml b/.github/workflows/docker-image-mariadb-py38-pymysql093.yml deleted file mode 100644 index ae6df2e..0000000 --- a/.github/workflows/docker-image-mariadb-py38-pymysql093.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mariadb-py38-pymysql093 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mariadb-py38-pymysql093/**' - - '.github/workflows/docker-image-mariadb-py38-pymysql093.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mariadb-py38-pymysql093 - context: test-containers/mariadb-py38-pymysql093 diff --git a/.github/workflows/docker-image-mariadb-py39-mysqlclient203.yml b/.github/workflows/docker-image-mariadb-py39-mysqlclient203.yml deleted file mode 100644 index 4efeef1..0000000 --- a/.github/workflows/docker-image-mariadb-py39-mysqlclient203.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mariadb-py39-mysqlclient203 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mariadb-py39-mysqlclient203/**' - - '.github/workflows/docker-image-mariadb-py39-mysqlclient203.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mariadb-py39-mysqlclient203 - context: test-containers/mariadb-py39-mysqlclient203 diff --git a/.github/workflows/docker-image-mariadb-py39-pymysql093.yml b/.github/workflows/docker-image-mariadb-py39-pymysql093.yml deleted file mode 100644 index a3205fb..0000000 --- a/.github/workflows/docker-image-mariadb-py39-pymysql093.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mariadb-py39-pymysql093 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mariadb-py39-pymysql093/**' - - '.github/workflows/docker-image-mariadb-py39-pymysql093.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mariadb-py39-pymysql093 - context: test-containers/mariadb-py39-pymysql093 diff --git a/.github/workflows/docker-image-my57-py38-mysqlclient201.yml b/.github/workflows/docker-image-my57-py38-mysqlclient201.yml deleted file mode 100644 index b256a47..0000000 --- a/.github/workflows/docker-image-my57-py38-mysqlclient201.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI my57-py38-mysqlclient201 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/my57-py38-mysqlclient201/**' - - '.github/workflows/docker-image-my57-py38-mysqlclient201.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-my57-py38-mysqlclient201 - context: test-containers/my57-py38-mysqlclient201 diff --git a/.github/workflows/docker-image-my57-py38-pymysql0711.yml b/.github/workflows/docker-image-my57-py38-pymysql0711.yml deleted file mode 100644 index 0064729..0000000 --- a/.github/workflows/docker-image-my57-py38-pymysql0711.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI my57-py38-pymysql0711 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/my57-py38-pymysql0711/**' - - '.github/workflows/docker-image-my57-py38-pymysql0711.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-my57-py38-pymysql0711 - context: test-containers/my57-py38-pymysql0711 diff --git a/.github/workflows/docker-image-my57-py38-pymysql093.yml b/.github/workflows/docker-image-my57-py38-pymysql093.yml deleted file mode 100644 index 58c7fed..0000000 --- a/.github/workflows/docker-image-my57-py38-pymysql093.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI my57-py38-pymysql093 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/my57-py38-pymysql093/**' - - '.github/workflows/docker-image-my57-py38-pymysql093.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-my57-py38-pymysql093 - context: test-containers/my57-py38-pymysql093 diff --git a/.github/workflows/docker-image-mysql-py310-mysqlclient211.yml b/.github/workflows/docker-image-mysql-py310-mysqlclient211.yml deleted file mode 100644 index dcb846f..0000000 --- a/.github/workflows/docker-image-mysql-py310-mysqlclient211.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mysql-py310-mysqlclient211 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mysql-py310-mysqlclient211/**' - - '.github/workflows/docker-image-mysql-py310-mysqlclient211.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mysql-py310-mysqlclient211 - context: test-containers/mysql-py310-mysqlclient211 diff --git a/.github/workflows/docker-image-mysql-py310-pymysql102.yml b/.github/workflows/docker-image-mysql-py310-pymysql102.yml deleted file mode 100644 index 815b923..0000000 --- a/.github/workflows/docker-image-mysql-py310-pymysql102.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mysql-py310-pymysql102 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mysql-py310-pymysql102/**' - - '.github/workflows/docker-image-mysql-py310-pymysql102.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mysql-py310-pymysql102 - context: test-containers/mysql-py310-pymysql102 diff --git a/.github/workflows/docker-image-mysql-py38-mysqlclient201.yml b/.github/workflows/docker-image-mysql-py38-mysqlclient201.yml deleted file mode 100644 index 93359a4..0000000 --- a/.github/workflows/docker-image-mysql-py38-mysqlclient201.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mysql-py38-mysqlclient201 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mysql-py38-mysqlclient201/**' - - '.github/workflows/docker-image-mysql-py38-mysqlclient201.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mysql-py38-mysqlclient201 - context: test-containers/mysql-py38-mysqlclient201 diff --git a/.github/workflows/docker-image-mysql-py38-pymysql093.yml b/.github/workflows/docker-image-mysql-py38-pymysql093.yml deleted file mode 100644 index ac572ea..0000000 --- a/.github/workflows/docker-image-mysql-py38-pymysql093.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mysql-py38-pymysql093 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mysql-py38-pymysql093/**' - - '.github/workflows/docker-image-mysql-py38-pymysql093.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mysql-py38-pymysql093 - context: test-containers/mysql-py38-pymysql093 diff --git a/.github/workflows/docker-image-mysql-py39-mysqlclient203.yml b/.github/workflows/docker-image-mysql-py39-mysqlclient203.yml deleted file mode 100644 index b314e57..0000000 --- a/.github/workflows/docker-image-mysql-py39-mysqlclient203.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mysql-py39-mysqlclient203 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mysql-py39-mysqlclient203/**' - - '.github/workflows/docker-image-mysql-py39-mysqlclient203.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mysql-py39-mysqlclient203 - context: test-containers/mysql-py39-mysqlclient203 diff --git a/.github/workflows/docker-image-mysql-py39-pymysql093.yml b/.github/workflows/docker-image-mysql-py39-pymysql093.yml deleted file mode 100644 index 55962fb..0000000 --- a/.github/workflows/docker-image-mysql-py39-pymysql093.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -name: Docker Image CI mysql-py39-pymysql093 - -on: # yamllint disable-line rule:truthy - push: - paths: - - 'test-containers/mysql-py39-pymysql093/*' - - '.github/workflows/docker-image-mysql-py39-pymysql093.yml' - - '.github/workflows/build-docker-image.yml' - branches-ignore: - - stable-* - -jobs: - - call-workflow-passing-data: - uses: ./.github/workflows/build-docker-image.yml - secrets: inherit - with: - registry: ghcr.io - image_name: test-container-mysql-py39-pymysql093 - context: test-containers/mysql-py39-pymysql093 diff --git a/Makefile b/Makefile index 7ea0785..1bf8fae 100644 --- a/Makefile +++ b/Makefile @@ -11,23 +11,6 @@ ifdef continue_on_errors _continue_on_errors = --retry-on-error --continue-on-error endif - -db_ver_tuple := $(subst ., , $(db_engine_version)) -db_engine_version_flat := $(word 1, $(db_ver_tuple))$(word 2, $(db_ver_tuple)) - -con_ver_tuple := $(subst ., , $(connector_version)) -connector_version_flat := $(word 1, $(con_ver_tuple))$(word 2, $(con_ver_tuple))$(word 3, $(con_ver_tuple)) - -py_ver_tuple := $(subst ., , $(python)) -python_version_flat := $(word 1, $(py_ver_tuple))$(word 2, $(py_ver_tuple)) - -ifeq ($(db_engine_version_flat), 57) - db_client := my57 -else - db_client := $(db_engine_name) -endif - - .PHONY: test-integration test-integration: @echo -n $(db_engine_name) > tests/integration/db_engine_name @@ -94,9 +77,8 @@ test-integration: https://github.com/ansible/ansible/archive/$(ansible).tar.gz; \ set -x; \ ansible-test integration $(target) -v --color --coverage --diff \ - --docker ghcr.io/ansible-collections/community.mysql/test-container\ - -$(db_client)-py$(python_version_flat)-$(connector_name)$(connector_version_flat):latest \ - --docker-network podman $(_continue_on_errors) $(_keep_containers_alive) --python $(python); \ + --docker --python $(python) \ + --docker-network podman $(_continue_on_errors) $(_keep_containers_alive); \ set +x # End of venv diff --git a/TESTING.md b/TESTING.md index f31db4a..54eb5ed 100644 --- a/TESTING.md +++ b/TESTING.md @@ -26,12 +26,9 @@ For now, the makefile only supports Podman. - Minimum 2GB of RAM -### Custom ansible-test containers +### ansible-test environment -Our integrations tests use custom containers for ansible-test. Those images have their definition file stored in the directory [test-containers](test-containers/). We build and publish the images on ghcr.io under the ansible-collection namespace: E.G.: -`ghcr.io/ansible-collections/community.mysql/test-container-mariadb106-py310-mysqlclient211:latest`. - -Availables images are listed [here](https://github.com/orgs/ansible-collections/packages). +Integration tests use the default container from ansible-test. Then required packages for the tests are installed from the `setup_controller` target located in the `tests/integration/targets` folder. ### Makefile options @@ -151,16 +148,6 @@ python run_all_tests.py ### Add a new Python, Connector or Database version -You can look into [.github/workflows/ansible-test-plugins.yml](https://github.com/ansible-collections/community.mysql/tree/main/.github/workflows) to see how those containers are built using [build-docker-image.yml](https://github.com/ansible-collections/community.mysql/blob/main/.github/workflows/build-docker-image.yml) and all [docker-image-xxx.yml](https://github.com/ansible-collections/community.mysql/blob/main/.github/workflows/docker-image-mariadb103-py38-mysqlclient201.yml) files. +New components version should be added to this file: [.github/workflows/ansible-test-plugins.yml](https://github.com/ansible-collections/community.mysql/tree/main/.github/workflows) -1. Add a workflow in [.github/workflows/](.github/workflows) -1. Add a new folder in [test-containers](test-containers) containing a new Dockerfile. Your container must contains 3 things: - - Python - - A connector: The python package to connect to the database (pymysql, mysqlclient, ...) - - A mysql client to prepare databases before our tests starts. This client must provide both `mysql` and `mysqldump` commands. -1. Add your version in the matrix of *.github/workflows/ansible-test-plugins.yml*. You can use [run_all_tests.py](run_all_tests.py) to help you see what the matrix will be. Simply comment out the line `os.system(make_cmd)` before runing the script. You can also add `print(len(matrix))` to display how many tests there will be on GitHub Action. -1. Ask the lead maintainer to mark your new image(s) as `public` under [https://github.com/orgs/ansible-collections/packages](https://github.com/orgs/ansible-collections/packages) - -After pushing your commit to the remote, the container will be built and published on ghcr.io. Have a look in the "Action" tab to see if it worked. In case of error `failed to copy: io: read/write on closed pipe` re-run the workflow, this append unfortunately a lot. - -To see the docker image produced, go to the package page in the ansible-collection namespace [https://github.com/orgs/ansible-collections/packages](https://github.com/orgs/ansible-collections/packages). This page indicate a "Published x days ago" that is updated infrequently. To see the last time the container has been updated you must click on its title and look in the right hands side bellow the title "Last published". +Be careful to not add too much tests. When adding a new version of Python, for instance, only test it agains the latest versions of Ansible and MySQL/MariaDB. When tests are run, you can see that we already start 40 virtual machines! diff --git a/test-containers/mariadb-py310-mysqlclient211/Dockerfile b/test-containers/mariadb-py310-mysqlclient211/Dockerfile deleted file mode 100644 index f7e9eb1..0000000 --- a/test-containers/mariadb-py310-mysqlclient211/Dockerfile +++ /dev/null @@ -1,21 +0,0 @@ -FROM quay.io/ansible/ubuntu2204-test-container:main -# ubuntu2204 comes with mariadb-client-10.6 - -# iproute2 # To grab docker network gateway address -# python3.10-dev # Reqs for mysqlclient -# default-libmysqlclient-dev # Reqs for mysqlclient -# build-essential # Reqs for mysqlclient -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.10 \ - python3.10-dev \ - mariadb-client \ - iproute2 \ - default-libmysqlclient-dev \ - build-essential - -RUN python3.10 -m pip install --disable-pip-version-check --no-cache-dir mysqlclient==2.1.1 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mariadb-py310-pymysql102/Dockerfile b/test-containers/mariadb-py310-pymysql102/Dockerfile deleted file mode 100644 index afe6a77..0000000 --- a/test-containers/mariadb-py310-pymysql102/Dockerfile +++ /dev/null @@ -1,15 +0,0 @@ -FROM quay.io/ansible/ubuntu2204-test-container:main -# ubuntu2204 comes with mariadb-client-10.6 - -# iproute2 # To grab docker network gateway address -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.10 \ - mariadb-client \ - iproute2 - -RUN python3.10 -m pip install --disable-pip-version-check --no-cache-dir pymysql==1.0.2 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mariadb-py38-mysqlclient201/Dockerfile b/test-containers/mariadb-py38-mysqlclient201/Dockerfile deleted file mode 100644 index 68ea3f6..0000000 --- a/test-containers/mariadb-py38-mysqlclient201/Dockerfile +++ /dev/null @@ -1,21 +0,0 @@ -FROM quay.io/ansible/ubuntu2004-test-container:main -# ubuntu2004 comes with mariadb-client-10.3 - -# iproute2 # To grab docker network gateway address -# python3.8-dev # Reqs for mysqlclient -# default-libmysqlclient-dev # Reqs for mysqlclient -# build-essential # Reqs for mysqlclient -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.8 \ - python3.8-dev \ - mariadb-client \ - iproute2 \ - default-libmysqlclient-dev \ - build-essential - -RUN python3.8 -m pip install --disable-pip-version-check --no-cache-dir mysqlclient==2.0.1 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mariadb-py38-pymysql093/Dockerfile b/test-containers/mariadb-py38-pymysql093/Dockerfile deleted file mode 100644 index 22c8c57..0000000 --- a/test-containers/mariadb-py38-pymysql093/Dockerfile +++ /dev/null @@ -1,15 +0,0 @@ -FROM quay.io/ansible/ubuntu2004-test-container:main -# ubuntu2004 comes with mariadb-client-10.3 - -# iproute2 # To grab docker network gateway address -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.8 \ - mariadb-client \ - iproute2 - -RUN python3.8 -m pip install --disable-pip-version-check --no-cache-dir pymysql==0.9.3 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mariadb-py39-mysqlclient203/Dockerfile b/test-containers/mariadb-py39-mysqlclient203/Dockerfile deleted file mode 100644 index b7837b2..0000000 --- a/test-containers/mariadb-py39-mysqlclient203/Dockerfile +++ /dev/null @@ -1,21 +0,0 @@ -FROM quay.io/ansible/ubuntu2004-test-container:main -# ubuntu2004 comes with mariadb-client-10.3 - -# iproute2 # To grab docker network gateway address -# python3.9-dev # Reqs for mysqlclient -# default-libmysqlclient-dev # Reqs for mysqlclient -# build-essential # Reqs for mysqlclient -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.9 \ - python3.9-dev \ - mariadb-client \ - iproute2 \ - default-libmysqlclient-dev \ - build-essential - -RUN python3.9 -m pip install --disable-pip-version-check --no-cache-dir mysqlclient==2.0.3 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mariadb-py39-pymysql093/Dockerfile b/test-containers/mariadb-py39-pymysql093/Dockerfile deleted file mode 100644 index a1451ff..0000000 --- a/test-containers/mariadb-py39-pymysql093/Dockerfile +++ /dev/null @@ -1,15 +0,0 @@ -FROM quay.io/ansible/ubuntu2004-test-container:main -# ubuntu2004 comes with mariadb-client-10.3 - -# iproute2 # To grab docker network gateway address -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.9 \ - mariadb-client \ - iproute2 - -RUN python3.9 -m pip install --disable-pip-version-check --no-cache-dir pymysql==0.9.3 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/my57-py38-mysqlclient201/Dockerfile b/test-containers/my57-py38-mysqlclient201/Dockerfile deleted file mode 100644 index 0eb1778..0000000 --- a/test-containers/my57-py38-mysqlclient201/Dockerfile +++ /dev/null @@ -1,21 +0,0 @@ -FROM quay.io/ansible/ubuntu1804-test-container:main -# ubuntu1804 comes with mysql-client-5.7 - -# iproute2 # To grab docker network gateway address -# python3.8-dev # Reqs for mysqlclient -# default-libmysqlclient-dev # Reqs for mysqlclient -# build-essential # Reqs for mysqlclient -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.8 \ - python3.8-dev \ - mysql-client \ - iproute2 \ - default-libmysqlclient-dev \ - build-essential - -RUN python3.8 -m pip install --disable-pip-version-check --no-cache-dir mysqlclient==2.0.1 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/my57-py38-pymysql0711/Dockerfile b/test-containers/my57-py38-pymysql0711/Dockerfile deleted file mode 100644 index 9141709..0000000 --- a/test-containers/my57-py38-pymysql0711/Dockerfile +++ /dev/null @@ -1,21 +0,0 @@ -FROM quay.io/ansible/ubuntu1804-test-container:main -# ubuntu1804 comes with mysql-client-5.7 - -# iproute2 # To grab docker network gateway address -# python3.8-dev # Reqs for mysqlclient -# default-libmysqlclient-dev # Reqs for mysqlclient -# build-essential # Reqs for mysqlclient -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.8 \ - python3.8-dev \ - mysql-client \ - iproute2 \ - default-libmysqlclient-dev \ - build-essential - -RUN python3.8 -m pip install --disable-pip-version-check --no-cache-dir pymysql==0.7.11 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/my57-py38-pymysql093/Dockerfile b/test-containers/my57-py38-pymysql093/Dockerfile deleted file mode 100644 index 6b0f519..0000000 --- a/test-containers/my57-py38-pymysql093/Dockerfile +++ /dev/null @@ -1,15 +0,0 @@ -FROM quay.io/ansible/ubuntu1804-test-container:main -# ubuntu1804 comes with mysql-client-5.7 - -# iproute2 # To grab docker network gateway address -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.8 \ - mysql-client \ - iproute2 - -RUN python3.8 -m pip install --disable-pip-version-check --no-cache-dir pymysql==0.9.3 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mysql-py310-mysqlclient211/Dockerfile b/test-containers/mysql-py310-mysqlclient211/Dockerfile deleted file mode 100644 index 1aea0cd..0000000 --- a/test-containers/mysql-py310-mysqlclient211/Dockerfile +++ /dev/null @@ -1,21 +0,0 @@ -FROM quay.io/ansible/ubuntu2204-test-container:main -# ubuntu2204 comes with mysql-client-8 - -# iproute2 # To grab docker network gateway address -# python3.10-dev # Reqs for mysqlclient -# default-libmysqlclient-dev # Reqs for mysqlclient -# build-essential # Reqs for mysqlclient -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.10 \ - python3.10-dev \ - mysql-client \ - iproute2 \ - default-libmysqlclient-dev \ - build-essential - -RUN python3.10 -m pip install --disable-pip-version-check --no-cache-dir mysqlclient==2.1.1 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mysql-py310-pymysql102/Dockerfile b/test-containers/mysql-py310-pymysql102/Dockerfile deleted file mode 100644 index 871a1e4..0000000 --- a/test-containers/mysql-py310-pymysql102/Dockerfile +++ /dev/null @@ -1,15 +0,0 @@ -FROM quay.io/ansible/ubuntu2204-test-container:main -# ubuntu2204 comes with mysql-client-8 - -# iproute2 # To grab docker network gateway address -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.10 \ - mysql-client \ - iproute2 - -RUN python3.10 -m pip install --disable-pip-version-check --no-cache-dir pymysql==1.0.2 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mysql-py38-mysqlclient201/Dockerfile b/test-containers/mysql-py38-mysqlclient201/Dockerfile deleted file mode 100644 index eb835c2..0000000 --- a/test-containers/mysql-py38-mysqlclient201/Dockerfile +++ /dev/null @@ -1,21 +0,0 @@ -FROM quay.io/ansible/ubuntu2004-test-container:main -# ubuntu2004 comes with mysql-client-8 - -# iproute2 # To grab docker network gateway address -# python3.8-dev # Reqs for mysqlclient -# default-libmysqlclient-dev # Reqs for mysqlclient -# build-essential # Reqs for mysqlclient -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.8 \ - python3.8-dev \ - mysql-client \ - iproute2 \ - default-libmysqlclient-dev \ - build-essential - -RUN python3.8 -m pip install --disable-pip-version-check --no-cache-dir mysqlclient==2.0.1 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mysql-py38-pymysql093/Dockerfile b/test-containers/mysql-py38-pymysql093/Dockerfile deleted file mode 100644 index e97e5e2..0000000 --- a/test-containers/mysql-py38-pymysql093/Dockerfile +++ /dev/null @@ -1,15 +0,0 @@ -FROM quay.io/ansible/ubuntu2004-test-container:main -# ubuntu2004 comes with mysql-client-8 - -# iproute2 # To grab docker network gateway address -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.8 \ - mysql-client \ - iproute2 - -RUN python3.8 -m pip install --disable-pip-version-check --no-cache-dir pymysql==0.9.3 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mysql-py39-mysqlclient203/Dockerfile b/test-containers/mysql-py39-mysqlclient203/Dockerfile deleted file mode 100644 index 396d895..0000000 --- a/test-containers/mysql-py39-mysqlclient203/Dockerfile +++ /dev/null @@ -1,21 +0,0 @@ -FROM quay.io/ansible/ubuntu2004-test-container:main -# ubuntu2004 comes with mysql-client-8 - -# iproute2 # To grab docker network gateway address -# python3.9-dev # Reqs for mysqlclient -# default-libmysqlclient-dev # Reqs for mysqlclient -# build-essential # Reqs for mysqlclient -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.9 \ - python3.9-dev \ - mysql-client \ - iproute2 \ - default-libmysqlclient-dev \ - build-essential - -RUN python3.9 -m pip install --disable-pip-version-check --no-cache-dir mysqlclient==2.0.3 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/test-containers/mysql-py39-pymysql093/Dockerfile b/test-containers/mysql-py39-pymysql093/Dockerfile deleted file mode 100644 index 57ef15e..0000000 --- a/test-containers/mysql-py39-pymysql093/Dockerfile +++ /dev/null @@ -1,16 +0,0 @@ -FROM quay.io/ansible/ubuntu2004-test-container:main -# ubuntu2004 comes with mysql-client-8 - -# iproute2 # To grab docker network gateway address -RUN apt-get update -y && \ - DEBIAN_FRONTEND=noninteractive apt-get upgrade -y --no-install-recommends && \ - DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ - python3.9 \ - mysql-client \ - iproute2 - -# cffi # To connect to MySQL 8 with Python3.9 and PyMySQL -RUN python3.9 -m pip install --disable-pip-version-check --no-cache-dir cffi pymysql==0.9.3 - -ENV container=docker -CMD ["/sbin/init"] diff --git a/tests/integration/targets/setup_controller/tasks/main.yml b/tests/integration/targets/setup_controller/tasks/main.yml index 0d5e36b..91b5f82 100644 --- a/tests/integration/targets/setup_controller/tasks/main.yml +++ b/tests/integration/targets/setup_controller/tasks/main.yml @@ -4,15 +4,18 @@ # and should not be used as examples of how to write Ansible roles # #################################################################### -- name: Prepare the fake root folder +- name: "{{ role_name }} | Main | Prepare the fake root folder" ansible.builtin.import_tasks: file: fake_root.yml -# setvars.yml requires the iproute2 package installed by install.yml -- name: Set variables +- name: "{{ role_name }} | Main | Set variables" ansible.builtin.import_tasks: file: setvars.yml -- name: Verify all components version under test +- name: "{{ role_name }} | Main | Install requirements" + ansible.builtin.import_tasks: + file: requirements.yml + +- name: "{{ role_name }} | Main | Verify all components version under test" ansible.builtin.import_tasks: file: verify.yml diff --git a/tests/integration/targets/setup_controller/tasks/requirements.yml b/tests/integration/targets/setup_controller/tasks/requirements.yml new file mode 100644 index 0000000..8bab1a0 --- /dev/null +++ b/tests/integration/targets/setup_controller/tasks/requirements.yml @@ -0,0 +1,20 @@ +--- + +- name: "{{ role_name }} | Requirements | Install Linux packages" + ansible.builtin.package: + name: + - bzip2 # To test mysql_db dump compression + - "{{ db_engine }}-client" + + # The command mysql-config must be present for mysqlclient python package. + # The package libmysqlclient-dev that provides this command have a + # different name between Ubuntu 20.04 and 22.04. Luckily, libmysql++ is + # available on both. + - "{{ 'libmysql++-dev' if db_engine == 'mysql' else 'libmariadb-dev' }}" + state: present + +- name: "{{ role_name }} | Requirements | Install Python packages" + ansible.builtin.pip: + name: + - "{{ connector_name }}=={{ connector_version }}" + state: present diff --git a/tests/integration/targets/setup_controller/tasks/setvars.yml b/tests/integration/targets/setup_controller/tasks/setvars.yml index 3e070a9..7c3e03b 100644 --- a/tests/integration/targets/setup_controller/tasks/setvars.yml +++ b/tests/integration/targets/setup_controller/tasks/setvars.yml @@ -1,13 +1,17 @@ --- -- name: "{{ role_name }} | Setvars | Extract Podman/Docker Network Gateway" - ansible.builtin.shell: - cmd: ip route|grep default|awk '{print $3}' - register: ip_route_output +- name: "{{ role_name }} | Setvars | Install tools gather network facts" + ansible.builtin.package: + name: + - iproute2 + state: present + +- name: "{{ role_name }} | Setvars | Gather facts" + ansible.builtin.setup: - name: "{{ role_name }} | Setvars | Set Fact" ansible.builtin.set_fact: - gateway_addr: "{{ ip_route_output.stdout }}" + gateway_addr: "{{ ansible_default_ipv4.gateway }}" connector_name_lookup: >- {{ lookup( 'file', diff --git a/tests/integration/targets/test_mysql_db/tasks/config_overrides_defaults.yml b/tests/integration/targets/test_mysql_db/tasks/config_overrides_defaults.yml index 390c6ae..dce0a43 100644 --- a/tests/integration/targets/test_mysql_db/tasks/config_overrides_defaults.yml +++ b/tests/integration/targets/test_mysql_db/tasks/config_overrides_defaults.yml @@ -93,7 +93,9 @@ - name: Config overrides | Add fake host to config file shell: 'echo "host = {{ fake_host }}" >> {{ config_file }}' -- name: Config overrides | Remove database using fake login_host +- name: >- + Config overrides | Fail to Remove database using fake login_host + because its default has been overriden by wrong value from config file mysql_db: login_user: '{{ mysql_user }}' login_password: '{{ mysql_password }}' @@ -102,15 +104,17 @@ name: '{{ db_to_create }}' state: absent config_file: '{{ config_file }}' - config_overrides_defaults: yes + config_overrides_defaults: true register: result - ignore_errors: yes - -- name: Config overrides | Must fail because login_host default has beed overriden by wrong value from config file - assert: - that: - - result is failed - - result.msg is search("Can't connect to MySQL server on '{{ fake_host }}'") or result.msg is search("Unknown MySQL server host '{{ fake_host }}'") + failed_when: + - result is succeeded + - result.msg is not search(pattern1) + - result.msg is not search(pattern2) + - result.msg is not search(pattern3) + vars: + pattern1: Can't connect to MySQL server on '{{ fake_host }}' + pattern2: Unknown MySQL server host '{{ fake_host }}' + pattern3: Unknown server host '{{ fake_host }}' - name: Config overrides | Clean up test database mysql_db: From 33e8754c4e0de108c5621818a9139c5d51cd2dfd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Thu, 27 Jun 2024 22:12:01 +0200 Subject: [PATCH 15/54] Fix mysql_user on_new_username IndexError (#642) * fix tuple indexerror when no accounts are found * Fix tests for update_password not executed * Add test for case where existing user have different password * lint to prevent warning about jinja templating in when clause * Refactor get_existing_authentication to return a list of all row found Previously we were returning only the first row found. We need to be able to see if there is a difference in the existing passwords. * Refactor host option to be optional This make it possible to use the same method from mysql_user to help update_password retrieve existing password for all account with the same username independently of their hostname. And from mysql_info to get the password of a specif user using WHERE user = '' AND host = '' * Add change log fragment * Add link to the PR in the change log * lint for ansible devel * Fix templating type error could not cconvert to bool with ansible devel * Revert changes made for ansible-devel that broke tests for Ansible 2.15 * Revert changes made for ansible-devel that broke tests * Cut unnecessary set, uniqueness is ensured by the group_by in the query * Cut auth plugin from returned values when multiple existing auths exists Discussed here: https://github.com/ansible-collections/community.mysql/pull/642/files#r1649720519 * fix convertion of list(dict) to list(tuple) * Fix test for empty password on MySQL 8+ --- .../lie_fix_mysql_user_on_new_username.yml | 6 ++ plugins/module_utils/user.py | 93 ++++++++++++------- plugins/modules/mysql_info.py | 2 +- .../targets/test_mysql_user/tasks/main.yml | 4 + .../tasks/test_update_password.yml | 26 ++++++ .../tasks/utils/assert_user_password.yml | 23 ++--- .../test_mysql_variables/tasks/issue-28.yml | 37 ++++---- .../tasks/mysql_variables.yml | 24 ++--- 8 files changed, 141 insertions(+), 74 deletions(-) create mode 100644 changelogs/fragments/lie_fix_mysql_user_on_new_username.yml diff --git a/changelogs/fragments/lie_fix_mysql_user_on_new_username.yml b/changelogs/fragments/lie_fix_mysql_user_on_new_username.yml new file mode 100644 index 0000000..7f13738 --- /dev/null +++ b/changelogs/fragments/lie_fix_mysql_user_on_new_username.yml @@ -0,0 +1,6 @@ +--- + +bugfixes: + + - mysql_user - Fixed an IndexError in the update_password functionality introduced in PR https://github.com/ansible-collections/community.mysql/pull/580 and released in community.mysql 3.8.0. If you used this functionality, please avoid versions 3.8.0 to 3.9.0 (https://github.com/ansible-collections/community.mysql/pull/642). + - mysql_user - Added a warning to update_password's on_new_username option if multiple accounts with the same username but different passwords exist (https://github.com/ansible-collections/community.mysql/pull/642). diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index 80da47e..bd71691 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -95,8 +95,12 @@ def get_grants(cursor, user, host): return grants.split(", ") -def get_existing_authentication(cursor, user, host): - # Return the plugin and auth_string if there is exactly one distinct existing plugin and auth_string. +def get_existing_authentication(cursor, user, host=None): + """ Return a list of dict containing the plugin and auth_string for the + specified username. + If hostname is provided, return only the information about this particular + account. + """ cursor.execute("SELECT VERSION()") srv_type = cursor.fetchone() # Mysql_info use a DictCursor so we must convert back to a list @@ -107,37 +111,50 @@ def get_existing_authentication(cursor, user, host): if 'mariadb' in srv_type[0].lower(): # before MariaDB 10.2.19 and 10.3.11, "password" and "authentication_string" can differ # when using mysql_native_password - cursor.execute("""select plugin, auth from ( - select plugin, password as auth from mysql.user where user=%(user)s - and host=%(host)s - union select plugin, authentication_string as auth from mysql.user where user=%(user)s - and host=%(host)s) x group by plugin, auth limit 2 - """, {'user': user, 'host': host}) + if host: + cursor.execute("""select plugin, auth from ( + select plugin, password as auth from mysql.user where user=%(user)s + and host=%(host)s + union select plugin, authentication_string as auth from mysql.user where user=%(user)s + and host=%(host)s) x group by plugin, auth + """, {'user': user, 'host': host}) + else: + cursor.execute("""select plugin, auth from ( + select plugin, password as auth from mysql.user where user=%(user)s + union select plugin, authentication_string as auth from mysql.user where user=%(user)s + ) x group by plugin, auth + """, {'user': user}) else: - cursor.execute("""select plugin, authentication_string as auth - from mysql.user where user=%(user)s and host=%(host)s - group by plugin, authentication_string limit 2""", {'user': user, 'host': host}) + if host: + cursor.execute("""select plugin, authentication_string as auth + from mysql.user where user=%(user)s and host=%(host)s + group by plugin, authentication_string""", {'user': user, 'host': host}) + else: + cursor.execute("""select plugin, authentication_string as auth + from mysql.user where user=%(user)s + group by plugin, authentication_string""", {'user': user}) + rows = cursor.fetchall() - # Mysql_info use a DictCursor so we must convert back to a list - # otherwise we get KeyError 0 - if isinstance(rows, dict): - rows = list(rows.values()) + if len(rows) == 0: + return [] - # 'plugin_auth_string' contains the hash string. Must be removed in c.mysql 4.0 - # See https://github.com/ansible-collections/community.mysql/pull/629 - if isinstance(rows[0], tuple): - return {'plugin': rows[0][0], - 'plugin_auth_string': rows[0][1], - 'plugin_hash_string': rows[0][1]} - - # 'plugin_auth_string' contains the hash string. Must be removed in c.mysql 4.0 - # See https://github.com/ansible-collections/community.mysql/pull/629 + # Mysql_info use a DictCursor so we must convert list(dict) + # to list(tuple) otherwise we get KeyError 0 if isinstance(rows[0], dict): - return {'plugin': rows[0].get('plugin'), - 'plugin_auth_string': rows[0].get('auth'), - 'plugin_hash_string': rows[0].get('auth')} - return None + rows = [tuple(row.values()) for row in rows] + + existing_auth_list = [] + + # 'plugin_auth_string' contains the hash string. Must be removed in c.mysql 4.0 + # See https://github.com/ansible-collections/community.mysql/pull/629 + for r in rows: + existing_auth_list.append({ + 'plugin': r[0], + 'plugin_auth_string': r[1], + 'plugin_hash_string': r[1]}) + + return existing_auth_list def user_add(cursor, user, host, host_all, password, encrypted, @@ -161,14 +178,24 @@ def user_add(cursor, user, host, host_all, password, encrypted, mogrify = do_not_mogrify_requires if old_user_mgmt else mogrify_requires + # This is for update_password: on_new_username used_existing_password = False if reuse_existing_password: - existing_auth = get_existing_authentication(cursor, user, host) + existing_auth = get_existing_authentication(cursor, user) if existing_auth: - plugin = existing_auth['plugin'] - plugin_hash_string = existing_auth['plugin_hash_string'] - password = None - used_existing_password = True + if len(existing_auth) != 1: + module.warn("An account with the username %s has a different " + "password than the others existing accounts. Thus " + "on_new_username can't decide which password to " + "reuse so it will use your provided password " + "instead. If no password is provided, the account " + "will have an empty password!" % user) + used_existing_password = False + else: + plugin_hash_string = existing_auth[0]['plugin_hash_string'] + password = None + used_existing_password = True + plugin = existing_auth[0]['plugin'] # What if plugin differ? if password and encrypted: if impl.supports_identified_by_password(cursor): query_with_args = "CREATE USER %s@%s IDENTIFIED BY PASSWORD %s", (user, host, password) diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index 6103589..9f0586a 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -639,7 +639,7 @@ class MySQL_Info(object): authentications = get_existing_authentication(self.cursor, user, host) if authentications: - output_dict.update(authentications) + output_dict.update(authentications[0]) # TODO password_option # TODO lock_option diff --git a/tests/integration/targets/test_mysql_user/tasks/main.yml b/tests/integration/targets/test_mysql_user/tasks/main.yml index 8ec0798..e77c443 100644 --- a/tests/integration/targets/test_mysql_user/tasks/main.yml +++ b/tests/integration/targets/test_mysql_user/tasks/main.yml @@ -295,3 +295,7 @@ - name: Mysql_user - test column case sensitive ansible.builtin.import_tasks: file: test_column_case_sensitive.yml + + - name: Mysql_user - test update_password + ansible.builtin.import_tasks: + file: test_update_password.yml diff --git a/tests/integration/targets/test_mysql_user/tasks/test_update_password.yml b/tests/integration/targets/test_mysql_user/tasks/test_update_password.yml index 428c1ef..adaa7c7 100644 --- a/tests/integration/targets/test_mysql_user/tasks/test_update_password.yml +++ b/tests/integration/targets/test_mysql_user/tasks/test_update_password.yml @@ -127,3 +127,29 @@ update_password: on_create - username: test3 update_password: on_new_username + + # another new user, another new password and multiple existing users with + # varying passwords without providing a password + - name: update_password | Create account with on_new_username while omit password + community.mysql.mysql_user: + login_user: '{{ mysql_parameters.login_user }}' + login_password: '{{ mysql_parameters.login_password }}' + login_host: '{{ mysql_parameters.login_host }}' + login_port: '{{ mysql_parameters.login_port }}' + state: present + name: test3 + host: '10.10.10.10' + update_password: on_new_username + + - name: update_password | Assert create account with on_new_username while omit password produce empty auth string + ansible.builtin.command: >- + {{ mysql_command }} -BNe "SELECT user, host, plugin, authentication_string + FROM mysql.user where user='test3' and host='10.10.10.10'" + register: test3_info + changed_when: false + failed_when: + # MariaDB default plugin is mysql_native_password + - "'test3\t10.10.10.10\tmysql_native_password\t' != test3_info.stdout" + + # MySQL 8+ default plugin is caching_sha2_password + - "'test3\t10.10.10.10\tcaching_sha2_password\t' != test3_info.stdout" diff --git a/tests/integration/targets/test_mysql_user/tasks/utils/assert_user_password.yml b/tests/integration/targets/test_mysql_user/tasks/utils/assert_user_password.yml index d95e53b..e6bd695 100644 --- a/tests/integration/targets/test_mysql_user/tasks/utils/assert_user_password.yml +++ b/tests/integration/targets/test_mysql_user/tasks/utils/assert_user_password.yml @@ -1,6 +1,6 @@ --- - name: Utils | Assert user password | Apply update_password to {{ username }} - mysql_user: + community.mysql.mysql_user: login_user: '{{ mysql_parameters.login_user }}' login_password: '{{ mysql_parameters.login_password }}' login_host: '{{ mysql_parameters.login_host }}' @@ -13,16 +13,17 @@ register: result - name: Utils | Assert user password | Assert a change occurred - assert: + ansible.builtin.assert: that: - - "result.changed | bool == {{ expect_change }} | bool" - - "result.password_changed == {{ expect_password_change }}" + - result.changed | bool == expect_change | bool + - result.password_changed == expect_password_change -- name: Utils | Assert user password | Query user {{ username }} - command: "{{ mysql_command }} -BNe \"SELECT plugin, authentication_string FROM mysql.user where user='{{ username }}' and host='{{ host }}'\"" +- name: Utils | Assert user password | Assert expect_hash is in user stdout for {{ username }} + ansible.builtin.command: >- + {{ mysql_command }} -BNe "SELECT plugin, authentication_string + FROM mysql.user where user='{{ username }}' and host='{{ host }}'" register: existing_user - -- name: Utils | Assert user password | Assert expect_hash is in user stdout - assert: - that: - - "'mysql_native_password\t{{ expect_password_hash }}' in existing_user.stdout_lines" + changed_when: false + failed_when: pattern not in existing_user.stdout_lines + vars: + pattern: "mysql_native_password\t{{ expect_password_hash }}" diff --git a/tests/integration/targets/test_mysql_variables/tasks/issue-28.yml b/tests/integration/targets/test_mysql_variables/tasks/issue-28.yml index 10a9154..89d3d26 100644 --- a/tests/integration/targets/test_mysql_variables/tasks/issue-28.yml +++ b/tests/integration/targets/test_mysql_variables/tasks/issue-28.yml @@ -1,8 +1,11 @@ --- - name: set fact tls_enabled - command: "{{ mysql_command }} \"-e SHOW VARIABLES LIKE 'have_ssl';\"" + ansible.builtin.command: + cmd: "{{ mysql_command }} \"-e SHOW VARIABLES LIKE 'have_ssl';\"" register: result -- set_fact: + +- name: Set tls_enabled fact + ansible.builtin.set_fact: tls_enabled: "{{ 'YES' in result.stdout | bool | default('false', true) }}" - vars: @@ -16,21 +19,21 @@ # ============================================================ - name: get server certificate - copy: + ansible.builtin.copy: content: "{{ lookup('pipe', \"openssl s_client -starttls mysql -connect localhost:3307 -showcerts 2>/dev/null = 0.7.11 is required' in result.msg + ignore_errors: true + failed_when: + - result is failed or 'pymysql >= 0.7.11 is required' not in result.msg - name: Drop mysql user - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ user_name_1 }}' host_all: true diff --git a/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml b/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml index 2d2318e..4a7fd00 100644 --- a/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml +++ b/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml @@ -47,8 +47,8 @@ # Verify mysql_variable successfully updates a variable (issue:4568) # - set_fact: - set_name: 'delay_key_write' - set_value: 'ON' + set_name: 'delay_key_write' + set_value: 'ON' - name: set mysql variable mysql_variables: @@ -74,8 +74,8 @@ # Verify mysql_variable successfully updates a variable using single quotes # - set_fact: - set_name: 'wait_timeout' - set_value: '300' + set_name: 'wait_timeout' + set_value: '300' - name: set mysql variable to a temp value mysql_variables: @@ -105,8 +105,8 @@ # Verify mysql_variable successfully updates a variable using double quotes # - set_fact: - set_name: "wait_timeout" - set_value: "400" + set_name: "wait_timeout" + set_value: "400" - name: set mysql variable to a temp value mysql_variables: @@ -132,8 +132,8 @@ # Verify mysql_variable successfully updates a variable using no quotes # - set_fact: - set_name: wait_timeout - set_value: 500 + set_name: wait_timeout + set_value: 500 - name: set mysql variable to a temp value mysql_variables: @@ -251,8 +251,8 @@ # Verify mysql_variable works with the login_user and login_password parameters # - set_fact: - set_name: wait_timeout - set_value: 77 + set_name: wait_timeout + set_value: 77 - name: query mysql_variable using login_user and password_password mysql_variables: @@ -291,8 +291,8 @@ # Verify mysql_variable fails with an incorrect login_password parameter # - set_fact: - set_name: connect_timeout - set_value: 10 + set_name: connect_timeout + set_value: 10 - name: query mysql_variable using incorrect login_password mysql_variables: From 4912f1a41b9b7a79fa526119879ff8159bb7c2da Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Fri, 28 Jun 2024 11:34:59 +0200 Subject: [PATCH 16/54] mysql_variables: fix boolean value handling (#653) * mysql_variables: fix boolean value handling * fix * Fix tests * Fix tests * Fix * Fix * Fix * Fix comment --- changelogs/fragments/2-mysql_variables.yml | 2 + plugins/modules/mysql_variables.py | 21 +++++ .../tasks/mysql_variables.yml | 93 +++++++++++++++++++ .../plugins/modules/test_mysql_variables.py | 26 ++++++ 4 files changed, 142 insertions(+) create mode 100644 changelogs/fragments/2-mysql_variables.yml create mode 100644 tests/unit/plugins/modules/test_mysql_variables.py diff --git a/changelogs/fragments/2-mysql_variables.yml b/changelogs/fragments/2-mysql_variables.yml new file mode 100644 index 0000000..9ef8d80 --- /dev/null +++ b/changelogs/fragments/2-mysql_variables.yml @@ -0,0 +1,2 @@ +bugfixes: +- mysql_variables - fix the module always changes on boolean values (https://github.com/ansible-collections/community.mysql/issues/652). diff --git a/plugins/modules/mysql_variables.py b/plugins/modules/mysql_variables.py index f912a27..8632a52 100644 --- a/plugins/modules/mysql_variables.py +++ b/plugins/modules/mysql_variables.py @@ -26,6 +26,7 @@ options: value: description: - If set, then sets variable value to this. + - With boolean values, use C(0)|C(1) or quoted C("ON")|C("OFF"). type: str mode: description: @@ -74,6 +75,11 @@ EXAMPLES = r''' variable: read_only value: 1 mode: persist + +- name: Set a boolean using ON/OFF notation + mysql_variables: + variable: log_slow_replica_statements + value: "ON" # Make sure it's quoted ''' RETURN = r''' @@ -176,6 +182,18 @@ def setvariable(cursor, mysqlvar, value, mode='global'): return result +def convert_bool_setting_value_wanted(val): + """Converts passed value from 0,1,on,off to ON/OFF + as it's represented in the server. + """ + if val in ('on', 1): + val = 'ON' + elif val in ('off', 0): + val = 'OFF' + + return val + + def main(): argument_spec = mysql_common_argument_spec() argument_spec.update( @@ -243,6 +261,9 @@ def main(): # Type values before using them value_wanted = typedvalue(value) value_actual = typedvalue(mysqlvar_val) + if value_actual in ('ON', 'OFF') and value_wanted not in ('ON', 'OFF'): + value_wanted = convert_bool_setting_value_wanted(value_wanted) + value_in_auto_cnf = None if var_in_mysqld_auto_cnf is not None: value_in_auto_cnf = typedvalue(var_in_mysqld_auto_cnf) diff --git a/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml b/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml index 4a7fd00..8194172 100644 --- a/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml +++ b/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml @@ -287,6 +287,99 @@ var_name: "{{set_name}}" var_value: '{{set_value}}' + #========================================================================= + # Bugfix https://github.com/ansible-collections/community.mysql/issues/652 + + - name: Get server version + register: result + mysql_info: + <<: *mysql_params + + - name: Set variable name when running on MySQL + set_fact: + log_slow_statements: log_slow_replica_statements + when: result.server_engine == 'MySQL' + + - name: Set variable name when running on MariaDB + set_fact: + log_slow_statements: log_slow_slave_statements + when: result.server_engine == 'MariaDB' + + - name: Set a boolean value using ON + mysql_variables: + <<: *mysql_params + variable: "{{ log_slow_statements }}" + value: "ON" + register: result + + - name: Check that it changed + assert: + that: + - result is changed or result.msg == "Variable is already set to requested value." + - result.msg == "Variable is already set to requested value." or result.queries == ["SET GLOBAL `{{ log_slow_statements }}` = ON"] + + - name: Set a boolean value again using ON + mysql_variables: + <<: *mysql_params + variable: "{{ log_slow_statements }}" + value: "ON" + register: result + + - name: Check that it didn't change + assert: + that: + - result is not changed + + - name: Set a boolean value again using 1 + mysql_variables: + <<: *mysql_params + variable: "{{ log_slow_statements }}" + value: 1 + register: result + + - name: Check that it didn't change + assert: + that: + - result is not changed + + - name: Set a boolean value using OFF + mysql_variables: + <<: *mysql_params + variable: "{{ log_slow_statements }}" + value: "OFF" + register: result + + - name: Check that it changed + assert: + that: + - result is changed + - result.queries == ["SET GLOBAL `{{ log_slow_statements }}` = OFF"] + + - name: Set a boolean value again using 0 + mysql_variables: + <<: *mysql_params + variable: "{{ log_slow_statements }}" + value: 0 + register: result + + - name: Check that it didn't change + assert: + that: + - result is not changed + + - name: Set a boolean value using on + mysql_variables: + <<: *mysql_params + variable: "{{ log_slow_statements }}" + value: "on" + register: result + + - name: Check that it changed + assert: + that: + - result is changed + - result.queries == ["SET GLOBAL `{{ log_slow_statements }}` = ON"] + #============================================================ # Verify mysql_variable fails with an incorrect login_password parameter # diff --git a/tests/unit/plugins/modules/test_mysql_variables.py b/tests/unit/plugins/modules/test_mysql_variables.py new file mode 100644 index 0000000..8960173 --- /dev/null +++ b/tests/unit/plugins/modules/test_mysql_variables.py @@ -0,0 +1,26 @@ +# -*- coding: utf-8 -*- + +from __future__ import (absolute_import, division, print_function) +__metaclass__ = type + +import pytest + +from ansible_collections.community.mysql.plugins.modules.mysql_variables import ( + convert_bool_setting_value_wanted, +) + + +@pytest.mark.parametrize( + 'value,output', + [ + (1, 'ON'), + (0, 'OFF'), + (2, 2), + ('on', 'ON'), + ('off', 'OFF'), + ('ON', 'ON'), + ('OFF', 'OFF'), + ] +) +def test_convert_bool_value(value, output): + assert convert_bool_setting_value_wanted(value) == output From 83ed4af4e13233c8ca1b8528cf9ba7bc536e03ed Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Tue, 9 Jul 2024 08:20:47 +0200 Subject: [PATCH 17/54] Deprecate mysqlclient/MySQLdb connector support (#655) * Deprecate mysqlclient/MySQLdb connector support * Update README * Put in README that mysqlclient is deprecated --- README.md | 9 ++------- .../fragments/3-deprecate_mysqlclient.yml | 2 ++ plugins/doc_fragments/mysql.py | 20 +++++++------------ plugins/module_utils/mysql.py | 7 +++++++ plugins/modules/mysql_info.py | 1 - 5 files changed, 18 insertions(+), 21 deletions(-) create mode 100644 changelogs/fragments/3-deprecate_mysqlclient.yml diff --git a/README.md b/README.md index 2678f31..98e90b2 100644 --- a/README.md +++ b/README.md @@ -122,17 +122,12 @@ For MariaDB, only Long Term releases are tested. - pymysql 0.7.11 (Only tested with MySQL 5.7) - pymysql 0.9.3 - pymysql 1.0.2 (only collection version >= 3.6.1) -- mysqlclient 2.0.1 -- mysqlclient 2.0.3 (only collection version >= 3.5.2) -- mysqlclient 2.1.1 (only collection version >= 3.5.2) ## External requirements -The MySQL modules rely on a MySQL connector. The list of supported drivers is below: +The MySQL modules rely on a [PyMySQL](https://github.com/PyMySQL/PyMySQL) connector. -- [PyMySQL](https://github.com/PyMySQL/PyMySQL) -- [mysqlclient](https://github.com/PyMySQL/mysqlclient) -- Support for other Python MySQL connectors may be added in a future release. +The `mysqlclient` connector support has been [deprecated](https://github.com/ansible-collections/community.mysql/issues/654) - use `PyMySQL` connector instead! We will stop testing against it in collection version 4.0.0 and remove the related code in 5.0.0. ## Using this collection diff --git a/changelogs/fragments/3-deprecate_mysqlclient.yml b/changelogs/fragments/3-deprecate_mysqlclient.yml new file mode 100644 index 0000000..9134413 --- /dev/null +++ b/changelogs/fragments/3-deprecate_mysqlclient.yml @@ -0,0 +1,2 @@ +breaking_changes: +- collection - support of mysqlclient connector is deprecated - use PyMySQL connector instead! We will stop testing against it in collection version 4.0.0 and remove the related code in 5.0.0 (https://github.com/ansible-collections/community.mysql/issues/654). diff --git a/plugins/doc_fragments/mysql.py b/plugins/doc_fragments/mysql.py index 27ec650..a52243b 100644 --- a/plugins/doc_fragments/mysql.py +++ b/plugins/doc_fragments/mysql.py @@ -71,24 +71,21 @@ options: - Whether to validate the server host name when an SSL connection is required. Corresponds to MySQL CLIs C(--ssl) switch. - Setting this to C(false) disables hostname verification. Use with caution. - Requires pymysql >= 0.7.11. - - This option has no effect on MySQLdb. type: bool version_added: '1.1.0' requirements: - - mysqlclient (Python 3.5+) or - - PyMySQL (Python 2.7 and Python 3.x) or - - MySQLdb (Python 2.x) + - PyMySQL (Python 2.7 and Python 3.x) notes: - - Requires the PyMySQL (Python 2.7 and Python 3.X) or MySQL-python (Python 2.X) package installed on the remote host. + - Requires the PyMySQL (Python 2.7 and Python 3.X) package installed on the remote host. The Python package may be installed with apt-get install python-pymysql (Ubuntu; see M(ansible.builtin.apt)) or yum install python2-PyMySQL (RHEL/CentOS/Fedora; see M(ansible.builtin.yum)). You can also use dnf install python2-PyMySQL for newer versions of Fedora; see M(ansible.builtin.dnf). - - Be sure you have mysqlclient, PyMySQL, or MySQLdb library installed on the target machine - for the Python interpreter Ansible discovers. For example if ansible discovers and uses Python 3, you need to install - the Python 3 version of PyMySQL or mysqlclient. If ansible discovers and uses Python 2, you need to install the Python 2 - version of either PyMySQL or MySQL-python. + - Be sure you have PyMySQL library installed on the target machine + for the Python interpreter Ansible discovers. For example if ansible discovers and uses Python 3, you need to install + the Python 3 version of PyMySQL. If ansible discovers and uses Python 2, you need to install the Python 2 + version of PyMySQL. - If you have trouble, it may help to force Ansible to use the Python interpreter you need by specifying - C(ansible_python_interpreter). For more information, see + C(ansible_python_interpreter). For more information, see U(https://docs.ansible.com/ansible/latest/reference_appendices/interpreter_discovery.html). - Both C(login_password) and C(login_user) are required when you are passing credentials. If none are present, the module will attempt to read @@ -99,9 +96,6 @@ notes: and later uses the unix_socket authentication plugin by default that without using I(login_unix_socket=/var/run/mysqld/mysqld.sock) (the default path) causes the error ``Host '127.0.0.1' is not allowed to connect to this MariaDB server``. - - Alternatively, you can use the mysqlclient library instead of MySQL-python (MySQLdb) - which supports both Python 2.X and Python >=3.5. - See U(https://pypi.org/project/mysqlclient/) how to install it. - "If credentials from the config file (for example, C(/root/.my.cnf)) are not needed to connect to a database server, but the file exists and does not contain a C([client]) section, before any other valid directives, it will be read and this will cause the connection to fail, to prevent this set it to an empty string, (for example C(config_file: ''))." diff --git a/plugins/module_utils/mysql.py b/plugins/module_utils/mysql.py index 10ccfcf..9758994 100644 --- a/plugins/module_utils/mysql.py +++ b/plugins/module_utils/mysql.py @@ -154,6 +154,13 @@ def mysql_connect(module, login_user=None, login_password=None, config_file='', db_connection = mysql_driver.connect(autocommit=autocommit, **config) else: # In case of MySQLdb driver + + # Will be deprecated and dropped + # https://github.com/ansible-collections/community.mysql/issues/654 + module.warn('Support of mysqlcline/MySQLdb connector is deprecated. ' + 'We\'ll stop testing against it in collection version 4.0.0 ' + 'and remove the related code in 5.0.0. Use PyMySQL connector instead.') + if mysql_driver.version_info[0] < 2 or (mysql_driver.version_info[0] == 2 and mysql_driver.version_info[1] < 1): # for MySQLdb < 2.1.0, use 'db' instead of 'database' and 'passwd' instead of 'password' if 'database' in config: diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index 9f0586a..d8bc88c 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -280,7 +280,6 @@ connector_name: type: str sample: - "pymysql" - - "MySQLdb" version_added: '3.6.0' connector_version: description: Version of the python connector used by the module. When the connector is not identified, returns C(Unknown). From c503dc5b6bdfa06373ff8e8ec7db0f12c911938a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Fri, 19 Jul 2024 11:04:13 +0200 Subject: [PATCH 18/54] [CI] Add 2024 versions to tests (#660) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Enable mysql_native_password for MySQL 8.2+ * Fix connection to MySQL 8 since Ubuntu 20.04 update * Cut mysqlclient form the documentation * Cut tests for Python 3.12 not supported by ansible-test * Upgrade integration controller to ubuntu2204 by removing python ansible-test uses python 3.10 if we specify ubuntu2204. Thus we lose the ability to chose specific version of python to test. But integrations tests are optional for a collection. And we don't catch a issue with Python that often (ever ? I don't recall seen one). This allow us to test MySQL 8.4, so it's a win. * Cut tests for EoL MariaDB 10.4 * Reduce number of test in the matrix * Cut support for intermediate LTS * Fix python command not found with ansible-devel and add the debug This is puzzling me. Why when using ansible devel the python command changes? I know ansible-test install python after starting ubuntu22.04 so the way python is install must changes. * Disable retry-on-error When reading log we tend to look at the bottom, but doing so we find often a idempotent error that are nothing to do with the first error. Disabling this can greatly speedup tests and makes logs more readable. Plus, now GHA jumps automatically at the latest error message. So with this modification, we will always jump to the latest real error message. * Enhance jobs title readability We can't expand the left column on GHA, so the shorter, the better. Use Ⓐ instead of Ansible. --- .github/workflows/ansible-test-plugins.yml | 250 +++++++++--------- Makefile | 27 +- README.md | 31 ++- TESTING.md | 44 ++- .../setup_controller/tasks/requirements.yml | 2 + .../setup_controller/tasks/setvars.yml | 7 - .../targets/setup_controller/tasks/verify.yml | 20 +- 7 files changed, 201 insertions(+), 180 deletions(-) diff --git a/.github/workflows/ansible-test-plugins.yml b/.github/workflows/ansible-test-plugins.yml index f3f440e..efc1537 100644 --- a/.github/workflows/ansible-test-plugins.yml +++ b/.github/workflows/ansible-test-plugins.yml @@ -17,7 +17,7 @@ on: # yamllint disable-line rule:truthy jobs: sanity: - name: "Sanity (Ansible: ${{ matrix.ansible }})" + name: "Sanity (Ⓐ${{ matrix.ansible }})" runs-on: ubuntu-22.04 strategy: matrix: @@ -35,8 +35,10 @@ jobs: testing-type: sanity pull-request-change-detection: true + # Use this to chose which version of Python vs Ansible to test: + # https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html#ansible-core-control-node-python-support integration: - name: "Integration (Python: ${{ matrix.python }}, Ansible: ${{ matrix.ansible }}, DB: ${{ matrix.db_engine_name }} ${{ matrix.db_engine_version }}, connector: ${{ matrix.connector_name }} ${{ matrix.connector_version }})" + name: "Integration (Ⓐ${{ matrix.ansible }}, DB: ${{ matrix.db_engine_name }} ${{ matrix.db_engine_version }}, connector: ${{ matrix.connector_name }} ${{ matrix.connector_version }})" runs-on: ubuntu-22.04 strategy: fail-fast: false @@ -50,142 +52,117 @@ jobs: - mysql - mariadb db_engine_version: - - 5.7.40 - - 8.0.31 - - 10.4.27 - - 10.5.18 - - 10.6.11 - python: - - '3.8' - - '3.9' - - '3.10' + - '8.0.38' + - '8.4.1' + - '10.5.25' + - '10.11.8' connector_name: - pymysql - mysqlclient connector_version: - - 0.7.11 - - 0.9.3 - - 1.0.2 - - 2.0.1 - - 2.0.3 - - 2.1.1 + - '0.9.3' + - '1.0.2' + - '1.1.1' + - '2.0.1' + - '2.0.3' + - '2.1.1' + + include: + + # RHEL8 context + - connector_name: pymysql + connector_version: '0.10.1' + ansible: stable-2.16 + db_engine_name: mariadb + db_engine_version: '10.11.8' + + # RHEL9 context + # - connector_name: pymysql + # connector_version: '1.1.1' + # ansible: stable-2.17 + # db_engine_name: mariadb + # db_engine_version: '10.11.8' + # This tests is already included in the matrix, no need repeating + exclude: - - db_engine_name: mysql - db_engine_version: 10.4.27 - db_engine_name: mysql - db_engine_version: 10.5.18 + db_engine_version: '10.5.25' - db_engine_name: mysql - db_engine_version: 10.6.11 + db_engine_version: '10.11.8' - db_engine_name: mariadb - db_engine_version: 5.7.40 + db_engine_version: '8.0.38' - db_engine_name: mariadb - db_engine_version: 8.0.31 + db_engine_version: '8.4.1' - connector_name: pymysql - connector_version: 2.0.1 + connector_version: '2.0.1' - connector_name: pymysql - connector_version: 2.0.3 + connector_version: '2.0.3' - connector_name: pymysql - connector_version: 2.1.1 + connector_version: '2.1.1' - connector_name: mysqlclient - connector_version: 0.7.11 + connector_version: '0.9.3' - connector_name: mysqlclient - connector_version: 0.9.3 + connector_version: '1.0.2' - connector_name: mysqlclient - connector_version: 1.0.2 + connector_version: '1.1.1' - - db_engine_name: mariadb - connector_version: 0.7.11 + - db_engine_version: '8.0.38' + ansible: stable-2.17 - - db_engine_version: 5.7.40 - python: '3.9' + - db_engine_version: '10.5.25' + ansible: stable-2.17 - - db_engine_version: 5.7.40 - python: '3.10' + - db_engine_version: '8.0.38' + ansible: devel - - db_engine_version: 5.7.40 + - db_engine_version: '10.5.25' + ansible: devel + + - db_engine_version: '8.4.1' + connector_version: '0.9.3' + + - db_engine_version: '8.4.1' + connector_version: '1.0.2' + + - db_engine_version: '8.4.1' + connector_version: '2.0.1' + + - db_engine_version: '8.4.1' + connector_version: '2.0.3' + + - db_engine_version: '10.11.8' + connector_version: '0.9.3' + + - db_engine_version: '10.11.8' + connector_version: '1.0.2' + + - db_engine_version: '10.11.8' + connector_version: '2.0.1' + + - db_engine_version: '10.11.8' + connector_version: '2.0.1' + + - db_engine_version: '10.11.8' ansible: stable-2.15 - - db_engine_version: 5.7.40 - ansible: stable-2.16 + - db_engine_version: '8.4.1' + ansible: stable-2.15 - - db_engine_version: 5.7.40 - ansible: devel + - connector_version: '1.1.1' + db_engine_version: '8.0.38' - - db_engine_version: 8.0.31 - python: '3.8' - - - db_engine_version: 10.4.27 - python: '3.10' - - - db_engine_version: 10.4.27 - ansible: devel - - - db_engine_version: 10.6.11 - python: '3.8' - - - db_engine_version: 10.6.11 - python: '3.9' - - - python: '3.8' - connector_version: 1.0.2 - - - python: '3.8' - connector_version: 2.0.3 - - - python: '3.8' - connector_version: 2.1.1 - - - python: '3.9' - connector_version: 0.7.11 - - - python: '3.9' - connector_version: 1.0.2 - - - python: '3.9' - connector_version: 2.0.1 - - - python: '3.9' - connector_version: 2.1.1 - - - python: '3.10' - connector_version: 0.7.11 - - - python: '3.10' - connector_version: 0.9.3 - - - python: '3.10' - connector_version: 2.0.1 - - - python: '3.10' - connector_version: 2.0.3 - - - python: '3.8' - ansible: stable-2.16 - - - python: '3.8' - ansible: stable-2.17 - - - python: '3.8' - ansible: devel - - - python: '3.9' - ansible: stable-2.16 - - - python: '3.9' - ansible: stable-2.17 - - - python: '3.9' - ansible: devel + - connector_version: '1.1.1' + db_engine_version: '10.5.25' services: db_primary: @@ -238,9 +215,22 @@ jobs: - name: Restart MySQL server with settings for replication run: | - docker exec ${{ job.services.db_primary.id }} bash -c 'echo -e [mysqld]\\nserver-id=1\\nlog-bin=/var/lib/mysql/primary-bin > /etc/mysql/conf.d/replication.cnf' - docker exec ${{ job.services.db_replica1.id }} bash -c 'echo -e [mysqld]\\nserver-id=2\\nlog-bin=/var/lib/mysql/replica1-bin > /etc/mysql/conf.d/replication.cnf' - docker exec ${{ job.services.db_replica2.id }} bash -c 'echo -e [mysqld]\\nserver-id=3\\nlog-bin=/var/lib/mysql/replica2-bin > /etc/mysql/conf.d/replication.cnf' + db_ver="${{ matrix.db_engine_version }}" + maj="${db_ver%.*.*}" + maj_min="${db_ver%.*}" + min="${maj_min#*.}" + if [[ "${{ matrix.db_engine_name }}" == "mysql" && "$maj" -eq 8 && "$min" -ge 2 ]]; then + prima_conf='[mysqld]\\nserver-id=1\\nlog-bin=/var/lib/mysql/primary-bin\\nmysql-native-password=1' + repl1_conf='[mysqld]\\nserver-id=2\\nlog-bin=/var/lib/mysql/replica1-bin\\nmysql-native-password=1' + repl2_conf='[mysqld]\\nserver-id=3\\nlog-bin=/var/lib/mysql/replica2-bin\\nmysql-native-password=1' + else + prima_conf='[mysqld]\\nserver-id=1\\nlog-bin=/var/lib/mysql/primary-bin' + repl1_conf='[mysqld]\\nserver-id=2\\nlog-bin=/var/lib/mysql/replica1-bin' + repl2_conf='[mysqld]\\nserver-id=3\\nlog-bin=/var/lib/mysql/replica2-bin' + fi + docker exec -e cnf=$prima_conf ${{ job.services.db_primary.id }} bash -c 'echo -e ${cnf//\\n/\n} > /etc/mysql/conf.d/replication.cnf' + docker exec -e cnf=$repl1_conf ${{ job.services.db_replica1.id }} bash -c 'echo -e ${cnf//\\n/\n} > /etc/mysql/conf.d/replication.cnf' + docker exec -e cnf=$repl2_conf ${{ job.services.db_replica2.id }} bash -c 'echo -e ${cnf//\\n/\n} > /etc/mysql/conf.d/replication.cnf' docker restart -t 30 ${{ job.services.db_primary.id }} docker restart -t 30 ${{ job.services.db_replica1.id }} docker restart -t 30 ${{ job.services.db_replica2.id }} @@ -255,10 +245,10 @@ jobs: - name: >- Perform integration testing against Ansible version ${{ matrix.ansible }} - under Python ${{ matrix.python }} uses: ansible-community/ansible-test-gh-action@release/v1 with: ansible-core-version: ${{ matrix.ansible }} + docker-image: ubuntu2204 pre-test-cmd: >- echo Setting db_engine_name to "${{ matrix.db_engine_name }}"...; echo -n "${{ matrix.db_engine_name }}" @@ -277,19 +267,15 @@ jobs: echo -n "${{ matrix.connector_version }}" > tests/integration/connector_version; - echo Setting Python version to "${{ matrix.python }}"...; - echo -n "${{ matrix.python }}" - > tests/integration/python; - echo Setting Ansible version to "${{ matrix.ansible }}"...; echo -n "${{ matrix.ansible }}" > tests/integration/ansible - target-python-version: ${{ matrix.python }} testing-type: integration + integration-retry-on-error: false units: runs-on: ubuntu-22.04 - name: Units (Ⓐ${{ matrix.ansible }}) + name: Units (Ⓐ${{ matrix.ansible }}, Python${{ matrix.python }}) strategy: # As soon as the first unit test fails, # cancel the others to free up the CI queue @@ -301,22 +287,46 @@ jobs: - stable-2.17 - devel python: - - 3.8 - - 3.9 + - '3.8' + - '3.9' + - '3.10' + - '3.11' exclude: - - python: '3.8' - ansible: stable-2.15 - python: '3.8' ansible: stable-2.16 + - python: '3.8' ansible: stable-2.17 + - python: '3.8' ansible: devel + - python: '3.9' + ansible: stable-2.15 + + - python: '3.9' + ansible: stable-2.17 + + - python: '3.9' + ansible: devel + + - python: '3.10' + ansible: stable-2.15 + + - python: '3.10' + ansible: stable-2.16 + + - python: '3.11' + ansible: stable-2.15 + + - python: '3.11' + ansible: stable-2.16 + steps: - name: >- Perform unit testing against - Ansible version ${{ matrix.ansible }} + Ansible version ${{ matrix.ansible }} and + python version ${{ matrix.python }} uses: ansible-community/ansible-test-gh-action@release/v1 with: ansible-core-version: ${{ matrix.ansible }} diff --git a/Makefile b/Makefile index 1bf8fae..5a11d1b 100644 --- a/Makefile +++ b/Makefile @@ -8,7 +8,7 @@ endif # This match what GitHub Action will do. Disabled by default. ifdef continue_on_errors - _continue_on_errors = --retry-on-error --continue-on-error + _continue_on_errors = --continue-on-error endif .PHONY: test-integration @@ -17,7 +17,6 @@ test-integration: @echo -n $(db_engine_version) > tests/integration/db_engine_version @echo -n $(connector_name) > tests/integration/connector_name @echo -n $(connector_version) > tests/integration/connector_version - @echo -n $(python) > tests/integration/python @echo -n $(ansible) > tests/integration/ansible # Create podman network for systems missing it. Error can be ignored @@ -55,10 +54,23 @@ test-integration: --health-cmd 'mysqladmin ping -P 3306 -pmsandbox | grep alive || exit 1' \ docker.io/library/$(db_engine_name):$(db_engine_version) \ mysqld - # Setup replication and restart containers - podman exec primary bash -c 'echo -e [mysqld]\\nserver-id=1\\nlog-bin=/var/lib/mysql/primary-bin > /etc/mysql/conf.d/replication.cnf' - podman exec replica1 bash -c 'echo -e [mysqld]\\nserver-id=2\\nlog-bin=/var/lib/mysql/replica1-bin > /etc/mysql/conf.d/replication.cnf' - podman exec replica2 bash -c 'echo -e [mysqld]\\nserver-id=3\\nlog-bin=/var/lib/mysql/replica2-bin > /etc/mysql/conf.d/replication.cnf' + # Setup replication and restart containers using the same subshell to keep variables alive + db_ver=$(db_engine_version); \ + maj="$${db_ver%.*.*}"; \ + maj_min="$${db_ver%.*}"; \ + min="$${maj_min#*.}"; \ + if [[ "$(db_engine_name)" == "mysql" && "$$maj" -eq 8 && "$$min" -ge 2 ]]; then \ + prima_conf='[mysqld]\\nserver-id=1\\nlog-bin=/var/lib/mysql/primary-bin\\nmysql-native-password=1'; \ + repl1_conf='[mysqld]\\nserver-id=2\\nlog-bin=/var/lib/mysql/replica1-bin\\nmysql-native-password=1'; \ + repl2_conf='[mysqld]\\nserver-id=3\\nlog-bin=/var/lib/mysql/replica2-bin\\nmysql-native-password=1'; \ + else \ + prima_conf='[mysqld]\\nserver-id=1\\nlog-bin=/var/lib/mysql/primary-bin'; \ + repl1_conf='[mysqld]\\nserver-id=2\\nlog-bin=/var/lib/mysql/replica1-bin'; \ + repl2_conf='[mysqld]\\nserver-id=3\\nlog-bin=/var/lib/mysql/replica2-bin'; \ + fi; \ + podman exec -e cnf="$$prima_conf" primary bash -c 'echo -e "$${cnf//\\n/\n}" > /etc/mysql/conf.d/replication.cnf'; \ + podman exec -e cnf="$$repl1_conf" replica1 bash -c 'echo -e "$${cnf//\\n/\n}" > /etc/mysql/conf.d/replication.cnf'; \ + podman exec -e cnf="$$repl2_conf" replica2 bash -c 'echo -e "$${cnf//\\n/\n}" > /etc/mysql/conf.d/replication.cnf' # Don't restart a container unless it is healthy while ! podman healthcheck run primary && [[ "$$SECONDS" -lt 120 ]]; do sleep 1; done podman restart -t 30 primary @@ -77,7 +89,7 @@ test-integration: https://github.com/ansible/ansible/archive/$(ansible).tar.gz; \ set -x; \ ansible-test integration $(target) -v --color --coverage --diff \ - --docker --python $(python) \ + --docker ubuntu2204 \ --docker-network podman $(_continue_on_errors) $(_keep_containers_alive); \ set +x # End of venv @@ -86,7 +98,6 @@ test-integration: rm tests/integration/db_engine_version rm tests/integration/connector_name rm tests/integration/connector_version - rm tests/integration/python rm tests/integration/ansible ifndef keep_containers_alive podman stop --time 0 --ignore primary replica1 replica2 diff --git a/README.md b/README.md index 98e90b2..05a7bde 100644 --- a/README.md +++ b/README.md @@ -104,24 +104,35 @@ Here is the table for the support timeline: - stable-2.17 - current development version +### Python + +- 3.8 (Unit tests only) +- 3.9 (Unit tests only) +- 3.10 (Sanity, Units and integrations tests) +- 3.11 (Unit tests only, collection version >= 3.10.0) + ### Databases -For MariaDB, only Long Term releases are tested. +For MariaDB, only Long Term releases are tested. When multiple LTS are available, we test the oldest and the newest only. Usually breaking changes introduced in the versions in between are also present in the latest version. -- mysql 5.7.40 -- mysql 8.0.31 -- mariadb:10.3.34 (only collection version <= 3.5.1) -- mariadb:10.4.24 (only collection version >= 3.5.2) -- mariadb:10.5.18 (only collection version >= 3.5.2) -- mariadb:10.6.11 (only collection version >= 3.5.2) -- mariadb:10.11.?? (waiting for release) +- mysql 5.7.40 (collection version < 3.10.0) +- mysql 8.0.31 (collection version < 3.10.0) +- mysql 8.4.1 (collection version >= 3.10.0) !!! FAILING, no support yet !!! +- mariadb:10.3.34 (collection version < 3.5.1) +- mariadb:10.4.24 (collection version >= 3.5.2, < 3.10.0) +- mariadb:10.5.18 (collection version >= 3.5.2, < 3.10.0) +- mariadb:10.5.25 (collection version >= 3.10.0) +- mariadb:10.6.11 (collection version >= 3.5.2, < 3.10.0) +- mariadb:10.11.8 (collection version >= 3.10.0) ### Database connectors -- pymysql 0.7.11 (Only tested with MySQL 5.7) +- pymysql 0.7.11 (collection version < 3.10 and MySQL 5.7) - pymysql 0.9.3 -- pymysql 1.0.2 (only collection version >= 3.6.1) +- pymysql 0.10.1 (for RHEL8 context) +- pymysql 1.0.2 (collection version >= 3.6.1) +- pymysql 1.1.1 (collection version >= 3.10.0) ## External requirements diff --git a/TESTING.md b/TESTING.md index 54eb5ed..1a22832 100644 --- a/TESTING.md +++ b/TESTING.md @@ -19,7 +19,7 @@ For now, the makefile only supports Podman. ### Requirements -- python >= 3.8 and <= 3.10 +- python >= 3.8 - make - podman - Minimum 15GB of free space on the device storing containers images and volumes. You can use this command to check: `podman system info --format='{{.Store.GraphRoot}}'|xargs findmnt --noheadings --nofsroot --output SOURCE --target|xargs df -h --output=size,used,avail,pcent,target` @@ -41,7 +41,8 @@ The Makefile accept the following options - "3.8" - "3.9" - "3.10" - - Description: If `Python -V` shows an unsupported version, use this option and choose one of the version available on your system. Use `ls /usr/bin/python3*|grep -v config` to list them. + - "3.11" (for stable-2.15+) + - Description: If `Python -V` shows an unsupported version, use this option to select a compatible Python version available on your system. Use `ls /usr/bin/python3*|grep -v config` to list the available versions (You may have to install one). Unsupported versions are those that are too recent for the Ansible version you are using. In such cases, you will see an error message similar to: 'This version of ansible-test cannot be executed with Python version 3.12.3. Supported Python versions are: 3.9, 3.10, 3.11'. - `ansible` - Mandatory: true @@ -62,11 +63,10 @@ The Makefile accept the following options - `db_engine_version` - Mandatory: true - Choices: - - "5.7.40" <- mysql - - "8.0.31" <- mysql - - "10.4.24" <- mariadb - - "10.5.18" <- mariadb - - "10.6.11" <- mariadb + - "8.0.38" <- mysql + - "8.4.1" <- mysql (NOT WORKING YET, ansible-test uses Ubuntu 20.04 which is too old to install mysql-community-client 8.4) + - "10.5.25" <- mariadb + - "10.11.8" <- mariadb - Description: The tag of the container to use for the service containers that will host a primary database and two replicas. Do not use short version, like `mysql:8` (don't do that) because our tests expect a full version to filter tests precisely. For instance: `when: db_version is version ('8.0.22', '>')`. You can use any tag available on [hub.docker.com/_/mysql](https://hub.docker.com/_/mysql) and [hub.docker.com/_/mariadb](https://hub.docker.com/_/mariadb) but GitHub Action will only use the versions listed above. - `connector_name` @@ -79,22 +79,12 @@ The Makefile accept the following options - `connector_version` - Mandatory: true - Choices: - - "0.7.11" <- pymysql (Only for MySQL 5.7) - "0.9.3" <- pymysql + - "0.10.1" <- pymysql - "1.0.2" <- pymysql - - "2.0.1" <- mysqlclient - - "2.0.3" <- mysqlclient - - "2.1.1" <- mysqlclient + - "1.1.1" <- pymysql - Description: The version of the python package of the connector to use. This value is used to filter tests meant for other connectors. -- `python` - - Mandatory: true - - Choices: - - "3.8" - - "3.9" - - "3.10" - - Description: The python version to use in the controller (ansible-test container). - - `target` - Mandatory: false - Choices: @@ -114,30 +104,30 @@ tests will overwrite the 3 databases containers so no need to kill them in advan - `continue_on_errors` - Mandatory: false - - Description: Tells ansible-test to retry on errors and also continue on errors. This is the way the GitHub Action's workflow runs the tests. This can be used to catch all errors in a single run, but you'll need to scroll up to find them. Add any value to activate this option: `continue_on_errors=1` + - Description: Tells ansible-test to continue on errors. This is the way the GitHub Action's workflow runs the tests. This can be used to catch all errors in a single run, but you'll need to scroll up to find them. Add any value to activate this option: `continue_on_errors=1` #### Makefile usage examples: ```sh # Run all targets -make ansible="stable-2.12" db_engine_name="mysql" db_engine_version="5.7.40" python="3.8" connector_name="pymysql" connector_version="0.7.11" +make ansible="stable-2.16" db_engine_name="mysql" db_engine_version="8.0.31" connector_name="pymysql" connector_version="1.0.2" # A single target -make ansible="stable-2.14" db_engine_name="mysql" db_engine_version="5.7.40" python="3.8" connector_name="pymysql" connector_version="0.7.11" target="test_mysql_info" +make ansible="stable-2.16" db_engine_name="mysql" db_engine_version="8.0.31" connector_name="pymysql" connector_version="1.0.2" target="test_mysql_info" # Keep databases and ansible tests containers alives # A single target and continue on errors -make ansible="stable-2.14" db_engine_name="mysql" db_engine_version="8.0.31" python="3.9" connector_name="mysqlclient" connector_version="2.0.3" target="test_mysql_query" keep_containers_alive=1 continue_on_errors=1 +make ansible="stable-2.17" db_engine_name="mysql" db_engine_version="8.0.31" connector_name="mysqlclient" connector_version="2.0.3" target="test_mysql_query" keep_containers_alive=1 continue_on_errors=1 # If your system has an usupported version of Python: -make local_python_version="3.8" ansible="stable-2.14" db_engine_name="mariadb" db_engine_version="10.6.11" python="3.9" connector_name="pymysql" connector_version="0.9.3" +make local_python_version="3.10" ansible="stable-2.17" db_engine_name="mariadb" db_engine_version="10.6.11" connector_name="pymysql" connector_version="1.0.2" ``` ### Run all tests -GitHub Action offer a test matrix that run every combination of Python, MySQL, MariaDB and Connector against each other. To reproduce this, this repo provides a script called *run_all_tests.py*. +GitHub Action offer a test matrix that run every combination of MySQL, MariaDB and Connector against each other. To reproduce this, this repo provides a script called *run_all_tests.py*. Examples: @@ -146,8 +136,8 @@ python run_all_tests.py ``` -### Add a new Python, Connector or Database version +### Add a new Connector or Database version New components version should be added to this file: [.github/workflows/ansible-test-plugins.yml](https://github.com/ansible-collections/community.mysql/tree/main/.github/workflows) -Be careful to not add too much tests. When adding a new version of Python, for instance, only test it agains the latest versions of Ansible and MySQL/MariaDB. When tests are run, you can see that we already start 40 virtual machines! +Be careful to not add too much tests. The matrix creates an exponential number of virtual machines! diff --git a/tests/integration/targets/setup_controller/tasks/requirements.yml b/tests/integration/targets/setup_controller/tasks/requirements.yml index 8bab1a0..c939098 100644 --- a/tests/integration/targets/setup_controller/tasks/requirements.yml +++ b/tests/integration/targets/setup_controller/tasks/requirements.yml @@ -1,5 +1,7 @@ --- +# We use the ubuntu2204 image provided by ansible-test. + - name: "{{ role_name }} | Requirements | Install Linux packages" ansible.builtin.package: name: diff --git a/tests/integration/targets/setup_controller/tasks/setvars.yml b/tests/integration/targets/setup_controller/tasks/setvars.yml index 7c3e03b..0bb8c0e 100644 --- a/tests/integration/targets/setup_controller/tasks/setvars.yml +++ b/tests/integration/targets/setup_controller/tasks/setvars.yml @@ -32,11 +32,6 @@ 'file', '/root/ansible_collections/community/mysql/tests/integration/db_engine_version' ) }} - python_version_lookup: >- - {{ lookup( - 'file', - '/root/ansible_collections/community/mysql/tests/integration/python' - ) }} ansible_version_lookup: >- {{ lookup( 'file', @@ -49,7 +44,6 @@ connector_version: "{{ connector_version_lookup.strip() }}" db_engine: "{{ db_engine_name_lookup.strip() }}" db_version: "{{ db_engine_version_lookup.strip() }}" - python_version: "{{ python_version_lookup.strip() }}" test_ansible_version: >- {%- if ansible_version_lookup == 'devel' -%} {{ ansible_version_lookup }} @@ -77,7 +71,6 @@ connector_version: {{ connector_version }} db_engine: {{ db_engine }} db_version: {{ db_version }} - python_version: {{ python_version }} test_ansible_version: {{ test_ansible_version }} ansible.builtin.debug: msg: "{{ msg.split('\n') }}" diff --git a/tests/integration/targets/setup_controller/tasks/verify.yml b/tests/integration/targets/setup_controller/tasks/verify.yml index e5b4c94..b47e354 100644 --- a/tests/integration/targets/setup_controller/tasks/verify.yml +++ b/tests/integration/targets/setup_controller/tasks/verify.yml @@ -41,16 +41,20 @@ when: - connector_name == 'mysqlclient' - - name: Display the python version in use - command: - cmd: python{{ python_version }} -V + - name: Get the python version in use + ansible.builtin.command: + cmd: python -V changed_when: false - register: python_in_use + failed_when: false + register: python_version_in_use - - name: Assert that expected Python is installed - assert: - that: - - python_in_use.stdout is search(python_version) + - name: Display the python version in use + ansible.builtin.debug: + msg: > + Python in use inside the test container: + ${{ python_version_in_use }} + when: + - python_version_in_use is defined - name: Assert that we run the expected ansible version assert: From cd9f4fcf57bd9d80340148798c383ee702bb4ae1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Mon, 5 Aug 2024 08:55:18 +0200 Subject: [PATCH 19/54] Fix deprecated options from MySQL 8.2 (#662) * Fix show master status for MySQL 8.2+ * Fix mysqldump option form --master-data to --source-data * Fix incompatibility between mysqldump 8.0 and MySQL 8.4 Installing the same version between the client and the server makes sense anyway. The incompatibility arise when you use mysqldump with --source-data. The the tool tries to perform a SHOW MASTER STATUS which is deprecated in MySQL 8.2+. * Fix missing condition * Fix unit tests * Add a query resolver depending on implementation and version * Sanity * Fix SHOW REPLICA STATUS queries * Fix mariadb's SHOW REPLICA HOSTS query * Fix CHANGE MASTER for MySQL 8.0.23+ * Fix integration test for CHANGE MASTER * Fix integration test for CHANGE MASTER * Fix replication queries for MySQL 8.0.23+ and 8.4+ * Revert file edited by mistake * Enhance tests format --- plugins/module_utils/command_resolver.py | 180 ++++++++++++++++++ plugins/modules/mysql_db.py | 25 ++- plugins/modules/mysql_info.py | 23 ++- plugins/modules/mysql_replication.py | 84 ++++---- .../targets/setup_controller/files/mysql.gpg | 49 +++++ .../setup_controller/tasks/requirements.yml | 32 ++++ .../test_mysql_db/tasks/state_dump_import.yml | 15 +- .../test_mysql_replication/tasks/main.yml | 5 +- .../tasks/mysql_replication_channel.yml | 31 ++- .../tasks/mysql_replication_initial.yml | 59 ++++-- .../tasks/mysql_replication_primary_delay.yml | 20 +- .../mysql_replication_resetprimary_mode.yml | 21 +- .../module_utils/test_command_resolver.py | 39 ++++ tests/unit/plugins/modules/test_mysql_info.py | 14 +- 14 files changed, 503 insertions(+), 94 deletions(-) create mode 100644 plugins/module_utils/command_resolver.py create mode 100644 tests/integration/targets/setup_controller/files/mysql.gpg create mode 100644 tests/unit/plugins/module_utils/test_command_resolver.py diff --git a/plugins/module_utils/command_resolver.py b/plugins/module_utils/command_resolver.py new file mode 100644 index 0000000..4374879 --- /dev/null +++ b/plugins/module_utils/command_resolver.py @@ -0,0 +1,180 @@ +# -*- coding: utf-8 -*- + +from __future__ import (absolute_import, division, print_function) +from ._version import LooseVersion +__metaclass__ = type + + +class CommandResolver(): + def __init__(self, server_implementation, server_version): + self.server_implementation = server_implementation + self.server_version = LooseVersion(server_version) + + def resolve_command(self, command): + """ + Resolves the appropriate SQL command based on the server implementation and version. + + Parameters: + command (str): The base SQL command to be resolved (e.g., "SHOW SLAVE HOSTS"). + + Returns: + str: The resolved SQL command suitable for the given server implementation and version. + + Raises: + ValueError: If the command is not supported or recognized. + + Example: + Given a server implementation `mysql` and server version `8.0.23`, and a command `SHOW SLAVE HOSTS`, + the method will resolve the command based on the following table of versions: + + Table: + [ + ("mysql", "default", "SHOW SLAVES HOSTS default"), + ("mysql", "5.7.0", "SHOW SLAVES HOSTS"), + ("mysql", "8.0.22", "SHOW REPLICAS"), + ("mysql", "8.4.0", "SHOW REPLICAS 8.4"), + ("mariadb", "10.5.1", "SHOW REPLICAS HOSTS"), + ] + + Example usage: + >>> resolver = CommandResolver("mysql", "8.0.23") + >>> resolver.resolve_command("SHOW SLAVE HOSTS") + 'SHOW REPLICAS' + + In this example, the resolver will: + - Filter and sort applicable versions: [ + ("8.4.0", "SHOW REPLICAS 8.4"), + ("8.0.22", "HOW REPLICAS"), + ("5.7.0", "SHOW SLAVES HOSTS") + ] + + - Iterate through the sorted list and find the first version less than or equal to 8.0.23, + which is 8.0.22, and return the corresponding command. + """ + + # Convert the command to uppercase to ensure case-insensitive lookup + command = command.upper() + + commands = { + "SHOW MASTER STATUS": { + ("mysql", "default"): "SHOW MASTER STATUS", + ("mariadb", "default"): "SHOW MASTER STATUS", + ("mysql", "8.2.0"): "SHOW BINARY LOG STATUS", + ("mariadb", "10.5.2"): "SHOW BINLOG STATUS", + }, + "SHOW SLAVE STATUS": { + ("mysql", "default"): "SHOW SLAVE STATUS", + ("mariadb", "default"): "SHOW SLAVE STATUS", + ("mysql", "8.0.22"): "SHOW REPLICA STATUS", + ("mariadb", "10.5.1"): "SHOW REPLICA STATUS", + }, + "SHOW SLAVE HOSTS": { + ("mysql", "default"): "SHOW SLAVE HOSTS", + ("mariadb", "default"): "SHOW SLAVE HOSTS", + ("mysql", "8.0.22"): "SHOW REPLICAS", + ("mariadb", "10.5.1"): "SHOW REPLICA HOSTS", + }, + "CHANGE MASTER": { + ("mysql", "default"): "CHANGE MASTER", + ("mariadb", "default"): "CHANGE MASTER", + ("mysql", "8.0.23"): "CHANGE REPLICATION SOURCE", + }, + "MASTER_HOST": { + ("mysql", "default"): "MASTER_HOST", + ("mariadb", "default"): "MASTER_HOST", + ("mysql", "8.0.23"): "SOURCE_HOST", + }, + "MASTER_USER": { + ("mysql", "default"): "MASTER_USER", + ("mariadb", "default"): "MASTER_USER", + ("mysql", "8.0.23"): "SOURCE_USER", + }, + "MASTER_PASSWORD": { + ("mysql", "default"): "MASTER_PASSWORD", + ("mariadb", "default"): "MASTER_PASSWORD", + ("mysql", "8.0.23"): "SOURCE_PASSWORD", + }, + "MASTER_PORT": { + ("mysql", "default"): "MASTER_PORT", + ("mariadb", "default"): "MASTER_PORT", + ("mysql", "8.0.23"): "SOURCE_PORT", + }, + "MASTER_CONNECT_RETRY": { + ("mysql", "default"): "MASTER_CONNECT_RETRY", + ("mariadb", "default"): "MASTER_CONNECT_RETRY", + ("mysql", "8.0.23"): "SOURCE_CONNECT_RETRY", + }, + "MASTER_LOG_FILE": { + ("mysql", "default"): "MASTER_LOG_FILE", + ("mariadb", "default"): "MASTER_LOG_FILE", + ("mysql", "8.0.23"): "SOURCE_LOG_FILE", + }, + "MASTER_LOG_POS": { + ("mysql", "default"): "MASTER_LOG_POS", + ("mariadb", "default"): "MASTER_LOG_POS", + ("mysql", "8.0.23"): "SOURCE_LOG_POS", + }, + "MASTER_DELAY": { + ("mysql", "default"): "MASTER_DELAY", + ("mariadb", "default"): "MASTER_DELAY", + ("mysql", "8.0.23"): "SOURCE_DELAY", + }, + "MASTER_SSL": { + ("mysql", "default"): "MASTER_SSL", + ("mariadb", "default"): "MASTER_SSL", + ("mysql", "8.0.23"): "SOURCE_SSL", + }, + "MASTER_SSL_CA": { + ("mysql", "default"): "MASTER_SSL_CA", + ("mariadb", "default"): "MASTER_SSL_CA", + ("mysql", "8.0.23"): "SOURCE_SSL_CA", + }, + "MASTER_SSL_CAPATH": { + ("mysql", "default"): "MASTER_SSL_CAPATH", + ("mariadb", "default"): "MASTER_SSL_CAPATH", + ("mysql", "8.0.23"): "SOURCE_SSL_CAPATH", + }, + "MASTER_SSL_CERT": { + ("mysql", "default"): "MASTER_SSL_CERT", + ("mariadb", "default"): "MASTER_SSL_CERT", + ("mysql", "8.0.23"): "SOURCE_SSL_CERT", + }, + "MASTER_SSL_KEY": { + ("mysql", "default"): "MASTER_SSL_KEY", + ("mariadb", "default"): "MASTER_SSL_KEY", + ("mysql", "8.0.23"): "SOURCE_SSL_KEY", + }, + "MASTER_SSL_CIPHER": { + ("mysql", "default"): "MASTER_SSL_CIPHER", + ("mariadb", "default"): "MASTER_SSL_CIPHER", + ("mysql", "8.0.23"): "SOURCE_SSL_CIPHER", + }, + "MASTER_SSL_VERIFY_SERVER_CERT": { + ("mysql", "default"): "MASTER_SSL_VERIFY_SERVER_CERT", + ("mariadb", "default"): "MASTER_SSL_VERIFY_SERVER_CERT", + ("mysql", "8.0.23"): "SOURCE_SSL_VERIFY_SERVER_CERT", + }, + "MASTER_AUTO_POSITION": { + ("mysql", "default"): "MASTER_AUTO_POSITION", + ("mariadb", "default"): "MASTER_AUTO_POSITION", + ("mysql", "8.0.23"): "SOURCE_AUTO_POSITION", + }, + "RESET MASTER": { + ("mysql", "default"): "RESET MASTER", + ("mariadb", "default"): "RESET MASTER", + ("mysql", "8.4.0"): "RESET BINARY LOGS AND GTIDS", + }, + # Add more command mappings here + } + + if command in commands: + cmd_syntaxes = commands[command] + applicable_versions = [(v, cmd) for (impl, v), cmd in cmd_syntaxes.items() if impl == self.server_implementation and v != 'default'] + applicable_versions.sort(reverse=True, key=lambda x: LooseVersion(x[0])) + + for version, cmd in applicable_versions: + if self.server_version >= LooseVersion(version): + return cmd + + return cmd_syntaxes[(self.server_implementation, "default")] + raise ValueError("Unsupported command: %s" % command) diff --git a/plugins/modules/mysql_db.py b/plugins/modules/mysql_db.py index 8742f3c..4a2c954 100644 --- a/plugins/modules/mysql_db.py +++ b/plugins/modules/mysql_db.py @@ -343,7 +343,15 @@ import traceback from ansible.module_utils.basic import AnsibleModule from ansible_collections.community.mysql.plugins.module_utils.database import mysql_quote_identifier -from ansible_collections.community.mysql.plugins.module_utils.mysql import mysql_connect, mysql_driver, mysql_driver_fail_msg, mysql_common_argument_spec +from ansible_collections.community.mysql.plugins.module_utils.mysql import ( + mysql_connect, + mysql_driver, + mysql_driver_fail_msg, + mysql_common_argument_spec, + get_server_implementation, + get_server_version, +) +from ansible_collections.community.mysql.plugins.module_utils.version import LooseVersion from ansible.module_utils.six.moves import shlex_quote from ansible.module_utils._text import to_native @@ -372,7 +380,8 @@ def db_delete(cursor, db): def db_dump(module, host, user, password, db_name, target, all_databases, port, - config_file, socket=None, ssl_cert=None, ssl_key=None, ssl_ca=None, + config_file, server_implementation, server_version, socket=None, + ssl_cert=None, ssl_key=None, ssl_ca=None, single_transaction=None, quick=None, ignore_tables=None, hex_blob=None, encoding=None, force=False, master_data=0, skip_lock_tables=False, dump_extra_args=None, unsafe_password=False, restrict_config_file=False, @@ -431,7 +440,11 @@ def db_dump(module, host, user, password, db_name, target, all_databases, port, if hex_blob: cmd += " --hex-blob" if master_data: - cmd += " --master-data=%s" % master_data + if (server_implementation == 'mysql' and + LooseVersion(server_version) >= LooseVersion("8.2.0")): + cmd += " --source-data=%s" % master_data + else: + cmd += " --master-data=%s" % master_data if dump_extra_args is not None: cmd += " " + dump_extra_args @@ -690,6 +703,9 @@ def main(): else: module.fail_json(msg="unable to find %s. Exception message: %s" % (config_file, to_native(e))) + server_implementation = get_server_implementation(cursor) + server_version = get_server_version(cursor) + changed = False if not os.path.exists(config_file): config_file = None @@ -730,7 +746,8 @@ def main(): module.exit_json(changed=True, db=db_name, db_list=db) rc, stdout, stderr = db_dump(module, login_host, login_user, login_password, db, target, all_databases, - login_port, config_file, socket, ssl_cert, ssl_key, + login_port, config_file, server_implementation, server_version, + socket, ssl_cert, ssl_key, ssl_ca, single_transaction, quick, ignore_tables, hex_blob, encoding, force, master_data, skip_lock_tables, dump_extra_args, unsafe_login_password, restrict_config_file, diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index d8bc88c..2d1fe94 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -293,6 +293,9 @@ connector_version: from decimal import Decimal from ansible.module_utils.basic import AnsibleModule +from ansible_collections.community.mysql.plugins.module_utils.command_resolver import ( + CommandResolver +) from ansible_collections.community.mysql.plugins.module_utils.mysql import ( mysql_connect, mysql_common_argument_spec, @@ -301,6 +304,7 @@ from ansible_collections.community.mysql.plugins.module_utils.mysql import ( get_connector_name, get_connector_version, get_server_implementation, + get_server_version, ) from ansible_collections.community.mysql.plugins.module_utils.user import ( @@ -335,11 +339,13 @@ class MySQL_Info(object): 5. add info about the new subset with an example to RETURN block """ - def __init__(self, module, cursor, server_implementation, user_implementation): + def __init__(self, module, cursor, server_implementation, server_version, user_implementation): self.module = module self.cursor = cursor self.server_implementation = server_implementation + self.server_version = server_version self.user_implementation = user_implementation + self.command_resolver = CommandResolver(self.server_implementation, self.server_version) self.info = { 'version': {}, 'databases': {}, @@ -501,7 +507,8 @@ class MySQL_Info(object): def __get_master_status(self): """Get master status if the instance is a master.""" - res = self.__exec_sql('SHOW MASTER STATUS') + query = self.command_resolver.resolve_command("SHOW MASTER STATUS") + res = self.__exec_sql(query) if res: for line in res: for vname, val in iteritems(line): @@ -509,10 +516,8 @@ class MySQL_Info(object): def __get_slave_status(self): """Get slave status if the instance is a slave.""" - if self.server_implementation == "mariadb": - res = self.__exec_sql('SHOW ALL SLAVES STATUS') - else: - res = self.__exec_sql('SHOW SLAVE STATUS') + query = self.command_resolver.resolve_command("SHOW SLAVE STATUS") + res = self.__exec_sql(query) if res: for line in res: host = line['Master_Host'] @@ -533,7 +538,8 @@ class MySQL_Info(object): def __get_slaves(self): """Get slave hosts info if the instance is a master.""" - res = self.__exec_sql('SHOW SLAVE HOSTS') + query = self.command_resolver.resolve_command("SHOW SLAVE HOSTS") + res = self.__exec_sql(query) if res: for line in res: srv_id = line['Server_id'] @@ -762,12 +768,13 @@ def main(): module.fail_json(msg) server_implementation = get_server_implementation(cursor) + server_version = get_server_version(cursor) user_implementation = get_user_implementation(cursor) ############################### # Create object and do main job - mysql = MySQL_Info(module, cursor, server_implementation, user_implementation) + mysql = MySQL_Info(module, cursor, server_implementation, server_version, user_implementation) module.exit_json(changed=False, server_engine='MariaDB' if server_implementation == 'mariadb' else 'MySQL', diff --git a/plugins/modules/mysql_replication.py b/plugins/modules/mysql_replication.py index b0caf11..723fc35 100644 --- a/plugins/modules/mysql_replication.py +++ b/plugins/modules/mysql_replication.py @@ -20,11 +20,12 @@ author: - Balazs Pocze (@banyek) - Andrew Klychkov (@Andersson007) - Dennis Urtubia (@dennisurtubia) +- Laurent Indermühle (@laurent-indermuehle) options: mode: description: - Module operating mode. Could be - C(changeprimary) (CHANGE MASTER TO), + C(changeprimary) (CHANGE MASTER TO) - also works for MySQL 8.0.23 and later since community.mysql 3.10.0, C(changereplication) (CHANGE REPLICATION SOURCE TO) - only supported in MySQL 8.0.23 and later, C(getprimary) (SHOW MASTER STATUS), C(getreplica) (SHOW REPLICA STATUS), @@ -298,8 +299,10 @@ queries: import os import warnings -from ansible_collections.community.mysql.plugins.module_utils.version import LooseVersion from ansible.module_utils.basic import AnsibleModule +from ansible_collections.community.mysql.plugins.module_utils.command_resolver import ( + CommandResolver +) from ansible_collections.community.mysql.plugins.module_utils.mysql import ( get_server_version, get_server_implementation, @@ -313,18 +316,9 @@ from ansible.module_utils._text import to_native executed_queries = [] -def get_primary_status(cursor): - term = "MASTER" - - version = get_server_version(cursor) - server_implementation = get_server_implementation(cursor) - if server_implementation == "mysql" and LooseVersion(version) >= LooseVersion("8.2.0"): - term = "BINARY LOG" - - if server_implementation == "mariadb" and LooseVersion(version) >= LooseVersion("10.5.2"): - term = "BINLOG" - - cursor.execute("SHOW %s STATUS" % term) +def get_primary_status(cursor, command_resolver): + query = command_resolver.resolve_command("SHOW MASTER STATUS") + cursor.execute(query) primarystatus = cursor.fetchone() return primarystatus @@ -410,8 +404,8 @@ def reset_replica_all(module, cursor, connection_name='', channel='', fail_on_er return reset -def reset_primary(module, cursor, fail_on_error=False): - query = 'RESET MASTER' +def reset_primary(module, cursor, command_resolver, fail_on_error=False): + query = command_resolver.resolve_command('RESET MASTER') try: executed_queries.append(query) cursor.execute(query) @@ -420,7 +414,7 @@ def reset_primary(module, cursor, fail_on_error=False): reset = False except Exception as e: if fail_on_error: - module.fail_json(msg="RESET MASTER failed: %s" % to_native(e)) + module.fail_json(msg="%s failed: %s" % (command_resolver.resolve_command('RESET MASTER'), to_native(e))) reset = False return reset @@ -447,11 +441,12 @@ def start_replica(module, cursor, connection_name='', channel='', fail_on_error= return started -def changeprimary(cursor, chm, connection_name='', channel=''): +def changeprimary(cursor, command_resolver, chm, connection_name='', channel=''): + query_head = command_resolver.resolve_command("CHANGE MASTER") if connection_name: - query = "CHANGE MASTER '%s' TO %s" % (connection_name, ','.join(chm)) + query = "%s '%s' TO %s" % (query_head, connection_name, ','.join(chm)) else: - query = 'CHANGE MASTER TO %s' % ','.join(chm) + query = '%s TO %s' % (query_head, ','.join(chm)) if channel: query += " FOR CHANNEL '%s'" % channel @@ -566,8 +561,11 @@ def main(): else: module.fail_json(msg="unable to find %s. Exception message: %s" % (config_file, to_native(e))) + server_version = get_server_version(cursor) + server_implementation = get_server_implementation(cursor) + command_resolver = CommandResolver(server_implementation, server_version) cursor.execute("SELECT VERSION()") - if 'mariadb' in cursor.fetchone()["VERSION()"].lower(): + if server_implementation == 'mariadb': from ansible_collections.community.mysql.plugins.module_utils.implementations.mariadb import replication as impl else: from ansible_collections.community.mysql.plugins.module_utils.implementations.mysql import replication as impl @@ -582,7 +580,7 @@ def main(): primary_use_gtid = 'slave_pos' if mode == 'getprimary': - status = get_primary_status(cursor) + status = get_primary_status(cursor, command_resolver) if status and "File" in status and "Position" in status: status['Is_Primary'] = True else: @@ -610,52 +608,52 @@ def main(): chm = [] result = {} if primary_host is not None: - chm.append("MASTER_HOST='%s'" % primary_host) + chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_HOST'), primary_host)) if primary_user is not None: - chm.append("MASTER_USER='%s'" % primary_user) + chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_USER'), primary_user)) if primary_password is not None: - chm.append("MASTER_PASSWORD='%s'" % primary_password) + chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_PASSWORD'), primary_password)) if primary_port is not None: - chm.append("MASTER_PORT=%s" % primary_port) + chm.append("%s=%s" % (command_resolver.resolve_command('MASTER_PORT'), primary_port)) if primary_connect_retry is not None: - chm.append("MASTER_CONNECT_RETRY=%s" % primary_connect_retry) + chm.append("%s=%s" % (command_resolver.resolve_command('MASTER_CONNECT_RETRY'), primary_connect_retry)) if primary_log_file is not None: - chm.append("MASTER_LOG_FILE='%s'" % primary_log_file) + chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_LOG_FILE'), primary_log_file)) if primary_log_pos is not None: - chm.append("MASTER_LOG_POS=%s" % primary_log_pos) + chm.append("%s=%s" % (command_resolver.resolve_command('MASTER_LOG_POS'), primary_log_pos)) if primary_delay is not None: - chm.append("MASTER_DELAY=%s" % primary_delay) + chm.append("%s=%s" % (command_resolver.resolve_command('MASTER_DELAY'), primary_delay)) if relay_log_file is not None: chm.append("RELAY_LOG_FILE='%s'" % relay_log_file) if relay_log_pos is not None: chm.append("RELAY_LOG_POS=%s" % relay_log_pos) if primary_ssl is not None: if primary_ssl: - chm.append("MASTER_SSL=1") + chm.append("%s=1" % command_resolver.resolve_command('MASTER_SSL')) else: - chm.append("MASTER_SSL=0") + chm.append("%s=0" % command_resolver.resolve_command('MASTER_SSL')) if primary_ssl_ca is not None: - chm.append("MASTER_SSL_CA='%s'" % primary_ssl_ca) + chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_SSL_CA'), primary_ssl_ca)) if primary_ssl_capath is not None: - chm.append("MASTER_SSL_CAPATH='%s'" % primary_ssl_capath) + chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_SSL_CAPATH'), primary_ssl_capath)) if primary_ssl_cert is not None: - chm.append("MASTER_SSL_CERT='%s'" % primary_ssl_cert) + chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_SSL_CERT'), primary_ssl_cert)) if primary_ssl_key is not None: - chm.append("MASTER_SSL_KEY='%s'" % primary_ssl_key) + chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_SSL_KEY'), primary_ssl_key)) if primary_ssl_cipher is not None: - chm.append("MASTER_SSL_CIPHER='%s'" % primary_ssl_cipher) + chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_SSL_CIPHER'), primary_ssl_cipher)) if primary_ssl_verify_server_cert: - chm.append("SOURCE_SSL_VERIFY_SERVER_CERT=1") + chm.append("%s=1" % command_resolver.resolve_command('MASTER_SSL_VERIFY_SERVER_CERT')) if primary_auto_position: - chm.append("MASTER_AUTO_POSITION=1") + chm.append("%s=1" % command_resolver.resolve_command('MASTER_AUTO_POSITION')) if primary_use_gtid is not None: - chm.append("MASTER_USE_GTID=%s" % primary_use_gtid) + chm.append("MASTER_USE_GTID=%s" % primary_use_gtid) # MariaDB only try: - changeprimary(cursor, chm, connection_name, channel) + changeprimary(cursor, command_resolver, chm, connection_name, channel) except mysql_driver.Warning as e: result['warning'] = to_native(e) except Exception as e: - module.fail_json(msg='%s. Query == CHANGE MASTER TO %s' % (to_native(e), chm)) + module.fail_json(msg='%s. Query == %s TO %s' % (to_native(e), command_resolver.resolve_command('CHANGE MASTER'), chm)) result['changed'] = True module.exit_json(queries=executed_queries, **result) elif mode == "startreplica": @@ -671,7 +669,7 @@ def main(): else: module.exit_json(msg="Replica already stopped", changed=False, queries=executed_queries) elif mode == 'resetprimary': - reset = reset_primary(module, cursor, fail_on_error) + reset = reset_primary(module, cursor, command_resolver, fail_on_error) if reset is True: module.exit_json(msg="Primary reset", changed=True, queries=executed_queries) else: diff --git a/tests/integration/targets/setup_controller/files/mysql.gpg b/tests/integration/targets/setup_controller/files/mysql.gpg new file mode 100644 index 0000000..117f1e7 --- /dev/null +++ b/tests/integration/targets/setup_controller/files/mysql.gpg @@ -0,0 +1,49 @@ +-----BEGIN PGP PUBLIC KEY BLOCK----- +Version: SKS 1.1.6 +Comment: Hostname: pgp.mit.edu + +mQINBGU2rNoBEACSi5t0nL6/Hj3d0PwsbdnbY+SqLUIZ3uWZQm6tsNhvTnahvPPZBGdl99iW +YTt2KmXp0KeN2s9pmLKkGAbacQP1RqzMFnoHawSMf0qTUVjAvhnI4+qzMDjTNSBq9fa3nHmO +YxownnrRkpiQUM/yD7/JmVENgwWb6akZeGYrXch9jd4XV3t8OD6TGzTedTki0TDNr6YZYhC7 +jUm9fK9Zs299pzOXSxRRNGd+3H9gbXizrBu4L/3lUrNf//rM7OvV9Ho7u9YYyAQ3L3+OABK9 +FKHNhrpi8Q0cbhvWkD4oCKJ+YZ54XrOG0YTg/YUAs5/3//FATI1sWdtLjJ5pSb0onV3LIbar +RTN8lC4Le/5kd3lcot9J8b3EMXL5p9OGW7wBfmNVRSUI74Vmwt+v9gyp0Hd0keRCUn8lo/1V +0YD9i92KsE+/IqoYTjnya/5kX41jB8vr1ebkHFuJ404+G6ETd0owwxq64jLIcsp/GBZHGU0R +KKAo9DRLH7rpQ7PVlnw8TDNlOtWt5EJlBXFcPL+NgWbqkADAyA/XSNeWlqonvPlYfmasnAHA +pMd9NhPQhC7hJTjCiAwG8UyWpV8Dj07DHFQ5xBbkTnKH2OrJtguPqSNYtTASbsWz09S8ujoT +DXFT17NbFM2dMIiq0a4VQB3SzH13H2io9Cbg/TzJrJGmwgoXgwARAQABtDZNeVNRTCBSZWxl +YXNlIEVuZ2luZWVyaW5nIDxteXNxbC1idWlsZEBvc3Mub3JhY2xlLmNvbT6JAlQEEwEIAD4W +IQS8pDQXw7SF3RKOxtS3s7eIqNN4XAUCZTas2gIbAwUJA8JnAAULCQgHAgYVCgkICwIEFgID +AQIeAQIXgAAKCRC3s7eIqNN4XLzoD/9PlpWtfHlI8eQTHwGsGIwFA+fgipyDElapHw3MO+K9 +VOEYRZCZSuBXHJe9kjGEVCGUDrfImvgTuNuqYmVUV+wyhP+w46W/cWVkqZKAW0hNp0TTvu3e +Dwap7gdk80VF24Y2Wo0bbiGkpPiPmB59oybGKaJ756JlKXIL4hTtK3/hjIPFnb64Ewe4YLZy +oJu0fQOyA8gXuBoalHhUQTbRpXI0XI3tpZiQemNbfBfJqXo6LP3/LgChAuOfHIQ8alvnhCwx +hNUSYGIRqx+BEbJw1X99Az8XvGcZ36VOQAZztkW7mEfH9NDPz7MXwoEvduc61xwlMvEsUIaS +fn6SGLFzWPClA98UMSJgF6sKb+JNoNbzKaZ8V5w13msLb/pq7hab72HH99XJbyKNliYj3+KA +3q0YLf+Hgt4Y4EhIJ8x2+g690Np7zJF4KXNFbi1BGloLGm78akY1rQlzpndKSpZq5KWw8FY/ +1PEXORezg/BPD3Etp0AVKff4YdrDlOkNB7zoHRfFHAvEuuqti8aMBrbRnRSG0xunMUOEhbYS +/wOOTl0g3bF9NpAkfU1Fun57N96Us2T9gKo9AiOY5DxMe+IrBg4zaydEOovgqNi2wbU0MOBQ +b23Puhj7ZCIXcpILvcx9ygjkONr75w+XQrFDNeux4Znzay3ibXtAPqEykPMZHsZ2sbkCDQRl +NqzaARAAsdvBo8WRqZ5WVVk6lReD8b6Zx83eJUkV254YX9zn5t8KDRjYOySwS75mJIaZLsv0 +YQjJk+5rt10tejyCrJIFo9CMvCmjUKtVbgmhfS5+fUDRrYCEZBBSa0Dvn68EBLiHugr+SPXF +6o1hXEUqdMCpB6oVp6X45JVQroCKIH5vsCtw2jU8S2/IjjV0V+E/zitGCiZaoZ1f6NG7ozyF +ep1CSAReZu/sssk0pCLlfCebRd9Rz3QjSrQhWYuJa+eJmiF4oahnpUGktxMD632I9aG+IMfj +tNJNtX32MbO+Se+cCtVc3cxSa/pR+89a3cb9IBA5tFF2Qoekhqo/1mmLi93Xn6uDUhl5tVxT +nB217dBT27tw+p0hjd9hXZRQbrIZUTyh3+8EMfmAjNSIeR+th86xRd9XFRr9EOqrydnALOUr +9cT7TfXWGEkFvn6ljQX7f4RvjJOTbc4jJgVFyu8K+VU6u1NnFJgDiNGsWvnYxAf7gDDbUSXE +uC2anhWvxPvpLGmsspngge4yl+3nv+UqZ9sm6LCebR/7UZ67tYz3p6xzAOVgYsYcxoIUuEZX +jHQtsYfTZZhrjUWBJ09jrMvlKUHLnS437SLbgoXVYZmcqwAWpVNOLZf+fFm4IE5aGBG5Dho2 +CZ6ujngW9Zkn98T1d4N0MEwwXa2V6T1ijzcqD7GApZUAEQEAAYkCPAQYAQgAJhYhBLykNBfD +tIXdEo7G1Lezt4io03hcBQJlNqzaAhsMBQkDwmcAAAoJELezt4io03hcXqMP/01aPT3A3Sg7 +oTQoHdCxj04ELkzrezNWGM+YwbSKrR2LoXR8zf2tBFzc2/Tl98V0+68f/eCvkvqCuOtq4392 +Ps23j9W3r5XG+GDOwDsx0gl0E+Qkw07pwdJctA6efsmnRkjF2YVO0N9MiJA1tc8NbNXpEEHJ +Z7F8Ri5cpQrGUz/AY0eae2b7QefyP4rpUELpMZPjc8Px39Fe1DzRbT+5E19TZbrpbwlSYs1i +CzS5YGFmpCRyZcLKXo3zS6N22+82cnRBSPPipiO6WaQawcVMlQO1SX0giB+3/DryfN9VuIYd +1EWCGQa3O0MVu6o5KVHwPgl9R1P6xPZhurkDpAd0b1s4fFxin+MdxwmG7RslZA9CXRPpzo7/ +fCMW8sYOH15DP+YfUckoEreBt+zezBxbIX2CGGWEV9v3UBXadRtwxYQ6sN9bqW4jm1b41vNA +17b6CVH6sVgtU3eN+5Y9an1e5jLD6kFYx+OIeqIIId/TEqwS61csY9aav4j4KLOZFCGNU0FV +ji7NQewSpepTcJwfJDOzmtiDP4vol1ApJGLRwZZZ9PB6wsOgDOoP6sr0YrDI/NNX2RyXXbgl +nQ1yJZVSH3/3eo6knG2qTthUKHCRDNKdy9Qqc1x4WWWtSRjh+zX8AvJK2q1rVLH2/3ilxe9w +cAZUlaj3id3TxquAlud4lWDz +=h5nH +-----END PGP PUBLIC KEY BLOCK----- diff --git a/tests/integration/targets/setup_controller/tasks/requirements.yml b/tests/integration/targets/setup_controller/tasks/requirements.yml index c939098..a576ce4 100644 --- a/tests/integration/targets/setup_controller/tasks/requirements.yml +++ b/tests/integration/targets/setup_controller/tasks/requirements.yml @@ -2,6 +2,38 @@ # We use the ubuntu2204 image provided by ansible-test. +# The GPG key is imported in the files folder from: +# https://dev.mysql.com/doc/refman/8.4/en/checking-gpg-signature.html +# Downloading the key on each iteration of the tests is too slow. +- name: Install MySQL PGP public key + ansible.builtin.copy: + src: files/mysql.gpg + dest: /usr/share/keyrings/mysql.gpg + owner: root + group: root + mode: '0644' + when: + - db_engine == 'mysql' + - db_version is version('8.4', '>=') + +- name: Add Apt signing key to keyring + ansible.builtin.apt_key: + id: A8D3785C + file: /usr/share/keyrings/mysql.gpg + state: present + when: + - db_engine == 'mysql' + - db_version is version('8.4', '>=') + +- name: Add MySQL 8.4 repository + ansible.builtin.apt_repository: + repo: deb http://repo.mysql.com/apt/ubuntu/ jammy mysql-8.4-lts mysql-tools + state: present + filename: mysql + when: + - db_engine == 'mysql' + - db_version is version('8.4', '>=') + - name: "{{ role_name }} | Requirements | Install Linux packages" ansible.builtin.package: name: diff --git a/tests/integration/targets/test_mysql_db/tasks/state_dump_import.yml b/tests/integration/targets/test_mysql_db/tasks/state_dump_import.yml index e4ae762..f8d2b4b 100644 --- a/tests/integration/targets/test_mysql_db/tasks/state_dump_import.yml +++ b/tests/integration/targets/test_mysql_db/tasks/state_dump_import.yml @@ -111,11 +111,24 @@ check_implicit_admin: no register: result -- name: Dump and Import | Assert successful completion of dump operation +- name: Dump and Import | Assert successful completion of dump operation for MariaDB and MySQL < 8.2 assert: that: - result is changed - result.executed_commands[0] is search(".department --master-data=1 --skip-triggers") + when: + - > + db_engine == 'mariadb' or + (db_engine == 'mysql' and db_version is version('8.2', '<')) + +- name: Dump and Import | Assert successful completion of dump operation for MySQL >= 8.2 + assert: + that: + - result is changed + - result.executed_commands[0] is search(".department --source-data=1 --skip-triggers") + when: + - db_engine == 'mysql' + - db_version is version('8.2', '>=') - name: Dump and Import | State dump/import - file name should exist (db_file_name) file: diff --git a/tests/integration/targets/test_mysql_replication/tasks/main.yml b/tests/integration/targets/test_mysql_replication/tasks/main.yml index 2baa536..a65cabd 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/main.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/main.yml @@ -1,3 +1,4 @@ +--- #################################################################### # WARNING: These are designed specifically for Ansible tests # # and should not be used as examples of how to write Ansible roles # @@ -18,8 +19,7 @@ # Tests of channel parameter: - import_tasks: mysql_replication_channel.yml when: - - db_engine == 'mysql' # FIXME: mariadb introduces FOR CHANNEL in 10.7 - - mysql8022_and_higher == true # FIXME: mysql 5.7 should work, but our tets fails, why? + - db_engine == 'mysql' # FIXME: mariadb introduces FOR CHANNEL in 10.7 # Tests of resetprimary mode: - import_tasks: mysql_replication_resetprimary_mode.yml @@ -30,3 +30,4 @@ - import_tasks: mysql_replication_changereplication_mode.yml when: - db_engine == 'mysql' + - db_version is version('8.0.23', '>=') diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml index 7d37df0..802865c 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml @@ -32,10 +32,15 @@ channel: '{{ test_channel }}' register: result - - assert: + - name: Assert that run replication with channel is changed and query matches for MariaDB and MySQL < 8.0.23 + ansible.builtin.assert: that: - result is changed - result.queries == result_query + when: + - > + db_engine == 'mariadb' or + (db_engine == 'mysql' and db_version is version('8.0.23', '<')) vars: result_query: ["CHANGE MASTER TO MASTER_HOST='{{ mysql_host }}',\ MASTER_USER='{{ replication_user }}',MASTER_PASSWORD='********',\ @@ -43,6 +48,21 @@ '{{ mysql_primary_status.File }}',MASTER_LOG_POS=\ {{ mysql_primary_status.Position }} FOR CHANNEL '{{ test_channel }}'"] + - name: Assert that run replication with channel is changed and query matches for MySQL >= 8.0.23 + ansible.builtin.assert: + that: + - result is changed + - result.queries == result_query + when: + - db_engine == 'mysql' + - db_version is version('8.0.23', '>=') + vars: + result_query: ["CHANGE REPLICATION SOURCE TO SOURCE_HOST='{{ mysql_host }}',\ + SOURCE_USER='{{ replication_user }}',SOURCE_PASSWORD='********',\ + SOURCE_PORT={{ mysql_primary_port }},SOURCE_LOG_FILE=\ + '{{ mysql_primary_status.File }}',SOURCE_LOG_POS=\ + {{ mysql_primary_status.Position }} FOR CHANNEL '{{ test_channel }}'"] + # Test startreplica mode: - name: Start replica with channel mysql_replication: @@ -83,7 +103,10 @@ mysql_host_value: '{{ mysql_host }}' mysql_primary_port_value: '{{ mysql_primary_port }}' test_channel_value: '{{ test_channel }}' - when: mysql8022_and_higher == false + when: + - > + db_engine == 'mariadb' or + (db_engine == 'mysql' and db_version is version('8.0.22', '<')) - assert: that: @@ -99,7 +122,9 @@ mysql_host_value: '{{ mysql_host }}' mysql_primary_port_value: '{{ mysql_primary_port }}' test_channel_value: '{{ test_channel }}' - when: mysql8022_and_higher == true + when: + - db_engine == 'mysql' + - db_version is version('8.0.22', '>=') # Test stopreplica mode: diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml index e08954b..30cd99f 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml @@ -9,16 +9,6 @@ login_host: '{{ mysql_host }}' block: - - name: Set mysql8022_and_higher - set_fact: - mysql8022_and_higher: false - - - name: Set mysql8022_and_higher - set_fact: - mysql8022_and_higher: true - when: - - db_engine == 'mysql' - - db_version is version('8.0.22', '>=') # We use iF NOT EXISTS because the GITHUB Action: # "ansible-community/ansible-test-gh-action" uses "--retry-on-error". @@ -136,11 +126,10 @@ that: - result is not failed - # Test changeprimary mode: # primary_ssl_ca will be set as '' to check the module's behaviour for #23976, # must be converted to an empty string - - name: Run replication - mysql_replication: + - name: Test changeprimary mode with empty primary_ssl_ca + community.mysql.mysql_replication: <<: *mysql_params login_port: '{{ mysql_replica1_port }}' mode: changeprimary @@ -151,14 +140,18 @@ primary_log_file: '{{ mysql_primary_status.File }}' primary_log_pos: '{{ mysql_primary_status.Position }}' primary_ssl_ca: '' - primary_ssl: no + primary_ssl: false register: result - - name: Assert that changeprimmary is changed and return expected query - assert: + - name: Assert that changeprimmary is changed and return expected query for MariaDB and MySQL < 8.0.23 + ansible.builtin.assert: that: - result is changed - result.queries == expected_queries + when: + - > + db_engine == 'mariadb' or + (db_engine == 'mysql' and db_version is version('8.0.23', '<')) vars: expected_queries: ["CHANGE MASTER TO MASTER_HOST='{{ mysql_host }}',\ MASTER_USER='{{ replication_user }}',MASTER_PASSWORD='********',\ @@ -166,6 +159,22 @@ '{{ mysql_primary_status.File }}',MASTER_LOG_POS=\ {{ mysql_primary_status.Position }},MASTER_SSL=0,MASTER_SSL_CA=''"] + - name: Assert that changeprimmary is changed and return expected query for MySQL > 8.0.23 + ansible.builtin.assert: + that: + - result is changed + - result.queries == expected_queries + when: + - db_engine == 'mysql' + - db_version is version('8.0.23', '>=') + vars: + expected_queries: ["CHANGE REPLICATION SOURCE TO \ + SOURCE_HOST='{{ mysql_host }}',\ + SOURCE_USER='{{ replication_user }}',SOURCE_PASSWORD='********',\ + SOURCE_PORT={{ mysql_primary_port }},SOURCE_LOG_FILE=\ + '{{ mysql_primary_status.File }}',SOURCE_LOG_POS=\ + {{ mysql_primary_status.Position }},SOURCE_SSL=0,SOURCE_SSL_CA=''"] + # Test startreplica mode: - name: Start replica mysql_replication: @@ -201,7 +210,10 @@ vars: mysql_host_value: "{{ mysql_host }}" mysql_primary_port_value: "{{ mysql_primary_port }}" - when: mysql8022_and_higher is falsy(convert_bool=True) + when: + - > + db_engine == 'mariadb' or + (db_engine == 'mysql' and db_version is version('8.0.22', '<')) - name: Assert that getreplica returns expected values for MySQL newer than 8.0.22 assert: @@ -216,7 +228,9 @@ vars: mysql_host_value: "{{ mysql_host }}" mysql_primary_port_value: "{{ mysql_primary_port }}" - when: mysql8022_and_higher is truthy(convert_bool=True) + when: + - db_engine == 'mysql' + - db_version is version('8.0.22', '>=') # Create test table and add data to it: - name: Create test table @@ -243,13 +257,18 @@ assert: that: - replica_status.Exec_Master_Log_Pos != mysql_primary_status.Position - when: mysql8022_and_higher == false + when: + - > + db_engine == 'mariadb' or + (db_engine == 'mysql' and db_version is version('8.0.22', '<')) - name: Assert that getreplica Log_Pos is different for MySQL newer than 8.0.22 assert: that: - replica_status.Exec_Source_Log_Pos != mysql_primary_status.Position - when: mysql8022_and_higher == true + when: + - db_engine == 'mysql' + - db_version is version('8.0.22', '>=') - name: Start replica that is already running mysql_replication: diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml index 5e967e8..3ae4339 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml @@ -18,10 +18,24 @@ primary_delay: '{{ test_primary_delay }}' register: result - - assert: + - name: Assert that run replication is changed and query match expectation for MariaDB and MySQL < 8.0.23 + ansible.builtin.assert: that: - - result is changed - - result.queries == ["CHANGE MASTER TO MASTER_DELAY=60"] + - result is changed + - result.queries == ["CHANGE MASTER TO MASTER_DELAY=60"] + when: + - > + db_engine == 'mariadb' or + (db_engine == 'mysql' and db_version is version('8.0.23', '<')) + + - name: Assert that run replication is changed and query match expectation for MySQL >= 8.0.23 + ansible.builtin.assert: + that: + - result is changed + - result.queries == ["CHANGE REPLICATION SOURCE TO SOURCE_DELAY=60"] + when: + - db_engine == 'mysql' + - db_version is version('8.0.23', '>=') # Auxiliary step: - name: Start replica diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_resetprimary_mode.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_resetprimary_mode.yml index 4bccc76..8968049 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_resetprimary_mode.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_resetprimary_mode.yml @@ -1,3 +1,4 @@ +--- # Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) @@ -38,10 +39,24 @@ mode: resetprimary register: result - - assert: + - name: Assert that reset primary is changed and query matches for MariaDB and MySQL < 8.4 + ansible.builtin.assert: that: - - result is changed - - result.queries == ["RESET MASTER"] + - result is changed + - result.queries == ["RESET MASTER"] + when: + - > + db_engine == 'mariadb' or + (db_engine == 'mysql' and db_version is version('8.4.0', '<')) + + - name: Assert that reset primary is changed and query matches for MySQL > 8.4 + ansible.builtin.assert: + that: + - result is changed + - result.queries == ["RESET BINARY LOGS AND GTIDS"] + when: + - db_engine == 'mysql' + - db_version is version('8.4.0', '>=') # Get primary final status: - name: Get primary status diff --git a/tests/unit/plugins/module_utils/test_command_resolver.py b/tests/unit/plugins/module_utils/test_command_resolver.py new file mode 100644 index 0000000..9653418 --- /dev/null +++ b/tests/unit/plugins/module_utils/test_command_resolver.py @@ -0,0 +1,39 @@ +# -*- coding: utf-8 -*- + +from __future__ import (absolute_import, division, print_function) +__metaclass__ = type + +import pytest + +from ansible_collections.community.mysql.plugins.module_utils.command_resolver import ( + CommandResolver, +) + + +@pytest.mark.parametrize( + 'server_implementation,server_version,command,expected_output,expected_exception,expected_message', + [ + ('mysql', '1.0.0', 'SHOW NOTHING', '', ValueError, 'Unsupported command: SHOW NOTHING'), + ('mysql', '8.0.20', 'SHOW MASTER STATUS', 'SHOW MASTER STATUS', None, None), # Case insensitive + ('mysql', '8.0.20', 'show master status', 'SHOW MASTER STATUS', None, None), # Case insensitive + ('mysql', '8.0.20', 'SHOW master STATUS', 'SHOW MASTER STATUS', None, None), # Case insensitive + ('mysql', '8.2.0', 'SHOW MASTER STATUS', 'SHOW BINARY LOG STATUS', None, None), + ('mysql', '9.0.0', 'SHOW MASTER STATUS', 'SHOW BINARY LOG STATUS', None, None), + ('mariadb', '10.4.23', 'SHOW MASTER STATUS', 'SHOW MASTER STATUS', None, None), # Default + ('mariadb', '10.5.1', 'SHOW MASTER STATUS', 'SHOW MASTER STATUS', None, None), # Default + ('mariadb', '10.5.2', 'SHOW MASTER STATUS', 'SHOW BINLOG STATUS', None, None), + ('mariadb', '10.6.17', 'SHOW MASTER STATUS', 'SHOW BINLOG STATUS', None, None), + ('mysql', '8.4.1', 'CHANGE MASTER', 'CHANGE REPLICATION SOURCE', None, None), + ] +) +def test_resolve_command(server_implementation, server_version, command, expected_output, expected_exception, expected_message): + """ + Tests that the CommandResolver method resolve_command return the correct query. + """ + resolver = CommandResolver(server_implementation, server_version) + if expected_exception: + with pytest.raises(expected_exception) as excinfo: + resolver.resolve_command(command) + assert str(excinfo.value) == expected_message + else: + assert resolver.resolve_command(command) == expected_output diff --git a/tests/unit/plugins/modules/test_mysql_info.py b/tests/unit/plugins/modules/test_mysql_info.py index 0d086f4..7b2de1c 100644 --- a/tests/unit/plugins/modules/test_mysql_info.py +++ b/tests/unit/plugins/modules/test_mysql_info.py @@ -14,15 +14,15 @@ from ansible_collections.community.mysql.plugins.modules.mysql_info import MySQL @pytest.mark.parametrize( - 'suffix,cursor_output,server_implementation,user_implementation', + 'suffix,cursor_output,server_implementation,server_version,user_implementation', [ - ('mysql', '5.5.1-mysql', 'mysql', 'mysql'), - ('log', '5.7.31-log', 'mysql', 'mysql'), - ('mariadb', '10.5.0-mariadb', 'mariadb', 'mariadb'), - ('', '8.0.22', 'mysql', 'mysql'), + ('mysql', '5.5.1-mysql', 'mysql', '5.5.1', 'mysql'), + ('log', '5.7.31-log', 'mysql', '5.7.31', 'mysql'), + ('mariadb', '10.5.0-mariadb', 'mariadb', '10.5.0', 'mariadb'), + ('', '8.0.22', 'mysql', '8.0.22', 'mysql'), ] ) -def test_get_info_suffix(suffix, cursor_output, server_implementation, user_implementation): +def test_get_info_suffix(suffix, cursor_output, server_implementation, server_version, user_implementation): def __cursor_return_value(input_parameter): if input_parameter == "SHOW GLOBAL VARIABLES": cursor.fetchall.return_value = [{"Variable_name": "version", "Value": cursor_output}] @@ -32,6 +32,6 @@ def test_get_info_suffix(suffix, cursor_output, server_implementation, user_impl cursor = MagicMock() cursor.execute.side_effect = __cursor_return_value - info = MySQL_Info(MagicMock(), cursor, server_implementation, user_implementation) + info = MySQL_Info(MagicMock(), cursor, server_implementation, server_version, user_implementation) assert info.get_info([], [], False)['version']['suffix'] == suffix From a9f9806728873e5003eb51eeb7fa96e6f1e783a3 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Mon, 19 Aug 2024 10:41:13 +0200 Subject: [PATCH 20/54] README: Add Communication section with Forum information (#665) --- README.md | 41 ++++++++++++++++------------------------- 1 file changed, 16 insertions(+), 25 deletions(-) diff --git a/README.md b/README.md index 05a7bde..1f5b47a 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,22 @@ We follow the [Ansible Code of Conduct](https://docs.ansible.com/ansible/latest/ If you encounter abusive behavior violating the [Ansible Code of Conduct](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html), please refer to the [policy violations](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html#policy-violations) section of the Code of Conduct for information on how to raise a complaint. +## Communication + +* Join the Ansible forum: + * [Get Help](https://forum.ansible.com/c/help/6): get help or help others. + * [Posts tagged with 'mysql'](https://forum.ansible.com/tag/mysql): leverage tags to narrow the scope. + * [MySQL Team](https://forum.ansible.com/g/MySQLTeam): by joining the team you will automatically get subscribed to the posts tagged with [mysql](https://forum.ansible.com/tag/mysql). + * [Social Spaces](https://forum.ansible.com/c/chat/4): gather and interact with fellow enthusiasts. + * [News & Announcements](https://forum.ansible.com/c/news/5): track project-wide announcements including social events. + +* The Ansible [Bullhorn newsletter](https://docs.ansible.com/ansible/devel/community/communication.html#the-bullhorn): used to announce releases and important changes. + +* Matrix chat: + * [#mysql:ansible.com](https://matrix.to/#/#mysql:ansible.com) room: questions on how to contribute to this collection. + +For more information about communication, see the [Ansible communication guide](https://docs.ansible.com/ansible/devel/community/communication.html). + ## Contributing The content of this collection is made by [people](https://github.com/ansible-collections/community.mysql/blob/main/CONTRIBUTORS) just like you, a community of individuals collaborating on making the world better through developing automation software. @@ -38,31 +54,6 @@ It is necessary for maintainers of this collection to be subscribed to: They also should be subscribed to Ansible's [The Bullhorn newsletter](https://docs.ansible.com/ansible/devel/community/communication.html#the-bullhorn). -## Communication - -> The `GitHub Discussions` feature is disabled in this repository. Use the `mysql` tag on the forum in the [Project Discussions](https://forum.ansible.com/new-topic?title=topic%20title&body=topic%20body&category=project&tags=mysql) or [Get Help](https://forum.ansible.com/new-topic?title=topic%20title&body=topic%20body&category=help&tags=mysql) category instead. - -### Asynchronous channels - -* Join the Ansible forum: - * [MySQL Team](https://forum.ansible.com/g/MySQLTeam): by joining the team you will automatically get subscribed to the posts tagged with [mysql](https://forum.ansible.com/tag/mysql). - * [Get Help](https://forum.ansible.com/c/help/6/none): get help or help others. - * [Posts tagged with 'mysql'](https://forum.ansible.com/tag/mysql): leverage tags to narrow the scope. - * [Social Spaces](https://forum.ansible.com/c/chat/4): gather and interact with fellow enthusiasts. - * [News & Announcements](https://forum.ansible.com/c/news/5/none): track project-wide announcements. - -* The Ansible's [Bullhorn newsletter](https://forum.ansible.com/t/about-the-newsletter-category/166): we use it to announce releases and important changes. - -### Real-time channels - -* Matrix: - * `#mysql:ansible.com` [room](https://matrix.to/#/#mysql:ansible.com): questions on how to contribute and use this collection. - * `#users:ansible.com` [room](https://matrix.to/#/#users:ansible.com): general use questions and support. - * `#ansible-community:ansible.com` [room](https://matrix.to/#/#community:ansible.com): community and collection development questions. - * other Matrix rooms; see the [Ansible Communication Guide](https://docs.ansible.com/ansible/devel/community/communication.html) for details. - -For more information about communication, refer to the [Ansible Communication guide](https://docs.ansible.com/ansible/devel/community/communication.html). - ## Governance We, [the MySQL team](https://forum.ansible.com/g/MySQLTeam), use [the forum](https://forum.ansible.com/tag/mysql) posts tagged with `mysql` for general announcements and discussions. From 37a718c66f5563f5d90e8af56a1e719ffa3f6c5d Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Thu, 22 Aug 2024 10:45:53 +0200 Subject: [PATCH 21/54] Release 3.10.0 commit (#667) --- CHANGELOG.rst | 43 +++++++++++++- changelogs/changelog.yaml | 59 +++++++++++++++++++ changelogs/fragments/0-mysql_user.yml | 2 - changelogs/fragments/1-mysql_info.yml | 2 - changelogs/fragments/2-mysql_variables.yml | 2 - .../fragments/3-deprecate_mysqlclient.yml | 2 - .../add_salt_param_to_gen_sha256_hash.yml | 3 - .../get_primary_show_binary_log_status.yml | 4 -- .../improve_get_replica_primary_status.yml | 4 -- .../lie_fix_mysql_user_on_new_username.yml | 6 -- .../lie_fix_plugin_hash_string_return.yml | 6 -- .../fragments/mysql_user_tls_requires.yml | 6 -- ...rts_mysql_change_replication_source_to.yml | 3 - galaxy.yml | 2 +- 14 files changed, 100 insertions(+), 44 deletions(-) delete mode 100644 changelogs/fragments/0-mysql_user.yml delete mode 100644 changelogs/fragments/1-mysql_info.yml delete mode 100644 changelogs/fragments/2-mysql_variables.yml delete mode 100644 changelogs/fragments/3-deprecate_mysqlclient.yml delete mode 100644 changelogs/fragments/add_salt_param_to_gen_sha256_hash.yml delete mode 100644 changelogs/fragments/get_primary_show_binary_log_status.yml delete mode 100644 changelogs/fragments/improve_get_replica_primary_status.yml delete mode 100644 changelogs/fragments/lie_fix_mysql_user_on_new_username.yml delete mode 100644 changelogs/fragments/lie_fix_plugin_hash_string_return.yml delete mode 100644 changelogs/fragments/mysql_user_tls_requires.yml delete mode 100644 changelogs/fragments/supports_mysql_change_replication_source_to.yml diff --git a/CHANGELOG.rst b/CHANGELOG.rst index cc7ab85..c5039ed 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -1,11 +1,48 @@ -======================================== -Community MySQL Collection Release Notes -======================================== +==================================================== +Community MySQL and MariaDB Collection Release Notes +==================================================== .. contents:: Topics This changelog describes changes after version 2.0.0. +v3.10.0 +======= + +Release Summary +--------------- + +This is a minor release of the ``community.mysql`` collection. +This changelog contains all changes to the modules and plugins in this +collection that have been made after the previous release. + +Minor Changes +------------- + +- mysql_info - Add ``tls_requires`` returned value for the ``users_info`` filter (https://github.com/ansible-collections/community.mysql/pull/628). +- mysql_info - return a database server engine used (https://github.com/ansible-collections/community.mysql/issues/644). +- mysql_replication - Adds support for `CHANGE REPLICATION SOURCE TO` statement (https://github.com/ansible-collections/community.mysql/issues/635). +- mysql_replication - Adds support for `SHOW BINARY LOG STATUS` and `SHOW BINLOG STATUS` on getprimary mode. +- mysql_replication - Improve detection of IsReplica and IsPrimary by inspecting the dictionary returned from the SQL query instead of relying on variable types. This ensures compatibility with changes in the connector or the output of SHOW REPLICA STATUS and SHOW MASTER STATUS, allowing for easier maintenance if these change in the future. +- mysql_user - Add salt parameter to generate static hash for `caching_sha2_password` and `sha256_password` plugins. + +Breaking Changes / Porting Guide +-------------------------------- + +- collection - support of mysqlclient connector is deprecated - use PyMySQL connector instead! We will stop testing against it in collection version 4.0.0 and remove the related code in 5.0.0 (https://github.com/ansible-collections/community.mysql/issues/654). +- mysql_info - The ``users_info`` filter returned variable ``plugin_auth_string`` contains the hashed password and it's misleading, it will be removed from community.mysql 4.0.0. Use the `plugin_hash_string` return value instead (https://github.com/ansible-collections/community.mysql/pull/629). + +Bugfixes +-------- + +- mysql_info - Add ``plugin_hash_string`` to ``users_info`` filter's output. The existing ``plugin_auth_string`` contained the hashed password and thus is missleading, it will be removed from community.mysql 4.0.0. (https://github.com/ansible-collections/community.mysql/pull/629). +- mysql_user - Added a warning to update_password's on_new_username option if multiple accounts with the same username but different passwords exist (https://github.com/ansible-collections/community.mysql/pull/642). +- mysql_user - Fix ``tls_requires`` not removing ``SSL`` and ``X509`` when sets as empty (https://github.com/ansible-collections/community.mysql/pull/628). +- mysql_user - Fix idempotence when using variables from the ``users_info`` filter of ``mysql_info`` as an input (https://github.com/ansible-collections/community.mysql/pull/628). +- mysql_user - Fixed an IndexError in the update_password functionality introduced in PR https://github.com/ansible-collections/community.mysql/pull/580 and released in community.mysql 3.8.0. If you used this functionality, please avoid versions 3.8.0 to 3.9.0 (https://github.com/ansible-collections/community.mysql/pull/642). +- mysql_user - add correct ``ed25519`` auth plugin handling (https://github.com/ansible-collections/community.mysql/issues/6). +- mysql_variables - fix the module always changes on boolean values (https://github.com/ansible-collections/community.mysql/issues/652). + v3.9.0 ====== diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index eb4264d..8c18264 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -97,6 +97,65 @@ releases: - 307-mysql_user_add_if_exists_to_drop.yml - 329-mysql_role-remove-redudant-connection-closing.yml release_date: '2022-04-26' + 3.10.0: + changes: + breaking_changes: + - collection - support of mysqlclient connector is deprecated - use PyMySQL + connector instead! We will stop testing against it in collection version 4.0.0 + and remove the related code in 5.0.0 (https://github.com/ansible-collections/community.mysql/issues/654). + - mysql_info - The ``users_info`` filter returned variable ``plugin_auth_string`` + contains the hashed password and it's misleading, it will be removed from + community.mysql 4.0.0. Use the `plugin_hash_string` return value instead (https://github.com/ansible-collections/community.mysql/pull/629). + bugfixes: + - mysql_info - Add ``plugin_hash_string`` to ``users_info`` filter's output. + The existing ``plugin_auth_string`` contained the hashed password and thus + is missleading, it will be removed from community.mysql 4.0.0. (https://github.com/ansible-collections/community.mysql/pull/629). + - mysql_user - Added a warning to update_password's on_new_username option if + multiple accounts with the same username but different passwords exist (https://github.com/ansible-collections/community.mysql/pull/642). + - mysql_user - Fix ``tls_requires`` not removing ``SSL`` and ``X509`` when sets + as empty (https://github.com/ansible-collections/community.mysql/pull/628). + - mysql_user - Fix idempotence when using variables from the ``users_info`` + filter of ``mysql_info`` as an input (https://github.com/ansible-collections/community.mysql/pull/628). + - mysql_user - Fixed an IndexError in the update_password functionality introduced + in PR https://github.com/ansible-collections/community.mysql/pull/580 and + released in community.mysql 3.8.0. If you used this functionality, please + avoid versions 3.8.0 to 3.9.0 (https://github.com/ansible-collections/community.mysql/pull/642). + - mysql_user - add correct ``ed25519`` auth plugin handling (https://github.com/ansible-collections/community.mysql/issues/6). + - mysql_variables - fix the module always changes on boolean values (https://github.com/ansible-collections/community.mysql/issues/652). + minor_changes: + - mysql_info - Add ``tls_requires`` returned value for the ``users_info`` filter + (https://github.com/ansible-collections/community.mysql/pull/628). + - mysql_info - return a database server engine used (https://github.com/ansible-collections/community.mysql/issues/644). + - mysql_replication - Adds support for `CHANGE REPLICATION SOURCE TO` statement + (https://github.com/ansible-collections/community.mysql/issues/635). + - mysql_replication - Adds support for `SHOW BINARY LOG STATUS` and `SHOW BINLOG + STATUS` on getprimary mode. + - mysql_replication - Improve detection of IsReplica and IsPrimary by inspecting + the dictionary returned from the SQL query instead of relying on variable + types. This ensures compatibility with changes in the connector or the output + of SHOW REPLICA STATUS and SHOW MASTER STATUS, allowing for easier maintenance + if these change in the future. + - mysql_user - Add salt parameter to generate static hash for `caching_sha2_password` + and `sha256_password` plugins. + release_summary: 'This is a minor release of the ``community.mysql`` collection. + + This changelog contains all changes to the modules and plugins in this + + collection that have been made after the previous release.' + fragments: + - 0-mysql_user.yml + - 1-mysql_info.yml + - 2-mysql_variables.yml + - 3-deprecate_mysqlclient.yml + - 3.10.0.yml + - add_salt_param_to_gen_sha256_hash.yml + - get_primary_show_binary_log_status.yml + - improve_get_replica_primary_status.yml + - lie_fix_mysql_user_on_new_username.yml + - lie_fix_plugin_hash_string_return.yml + - mysql_user_tls_requires.yml + - supports_mysql_change_replication_source_to.yml + release_date: '2024-08-22' 3.2.0: changes: bugfixes: diff --git a/changelogs/fragments/0-mysql_user.yml b/changelogs/fragments/0-mysql_user.yml deleted file mode 100644 index 6b812ab..0000000 --- a/changelogs/fragments/0-mysql_user.yml +++ /dev/null @@ -1,2 +0,0 @@ -bugfixes: -- mysql_user - add correct ``ed25519`` auth plugin handling (https://github.com/ansible-collections/community.mysql/issues/6). diff --git a/changelogs/fragments/1-mysql_info.yml b/changelogs/fragments/1-mysql_info.yml deleted file mode 100644 index 1ab4d2c..0000000 --- a/changelogs/fragments/1-mysql_info.yml +++ /dev/null @@ -1,2 +0,0 @@ -minor_changes: -- mysql_info - return a database server engine used (https://github.com/ansible-collections/community.mysql/issues/644). diff --git a/changelogs/fragments/2-mysql_variables.yml b/changelogs/fragments/2-mysql_variables.yml deleted file mode 100644 index 9ef8d80..0000000 --- a/changelogs/fragments/2-mysql_variables.yml +++ /dev/null @@ -1,2 +0,0 @@ -bugfixes: -- mysql_variables - fix the module always changes on boolean values (https://github.com/ansible-collections/community.mysql/issues/652). diff --git a/changelogs/fragments/3-deprecate_mysqlclient.yml b/changelogs/fragments/3-deprecate_mysqlclient.yml deleted file mode 100644 index 9134413..0000000 --- a/changelogs/fragments/3-deprecate_mysqlclient.yml +++ /dev/null @@ -1,2 +0,0 @@ -breaking_changes: -- collection - support of mysqlclient connector is deprecated - use PyMySQL connector instead! We will stop testing against it in collection version 4.0.0 and remove the related code in 5.0.0 (https://github.com/ansible-collections/community.mysql/issues/654). diff --git a/changelogs/fragments/add_salt_param_to_gen_sha256_hash.yml b/changelogs/fragments/add_salt_param_to_gen_sha256_hash.yml deleted file mode 100644 index c49ba1d..0000000 --- a/changelogs/fragments/add_salt_param_to_gen_sha256_hash.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -minor_changes: - - mysql_user - Add salt parameter to generate static hash for `caching_sha2_password` and `sha256_password` plugins. diff --git a/changelogs/fragments/get_primary_show_binary_log_status.yml b/changelogs/fragments/get_primary_show_binary_log_status.yml deleted file mode 100644 index 8757aa1..0000000 --- a/changelogs/fragments/get_primary_show_binary_log_status.yml +++ /dev/null @@ -1,4 +0,0 @@ ---- -minor_changes: - - - mysql_replication - Adds support for `SHOW BINARY LOG STATUS` and `SHOW BINLOG STATUS` on getprimary mode. diff --git a/changelogs/fragments/improve_get_replica_primary_status.yml b/changelogs/fragments/improve_get_replica_primary_status.yml deleted file mode 100644 index 512d7ef..0000000 --- a/changelogs/fragments/improve_get_replica_primary_status.yml +++ /dev/null @@ -1,4 +0,0 @@ ---- -minor_changes: - - - mysql_replication - Improve detection of IsReplica and IsPrimary by inspecting the dictionary returned from the SQL query instead of relying on variable types. This ensures compatibility with changes in the connector or the output of SHOW REPLICA STATUS and SHOW MASTER STATUS, allowing for easier maintenance if these change in the future. diff --git a/changelogs/fragments/lie_fix_mysql_user_on_new_username.yml b/changelogs/fragments/lie_fix_mysql_user_on_new_username.yml deleted file mode 100644 index 7f13738..0000000 --- a/changelogs/fragments/lie_fix_mysql_user_on_new_username.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- - -bugfixes: - - - mysql_user - Fixed an IndexError in the update_password functionality introduced in PR https://github.com/ansible-collections/community.mysql/pull/580 and released in community.mysql 3.8.0. If you used this functionality, please avoid versions 3.8.0 to 3.9.0 (https://github.com/ansible-collections/community.mysql/pull/642). - - mysql_user - Added a warning to update_password's on_new_username option if multiple accounts with the same username but different passwords exist (https://github.com/ansible-collections/community.mysql/pull/642). diff --git a/changelogs/fragments/lie_fix_plugin_hash_string_return.yml b/changelogs/fragments/lie_fix_plugin_hash_string_return.yml deleted file mode 100644 index e1a71ea..0000000 --- a/changelogs/fragments/lie_fix_plugin_hash_string_return.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- -bugfixes: - - mysql_info - Add ``plugin_hash_string`` to ``users_info`` filter's output. The existing ``plugin_auth_string`` contained the hashed password and thus is missleading, it will be removed from community.mysql 4.0.0. (https://github.com/ansible-collections/community.mysql/pull/629). - -breaking_changes: - - mysql_info - The ``users_info`` filter returned variable ``plugin_auth_string`` contains the hashed password and it's misleading, it will be removed from community.mysql 4.0.0. Use the `plugin_hash_string` return value instead (https://github.com/ansible-collections/community.mysql/pull/629). diff --git a/changelogs/fragments/mysql_user_tls_requires.yml b/changelogs/fragments/mysql_user_tls_requires.yml deleted file mode 100644 index 1fa0c94..0000000 --- a/changelogs/fragments/mysql_user_tls_requires.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- -minor_changes: - - mysql_info - Add ``tls_requires`` returned value for the ``users_info`` filter (https://github.com/ansible-collections/community.mysql/pull/628). -bugfixes: - - mysql_user - Fix idempotence when using variables from the ``users_info`` filter of ``mysql_info`` as an input (https://github.com/ansible-collections/community.mysql/pull/628). - - mysql_user - Fix ``tls_requires`` not removing ``SSL`` and ``X509`` when sets as empty (https://github.com/ansible-collections/community.mysql/pull/628). diff --git a/changelogs/fragments/supports_mysql_change_replication_source_to.yml b/changelogs/fragments/supports_mysql_change_replication_source_to.yml deleted file mode 100644 index 955d62e..0000000 --- a/changelogs/fragments/supports_mysql_change_replication_source_to.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -minor_changes: - - mysql_replication - Adds support for `CHANGE REPLICATION SOURCE TO` statement (https://github.com/ansible-collections/community.mysql/issues/635). diff --git a/galaxy.yml b/galaxy.yml index 512c668..353a6f8 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,7 +1,7 @@ --- namespace: community name: mysql -version: 3.9.0 +version: 3.10.0 readme: README.md authors: - Ansible community From 87be61ccf3601ed02711ce893c5f40af71f656ac Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Thu, 29 Aug 2024 08:47:48 +0200 Subject: [PATCH 22/54] CI: Fix sanity errors (#668) --- plugins/module_utils/user.py | 1 + plugins/modules/mysql_user.py | 1 + tests/sanity/ignore-2.15.txt | 1 - tests/sanity/ignore-2.16.txt | 1 - tests/sanity/ignore-2.17.txt | 1 - tests/sanity/ignore-2.18.txt | 1 - 6 files changed, 2 insertions(+), 4 deletions(-) diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index bd71691..5e0196a 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -393,6 +393,7 @@ def user_mod(cursor, user, host, host_all, password, encrypted, update = True if update: + query_with_args = None if plugin_hash_string: query_with_args = "ALTER USER %s@%s IDENTIFIED WITH %s AS %s", (user, host, plugin, plugin_hash_string) elif plugin_auth_string: diff --git a/plugins/modules/mysql_user.py b/plugins/modules/mysql_user.py index 0c7021b..2ee5e01 100644 --- a/plugins/modules/mysql_user.py +++ b/plugins/modules/mysql_user.py @@ -20,6 +20,7 @@ options: - Name of the user (role) to add or remove. type: str required: true + aliases: ['user'] password: description: - Set the user's password. Only for C(mysql_native_password) authentication. diff --git a/tests/sanity/ignore-2.15.txt b/tests/sanity/ignore-2.15.txt index 55b2904..152162d 100644 --- a/tests/sanity/ignore-2.15.txt +++ b/tests/sanity/ignore-2.15.txt @@ -1,4 +1,3 @@ plugins/modules/mysql_db.py validate-modules:use-run-command-not-popen -plugins/modules/mysql_user.py validate-modules:undocumented-parameter plugins/module_utils/mysql.py pylint:unused-import plugins/module_utils/version.py pylint:unused-import diff --git a/tests/sanity/ignore-2.16.txt b/tests/sanity/ignore-2.16.txt index 55b2904..152162d 100644 --- a/tests/sanity/ignore-2.16.txt +++ b/tests/sanity/ignore-2.16.txt @@ -1,4 +1,3 @@ plugins/modules/mysql_db.py validate-modules:use-run-command-not-popen -plugins/modules/mysql_user.py validate-modules:undocumented-parameter plugins/module_utils/mysql.py pylint:unused-import plugins/module_utils/version.py pylint:unused-import diff --git a/tests/sanity/ignore-2.17.txt b/tests/sanity/ignore-2.17.txt index 55b2904..152162d 100644 --- a/tests/sanity/ignore-2.17.txt +++ b/tests/sanity/ignore-2.17.txt @@ -1,4 +1,3 @@ plugins/modules/mysql_db.py validate-modules:use-run-command-not-popen -plugins/modules/mysql_user.py validate-modules:undocumented-parameter plugins/module_utils/mysql.py pylint:unused-import plugins/module_utils/version.py pylint:unused-import diff --git a/tests/sanity/ignore-2.18.txt b/tests/sanity/ignore-2.18.txt index 55b2904..152162d 100644 --- a/tests/sanity/ignore-2.18.txt +++ b/tests/sanity/ignore-2.18.txt @@ -1,4 +1,3 @@ plugins/modules/mysql_db.py validate-modules:use-run-command-not-popen -plugins/modules/mysql_user.py validate-modules:undocumented-parameter plugins/module_utils/mysql.py pylint:unused-import plugins/module_utils/version.py pylint:unused-import From 0de9685cf1db355fac194f70e154fa48ecd06705 Mon Sep 17 00:00:00 2001 From: Fran <51233345+francescsanjuanmrf@users.noreply.github.com> Date: Fri, 30 Aug 2024 11:15:16 +0200 Subject: [PATCH 23/54] Fix user plugin changes in check mode (#596) * Fix user plugin changes in check mode * Add auth plugin tests * Undo local changes * Improve task names * Fix query * Changes * Add check * Add check * Add check * Add one more check * Add one more check * Fix typo * Change parameter * Testing * Remove tests * Add tests * Test first stteps * Readd tests * Test without check mode * Test with check mode * Test with check mode * Testing * Testing * Add missing tests * Changes for ansible-lint complaints * Fix condition * Update changelogs/fragments/596-fix-check-changes.yaml Co-authored-by: Andrew Klychkov * refactor * Add more tests * Fix newpass var * Remove extra test --------- Co-authored-by: Andrew Klychkov --- .../fragments/596-fix-check-changes.yaml | 2 + plugins/module_utils/user.py | 3 +- .../tasks/test_user_plugin_auth.yml | 227 ++++++++++++------ .../tasks/utils/assert_plugin.yml | 11 + 4 files changed, 175 insertions(+), 68 deletions(-) create mode 100644 changelogs/fragments/596-fix-check-changes.yaml create mode 100644 tests/integration/targets/test_mysql_user/tasks/utils/assert_plugin.yml diff --git a/changelogs/fragments/596-fix-check-changes.yaml b/changelogs/fragments/596-fix-check-changes.yaml new file mode 100644 index 0000000..e7c24f1 --- /dev/null +++ b/changelogs/fragments/596-fix-check-changes.yaml @@ -0,0 +1,2 @@ +bugfixes: + - mysql_user - module makes changes when is executed with ``plugin_auth_string`` parameter and check mode. diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index 5e0196a..7d7d304 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -411,7 +411,8 @@ def user_mod(cursor, user, host, host_all, password, encrypted, else: query_with_args = "ALTER USER %s@%s IDENTIFIED WITH %s", (user, host, plugin) - cursor.execute(*query_with_args) + if not module.check_mode: + cursor.execute(*query_with_args) password_changed = True changed = True diff --git a/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml b/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml index b5ed6c5..f6f3c2e 100644 --- a/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml +++ b/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml @@ -24,7 +24,7 @@ # - name: Plugin auth | Create user with plugin auth (with hash string) - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' host: '%' @@ -34,28 +34,28 @@ register: result - name: Plugin auth | Get user information (with hash string) - command: "{{ mysql_command }} -e \"SELECT user, host, plugin FROM mysql.user WHERE user = '{{ test_user_name }}' and host = '%'\"" + ansible.builtin.command: "{{ mysql_command }} -e \"SELECT user, host, plugin FROM mysql.user WHERE user = '{{ test_user_name }}' and host = '%'\"" register: show_create_user - name: Plugin auth | Check that the module made a change (with hash string) - assert: + ansible.builtin.assert: that: - result is changed - name: Plugin auth | Check that the expected plugin type is set (with hash string) - assert: + ansible.builtin.assert: that: - "'{{ test_plugin_type }}' in show_create_user.stdout" when: db_engine == 'mysql' or (db_engine == 'mariadb' and db_version is version('10.3', '>=')) - - include_tasks: utils/assert_user.yml + - ansible.builtin.include_tasks: utils/assert_user.yml vars: user_name: "{{ test_user_name }}" user_host: "%" priv: "{{ test_default_priv_type }}" - name: Plugin auth | Get the MySQL version using the newly created creds - mysql_info: + community.mysql.mysql_info: login_user: '{{ test_user_name }}' login_password: '{{ test_plugin_auth_string }}' login_host: '{{ mysql_host }}' @@ -64,12 +64,12 @@ register: result - name: Plugin auth | Assert that mysql_info was successful - assert: + ansible.builtin.assert: that: - result is succeeded - name: Plugin auth | Update the user with a different hash - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' host: '%' @@ -78,18 +78,18 @@ register: result - name: Plugin auth | Check that the module makes the change because the hash changed - assert: + ansible.builtin.assert: that: - result is changed - - include_tasks: utils/assert_user.yml + - ansible.builtin.include_tasks: utils/assert_user.yml vars: user_name: "{{ test_user_name }}" user_host: "%" priv: "{{ test_default_priv_type }}" - name: Plugin auth | Getting the MySQL info with the new password should work - mysql_info: + community.mysql.mysql_info: login_user: '{{ test_user_name }}' login_password: '{{ test_plugin_new_auth_string }}' login_host: '{{ mysql_host }}' @@ -98,12 +98,12 @@ register: result - name: Plugin auth | Assert that mysql_info was successful - assert: + ansible.builtin.assert: that: - result is succeeded # Cleanup - - include_tasks: utils/remove_user.yml + - ansible.builtin.include_tasks: utils/remove_user.yml vars: user_name: "{{ test_user_name }}" @@ -112,7 +112,7 @@ # - name: Plugin auth | Create user with plugin auth (with hash string) - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' host: '%' @@ -122,28 +122,28 @@ register: result - name: Plugin auth | Get user information - command: "{{ mysql_command }} -e \"SELECT user, host, plugin FROM mysql.user WHERE user = '{{ test_user_name }}' and host = '%'\"" + ansible.builtin.command: "{{ mysql_command }} -e \"SELECT user, host, plugin FROM mysql.user WHERE user = '{{ test_user_name }}' and host = '%'\"" register: show_create_user - name: Plugin auth | Check that the module made a change (with hash string) - assert: + ansible.builtin.assert: that: - result is changed - name: Plugin auth | Check that the expected plugin type is set (with hash string) - assert: + ansible.builtin.assert: that: - "'{{ test_plugin_type }}' in show_create_user.stdout" when: db_engine == 'mysql' or (db_engine == 'mariadb' and db_version is version('10.3', '>=')) - - include_tasks: utils/assert_user.yml + - ansible.builtin.include_tasks: utils/assert_user.yml vars: user_name: "{{ test_user_name }}" user_host: "%" priv: "{{ test_default_priv_type }}" - name: Plugin auth | Get the MySQL version using the newly created creds - mysql_info: + community.mysql.mysql_info: login_user: '{{ test_user_name }}' login_password: '{{ test_plugin_auth_string }}' login_host: '{{ mysql_host }}' @@ -152,12 +152,12 @@ register: result - name: Plugin auth | Assert that mysql_info was successful - assert: + ansible.builtin.assert: that: - result is succeeded - name: Plugin auth | Update the user with the same hash (no change expected) - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' host: '%' @@ -167,19 +167,19 @@ # FIXME: on mariadb 10.2 there's always a change - name: Plugin auth | Check that the module doesn't make a change when the same hash is passed in - assert: + ansible.builtin.assert: that: - result is not changed when: db_engine == 'mysql' or (db_engine == 'mariadb' and db_version is version('10.3', '>=')) - - include_tasks: utils/assert_user.yml + - ansible.builtin.include_tasks: utils/assert_user.yml vars: user_name: "{{ test_user_name }}" user_host: "%" priv: "{{ test_default_priv_type }}" - name: Plugin auth | Change the user using the same plugin, but switch to the same auth string in plaintext form - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' host: '%' @@ -189,12 +189,12 @@ # Expecting a change is currently by design (see comment in source). - name: Plugin auth | Check that the module did not change the password - assert: + ansible.builtin.assert: that: - result is changed - name: Plugin auth | Getting the MySQL info should still work - mysql_info: + community.mysql.mysql_info: login_user: '{{ test_user_name }}' login_password: '{{ test_plugin_auth_string }}' login_host: '{{ mysql_host }}' @@ -203,12 +203,12 @@ register: result - name: Plugin auth | Assert that mysql_info was successful - assert: + ansible.builtin.assert: that: - result is succeeded # Cleanup - - include_tasks: utils/remove_user.yml + - ansible.builtin.include_tasks: utils/remove_user.yml vars: user_name: "{{ test_user_name }}" @@ -217,7 +217,7 @@ # - name: Plugin auth | Create user with plugin auth (with auth string) - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' host: '%' @@ -227,28 +227,28 @@ register: result - name: Plugin auth | Get user information(with auth string) - command: "{{ mysql_command }} -e \"SHOW CREATE USER '{{ test_user_name }}'@'%'\"" + ansible.builtin.command: "{{ mysql_command }} -e \"SHOW CREATE USER '{{ test_user_name }}'@'%'\"" register: show_create_user - name: Plugin auth | Check that the module made a change (with auth string) - assert: + ansible.builtin.assert: that: - result is changed - name: Plugin auth | Check that the expected plugin type is set (with auth string) - assert: + ansible.builtin.assert: that: - test_plugin_type in show_create_user.stdout when: db_engine == 'mysql' or (db_engine == 'mariadb' and db_version is version('10.3', '>=')) - - include_tasks: utils/assert_user.yml + - ansible.builtin.include_tasks: utils/assert_user.yml vars: user_name: "{{ test_user_name }}" user_host: "%" priv: "{{ test_default_priv_type }}" - name: Plugin auth | Get the MySQL version using the newly created creds - mysql_info: + community.mysql.mysql_info: login_user: '{{ test_user_name }}' login_password: '{{ test_plugin_auth_string }}' login_host: '{{ mysql_host }}' @@ -257,12 +257,12 @@ register: result - name: Plugin auth | Assert that mysql_info was successful - assert: + ansible.builtin.assert: that: - result is succeeded - name: Plugin auth | Update the user with the same auth string - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' host: '%' @@ -273,18 +273,18 @@ # This is the current expected behavior because there isn't a reliable way to hash the password in the mysql_user # module in order to be able to compare this password with the stored hash. See the source for more info. - name: Plugin auth | The module should detect a change even though the password is the same - assert: + ansible.builtin.assert: that: - result is changed - - include_tasks: utils/assert_user.yml + - ansible.builtin.include_tasks: utils/assert_user.yml vars: user_name: "{{ test_user_name }}" user_host: "%" priv: "{{ test_default_priv_type }}" - name: Plugin auth | Change the user using the same plugin, but switch to the same auth string in hash form - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' host: '%' @@ -293,12 +293,12 @@ register: result - name: Plugin auth | Check that the module did not change the password - assert: + ansible.builtin.assert: that: - result is not changed - name: Plugin auth | Get the MySQL version using the newly created creds - mysql_info: + community.mysql.mysql_info: login_user: '{{ test_user_name }}' login_password: '{{ test_plugin_auth_string }}' login_host: '{{ mysql_host }}' @@ -307,12 +307,12 @@ register: result - name: Plugin auth | Assert that mysql_info was successful - assert: + ansible.builtin.assert: that: - result is succeeded # Cleanup - - include_tasks: utils/remove_user.yml + - ansible.builtin.include_tasks: utils/remove_user.yml vars: user_name: "{{ test_user_name }}" @@ -321,7 +321,7 @@ # - name: Plugin auth | Create user with plugin auth (empty auth string) - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' host: '%' @@ -330,28 +330,28 @@ register: result - name: Plugin auth | Get user information (empty auth string) - command: "{{ mysql_command }} -e \"SHOW CREATE USER '{{ test_user_name }}'@'%'\"" + ansible.builtin.command: "{{ mysql_command }} -e \"SHOW CREATE USER '{{ test_user_name }}'@'%'\"" register: show_create_user - name: Plugin auth | Check that the module made a change (empty auth string) - assert: + ansible.builtin.assert: that: - result is changed - name: Plugin auth | Check that the expected plugin type is set (empty auth string) - assert: + ansible.builtin.assert: that: - "'{{ test_plugin_type }}' in show_create_user.stdout" when: db_engine == 'mysql' or (db_engine == 'mariadb' and db_version is version('10.3', '>=')) - - include_tasks: utils/assert_user.yml + - ansible.builtin.include_tasks: utils/assert_user.yml vars: user_name: "{{ test_user_name }}" user_host: "%" priv: "{{ test_default_priv_type }}" - name: Plugin auth | Get the MySQL version using an empty password for the newly created user - mysql_info: + community.mysql.mysql_info: login_user: '{{ test_user_name }}' login_password: '' login_host: '{{ mysql_host }}' @@ -361,12 +361,12 @@ ignore_errors: true - name: Plugin auth | Assert that mysql_info was successful - assert: + ansible.builtin.assert: that: - result is succeeded - name: Plugin auth | Get the MySQL version using an non-empty password (should fail) - mysql_info: + community.mysql.mysql_info: login_user: '{{ test_user_name }}' login_password: 'some_password' login_host: '{{ mysql_host }}' @@ -376,12 +376,12 @@ ignore_errors: true - name: Plugin auth | Assert that mysql_info failed - assert: + ansible.builtin.assert: that: - result is failed - name: Plugin auth | Update the user without changing the auth mechanism - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' host: '%' @@ -390,12 +390,12 @@ register: result - name: Plugin auth | Assert that the user wasn't changed because the auth string is still empty - assert: + ansible.builtin.assert: that: - result is not changed # Cleanup - - include_tasks: utils/remove_user.yml + - ansible.builtin.include_tasks: utils/remove_user.yml vars: user_name: "{{ test_user_name }}" @@ -415,7 +415,7 @@ block: - name: Plugin auth | Create user with plugin auth (empty auth string) - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' plugin: '{{ test_plugin_type }}' @@ -423,28 +423,28 @@ register: result - name: Plugin auth | Get user information (empty auth string) - command: "{{ mysql_command }} -e \"SHOW CREATE USER '{{ test_user_name }}'@'localhost'\"" + ansible.builtin.command: "{{ mysql_command }} -e \"SHOW CREATE USER '{{ test_user_name }}'@'localhost'\"" register: show_create_user - name: Plugin auth | Check that the module made a change (empty auth string) - assert: + ansible.builtin.assert: that: - result is changed - name: Plugin auth | Check that the expected plugin type is set (empty auth string) - assert: + ansible.builtin.assert: that: - test_plugin_type in show_create_user.stdout when: db_engine == 'mysql' or (db_engine == 'mariadb' and db_version is version('10.3', '>=')) - - include_tasks: utils/assert_user.yml + - ansible.builtin.include_tasks: utils/assert_user.yml vars: user_name: "{{ test_user_name }}" user_host: localhost priv: "{{ test_default_priv_type }}" - name: Plugin auth | Switch user to sha256_password auth plugin - mysql_user: + community.mysql.mysql_user: <<: *mysql_params name: '{{ test_user_name }}' plugin: sha256_password @@ -452,28 +452,28 @@ register: result - name: Plugin auth | Get user information (sha256_password) - command: "{{ mysql_command }} -e \"SHOW CREATE USER '{{ test_user_name }}'@'localhost'\"" + ansible.builtin.command: "{{ mysql_command }} -e \"SHOW CREATE USER '{{ test_user_name }}'@'localhost'\"" register: show_create_user - name: Plugin auth | Check that the module made a change (sha256_password) - assert: + ansible.builtin.assert: that: - result is changed - name: Plugin auth | Check that the expected plugin type is set (sha256_password) - assert: + ansible.builtin.assert: that: - "'sha256_password' in show_create_user.stdout" when: db_engine == 'mysql' or (db_engine == 'mariadb' and db_version is version('10.3', '>=')) - - include_tasks: utils/assert_user.yml + - ansible.builtin.include_tasks: utils/assert_user.yml vars: user_name: "{{ test_user_name }}" user_host: localhost priv: "{{ test_default_priv_type }}" # Cleanup - - include_tasks: utils/remove_user.yml + - ansible.builtin.include_tasks: utils/remove_user.yml vars: user_name: "{{ test_user_name }}" @@ -505,7 +505,7 @@ register: result failed_when: result is changed - - name: cleanup user + - name: Cleanup user ansible.builtin.include_tasks: utils/remove_user.yml vars: user_name: "{{ test_user_name }}" @@ -544,3 +544,96 @@ priv: "{{ test_default_priv }}" register: result failed_when: result is success + + # ============================================================ + # Test auth plugin change + # + + - name: Plugin auth | Test plugin auth switching which doesn't work on pymysql < 0.9 + when: + - > + connector_name != 'pymysql' + or ( + connector_name == 'pymysql' + and connector_version is version('0.9', '>=') + ) + block: + + - name: Cleanup user + ansible.builtin.include_tasks: utils/remove_user.yml + vars: + user_name: "{{ test_user_name }}" + + - name: Plugin auth | Create user with mysql_native_password + community.mysql.mysql_user: + <<: *mysql_params + name: "{{ test_user_name }}" + host: "%" + plugin: "{{ test_plugin_type }}" + password: "{{ test_plugin_auth_string }}" + priv: "{{ test_default_priv }}" + + - name: Plugin auth | Check that the expected plugin type is set + ansible.builtin.include_tasks: utils/assert_plugin.yml + vars: + user_name: "{{ test_user_name }}" + plugin_type: "{{ test_plugin_type }}" + + - name: Plugin auth | Connect with user and password + ansible.builtin.command: '{{ mysql_command }} -u {{ test_user_name }} -p{{ test_plugin_auth_string }} -e "SELECT 1"' + changed_when: false + + - name: Plugin auth | Change auth user plugin in check mode + community.mysql.mysql_user: + <<: *mysql_params + name: "{{ test_user_name }}" + host: '%' + plugin: caching_sha2_password + plugin_auth_string: "{{ test_plugin_auth_string }}" + salt: "{{ test_salt }}" + priv: "{{ test_default_priv }}" + check_mode: true + register: result + failed_when: result is not changed + + - name: Plugin auth | Check that the expected plugin type is set (not changed) + ansible.builtin.include_tasks: utils/assert_plugin.yml + vars: + user_name: "{{ test_user_name }}" + plugin_type: "{{ test_plugin_type }}" + + - name: Plugin auth | Change auth user plugin + community.mysql.mysql_user: + <<: *mysql_params + name: "{{ test_user_name }}" + host: '%' + plugin: caching_sha2_password + plugin_auth_string: "{{ test_plugin_auth_string }}" + salt: "{{ test_salt }}" + priv: "{{ test_default_priv }}" + register: result + failed_when: result is not changed + + - name: Plugin auth | Check that the expected (new) plugin type is set + ansible.builtin.include_tasks: utils/assert_plugin.yml + vars: + user_name: "{{ test_user_name }}" + plugin_type: caching_sha2_password + + - name: Plugin auth | Change auth user plugin again (should not change) + community.mysql.mysql_user: + <<: *mysql_params + name: "{{ test_user_name }}" + host: '%' + plugin: caching_sha2_password + plugin_auth_string: "{{ test_plugin_auth_string }}" + salt: "{{ test_salt }}" + priv: "{{ test_default_priv }}" + register: result + failed_when: result is changed + + - name: Plugin auth | Check that the expected (not changed) plugin type is set + ansible.builtin.include_tasks: utils/assert_plugin.yml + vars: + user_name: "{{ test_user_name }}" + plugin_type: caching_sha2_password diff --git a/tests/integration/targets/test_mysql_user/tasks/utils/assert_plugin.yml b/tests/integration/targets/test_mysql_user/tasks/utils/assert_plugin.yml new file mode 100644 index 0000000..7d3b5a1 --- /dev/null +++ b/tests/integration/targets/test_mysql_user/tasks/utils/assert_plugin.yml @@ -0,0 +1,11 @@ +--- + +- name: Utils | Assert plugin | Query for user {{ user_name }} + ansible.builtin.command: "{{ mysql_command }} -e \"SELECT plugin FROM mysql.user where user='{{ user_name }}'\"" + register: result + changed_when: False + +- name: Utils | Assert plugin | Assert plugin is correct + ansible.builtin.assert: + that: + - plugin_type in result.stdout From 59c26211ca325553214105e52f460d3bf035e561 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Mon, 2 Sep 2024 18:07:11 +0200 Subject: [PATCH 24/54] mysql_user: deprecate alias user for name argument (#670) * mysql_user: deprecate alias user for name argument * Fix module and tests --- changelogs/fragments/0-mysql_user.yml | 2 ++ plugins/modules/mysql_user.py | 10 ++++++++-- .../targets/test_mysql_user/tasks/issue-265.yml | 8 ++++---- .../targets/test_mysql_user/tasks/test_idempotency.yml | 4 ++-- 4 files changed, 16 insertions(+), 8 deletions(-) create mode 100644 changelogs/fragments/0-mysql_user.yml diff --git a/changelogs/fragments/0-mysql_user.yml b/changelogs/fragments/0-mysql_user.yml new file mode 100644 index 0000000..b75533f --- /dev/null +++ b/changelogs/fragments/0-mysql_user.yml @@ -0,0 +1,2 @@ +breaking_changes: +- mysql_user - the ``user`` alias of the ``name`` argument has been deprecated and will be removed in collection version 5.0.0. Use the ``name`` argument instead. diff --git a/plugins/modules/mysql_user.py b/plugins/modules/mysql_user.py index 2ee5e01..78f11a9 100644 --- a/plugins/modules/mysql_user.py +++ b/plugins/modules/mysql_user.py @@ -439,7 +439,13 @@ from ansible.module_utils._text import to_native def main(): argument_spec = mysql_common_argument_spec() argument_spec.update( - user=dict(type='str', required=True, aliases=['name']), + name=dict(type='str', required=True, aliases=['user'], deprecated_aliases=[ + { + 'name': 'user', + 'version': '5.0.0', + 'collection_name': 'community.mysql', + }], + ), password=dict(type='str', no_log=True), encrypted=dict(type='bool', default=False), host=dict(type='str', default='localhost'), @@ -471,7 +477,7 @@ def main(): ) login_user = module.params["login_user"] login_password = module.params["login_password"] - user = module.params["user"] + user = module.params["name"] password = module.params["password"] encrypted = module.boolean(module.params["encrypted"]) host = module.params["host"].lower() diff --git a/tests/integration/targets/test_mysql_user/tasks/issue-265.yml b/tests/integration/targets/test_mysql_user/tasks/issue-265.yml index 2d8db77..dfceda7 100644 --- a/tests/integration/targets/test_mysql_user/tasks/issue-265.yml +++ b/tests/integration/targets/test_mysql_user/tasks/issue-265.yml @@ -64,7 +64,7 @@ - name: Issue-265 | Remove blank mysql user with hosts=all (expect changed) mysql_user: <<: *mysql_params - user: "" + name: "" host_all: true state: absent force_context: yes @@ -78,7 +78,7 @@ - name: Issue-265 | Remove blank mysql user with hosts=all (expect ok) mysql_user: <<: *mysql_params - user: "" + name: "" host_all: true force_context: yes state: absent @@ -151,7 +151,7 @@ - name: Issue-265 | Remove blank mysql user with hosts=all (expect changed) mysql_user: <<: *mysql_params - user: "" + name: "" host_all: true state: absent force_context: no @@ -165,7 +165,7 @@ - name: Issue-265 | Remove blank mysql user with hosts=all (expect ok) mysql_user: <<: *mysql_params - user: "" + name: "" host_all: true force_context: no state: absent diff --git a/tests/integration/targets/test_mysql_user/tasks/test_idempotency.yml b/tests/integration/targets/test_mysql_user/tasks/test_idempotency.yml index fb60139..f76934b 100644 --- a/tests/integration/targets/test_mysql_user/tasks/test_idempotency.yml +++ b/tests/integration/targets/test_mysql_user/tasks/test_idempotency.yml @@ -66,7 +66,7 @@ - name: Idempotency | Remove blank user with hosts=all (expect changed) mysql_user: <<: *mysql_params - user: "" + name: "" host_all: true state: absent register: result @@ -79,7 +79,7 @@ - name: Idempotency | Remove blank user with hosts=all (expect ok) mysql_user: <<: *mysql_params - user: "" + name: "" host_all: true state: absent register: result From 2db131f8c054ce1dee88eb1f575603aaec4c4c8b Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Wed, 4 Sep 2024 07:19:59 +0200 Subject: [PATCH 25/54] Release 3.10.1 commit (#673) --- CHANGELOG.rst | 19 +++++++++++++++++++ changelogs/changelog.yaml | 17 +++++++++++++++++ changelogs/fragments/0-mysql_user.yml | 2 -- .../fragments/596-fix-check-changes.yaml | 2 -- galaxy.yml | 2 +- 5 files changed, 37 insertions(+), 5 deletions(-) delete mode 100644 changelogs/fragments/0-mysql_user.yml delete mode 100644 changelogs/fragments/596-fix-check-changes.yaml diff --git a/CHANGELOG.rst b/CHANGELOG.rst index c5039ed..19b018b 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -6,6 +6,25 @@ Community MySQL and MariaDB Collection Release Notes This changelog describes changes after version 2.0.0. +v3.10.1 +======= + +Release Summary +--------------- + +This is a patch release of the ``community.mysql`` collection. +Besides a bugfix, it contains an important upcoming breaking-change information. + +Breaking Changes / Porting Guide +-------------------------------- + +- mysql_user - the ``user`` alias of the ``name`` argument has been deprecated and will be removed in collection version 5.0.0. Use the ``name`` argument instead. + +Bugfixes +-------- + +- mysql_user - module makes changes when is executed with ``plugin_auth_string`` parameter and check mode. + v3.10.0 ======= diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index 8c18264..1b8048a 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -156,6 +156,23 @@ releases: - mysql_user_tls_requires.yml - supports_mysql_change_replication_source_to.yml release_date: '2024-08-22' + 3.10.1: + changes: + breaking_changes: + - mysql_user - the ``user`` alias of the ``name`` argument has been deprecated + and will be removed in collection version 5.0.0. Use the ``name`` argument + instead. + bugfixes: + - mysql_user - module makes changes when is executed with ``plugin_auth_string`` + parameter and check mode. + release_summary: 'This is a patch release of the ``community.mysql`` collection. + + Besides a bugfix, it contains an important upcoming breaking-change information.' + fragments: + - 0-mysql_user.yml + - 3.10.1.yml + - 596-fix-check-changes.yaml + release_date: '2024-09-04' 3.2.0: changes: bugfixes: diff --git a/changelogs/fragments/0-mysql_user.yml b/changelogs/fragments/0-mysql_user.yml deleted file mode 100644 index b75533f..0000000 --- a/changelogs/fragments/0-mysql_user.yml +++ /dev/null @@ -1,2 +0,0 @@ -breaking_changes: -- mysql_user - the ``user`` alias of the ``name`` argument has been deprecated and will be removed in collection version 5.0.0. Use the ``name`` argument instead. diff --git a/changelogs/fragments/596-fix-check-changes.yaml b/changelogs/fragments/596-fix-check-changes.yaml deleted file mode 100644 index e7c24f1..0000000 --- a/changelogs/fragments/596-fix-check-changes.yaml +++ /dev/null @@ -1,2 +0,0 @@ -bugfixes: - - mysql_user - module makes changes when is executed with ``plugin_auth_string`` parameter and check mode. diff --git a/galaxy.yml b/galaxy.yml index 353a6f8..ffcb55b 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,7 +1,7 @@ --- namespace: community name: mysql -version: 3.10.0 +version: 3.10.1 readme: README.md authors: - Ansible community From 3425fdb839615203e50a84b3e2ee07f5c2da4b67 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Thu, 5 Sep 2024 12:19:33 +0200 Subject: [PATCH 26/54] mysql_user: add correct ed25519 plugin handling when creating a user (#674) --- changelogs/fragments/0-mysql_user.yml | 2 ++ plugins/module_utils/user.py | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) create mode 100644 changelogs/fragments/0-mysql_user.yml diff --git a/changelogs/fragments/0-mysql_user.yml b/changelogs/fragments/0-mysql_user.yml new file mode 100644 index 0000000..61a9a01 --- /dev/null +++ b/changelogs/fragments/0-mysql_user.yml @@ -0,0 +1,2 @@ +bugfixes: +- mysql_user - add correct ``ed25519`` auth plugin handling when creating a user (https://github.com/ansible-collections/community.mysql/issues/672). diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index 7d7d304..58ed607 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -212,7 +212,7 @@ def user_add(cursor, user, host, host_all, password, encrypted, query_with_args = "CREATE USER %s@%s IDENTIFIED WITH %s AS %s", (user, host, plugin, plugin_hash_string) elif plugin and plugin_auth_string: # Mysql and MariaDB differ in naming pam plugin and Syntax to set it - if plugin == 'pam': # Used by MariaDB which requires the USING keyword, not BY + if plugin in ('pam', 'ed25519'): # Used by MariaDB which requires the USING keyword, not BY query_with_args = "CREATE USER %s@%s IDENTIFIED WITH %s USING %s", (user, host, plugin, plugin_auth_string) elif salt: if plugin in ['caching_sha2_password', 'sha256_password']: From 7188bea0c827fab6e190984c4d6fd3acb3668e35 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Fri, 6 Sep 2024 08:21:45 +0200 Subject: [PATCH 27/54] Release 3.10.2 commit (#675) --- CHANGELOG.rst | 15 +++++++++++++++ changelogs/changelog.yaml | 14 ++++++++++++++ changelogs/fragments/0-mysql_user.yml | 2 -- galaxy.yml | 2 +- 4 files changed, 30 insertions(+), 3 deletions(-) delete mode 100644 changelogs/fragments/0-mysql_user.yml diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 19b018b..55e08f2 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -6,6 +6,21 @@ Community MySQL and MariaDB Collection Release Notes This changelog describes changes after version 2.0.0. +v3.10.2 +======= + +Release Summary +--------------- + +This is a bugfix release of the ``community.mysql`` collection. +This changelog contains all changes to the modules and plugins in this +collection that have been made after the previous release. + +Bugfixes +-------- + +- mysql_user - add correct ``ed25519`` auth plugin handling when creating a user (https://github.com/ansible-collections/community.mysql/issues/672). + v3.10.1 ======= diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index 1b8048a..56b9a53 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -173,6 +173,20 @@ releases: - 3.10.1.yml - 596-fix-check-changes.yaml release_date: '2024-09-04' + 3.10.2: + changes: + bugfixes: + - mysql_user - add correct ``ed25519`` auth plugin handling when creating a + user (https://github.com/ansible-collections/community.mysql/issues/672). + release_summary: 'This is a bugfix release of the ``community.mysql`` collection. + + This changelog contains all changes to the modules and plugins in this + + collection that have been made after the previous release.' + fragments: + - 0-mysql_user.yml + - 3.10.2.yml + release_date: '2024-09-06' 3.2.0: changes: bugfixes: diff --git a/changelogs/fragments/0-mysql_user.yml b/changelogs/fragments/0-mysql_user.yml deleted file mode 100644 index 61a9a01..0000000 --- a/changelogs/fragments/0-mysql_user.yml +++ /dev/null @@ -1,2 +0,0 @@ -bugfixes: -- mysql_user - add correct ``ed25519`` auth plugin handling when creating a user (https://github.com/ansible-collections/community.mysql/issues/672). diff --git a/galaxy.yml b/galaxy.yml index ffcb55b..99a5a39 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,7 +1,7 @@ --- namespace: community name: mysql -version: 3.10.1 +version: 3.10.2 readme: README.md authors: - Ansible community From eec6e7091f5dd1ecb7fbb114be7b8c71e94d909e Mon Sep 17 00:00:00 2001 From: hubiongithub <79990207+hubiongithub@users.noreply.github.com> Date: Mon, 9 Sep 2024 15:01:26 +0200 Subject: [PATCH 28/54] Update user.py (#676) * Update user.py Added correct syntax to ed25519 password plugin. on create user on update user This only accepts cleartext passwords (PASSWORD(%s)) not pregenerated ed25519 hashes. * Update plugins/module_utils/user.py Co-authored-by: Andrew Klychkov * Update plugins/module_utils/user.py Co-authored-by: Andrew Klychkov * Update plugins/module_utils/user.py Co-authored-by: Andrew Klychkov * Update plugins/module_utils/user.py Co-authored-by: Andrew Klychkov * Update plugins/module_utils/user.py * Update plugins/module_utils/user.py --------- Co-authored-by: Andrew Klychkov --- plugins/module_utils/user.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index 58ed607..7b6914f 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -212,8 +212,10 @@ def user_add(cursor, user, host, host_all, password, encrypted, query_with_args = "CREATE USER %s@%s IDENTIFIED WITH %s AS %s", (user, host, plugin, plugin_hash_string) elif plugin and plugin_auth_string: # Mysql and MariaDB differ in naming pam plugin and Syntax to set it - if plugin in ('pam', 'ed25519'): # Used by MariaDB which requires the USING keyword, not BY + if plugin == 'pam': # Used by MariaDB which requires the USING keyword, not BY query_with_args = "CREATE USER %s@%s IDENTIFIED WITH %s USING %s", (user, host, plugin, plugin_auth_string) + elif plugin == 'ed25519': # Used by MariaDB which requires the USING keyword, not BY + query_with_args = "CREATE USER %s@%s IDENTIFIED WITH %s USING PASSWORD(%s)", (user, host, plugin, plugin_auth_string) elif salt: if plugin in ['caching_sha2_password', 'sha256_password']: generated_hash_string = mysql_sha256_password_hash_hex(password=plugin_auth_string, salt=salt) @@ -398,8 +400,10 @@ def user_mod(cursor, user, host, host_all, password, encrypted, query_with_args = "ALTER USER %s@%s IDENTIFIED WITH %s AS %s", (user, host, plugin, plugin_hash_string) elif plugin_auth_string: # Mysql and MariaDB differ in naming pam plugin and syntax to set it - if plugin in ('pam', 'ed25519'): + if plugin == 'pam': query_with_args = "ALTER USER %s@%s IDENTIFIED WITH %s USING %s", (user, host, plugin, plugin_auth_string) + elif plugin == 'ed25519': + query_with_args = "ALTER USER %s@%s IDENTIFIED WITH %s USING PASSWORD(%s)", (user, host, plugin, plugin_auth_string) elif salt: if plugin in ['caching_sha2_password', 'sha256_password']: generated_hash_string = mysql_sha256_password_hash_hex(password=plugin_auth_string, salt=salt) From a75d71a7ff9d6929a08616b90ee4b50d2b15b841 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Mon, 9 Sep 2024 15:05:25 +0200 Subject: [PATCH 29/54] Release 3.10.3 commit (#678) --- CHANGELOG.rst | 15 +++++++++++++++ changelogs/changelog.yaml | 14 ++++++++++++++ galaxy.yml | 2 +- 3 files changed, 30 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 55e08f2..76d83fe 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -6,6 +6,21 @@ Community MySQL and MariaDB Collection Release Notes This changelog describes changes after version 2.0.0. +v3.10.3 +======= + +Release Summary +--------------- + +This is a bugfix release of the ``community.mysql`` collection. +This changelog contains all changes to the modules and plugins in this +collection that have been made after the previous release. + +Bugfixes +-------- + +- mysql_user - add correct ``ed25519`` auth plugin handling when creating a user (https://github.com/ansible-collections/community.mysql/pull/676). + v3.10.2 ======= diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index 56b9a53..ea7c09f 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -187,6 +187,20 @@ releases: - 0-mysql_user.yml - 3.10.2.yml release_date: '2024-09-06' + 3.10.3: + changes: + bugfixes: + - mysql_user - add correct ``ed25519`` auth plugin handling when creating a + user (https://github.com/ansible-collections/community.mysql/pull/676). + release_summary: 'This is a bugfix release of the ``community.mysql`` collection. + + This changelog contains all changes to the modules and plugins in this + + collection that have been made after the previous release.' + fragments: + - 0-mysql_user.yml + - 3.10.3.yml + release_date: '2024-09-09' 3.2.0: changes: bugfixes: diff --git a/galaxy.yml b/galaxy.yml index 99a5a39..0046b5a 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,7 +1,7 @@ --- namespace: community name: mysql -version: 3.10.2 +version: 3.10.3 readme: README.md authors: - Ansible community From 28bf7093be36e0bd47866e28aefff1a38cc5b2b0 Mon Sep 17 00:00:00 2001 From: Maxwell G Date: Wed, 11 Sep 2024 07:35:02 -0500 Subject: [PATCH 30/54] changelogs: categorize deprecations under deprecated_features (#679) These should be put under deprecated_features so they show up properly in the generated changelog. --- CHANGELOG.rst | 8 ++++---- changelogs/changelog.yaml | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 76d83fe..cf1162f 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -45,8 +45,8 @@ Release Summary This is a patch release of the ``community.mysql`` collection. Besides a bugfix, it contains an important upcoming breaking-change information. -Breaking Changes / Porting Guide --------------------------------- +Deprecated Features +------------------- - mysql_user - the ``user`` alias of the ``name`` argument has been deprecated and will be removed in collection version 5.0.0. Use the ``name`` argument instead. @@ -75,8 +75,8 @@ Minor Changes - mysql_replication - Improve detection of IsReplica and IsPrimary by inspecting the dictionary returned from the SQL query instead of relying on variable types. This ensures compatibility with changes in the connector or the output of SHOW REPLICA STATUS and SHOW MASTER STATUS, allowing for easier maintenance if these change in the future. - mysql_user - Add salt parameter to generate static hash for `caching_sha2_password` and `sha256_password` plugins. -Breaking Changes / Porting Guide --------------------------------- +Deprecated Features +------------------- - collection - support of mysqlclient connector is deprecated - use PyMySQL connector instead! We will stop testing against it in collection version 4.0.0 and remove the related code in 5.0.0 (https://github.com/ansible-collections/community.mysql/issues/654). - mysql_info - The ``users_info`` filter returned variable ``plugin_auth_string`` contains the hashed password and it's misleading, it will be removed from community.mysql 4.0.0. Use the `plugin_hash_string` return value instead (https://github.com/ansible-collections/community.mysql/pull/629). diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index ea7c09f..27ae315 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -99,7 +99,7 @@ releases: release_date: '2022-04-26' 3.10.0: changes: - breaking_changes: + deprecated_features: - collection - support of mysqlclient connector is deprecated - use PyMySQL connector instead! We will stop testing against it in collection version 4.0.0 and remove the related code in 5.0.0 (https://github.com/ansible-collections/community.mysql/issues/654). @@ -158,7 +158,7 @@ releases: release_date: '2024-08-22' 3.10.1: changes: - breaking_changes: + deprecated_features: - mysql_user - the ``user`` alias of the ``name`` argument has been deprecated and will be removed in collection version 5.0.0. Use the ``name`` argument instead. From a5afa1a375ebd7dc676ff6ab6f7323ce0b88b299 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Thu, 26 Sep 2024 14:31:08 +0200 Subject: [PATCH 31/54] CI: add stable-2.18, fix README (#681) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * CI: add stable-2.18, fix README * Update .github/workflows/ansible-test-plugins.yml Co-authored-by: Laurent Indermühle * Update .github/workflows/ansible-test-plugins.yml Co-authored-by: Laurent Indermühle * Update .github/workflows/ansible-test-plugins.yml Co-authored-by: Laurent Indermühle * Update README.md Co-authored-by: Laurent Indermühle --------- Co-authored-by: Laurent Indermühle --- .github/workflows/ansible-test-plugins.yml | 6 +++--- README.md | 2 +- tests/sanity/ignore-2.19.txt | 3 +++ 3 files changed, 7 insertions(+), 4 deletions(-) create mode 100644 tests/sanity/ignore-2.19.txt diff --git a/.github/workflows/ansible-test-plugins.yml b/.github/workflows/ansible-test-plugins.yml index efc1537..ad8c4b5 100644 --- a/.github/workflows/ansible-test-plugins.yml +++ b/.github/workflows/ansible-test-plugins.yml @@ -22,9 +22,9 @@ jobs: strategy: matrix: ansible: - - stable-2.15 - stable-2.16 - stable-2.17 + - stable-2.18 - devel steps: # https://github.com/ansible-community/ansible-test-gh-action @@ -44,9 +44,9 @@ jobs: fail-fast: false matrix: ansible: - - stable-2.15 - stable-2.16 - stable-2.17 + - stable-2.18 - devel db_engine_name: - mysql @@ -282,9 +282,9 @@ jobs: fail-fast: true matrix: ansible: - - stable-2.15 - stable-2.16 - stable-2.17 + - stable-2.18 - devel python: - '3.8' diff --git a/README.md b/README.md index 1f5b47a..5db2f05 100644 --- a/README.md +++ b/README.md @@ -90,9 +90,9 @@ Here is the table for the support timeline: ### ansible-core -- stable-2.15 - stable-2.16 - stable-2.17 +- stable-2.18 - current development version ### Python diff --git a/tests/sanity/ignore-2.19.txt b/tests/sanity/ignore-2.19.txt new file mode 100644 index 0000000..152162d --- /dev/null +++ b/tests/sanity/ignore-2.19.txt @@ -0,0 +1,3 @@ +plugins/modules/mysql_db.py validate-modules:use-run-command-not-popen +plugins/module_utils/mysql.py pylint:unused-import +plugins/module_utils/version.py pylint:unused-import From 93cd1850d93b8b9356a8310e461dfb6bd6f989b7 Mon Sep 17 00:00:00 2001 From: JS <26802713+rujschafer@users.noreply.github.com> Date: Wed, 23 Oct 2024 04:31:40 -0400 Subject: [PATCH 32/54] Update mysql_user.py - table/privilege spacing update (#687) * Update mysql_user.py - table/privilege spacing update Add note for no spacing between the table and the privilege as this will make the task not idempotent in check mode but still make it idempotent when in normal mode. * Update plugins/modules/mysql_user.py Co-authored-by: Andrew Klychkov --------- Co-authored-by: Andrew Klychkov --- plugins/modules/mysql_user.py | 1 + 1 file changed, 1 insertion(+) diff --git a/plugins/modules/mysql_user.py b/plugins/modules/mysql_user.py index 78f11a9..cf210a3 100644 --- a/plugins/modules/mysql_user.py +++ b/plugins/modules/mysql_user.py @@ -46,6 +46,7 @@ options: priv: description: - "MySQL privileges string in the format: C(db.table:priv1,priv2)." + - Additionally, there must be no spaces between the table and the privilege as this will yield a non-idempotent check mode. - "Multiple privileges can be specified by separating each one using a forward slash: C(db.table1:priv/db.table2:priv)." - The format is based on MySQL C(GRANT) statement. From 90bd0b0a75e2dd8b893058cf98b5bc98ca0ac5d6 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Thu, 24 Oct 2024 10:57:36 +0200 Subject: [PATCH 33/54] Update contributor's email (#684) --- plugins/modules/mysql_info.py | 2 +- plugins/modules/mysql_query.py | 2 +- plugins/modules/mysql_replication.py | 2 +- plugins/modules/mysql_role.py | 2 +- tests/integration/old_mariadb_replication/tasks/main.yml | 2 +- .../old_mariadb_replication/tasks/mariadb_master_use_gtid.yml | 2 +- .../tasks/mariadb_replication_connection_name.yml | 2 +- .../tasks/mariadb_replication_initial.yml | 2 +- tests/integration/targets/test_mysql_info/tasks/main.yml | 2 +- .../targets/test_mysql_query/tasks/mysql_query_initial.yml | 2 +- tests/integration/targets/test_mysql_replication/tasks/main.yml | 2 +- .../test_mysql_replication/tasks/mysql_replication_channel.yml | 2 +- .../test_mysql_replication/tasks/mysql_replication_initial.yml | 2 +- .../tasks/mysql_replication_primary_delay.yml | 2 +- .../tasks/mysql_replication_resetprimary_mode.yml | 2 +- tests/unit/plugins/module_utils/test_mariadb_replication.py | 2 +- tests/unit/plugins/module_utils/test_mysql_replication.py | 2 +- 17 files changed, 17 insertions(+), 17 deletions(-) diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index 2d1fe94..3a30597 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -1,7 +1,7 @@ #!/usr/bin/python # -*- coding: utf-8 -*- -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) from __future__ import absolute_import, division, print_function diff --git a/plugins/modules/mysql_query.py b/plugins/modules/mysql_query.py index 13a07de..2cdf096 100644 --- a/plugins/modules/mysql_query.py +++ b/plugins/modules/mysql_query.py @@ -1,7 +1,7 @@ #!/usr/bin/python # -*- coding: utf-8 -*- -# Copyright: (c) 2020, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2020, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) from __future__ import (absolute_import, division, print_function) diff --git a/plugins/modules/mysql_replication.py b/plugins/modules/mysql_replication.py index 723fc35..35659d3 100644 --- a/plugins/modules/mysql_replication.py +++ b/plugins/modules/mysql_replication.py @@ -2,7 +2,7 @@ # -*- coding: utf-8 -*- # Copyright: (c) 2013, Balazs Pocze -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # Certain parts are taken from Mark Theunissen's mysqldb module # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) diff --git a/plugins/modules/mysql_role.py b/plugins/modules/mysql_role.py index 032b41e..c88392b 100644 --- a/plugins/modules/mysql_role.py +++ b/plugins/modules/mysql_role.py @@ -1,7 +1,7 @@ #!/usr/bin/python # -*- coding: utf-8 -*- -# Copyright: (c) 2021, Andrew Klychkov +# Copyright: (c) 2021, Andrew Klychkov # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) from __future__ import absolute_import, division, print_function diff --git a/tests/integration/old_mariadb_replication/tasks/main.yml b/tests/integration/old_mariadb_replication/tasks/main.yml index 4ea76a9..321ba4d 100644 --- a/tests/integration/old_mariadb_replication/tasks/main.yml +++ b/tests/integration/old_mariadb_replication/tasks/main.yml @@ -1,4 +1,4 @@ -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) # Initial CI tests of mysql_replication module diff --git a/tests/integration/old_mariadb_replication/tasks/mariadb_master_use_gtid.yml b/tests/integration/old_mariadb_replication/tasks/mariadb_master_use_gtid.yml index 699b61f..8977c10 100644 --- a/tests/integration/old_mariadb_replication/tasks/mariadb_master_use_gtid.yml +++ b/tests/integration/old_mariadb_replication/tasks/mariadb_master_use_gtid.yml @@ -1,4 +1,4 @@ -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) # Tests for master_use_gtid parameter. diff --git a/tests/integration/old_mariadb_replication/tasks/mariadb_replication_connection_name.yml b/tests/integration/old_mariadb_replication/tasks/mariadb_replication_connection_name.yml index 3928c78..337a839 100644 --- a/tests/integration/old_mariadb_replication/tasks/mariadb_replication_connection_name.yml +++ b/tests/integration/old_mariadb_replication/tasks/mariadb_replication_connection_name.yml @@ -1,4 +1,4 @@ -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) # Needs for further tests: diff --git a/tests/integration/old_mariadb_replication/tasks/mariadb_replication_initial.yml b/tests/integration/old_mariadb_replication/tasks/mariadb_replication_initial.yml index f65d090..1a95a55 100644 --- a/tests/integration/old_mariadb_replication/tasks/mariadb_replication_initial.yml +++ b/tests/integration/old_mariadb_replication/tasks/mariadb_replication_initial.yml @@ -1,4 +1,4 @@ -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) # Preparation: diff --git a/tests/integration/targets/test_mysql_info/tasks/main.yml b/tests/integration/targets/test_mysql_info/tasks/main.yml index 93570f2..42350c6 100644 --- a/tests/integration/targets/test_mysql_info/tasks/main.yml +++ b/tests/integration/targets/test_mysql_info/tasks/main.yml @@ -5,7 +5,7 @@ #################################################################### # Test code for mysql_info module -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) ################### diff --git a/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml b/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml index 82665af..fbf5ca8 100644 --- a/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml +++ b/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml @@ -1,6 +1,6 @@ --- # Test code for mysql_query module -# Copyright: (c) 2020, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2020, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) - vars: mysql_parameters: &mysql_params diff --git a/tests/integration/targets/test_mysql_replication/tasks/main.yml b/tests/integration/targets/test_mysql_replication/tasks/main.yml index a65cabd..32ce553 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/main.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/main.yml @@ -4,7 +4,7 @@ # and should not be used as examples of how to write Ansible roles # #################################################################### -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) # Initial CI tests of mysql_replication module: diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml index 802865c..0bcc6e6 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml @@ -1,5 +1,5 @@ --- -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) - vars: diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml index 30cd99f..00699c1 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml @@ -1,5 +1,5 @@ --- -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) - vars: diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml index 3ae4339..2093b70 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml @@ -1,4 +1,4 @@ -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) - vars: diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_resetprimary_mode.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_resetprimary_mode.yml index 8968049..cdd5fa7 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_resetprimary_mode.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_resetprimary_mode.yml @@ -1,5 +1,5 @@ --- -# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) - vars: diff --git a/tests/unit/plugins/module_utils/test_mariadb_replication.py b/tests/unit/plugins/module_utils/test_mariadb_replication.py index deb3099..513d8cf 100644 --- a/tests/unit/plugins/module_utils/test_mariadb_replication.py +++ b/tests/unit/plugins/module_utils/test_mariadb_replication.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Copyright: (c) 2020, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2020, Andrew Klychkov (@Andersson007) from __future__ import (absolute_import, division, print_function) __metaclass__ = type diff --git a/tests/unit/plugins/module_utils/test_mysql_replication.py b/tests/unit/plugins/module_utils/test_mysql_replication.py index 96d4d9a..c4126a5 100644 --- a/tests/unit/plugins/module_utils/test_mysql_replication.py +++ b/tests/unit/plugins/module_utils/test_mysql_replication.py @@ -1,5 +1,5 @@ # -*- coding: utf-8 -*- -# Copyright: (c) 2020, Andrew Klychkov (@Andersson007) +# Copyright: (c) 2020, Andrew Klychkov (@Andersson007) from __future__ import (absolute_import, division, print_function) __metaclass__ = type From ebb37ae7a3b126603cfe4066aa69e3e9c7cc93e7 Mon Sep 17 00:00:00 2001 From: Soledad208 Date: Thu, 7 Nov 2024 15:56:31 +0700 Subject: [PATCH 34/54] sql_mode can be set in session, therefore we should look for ANSI_QUOTES in session variable instead of global variable (#677) * issue-671: get ASNI_QUOTES from session sql_mode instead of GLOBAL sql_mode --- .../fragments/671-modules_util_user.yml | 12 ++ plugins/module_utils/user.py | 2 +- .../test_mysql_user/tasks/issue-671.yaml | 112 ++++++++++++++++++ .../targets/test_mysql_user/tasks/main.yml | 6 + 4 files changed, 131 insertions(+), 1 deletion(-) create mode 100644 changelogs/fragments/671-modules_util_user.yml create mode 100644 tests/integration/targets/test_mysql_user/tasks/issue-671.yaml diff --git a/changelogs/fragments/671-modules_util_user.yml b/changelogs/fragments/671-modules_util_user.yml new file mode 100644 index 0000000..a913651 --- /dev/null +++ b/changelogs/fragments/671-modules_util_user.yml @@ -0,0 +1,12 @@ +bugfixes: + - mysql_user,mysql_role - The sql_mode ANSI_QUOTES affects how the modules mysql_user + and mysql_role compare the existing privileges with the configured privileges, + as well as decide whether double quotes or backticks should be used in the GRANT + statements. Pointing out in issue 671, the modules mysql_user and mysql_role allow + users to enable/disable ANSI_QUOTES in session variable (within a DB session, the + session variable always overwrites the global one). But due to the issue, the modules + do not check for ANSI_MODE in the session variable, instead, they only check in the + GLOBAL one.That behavior is not only limiting the users' flexibility, but also not + allowing users to explicitly disable ANSI_MODE to work around such bugs like + https://bugs.mysql.com/bug.php?id=115953. + (https://github.com/ansible-collections/community.mysql/issues/671) \ No newline at end of file diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index 7b6914f..307ef6e 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -32,7 +32,7 @@ class InvalidPrivsError(Exception): def get_mode(cursor): - cursor.execute('SELECT @@GLOBAL.sql_mode') + cursor.execute('SELECT @@sql_mode') result = cursor.fetchone() mode_str = result[0] if 'ANSI' in mode_str: diff --git a/tests/integration/targets/test_mysql_user/tasks/issue-671.yaml b/tests/integration/targets/test_mysql_user/tasks/issue-671.yaml new file mode 100644 index 0000000..3696cf0 --- /dev/null +++ b/tests/integration/targets/test_mysql_user/tasks/issue-671.yaml @@ -0,0 +1,112 @@ +--- +# Due to https://bugs.mysql.com/bug.php?id=115953, in Mysql 8, if ANSI_QUOTES is enabled, +# backticks will be used instead of double quotes to quote functions or procedures name. +# As a consequence, mysql_user and mysql_roles will always report "changed" for functions +# and procedures no matter the privileges are granted or not. +# Workaround for the mysql bug 116953 is removing ANSI_QUOTES from the module's session +# sql_mode. But because issue 671, ANSI_QUOTES is always got from GLOBAL sql_mode, thus +# this workaround can't work. Even without the Mysql bug, because sql_mode in session +# precedes GLOBAL sql_mode. we should check for sql_mode in session variable instead of +# the GLOBAL one. +- vars: + mysql_parameters: &mysql_params + login_user: '{{ mysql_user }}' + login_password: '{{ mysql_password }}' + login_host: '{{ mysql_host }}' + login_port: '{{ mysql_primary_port }}' + + block: + - name: Issue-671| test setup | drop database + community.mysql.mysql_db: + <<: *mysql_params + name: "{{ item }}" + state: absent + loop: + - foo + - bar + + - name: Issue-671| test setup | create database + community.mysql.mysql_db: + <<: *mysql_params + name: "{{ item }}" + state: present + loop: + - foo + - bar + + - name: Issue-671| test setup | get value of GLOBAL.sql_mode + community.mysql.mysql_query: + <<: *mysql_params + query: 'select @@GLOBAL.sql_mode AS sql_mode' + register: sql_mode_orig + + - name: Issue-671| Assert sql_mode_orig + ansible.builtin.assert: + that: + - sql_mode_orig.query_result[0][0].sql_mode != None + + - name: Issue-671| enable sql_mode ANSI_QUOTES + community.mysql.mysql_variables: + <<: *mysql_params + variable: sql_mode + value: '{{ sql_mode_orig.query_result[0][0].sql_mode }},ANSI_QUOTES' + mode: "{% if db_engine == 'mariadb' %}global{% else %}persist{% endif %}" + + - name: Issue-671| Copy SQL scripts to remote + ansible.builtin.copy: + src: "{{ item }}" + dest: "{{ remote_tmp_dir }}/{{ item | basename }}" + loop: + - create-function.sql + - create-procedure.sql + + - name: Issue-671| Create function for test + ansible.builtin.shell: + cmd: "{{ mysql_command }} < {{ remote_tmp_dir }}/create-function.sql" + + - name: Issue-671| Create procedure for test + ansible.builtin.shell: + cmd: "{{ mysql_command }} < {{ remote_tmp_dir }}/create-procedure.sql" + + - name: Issue-671| Create user with FUNCTION and PROCEDURE privileges + community.mysql.mysql_user: + <<: *mysql_params + name: '{{ user_name_2 }}' + password: '{{ user_password_2 }}' + state: present + priv: 'FUNCTION foo.function:EXECUTE/foo.*:SELECT/PROCEDURE bar.procedure:EXECUTE' + + - name: Issue-671| Grant the privileges again, remove ANSI_QUOTES from the session variable + community.mysql.mysql_user: + <<: *mysql_params + session_vars: + sql_mode: "" + name: '{{ user_name_2 }}' + password: '{{ user_password_2 }}' + state: present + priv: 'FUNCTION foo.function:EXECUTE/foo.*:SELECT/PROCEDURE bar.procedure:EXECUTE' + register: result + failed_when: + - result is failed or result is changed + + - name: Issue-671| Test teardown | cleanup databases + community.mysql.mysql_db: + <<: *mysql_params + name: "{{ item }}" + state: absent + loop: + - foo + - bar + + - name: Issue-671| set sql_mode back to original value + community.mysql.mysql_variables: + <<: *mysql_params + variable: sql_mode + value: '{{ sql_mode_orig.query_result[0][0].sql_mode }}' + mode: "{% if db_engine == 'mariadb' %}global{% else %}persist{% endif %}" + + - name: Issue-671| Teardown user_name_2 + ansible.builtin.include_tasks: + file: utils/remove_user.yml + vars: + user_name: "{{ user_name_2 }}" \ No newline at end of file diff --git a/tests/integration/targets/test_mysql_user/tasks/main.yml b/tests/integration/targets/test_mysql_user/tasks/main.yml index e77c443..9244570 100644 --- a/tests/integration/targets/test_mysql_user/tasks/main.yml +++ b/tests/integration/targets/test_mysql_user/tasks/main.yml @@ -282,6 +282,12 @@ - import_tasks: issue-64560.yaml tags: - issue-64560 + + - name: Test ANSI_QUOTES + ansible.builtin.import_tasks: + file: issue-671.yaml + tags: + - issue-671 # Test that mysql_user still works with force_context enabled (database set to "mysql") # (https://github.com/ansible-collections/community.mysql/issues/265) From 7d787eb238738e158f6ad8626d65b61a0a94b902 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Thu, 7 Nov 2024 10:37:10 +0100 Subject: [PATCH 35/54] Add contributors from last 10 PR pages (#688) I've applied a sort on the whole file. This Patch is hard to read, sorry. I've remove nobody! Only move! --- CONTRIBUTORS | 36 +++++++++++++++++++++++++++++++----- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/CONTRIBUTORS b/CONTRIBUTORS index 06fb579..6d946cc 100644 --- a/CONTRIBUTORS +++ b/CONTRIBUTORS @@ -17,9 +17,11 @@ amitk79 amree Andersson007 andrewhowdencom +aneustroev ansibot anthonyxpalermo antonioribeiro +Aohzan apollo13 aquach arcmop @@ -33,6 +35,8 @@ baldpale banyek BarbzYHOOL Berbe +betanummeric +bigo8525 bizmate bjne bmalynovytch @@ -46,6 +50,7 @@ candeira caphrim007 cdalbergue checkphi +chriscroome chrismeyersfsu ChristopherGAndrews cmodijk @@ -56,13 +61,14 @@ CormacBracken cosmix cptMikky crashes +d-lee +d-rupp dagwieers damianmoore Davidffry denisemauldin +dennisurtubia diclophis -d-lee -d-rupp dmp1ce dnelson dramaley @@ -72,9 +78,11 @@ DSpeichert dungdm93 dwagelaar dylanjbarth -einarc E-M +einarc +elpavel eowin +eRadical Ernest0x esamattis Everspace @@ -82,24 +90,30 @@ F21 faitno felixfontein flatrocks +FlorianPerrot fourjay fraff +francescsanjuanmrf g00fy- geerlingguy georgeOsdDev ghjm ghost +GhostLyrics giacmir giorgio-v gkoller +gotmax23 gottwald gstorme gundalow hansbaer hchargois hluaces +hubiongithub hwali hyperfocus1338 +IBims1NicerTobi igormukhingmailcom imjoseangel infigoKriti @@ -164,8 +178,8 @@ markdorison markotitel marktheunissen markuman -mattclay matt-horwood-mayden +mattclay mavimo maxamillion maxbube @@ -184,11 +198,15 @@ mkrizek mmoya mohag mohsenSy +moledzki mpdehaan +MRMegaNova MRwangyd +mstinsky mverwijs mvgrimes mysqlbox +n-cc netmonk nhojpatrick nicolas-g @@ -202,7 +220,9 @@ organman91 p53 pakal paulbadcock +paulcampbell-ayroc pennycoders +perlun petoju petracvv pgrenaud @@ -223,12 +243,14 @@ richlv riupie rndmh3ro robertdebock +robertsilen robpblake rokka-n Roxyrob roysmith rsicart rthouvenin +rujschafer ruudk samccann samdoran @@ -242,6 +264,7 @@ shrikeh sivel skalfyfan skoriy88 +SoledaD208 sperantus spoyd steverweber @@ -262,19 +285,22 @@ time-palominodb timorunge Tomasthanes tomdymond +tompal3 Tronde tuhoanganh tvlooy tyll UncertaintyP unnecessary-username +v-zhuravlev vamshi8 vanne vdboor vmahadev -v-zhuravlev +webknjaz webmat wedi +wfelipew whysthatso willthames windowsansiblernew From d613fa19938d24ce6adccf792040d2f849ca3083 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Mon, 18 Nov 2024 15:44:39 +0100 Subject: [PATCH 36/54] Fix wrong documentation assertion (#690) --- plugins/modules/mysql_db.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/modules/mysql_db.py b/plugins/modules/mysql_db.py index 4a2c954..e1d1a7a 100644 --- a/plugins/modules/mysql_db.py +++ b/plugins/modules/mysql_db.py @@ -159,7 +159,7 @@ options: pipefail: description: - Use C(bash) instead of C(sh) and add C(-o pipefail) to catch errors from the - mysql_dump command when I(state=import) and compression is used. + mysql_dump command when I(state=dump) and compression is used. - The default is C(no) to prevent issues on systems without bash as a default interpreter. - The default will change to C(yes) in community.mysql 4.0.0. type: bool From 9057637844d81cc84ac7f0d9a80bfa1df2de3275 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Tue, 19 Nov 2024 08:51:03 +0100 Subject: [PATCH 37/54] mysql_info - add table count to the databases returned values (#691) * Add tables count per database * Add integrations tests * Deduplicate tests between main and new task file --- .../591-mysql_info-db_tables_count.yml | 3 + plugins/modules/mysql_info.py | 65 +++---- .../tasks/filter_databases.yml | 161 ++++++++++++++++++ .../targets/test_mysql_info/tasks/main.yml | 89 +--------- 4 files changed, 202 insertions(+), 116 deletions(-) create mode 100644 changelogs/fragments/591-mysql_info-db_tables_count.yml create mode 100644 tests/integration/targets/test_mysql_info/tasks/filter_databases.yml diff --git a/changelogs/fragments/591-mysql_info-db_tables_count.yml b/changelogs/fragments/591-mysql_info-db_tables_count.yml new file mode 100644 index 0000000..abbc1cb --- /dev/null +++ b/changelogs/fragments/591-mysql_info-db_tables_count.yml @@ -0,0 +1,3 @@ +--- +minor_changes: + - mysql_info - adds the count of tables for each database to the returned values. It is possible to exclude this new field using the ``db_table_count`` exclusion filter. (https://github.com/ansible-collections/community.mysql/pull/691) diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index 3a30597..8c3845d 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -35,7 +35,7 @@ options: exclude_fields: description: - List of fields which are not needed to collect. - - "Supports elements: C(db_size). Unsupported elements will be ignored." + - "Supports elements: C(db_size), C(db_table_count). Unsupported elements will be ignored." type: list elements: str version_added: '0.1.0' @@ -204,13 +204,19 @@ databases: returned: if not excluded by filter type: dict sample: - - { "mysql": { "size": 656594 }, "information_schema": { "size": 73728 } } + - { "mysql": { "size": 656594, "tables": 31 }, "information_schema": { "size": 73728, "tables": 79 } } contains: size: description: Database size in bytes. returned: if not excluded by filter type: dict sample: { 'size': 656594 } + tables: + description: Count of tables and views in that database. + returned: if not excluded by filter + type: dict + sample: { 'tables': 12 } + version_added: '3.11.0' settings: description: Global settings (variables) information. returned: if not excluded by filter @@ -656,40 +662,39 @@ class MySQL_Info(object): def __get_databases(self, exclude_fields, return_empty_dbs): """Get info about databases.""" - if not exclude_fields: - query = ('SELECT table_schema AS "name", ' - 'SUM(data_length + index_length) AS "size" ' - 'FROM information_schema.TABLES GROUP BY table_schema') - else: - if 'db_size' in exclude_fields: - query = ('SELECT table_schema AS "name" ' - 'FROM information_schema.TABLES GROUP BY table_schema') - res = self.__exec_sql(query) + def is_field_included(field_name): + return not exclude_fields or 'db_{}'.format(field_name) not in exclude_fields - if res: - for db in res: - self.info['databases'][db['name']] = {} + def create_db_info(db_data): + info = {} + if is_field_included('size'): + info['size'] = int(db_data.get('size', 0) or 0) + if is_field_included('table_count'): + info['tables'] = int(db_data.get('tables', 0) or 0) + return info - if not exclude_fields or 'db_size' not in exclude_fields: - if db['size'] is None: - db['size'] = 0 + # Build the main query + query_parts = ['SELECT table_schema AS "name"'] + if is_field_included('size'): + query_parts.append('SUM(data_length + index_length) AS "size"') + if is_field_included('table_count'): + query_parts.append('COUNT(table_name) as "tables"') - self.info['databases'][db['name']]['size'] = int(db['size']) + query = "{} FROM information_schema.TABLES GROUP BY table_schema".format(", ".join(query_parts)) - # If empty dbs are not needed in the returned dict, exit from the method - if not return_empty_dbs: - return None + # Get and process databases with tables + databases = self.__exec_sql(query) or [] + for db in databases: + self.info['databases'][db['name']] = create_db_info(db) - # Add info about empty databases (issue #65727): - res = self.__exec_sql('SHOW DATABASES') - if res: - for db in res: - if db['Database'] not in self.info['databases']: - self.info['databases'][db['Database']] = {} - - if not exclude_fields or 'db_size' not in exclude_fields: - self.info['databases'][db['Database']]['size'] = 0 + # Handle empty databases if requested + if return_empty_dbs: + empty_databases = self.__exec_sql('SHOW DATABASES') or [] + for db in empty_databases: + db_name = db['Database'] + if db_name not in self.info['databases']: + self.info['databases'][db_name] = create_db_info({}) def __exec_sql(self, query, ddl=False): """Execute SQL. diff --git a/tests/integration/targets/test_mysql_info/tasks/filter_databases.yml b/tests/integration/targets/test_mysql_info/tasks/filter_databases.yml new file mode 100644 index 0000000..da1058b --- /dev/null +++ b/tests/integration/targets/test_mysql_info/tasks/filter_databases.yml @@ -0,0 +1,161 @@ +--- + +- module_defaults: + community.mysql.mysql_db: &mysql_defaults + login_user: "{{ mysql_user }}" + login_password: "{{ mysql_password }}" + login_host: "{{ mysql_host }}" + login_port: "{{ mysql_primary_port }}" + community.mysql.mysql_query: *mysql_defaults + community.mysql.mysql_info: *mysql_defaults + community.mysql.mysql_user: *mysql_defaults + + block: + + # ================================ Prepare ============================== + - name: Mysql_info databases | Prepare | Create databases + community.mysql.mysql_db: + name: + - db_tables_count_empty + - db_tables_count_1 + - db_tables_count_2 + - db_only_views # https://github.com/ansible-Getions/community.mysql/issues/204 + state: present + + - name: Mysql_info databases | Prepare | Create tables + community.mysql.mysql_query: + query: + - >- + CREATE TABLE IF NOT EXISTS db_tables_count_1.t1 + (id int, name varchar(9)) + - >- + CREATE TABLE IF NOT EXISTS db_tables_count_2.t1 + (id int, name1 varchar(9)) + - >- + CREATE TABLE IF NOT EXISTS db_tables_count_2.t2 + (id int, name1 varchar(9)) + - >- + CREATE VIEW db_only_views.v_today (today) AS SELECT CURRENT_DATE + + # ================================== Tests ============================== + + - name: Mysql_info databases | Get all non-empty databases fields + community.mysql.mysql_info: + filter: + - databases + register: result + failed_when: + - > + result.databases['db_tables_count_1'].size != 16384 or + result.databases['db_tables_count_1'].tables != 1 or + result.databases['db_tables_count_2'].size != 32768 or + result.databases['db_tables_count_2'].tables != 2 or + result.databases['db_only_views'].size != 0 or + result.databases['db_only_views'].tables != 1 or + 'db_tables_count_empty' in result.databases | dict2items + | map(attribute='key') + + - name: Mysql_info databases | Get all dbs fields except db_size + community.mysql.mysql_info: + filter: + - databases + exclude_fields: + - db_size + register: result + failed_when: + - > + result.databases['db_tables_count_1'].size is defined or + result.databases['db_tables_count_1'].tables != 1 or + result.databases['db_tables_count_2'].size is defined or + result.databases['db_tables_count_2'].tables != 2 or + result.databases['db_only_views'].size is defined or + result.databases['db_only_views'].tables != 1 or + 'db_tables_count_empty' in result.databases | dict2items + | map(attribute='key') + + # 'unsupported' element is passed to check that an unsupported value + # won't break anything (will be ignored regarding to the module's + # documentation). + - name: Mysql_info databases | Get all dbs fields with unsupported value + community.mysql.mysql_info: + filter: + - databases + exclude_fields: + - db_size + - unsupported + register: result + failed_when: + - > + result.databases['db_tables_count_1'].size is defined or + result.databases['db_tables_count_1'].tables != 1 or + result.databases['db_tables_count_2'].size is defined or + result.databases['db_tables_count_2'].tables != 2 or + result.databases['db_only_views'].size is defined or + result.databases['db_only_views'].tables != 1 or + 'db_tables_count_empty' in result.databases | dict2items + | map(attribute='key') + + - name: Mysql_info databases | Get all dbs fields except tables + community.mysql.mysql_info: + filter: + - databases + exclude_fields: + - db_table_count + register: result + failed_when: + - > + result.databases['db_tables_count_1'].size != 16384 or + result.databases['db_tables_count_1'].tables is defined or + result.databases['db_tables_count_2'].size != 32768 or + result.databases['db_tables_count_2'].tables is defined or + result.databases['db_only_views'].size != 0 or + result.databases['db_only_views'].tables is defined or + 'db_tables_count_empty' in result.databases | dict2items + | map(attribute='key') + + - name: Mysql_info databases | Get all dbs even empty ones + community.mysql.mysql_info: + filter: + - databases + return_empty_dbs: true + register: result + failed_when: + - > + result.databases['db_tables_count_1'].size != 16384 or + result.databases['db_tables_count_1'].tables != 1 or + result.databases['db_tables_count_2'].size != 32768 or + result.databases['db_tables_count_2'].tables != 2 or + result.databases['db_only_views'].size != 0 or + result.databases['db_only_views'].tables != 1 or + result.databases['db_tables_count_empty'].size != 0 or + result.databases['db_tables_count_empty'].tables != 0 + + - name: Mysql_info databases | Get all dbs even empty ones without size + community.mysql.mysql_info: + filter: + - databases + exclude_fields: + - db_size + return_empty_dbs: true + register: result + failed_when: + - > + result.databases['db_tables_count_1'].size is defined or + result.databases['db_tables_count_1'].tables != 1 or + result.databases['db_tables_count_2'].size is defined or + result.databases['db_tables_count_2'].tables != 2 or + result.databases['db_only_views'].size is defined or + result.databases['db_only_views'].tables != 1 or + result.databases['db_tables_count_empty'].size is defined or + result.databases['db_tables_count_empty'].tables != 0 + + # ================================== Cleanup ============================ + + - name: Mysql_info databases | Cleanup databases + community.mysql.mysql_db: + name: + - db_tables_count_empty + - db_tables_count_1 + - db_tables_count_2 + - db_only_views + state: absent diff --git a/tests/integration/targets/test_mysql_info/tasks/main.yml b/tests/integration/targets/test_mysql_info/tasks/main.yml index 42350c6..61f238f 100644 --- a/tests/integration/targets/test_mysql_info/tasks/main.yml +++ b/tests/integration/targets/test_mysql_info/tasks/main.yml @@ -132,94 +132,11 @@ - result.global_status is not defined - result.users is not defined - # Test exclude_fields: db_size - # 'unsupported' element is passed to check that an unsupported value - # won't break anything (will be ignored regarding to the module's documentation). - - name: Collect info about databases excluding their sizes - mysql_info: - <<: *mysql_params - filter: - - databases - exclude_fields: - - db_size - - unsupported - register: result - - - assert: - that: - - result is not changed - - result.databases != {} - - result.databases.mysql == {} - - ######################################################## - # Issue #65727, empty databases must be in returned dict - # - - name: Create empty database acme - mysql_db: - <<: *mysql_params - name: acme - - - name: Collect info about databases - mysql_info: - <<: *mysql_params - filter: - - databases - return_empty_dbs: true - register: result - - # Check acme is in returned dict - - assert: - that: - - result is not changed - - result.databases.acme.size == 0 - - result.databases.mysql != {} - - - name: Collect info about databases excluding their sizes - mysql_info: - <<: *mysql_params - filter: - - databases - exclude_fields: - - db_size - return_empty_dbs: true - register: result - - # Check acme is in returned dict - - assert: - that: - - result is not changed - - result.databases.acme == {} - - result.databases.mysql == {} - - - name: Remove acme database - mysql_db: - <<: *mysql_params - name: acme - state: absent - - include_tasks: issue-28.yml - # https://github.com/ansible-collections/community.mysql/issues/204 - - name: Create database containing only views - mysql_db: - <<: *mysql_params - name: allviews - - - name: Create view - mysql_query: - <<: *mysql_params - login_db: allviews - query: 'CREATE VIEW v_today (today) AS SELECT CURRENT_DATE' - - - name: Fetch info - mysql_info: - <<: *mysql_params - register: result - - - name: Check - assert: - that: - - result.databases.allviews.size == 0 + - name: Import tasks file to tests tables count in database filter + ansible.builtin.import_tasks: + file: filter_databases.yml - name: Import tasks file to tests users_info filter ansible.builtin.import_tasks: From e437d562c1fec1979906c639bc579a69072a38ed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Tue, 19 Nov 2024 10:51:58 +0100 Subject: [PATCH 38/54] Release 3.11.0 commit (#692) --- CHANGELOG.rst | 20 ++++++++ changelogs/changelog.yaml | 49 +++++++++++++++---- .../591-mysql_info-db_tables_count.yml | 3 -- .../fragments/671-modules_util_user.yml | 12 ----- galaxy.yml | 2 +- 5 files changed, 60 insertions(+), 26 deletions(-) delete mode 100644 changelogs/fragments/591-mysql_info-db_tables_count.yml delete mode 100644 changelogs/fragments/671-modules_util_user.yml diff --git a/CHANGELOG.rst b/CHANGELOG.rst index cf1162f..a6ada35 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -6,6 +6,26 @@ Community MySQL and MariaDB Collection Release Notes This changelog describes changes after version 2.0.0. +v3.11.0 +======= + +Release Summary +--------------- + +This is a minor release of the ``community.mysql`` collection. +This changelog contains all changes to the modules and plugins in this +collection that have been made after the previous release. + +Minor Changes +------------- + +- mysql_info - adds the count of tables for each database to the returned values. It is possible to exclude this new field using the ``db_table_count`` exclusion filter. (https://github.com/ansible-collections/community.mysql/pull/691) + +Bugfixes +-------- + +- mysql_user,mysql_role - The sql_mode ANSI_QUOTES affects how the modules mysql_user and mysql_role compare the existing privileges with the configured privileges, as well as decide whether double quotes or backticks should be used in the GRANT statements. Pointing out in issue 671, the modules mysql_user and mysql_role allow users to enable/disable ANSI_QUOTES in session variable (within a DB session, the session variable always overwrites the global one). But due to the issue, the modules do not check for ANSI_MODE in the session variable, instead, they only check in the GLOBAL one.That behavior is not only limiting the users' flexibility, but also not allowing users to explicitly disable ANSI_MODE to work around such bugs like https://bugs.mysql.com/bug.php?id=115953. (https://github.com/ansible-collections/community.mysql/issues/671) + v3.10.3 ======= diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index 27ae315..8e5aeaf 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -99,13 +99,6 @@ releases: release_date: '2022-04-26' 3.10.0: changes: - deprecated_features: - - collection - support of mysqlclient connector is deprecated - use PyMySQL - connector instead! We will stop testing against it in collection version 4.0.0 - and remove the related code in 5.0.0 (https://github.com/ansible-collections/community.mysql/issues/654). - - mysql_info - The ``users_info`` filter returned variable ``plugin_auth_string`` - contains the hashed password and it's misleading, it will be removed from - community.mysql 4.0.0. Use the `plugin_hash_string` return value instead (https://github.com/ansible-collections/community.mysql/pull/629). bugfixes: - mysql_info - Add ``plugin_hash_string`` to ``users_info`` filter's output. The existing ``plugin_auth_string`` contained the hashed password and thus @@ -122,6 +115,13 @@ releases: avoid versions 3.8.0 to 3.9.0 (https://github.com/ansible-collections/community.mysql/pull/642). - mysql_user - add correct ``ed25519`` auth plugin handling (https://github.com/ansible-collections/community.mysql/issues/6). - mysql_variables - fix the module always changes on boolean values (https://github.com/ansible-collections/community.mysql/issues/652). + deprecated_features: + - collection - support of mysqlclient connector is deprecated - use PyMySQL + connector instead! We will stop testing against it in collection version 4.0.0 + and remove the related code in 5.0.0 (https://github.com/ansible-collections/community.mysql/issues/654). + - mysql_info - The ``users_info`` filter returned variable ``plugin_auth_string`` + contains the hashed password and it's misleading, it will be removed from + community.mysql 4.0.0. Use the `plugin_hash_string` return value instead (https://github.com/ansible-collections/community.mysql/pull/629). minor_changes: - mysql_info - Add ``tls_requires`` returned value for the ``users_info`` filter (https://github.com/ansible-collections/community.mysql/pull/628). @@ -158,13 +158,13 @@ releases: release_date: '2024-08-22' 3.10.1: changes: + bugfixes: + - mysql_user - module makes changes when is executed with ``plugin_auth_string`` + parameter and check mode. deprecated_features: - mysql_user - the ``user`` alias of the ``name`` argument has been deprecated and will be removed in collection version 5.0.0. Use the ``name`` argument instead. - bugfixes: - - mysql_user - module makes changes when is executed with ``plugin_auth_string`` - parameter and check mode. release_summary: 'This is a patch release of the ``community.mysql`` collection. Besides a bugfix, it contains an important upcoming breaking-change information.' @@ -201,6 +201,35 @@ releases: - 0-mysql_user.yml - 3.10.3.yml release_date: '2024-09-09' + 3.11.0: + changes: + bugfixes: + - mysql_user,mysql_role - The sql_mode ANSI_QUOTES affects how the modules mysql_user + and mysql_role compare the existing privileges with the configured privileges, + as well as decide whether double quotes or backticks should be used in the + GRANT statements. Pointing out in issue 671, the modules mysql_user and mysql_role + allow users to enable/disable ANSI_QUOTES in session variable (within a DB + session, the session variable always overwrites the global one). But due to + the issue, the modules do not check for ANSI_MODE in the session variable, + instead, they only check in the GLOBAL one.That behavior is not only limiting + the users' flexibility, but also not allowing users to explicitly disable + ANSI_MODE to work around such bugs like https://bugs.mysql.com/bug.php?id=115953. + (https://github.com/ansible-collections/community.mysql/issues/671) + minor_changes: + - mysql_info - adds the count of tables for each database to the returned values. + It is possible to exclude this new field using the ``db_table_count`` exclusion + filter. (https://github.com/ansible-collections/community.mysql/pull/691) + release_summary: 'This is a minor release of the ``community.mysql`` collection. + + + This changelog contains all changes to the modules and plugins in this + + collection that have been made after the previous release.' + fragments: + - 3.11.0.yml + - 591-mysql_info-db_tables_count.yml + - 671-modules_util_user.yml + release_date: '2024-11-19' 3.2.0: changes: bugfixes: diff --git a/changelogs/fragments/591-mysql_info-db_tables_count.yml b/changelogs/fragments/591-mysql_info-db_tables_count.yml deleted file mode 100644 index abbc1cb..0000000 --- a/changelogs/fragments/591-mysql_info-db_tables_count.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -minor_changes: - - mysql_info - adds the count of tables for each database to the returned values. It is possible to exclude this new field using the ``db_table_count`` exclusion filter. (https://github.com/ansible-collections/community.mysql/pull/691) diff --git a/changelogs/fragments/671-modules_util_user.yml b/changelogs/fragments/671-modules_util_user.yml deleted file mode 100644 index a913651..0000000 --- a/changelogs/fragments/671-modules_util_user.yml +++ /dev/null @@ -1,12 +0,0 @@ -bugfixes: - - mysql_user,mysql_role - The sql_mode ANSI_QUOTES affects how the modules mysql_user - and mysql_role compare the existing privileges with the configured privileges, - as well as decide whether double quotes or backticks should be used in the GRANT - statements. Pointing out in issue 671, the modules mysql_user and mysql_role allow - users to enable/disable ANSI_QUOTES in session variable (within a DB session, the - session variable always overwrites the global one). But due to the issue, the modules - do not check for ANSI_MODE in the session variable, instead, they only check in the - GLOBAL one.That behavior is not only limiting the users' flexibility, but also not - allowing users to explicitly disable ANSI_MODE to work around such bugs like - https://bugs.mysql.com/bug.php?id=115953. - (https://github.com/ansible-collections/community.mysql/issues/671) \ No newline at end of file diff --git a/galaxy.yml b/galaxy.yml index 0046b5a..1ecd6f2 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,7 +1,7 @@ --- namespace: community name: mysql -version: 3.10.3 +version: 3.11.0 readme: README.md authors: - Ansible community From 3d3f115574adf10a6c8552b5d811a45aef2597ba Mon Sep 17 00:00:00 2001 From: Laurent Indermuehle Date: Tue, 19 Nov 2024 10:56:37 +0100 Subject: [PATCH 39/54] Add next expected version --- galaxy.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/galaxy.yml b/galaxy.yml index 1ecd6f2..4830311 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,7 +1,7 @@ --- namespace: community name: mysql -version: 3.11.0 +version: 3.11.1 readme: README.md authors: - Ansible community From 022ed60906c36beb9082b7d39ba1aa4602199306 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Fri, 13 Dec 2024 09:21:06 +0100 Subject: [PATCH 40/54] Fix linting issues (#693) --- plugins/modules/mysql_replication.py | 1 - plugins/modules/mysql_user.py | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/plugins/modules/mysql_replication.py b/plugins/modules/mysql_replication.py index 35659d3..b902da0 100644 --- a/plugins/modules/mysql_replication.py +++ b/plugins/modules/mysql_replication.py @@ -284,7 +284,6 @@ EXAMPLES = r''' community.mysql.mysql_replication: mode: changeprimary fail_on_error: true - ''' RETURN = r''' diff --git a/plugins/modules/mysql_user.py b/plugins/modules/mysql_user.py index cf210a3..499f2a0 100644 --- a/plugins/modules/mysql_user.py +++ b/plugins/modules/mysql_user.py @@ -269,7 +269,7 @@ EXAMPLES = r''' priv: '*.*:ALL,GRANT' state: present session_vars: - wsrep_on: off + wsrep_on: 'off' - name: Create user with password, all database privileges and 'WITH GRANT OPTION' in db1 and db2 community.mysql.mysql_user: From a45a0d006d5654da57ea6a0f6692fba238646113 Mon Sep 17 00:00:00 2001 From: Sergio <45396489+Sergio-IME@users.noreply.github.com> Date: Thu, 16 Jan 2025 09:35:04 +0100 Subject: [PATCH 41/54] mysql_db: added `zstd` support (#696) --- changelogs/fragments/696-mysql-db-add-zstd-support.yml | 3 +++ plugins/modules/mysql_db.py | 8 ++++++-- 2 files changed, 9 insertions(+), 2 deletions(-) create mode 100644 changelogs/fragments/696-mysql-db-add-zstd-support.yml diff --git a/changelogs/fragments/696-mysql-db-add-zstd-support.yml b/changelogs/fragments/696-mysql-db-add-zstd-support.yml new file mode 100644 index 0000000..537fc6e --- /dev/null +++ b/changelogs/fragments/696-mysql-db-add-zstd-support.yml @@ -0,0 +1,3 @@ +minor_changes: +- mysql_db - added ``zstd`` (de)compression support for ``import``/``dump`` states + (https://github.com/ansible-collections/community.mysql/issues/696). diff --git a/plugins/modules/mysql_db.py b/plugins/modules/mysql_db.py index e1d1a7a..e108054 100644 --- a/plugins/modules/mysql_db.py +++ b/plugins/modules/mysql_db.py @@ -46,8 +46,8 @@ options: target: description: - Location, on the remote host, of the dump file to read from or write to. - - Uncompressed SQL files (C(.sql)) as well as bzip2 (C(.bz2)), gzip (C(.gz)) and - xz (Added in 2.0) compressed files are supported. + - Uncompressed SQL files (C(.sql)) as well as bzip2 (C(.bz2)), gzip (C(.gz)), + xz (Added in 2.0) and zstd (C(.zst)) (Added in 3.12.0) compressed files are supported. type: path single_transaction: description: @@ -455,6 +455,8 @@ def db_dump(module, host, user, password, db_name, target, all_databases, port, path = module.get_bin_path('bzip2', True) elif os.path.splitext(target)[-1] == '.xz': path = module.get_bin_path('xz', True) + elif os.path.splitext(target)[-1] == '.zst': + path = module.get_bin_path('zstd', True) if path: cmd = '%s | %s > %s' % (cmd, path, shlex_quote(target)) @@ -526,6 +528,8 @@ def db_import(module, host, user, password, db_name, target, all_databases, port comp_prog_path = module.get_bin_path('bzip2', required=True) elif os.path.splitext(target)[-1] == '.xz': comp_prog_path = module.get_bin_path('xz', required=True) + elif os.path.splitext(target)[-1] == '.zst': + comp_prog_path = module.get_bin_path('zstd', required=True) if comp_prog_path: # The line below is for returned data only: executed_commands.append('%s -dc %s | %s' % (comp_prog_path, target, cmd)) From 960ac32adffac3ff91c1c307ca04c62667a11b2b Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Thu, 16 Jan 2025 15:49:53 +0100 Subject: [PATCH 42/54] mysql_query: returns execution_time_ms list containing execution time per query (#697) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * mysql_query: returns execution_time_ms list containing execution time per query * Update changelogs/fragments/0-mysql_query-returns-exec-time-ms.yml Co-authored-by: Laurent Indermühle --- .../0-mysql_query-returns-exec-time-ms.yml | 2 ++ plugins/modules/mysql_query.py | 28 +++++++++++++++++-- .../tasks/mysql_query_initial.yml | 3 ++ 3 files changed, 31 insertions(+), 2 deletions(-) create mode 100644 changelogs/fragments/0-mysql_query-returns-exec-time-ms.yml diff --git a/changelogs/fragments/0-mysql_query-returns-exec-time-ms.yml b/changelogs/fragments/0-mysql_query-returns-exec-time-ms.yml new file mode 100644 index 0000000..d17628c --- /dev/null +++ b/changelogs/fragments/0-mysql_query-returns-exec-time-ms.yml @@ -0,0 +1,2 @@ +minor_changes: +- mysql_query - returns the ``execution_time_ms`` list containing execution time per query in milliseconds. diff --git a/plugins/modules/mysql_query.py b/plugins/modules/mysql_query.py index 2cdf096..35beeb3 100644 --- a/plugins/modules/mysql_query.py +++ b/plugins/modules/mysql_query.py @@ -62,7 +62,6 @@ author: - Andrew Klychkov (@Andersson007) extends_documentation_fragment: - community.mysql.mysql - ''' EXAMPLES = r''' @@ -117,8 +116,18 @@ rowcount: returned: changed type: list sample: [5, 1] +execution_time_ms: + description: + - A list containing execution time per query in milliseconds. + - The measurements are done right before and after passing + the query to the driver for execution. + returned: success + type: list + sample: [7104, 85] + version_added: '3.12.0' ''' +import time import warnings from ansible.module_utils.basic import AnsibleModule @@ -139,6 +148,18 @@ DDL_QUERY_KEYWORDS = ('CREATE', 'DROP', 'ALTER', 'RENAME', 'TRUNCATE') # Module execution. # + +def execute_and_return_time(cursor, query, args): + # Measure query execution time in milliseconds + start_time = time.perf_counter() + + cursor.execute(query, args) + + # Calculate the execution time rounding it to 4 decimal places + exec_time_ms = round((time.perf_counter() - start_time) * 1000, 4) + return cursor, exec_time_ms + + def main(): argument_spec = mysql_common_argument_spec() argument_spec.update( @@ -213,6 +234,7 @@ def main(): query_result = [] executed_queries = [] rowcount = [] + execution_time_ms = [] already_exists = False for q in query: @@ -223,7 +245,8 @@ def main(): category=mysql_driver.Warning) try: - cursor.execute(q, arguments) + cursor, exec_time_ms = execute_and_return_time(cursor, q, arguments) + execution_time_ms.append(exec_time_ms) except mysql_driver.Warning: # When something is run with IF NOT EXISTS # and there's "already exists" MySQL warning, @@ -280,6 +303,7 @@ def main(): 'executed_queries': executed_queries, 'query_result': query_result, 'rowcount': rowcount, + 'execution_time_ms': execution_time_ms, } # Exit: diff --git a/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml b/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml index fbf5ca8..310f925 100644 --- a/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml +++ b/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml @@ -35,6 +35,7 @@ that: - result is changed - result.executed_queries == ['CREATE TABLE {{ test_table1 }} (id int)'] + - result.execution_time_ms[0] > 0 - name: Insert test data mysql_query: @@ -52,6 +53,8 @@ - result is changed - result.rowcount == [2, 1] - result.executed_queries == ['INSERT INTO {{ test_table1 }} VALUES (1), (2)', 'INSERT INTO {{ test_table1 }} VALUES (3)'] + - result.execution_time_ms[0] > 0 + - result.execution_time_ms[1] > 0 - name: Check data in {{ test_table1 }} mysql_query: From e9845b0a1caba4344aab9e957865ac74ab17fc7f Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Fri, 17 Jan 2025 10:11:27 +0100 Subject: [PATCH 43/54] Release 3.12.0 commit (#698) --- CHANGELOG.rst | 17 +++++++++++++++++ changelogs/changelog.yaml | 17 +++++++++++++++++ .../0-mysql_query-returns-exec-time-ms.yml | 2 -- .../fragments/696-mysql-db-add-zstd-support.yml | 3 --- galaxy.yml | 2 +- 5 files changed, 35 insertions(+), 6 deletions(-) delete mode 100644 changelogs/fragments/0-mysql_query-returns-exec-time-ms.yml delete mode 100644 changelogs/fragments/696-mysql-db-add-zstd-support.yml diff --git a/CHANGELOG.rst b/CHANGELOG.rst index a6ada35..ba19887 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -6,6 +6,22 @@ Community MySQL and MariaDB Collection Release Notes This changelog describes changes after version 2.0.0. +v3.12.0 +======= + +Release Summary +--------------- + +This is a minor release of the ``community.mysql`` collection. +This changelog contains all changes to the modules and plugins in this +collection that have been made after the previous release. + +Minor Changes +------------- + +- mysql_db - added ``zstd`` (de)compression support for ``import``/``dump`` states (https://github.com/ansible-collections/community.mysql/issues/696). +- mysql_query - returns the ``execution_time_ms`` list containing execution time per query in milliseconds. + v3.11.0 ======= @@ -13,6 +29,7 @@ Release Summary --------------- This is a minor release of the ``community.mysql`` collection. + This changelog contains all changes to the modules and plugins in this collection that have been made after the previous release. diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index 8e5aeaf..fa08150 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -230,6 +230,23 @@ releases: - 591-mysql_info-db_tables_count.yml - 671-modules_util_user.yml release_date: '2024-11-19' + 3.12.0: + changes: + minor_changes: + - mysql_db - added ``zstd`` (de)compression support for ``import``/``dump`` + states (https://github.com/ansible-collections/community.mysql/issues/696). + - mysql_query - returns the ``execution_time_ms`` list containing execution + time per query in milliseconds. + release_summary: 'This is a minor release of the ``community.mysql`` collection. + + This changelog contains all changes to the modules and plugins in this + + collection that have been made after the previous release.' + fragments: + - 0-mysql_query-returns-exec-time-ms.yml + - 3.12.0.yml + - 696-mysql-db-add-zstd-support.yml + release_date: '2025-01-17' 3.2.0: changes: bugfixes: diff --git a/changelogs/fragments/0-mysql_query-returns-exec-time-ms.yml b/changelogs/fragments/0-mysql_query-returns-exec-time-ms.yml deleted file mode 100644 index d17628c..0000000 --- a/changelogs/fragments/0-mysql_query-returns-exec-time-ms.yml +++ /dev/null @@ -1,2 +0,0 @@ -minor_changes: -- mysql_query - returns the ``execution_time_ms`` list containing execution time per query in milliseconds. diff --git a/changelogs/fragments/696-mysql-db-add-zstd-support.yml b/changelogs/fragments/696-mysql-db-add-zstd-support.yml deleted file mode 100644 index 537fc6e..0000000 --- a/changelogs/fragments/696-mysql-db-add-zstd-support.yml +++ /dev/null @@ -1,3 +0,0 @@ -minor_changes: -- mysql_db - added ``zstd`` (de)compression support for ``import``/``dump`` states - (https://github.com/ansible-collections/community.mysql/issues/696). diff --git a/galaxy.yml b/galaxy.yml index 4830311..cf87c64 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,7 +1,7 @@ --- namespace: community name: mysql -version: 3.11.1 +version: 3.12.0 readme: README.md authors: - Ansible community From dd7e297d509d833dac5bd721d1e48a170079748e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Mon, 10 Mar 2025 18:55:42 +0100 Subject: [PATCH 44/54] Add support for MariaDB 11.4 (#703) * fix missing symlink to mysql binaries for MariaDB 11+ * update tested version of MariaDB 11.4 instead of 10.5 * add changelog fragment * [CI] add way to trigger workflow manually Useful in the case we don't modifiy any files in the paths: sections of the push event. * add version check for mariadb < 10.4.6 without mariadb* binaries * Use same concatenation method between functions to avoid future confusion I didn't notice that db_dump and db_import were different, thus I introduced a bug with the initialization of the variable cmd. This commit fixes that. --- .github/workflows/ansible-test-plugins.yml | 20 +++--- Makefile | 23 +++++-- README.md | 4 +- TESTING.md | 4 +- changelogs/fragments/tests_mariadb_11_4.yml | 5 ++ plugins/modules/mysql_db.py | 76 +++++++++++++-------- plugins/modules/mysql_info.py | 1 + 7 files changed, 84 insertions(+), 49 deletions(-) create mode 100644 changelogs/fragments/tests_mariadb_11_4.yml diff --git a/.github/workflows/ansible-test-plugins.yml b/.github/workflows/ansible-test-plugins.yml index ad8c4b5..0b6c184 100644 --- a/.github/workflows/ansible-test-plugins.yml +++ b/.github/workflows/ansible-test-plugins.yml @@ -13,7 +13,7 @@ on: # yamllint disable-line rule:truthy - '.github/workflows/ansible-test-plugins.yml' schedule: - cron: '0 6 * * *' - + workflow_dispatch: jobs: sanity: @@ -54,8 +54,8 @@ jobs: db_engine_version: - '8.0.38' - '8.4.1' - - '10.5.25' - '10.11.8' + - '11.4.5' connector_name: - pymysql - mysqlclient @@ -87,10 +87,10 @@ jobs: exclude: - db_engine_name: mysql - db_engine_version: '10.5.25' + db_engine_version: '10.11.8' - db_engine_name: mysql - db_engine_version: '10.11.8' + db_engine_version: '11.4.5' - db_engine_name: mariadb db_engine_version: '8.0.38' @@ -119,13 +119,13 @@ jobs: - db_engine_version: '8.0.38' ansible: stable-2.17 - - db_engine_version: '10.5.25' + - db_engine_version: '10.11.8' ansible: stable-2.17 - db_engine_version: '8.0.38' ansible: devel - - db_engine_version: '10.5.25' + - db_engine_version: '10.11.8' ansible: devel - db_engine_version: '8.4.1' @@ -162,7 +162,7 @@ jobs: db_engine_version: '8.0.38' - connector_version: '1.1.1' - db_engine_version: '10.5.25' + db_engine_version: '10.11.8' services: db_primary: @@ -175,7 +175,7 @@ jobs: # We write our own health-cmd because the mariadb container does not # provide a healthcheck options: >- - --health-cmd "mysqladmin ping -P 3306 -pmsandbox |grep alive || exit 1" + --health-cmd "${{ matrix.db_engine_name == 'mysql' && 'mysqladmin' || 'mariadb-admin' }} ping -P 3306 -pmsandbox |grep alive || exit 1" --health-start-period 10s --health-interval 10s --health-timeout 5s @@ -189,7 +189,7 @@ jobs: ports: - 3308:3306 options: >- - --health-cmd "mysqladmin ping -P 3306 -pmsandbox |grep alive || exit 1" + --health-cmd "${{ matrix.db_engine_name == 'mysql' && 'mysqladmin' || 'mariadb-admin' }} ping -P 3306 -pmsandbox |grep alive || exit 1" --health-start-period 10s --health-interval 10s --health-timeout 5s @@ -203,7 +203,7 @@ jobs: ports: - 3309:3306 options: >- - --health-cmd "mysqladmin ping -P 3306 -pmsandbox |grep alive || exit 1" + --health-cmd "${{ matrix.db_engine_name == 'mysql' && 'mysqladmin' || 'mariadb-admin' }} ping -P 3306 -pmsandbox |grep alive || exit 1" --health-start-period 10s --health-interval 10s --health-timeout 5s diff --git a/Makefile b/Makefile index 5a11d1b..b503e2f 100644 --- a/Makefile +++ b/Makefile @@ -11,6 +11,17 @@ ifdef continue_on_errors _continue_on_errors = --continue-on-error endif +# Set command variables based on database engine +# Required for MariaDB 11+ which no longer includes mysql named compatible +# executable symlinks +ifeq ($(db_engine_name),mysql) + _command = mysqld + _health_cmd = mysqladmin +else + _command = mariadbd + _health_cmd = mariadb-admin +endif + .PHONY: test-integration test-integration: @echo -n $(db_engine_name) > tests/integration/db_engine_name @@ -29,9 +40,9 @@ test-integration: --env MYSQL_ROOT_PASSWORD=msandbox \ --network podman \ --publish 3307:3306 \ - --health-cmd 'mysqladmin ping -P 3306 -pmsandbox | grep alive || exit 1' \ + --health-cmd '$(_health_cmd) ping -P 3306 -pmsandbox | grep alive || exit 1' \ docker.io/library/$(db_engine_name):$(db_engine_version) \ - mysqld + $(_command) podman run \ --detach \ --replace \ @@ -40,9 +51,9 @@ test-integration: --env MYSQL_ROOT_PASSWORD=msandbox \ --network podman \ --publish 3308:3306 \ - --health-cmd 'mysqladmin ping -P 3306 -pmsandbox | grep alive || exit 1' \ + --health-cmd '$(_health_cmd) ping -P 3306 -pmsandbox | grep alive || exit 1' \ docker.io/library/$(db_engine_name):$(db_engine_version) \ - mysqld + $(_command) podman run \ --detach \ --replace \ @@ -51,9 +62,9 @@ test-integration: --env MYSQL_ROOT_PASSWORD=msandbox \ --network podman \ --publish 3309:3306 \ - --health-cmd 'mysqladmin ping -P 3306 -pmsandbox | grep alive || exit 1' \ + --health-cmd '$(_health_cmd) ping -P 3306 -pmsandbox | grep alive || exit 1' \ docker.io/library/$(db_engine_name):$(db_engine_version) \ - mysqld + $(_command) # Setup replication and restart containers using the same subshell to keep variables alive db_ver=$(db_engine_version); \ maj="$${db_ver%.*.*}"; \ diff --git a/README.md b/README.md index 5db2f05..df2404f 100644 --- a/README.md +++ b/README.md @@ -112,10 +112,10 @@ For MariaDB, only Long Term releases are tested. When multiple LTS are available - mariadb:10.3.34 (collection version < 3.5.1) - mariadb:10.4.24 (collection version >= 3.5.2, < 3.10.0) - mariadb:10.5.18 (collection version >= 3.5.2, < 3.10.0) -- mariadb:10.5.25 (collection version >= 3.10.0) +- mariadb:10.5.25 (collection version >= 3.10.0, <3.13.0) - mariadb:10.6.11 (collection version >= 3.5.2, < 3.10.0) - mariadb:10.11.8 (collection version >= 3.10.0) - +- mariadb:11.4.5 (collection version >= 3.13.0) ### Database connectors diff --git a/TESTING.md b/TESTING.md index 1a22832..45e6bba 100644 --- a/TESTING.md +++ b/TESTING.md @@ -65,8 +65,8 @@ The Makefile accept the following options - Choices: - "8.0.38" <- mysql - "8.4.1" <- mysql (NOT WORKING YET, ansible-test uses Ubuntu 20.04 which is too old to install mysql-community-client 8.4) - - "10.5.25" <- mariadb - "10.11.8" <- mariadb + - "11.4.5" <- mariadb - Description: The tag of the container to use for the service containers that will host a primary database and two replicas. Do not use short version, like `mysql:8` (don't do that) because our tests expect a full version to filter tests precisely. For instance: `when: db_version is version ('8.0.22', '>')`. You can use any tag available on [hub.docker.com/_/mysql](https://hub.docker.com/_/mysql) and [hub.docker.com/_/mariadb](https://hub.docker.com/_/mariadb) but GitHub Action will only use the versions listed above. - `connector_name` @@ -121,7 +121,7 @@ make ansible="stable-2.16" db_engine_name="mysql" db_engine_version="8.0.31" con make ansible="stable-2.17" db_engine_name="mysql" db_engine_version="8.0.31" connector_name="mysqlclient" connector_version="2.0.3" target="test_mysql_query" keep_containers_alive=1 continue_on_errors=1 # If your system has an usupported version of Python: -make local_python_version="3.10" ansible="stable-2.17" db_engine_name="mariadb" db_engine_version="10.6.11" connector_name="pymysql" connector_version="1.0.2" +make local_python_version="3.10" ansible="stable-2.17" db_engine_name="mariadb" db_engine_version="11.4.5" connector_name="pymysql" connector_version="1.0.2" ``` diff --git a/changelogs/fragments/tests_mariadb_11_4.yml b/changelogs/fragments/tests_mariadb_11_4.yml new file mode 100644 index 0000000..46927bf --- /dev/null +++ b/changelogs/fragments/tests_mariadb_11_4.yml @@ -0,0 +1,5 @@ +--- +minor_changes: + - Integration tests for MariaDB 11.4 have replaced those for 10.5. The previous version is now 10.11. +bugfixes: + - mysql_db - fix dump and import to find MariaDB binaries (mariadb and mariadb-dump) when MariaDB 11+ is used and symbolic links to MySQL binaries are absent. diff --git a/plugins/modules/mysql_db.py b/plugins/modules/mysql_db.py index e108054..6ef578c 100644 --- a/plugins/modules/mysql_db.py +++ b/plugins/modules/mysql_db.py @@ -386,67 +386,75 @@ def db_dump(module, host, user, password, db_name, target, all_databases, port, encoding=None, force=False, master_data=0, skip_lock_tables=False, dump_extra_args=None, unsafe_password=False, restrict_config_file=False, check_implicit_admin=False, pipefail=False): - cmd = module.get_bin_path('mysqldump', True) + + cmd_str = 'mysqldump' + if server_implementation == 'mariadb' and LooseVersion(server_version) >= LooseVersion("10.4.6"): + cmd_str = 'mariadb-dump' + try: + cmd = [module.get_bin_path(cmd_str, True)] + except Exception as e: + return 1, "", "Error determining dump command: %s" % str(e) + # If defined, mysqldump demands --defaults-extra-file be the first option if config_file: if restrict_config_file: - cmd += " --defaults-file=%s" % shlex_quote(config_file) + cmd.append("--defaults-file=%s" % shlex_quote(config_file)) else: - cmd += " --defaults-extra-file=%s" % shlex_quote(config_file) + cmd.append("--defaults-extra-file=%s" % shlex_quote(config_file)) if check_implicit_admin: - cmd += " --user=root --password=''" + cmd.append("--user=root --password=''") else: if user is not None: - cmd += " --user=%s" % shlex_quote(user) + cmd.append("--user=%s" % shlex_quote(user)) if password is not None: if not unsafe_password: - cmd += " --password=%s" % shlex_quote(password) + cmd.append("--password=%s" % shlex_quote(password)) else: - cmd += " --password=%s" % password + cmd.append("--password=%s" % password) if ssl_cert is not None: - cmd += " --ssl-cert=%s" % shlex_quote(ssl_cert) + cmd.append("--ssl-cert=%s" % shlex_quote(ssl_cert)) if ssl_key is not None: - cmd += " --ssl-key=%s" % shlex_quote(ssl_key) + cmd.append("--ssl-key=%s" % shlex_quote(ssl_key)) if ssl_ca is not None: - cmd += " --ssl-ca=%s" % shlex_quote(ssl_ca) + cmd.append("--ssl-ca=%s" % shlex_quote(ssl_ca)) if force: - cmd += " --force" + cmd.append("--force") if socket is not None: - cmd += " --socket=%s" % shlex_quote(socket) + cmd.append("--socket=%s" % shlex_quote(socket)) else: - cmd += " --host=%s --port=%i" % (shlex_quote(host), port) + cmd.append("--host=%s --port=%i" % (shlex_quote(host), port)) if all_databases: - cmd += " --all-databases" + cmd.append("--all-databases") elif len(db_name) > 1: - cmd += " --databases {0}".format(' '.join(db_name)) + cmd.append("--databases {0}".format(' '.join(db_name))) else: - cmd += " %s" % shlex_quote(' '.join(db_name)) + cmd.append("%s" % shlex_quote(' '.join(db_name))) if skip_lock_tables: - cmd += " --skip-lock-tables" + cmd.append("--skip-lock-tables") if (encoding is not None) and (encoding != ""): - cmd += " --default-character-set=%s" % shlex_quote(encoding) + cmd.append("--default-character-set=%s" % shlex_quote(encoding)) if single_transaction: - cmd += " --single-transaction=true" + cmd.append("--single-transaction=true") if quick: - cmd += " --quick" + cmd.append("--quick") if ignore_tables: for an_ignored_table in ignore_tables: - cmd += " --ignore-table={0}".format(an_ignored_table) + cmd.append("--ignore-table={0}".format(an_ignored_table)) if hex_blob: - cmd += " --hex-blob" + cmd.append("--hex-blob") if master_data: if (server_implementation == 'mysql' and LooseVersion(server_version) >= LooseVersion("8.2.0")): - cmd += " --source-data=%s" % master_data + cmd.append("--source-data=%s" % master_data) else: - cmd += " --master-data=%s" % master_data + cmd.append("--master-data=%s" % master_data) if dump_extra_args is not None: - cmd += " " + dump_extra_args + cmd.append(dump_extra_args) path = None if os.path.splitext(target)[-1] == '.gz': @@ -458,6 +466,8 @@ def db_dump(module, host, user, password, db_name, target, all_databases, port, elif os.path.splitext(target)[-1] == '.zst': path = module.get_bin_path('zstd', True) + cmd = ' '.join(cmd) + if path: cmd = '%s | %s > %s' % (cmd, path, shlex_quote(target)) if pipefail: @@ -476,13 +486,21 @@ def db_dump(module, host, user, password, db_name, target, all_databases, port, def db_import(module, host, user, password, db_name, target, all_databases, port, config_file, - socket=None, ssl_cert=None, ssl_key=None, ssl_ca=None, encoding=None, force=False, + server_implementation, server_version, socket=None, ssl_cert=None, ssl_key=None, ssl_ca=None, + encoding=None, force=False, use_shell=False, unsafe_password=False, restrict_config_file=False, check_implicit_admin=False): if not os.path.exists(target): return module.fail_json(msg="target %s does not exist on the host" % target) - cmd = [module.get_bin_path('mysql', True)] + cmd_str = 'mysql' + if server_implementation == 'mariadb' and LooseVersion(server_version) >= LooseVersion("10.4.6"): + cmd_str = 'mariadb' + try: + cmd = [module.get_bin_path(cmd_str, True)] + except Exception as e: + return 1, "", "Error determining mysql/mariadb command: %s" % str(e) + # --defaults-file must go first, or errors out if config_file: if restrict_config_file: @@ -772,8 +790,8 @@ def main(): rc, stdout, stderr = db_import(module, login_host, login_user, login_password, db, target, all_databases, - login_port, config_file, - socket, ssl_cert, ssl_key, ssl_ca, + login_port, config_file, server_implementation, + server_version, socket, ssl_cert, ssl_key, ssl_ca, encoding, force, use_shell, unsafe_login_password, restrict_config_file, check_implicit_admin) if rc != 0: diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index 8c3845d..9bf89ae 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -4,6 +4,7 @@ # Copyright: (c) 2019, Andrew Klychkov (@Andersson007) # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) + from __future__ import absolute_import, division, print_function __metaclass__ = type From 45a29408ad41fb42271b05617ca6e44c3c384208 Mon Sep 17 00:00:00 2001 From: Keeper-of-the-Keys Date: Wed, 19 Mar 2025 15:40:59 +0200 Subject: [PATCH 45/54] User locking (#702) * function to check if a user is locked already Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Add the location and logic of where I think user locking would happen. Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Fix missing parameters for execute() Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Add the locked attribute Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Initial user locking integration tests Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Add attribute documentation Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * More descriptive names in the integration tests Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * - Changes requested/suggested by @Andersson007 - Example usage - Changelog fragment Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Fix user_is_locked and remove host_all option. Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Fix host of user (was % should have been localhost after deleting `host:` earlier) Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Switch locked to named instead of positional. Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Add check_mode support. Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Add check_mode: true test cases Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Fix names that included `check_mode: true` Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Add idempotence checks Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Switch calls to user_mod with sequences of None positional arguments to full named arguments Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * locked check should not run for roles. Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * check_mode is set at the task level and not the module level Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Add user locking to info module and test. Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Handle DictCursor Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Add check_mode feedback Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Add another builtin account to the exclusion list Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Initial switch to default=None for locked, will need to add a test for it. Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys * Add check that missing locked argument does not unlock a user Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys --------- Signed-off-by: E.S. Rosenberg a.k.a. Keeper of the Keys --- changelogs/fragments/702-user_locking.yaml | 2 + plugins/module_utils/user.py | 42 +++- plugins/modules/mysql_info.py | 5 +- plugins/modules/mysql_role.py | 11 +- plugins/modules/mysql_user.py | 33 ++- .../tasks/filter_users_info.yml | 2 + .../targets/test_mysql_user/tasks/main.yml | 4 + .../tasks/test_user_locking.yml | 200 ++++++++++++++++++ 8 files changed, 285 insertions(+), 14 deletions(-) create mode 100644 changelogs/fragments/702-user_locking.yaml create mode 100644 tests/integration/targets/test_mysql_user/tasks/test_user_locking.yml diff --git a/changelogs/fragments/702-user_locking.yaml b/changelogs/fragments/702-user_locking.yaml new file mode 100644 index 0000000..1378793 --- /dev/null +++ b/changelogs/fragments/702-user_locking.yaml @@ -0,0 +1,2 @@ +minor_changes: +- mysql_user - add ``locked`` option to lock/unlock users, this is mainly used to have users that will act as definers on stored procedures. diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index 307ef6e..9de1c6d 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -52,6 +52,25 @@ def user_exists(cursor, user, host, host_all): return count[0] > 0 +def user_is_locked(cursor, user, host): + cursor.execute("SHOW CREATE USER %s@%s", (user, host)) + + # Per discussions on irc:libera.chat:#maria the query may return up to 2 rows but "ACCOUNT LOCK" should always be in the first row. + result = cursor.fetchone() + + # ACCOUNT LOCK does not have to be the last option in the CREATE USER query. + # Need to handle both DictCursor and non-DictCursor + if isinstance(result, tuple): + if result[0].find('ACCOUNT LOCK') > 0: + return True + elif isinstance(result, dict): + for res in result.values(): + if res.find('ACCOUNT LOCK') > 0: + return True + + return False + + def sanitize_requires(tls_requires): sanitized_requires = {} if tls_requires: @@ -160,7 +179,7 @@ def get_existing_authentication(cursor, user, host=None): def user_add(cursor, user, host, host_all, password, encrypted, plugin, plugin_hash_string, plugin_auth_string, salt, new_priv, attributes, tls_requires, reuse_existing_password, module, - password_expire, password_expire_interval): + password_expire, password_expire_interval, locked=False): # If attributes are set, perform a sanity check to ensure server supports user attributes before creating user if attributes and not get_attribute_support(cursor): module.fail_json(msg="user attributes were specified but the server does not support user attributes") @@ -250,6 +269,9 @@ def user_add(cursor, user, host, host_all, password, encrypted, cursor.execute("ALTER USER %s@%s ATTRIBUTE %s", (user, host, json.dumps(attributes))) final_attributes = attributes_get(cursor, user, host) + if locked: + cursor.execute("ALTER USER %s@%s ACCOUNT LOCK", (user, host)) + return {'changed': True, 'password_changed': not used_existing_password, 'attributes': final_attributes} @@ -264,7 +286,7 @@ def is_hash(password): def user_mod(cursor, user, host, host_all, password, encrypted, plugin, plugin_hash_string, plugin_auth_string, salt, new_priv, append_privs, subtract_privs, attributes, tls_requires, module, - password_expire, password_expire_interval, role=False, maria_role=False): + password_expire, password_expire_interval, locked=None, role=False, maria_role=False): changed = False msg = "User unchanged" grant_option = False @@ -536,6 +558,22 @@ def user_mod(cursor, user, host, host_all, password, encrypted, if attribute_support: final_attributes = attributes_get(cursor, user, host) + if not role and locked is not None and user_is_locked(cursor, user, host) != locked: + if not module.check_mode: + if locked: + cursor.execute("ALTER USER %s@%s ACCOUNT LOCK", (user, host)) + msg = 'User locked' + else: + cursor.execute("ALTER USER %s@%s ACCOUNT UNLOCK", (user, host)) + msg = 'User unlocked' + else: + if locked: + msg = 'User will be locked' + else: + msg = 'User will be unlocked' + + changed = True + if role: continue diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index 9bf89ae..2360d01 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -319,6 +319,7 @@ from ansible_collections.community.mysql.plugins.module_utils.user import ( get_resource_limits, get_existing_authentication, get_user_implementation, + user_is_locked, ) from ansible.module_utils.six import iteritems from ansible.module_utils._text import to_native @@ -653,8 +654,10 @@ class MySQL_Info(object): if authentications: output_dict.update(authentications[0]) + if line.get('is_role') and line['is_role'] == 'N': + output_dict['locked'] = user_is_locked(self.cursor, user, host) + # TODO password_option - # TODO lock_option # but both are not supported by mysql_user atm. So no point yet. output.append(output_dict) diff --git a/plugins/modules/mysql_role.py b/plugins/modules/mysql_role.py index c88392b..382445c 100644 --- a/plugins/modules/mysql_role.py +++ b/plugins/modules/mysql_role.py @@ -930,11 +930,12 @@ class Role(): set_default_role_all=set_default_role_all) if privs: - result = user_mod(self.cursor, self.name, self.host, - None, None, None, None, None, None, None, - privs, append_privs, subtract_privs, None, None, - self.module, None, None, role=True, - maria_role=self.is_mariadb) + result = user_mod(cursor=self.cursor, user=self.name, host=self.host, + host_all=None, password=None, encrypted=None, plugin=None, + plugin_auth_string=None, plugin_hash_string=None, salt=None, + new_priv=privs, append_privs=append_privs, subtract_privs=subtract_privs, + attributes=None, tls_requires=None, module=self.module, password_expire=None, + password_expire_interval=None, role=True, maria_role=self.is_mariadb) changed = result['changed'] if admin: diff --git a/plugins/modules/mysql_user.py b/plugins/modules/mysql_user.py index 499f2a0..2a5855c 100644 --- a/plugins/modules/mysql_user.py +++ b/plugins/modules/mysql_user.py @@ -189,6 +189,15 @@ options: fields names in privileges. type: bool version_added: '3.8.0' + + locked: + description: + - Lock account to prevent connections using it. + - This is primarily used for creating a user that will act as a DEFINER on stored procedures. + - If not specified leaves the lock state as is (for a new user creates unlocked). + type: bool + version_added: '3.13.0' + attributes: description: - "Create, update, or delete user attributes (arbitrary 'key: value' comments) for the user." @@ -225,6 +234,7 @@ author: - Lukasz Tomaszkiewicz (@tomaszkiewicz) - kmarse (@kmarse) - Laurent Indermühle (@laurent-indermuehle) +- E.S. Rosenberg (@Keeper-of-the-Keys) extends_documentation_fragment: - community.mysql.mysql @@ -400,6 +410,13 @@ EXAMPLES = r''' priv: 'db1.*': DELETE +- name: Create locked user to act as a definer on procedures + community.mysql.mysql_user: + name: readonly_procedures_locked + locked: true + priv: + db1.*: SELECT + # Example .my.cnf file for setting the root password # [client] # user=root @@ -470,6 +487,7 @@ def main(): column_case_sensitive=dict(type='bool', default=None), # TODO 4.0.0 add default=True password_expire=dict(type='str', choices=['now', 'never', 'default', 'interval'], no_log=True), password_expire_interval=dict(type='int', required_if=[('password_expire', 'interval', True)], no_log=True), + locked=dict(type='bool'), ) module = AnsibleModule( argument_spec=argument_spec, @@ -510,6 +528,7 @@ def main(): column_case_sensitive = module.params["column_case_sensitive"] password_expire = module.params["password_expire"] password_expire_interval = module.params["password_expire_interval"] + locked = module.boolean(module.params['locked']) if priv and not isinstance(priv, (str, dict)): module.fail_json(msg="priv parameter must be str or dict but %s was passed" % type(priv)) @@ -577,13 +596,15 @@ def main(): result = user_mod(cursor, user, host, host_all, password, encrypted, plugin, plugin_hash_string, plugin_auth_string, salt, priv, append_privs, subtract_privs, attributes, tls_requires, module, - password_expire, password_expire_interval) + password_expire, password_expire_interval, locked=locked) else: - result = user_mod(cursor, user, host, host_all, None, encrypted, - None, None, None, None, - priv, append_privs, subtract_privs, attributes, tls_requires, module, - password_expire, password_expire_interval) + result = user_mod(cursor=cursor, user=user, host=host, host_all=host_all, password=None, + encrypted=encrypted, plugin=None, plugin_hash_string=None, plugin_auth_string=None, + salt=None, new_priv=priv, append_privs=append_privs, subtract_privs=subtract_privs, + attributes=attributes, tls_requires=tls_requires, module=module, + password_expire=password_expire, password_expire_interval=password_expire_interval, + locked=locked) changed = result['changed'] msg = result['msg'] password_changed = result['password_changed'] @@ -601,7 +622,7 @@ def main(): result = user_add(cursor, user, host, host_all, password, encrypted, plugin, plugin_hash_string, plugin_auth_string, salt, priv, attributes, tls_requires, reuse_existing_password, module, - password_expire, password_expire_interval) + password_expire, password_expire_interval, locked=locked) changed = result['changed'] password_changed = result['password_changed'] final_attributes = result['attributes'] diff --git a/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml b/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml index 36508f3..558d309 100644 --- a/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml +++ b/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml @@ -261,6 +261,7 @@ resource_limits: "{{ item.resource_limits | default(omit) }}" column_case_sensitive: true state: present + locked: "{{ item.locked | default(omit) }}" loop: "{{ result.users_info }}" loop_control: label: "{{ item.name }}@{{ item.host }}" @@ -275,6 +276,7 @@ - item.name != 'mariadb.sys' - item.name != 'mysql.sys' - item.name != 'mysql.infoschema' + - item.name != 'mysql.session' # ================================== Cleanup ============================ diff --git a/tests/integration/targets/test_mysql_user/tasks/main.yml b/tests/integration/targets/test_mysql_user/tasks/main.yml index 9244570..7212886 100644 --- a/tests/integration/targets/test_mysql_user/tasks/main.yml +++ b/tests/integration/targets/test_mysql_user/tasks/main.yml @@ -305,3 +305,7 @@ - name: Mysql_user - test update_password ansible.builtin.import_tasks: file: test_update_password.yml + + - name: Mysql_user - test user_locking + ansible.builtin.import_tasks: + file: test_user_locking.yml diff --git a/tests/integration/targets/test_mysql_user/tasks/test_user_locking.yml b/tests/integration/targets/test_mysql_user/tasks/test_user_locking.yml new file mode 100644 index 0000000..3990610 --- /dev/null +++ b/tests/integration/targets/test_mysql_user/tasks/test_user_locking.yml @@ -0,0 +1,200 @@ +--- + +- vars: + mysql_parameters: &mysql_params + login_user: '{{ mysql_user }}' + login_password: '{{ mysql_password }}' + login_host: '{{ mysql_host }}' + login_port: '{{ mysql_primary_port }}' + + block: + + # ========================= Prepare ======================================= + - name: Mysql_user Lock user | Create a test database + community.mysql.mysql_db: + <<: *mysql_params + name: mysql_lock_user_test + state: present + + # ========================== Tests ======================================== + + - name: Mysql_user Lock user | create locked | Create test user + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + password: 'msandbox' + locked: true + priv: + 'mysql_lock_user_test.*': 'SELECT' + + - name: Mysql_user Lock user | create locked | Assert that test user is locked + community.mysql.mysql_query: + <<: *mysql_params + query: + - SHOW CREATE USER 'mysql_locked_user'@'localhost' + register: locked_user_creation + failed_when: + - locked_user_creation.query_result[0][0] is not search('ACCOUNT LOCK') + + - name: 'Mysql_user Lock user | create locked | Idempotence check' + check_mode: true + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + locked: true + priv: + 'mysql_lock_user_test.*': 'SELECT' + register: idempotence_check + failed_when: idempotence_check is changed + + - name: 'Mysql_user Lock user | create locked | Check that absense of locked does not unlock the user' + check_mode: true + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + priv: + 'mysql_lock_user_test.*': 'SELECT' + register: idempotence_check + failed_when: idempotence_check is changed + + - name: 'Mysql_user Lock user | create locked | Unlock test user check_mode: true' + check_mode: true + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + locked: false + priv: + 'mysql_lock_user_test.*': 'SELECT' + + - name: Mysql_user Lock user | create locked | Assert that test user is locked + community.mysql.mysql_query: + <<: *mysql_params + query: + - SHOW CREATE USER 'mysql_locked_user'@'localhost' + register: locked_user_creation + failed_when: + - locked_user_creation.query_result[0][0] is not search('ACCOUNT LOCK') + + - name: Mysql_user Lock user | create locked | Unlock test user + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + locked: false + priv: + 'mysql_lock_user_test.*': 'SELECT' + + - name: Mysql_user Lock user | create locked | Assert that test user is not locked + community.mysql.mysql_query: + <<: *mysql_params + query: + - SHOW CREATE USER 'mysql_locked_user'@'localhost' + register: locked_user_creation + failed_when: + - locked_user_creation.query_result[0][0] is search('ACCOUNT LOCK') + + - name: Mysql_user Lock user | create locked | Remove test user + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + state: absent + + - name: Mysql_user Lock user | create unlocked | Create test user + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + password: 'msandbox' + locked: false + priv: + 'mysql_lock_user_test.*': 'SELECT' + + - name: Mysql_user Lock user | create unlocked | Assert that test user is not locked + community.mysql.mysql_query: + <<: *mysql_params + query: + - SHOW CREATE USER 'mysql_locked_user'@'localhost' + register: locked_user_creation + failed_when: + - locked_user_creation.query_result[0][0] is search('ACCOUNT LOCK') + + - name: 'Mysql_user Lock user | create unlocked | Idempotence check' + check_mode: true + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + locked: false + priv: + 'mysql_lock_user_test.*': 'SELECT' + register: idempotence_check + failed_when: idempotence_check is changed + + - name: 'Mysql_user Lock user | create unlocked | Lock test user check_mode: true' + check_mode: true + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + locked: true + priv: + 'mysql_lock_user_test.*': 'SELECT' + + - name: Mysql_user Lock user | create unlocked | Assert that test user is not locked + community.mysql.mysql_query: + <<: *mysql_params + query: + - SHOW CREATE USER 'mysql_locked_user'@'localhost' + register: locked_user_creation + failed_when: + - locked_user_creation.query_result[0][0] is search('ACCOUNT LOCK') + + - name: Mysql_user Lock user | create unlocked | Lock test user + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + locked: true + priv: + 'mysql_lock_user_test.*': 'SELECT' + + - name: Mysql_user Lock user | create unlocked | Assert that test user is locked + community.mysql.mysql_query: + <<: *mysql_params + query: + - SHOW CREATE USER 'mysql_locked_user'@'localhost' + register: locked_user_creation + failed_when: + - locked_user_creation.query_result[0][0] is not search('ACCOUNT LOCK') + + - name: Mysql_user Lock user | create unlocked | Remove test user + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + state: absent + + - name: Mysql_user Lock user | create default | Create test user + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + password: 'msandbox' + priv: + 'mysql_lock_user_test.*': 'SELECT' + + - name: Mysql_user Lock user | create default | Assert that test user is not locked + community.mysql.mysql_query: + <<: *mysql_params + query: + - SHOW CREATE USER 'mysql_locked_user'@'localhost' + register: locked_user_creation + failed_when: + - locked_user_creation.query_result[0][0] is search('ACCOUNT LOCK') + + - name: Mysql_user Lock user | create default | Remove test user + community.mysql.mysql_user: + <<: *mysql_params + name: mysql_locked_user + state: absent + + # ========================= Teardown ====================================== + + - name: Mysql_user Lock user | Delete test database + community.mysql.mysql_db: + <<: *mysql_params + name: mysql_lock_user_test + state: absent From b26235b7d7f571895245cf5d1137096951e44294 Mon Sep 17 00:00:00 2001 From: Andrew Klychkov Date: Fri, 21 Mar 2025 07:02:43 +0100 Subject: [PATCH 46/54] Release 3.13.0 commit (#705) --- CHANGELOG.rst | 21 +++++++++++++++++++++ changelogs/changelog.yaml | 20 ++++++++++++++++++++ changelogs/fragments/702-user_locking.yaml | 2 -- changelogs/fragments/tests_mariadb_11_4.yml | 5 ----- galaxy.yml | 2 +- 5 files changed, 42 insertions(+), 8 deletions(-) delete mode 100644 changelogs/fragments/702-user_locking.yaml delete mode 100644 changelogs/fragments/tests_mariadb_11_4.yml diff --git a/CHANGELOG.rst b/CHANGELOG.rst index ba19887..b318076 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -6,6 +6,27 @@ Community MySQL and MariaDB Collection Release Notes This changelog describes changes after version 2.0.0. +v3.13.0 +======= + +Release Summary +--------------- + +This is a minor release of the ``community.mysql`` collection. +This changelog contains all changes to the modules and plugins in this +collection that have been made after the previous release. + +Minor Changes +------------- + +- Integration tests for MariaDB 11.4 have replaced those for 10.5. The previous version is now 10.11. +- mysql_user - add ``locked`` option to lock/unlock users, this is mainly used to have users that will act as definers on stored procedures. + +Bugfixes +-------- + +- mysql_db - fix dump and import to find MariaDB binaries (mariadb and mariadb-dump) when MariaDB 11+ is used and symbolic links to MySQL binaries are absent. + v3.12.0 ======= diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index fa08150..5ec7dc9 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -247,6 +247,26 @@ releases: - 3.12.0.yml - 696-mysql-db-add-zstd-support.yml release_date: '2025-01-17' + 3.13.0: + changes: + bugfixes: + - mysql_db - fix dump and import to find MariaDB binaries (mariadb and mariadb-dump) + when MariaDB 11+ is used and symbolic links to MySQL binaries are absent. + minor_changes: + - Integration tests for MariaDB 11.4 have replaced those for 10.5. The previous + version is now 10.11. + - mysql_user - add ``locked`` option to lock/unlock users, this is mainly used + to have users that will act as definers on stored procedures. + release_summary: 'This is a minor release of the ``community.mysql`` collection. + + This changelog contains all changes to the modules and plugins in this + + collection that have been made after the previous release.' + fragments: + - 3.13.0.yml + - 702-user_locking.yaml + - tests_mariadb_11_4.yml + release_date: '2025-03-21' 3.2.0: changes: bugfixes: diff --git a/changelogs/fragments/702-user_locking.yaml b/changelogs/fragments/702-user_locking.yaml deleted file mode 100644 index 1378793..0000000 --- a/changelogs/fragments/702-user_locking.yaml +++ /dev/null @@ -1,2 +0,0 @@ -minor_changes: -- mysql_user - add ``locked`` option to lock/unlock users, this is mainly used to have users that will act as definers on stored procedures. diff --git a/changelogs/fragments/tests_mariadb_11_4.yml b/changelogs/fragments/tests_mariadb_11_4.yml deleted file mode 100644 index 46927bf..0000000 --- a/changelogs/fragments/tests_mariadb_11_4.yml +++ /dev/null @@ -1,5 +0,0 @@ ---- -minor_changes: - - Integration tests for MariaDB 11.4 have replaced those for 10.5. The previous version is now 10.11. -bugfixes: - - mysql_db - fix dump and import to find MariaDB binaries (mariadb and mariadb-dump) when MariaDB 11+ is used and symbolic links to MySQL binaries are absent. diff --git a/galaxy.yml b/galaxy.yml index cf87c64..624c7d6 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,7 +1,7 @@ --- namespace: community name: mysql -version: 3.12.0 +version: 3.13.0 readme: README.md authors: - Ansible community From da2dc9ab5d0cc0c3d444ef5b5694fde421eab1ff Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Thu, 24 Apr 2025 13:44:50 +0200 Subject: [PATCH 47/54] Doc: locked returned in users info (#706) * doc: users_info returns a new field "locked" * doc: fix hash syntax Co-authored-by: Andrew Klychkov --------- Co-authored-by: Andrew Klychkov --- plugins/modules/mysql_info.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index 2360d01..a630f18 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -151,6 +151,7 @@ EXAMPLES = r''' tls_requires: "{{ item.tls_requires | default(omit) }}" priv: "{{ item.priv | default(omit) }}" resource_limits: "{{ item.resource_limits | default(omit) }}" + locked: "{{ item.locked | default(omit) }}" column_case_sensitive: true state: present loop: "{{ result.users_info }}" @@ -246,6 +247,7 @@ users_info: If the output is fed to M(community.mysql.mysql_user), the ``plugin_auth_string`` will most likely be unreadable due to non-binary characters. + - The "locked" field was aded in ``community.mysql`` 3.13. returned: if not excluded by filter type: dict sample: @@ -255,7 +257,8 @@ users_info: "plugin": "mysql_native_password", "priv": "db1.*:SELECT/db2.*:SELECT", "resource_limits": { "MAX_USER_CONNECTIONS": 100 }, - "tls_requires": { "SSL": null } } + "tls_requires": { "SSL": null }, + "locked": false } version_added: '3.8.0' engines: description: Information about the server's storage engines. From d44f8f20396b5963f6a8fb03bf20fde68837dcb7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Mon, 28 Apr 2025 13:52:40 +0200 Subject: [PATCH 48/54] CI: fix string templating for ansible devel (core 2.19, 12 beta) in tests (#713) * fix string templating for ansible devel (core 2.19, 12 beta) * fix order to prevent [ERROR]: Task failed: Module failed: 'Server_id' * cut jinja templates from conditionals * fix 'item' already in use --- .../tasks/multi_db_create_delete.yml | 120 +++++++++--------- .../test_mysql_db/tasks/state_dump_import.yml | 16 ++- .../tasks/state_present_absent.yml | 36 ++++-- .../tasks/mysql_query_initial.yml | 63 ++++++--- .../tasks/mysql_replication_channel.yml | 2 +- .../tasks/mysql_replication_initial.yml | 3 +- .../tasks/mysql_replication_primary_delay.yml | 2 +- .../test_mysql_user/tasks/test_privs.yml | 4 +- .../tasks/test_user_plugin_auth.yml | 2 +- .../tasks/utils/assert_user.yml | 4 +- .../test_mysql_variables/tasks/assert_var.yml | 6 +- .../tasks/assert_var_output.yml | 6 +- .../tasks/mysql_variables.yml | 22 +++- 13 files changed, 173 insertions(+), 113 deletions(-) diff --git a/tests/integration/targets/test_mysql_db/tasks/multi_db_create_delete.yml b/tests/integration/targets/test_mysql_db/tasks/multi_db_create_delete.yml index 0bd7d58..d65b422 100644 --- a/tests/integration/targets/test_mysql_db/tasks/multi_db_create_delete.yml +++ b/tests/integration/targets/test_mysql_db/tasks/multi_db_create_delete.yml @@ -33,9 +33,9 @@ - name: assert that databases does not exist assert: that: - - "'{{ db1_name }}' not in mysql_result.stdout" - - "'{{ db2_name }}' not in mysql_result.stdout" - - "'{{ db3_name }}' not in mysql_result.stdout" + - db1_name not in mysql_result.stdout + - db2_name not in mysql_result.stdout + - db3_name not in mysql_result.stdout # ========================================================================== # Create multiple databases that does not exists (check mode) @@ -65,9 +65,9 @@ - name: assert that databases does not exist (since created via check mode) assert: that: - - "'{{ db1_name }}' not in mysql_result.stdout" - - "'{{ db2_name }}' not in mysql_result.stdout" - - "'{{ db3_name }}' not in mysql_result.stdout" + - db1_name not in mysql_result.stdout + - db2_name not in mysql_result.stdout + - db3_name not in mysql_result.stdout # ========================================================================== # Create multiple databases @@ -88,7 +88,7 @@ assert: that: - result is changed - - result.db_list == ['{{ db1_name }}', '{{ db2_name }}', '{{ db3_name }}'] + - result.db_list == [db1_name, db2_name, db3_name] - name: run command to list databases like specified database name command: "{{ mysql_command }} \"-e show databases like 'database%'\"" @@ -97,9 +97,9 @@ - name: assert that databases exist after creation assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name in mysql_result.stdout + - db3_name in mysql_result.stdout # ========================================================================= # Recreate already existing databases (check mode) @@ -129,9 +129,9 @@ - name: assert that databases exist (since performed recreation of existing databases via check mode) assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name in mysql_result.stdout + - db3_name in mysql_result.stdout # ========================================================================== # Recreate same databases @@ -160,9 +160,9 @@ - name: assert that databases does priorly exist assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name in mysql_result.stdout + - db3_name in mysql_result.stdout # ========================================================================== # Delete one of the databases (db2 here) @@ -189,9 +189,9 @@ - name: assert that only db2 database does not exist assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' not in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name not in mysql_result.stdout + - db3_name in mysql_result.stdout # ========================================================================= # Recreate multiple databases in which few databases does not exists (check mode) @@ -221,9 +221,9 @@ - name: assert that recreated non existing databases does not exist (since created via check mode) assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' not in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name not in mysql_result.stdout + - db3_name in mysql_result.stdout # ========================================================================== # Create multiple databases @@ -252,9 +252,9 @@ - name: assert that databases exist assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name in mysql_result.stdout + - db3_name in mysql_result.stdout # ============================== DUMP TEST ================================= # @@ -293,9 +293,9 @@ - name: assert that databases exist (check mode) assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name in mysql_result.stdout + - db3_name in mysql_result.stdout - name: state dump - file name should not exist (since dumped via check mode) file: @@ -332,10 +332,10 @@ - name: assert that databases exist (since check mode) assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" - - "'{{ db4_name }}' not in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name in mysql_result.stdout + - db3_name in mysql_result.stdout + - db4_name not in mysql_result.stdout - name: state dump - file name should not exist (since prior dump operation performed via check mode) file: @@ -371,11 +371,11 @@ - name: assert that databases exist (since delete via check mode) assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" - - "'{{ db4_name }}' not in mysql_result.stdout" - - "'{{ db5_name }}' not in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name in mysql_result.stdout + - db3_name in mysql_result.stdout + - db4_name not in mysql_result.stdout + - db5_name not in mysql_result.stdout - name: state dump - file name should not exist (since prior dump operation performed via check mode) file: @@ -403,7 +403,7 @@ assert: that: - dump_result is changed - - dump_result.db_list == ['{{ db1_name }}', '{{ db2_name }}', '{{ db3_name }}'] + - dump_result.db_list == [db1_name, db2_name, db3_name] - name: Run command to list databases like specified database name command: "{{ mysql_command }} \"-e show databases like 'database%'\"" @@ -412,9 +412,9 @@ - name: assert that databases exist assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name in mysql_result.stdout + - db3_name in mysql_result.stdout - name: State dump - file name should exist (dump1_file) file: @@ -461,11 +461,11 @@ - name: assert that databases exist assert: that: - - "'{{ db1_name }}' in mysql_result.stdout" - - "'{{ db2_name }}' in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" - - "'{{ db4_name }}' not in mysql_result.stdout" - - "'{{ db5_name }}' not in mysql_result.stdout" + - db1_name in mysql_result.stdout + - db2_name in mysql_result.stdout + - db3_name in mysql_result.stdout + - db4_name not in mysql_result.stdout + - db5_name not in mysql_result.stdout - name: state dump - file name should exist (dump2_file) file: @@ -501,8 +501,8 @@ - name: assert that databases exist even after deleting (since deleted via check mode) assert: that: - - "'{{ db2_name }}' in mysql_result.stdout" - - "'{{ db3_name }}' in mysql_result.stdout" + - db2_name in mysql_result.stdout + - db3_name in mysql_result.stdout # ========================================================================== # Delete multiple databases @@ -522,7 +522,7 @@ assert: that: - result is changed - - result.db_list == ['{{ db2_name }}', '{{ db3_name }}'] + - result.db_list == [db2_name, db3_name] - name: run command to list databases like specified database name command: "{{ mysql_command }} \"-e show databases like 'database%'\"" @@ -531,8 +531,8 @@ - name: assert that databases does not exist assert: that: - - "'{{ db2_name }}' not in mysql_result.stdout" - - "'{{ db3_name }}' not in mysql_result.stdout" + - db2_name not in mysql_result.stdout + - db3_name not in mysql_result.stdout # ========================================================================== # Delete non existing databases (check mode) @@ -561,8 +561,8 @@ - name: assert that databases does not exist since were deleted priorly (check mode) assert: that: - - "'{{ db2_name }}' not in mysql_result.stdout" - - "'{{ db4_name }}' not in mysql_result.stdout" + - db2_name not in mysql_result.stdout + - db4_name not in mysql_result.stdout # ========================================================================== # Delete already deleted databases @@ -590,8 +590,8 @@ - name: assert that databases does not exists assert: that: - - "'{{ db2_name }}' not in mysql_result.stdout" - - "'{{ db4_name }}' not in mysql_result.stdout" + - db2_name not in mysql_result.stdout + - db4_name not in mysql_result.stdout # ========================================================================== # Delete all databases @@ -622,11 +622,11 @@ - name: assert that specific databases does not exist assert: that: - - "'{{ db1_name }}' not in mysql_result.stdout" - - "'{{ db2_name }}' not in mysql_result.stdout" - - "'{{ db3_name }}' not in mysql_result.stdout" - - "'{{ db4_name }}' not in mysql_result.stdout" - - "'{{ db5_name }}' not in mysql_result.stdout" + - db1_name not in mysql_result.stdout + - db2_name not in mysql_result.stdout + - db3_name not in mysql_result.stdout + - db4_name not in mysql_result.stdout + - db5_name not in mysql_result.stdout - name: state dump - dump 1 file name should be removed file: diff --git a/tests/integration/targets/test_mysql_db/tasks/state_dump_import.yml b/tests/integration/targets/test_mysql_db/tasks/state_dump_import.yml index f8d2b4b..96e6371 100644 --- a/tests/integration/targets/test_mysql_db/tasks/state_dump_import.yml +++ b/tests/integration/targets/test_mysql_db/tasks/state_dump_import.yml @@ -283,7 +283,7 @@ - name: Dump and Import | Assert that db_name2 database does not exist assert: that: - - "'{{ db_name2 }}' not in mysql_result.stdout" + - db_name2 not in mysql_result.stdout - name: Dump and Import | Test state=import to restore a database from dumped file2 (check mode) mysql_db: @@ -309,7 +309,7 @@ - name: Dump and Import | Assert that db_name2 database does not exist (check mode) assert: that: - - "'{{ db_name2 }}' not in mysql_result.stdout" + - db_name2 not in mysql_result.stdout - name: Dump and Import | Test state=import to restore a database from multiple database dumped file2 mysql_db: @@ -326,7 +326,7 @@ assert: that: - import_result2 is changed - - import_result2.db_list == ['{{ db_name2 }}'] + - import_result2.db_list == [db_name2] - name: Dump and Import | Run command to list databases command: "{{ mysql_command }} \"-e show databases like 'data%'\"" @@ -335,7 +335,7 @@ - name: Dump and Import | Assert that db_name2 database does exist after import assert: that: - - "'{{ db_name2 }}' in mysql_result.stdout" + - db_name2 in mysql_result.stdout - name: Dump and Import | Test state=dump to backup the database of type {{ format_type }} (expect changed=true) mysql_db: @@ -487,18 +487,22 @@ login_password: '{{ mysql_password }}' login_host: '{{ mysql_host }}' login_port: '{{ mysql_primary_port }}' - name: '{{ item }}' + name: '{{ cleanup_db }}' state: absent loop: - '{{ db_name }}' - '{{ db_name2 }}' + loop_control: + loop_var: cleanup_db - name: Dump and Import | Clean up files file: - name: '{{ item }}' + name: '{{ cleanup_file }}' state: absent loop: - '{{ db_file_name }}' - '{{ wrong_sql_file }}' - '{{ dump_file1 }}' - '{{ dump_file2 }}' + loop_control: + loop_var: cleanup_file diff --git a/tests/integration/targets/test_mysql_db/tasks/state_present_absent.yml b/tests/integration/targets/test_mysql_db/tasks/state_present_absent.yml index 12633f2..6b3c772 100644 --- a/tests/integration/targets/test_mysql_db/tasks/state_present_absent.yml +++ b/tests/integration/targets/test_mysql_db/tasks/state_present_absent.yml @@ -41,8 +41,10 @@ assert: that: - result is changed - - result.db == '{{ db_name }}' - - result.executed_commands == ["CREATE DATABASE `{{ db_name }}`"] + - result.db == db_name + - result.executed_commands == expected_commands + vars: + expected_commands: ["CREATE DATABASE `{{ db_name }}`"] - name: State Present Absent | Run command to test state=present for a database name (expect db_name in stdout) command: "{{ mysql_command }} -e \"show databases like '{{ db_name | regex_replace(\"([%_\\\\])\", \"\\\\\\1\") }}'\"" @@ -51,7 +53,7 @@ - name: State Present Absent | Assert database exist assert: that: - - "'{{ db_name }}' in result.stdout" + - db_name in result.stdout # ============================================================ - name: State Present Absent | Test state=absent for a database name (expect changed=true) @@ -68,8 +70,10 @@ assert: that: - result is changed - - result.db == '{{ db_name }}' - - result.executed_commands == ["DROP DATABASE `{{ db_name }}`"] + - result.db == db_name + - result.executed_commands == expected_commands + vars: + expected_commands: ["DROP DATABASE `{{ db_name }}`"] - name: State Present Absent | Run command to test state=absent for a database name (expect db_name not in stdout) command: "{{ mysql_command }} -e \"show databases like '{{ db_name | regex_replace(\"([%_\\\\])\", \"\\\\\\1\") }}'\"" @@ -78,7 +82,7 @@ - name: State Present Absent | Assert database does not exist assert: that: - - "'{{ db_name }}' not in result.stdout" + - db_name not in result.stdout # ============================================================ - name: State Present Absent | Test mysql_db encoding param not valid - issue 8075 @@ -116,7 +120,9 @@ assert: that: - result is changed - - result.executed_commands == ["CREATE DATABASE `en{{ db_name }}` CHARACTER SET 'utf8'"] + - result.executed_commands == expected_commands + vars: + expected_commands: ["CREATE DATABASE `en{{ db_name }}` CHARACTER SET 'utf8'"] - name: State Present Absent | Test database was created command: "{{ mysql_command }} -e \"SHOW CREATE DATABASE `en{{ db_name }}`\"" @@ -152,7 +158,9 @@ assert: that: - result is changed - - result.executed_commands == ["CREATE DATABASE `en{{ db_name }}` CHARACTER SET 'binary'"] + - result.executed_commands == expected_commands + vars: + expected_commands: ["CREATE DATABASE `en{{ db_name }}` CHARACTER SET 'binary'"] - name: State Present Absent | Run command to test database was created command: "{{ mysql_command }} -e \"SHOW CREATE DATABASE `en{{ db_name }}`\"" @@ -207,7 +215,7 @@ - name: State Present Absent | Assert database exist assert: that: - - "'{{ db_user1 }}' in result.stdout" + - db_user1 in result.stdout # ============================================================ - name: State Present Absent | Create user2 to access database with privilege select only @@ -245,7 +253,7 @@ - name: State Present Absent | Assert database does not exist assert: that: - - "'{{ db_user2 }}' not in result.stdout" + - db_user2 not in result.stdout # ============================================================ - name: State Present Absent | Delete database using user2 with no privilege to delete (expect failed=true) @@ -272,7 +280,7 @@ - name: State Present Absent | Assert database still exist assert: that: - - "'{{ db_user1 }}' in result.stdout" + - db_user1 in result.stdout # ============================================================ - name: State Present Absent | Delete database using user1 with all privilege to delete a database (expect changed=true) @@ -290,7 +298,9 @@ assert: that: - result is changed - - result.executed_commands == ["DROP DATABASE `{{ db_user1 }}`"] + - result.executed_commands == expected_commands + vars: + expected_commands: ["DROP DATABASE `{{ db_user1 }}`"] - name: State Present Absent | Run command to test database was deleted using user1 command: "{{ mysql_command }} -e \"show databases like '{{ db_name | regex_replace(\"([%_\\\\])\", \"\\\\\\1\") }}'\"" @@ -299,4 +309,4 @@ - name: State Present Absent | Assert database does not exist assert: that: - - "'{{ db_user1 }}' not in result.stdout" + - db_user1 not in result.stdout diff --git a/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml b/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml index 310f925..5545111 100644 --- a/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml +++ b/tests/integration/targets/test_mysql_query/tasks/mysql_query_initial.yml @@ -21,7 +21,9 @@ assert: that: - result is changed - - result.executed_queries == ['CREATE DATABASE {{ test_db }}'] + - result.executed_queries == expected_queries + vars: + expected_queries: ['CREATE DATABASE {{ test_db }}'] - name: Create {{ test_table1 }} mysql_query: @@ -34,8 +36,10 @@ assert: that: - result is changed - - result.executed_queries == ['CREATE TABLE {{ test_table1 }} (id int)'] + - result.executed_queries == expected_queries - result.execution_time_ms[0] > 0 + vars: + expected_queries: ['CREATE TABLE {{ test_table1 }} (id int)'] - name: Insert test data mysql_query: @@ -52,9 +56,14 @@ that: - result is changed - result.rowcount == [2, 1] - - result.executed_queries == ['INSERT INTO {{ test_table1 }} VALUES (1), (2)', 'INSERT INTO {{ test_table1 }} VALUES (3)'] + - result.executed_queries == expected_queries - result.execution_time_ms[0] > 0 - result.execution_time_ms[1] > 0 + vars: + expected_queries: [ + 'INSERT INTO {{ test_table1 }} VALUES (1), (2)', + 'INSERT INTO {{ test_table1 }} VALUES (3)', + ] - name: Check data in {{ test_table1 }} mysql_query: @@ -67,11 +76,13 @@ assert: that: - result is not changed - - result.executed_queries == ['SELECT * FROM {{ test_table1 }}'] + - result.executed_queries == expected_queries - result.rowcount == [3] - result.query_result[0][0].id == 1 - result.query_result[0][1].id == 2 - result.query_result[0][2].id == 3 + vars: + expected_queries: ['SELECT * FROM {{ test_table1 }}'] - name: Check data in {{ test_table1 }} using positional args mysql_query: @@ -86,9 +97,11 @@ assert: that: - result is not changed - - result.executed_queries == ["SELECT * FROM {{ test_table1 }} WHERE id = 1"] + - result.executed_queries == expected_queries - result.rowcount == [1] - result.query_result[0][0].id == 1 + vars: + expected_queries: ["SELECT * FROM {{ test_table1 }} WHERE id = 1"] - name: Check data in {{ test_table1 }} using named args mysql_query: @@ -103,9 +116,11 @@ assert: that: - result is not changed - - result.executed_queries == ["SELECT * FROM {{ test_table1 }} WHERE id = 1"] + - result.executed_queries == expected_queries - result.rowcount == [1] - result.query_result[0][0].id == 1 + vars: + expected_queries: ["SELECT * FROM {{ test_table1 }} WHERE id = 1"] - name: Update data in {{ test_table1 }} mysql_query: @@ -121,8 +136,10 @@ assert: that: - result is changed - - result.executed_queries == ['UPDATE {{ test_table1 }} SET id = 0 WHERE id = 1'] + - result.executed_queries == expected_queries - result.rowcount == [1] + vars: + expected_queries: ['UPDATE {{ test_table1 }} SET id = 0 WHERE id = 1'] - name: Check the prev update - row with value 1 does not exist anymore mysql_query: @@ -137,8 +154,10 @@ assert: that: - result is not changed - - result.executed_queries == ['SELECT * FROM {{ test_table1 }} WHERE id = 1'] + - result.executed_queries == expected_queries - result.rowcount == [0] + vars: + expected_queries: ['SELECT * FROM {{ test_table1 }} WHERE id = 1'] - name: Check the prev update - row with value - exist mysql_query: @@ -153,8 +172,10 @@ assert: that: - result is not changed - - result.executed_queries == ['SELECT * FROM {{ test_table1 }} WHERE id = 0'] + - result.executed_queries == expected_queries - result.rowcount == [1] + vars: + expected_queries: ['SELECT * FROM {{ test_table1 }} WHERE id = 0'] - name: Update data in {{ test_table1 }} again mysql_query: @@ -170,8 +191,10 @@ assert: that: - result is not changed - - result.executed_queries == ['UPDATE {{ test_table1 }} SET id = 0 WHERE id = 1'] + - result.executed_queries == expected_queries - result.rowcount == [0] + vars: + expected_queries: ['UPDATE {{ test_table1 }} SET id = 0 WHERE id = 1'] - name: Delete data from {{ test_table1 }} mysql_query: @@ -186,8 +209,10 @@ assert: that: - result is changed - - result.executed_queries == ['DELETE FROM {{ test_table1 }} WHERE id = 0', 'SELECT * FROM {{ test_table1 }} WHERE id = 0'] + - result.executed_queries == expected_queries - result.rowcount == [1, 0] + vars: + expected_queries: ['DELETE FROM {{ test_table1 }} WHERE id = 0', 'SELECT * FROM {{ test_table1 }} WHERE id = 0'] - name: Delete data from {{ test_table1 }} again mysql_query: @@ -200,8 +225,10 @@ assert: that: - result is not changed - - result.executed_queries == ['DELETE FROM {{ test_table1 }} WHERE id = 0'] + - result.executed_queries == expected_queries - result.rowcount == [0] + vars: + expected_queries: ['DELETE FROM {{ test_table1 }} WHERE id = 0'] - name: Truncate {{ test_table1 }} mysql_query: @@ -216,8 +243,10 @@ assert: that: - result is changed - - result.executed_queries == ['TRUNCATE {{ test_table1 }}', 'SELECT * FROM {{ test_table1 }}'] + - result.executed_queries == expected_queries - result.rowcount == [0, 0] + vars: + expected_queries: ['TRUNCATE {{ test_table1 }}', 'SELECT * FROM {{ test_table1 }}'] - name: Rename {{ test_table1 }} mysql_query: @@ -230,8 +259,10 @@ assert: that: - result is changed - - result.executed_queries == ['RENAME TABLE {{ test_table1 }} TO {{ test_table2 }}'] + - result.executed_queries == expected_queries - result.rowcount == [0] + vars: + expected_queries: ['RENAME TABLE {{ test_table1 }} TO {{ test_table2 }}'] - name: Check the prev rename mysql_query: @@ -398,7 +429,9 @@ assert: that: - result is changed - - result.executed_queries == ['DROP DATABASE {{ test_db }}'] + - result.executed_queries == expected_queries + vars: + expected_queries: ['DROP DATABASE {{ test_db }}'] always: diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml index 0bcc6e6..67d9261 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_channel.yml @@ -75,7 +75,7 @@ - assert: that: - result is changed - - result.queries == result_query or result_query2 + - result.queries in [result_query, result_query2] vars: result_query: ["START SLAVE FOR CHANNEL '{{ test_channel }}'"] result_query2: ["START REPLICA FOR CHANNEL '{{ test_channel }}'"] diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml index 00699c1..182ea5c 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_initial.yml @@ -337,5 +337,6 @@ - name: Assert that stopslave returns expected error message assert: that: - - result.msg == "value of mode must be one of{{ ":" }} getprimary, getreplica, changeprimary, stopreplica, startreplica, resetprimary, resetreplica, resetreplicaall, changereplication, got{{ ":" }} stopslave" + - + "result.msg == 'value of mode must be one of: getprimary, getreplica, changeprimary, stopreplica, startreplica, resetprimary, resetreplica, resetreplicaall, changereplication, got: stopslave'" - result is failed diff --git a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml index 2093b70..fe22db8 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/mysql_replication_primary_delay.yml @@ -55,5 +55,5 @@ - assert: that: - - replica_status.SQL_Delay == {{ test_primary_delay }} + - replica_status.SQL_Delay == test_primary_delay - replica_status is not changed diff --git a/tests/integration/targets/test_mysql_user/tasks/test_privs.yml b/tests/integration/targets/test_mysql_user/tasks/test_privs.yml index 95d44aa..cd50427 100644 --- a/tests/integration/targets/test_mysql_user/tasks/test_privs.yml +++ b/tests/integration/targets/test_mysql_user/tasks/test_privs.yml @@ -220,7 +220,7 @@ - name: Privs | Assert that 'GRANT' permission is present assert: that: - - mysql_info_about_users.users.localhost.{{ user_name_2 }}.Grant_priv == 'Y' + - mysql_info_about_users.users.localhost.db_user2.Grant_priv == 'Y' - name: Privs | Test idempotency (expect ok) mysql_user: @@ -246,7 +246,7 @@ - name: Privs | Assert that 'GRANT' permission is present (by host) assert: that: - - mysql_info_about_users.users.localhost.{{ user_name_2 }}.Grant_priv == 'Y' + - mysql_info_about_users.users.localhost.db_user2.Grant_priv == 'Y' # ============================================================ - name: Privs | Update user with invalid privileges diff --git a/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml b/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml index f6f3c2e..164fea7 100644 --- a/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml +++ b/tests/integration/targets/test_mysql_user/tasks/test_user_plugin_auth.yml @@ -45,7 +45,7 @@ - name: Plugin auth | Check that the expected plugin type is set (with hash string) ansible.builtin.assert: that: - - "'{{ test_plugin_type }}' in show_create_user.stdout" + - test_plugin_type in show_create_user.stdout when: db_engine == 'mysql' or (db_engine == 'mariadb' and db_version is version('10.3', '>=')) - ansible.builtin.include_tasks: utils/assert_user.yml diff --git a/tests/integration/targets/test_mysql_user/tasks/utils/assert_user.yml b/tests/integration/targets/test_mysql_user/tasks/utils/assert_user.yml index e6bd23f..e03386e 100644 --- a/tests/integration/targets/test_mysql_user/tasks/utils/assert_user.yml +++ b/tests/integration/targets/test_mysql_user/tasks/utils/assert_user.yml @@ -17,5 +17,7 @@ - name: Utils | Assert user | Assert user has given privileges ansible.builtin.assert: that: - - "'GRANT {{ priv }} ON *.*' in result.stdout" + - expected_command in result.stdout when: priv is defined + vars: + expected_command: "GRANT {{ priv }} ON *.*" diff --git a/tests/integration/targets/test_mysql_variables/tasks/assert_var.yml b/tests/integration/targets/test_mysql_variables/tasks/assert_var.yml index e64c5a7..d72c7e8 100644 --- a/tests/integration/targets/test_mysql_variables/tasks/assert_var.yml +++ b/tests/integration/targets/test_mysql_variables/tasks/assert_var.yml @@ -23,7 +23,7 @@ - name: Assert output message changed value assert: that: - - "output.changed | bool == changed | bool" + - output.changed | bool == changed | bool - name: Run mysql command to show variable command: "{{ mysql_command }} \"-e show variables like '{{ var_name }}'\"" @@ -33,5 +33,5 @@ assert: that: - result is changed - - "'{{ var_name }}' in result.stdout" - - "'{{ var_value }}' in result.stdout" + - var_name | string in result.stdout + - var_value | string in result.stdout diff --git a/tests/integration/targets/test_mysql_variables/tasks/assert_var_output.yml b/tests/integration/targets/test_mysql_variables/tasks/assert_var_output.yml index 6f26386..0e9874c 100644 --- a/tests/integration/targets/test_mysql_variables/tasks/assert_var_output.yml +++ b/tests/integration/targets/test_mysql_variables/tasks/assert_var_output.yml @@ -22,7 +22,7 @@ - name: assert output message changed value assert: that: - - "output.changed | bool == changed | bool" + - output.changed | bool == changed | bool - set_fact: key_name: "{{ var_name }}" @@ -36,5 +36,5 @@ assert: that: - result is changed - - "key_name in result.stdout" - - "key_value in result.stdout" + - key_name in result.stdout + - key_value in result.stdout diff --git a/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml b/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml index 8194172..e2fe9ab 100644 --- a/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml +++ b/tests/integration/targets/test_mysql_variables/tasks/mysql_variables.yml @@ -92,7 +92,9 @@ - assert: that: - - result.queries == ["SET GLOBAL `{{ set_name }}` = {{ set_value }}"] + - result.queries == expected_queries + vars: + expected_queries: ["SET GLOBAL `{{ set_name }}` = {{ set_value }}"] - include_tasks: assert_var.yml vars: @@ -291,9 +293,9 @@ # Bugfix https://github.com/ansible-collections/community.mysql/issues/652 - name: Get server version - register: result mysql_info: <<: *mysql_params + register: result - name: Set variable name when running on MySQL set_fact: @@ -316,7 +318,9 @@ assert: that: - result is changed or result.msg == "Variable is already set to requested value." - - result.msg == "Variable is already set to requested value." or result.queries == ["SET GLOBAL `{{ log_slow_statements }}` = ON"] + - result.msg == "Variable is already set to requested value." or result.queries == expected_queries + vars: + expected_queries: ["SET GLOBAL `{{ log_slow_statements }}` = ON"] - name: Set a boolean value again using ON mysql_variables: @@ -378,7 +382,9 @@ assert: that: - result is changed - - result.queries == ["SET GLOBAL `{{ log_slow_statements }}` = ON"] + - result.queries == expected_queries + vars: + expected_queries: ["SET GLOBAL `{{ log_slow_statements }}` = ON"] #============================================================ # Verify mysql_variable fails with an incorrect login_password parameter @@ -452,7 +458,9 @@ - assert: that: - - result.queries == ["SET PERSIST `{{ set_name }}` = {{ set_value }}"] + - result.queries == expected_queries + vars: + expected_queries: ["SET PERSIST `{{ set_name }}` = {{ set_value }}"] - include_tasks: assert_var.yml vars: @@ -494,7 +502,9 @@ - assert: that: - result is changed - - result.queries == ["SET PERSIST_ONLY `{{ set_name }}` = {{ set_value }}"] + - result.queries == expected_queries + vars: + expected_queries: ["SET PERSIST_ONLY `{{ set_name }}` = {{ set_value }}"] - name: try to update mysql variable value (expect changed=false) in persist_only mode again mysql_variables: From 74ea0438ce214931b6849de8043a3f9014049925 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Mon, 28 Apr 2025 14:45:33 +0200 Subject: [PATCH 49/54] fix mysql_info crash if PUBLIC role is used on MariaDB 10.11+ (#712) * tests: grant a privilege to the PUBLIC role to force the module to fail * skip the PUBLIC role from the users migration example * Cut the 3rd matching group as it fails to match roles without quotes The group purpose was to match the closing quotes. We don't need the value after the TO. So we can ignore this entirely. * add MariaDB role detection * Add changelog fragment * doc: add notes that users_info doesn't support MariaDB roles. --- changelogs/fragments/grant_to_public.yml | 3 +++ plugins/module_utils/user.py | 2 +- plugins/modules/mysql_info.py | 6 +++++- .../test_mysql_info/tasks/filter_users_info.yml | 15 ++++++++++++++- 4 files changed, 23 insertions(+), 3 deletions(-) create mode 100644 changelogs/fragments/grant_to_public.yml diff --git a/changelogs/fragments/grant_to_public.yml b/changelogs/fragments/grant_to_public.yml new file mode 100644 index 0000000..ea24da0 --- /dev/null +++ b/changelogs/fragments/grant_to_public.yml @@ -0,0 +1,3 @@ +--- +bugfixes: + - mysql_info - fix a crash (ERROR 1141, There is no such grant defined for user 'PUBLIC' on host '%') when using the ``users_info`` filter with a PUBLIC role present in MariaDB 10.11+. Do note that the fix doesn't change the fact that the module won't return the privileges from the PUBLIC role in the users privileges list. It can't do that because you have to login as the particular user and use `SHOW GRANTS FOR CURRENT_USER`. We considered using an aggregation with the `SHOW GRANTS FOR PUBLIC` command. However, this approach would make copying users from one server to another transform the privileges inherited from the role as if they were direct privileges on the user. diff --git a/plugins/module_utils/user.py b/plugins/module_utils/user.py index 9de1c6d..337cc67 100644 --- a/plugins/module_utils/user.py +++ b/plugins/module_utils/user.py @@ -662,7 +662,7 @@ def privileges_get(cursor, user, host, maria_role=False): if not maria_role: res = re.match("""GRANT (.+) ON (.+) TO (['`"]).*\\3@(['`"]).*\\4( IDENTIFIED BY PASSWORD (['`"]).+\\6)? ?(.*)""", grant[0]) else: - res = re.match("""GRANT (.+) ON (.+) TO (['`"]).*\\3""", grant[0]) + res = re.match("""GRANT (.+) ON (.+) TO .*""", grant[0]) if res is None: # If a user has roles assigned, we'll have one of priv tuples looking like diff --git a/plugins/modules/mysql_info.py b/plugins/modules/mysql_info.py index a630f18..0447743 100644 --- a/plugins/modules/mysql_info.py +++ b/plugins/modules/mysql_info.py @@ -50,6 +50,7 @@ notes: - Compatible with MariaDB or MySQL. - Calculating the size of a database might be slow, depending on the number and size of tables in it. To avoid this, use I(exclude_fields=db_size). +- filters C(users_info) doesn't support MariaDB roles. attributes: check_mode: @@ -161,6 +162,7 @@ EXAMPLES = r''' - item.name != 'root' # In case you don't want to import admin accounts - item.name != 'mariadb.sys' - item.name != 'mysql' + - item.name != 'PUBLIC' # MariaDB roles are not supported ''' RETURN = r''' @@ -606,7 +608,9 @@ class MySQL_Info(object): user = line['User'] host = line['Host'] - user_priv = privileges_get(self.cursor, user, host) + # MariaDB roles have no host + is_role = self.server_implementation == 'mariadb' and not host + user_priv = privileges_get(self.cursor, user, host, maria_role=is_role) if not user_priv: self.module.warn("No privileges found for %s on host %s" % (user, host)) diff --git a/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml b/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml index 558d309..102bf59 100644 --- a/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml +++ b/tests/integration/targets/test_mysql_info/tasks/filter_users_info.yml @@ -31,6 +31,19 @@ CREATE TABLE IF NOT EXISTS users_info_db.T_UPPER (id int, name1 varchar(9), NAME2 varchar(9), Name3 varchar(9)) + # No need for a specific test later. When the module will retrieve the + # users privileges, it will fail with an error "1141 - There is no such + # grant defined for user 'PUBLIC' on host'%'" if the PUBLIC role is not + # handled properly by our module. + - name: Mysql_info users_info | Grant to PUBLIC for MariaDB 10.11+ + community.mysql.mysql_query: + query: + - >- + GRANT SELECT,INSERT,UPDATE,DELETE on users_info_db.* TO PUBLIC + when: + - db_engine == 'mariadb' + - db_version is version('10.11.1', '>=') + # I failed to create a procedure using community.mysql.mysql_query. # Maybe it's because we must changed the delimiter. - name: Mysql_info users_info | Create procedure SQL file @@ -277,7 +290,7 @@ - item.name != 'mysql.sys' - item.name != 'mysql.infoschema' - item.name != 'mysql.session' - + - item.name != 'PUBLIC' # MariaDB roles are not supported # ================================== Cleanup ============================ From 06e23c8ac3b96b0dbef31eb9020677560ec0dc65 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Tue, 6 May 2025 09:05:41 +0200 Subject: [PATCH 50/54] Add query to find command between python and python3 (#715) * Add query to find command between python and python3 * Cut extra $ --- tests/integration/targets/setup_controller/tasks/verify.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/integration/targets/setup_controller/tasks/verify.yml b/tests/integration/targets/setup_controller/tasks/verify.yml index b47e354..d3d615f 100644 --- a/tests/integration/targets/setup_controller/tasks/verify.yml +++ b/tests/integration/targets/setup_controller/tasks/verify.yml @@ -42,8 +42,8 @@ - connector_name == 'mysqlclient' - name: Get the python version in use - ansible.builtin.command: - cmd: python -V + ansible.builtin.shell: + cmd: echo $( $(command -v python || command -v python3) -V ) changed_when: false failed_when: false register: python_version_in_use @@ -52,7 +52,7 @@ ansible.builtin.debug: msg: > Python in use inside the test container: - ${{ python_version_in_use }} + {{ python_version_in_use.stdout }} when: - python_version_in_use is defined From 7307a51f200b2236854a193f2956db78daf9f2af Mon Sep 17 00:00:00 2001 From: Joakim Soderlund Date: Wed, 7 May 2025 14:32:55 +0200 Subject: [PATCH 51/54] Add action group containing all modules (#704) Helpful for setting module defaults. For example, when using the same credentials across multiple tasks while only declaring them once. This change creates the group `community.mysql.all` for that purpose. --- meta/runtime.yml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/meta/runtime.yml b/meta/runtime.yml index 2ee3c9f..0a472e8 100644 --- a/meta/runtime.yml +++ b/meta/runtime.yml @@ -1,2 +1,11 @@ --- requires_ansible: '>=2.9.10' +action_groups: + all: + - mysql_db + - mysql_info + - mysql_query + - mysql_replication + - mysql_role + - mysql_user + - mysql_variables From 49be739e89373503f3a38aa9192ce6e004c0ea0d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Fri, 9 May 2025 11:00:29 +0200 Subject: [PATCH 52/54] Fix ssl verification always enabled for replication even if set to false (#707) * Fix ssl verification always enabled for replication even if set to false * add changelog fragment * fix test when multiple replication channels are present * test: add check for changereplication * fix mismatch default value We use "None" as a default to know if the user provide this option or not. But a bool can't have a default of "None" for the sanity tests. So we must erase the line. * doc: add var name used by MySQL and MariaDB * Revert the change of default value * style * fix indentation * Revert "Revert the change of default value" This reverts commit db047fda9044a31fa9f3d47b3c2d60d58731fb12. * add changelog about changed default value * add link to issue and PR in changelog * doc: explain false had no effect prior to 3.14.0 --- .../707-source_ssl_verify_server_cert.yml | 6 ++ plugins/modules/mysql_replication.py | 21 ++++--- .../tasks/issue-689.yml | 62 +++++++++++++++++++ .../test_mysql_replication/tasks/main.yml | 4 ++ 4 files changed, 86 insertions(+), 7 deletions(-) create mode 100644 changelogs/fragments/707-source_ssl_verify_server_cert.yml create mode 100644 tests/integration/targets/test_mysql_replication/tasks/issue-689.yml diff --git a/changelogs/fragments/707-source_ssl_verify_server_cert.yml b/changelogs/fragments/707-source_ssl_verify_server_cert.yml new file mode 100644 index 0000000..592a831 --- /dev/null +++ b/changelogs/fragments/707-source_ssl_verify_server_cert.yml @@ -0,0 +1,6 @@ +--- +bugfixes: + - mysql_replication - fixed an issue where setting ``primary_ssl_verify_server_cert`` to false had no effect (https://github.com/ansible-collections/community.mysql/issues/689). + +minor_changes: + - mysql_replication - change default value for ``primary_ssl_verify_server_cert`` from False to None. This should not affect existing playbooks (https://github.com/ansible-collections/community.mysql/pull/707). diff --git a/plugins/modules/mysql_replication.py b/plugins/modules/mysql_replication.py index b902da0..a743b82 100644 --- a/plugins/modules/mysql_replication.py +++ b/plugins/modules/mysql_replication.py @@ -137,9 +137,10 @@ options: aliases: [master_ssl_cipher] primary_ssl_verify_server_cert: description: - - Same as mysql variable. + - Same as C(MASTER_SSL_VERIFY_SERVER_CERT) MySQL/MariaDB variable. + - The module switch automatically to C(SOURCE_SSL_VERIFY_SERVER_CERT) for MySQL 8.0.23 and later. + - Prior to community.mysql 3.14.0 C(false) had no effect. type: bool - default: false version_added: '3.5.0' primary_auto_position: description: @@ -493,7 +494,7 @@ def main(): primary_ssl_cert=dict(type='str', aliases=['master_ssl_cert']), primary_ssl_key=dict(type='str', no_log=False, aliases=['master_ssl_key']), primary_ssl_cipher=dict(type='str', aliases=['master_ssl_cipher']), - primary_ssl_verify_server_cert=dict(type='bool', default=False), + primary_ssl_verify_server_cert=dict(type='bool'), primary_use_gtid=dict(type='str', choices=[ 'current_pos', 'replica_pos', 'disabled'], aliases=['master_use_gtid']), primary_delay=dict(type='int', aliases=['master_delay']), @@ -641,8 +642,11 @@ def main(): chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_SSL_KEY'), primary_ssl_key)) if primary_ssl_cipher is not None: chm.append("%s='%s'" % (command_resolver.resolve_command('MASTER_SSL_CIPHER'), primary_ssl_cipher)) - if primary_ssl_verify_server_cert: - chm.append("%s=1" % command_resolver.resolve_command('MASTER_SSL_VERIFY_SERVER_CERT')) + if primary_ssl_verify_server_cert is not None: + if primary_ssl_verify_server_cert: + chm.append("%s=1" % command_resolver.resolve_command('MASTER_SSL_VERIFY_SERVER_CERT')) + else: + chm.append("%s=0" % command_resolver.resolve_command('MASTER_SSL_VERIFY_SERVER_CERT')) if primary_auto_position: chm.append("%s=1" % command_resolver.resolve_command('MASTER_AUTO_POSITION')) if primary_use_gtid is not None: @@ -723,8 +727,11 @@ def main(): chm.append("SOURCE_SSL_KEY='%s'" % primary_ssl_key) if primary_ssl_cipher is not None: chm.append("SOURCE_SSL_CIPHER='%s'" % primary_ssl_cipher) - if primary_ssl_verify_server_cert: - chm.append("SOURCE_SSL_VERIFY_SERVER_CERT=1") + if primary_ssl_verify_server_cert is not None: + if primary_ssl_verify_server_cert: + chm.append("%s=1" % command_resolver.resolve_command('MASTER_SSL_VERIFY_SERVER_CERT')) + else: + chm.append("%s=0" % command_resolver.resolve_command('MASTER_SSL_VERIFY_SERVER_CERT')) if primary_auto_position: chm.append("SOURCE_AUTO_POSITION=1") try: diff --git a/tests/integration/targets/test_mysql_replication/tasks/issue-689.yml b/tests/integration/targets/test_mysql_replication/tasks/issue-689.yml new file mode 100644 index 0000000..74fa1e0 --- /dev/null +++ b/tests/integration/targets/test_mysql_replication/tasks/issue-689.yml @@ -0,0 +1,62 @@ +--- + +- vars: + mysql_parameters: &mysql_params + login_user: '{{ mysql_user }}' + login_password: '{{ mysql_password }}' + login_host: '{{ mysql_host }}' + login_port: '{{ mysql_primary_port }}' + block: + + - name: Disable ssl verification + community.mysql.mysql_replication: + <<: *mysql_params + login_port: '{{ mysql_replica1_port }}' + mode: changeprimary + primary_ssl_verify_server_cert: false + register: result + + - name: Assert that changeprimmary is changed and return expected query for MariaDB and MySQL < 8.0.23 + ansible.builtin.assert: + that: + - result is changed + - result.queries == expected_queries + when: + - > + db_engine == 'mariadb' or + (db_engine == 'mysql' and db_version is version('8.0.23', '<')) + vars: + expected_queries: ["CHANGE MASTER TO MASTER_SSL_VERIFY_SERVER_CERT=0"] + + - name: Assert that changeprimmary is changed and return expected query for MySQL > 8.0.23 + ansible.builtin.assert: + that: + - result is changed + - result.queries == expected_queries + when: + - db_engine == 'mysql' + - db_version is version('8.0.23', '>=') + vars: + expected_queries: ["CHANGE REPLICATION SOURCE TO SOURCE_SSL_VERIFY_SERVER_CERT=0"] + + - name: Disable ssl verification for MySQL 8.0.23+ + community.mysql.mysql_replication: + <<: *mysql_params + login_port: '{{ mysql_replica1_port }}' + mode: changereplication + primary_ssl_verify_server_cert: false + register: result + when: + - db_engine == 'mysql' + - db_version is version('8.0.23', '>=') + + - name: Assert that changereplication is changed and return expected query for MySQL > 8.0.23 + ansible.builtin.assert: + that: + - result is changed + - result.queries == expected_queries + when: + - db_engine == 'mysql' + - db_version is version('8.0.23', '>=') + vars: + expected_queries: ["CHANGE REPLICATION SOURCE TO SOURCE_SSL_VERIFY_SERVER_CERT=0"] diff --git a/tests/integration/targets/test_mysql_replication/tasks/main.yml b/tests/integration/targets/test_mysql_replication/tasks/main.yml index 32ce553..e77af38 100644 --- a/tests/integration/targets/test_mysql_replication/tasks/main.yml +++ b/tests/integration/targets/test_mysql_replication/tasks/main.yml @@ -13,6 +13,10 @@ # Tests of replication filters and force_context - include_tasks: issue-265.yml +# primary_ssl_verify_server_cert +# Must run before mysql add channels in mysql_replication_channel.yml +- import_tasks: issue-689.yml + # Tests of primary_delay parameter: - import_tasks: mysql_replication_primary_delay.yml From fa3c72b2c0acd682a4be0ddcf8b3f69b0526b6df Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Laurent=20Inderm=C3=BChle?= Date: Fri, 23 May 2025 11:33:43 +0200 Subject: [PATCH 53/54] Release 3.14.0 commit (#717) --- CHANGELOG.rst | 21 +++++++++++++++ changelogs/changelog.yaml | 27 +++++++++++++++++++ .../707-source_ssl_verify_server_cert.yml | 6 ----- changelogs/fragments/grant_to_public.yml | 3 --- galaxy.yml | 2 +- 5 files changed, 49 insertions(+), 10 deletions(-) delete mode 100644 changelogs/fragments/707-source_ssl_verify_server_cert.yml delete mode 100644 changelogs/fragments/grant_to_public.yml diff --git a/CHANGELOG.rst b/CHANGELOG.rst index b318076..114b359 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -6,6 +6,27 @@ Community MySQL and MariaDB Collection Release Notes This changelog describes changes after version 2.0.0. +v3.14.0 +======= + +Release Summary +--------------- + +This is a minor release of the ``community.mysql`` collection. +This changelog contains all changes to the modules and plugins in this collection +that have been made after the previous release.' + +Minor Changes +------------- + +- mysql_replication - change default value for ``primary_ssl_verify_server_cert`` from False to None. This should not affect existing playbooks (https://github.com/ansible-collections/community.mysql/pull/707). + +Bugfixes +-------- + +- mysql_info - fix a crash (ERROR 1141, There is no such grant defined for user 'PUBLIC' on host '%') when using the ``users_info`` filter with a PUBLIC role present in MariaDB 10.11+. Do note that the fix doesn't change the fact that the module won't return the privileges from the PUBLIC role in the users privileges list. It can't do that because you have to login as the particular user and use `SHOW GRANTS FOR CURRENT_USER`. We considered using an aggregation with the `SHOW GRANTS FOR PUBLIC` command. However, this approach would make copying users from one server to another transform the privileges inherited from the role as if they were direct privileges on the user. +- mysql_replication - fixed an issue where setting ``primary_ssl_verify_server_cert`` to false had no effect (https://github.com/ansible-collections/community.mysql/issues/689). + v3.13.0 ======= diff --git a/changelogs/changelog.yaml b/changelogs/changelog.yaml index 5ec7dc9..79ee9cf 100644 --- a/changelogs/changelog.yaml +++ b/changelogs/changelog.yaml @@ -267,6 +267,33 @@ releases: - 702-user_locking.yaml - tests_mariadb_11_4.yml release_date: '2025-03-21' + 3.14.0: + changes: + bugfixes: + - mysql_info - fix a crash (ERROR 1141, There is no such grant defined for user + 'PUBLIC' on host '%') when using the ``users_info`` filter with a PUBLIC role + present in MariaDB 10.11+. Do note that the fix doesn't change the fact that + the module won't return the privileges from the PUBLIC role in the users privileges + list. It can't do that because you have to login as the particular user and + use `SHOW GRANTS FOR CURRENT_USER`. We considered using an aggregation with + the `SHOW GRANTS FOR PUBLIC` command. However, this approach would make copying + users from one server to another transform the privileges inherited from the + role as if they were direct privileges on the user. + - mysql_replication - fixed an issue where setting ``primary_ssl_verify_server_cert`` + to false had no effect (https://github.com/ansible-collections/community.mysql/issues/689). + minor_changes: + - mysql_replication - change default value for ``primary_ssl_verify_server_cert`` + from False to None. This should not affect existing playbooks (https://github.com/ansible-collections/community.mysql/pull/707). + release_summary: 'This is a minor release of the ``community.mysql`` collection. + + This changelog contains all changes to the modules and plugins in this collection + + that have been made after the previous release.''' + fragments: + - 707-source_ssl_verify_server_cert.yml + - grant_to_public.yml + - release_3_14_0.yml + release_date: '2025-05-23' 3.2.0: changes: bugfixes: diff --git a/changelogs/fragments/707-source_ssl_verify_server_cert.yml b/changelogs/fragments/707-source_ssl_verify_server_cert.yml deleted file mode 100644 index 592a831..0000000 --- a/changelogs/fragments/707-source_ssl_verify_server_cert.yml +++ /dev/null @@ -1,6 +0,0 @@ ---- -bugfixes: - - mysql_replication - fixed an issue where setting ``primary_ssl_verify_server_cert`` to false had no effect (https://github.com/ansible-collections/community.mysql/issues/689). - -minor_changes: - - mysql_replication - change default value for ``primary_ssl_verify_server_cert`` from False to None. This should not affect existing playbooks (https://github.com/ansible-collections/community.mysql/pull/707). diff --git a/changelogs/fragments/grant_to_public.yml b/changelogs/fragments/grant_to_public.yml deleted file mode 100644 index ea24da0..0000000 --- a/changelogs/fragments/grant_to_public.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -bugfixes: - - mysql_info - fix a crash (ERROR 1141, There is no such grant defined for user 'PUBLIC' on host '%') when using the ``users_info`` filter with a PUBLIC role present in MariaDB 10.11+. Do note that the fix doesn't change the fact that the module won't return the privileges from the PUBLIC role in the users privileges list. It can't do that because you have to login as the particular user and use `SHOW GRANTS FOR CURRENT_USER`. We considered using an aggregation with the `SHOW GRANTS FOR PUBLIC` command. However, this approach would make copying users from one server to another transform the privileges inherited from the role as if they were direct privileges on the user. diff --git a/galaxy.yml b/galaxy.yml index 624c7d6..01e50fc 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,7 +1,7 @@ --- namespace: community name: mysql -version: 3.13.0 +version: 3.14.0 readme: README.md authors: - Ansible community From 67f1460070e8a349a11f190d55bd0b5bd9c424b2 Mon Sep 17 00:00:00 2001 From: Laurent Indermuehle Date: Fri, 23 May 2025 11:38:52 +0200 Subject: [PATCH 54/54] Add next expected version --- galaxy.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/galaxy.yml b/galaxy.yml index 01e50fc..e69e4e8 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -1,7 +1,7 @@ --- namespace: community name: mysql -version: 3.14.0 +version: 3.14.1 readme: README.md authors: - Ansible community