Add sops lookup plugin (#374)

* add sops lookup plugin * fix pylint * fix undefined encrypted_file variable * decode sops output as text by default * add variable to control decrypted content print in logs * use Sops class decryption method * lookup should return text, use appropriate ansible facility * use ansible.module_utils._text.to_native As required by Ansible documentation on [raising errors][raising-errors] from plugins, use to_native to wrap errors to ensure string compatibility between Python versions. [raising-errors]: https://docs.ansible.com/ansible/latest/dev_guide/developing_plugins.html#id3 * use with_items instead of with_file in sops lookup documentation [with_file][with-file], per Ansible documentation, returns the content of the file. As sops is not able to decrypt a string by itself but requires the file is passed as argument, passing the content breaks the lookup plugin as reported by [here][bug-report]. [with_items][with-items] should be used instead. [with-file]: https://docs.ansible.com/ansible/2.4/playbooks_loops.html#looping-over-files [with-items]: https://docs.ansible.com/ansible/2.4/playbooks_loops.html#standard-loops [bug-report]: https://github.com/ansible/ansible/pull/59639#issuecomment-540803722 * uniform sops exception handling between plugins * Apply suggestions from code review Co-Authored-By: Felix Fontein <felix@fontein.de> * remove sops lookup plugin print option Is no longer possible to print the decrypted secrets directly from this plugin, but `debug` module can be used instead. * add github handle to author * add setup_sops integration target * extract sops module * add lookup_sops integration tests * use sops module * Update plugins/module_utils/sops.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update plugins/module_utils/sops.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update plugins/lookup/sops.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update plugins/module_utils/sops.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update plugins/module_utils/sops.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update test/integration/targets/lookup_sops/tasks/ubuntu.yml Co-authored-by: Felix Fontein <felix@fontein.de> * Update plugins/module_utils/sops.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update test/integration/targets/lookup_sops/files/simple.sops.yaml Co-authored-by: Felix Fontein <felix@fontein.de> * Adding aliases file * Emtpy spaces * Update plugins/lookup/sops.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update plugins/lookup/sops.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update plugins/lookup/sops.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update test/integration/targets/lookup_sops/tasks/ubuntu.yml Co-authored-by: Felix Fontein <felix@fontein.de> * Update plugins/lookup/sops.py Co-authored-by: Felix Fontein <felix@fontein.de> * Update test/integration/targets/lookup_sops/tasks/ubuntu.yml Co-authored-by: Felix Fontein <felix@fontein.de> * Update test/integration/targets/lookup_sops/tasks/ubuntu.yml Co-authored-by: Felix Fontein <felix@fontein.de> * gpg -> gnupg2 * with_items -> loop * Move error logic to module_utils. * Make Sops.decrypt() also handle errors and decode output. * Improve error handling. * Improve example formatting. * Reorganize tests. * Add test. * Remove version_added. Co-authored-by: Edoardo Tenani <edoardo.tenani@protonmail.com> Co-authored-by: Edoardo Tenani <edoardo.tenani@gmail.com> Co-authored-by: Edoardo T <endorama@users.noreply.github.com> Co-authored-by: Felix Fontein <felix@fontein.de>
2025-06-27 18:50:21 -07:00 · 2020-06-06 12:36:28 -07:00 · 2020-06-06 12:36:28 -07:00 · 1a13287788
commit 1a13287788
parent 77c29a1542
11 changed files with 273 additions and 0 deletions
--- a/tests/integration/targets/lookup_sops/aliases
+++ b/tests/integration/targets/lookup_sops/aliases
@ -0,0 +1,5 @@
+shippable/posix/group1
+skip/aix
+skip/osx
+skip/freebsd
+skip/python2.6  # lookups are controller only, and we no longer support Python 2.6 on the controller
--- a/tests/integration/targets/lookup_sops/files/.sops.yaml
+++ b/tests/integration/targets/lookup_sops/files/.sops.yaml
@ -0,0 +1,3 @@
+---
+creation_rules:
+    - pgp: FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4
--- a/tests/integration/targets/lookup_sops/files/simple.sops.yaml
+++ b/tests/integration/targets/lookup_sops/files/simple.sops.yaml
@ -0,0 +1,25 @@
+foo: ENC[AES256_GCM,data:a25L,iv:X8ILHZr+YiyLWa90Y+cwoMD1nVuel7JyTs0A5+oiOOo=,tag:GbBtp+Yqx1KEjdyztqS4EQ==,type:str]
+sops:
+    kms: []
+    gcp_kms: []
+    azure_kv: []
+    lastmodified: '2020-02-20T10:44:32Z'
+    mac: ENC[AES256_GCM,data:BAwQqD9sHgHkmlxPQLKq28Xy48qPp1B/+GDLEsIxir6WNhZgw8OgjVF1u/wCAad6qHkmN02Bwenr+aay6uKfCuOEsTRSvZ7v80yAU+h0wL3zJ/KMkRsE3QP3CWxcLQxInt+YaBjR+Q0IUjDXKm3u6ZomixZe5F5pwWr36ErV6Y0=,iv:e/iiyXQiCh8C2w/bc8mr/Psv+ehmqEMqEC1/bbGFHpY=,tag:NSDo2HISIBJhYvsqrU0mSA==,type:str]
+    pgp:
+      - created_at: '2020-02-20T10:44:32Z'
+        enc: |-
+            -----BEGIN PGP MESSAGE-----
+
+            wcBMAyUpShfNkFB/AQgALJTUwdx6rAPckJ+reP5TEq+lXzHI1Zi7aHYOqZQBnA2s
+            z8h1gRce/fn7RPkmdsjsdSYmxGGKqwDXxUYsbN1aWXk6mb4Juktdvjl/GndF6PkU
+            TiN/l1GM6upgS+GPxA01NKsGkVmEtKR5NhsNEnE6OzY29+PFLsBX2vO1Zfg7kzBz
+            cDl6PT8fbFTEaFeyuYl9IslIV8yYsj1oHL3CF76RjCP6b18NSOHM23ytlH+KVaBV
+            ntoSVkTyWDx5o9iEHBEWSEGNpaCWWiEgkDEkA1VqMHdUlsW+IjZ8ggg5NJbcVtrG
+            YkN8rlGsNEzx+g4O4b1160A2K6AdTBcoGHwHD3u3XdLgAeTqT1ekE2N3yNT6w4sm
+            6uET4eTS4Cvg1OFCgOC34uUzlY3gbuVy20h8RNyQoAfhSN4DD2MexKqcMMCVCtn0
+            OhRMTP2jjOCe5Ex3/p3awcVxwx7qeJ26Vnfiwtg6ueFI5AA=
+            =tcnq
+            -----END PGP MESSAGE-----
+        fp: FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4
+    unencrypted_suffix: _unencrypted
+    version: 3.4.0
--- a/tests/integration/targets/lookup_sops/files/wrong.yaml
+++ b/tests/integration/targets/lookup_sops/files/wrong.yaml
@ -0,0 +1,2 @@
+---
+this-is-not: a sops file
--- a/tests/integration/targets/lookup_sops/meta/main.yml
+++ b/tests/integration/targets/lookup_sops/meta/main.yml
@ -0,0 +1,3 @@
+---
+dependencies:
+  - setup_sops
--- a/tests/integration/targets/lookup_sops/tasks/main.yml
+++ b/tests/integration/targets/lookup_sops/tasks/main.yml
@ -0,0 +1,36 @@
+---
+- when: sops_installed
+  block:
+
+    - name: Test lookup with missing file
+      set_fact:
+        sops_file_does_not_exists: "{{ lookup('community.general.sops', 'file-does-not-exists.sops.yml') }}"
+      ignore_errors: yes
+      register: sops_lookup_missing_file
+
+    - assert:
+        that:
+          - "sops_lookup_missing_file is failed"
+          - "'could not locate file in lookup: file-does-not-exists.sops.yml' in sops_lookup_missing_file.msg"
+
+    - name: Test lookup of non-sops file
+      set_fact:
+        sops_wrong_file: "{{ lookup('community.general.sops', 'wrong.yaml') }}"
+      ignore_errors: yes
+      register: sops_lookup_wrong_file
+
+    - assert:
+        that:
+          - "sops_lookup_wrong_file is failed"
+          - "'sops metadata not found' in sops_lookup_wrong_file.msg"
+
+    - name: Test simple lookup
+      set_fact:
+        sops_success: "{{ lookup('community.general.sops', 'simple.sops.yaml') }}"
+      ignore_errors: yes
+      register: sops_lookup_simple
+
+    - assert:
+        that:
+          - "sops_lookup_simple is success"
+          - "sops_success == 'foo: bar'"
--- a/tests/integration/targets/setup_sops/meta/main.yml
+++ b/tests/integration/targets/setup_sops/meta/main.yml
@ -0,0 +1 @@
+---
--- a/tests/integration/targets/setup_sops/tasks/main.yml
+++ b/tests/integration/targets/setup_sops/tasks/main.yml
@ -0,0 +1,7 @@
+---
+
+- set_fact:
+    sops_installed: no
+
+- include: ubuntu.yml
+  when: ansible_distribution == 'Ubuntu'
--- a/tests/integration/targets/setup_sops/tasks/ubuntu.yml
+++ b/tests/integration/targets/setup_sops/tasks/ubuntu.yml
@ -0,0 +1,20 @@
+---
+
+- name: install sops release
+  apt:
+    deb: https://github.com/mozilla/sops/releases/download/v3.5.0/sops_3.5.0_amd64.deb
+
+- name: install gnupg2
+  apt:
+    package: gnupg2
+
+- name: download sops test GPG key
+  get_url:
+    url: https://raw.githubusercontent.com/mozilla/sops/master/pgp/sops_functional_tests_key.asc
+    dest: /tmp/sops_functional_tests_key.asc
+
+- name: import sops test GPG key
+  command: gpg --import /tmp/sops_functional_tests_key.asc
+
+- set_fact:
+    sops_installed: yes