New module: iso_customize (#5190)

* add ansible module iso_customize.py

* rerun CI testing due to "Failed to send request to https://api.github.com/repos/ansible/ansible/issues/23642: HTTP Error 403: rate limit exceeded"

* Rerun CI testing due to "Failed to send request to https://api.github....."

* rerun CI testing due to failure "Unknown error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/v2/collections/netbox/netbox/versions/3.1.0/': The read operation timed out"

* change document part as  felixfontein's careful review

* modify test file as russoz's comments

* modify comment part of module

* add comment for the example

* add more tests: check the files are deleted / added in customized ISO

* fix it: failed to run ansible.posix.mount in ubuntu

* fix it:  ansible.posix.mount is not working well in some OS.

* change DOCUMENTATION part

* change files according to the comment from code review

* fix issue: E231: missing whitespace after ':'

* modify the description of Document

* modify code for code review

* delete extra blank line in yml file

* Try to fix CI testing issue: "Caught \"'foo' is undefined. 'foo' is undefined\" while evaluating 'b' with item == {'a': 1}"

* delete extra blank line in the end of file

* change code as the comment from code review

* change code from code review

* change type: str to type: path

* change type:str to type:path

* delete unused variable

* fix CI testing error: return-syntax-error: RETURN.dest_iso.type: not a valid value for dictionary value @ data['dest_iso']['type']. Got 'path'

* add testcase: test add files / delete files separately

* add more testcases: test if we can catch exception from error input of users

* change code from code review

* fix issue: E231: missing whitespace after ','

* change code from code review

* add notes to document

* modify notes in document part

* /rebuild_failed

/rebuild_failed

* Try to support running testcases not only in MAC but also in other OS.

* modify document

* change mount to ansible.posix.mount

* skip the test platform which report "Error mounting"

* fix mount failed: Operation not permitted

* change code from code review

* change document from code review

* fix CI testing issue in some platforms

* Update plugins/modules/files/iso_customize.py

* change code from code review

1) change testcase
2) try to fix "mount: not permitted"

* modify aliases file

* change document and rerun CI testing

* add skip/docker as suggested

* add debug task

* fix issue in redhat 7.9: occurred while running the lookup plugin 'file'. ..could not locate file in lookup..

* change code from the code review

* modify function "iso_rr_check_file_exist" to "iso_check_file_exists" to make it works in all types of ISO

1. modify function "iso_rr_check_file_exist" to "iso_check_file_exists" to make it works in all types of ISO

2. run main.yml  with newer python 3.10.6
ansible [core 2.13.4]
  config file = None
  configured module search path = ['/Users/zouy/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/Cellar/ansible/6.4.0/libexec/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/zouy/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.10.6 (main, Aug 30 2022, 05:12:36) [Clang 13.1.6 (clang-1316.0.21.2.5)]
  jinja version = 3.1.2
  libyaml = True

* delete blank

* simply the code as suggested.

* Two small docs updates.

Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>

tests/integration/targets/iso_customize/aliases

@@ -0,0 +1,14 @@
+# Copyright (c) 2022, Ansible Project
+# Copyright (c) 2022, VMware, Inc. All Rights Reserved.
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+shippable/posix/group1
+destructive
+skip/aix
+skip/freebsd
+skip/alpine
+skip/python2.6
+skip/docker
+needs/root
+

tests/integration/targets/iso_customize/meta/main.yml

@@ -0,0 +1,7 @@
+---
+# Copyright (c) Ansible Project
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+dependencies:
+  - setup_remote_tmp_dir

tests/integration/targets/iso_customize/tasks/iso_customize.yml

@@ -0,0 +1,75 @@
+# Copyright (c) 2022, Ansible Project
+# Copyright (c) 2022, VMware, Inc. All Rights Reserved.
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+- name: Add a line to the file test02.cfg and make sure it succeed
+  ansible.builtin.lineinfile:
+    path: "{{ test_dir }}/test02.cfg"
+    regexp: "^test"
+    line: "test"
+
+- name: "Customize ISO file: add file, delete file and change file"
+  community.general.iso_customize:
+    src_iso: "{{ test_dir }}/test.iso"
+    dest_iso: "{{ test_dir }}/{{ dest_iso_name }}"
+    delete_files:
+      - "/test01.cfg"
+    add_files:
+      - src_file: "{{ test_dir }}/test01.cfg"
+        dest_file: "/preseed/ubuntu.seed"
+      - src_file: "{{ test_dir }}/test02.cfg"
+        dest_file: "/test02.cfg"
+
+- include_tasks: iso_mount.yml
+  vars:
+    iso_name: "{{ dest_iso_name }}"
+
+- debug: var=mount_root_dir
+
+- name: Check the file test01.cfg is deleted
+  stat:
+    path: "{{ mount_root_dir }}/test01.cfg"
+  register: check_file
+
+- assert:
+    that:
+    - check_file.stat.exists == False
+
+- name: Check the file /preseed/ubuntu.seed is added
+  stat:
+    path: "{{ mount_root_dir }}/preseed/ubuntu.seed"
+  register: check_file
+
+- assert:
+    that:
+    - check_file.stat.exists == True
+
+- block:
+    - name: Get the content of file test02.cfg
+      command: "cat {{ mount_root_dir }}/test02.cfg"
+      register: get_file_content
+
+    - set_fact:
+        file_contents: "{{ get_file_content.stdout }}"
+  when: ansible_distribution == 'RedHat' and ansible_distribution_version is version('7.9', '==')
+
+- name: Get the content of file test02.cfg
+  set_fact:
+    file_contents: "{{ lookup('file', mount_root_dir + '/test02.cfg') }}"
+  when: not (ansible_distribution == 'RedHat' and ansible_distribution_version is version('7.9', '=='))
+
+- fail: msg="Failed to replace the file test02.cfg"
+  when: file_contents != "test"
+
+- name: Umount ISO
+  mount:
+    path: "{{ mount_root_dir }}"
+    fstab: "{{ test_dir }}/temp.fstab"
+    state: unmounted
+
+- name: Delete line of file test02.cfg
+  ansible.builtin.lineinfile:
+    path: "{{ test_dir }}/test02.cfg"
+    regexp: "test" 
+    state: absent

tests/integration/targets/iso_customize/tasks/iso_customize_add_files.yml

@@ -0,0 +1,34 @@
+# Copyright (c) 2022, Ansible Project
+# Copyright (c) 2022, VMware, Inc. All Rights Reserved.
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+- name: "Customize ISO file: add file"
+  community.general.iso_customize:
+    src_iso: "{{ test_dir }}/test1.iso"
+    dest_iso: "{{ test_dir }}/{{ dest_iso_name }}"
+    add_files:
+      - src_file: "{{ test_dir }}/test01.cfg"
+        dest_file: "preseed/ubuntu.seed"
+
+
+- include_tasks: iso_mount.yml
+  vars:
+    iso_name: "{{ dest_iso_name }}"
+
+- debug: var=mount_root_dir
+
+- name: Check the file /preseed/ubuntu.seed is added
+  stat:
+    path: "{{ mount_root_dir }}/preseed/ubuntu.seed"
+  register: check_file
+
+- assert:
+    that:
+    - check_file.stat.exists == True
+
+- name: Umount ISO
+  mount:
+    path: "{{ mount_root_dir }}"
+    fstab: "{{ test_dir }}/temp.fstab"
+    state: unmounted

tests/integration/targets/iso_customize/tasks/iso_customize_delete_files.yml

@@ -0,0 +1,34 @@
+# Copyright (c) 2022, Ansible Project
+# Copyright (c) 2022, VMware, Inc. All Rights Reserved.
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+- name: "Customize ISO file: delete file"
+  community.general.iso_customize:
+    src_iso: "{{ test_dir }}/test1.iso"
+    dest_iso: "{{ test_dir }}/{{ dest_iso_name }}"
+    delete_files:
+      - "test01.cfg"
+
+- debug: var=ansible_distribution
+
+- include_tasks: iso_mount.yml
+  vars:
+    iso_name: "{{ dest_iso_name }}"
+
+- debug: var=mount_root_dir
+
+- name: Check the file test01.cfg is deleted
+  stat:
+    path: "{{ mount_root_dir }}/test01.cfg"
+  register: check_file
+
+- assert:
+    that:
+    - check_file.stat.exists == False
+
+- name: Umount ISO
+  mount:
+    path: "{{ mount_root_dir }}"
+    fstab: "{{ test_dir }}/temp.fstab"
+    state: unmounted

tests/integration/targets/iso_customize/tasks/iso_customize_exception.yml

@@ -0,0 +1,81 @@
+# Copyright (c) 2022, Ansible Project
+# Copyright (c) 2022, VMware, Inc. All Rights Reserved.
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+- name: "Testcase: local resource ISO does not exists"
+  community.general.iso_customize:
+    src_iso: "{{ test_dir }}/test11.iso"
+    dest_iso: "{{ test_dir }}/{{ dest_iso_name }}"
+  register: customized_result
+  failed_when: customized_result.msg.find('does not exist') == -1
+
+- name: "Testcase:: dest dir does not exists"
+  community.general.iso_customize:
+    src_iso: "{{ test_dir }}/test1.iso"
+    dest_iso: "/aaa/{{ dest_iso_name }}"
+  register: customized_result
+  failed_when: customized_result.msg.find('does not exist') == -1
+
+# Test: Get MODULE FAILURE when no add files data and no delete files data
+- name: "Testcase:: no add files data and no delete files data"
+  community.general.iso_customize:
+    src_iso: "{{ test_dir }}/test1.iso"
+    dest_iso: "{{ test_dir }}/iso_customize_nodata.iso"
+    delete_files:
+    add_files:
+  register: customized_result
+  failed_when: customized_result.msg.find("MODULE FAILURE") == -1
+
+# Test: nothing is changed when no options "add files" and "delete files"
+- block:
+    - name: "Testcase: no options 'add files' and 'delete files'"
+      community.general.iso_customize:
+        src_iso: "{{ test_dir }}/test1.iso"
+        dest_iso: "{{ test_dir }}/iso_customize_nochanged.iso"
+
+    - name: Get stats of a file test1.iso
+      ansible.builtin.stat:
+        path: "{{ test_dir }}/test1.iso"
+      register: iso_orginal
+
+    - name: Get stats of a file iso_customize_nochanged.iso
+      ansible.builtin.stat:
+        path: "{{ test_dir }}/iso_customize_nochanged.iso"
+      register: iso_customized
+
+    - name: compare size
+      fail: msg="Check we have nothing changed for customized ISO"
+      when: iso_orginal.stat.size != iso_customized.stat.size
+
+- name: "Testcase: delete the non-existing file in ISO"
+  community.general.iso_customize:
+    src_iso: "{{ test_dir }}/test1.iso"
+    dest_iso: "{{ test_dir }}/{{ dest_iso_name }}"
+    delete_files:
+      - "/test03.cfg"
+  register: customized_result
+  failed_when: customized_result.msg.find("does not exist") == -1
+
+# Test: failed when local src file does not exists
+- name: "Testcase: local src file does not exists"
+  community.general.iso_customize:
+    src_iso: "{{ test_dir }}/test.iso"
+    dest_iso: "{{ test_dir }}/{{ dest_iso_name }}"
+    add_files:
+      - src_file: "{{ test_dir }}/test03.cfg"
+        dest_file: "/preseed/ubuntu.seed"
+  register: customized_result
+  failed_when: customized_result.msg.find("does not exist") == -1
+
+# Test: filenames with whitespaces
+# We report error: the user should be reponsible for the it
+- name: "Testcase: filenames with whitespaces"
+  community.general.iso_customize:
+    src_iso: "{{ test_dir }}/test.iso"
+    dest_iso: "{{ test_dir }}/{{ dest_iso_name }}"
+    add_files:
+      - src_file: " {{ test_dir }}/test01.cfg "
+        dest_file: "/preseed/ubuntu.seed"
+  register: customized_result
+  failed_when: customized_result.msg.find("does not exist") == -1

tests/integration/targets/iso_customize/tasks/iso_mount.yml

@@ -0,0 +1,39 @@
+# Copyright (c) 2022, Ansible Project
+# Copyright (c) 2022, VMware, Inc. All Rights Reserved.
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+- debug: var=ansible_distribution
+
+- block:
+    - name: "Mount customized ISO on MAC"
+      command: "hdiutil attach {{ test_dir }}/{{ iso_name }} -mountroot {{ test_dir }}/iso_mount"
+
+    # For MAC, we have different root directory for different type of ISO
+    - set_fact:
+        mount_root_dir: "{{ test_dir }}/iso_mount/disk_image"
+
+    - set_fact:
+        mount_root_dir: "{{ test_dir }}/iso_mount/AUTOINSTALL"
+      when: iso_name.find('joliet') != -1
+
+    - set_fact:
+        mount_root_dir: "{{ test_dir }}/iso_mount/CDROM"
+      when: iso_name.find('udf') != -1
+  when: ansible_distribution == "MacOSX"
+
+- block:
+    - name: "Mount {{ iso_name }} to {{ test_dir }}/iso_mount on localhost"
+      become: true
+      ansible.posix.mount:
+        path: "{{ test_dir }}/iso_mount"
+        src: "{{ test_dir }}/{{ iso_name }}"
+        opts: "ro,noauto"
+        fstab: "{{ test_dir }}/temp.fstab"
+        fstype: "iso9660"
+        state: mounted
+
+    - set_fact:
+        mount_root_dir: "{{ test_dir }}/iso_mount"
+  when: 
+    - ansible_distribution != "MacOSX"

tests/integration/targets/iso_customize/tasks/main.yml

@@ -0,0 +1,94 @@
+####################################################################
+# WARNING: These are designed specifically for Ansible tests       #
+# and should not be used as examples of how to write Ansible roles #
+####################################################################
+
+# Copyright (c) 2022, Ansible Project
+# Copyright (c) 2022, VMware, Inc. All Rights Reserved.
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+- name: Skip some platforms which does not support ansible.posix.mount
+  meta: end_play
+  when: ansible_distribution in ['Alpine']
+
+- set_fact:
+    test_dir: '{{ remote_tmp_dir }}/test_iso_customize'
+
+- include_tasks: prepare.yml
+
+- name: Create iso file with a specified file and directory
+  community.general.iso_create:
+    src_files:
+      - "{{ test_dir }}/test01.cfg"
+      - "{{ test_dir }}/test02.cfg"
+    dest_iso: "{{ test_dir }}/test.iso"
+    interchange_level: 3
+
+- include_tasks: iso_customize.yml  
+  vars:
+    dest_iso_name: "iso_customize.iso"
+
+- name: Create an ISO file with Rock Ridge extension
+  community.general.iso_create:
+    src_files:
+      - "{{ test_dir }}/test01.cfg"
+      - "{{ test_dir }}/test02.cfg"
+    dest_iso: "{{ test_dir }}/test.iso"
+    rock_ridge: "1.09"
+
+- include_tasks: iso_customize.yml
+  vars:
+    dest_iso_name: "iso_customize_rr.iso"
+
+- name: Create an ISO file with Joliet support
+  community.general.iso_create:
+    src_files:
+      - "{{ test_dir }}/test01.cfg"
+      - "{{ test_dir }}/test02.cfg"
+    dest_iso: "{{ test_dir }}/test.iso"
+    interchange_level: 3
+    joliet: 3
+    vol_ident: AUTOINSTALL
+
+- include_tasks: iso_customize.yml 
+  vars:
+    dest_iso_name: "iso_customize_joliet.iso"
+
+- name: Create iso file with UDF enabled
+  community.general.iso_create:
+    src_files:
+      - "{{ test_dir }}/test01.cfg"
+      - "{{ test_dir }}/test02.cfg"
+    dest_iso: "{{ test_dir }}/test.iso"
+    udf: True
+
+- include_tasks: iso_customize.yml
+  vars:
+    dest_iso_name: "iso_customize_udf.iso"
+
+# Create initial iso for customzing with only option add_files/delete_files
+- name: Create iso file with a specified file and directory
+  community.general.iso_create:
+    src_files:
+      - "{{ test_dir }}/test01.cfg"
+    dest_iso: "{{ test_dir }}/test1.iso"
+    interchange_level: 3
+
+- include_tasks: iso_customize_add_files.yml
+  vars:
+    dest_iso_name: "iso_customize_add.iso"
+
+- include_tasks: iso_customize_delete_files.yml
+  vars:
+    dest_iso_name: "iso_customize_delete.iso"
+
+# Test: misc exception
+- include_tasks: iso_customize_exception.yml
+  vars:
+    dest_iso_name: "iso_customize_exception.iso"
+
+- name: Delete testing sub-directory
+  ansible.builtin.file:
+    path: '{{ test_dir }}'
+    state: absent

tests/integration/targets/iso_customize/tasks/prepare.yml

@@ -0,0 +1,39 @@
+# Copyright (c) 2022, Ansible Project
+# Copyright (c) 2022, VMware, Inc. All Rights Reserved.
+# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
+# SPDX-License-Identifier: GPL-3.0-or-later
+
+- name: install pycdlib
+  ansible.builtin.pip:
+    name: pycdlib
+
+- name: Make sure the previous testing sub-directory is deleted
+  ansible.builtin.file:
+    path: '{{ test_dir }}'
+    state: absent
+
+- name: Create our testing sub-directory
+  ansible.builtin.file:
+    path: '{{ test_dir }}'
+    state: directory
+
+- name: Create sub directory to mount customized ISO
+  ansible.builtin.file:
+    path: '{{ test_dir }}/iso_mount'
+    state: directory
+
+- name: Create temporary file test01.cfg for testing
+  ansible.builtin.file:
+    path: "{{ test_dir }}/test01.cfg"
+    state: touch
+
+- name: Add a line to the file test01.cfg and make sure it succeed
+  ansible.builtin.lineinfile:
+    path: "{{ test_dir }}/test01.cfg"
+    regexp: "^aaa"
+    line: "aaa"
+
+- name: Create temporary file test02.cfg for testing
+  ansible.builtin.file:
+    path: "{{ test_dir }}/test02.cfg"
+    state: touch