ansible-collections/community.general
1
0
Fork 0
mirror of https://github.com/ansible-collections/community.general.git synced 2025-07-25 14:20:22 -07:00
Code Issues Projects Releases Packages Wiki Activity Actions 6

Reflect new process for network integration tests (#35577)

Browse source 
With `ansible-test network-integration` we don't need the top level yaml files
except NXOS and Dell (for now)
This commit is contained in:
Alicia Cozine 2018-01-31 14:56:28 -06:00 • committed by GitHub
commit adf76b7d3f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
11 changed files with 36 additions and 934 deletions
Download patch file Download diff file Expand all files Collapse all files

101
docs/docsite/rst/dev_guide/testing_integration.rst
View file

@ -184,93 +184,50 @@ privileges.
Network Tests
=============
This page details the specifics around testing Ansible Networking modules.
.. important:: Network testing requirements for Ansible 2.4
Starting with Ansible 2.4, all network modules MUST include corresponding unit tests to defend functionality.
The unit tests must be added in the same PR that includes the new network module, or extends functionality.
Integration tests, although not required, are a welcome addition.
How to do this is explained in the rest of this document.
Network integration tests can be ran by doing::
cd test/integration
ANSIBLE_ROLES_PATH=targets ansible-playbook network-all.yaml
.. note::
* To run the network tests you will need a number of test machines and suitably configured inventory file. A sample is included in ``test/integration/inventory.network``
* As with the rest of the integration tests, they can be found grouped by module in ``test/integration/targets/MODULENAME/``
To filter a set of test cases set ``limit_to`` to the name of the group, generally this is the name of the module::
ANSIBLE_ROLES_PATH=targets ansible-playbook -i inventory.network network-all.yaml -e "limit_to=eos_command"
To filter a singular test case set the tags options to eapi or cli, set limit_to to the test group,
and test_cases to the name of the test::
ANSIBLE_ROLES_PATH=targets ansible-playbook -i inventory.network network-all.yaml --tags="cli" -e "limit_to=eos_command test_case=notequal"
Starting with Ansible 2.4, all network modules MUST include unit tests that cover all functionality. You must add unit tests for each new network module and for each added feature. Please submit the unit tests and the code in a single PR. Integration tests are also strongly encouraged.
Writing network integration tests
---------------------------------
Test cases are added to roles based on the module being testing. Test cases
should include both cli and API test cases. Cli test cases should be
added to ``test/integration/targets/modulename/tests/cli`` and API tests should be added to
``test/integration/targets/modulename/tests/eapi``, or ``nxapi``.
In addition to positive testing, negative tests are required to ensure user friendly warnings & errors are generated, rather than backtraces, for example:
.. code-block: yaml
- name: test invalid subset (foobar)
eos_facts:
provider: "{{ cli }}"
gather_subset:
- "foobar"
register: result
ignore_errors: true
- assert:
that:
# Failures shouldn't return changes
- "result.changed == false"
# It's a failure
- "result.failed == true"
# Sensible Failure message
- "'Subset must be one of' in result.msg"
For guidance on writing network test see the `adding tests for Network modules guide <https://github.com/ansible/community/blob/master/group-network/network_test.rst>`_.
Conventions
```````````
Running network integration tests locally
-----------------------------------------
- Each test case should generally follow the pattern:
Ansible uses Shippable to run an integration test suite on every PR, including new tests introduced by that PR. To find and fix problems in network modules, run the network integration test locally before you submit a PR.
setup —> test —> assert —> test again (idempotent) —> assert —> teardown (if needed) -> done
To run the network integration tests, use a command in the form::
This keeps test playbooks from becoming monolithic and difficult to
troubleshoot.
ansible-test network-integration --inventory /path/to/inventory tests_to_run
- Include a name for each task that is not an assertion. (It's OK to add names
to assertions too. But to make it easy to identify the broken task within a failed
test, at least provide a helpful name for each task.)
First, define a network inventory file::
- Files containing test cases must end in `.yaml`
cd test/integration
cp inventory.network.template inventory.networking
${EDITOR:-vi} inventory.networking
# Add in machines for the platform(s) you wish to test
To run all Network tests for a particular platform::
ansible-test network-integration --inventory /path/to/ansible/test/integration/inventory.networking vyos_.*
This example will run against all vyos modules. Note that ``vyos_.*`` is a regex match, not a bash wildcard - include the `.` if you modify this example.
Adding a new Network Platform
`````````````````````````````
To run integration tests for a specific module::
ansible-test network-integration --inventory /path/to/ansible/test/integration/inventory.networking vyos_vlan
For more options::
ansible-test network-integration --help
If you need additional help or feedback, reach out in ``#ansible-network`` on Freenode.
A top level playbook is required such as ``ansible/test/integration/eos.yaml`` which needs to be references by ``ansible/test/integration/network-all.yaml``
Where to find out more
======================
If you'd like to know more about the plans for improving testing Ansible then why not join the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
If you'd like to know more about the plans for improving testing Ansible, join the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.