more updates to plugin/config generation (#30787)

* fixed module generation

added missing lookup page
point to plugins when plugins
made modules singular
add display for verbose an debug messages
nicer templating, changed generation order for ref
corrected links
moved most of lookup docs to plugin section

* Copy edits
* Fixed typos
* Clarified wording

docs/docsite/rst/plugins/action.rst

@@ -6,11 +6,20 @@ They usually execute automatically in the background doing prerequisite work bef
 
 The 'normal' action plugin is used for modules that do not already have an action plugin.
 
+.. _enabling_action:
+
 Enabling Action Plugins
 +++++++++++++++++++++++
 
 You can enable a custom action plugin by either dropping it into the ``action_plugins`` directory adjacent to your play, inside a role, or by putting it in one of the action plugin directory sources configured in :doc:`ansible.cfg <../config>`.
 
+.. _using_action:
+
+Using Action Plugins
++++++++++++++++++++++
+
+Action plugin are executed by default when an associated module is used; no action is required.
+
 .. seealso::
 
    :doc:`cache`

docs/docsite/rst/plugins/cache.rst

@@ -10,11 +10,12 @@ without the performance hit of retrieving them from source.
 The default cache plugin is the :doc:`memory <cache/memory>` plugin, which only caches the data for the current execution of Ansible. Other plugins with persistent storage are available to allow caching the data across runs.
 
 
+.. _enabling_cache:
+
 Enabling Cache Plugins
 ++++++++++++++++++++++
 
 Only one cache plugin can be active at a time.
-
 You can enable a cache plugin in the Ansible configuration, either via environment variable:
 
 .. code-block:: shell
@@ -31,10 +32,23 @@ or in the ``ansible.cfg`` file:
 You will also need to configure other settings specific to each plugin. Consult the individual plugin documentation
 or the Ansible :doc:`configuration <../config>` for more details.
 
+A custom cache plugin is enabled by dropping it into a ``cache_plugins`` directory adjacent to your play, inside a role, or by putting it in one of the directory sources configured in :doc:`ansible.cfg <../config>`.
+
+
+.. _using_cache:
+
+Using Cache Plugins
++++++++++++++++++++
+
+Cache plugins are used autoamtically once they are enabled.
+
+
+.. _cache_plugin_list:
+
 Plugin List
 +++++++++++
 
-You can use ``ansible-doc -t cache -l`` to see the list of available plugins. 
+You can use ``ansible-doc -t cache -l`` to see the list of available plugins.
 Use ``ansible-doc -t cache <plugin name>`` to see specific documentation and examples.
 
 .. toctree:: :maxdepth: 1

docs/docsite/rst/plugins/callback.rst

@@ -70,6 +70,8 @@ You can also set this as an environment variable:
     export ANSIBLE_LOAD_CALLBACK_PLUGINS=1
 
 
+.. _callback_plugin_list:
+
 Plugin List
 +++++++++++
 

docs/docsite/rst/plugins/connection.rst

@@ -11,7 +11,6 @@ By default, Ansible ships with several plugins. The most commonly used are the '
 The basics of these connection types are covered in the :doc:`../intro_getting_started` section.  
 
 
-.. contents:: Topics
 
 .. _ssh_plugins:
 
@@ -20,28 +19,7 @@ ssh Plugins
 
 Because ssh is the default protocol used in system administration and the protocol most used in Ansible, ssh options are included in the command line tools. See :doc:`../ansible-playbook` for more details.
 
-
-.. _using_connection_plugins:
-
-Using Connection Plugins
-++++++++++++++++++++++++
-
-The transport can be changed via :doc:`configuration <../config>`, in the command line (``-c``, ``--connection``), as a keyword (:ref:`connection`)
-in your play or by setting the a connection variable (:ref:`ansible_connection`), most often, in your inventory.
-For example, for windows machines you might want o use the :doc:`winrm <connection/winrm>` plugin instead.
-
-Most connection plugins can operate with a minimum configuration. By default they use the :ref:`inventory_hostname` and defaults to find the target host.
-
-Plugins are self-documenting. Each plugin should document its configuration options. The following are connection variables common to most connection plugins:
-
-:ref:ansible_host
-    The name of the host to connect to, if different from the :ref:`inventory_hostname`.
-:ref:ansible_port
-    The ssh port number. For :doc:`ssh <connection/ssh>` and :doc:`paramiko <connection/paramiko>` the default value is 22.
-:ref:ansible_user
-    The default user name to use for log in. Most plugins default to the current user running Ansible.
-
-Each plugin might also have a specific version of a variable that overrides the general version. For example, :ref:`ansible_ssh_host` for the :doc:`ssh <connection/ssh>` plugin.
+.. _enabling_connection:
 
 Enabling Connection Plugins
 +++++++++++++++++++++++++++
@@ -49,6 +27,31 @@ Enabling Connection Plugins
 You can extend Ansible to support other transports (such as SNMP or message bus) by dropping a custom plugin
 into the ``connection_plugins`` directory.
 
+
+.. _using_connection:
+
+Using Connection Plugins
+++++++++++++++++++++++++
+
+The transport can be changed via :doc:`configuration <../config>`, in the command line (``-c``, ``--connection``), as a keyword (:ref:`connection`)
+in your play, or by setting the a connection variable (:ref:`ansible_connection`), most often in your inventory.
+For example, for Windows machines you might want to use the :doc:`winrm <connection/winrm>` plugin.
+
+Most connection plugins can operate with a minimum configuration. By default they use the :ref:`inventory_hostname` and defaults to find the target host.
+
+Plugins are self-documenting. Each plugin should document its configuration options. The following are connection variables common to most connection plugins:
+
+:ref:`ansible_host`
+    The name of the host to connect to, if different from the :ref:`inventory_hostname`.
+:ref:`ansible_port`
+    The ssh port number, for :doc:`ssh <connection/ssh>` and :doc:`paramiko <connection/paramiko>` it defaults to 22.
+:ref:`ansible_user`
+    The default user name to use for log in. Most plugins default to the 'current user running Ansible'.
+
+Each plugin might also have a specific version of a variable that overrides the general version. For example, :ref:`ansible_ssh_host` for the :doc:`ssh <connection/ssh>` plugin.
+
+.. _connection_plugin_list:
+
 Plugin List
 +++++++++++
 

docs/docsite/rst/plugins/inventory.rst

@@ -6,7 +6,8 @@ Inventory Plugins
 
 Inventory plugins allow users to point at data sources to compile the inventory of hosts that Ansible uses to target tasks, either via the ``-i /path/to/file`` and/or ``-i 'host1, host2`` command line parameters or from other configuration sources.
 
-.. _enabling_inventory_plugins:
+
+.. _enabling_inventory:
 
 Enabling Inventory Plugins
 ++++++++++++++++++++++++++
@@ -28,6 +29,18 @@ This list also establishes the order in which each plugin tries to parse an inve
    enable_plugins = advanced_host_list, constructed, yaml
 
 
+.. _using_inventory:
+
+Using Inventory Plugins
++++++++++++++++++++++++
+
+The only requirement for using an inventory plugin after it is enabled is to provide an inventory source to parse.
+Ansible will try to use the list of enabled inventory plugins, in order, against each inventory source provided.
+Once an inventory plugin succeeds at parsing a source, the any remaining inventory plugins will be skipped for that source.
+
+
+.. _inventory_plugin_list:
+
 Plugin List
 +++++++++++
 

docs/docsite/rst/plugins/lookup.rst

@@ -0,0 +1,99 @@
+.. contents:: Topics
+
+
+Lookup Plugins
+--------------
+
+Lookup plugins allow Ansible to access data from outside sources.
+This can include reading the filesystem in addition to contacting external datastores and services.
+Like all templating, these plugins are evaluated on the Ansible control machine, not on the target/remote.
+
+The data returned by a lookup plugin is made available using the standard templating system in Ansible,
+and are typically used to load variables or templates with information from those systems.
+
+Lookups are an Ansible-specific extension to the Jinja2 templating language.
+
+.. note::
+   - Lookups are executed with a working directory relative to the role or play,
+     as opposed to local tasks, which are executed relative the executed script.
+   - Since Ansible version 1.9, you can pass wantlist=True to lookups to use in Jinja2 template "for" loops.
+   - Lookup plugins are an advanced feature; to best leverage them you should have a good working knowledge of how to use Ansible plays.  
+
+.. warning::
+   - Some lookups pass arguments to a shell. When using variables from a remote/untrusted source, use the `|quote` filter to ensure safe usage.
+
+
+.. _enabling_lookup:
+
+Enabling Lookup Plugins
++++++++++++++++++++++++
+
+You can activate a custom lookup by either dropping it into a ``lookup_plugins`` directory adjacent to your play, inside a role, or by putting it in one of the lookup directory sources configured in :doc:`ansible.cfg <../config>`.
+
+
+.. _using_lookup:
+
+Using Lookup Plugins
+++++++++++++++++++++
+
+Lookup plugins can be used anywhere you can use templating in Ansible: in a play, in variables file, or in a Jinja2 template for the :doc:`template <../template_module>` module.
+
+.. code-block:: yaml
+
+  vars:
+    file_contents: "{{lookup('file', 'path/to/file.txt')}}"
+
+Lookups are an integral part of loops. Wherever you see ``with_``, the part after the underscore is the name of a lookup.
+This is also the reason most lookups output lists and take lists as input; for example, ``with_items`` uses the :doc:`items <lookup/items>` lookup:
+
+.. code-block:: yaml
+
+  tasks:
+    - name: count to 3
+      debug: msg={{item}}
+      with_items: [1, 2, 3]
+
+You can combine lookups with :doc:`../playbooks_filters`, :doc:`../playbooks_tests` and even each other to do some complex data generation and maniplulation. For example:
+
+.. code-block:: yaml
+
+  tasks:
+    - name: valid but useless and over complicated chained lookups and filters
+      debug: msg="find the answer here:\n{{ lookup('url', 'http://google.com/search/?q=' + item|urlencode)|join(' ') }}"
+      with_nested:
+        - "{{lookup('consul_kv', 'bcs/' + lookup('file', '/the/question') + ', host=localhost, port=2000')|shuffle}}"
+        - "{{lookup('sequence', 'end=42 start=2 step=2')|map('log', 4)|list)}}"
+        - ['a', 'c', 'd', 'c']
+
+
+.. _lookup_plugins_list:
+
+Plugin List
++++++++++++
+
+You can use ``ansible-doc -t lookup -l`` to see the list of available plugins. Use ``ansible-doc -t lookup <plugin name>`` to see specific documents and examples.
+
+
+.. toctree:: :maxdepth: 1
+    :glob:
+
+    lookup/*
+
+.. seealso::
+
+   :doc:`../playbooks`
+       An introduction to playbooks
+   :doc:`inventory`
+       Ansible inventory plugins
+   :doc:`callback`
+       Ansible callback plugins
+   :doc:`../playbooks_filters`
+       Jinja2 filter plugins
+   :doc:`../playbooks_tests`
+       Jinja2 test plugins
+   :doc:`../playbooks_lookups`
+       Jinja2 lookup plugins
+   `User Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Have a question?  Stop by the google group!
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/plugins/shell.rst

@@ -1,15 +1,27 @@
 Shell Plugins
 -------------
 
-Shell plugins work transparently to ensure that the basic commands Ansible runs are properly formated to work with the target machine.
+Shell plugins work transparently to ensure that the basic commands Ansible runs are properly formatted to work with the target machine.
+
+.. _enabling_shell:
 
 Enabling Shell Plugins
 ++++++++++++++++++++++
 
-.. warning:: These plugins should not be reconfigured unless you have a restricted or exotic setup
-             in which the default ``/bin/sh`` is not a POSIX compatible shell or not availble for execution. 
+You can add a custom shell plugin by dropping it into a ``shell_plugins`` directory adjacent to your play, inside a role,
+or by putting it in one of the shell plugin directory sources configured in :doc:`ansible.cfg <../config>`.
 
-In addition to modifying the default configuration settings in :doc:`../config`, you can use a 'connection variable' :ref:`ansible_shell_type` to select a shell plugin, and update the :ref:`ansible_executable` to match.
+.. warning:: You should not alter the configuration for these plugins unless you have a setup
+             in which the default ``/bin/sh`` is not a POSIX compatible shell or is not availble for execution.
+
+.. _using_shell:
+
+Using Shell Plugins
++++++++++++++++++++
+
+In addition to the default configuration settings in :doc:`../config`,
+you can use a 'connection variable' :ref:`ansible_shell_type` to select the plugin to use. 
+In this case, you will also want to update the :ref:`ansible_executable` to match.
 
 .. seealso::
 

docs/docsite/rst/plugins/strategy.rst

@@ -6,11 +6,21 @@ Strategy Plugins
 
 Strategy plugins control the flow of play execution by handling task and host scheduling.
 
+.. _enable_strategy:
 
 Enabling Strategy Plugins
 +++++++++++++++++++++++++
 
-Only one strategy plugin can be used in a play, but you can use different ones for each play in a playbook or Ansible run.
+Strategy plugins shipped with Ansible are enabled by default. You can enable a custom strategy plugin by
+putting it in one of the lookup directory sources configured in :doc:`ansible.cfg <../config>`.
+
+
+.. _using_strategy:
+
+Using Strategy Plugins
+++++++++++++++++++++++
+
+Only one strategy plugin can be used in a play, but you can use different ones for each play in a playbook or ansible run.
 The default is the :doc:`linear <strategy/linear>` plugin. You can change this default in Ansible :doc:`configuration <../config>` using an environment variable:
 
 .. code-block:: shell
@@ -38,6 +48,7 @@ You can also specify the strategy plugin in the play via the :ref:`strategy` key
       - name: restart_tomcat
         service: name=tomcat state=restarted
 
+.. _strategy_plugin_list:
 
 Plugin List
 +++++++++++

docs/docsite/rst/plugins/vars.rst

@@ -11,11 +11,24 @@ Vars plugins were partially implented in Ansible 2.0 and rewritten to be fully i
 The :doc:`host_group_vars <vars/host_group_vars>` plugin shipped with Ansible enables reading variables from :ref:`host_vars` and :ref:`group_vars`.
 
 
+.. _enable_vars:
+
 Enabling Vars Plugins
 +++++++++++++++++++++
 
 You can activate a custom vars plugins by either dropping it into a ``vars_plugins`` directory adjacent to your play,  inside a role, or by putting it in one of the directory sources configured in :doc:`ansible.cfg <../config>`.
 
+
+.. _using_vars:
+
+Using Vars Plugins
+++++++++++++++++++
+
+Vars plugins are used automatically after they are enabled.
+
+
+.. _vars_plugin_list:
+
 Plugin Lists
 ++++++++++++