Change default smart connection to ssh on macOS and remove paramiko from requirements.txt (#54738)

* Remove default use of paramiko connection plugin on macOS
    This fix was originally to work around a bug that caused a kernel panic on macOS
    that has since been fixed.
* Remove paramiko from requirements.txt
* Move paramiko checking to common place
* Drop the warnings obfiscation code
* Update pip installation instructions to reflect upstream instructions
* Fix tests on CentOS 6 (Python 2.6) that now show Python deprecation warnings
* Add changelog fragment

docs/docsite/rst/installation_guide/intro_installation.rst

@@ -31,10 +31,10 @@ Major bugs will still have maintenance releases when needed, though these are in
 
 If you are wishing to run the latest released version of Ansible and you are running Red Hat Enterprise Linux (TM), CentOS, Fedora, Debian, or Ubuntu, we recommend using the OS package manager.
 
-For other installation options, we recommend installing via "pip", which is the Python package manager, though other options are also available.
+For other installation options, we recommend installing via ``pip``, which is the Python package manager.
 
 If you wish to track the development release to use and test the latest features, we will share
-information about running from source.  It's not necessary to install the program to run from source.
+information about running from source. It's not necessary to install the program to run from source.
 
 .. _control_node_requirements:
 
@@ -230,9 +230,9 @@ Older versions of FreeBSD worked with something like this (substitute for your c
 Latest Releases on macOS
 ++++++++++++++++++++++++++
 
-The preferred way to install Ansible on a Mac is via pip.
+The preferred way to install Ansible on a Mac is via ``pip``.
 
-The instructions can be found in `Latest Releases via Pip`_ section. If you are running macOS version 10.12 or older, then you ought to upgrade to the latest pip (9.0.3 or newer) to connect to the Python Package Index securely.
+The instructions can be found in `Latest Releases via Pip`_ section. If you are running macOS version 10.12 or older, then you should upgrade to the latest ``pip`` to connect to the Python Package Index securely.
 
 .. _from_pkgutil:
 
@@ -293,30 +293,47 @@ Update of the software will be managed by the swupd tool::
 Latest Releases via Pip
 +++++++++++++++++++++++
 
-Ansible can be installed via "pip", the Python package manager.  If 'pip' isn't already available in
-your version of Python, you can get pip by::
+Ansible can be installed via ``pip``, the Python package manager.  If ``pip`` isn't already available on your system of Python, run the following commands to install it::
 
-   $ sudo easy_install pip
+    $ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
+    $ python get-pip.py --user
 
-Then install Ansible with [1]_::
+Then install Ansible [1]_::
 
-   $ sudo pip install ansible
+    $ pip install --user ansible
 
 Or if you are looking for the latest development version::
 
-   $ pip install git+https://github.com/ansible/ansible.git@devel
+    $ pip install --user git+https://github.com/ansible/ansible.git@devel
 
-If you are installing on macOS Mavericks, you may encounter some noise from your compiler.  A workaround is to do the following::
+If you are installing on macOS Mavericks (10.9), you may encounter some noise from your compiler. A workaround is to do the following::
 
-   $ sudo CFLAGS=-Qunused-arguments CPPFLAGS=-Qunused-arguments pip install ansible
+    $ CFLAGS=-Qunused-arguments CPPFLAGS=-Qunused-arguments pip install --user ansible
 
-Readers that use virtualenv can also install Ansible under virtualenv, though we'd recommend to not worry about it and just install Ansible globally. Do not use easy_install to install Ansible directly.
+In order to use the ``paramiko`` connection plugin or modules that require ``paramiko``, install the required module [2]_::
+
+    $ pip install --user paramiko
+
+Ansble can also be installed inside a new or existing ``virtualenv``::
+
+    $ python -m virtualenv ansible  # Create a virtualenv if one does not already exist
+    $ source ansible/bin/activate   # Activate the virtual environment
+    $ pip install ansible
+
+If you wish to install Ansible globally, run the following commands::
+
+    $ sudo python get-pip.py
+    $ sudo pip install ansible
 
 .. note::
 
-    Older versions of pip defaults to http://pypi.python.org/simple, which no longer works.
-    Please make sure you have an updated pip (version 10 or greater) installed before installing Ansible.
-    Refer `here <https://pip.pypa.io/en/stable/installing/#installation>`_ about installing latest pip.
+    Running ``pip`` with ``sudo`` will make global changes to the system. Since ``pip`` does not coordinate with system package managers, it could make changes to you system that leave it in an inconsistent on non-functioning state. This is particularly true for macOS. Installing with ``--user`` is recommended unless you understand fully the implications of modifying global files on the system.
+
+.. note::
+
+    Older versions of ``pip`` default to http://pypi.python.org/simple, which no longer works.
+    Please make sure you have the latest version of ``pip`` before installing Ansible.
+    If you have an older version of ``pip`` installed, you can upgrade by following `pip's upgrade instructions <https://pip.pypa.io/en/stable/installing/#upgrading-pip>`_ .
 
 .. _tagged_releases:
 
@@ -335,9 +352,9 @@ These releases are also tagged in the `git repository <https://github.com/ansibl
 Running From Source
 +++++++++++++++++++
 
-Ansible is easy to run from a checkout - root permissions are not required
-to use it and there is no software to actually install.  No daemons
-or database setup are required.  Because of this, many users in our community use the
+Ansible is easy to run from source. You do not need ``root`` permissions
+to use it and there is no software to actually install. No daemons
+or database setup are required. Because of this, many users in our community use the
 development version of Ansible all of the time so they can take advantage of new features
 when they are implemented and easily contribute to the project. Because there is
 nothing to install, following the development version is significantly easier than most
@@ -345,7 +362,7 @@ open source projects.
 
 .. note::
 
-   If you are intending to use Tower as the Control Node, do not use a source install. Please use OS package manager (like ``apt/yum``) or ``pip`` to install a stable version.
+   If you are want to use Ansible Tower as the Control Node, do not use a source installation of Ansible. Please use an OS package manager (like ``apt`` or ``yum``) or ``pip`` to install a stable version.
 
 
 To install from source, clone the Ansible git repository:
@@ -355,7 +372,7 @@ To install from source, clone the Ansible git repository:
     $ git clone https://github.com/ansible/ansible.git
     $ cd ./ansible
 
-Once git has cloned the Ansible repository, setup the Ansible environment:
+Once ``git`` has cloned the Ansible repository, setup the Ansible environment:
 
 Using Bash:
 
@@ -371,15 +388,16 @@ If you want to suppress spurious warnings/errors, use::
 
     $ source ./hacking/env-setup -q
 
-If you don't have pip installed in your version of Python, install pip::
+If you don't have ``pip`` installed in your version of Python, install it::
 
-    $ sudo easy_install pip
+    $ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
+    $ python get-pip.py --user
 
 Ansible also uses the following Python modules that need to be installed [1]_:
 
 .. code-block:: bash
 
-    $ sudo pip install -r ./requirements.txt
+    $ pip install --user -r ./requirements.txt
 
 To update ansible checkouts, use pull-with-rebase so any local changes are replayed.
 
@@ -387,18 +405,14 @@ To update ansible checkouts, use pull-with-rebase so any local changes are repla
 
     $ git pull --rebase
 
-Note: when updating Ansible checkouts that are v2.2 and older, be sure to not
-only update the source tree, but also the "submodules" in git which point at
-Ansible's own modules.
-
 .. code-block:: bash
 
     $ git pull --rebase #same as above
     $ git submodule update --init --recursive
 
 Once running the env-setup script you'll be running from checkout and the default inventory file
-will be /etc/ansible/hosts.  You can optionally specify an inventory file (see :ref:`inventory`)
-other than /etc/ansible/hosts:
+will be ``/etc/ansible/hosts``. You can optionally specify an inventory file (see :ref:`inventory`)
+other than ``/etc/ansible/hosts``:
 
 .. code-block:: bash
 
@@ -421,7 +435,7 @@ Ansible on GitHub
 `````````````````
 
 You may also wish to follow the `GitHub project <https://github.com/ansible/ansible>`_ if
-you have a GitHub account.  This is also where we keep the issue tracker for sharing
+you have a GitHub account. This is also where we keep the issue tracker for sharing
 bugs and feature ideas.
 
 
@@ -439,3 +453,4 @@ bugs and feature ideas.
        #ansible IRC chat channel
 
 .. [1] If you have issues with the "pycrypto" package install on macOS, then you may need to try ``CC=clang sudo -E pip install pycrypto``.
+.. [2] ``paramiko`` was included in Ansible's ``requirements.txt`` prior to 2.8.

docs/docsite/rst/porting_guides/porting_guide_2.8.rst

@@ -323,7 +323,7 @@ Noteworthy module changes
   You should use the ``win_service`` module to control the running state of the service. This will be removed in Ansible 2.12.
 
 * The ``status`` module option for ``win_nssm`` has changed its default value to ``present``. Before, the default was ``start``.
-  Consequently, the service is no longer started by default after creation with ``win_nssm``, and you should use 
+  Consequently, the service is no longer started by default after creation with ``win_nssm``, and you should use
   the ``win_service`` module to start it if needed.
 
 * The ``app_parameters`` module option for ``win_nssm`` has been deprecated; use ``argument`` instead. This will be removed in Ansible 2.12.
@@ -340,6 +340,8 @@ Noteworthy module changes
 Plugins
 =======
 
+* Ansible no longer defaults to the ``paramiko`` connection plugin when using macOS as the control node. Ansible will now use the ``ssh`` connection plugin by default on a macOS control node.  Since ``ssh`` supports connection persistence between tasks and playbook runs, it performs better than ``paramiko``. If you are using password authentication, you will need to install ``sshpass`` when using the ``ssh`` connection plugin. Or you can explicitly set the connection type to ``paramiko`` to maintain the pre-2.8 behavior on macOS.
+
 * Connection plugins have been standardized to allow use of ``ansible_<conn-type>_user``
   and ``ansible_<conn-type>_password`` variables.  Variables such as
   ``ansible_<conn-type>_pass`` and ``ansible_<conn-type>_username`` are treated