Implement vault encrypted yaml variables. (#16274)

Make !vault-encrypted create a AnsibleVaultUnicode
yaml object that can be used as a regular string object.

This allows a playbook to include a encrypted vault
blob for the value of a yaml variable. A 'secret_password'
variable can have it's value encrypted instead of having
to vault encrypt an entire vars file.

Add __ENCRYPTED__ to the vault yaml types so
template.Template can treat it similar
to __UNSAFE__ flags.

vault.VaultLib api changes:
    - Split VaultLib.encrypt to encrypt and encrypt_bytestring

    - VaultLib.encrypt() previously accepted the plaintext data
      as either a byte string or a unicode string.
      Doing the right thing based on the input type would fail
      on py3 if given a arg of type 'bytes'. To simplify the
      API, vaultlib.encrypt() now assumes input plaintext is a
      py2 unicode or py3 str. It will encode to utf-8 then call
      the new encrypt_bytestring(). The new methods are less
      ambiguous.

    - moved VaultLib.is_encrypted logic to vault module scope
      and split to is_encrypted() and is_encrypted_file().

Add a test/unit/mock/yaml_helper.py
It has some helpers for testing parsing/yaml

Integration tests added as roles test_vault and test_vault_embedded

lib/ansible/parsing/yaml/objects.py

@@ -19,7 +19,11 @@
 from __future__ import (absolute_import, division, print_function)
 __metaclass__ = type
 
+import yaml
+
 from ansible.compat.six import text_type
+from ansible.errors import AnsibleError
+from ansible.utils.unicode import to_bytes
 
 
 class AnsibleBaseYAMLObject(object):
@@ -63,3 +67,69 @@ class AnsibleUnicode(AnsibleBaseYAMLObject, text_type):
 class AnsibleSequence(AnsibleBaseYAMLObject, list):
     ''' sub class for lists '''
     pass
+
+
+# Unicode like object that is not evaluated (decrypted) until it needs to be
+# TODO: is there a reason these objects are subclasses for YAMLObject?
+class AnsibleVaultEncryptedUnicode(yaml.YAMLObject, AnsibleUnicode):
+    __UNSAFE__ = True
+    __ENCRYPTED__ = True
+    yaml_tag = u'!vault-encrypted'
+
+    @classmethod
+    def from_plaintext(cls, seq, vault):
+        if not vault:
+            raise vault.AnsibleVaultError('Error creating AnsibleVaultEncryptedUnicode, invalid vault (%s) provided' % vault)
+
+        ciphertext = vault.encrypt(seq)
+        avu = cls(ciphertext)
+        avu.vault = vault
+        return avu
+
+    def __init__(self, ciphertext):
+        '''A AnsibleUnicode with a Vault attribute that can decrypt it.
+
+        ciphertext is a byte string (str on PY2, bytestring on PY3).
+
+        The .data atttribute is a property that returns the decrypted plaintext
+        of the ciphertext as a PY2 unicode or PY3 string object.
+        '''
+        super(AnsibleVaultEncryptedUnicode, self).__init__()
+
+        # after construction, calling code has to set the .vault attribute to a vaultlib object
+        self.vault = None
+        self._ciphertext = to_bytes(ciphertext)
+
+    @property
+    def data(self):
+        if not self.vault:
+            # FIXME: raise exception?
+            return self._ciphertext
+        return self.vault.decrypt(self._ciphertext).decode()
+
+    @data.setter
+    def data(self, value):
+        self._ciphertext = value
+
+    def __repr__(self):
+        return 'AnsibleVaultEncryptedUnicode(%s)' % self._ciphertext
+
+    # Compare a regular str/text_type with the decrypted hypertext
+    def __eq__(self, other):
+        if self.vault:
+            return other == self.data
+        return False
+
+    def __hash__(self):
+        return id(self)
+
+    def __ne__(self, other):
+        if self.vault:
+            return other != self.data
+        return True
+
+    def __str__(self):
+        return str(self.data)
+
+    def __unicode__(self):
+        return unicode(self.data)