monit: fix module detection of monitored process state (#1107) (#1163)

* refactor and test

* require version >= 5.21.0

Prior to this version the status output was different

* python version compatability

* use exception classes from utils

* modify monit to use 'status' output instead of 'summary' output

The summary output is a fixed width table which truncates the
contents and prevents us from parsing the actual status of the
program.

* add integration tests + fixes

* remove unused handlers in monit integration test

* fix lint

* add '__metaclass__ = type' to integration python files

* raise AttributeError

* simplify status

* lint: add type to parameter docs

* remove lint ignore

* move monit process config into main file

* specify path to monit PID file

* set config location based on os_family

* create required directories

* update aliases to set group and skips

* add changelog

* add author

* add types to docs

* add EPEL repo

* custom vars for centos-6

* uninstall EPEL

* support older versions

* wait for status to change before exiting

* use 'validate' to force status updates

* handle 'execution failed'

* better status output for errors

* add more context to failure + standardize

* don't check rc for validate

* legacy string format support

* add integration test for 'reloaded' and 'present'

* don't wait after reload

* lint

* Revert "uninstall EPEL"

This reverts commit 4d548718d0.

* make 'present' more robust

* Apply suggestions from code review

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>

* add license header

* drop daemon.py and use python-daemon instead

* skip python2.6 which is not supported by python-daemon

* refactor test tasks for reuse

* cleanup files after test

* lint

* start process before enabling monit

This shouldn't be necessary but I'm adding it in the hopes
it will make tests more robust.

* retry task

* attempt to rescue the task on failure

* fix indentation

* ignore check if rescue ran

* restart monit instead of reload

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
(cherry picked from commit 8de1c0c205)

Co-authored-by: Simon Kelly <skelly@dimagi.com>

tests/integration/targets/monit/files/httpd_echo.py

@@ -0,0 +1,50 @@
+# (c) 2020, Simon Kelly <simongdkelly@gmail.com>
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import (absolute_import, division, print_function)
+__metaclass__ = type
+
+import daemon
+
+try:
+    from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
+
+    def write_to_output(stream, content):
+        stream.write(content)
+except ImportError:
+    from http.server import BaseHTTPRequestHandler, HTTPServer
+
+    def write_to_output(stream, content):
+        stream.write(bytes(content, "utf-8"))
+
+
+hostname = "localhost"
+server_port = 8082
+
+
+class EchoServer(BaseHTTPRequestHandler):
+    def do_GET(self):
+        self.send_response(200)
+        self.send_header("Content-type", "text/plain")
+        self.end_headers()
+        write_to_output(self.wfile, self.path)
+
+
+def run_webserver():
+    webServer = HTTPServer((hostname, server_port), EchoServer)
+    print("Server started http://%s:%s" % (hostname, server_port))
+
+    try:
+        webServer.serve_forever()
+    except KeyboardInterrupt:
+        pass
+
+    webServer.server_close()
+    print("Server stopped.")
+
+
+if __name__ == "__main__":
+    context = daemon.DaemonContext()
+
+    with context:
+        run_webserver()