Implement part of #27617 [expend checksum format to <algorithm>:(<checksum>|<url>)] (#43751)

* expend checksum format to <algorithm>:(<checksum>|<url>)

* continue to code at office

* ALPHA - expend checksum format to <algorithm>:(<checksum>|<url>)

* clean up tmpfile and comment

* try to add test code for 27617

* try to add test code for 27617

* try to add test code for 27617

* try to fix [Could not find or access 'testserver.py']

* fix test code [Could not find or access 'testserver.py']

* fix test code [add files dir]

* fix test code [files dir not exists]

* as [connection was closed before a valid response was received]

* [connection was closed before a valid response was received]

* [connection was closed before a valid response was received]

* add test item [sha1 and sha256]

* since [connection was closed before a valid response was received]

* fix [connection was closed before a valid response was received]

* fix test code typo

* add docs for #27617

* PR #43751 is minor change

* fix pep8 issue.

* fix test code style.

* fix unexpected quote
This commit is contained in:
Shuang Wang 2018-08-25 01:45:32 +09:00 committed by Sam Doran
commit b03feb6d40
5 changed files with 125 additions and 4 deletions

View file

@ -85,7 +85,8 @@ options:
- 'If a checksum is passed to this parameter, the digest of the
destination file will be calculated after it is downloaded to ensure
its integrity and verify that the transfer completed successfully.
Format: <algorithm>:<checksum>, e.g. checksum="sha256:D98291AC[...]B6DC7B97"'
Format: <algorithm>:<checksum|url>, e.g. checksum="sha256:D98291AC[...]B6DC7B97",
checksum="sha256:http://example.com/path/sha256sum.txt"'
- If you worry about portability, only the sha1 algorithm is available
on all platforms and python versions.
- The third party hashlib library can be installed for access to additional algorithms.
@ -192,6 +193,12 @@ EXAMPLES = r'''
dest: /etc/foo.conf
checksum: md5:66dffb5228a211e61d6d7ef4a86f5758
- name: Download file with checksum url (sha256)
get_url:
url: http://example.com/path/file.conf
dest: /etc/foo.conf
checksum: 'sha256:http://example.com/path/sha256sum.txt'
- name: Download file from a file path
get_url:
url: file:///tmp/afile.txt
@ -433,7 +440,16 @@ def main():
# checksum specified, parse for algorithm and checksum
if checksum:
try:
algorithm, checksum = checksum.rsplit(':', 1)
algorithm, checksum = checksum.split(':', 1)
if checksum.startswith('http://') or checksum.startswith('https://') or checksum.startswith('ftp://'):
checksum_url = checksum
# download checksum file to checksum_tmpsrc
checksum_tmpsrc, checksum_info = url_get(module, checksum_url, dest, use_proxy, last_mod_time, force, timeout, headers, tmp_dest)
lines = [line.rstrip('\n') for line in open(checksum_tmpsrc)]
os.remove(checksum_tmpsrc)
lines = dict(s.split(None, 1) for s in lines)
filename = url_filename(url)
[checksum] = (k for (k, v) in lines.items() if v == filename)
# Remove any non-alphanumeric characters, including the infamous
# Unicode zero-width space
checksum = re.sub(r'\W+', '', checksum).lower()