Commit graph

53 commits

Author SHA1 Message Date
@
3fa745eef9 Fixing error exception handling for python3. Does not need to be compa… (#3840)
* Fixing error exception handling for python. Does not need to be compatible with Python2.4 b/c boto is Python 2.6 and above.

* Fixing error exception handling for python. Does not need to be compatible with Python2.4 b/c boto is Python 2.6 and above.

* Fixing compile time errors IRT error exception handling for Python 3.5.
This does not need to be compatible with Python2.4 b/c Boto is Python 2.6 and above.
2016-12-08 11:24:34 -05:00
Doug Luce
9167abdd10 Fix S3 unavailable region error
This is to address this error:

  fatal: [site]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to connect to S3: Region  does not seem to be available for awsmodule boto.s3. If the region definitely exists, you may need to upgrade boto or extend with endpoints_path"}

Commit 0dd58e9 changed the logic so an exception is thrown (by
`connect_to_aws`) before the `s3 is None` check is performed. This
changes the `None` check to a catch so the old logic can compensate.
2016-12-08 11:24:30 -05:00
Brian Coca
fdcc1fe763 clarified docs on overwrite 2016-12-08 11:24:25 -05:00
Ted Timmons
8c6741b2bb fix typo in documentation
let's -> lets
2016-12-08 11:24:13 -05:00
Brian Coca
b61d531dfc corrected typo on aws connection arguments var 2016-12-08 11:24:10 -05:00
Will Thames
52accd7d1f Use connect_to_aws where possible
`connect_to_aws` fixes a bug with security tokens in AWS.
Modules should use that rather than calling
`boto.x.connect_to_region`
2016-12-08 11:24:09 -05:00
Matt Martz
aaf762573e Various simple linting type cleanups on modules 2016-12-08 11:24:06 -05:00
Matt Ferrante
2dd53a8d91 updated s3 module documentation 2016-12-08 11:23:59 -05:00
Brian Coca
f50fbe9dd2 updated s3 docs to reflect mode choices
fixes #2318
2016-12-08 11:23:58 -05:00
Greg DeKoenigsberg
248ea2deea Remove @ralph-tice from maintainership per his request 2016-12-08 11:23:45 -05:00
Val Komarov
0b294d69c5 Remove duplicate logic 2016-12-08 11:23:41 -05:00
Frank van Tol
db6c6ec5c9 Update s3.py
We are copying from S3,  the bucket is the source, not the target.
2016-12-08 11:23:36 -05:00
Taneli Lepp
fb317c72b6 Added option to set multiple ACLs for S3 objects. Also verifies the selected permissions against Boto's canned ACL strings list. 2016-12-08 11:23:35 -05:00
Till Backhaus
e005ef0c36 Delete dead and broken code 2016-12-08 11:23:33 -05:00
Jason Cowley
2c511a9470 Add support for S3 canned permissions. resolves #1939 2016-12-08 11:23:32 -05:00
James Cammarata
99075976a8 Fix missing params to download_s3file in s3 2016-12-08 11:23:27 -05:00
James Cammarata
689f13548f Fixing s3 failures when bucket names contain dots 2016-12-08 11:23:27 -05:00
Brian Coca
cd072e3090 added version added to headers in s3 2016-12-08 11:23:26 -05:00
Feanil Patel
e4898a4348 If overwrite is set, download eagerly.
If the we are going to overwrite the file anyway,
there is no need to do any checksums locally or
grab the remote etag.
2016-12-08 11:23:26 -05:00
Ash Caire
682bf34a43 add headers param to s3 uploads 2016-12-08 11:23:25 -05:00
khassen
8b60dd349d Use the common/shared MD5 function. 2016-12-08 11:23:25 -05:00
ayush
f46daacf67 Updated doc strings so each character isn't considered a line 2016-12-08 11:23:23 -05:00
Apoorva Kulkarni
1f35e578b4 s3 module: Add missing version tag to "encrypt" parameter 2016-12-08 11:23:20 -05:00
Patrick Roby
e638913c97 iterate through all keys in a more pythonic manner 2016-12-08 11:23:15 -05:00
Patrick Roby
bc5d79fc30 update documentation, adding new params 2016-12-08 11:23:15 -05:00
Patrick Roby
b318bc8b43 Add support for listing keys in a specific S3 bucket
Includes support for specifying a prefix, marker, and/or max_keys. Returns a list of key names (as strings).
2016-12-08 11:23:11 -05:00
Dan Abel
a51a8e1fd6 use aws connect calls that allow boto profile use 2016-12-08 11:23:10 -05:00
Greg DeKoenigsberg
18ca7aee07 Updating cloud modules with proper github author information 2016-12-08 11:23:07 -05:00
Tyler Cross
9babe9b07d Get specific object version with S3 module.
- allow specifying version for mode=get and mode=getstr
- when version specified doesn't exist give an error message that indicates so
2016-12-08 11:23:07 -05:00
Ching Yi, Chan
be9cddbddf Refactoring for easier to read 2016-12-08 11:23:06 -05:00
Ching Yi, Chan
7cafbb5f05 Prevent memory-error from a large file 2016-12-08 11:23:06 -05:00
Bruce Pennypacker
4cb7df2631 Added delobj command to delete an object within a bucket 2016-12-08 11:23:06 -05:00
tedder
dd6d5f0cb5 feature pull request: catch and retry recoverable errors
boto can throw SSLError when timeouts occur (among other SSL errors). Catch these so proper JSON can be returned, and also add the ability to retry the operation.

There's an open issue in boto for this: https://github.com/boto/boto/issues/2409

Here's a sample stacktrace that inspired me to work on this. I'm on 1.7, but there's no meaningful differences in the 1.8 release that would affect this. I've added line breaks to the trace for readability.

    failed to parse: Traceback (most recent call last):
      File "/home/ubuntu/.ansible/tmp/ansible-tmp-1419895753.17-160808281985012/s3", line 2031, in <module> main()
      File "/home/ubuntu/.ansible/tmp/ansible-tmp-1419895753.17-160808281985012/s3", line 353, in main download_s3file(module, s3, bucket, obj, dest)
      File "/home/ubuntu/.ansible/tmp/ansible-tmp-1419895753.17-160808281985012/s3", line 234, in download_s3file key.get_contents_to_filename(dest)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1665, in get_contents_to_filename response_headers=response_headers)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1603, in get_contents_to_file response_headers=response_headers)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1435, in get_file query_args=None)
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 1488, in _get_file_internal for bytes in self:
      File "/usr/local/lib/python2.7/dist-packages/boto/s3/key.py", line 368, in next data = self.resp.read(self.BufferSize)
      File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 416, in read return httplib.HTTPResponse.read(self, amt)
      File "/usr/lib/python2.7/httplib.py", line 567, in read s = self.fp.read(amt)
      File "/usr/lib/python2.7/socket.py", line 380, in read data = self._sock.recv(left)
      File "/usr/lib/python2.7/ssl.py", line 341, in recv return self.read(buflen)
      File "/usr/lib/python2.7/ssl.py", line 260, in read return self._sslobj.read(len) ssl.SSLError: The read operation timed out
2016-12-08 11:23:04 -05:00
Benno Joy
70188fe02a Fixes issue 1197 for s3 module, where the file gets downloaded even if checksums match 2016-12-08 11:23:03 -05:00
billwanjohi
a6fd78af37 s3 put: add support for server-side encryption
- added 'encrypt' boolean option
- reordered module options alphanumerically
2016-12-08 11:23:02 -05:00
Toshio Kuratomi
b6a312e489 Minor fixups found during review of #582 2016-12-08 11:23:02 -05:00
Brian Coca
953ed0aecc avoid exception when boto is not available 2016-12-08 11:22:56 -05:00
steenzout
67f769d9a6 issue #994: use HAS_BOTO to determine if import was successful: - removed import of sys module. - HAS_BOTO constant to check if import was successful. - trigger a failure when import fails. - removed unnecessary imports. 2016-12-08 11:22:53 -05:00
Mick Bass
5de4c053b6 clean rebase of https://github.com/ansible/ansible-modules-core/pull/1024 on a new branch 2016-12-08 11:22:51 -05:00
Brian Coca
4f7bc895d2 removed requirements/boto as it broke documentation as it is duplicated in shared aws fragements 2016-12-08 11:22:51 -05:00
Jonathan Davila
de85294d2d allow boto profile use with s3 2016-12-08 11:22:51 -05:00
Kohei YOSHIDA
f0db5c0cd0 support secure connection on fakes3 2016-12-08 11:22:51 -05:00
Mick Bass
9980205edc Add support for AWS Security Token Service (temporary credentials) to all AWS cloud modules. 2016-12-08 11:22:51 -05:00
Brian Coca
7cd7bad2e8 added patch to workaround some s3 connection faliures 2016-12-08 11:22:50 -05:00
Brian Coca
252df8685a refactrored connection exception handling, added check to see if we actually get a connection before proceeding
Fixes #964
2016-12-08 11:22:50 -05:00
Alex Rowley
eeb867daf0 cloud/amazon/s3: update documentation
update description to show that module can do more than just put files into s3
2016-12-08 11:22:48 -05:00
Toshio Kuratomi
ba90eea165 Also don't fail if files are different and overwrite is false for uploads (this mimics copy and template).
Related to #489
2016-12-08 11:22:48 -05:00
Michael Laccetti
8efb138fdc Moving the import to the top of the file and getting rid of the redundancy
I sure do wish I knew where the whitespace change was coming from, though. Frustrating.
2016-12-08 11:22:46 -05:00
Michael Laccetti
0bd25a9a38 Change s3 to use connect_to_region
* Allows us to use Ansible/s3 for GovCloud when accessing a pre-existing bucket
2016-12-08 11:22:46 -05:00
Peter Gehres
878ff1e929 Issue #489: s3 should not fail on checksum mismtach when overwrite=no 2016-12-08 11:22:46 -05:00