Commit graph

883 commits

Author SHA1 Message Date
Will Thames
7ae4027c58 Improve ec2_vpc_subnet check mode (#23108)
check_mode should behave pretty similarly to non-check mode -
just don't actually create or delete subnets or change tags.

Using DryRun for check_mode behaves very differently and results
in the following module failure:

```
"msg": "Unable to update tags for subnet-abcd1234,
error: EC2ResponseError: 412 Precondition Failed
<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<Response><Errors><Error><Code>DryRunOperation</Code>
<Message>Request would have succeeded, but DryRun flag
is set.</Message></Error></Errors>
<RequestID>12345678-abcd-1234-abcd-abcd1234abcd</RequestID></Response>"
```
2017-06-30 08:04:48 -04:00
Vinay Dandekar
c884d4ab7f Add support for EC2 dynamic data in ec2_facts (#21532)
* Add support for EC2 dynamic data in ec2_facts

- Flattens out JSON in the instance identity document and IAM info/credentials for easy access to facts
- This changes region fact from ‘ansible_ec2_placement_region’ to ’ansible_ec2_instance_identity_document_region’

* Maintain backwards compatibility by putting the region into the old key

* Improve JSON parsing logic and split security group IDs

* Add documentation, backwards compatibility, fix bug and formatting

- Update documentation for ec2_facts with return values
- Preserve JSON value from the metadata service for backwards compatibility
- Fix bug in fix_invalid_varnames
  - The keys in the dict were being modified in place; new dict now created to hold the sanitized keys
  - Consolidate two replace calls with a regex substitution
- Move imports for ec2_facts to the top

* Add support for parsing the IAM instance profile role
2017-06-30 16:27:49 +10:00
Rob
d1652aecf0 Upgrade ec2_remote_facts to use boto3 (#22937)
* Deprecate ec2_remote_facts in favor of ec2_instance_facts which uses boto3

* Update legacy-files.txt with deprecated file name change
2017-06-30 14:19:08 +10:00
Brandon Davidson
7bb3467db9 Convert ec2_vpc_subnet module to boto3 and add map_public option. (#23783) 2017-06-29 23:39:21 +10:00
Sloane Hertel
338bf0fd3f ec2_vpc_dhcp_options_facts: if tags don't exist set them to default list instead of crashing - fixes #23825 (#23967)
* if tags don't exist set them to an empty list

* dhcp_option will always have a name tag

* make requested change
2017-06-29 22:47:05 +10:00
Daniel Kozlowski
dcd1ff2809 Initial commit of iam_managed_policy file (#22097)
This module can add, remove, update versions, and set default versions
of managed policies. It will cycle out old versions of policies if too
many are present. It will check and set the version of the policy that
matches the pased in policy document if one already exists.

Incorporating changes from PR

Descriptions now have full stops, and pep8 error has been
addressed. Also added requirements, author, and updated interface to
"preview"

Additional change to pass CI

Previous commit added in some whitespace errors. Additinoally added
correct value for version_added, added in a RETURN block for
documentation, and moved import to top of file

Fixed error detaching policy from users

Updates to pass 2.4 CI

Updating iam_managed_policy supporting feedback
2017-06-29 12:08:27 +10:00
rip777
4e54df71a2 Further iam_group improvements
added `purge_users` and `purge_policy` option
added `convert_friendly_names_to_arns` function
flake8 fixes
2017-06-29 11:38:49 +10:00
Nick Aslanidis
dc8a05dfe2 New AWS module for managing IAM groups 2017-06-29 11:38:48 +10:00
MaciejCetler
0d179c77a7 fix for iam (#23347)
* fix for iam

* Update iam.py

removed extra space
2017-06-28 11:11:19 -04:00
Sloane Hertel
3ba4fc2d54 boto only returns tags in the described security group if they exist (#26053) 2017-06-28 09:14:38 +10:00
Will Thames
0e0b91534a [cloud] Return target_group_names where sensible in ec2_asg_facts module (#26078)
If target_group_arns is an empty list, then return
an empty target_group_names list.

If a connection to elbv2 is not obtainable, then it is
not possible to return target_group_names
2017-06-27 17:04:15 -04:00
Josh Souza
e49f15d6e4 [cloud] Fix logical flaw in route53_health_check, use string ports everywhere (#25706)
* Fix logical flaw (update when diff), use string ports everywhere

* Change port comparison to integer vs. string

The comparison works either way as long as it's consistent. Boto docs
state that it takes in an integer, but if given a string apparently
keeps it as such. This change just ensures that when we compare, we
specifically deal with integers.
2017-06-27 15:58:21 -04:00
Toshio Kuratomi
e238ae999b Cyptography pr 20566 rebase (#25560)
Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto

pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.

This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.

* Handle wrong password given for VaultAES format

* Do not display deprecation warning for cryptography on python-2.6

* Namespace all of the pycrypto imports and always import them

  Makes unittests better and the code less likely to get stupid mistakes
  (like using HMAC from cryptogrpahy when the one from pycrypto is needed)

* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko

* contrib/inventory/gce: Remove spurious require on pycrypto

(cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271)

* Add cryptography to ec2_win_password module requirements
  * Fix python3 bug which would pass text strings to a function which
    requires byte strings.

* Attempt to add pycrypto version to setup deps

* Change hacking README for dual pycrypto/cryptography

* update dependencies for various CI scripts

* additional CI dockerfile/script updates

* add paramiko to the windows and sanity requirement set

  This is needed because ansible lists it as a requirement. Previously
  the missing dep wasn't enforced, but cryptography imports pkg_resources
  so you can't ignore a requirement any more

* Add integration test cases for old vault and for wrong passwords

* helper script for manual testing of pycrypto/cryptography

* Skip the pycrypto tests so that users without it installed can still run the unittests

* Run unittests for vault with both cryptography and pycrypto backend
2017-06-27 06:00:15 -07:00
Sloane Hertel
dc2d9a2134 [cloud] Python3 fixes: use string_types instead of basestring (#26102)
Don't modify a dict while iterating over it
2017-06-26 15:06:53 -04:00
Mark Szymanski
91c1d8d8e2 Fixing bug introduced in 20622 2017-06-26 11:38:47 -04:00
mihu
6b76bc924f [cloud] New feature for ec2_group: allow deleting groups by id (#26022) 2017-06-26 09:07:29 -04:00
Ryan Fitzpatrick
3e52f47b51 [cloud] Fixes #26045 Ensure instance.instance_profile is not None before casting to dict (#26047) 2017-06-23 12:33:55 -04:00
Mark Szymanski
1703db4211 Allow iam_cert to be renamed or moved without needing to specify cert or key data (#20622)
Generally Pep8 and flake8 compliant so removing from legacy-files.txt
2017-06-23 15:32:15 +10:00
Constantin
9cc35b094f Fix of argument error in #25940 (#25979) 2017-06-22 16:59:36 +10:00
mihu
60afe93521 amazon: allow NACL to be removed by its id (#25925) 2017-06-22 16:46:46 +10:00
Wouter de Geus
dd07d11ae5 [cloud] Add ChangeSet support to cloudformation module (#23490) (#24497)
*  * Implements Change Sets on updating a cloudformation stack when create_changeset=true (#23490)

*  * Silence test complaints ;)

*  * Added optional changeset_name parameter.
 * Check if changeset with the requested name already exist.
 * Documentation fix

*  * Added warning when cloudformation stack has pending changesets.
 * Fix documentation
2017-06-21 16:05:17 -04:00
Tom Paine
f831be680e [cloud] Update ec2.py default for public IP assignment (#25896)
* Update ec2.py

Update docs to reflect default.

* Update ec2.py

Further default update.
2017-06-21 14:58:26 -04:00
Sloane Hertel
04e93e22cb add traceback and more helpful errors for ec2_vpc_route_table (#25704) 2017-06-21 12:01:52 +10:00
Yang Liu
b41b4fe2fd use proper pagination variable (#25838) 2017-06-21 12:00:41 +10:00
Damir Suleymanov
74f6b64fb0 Handle rate limits on every iteration (#23726) 2017-06-20 16:49:05 -04:00
Will Thames
f3c98f482a [cloud] New module ec2_vpc_endpoint_facts for AWS EC2 VPC Endpoints (#20211)
* New facts module for AWS EC2 VPC Endpoints

* ec2_vpc_endpoint_facts - meet latest Ansible standards

Fix exception syntax and use of `iteritems` for python3

Fix undefined `ec2` variable (should have been `connection`
Address various flake8 issues

Use `ansible_dict_to_boto3_filter_list` rather than
duplicating its implementation

* Remove max_items and next_token from vpc_endpoint_facts

max_items and next_token should be a module concern, not
a caller concern. It would be very difficult for a module
consumer to use next_token properly, whereas it's easy for
the module to handle it.

* ec2_vpc_endpoint_facts trivially supports check mode

Add supports_check_mode=True to the argument spec.

* Improve RETURN documentation for ec2_vpc_endpoint_facts

Fix bug in EXAMPLE documentation too

* fix return type for validate-modules
2017-06-20 08:31:31 -04:00
Will Rouesnel
3adf08e10f [cloud] iam_cert return arn and allow use with ansible vault (#20787)
* iam_cert.py Fix duplicate certificate detection with included chains.

The iam_cert module would fail to detect certificates as duplicates
if the certificate body included the authority chain directly.

This commit fixes the problem by checking if a given certificate
matches the start of the data returned by AWS, since in all cases
where they would match the certificate will come first.

* iam_cert.py Return certificate ARN in all success cases.

When uploading certificates or interacting with IAM, the certificate ARN
is needed for other operations with AWS such as provisioning elastic load
balancers.

This commit returns the certificate ARN in all success cases, which allows
it to be used to idempotently provision other Amazon services depending on
it (ELBs being an immediate example).
2017-06-20 08:01:56 -04:00
Sloane Hertel
06c21b4bec [cloud] fix dynamodb_ttl exception handling for ParamValidationError (#25870)
Also adds tracebacks to exceptions and imports `HAS_BOTO3` instead of checking directly.
2017-06-19 15:29:14 -04:00
Sloane Hertel
e079498485 [cloud] document route53 state choices added in 2.4 (#25866) 2017-06-19 15:16:58 -04:00
Ted Timmons
346cbf7d7a [cloud] New module: Add module to set Amazon AWS DynamoDB TTL (cloud/amazon/dynamodb_ttl) (#22588)
* add aws dynamo_ttl module, small parameter setter

 - New Module Pull Request

`dynamodb_ttl`

2.3.0/devel

Very self-contained TTL setter. This is independent of the dynamodb_table module
as it's really designed to be a helper for tables that may be created in other
ways (say, CloudFormation, which doesn't support setting TTL).

* committer is no longer a valid value

* bump version_added, catch common exceptions

* pep8 fixes

* one more pep8
2017-06-19 09:57:04 -04:00
Abhijeet Kasurde
6bc1e802e3 Rectify traceback.format_exc() call (#25773)
Fix adds correct call to traceback.format_exc method

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-06-16 11:38:36 -04:00
Sloane Hertel
938d60472a [cloud] ec2.py: add region to the aws_connect_kwargs before creating vpc connection Fixes #25673 (#25749)
* add region to the aws_connect_kwargs before creating vpc connection

* remove unused import
2017-06-15 15:52:56 -04:00
Robin Miller
540e1bbd69 [cloud] Route53 Avoid throttling errors and unnecessary processing when checking rrsets. (#22104)
The boto Route53 get_all_rrsets method will return the record set
matching the name, type, and identifier specified, followed by ALL
subsequent sets in alphabetical order based on name. If the specified
set does not exist, the method will still return all the sets that
_would_ have come after it. Searching through sets we know will not
match is not just a waste of resources but, more importantly, often
triggers AWS API throttling when used on zones with large numbers of
records.
2017-06-15 15:06:16 -04:00
Rob
6d34ce3f2a [cloud] elb_target_group_facts.pyFix tags being snaked (#25742) 2017-06-15 13:19:28 -04:00
Rob
f8d027b1ba [cloud] New module elb_target_group_facts (#24583) 2017-06-14 12:22:50 -04:00
Sloane Hertel
dfcf43efa9 ec2: Allow EC2-VPC instances to modify security groups - fixes #18928 (#23831)
* Allow EC2-VPC instances to update SG

make ec2 pep8

* use sets instead of loop and a break

* bring things in an indentation level

* Use to_text instead of str, text_type instead of basestring, - instead of difference

* basestrings not unicode

* simplifying syntax
2017-06-14 11:10:29 -04:00
Sloane Hertel
bd4f08d434 [cloud] Fix Route53 module value option, defaulting to empty list (#24555) 2017-06-13 15:55:19 -04:00
Sloane Hertel
3bba43a487 [cloud] ec2.py: region is documented as optional; allow endpoints to be used - fixes #24382 (#24470)
* region isn't required for ec2.py; allow endpoints to be used

* move where aws_connect_kwargs is set

* remove camel_dict_to_snake_dict and display error message
2017-06-13 15:12:47 -04:00
Sloane Hertel
00d5de3dec fix return value (#25611)
document the returned
2017-06-13 09:18:58 -04:00
Simon Staszkiewicz
eb7530df95 Correct link to DescribeInternetGateway docs (#25622)
Old link pointed to DescribeRoutingTable documentation instead of DescribeInternetGateway documentation.
2017-06-12 20:30:30 -04:00
Sloane Hertel
3f0c47196e [cloud] s3: deleting a nonexistent bucket should not give a traceback Fixes #25445 (#25487)
* trying to delete a nonexistent bucket should not fail

* Improve error handling for deleting s3 bucket

* Allow successful deletion

* Add test for deleting a nonexistent bucket

rename integration test target from s3 to aws_s3
2017-06-12 13:52:25 -04:00
Alan Loi
3ac9273507 Fix hash_key_name to be optional for delete (#25009) 2017-06-12 10:25:43 -04:00
Alan Loi
c23e3c1913 Fix version_added for tags and wait_for_active_timeout params (#24993) 2017-06-12 10:19:41 -04:00
willricardo
dd1f5d7acd ec2 module conver basestring to string_types (#25503) 2017-06-12 08:48:38 -04:00
Will Thames
16b877e2b3 ec2_asg and ec2_asg_facts module improvements (#25166)
* ec2_asg and ec2_asg_facts module improvements

Return target group information for both ec2_asg and ec2_asg_facts
modules

Provide RETURN documentation for ec2_asg module

PEP8 fixes for ec2_asg_facts

* ec2_asg: use pagination when describing target groups

In case an ASG has 100s of target groups, ensure that
we get the full result using build_full_result
2017-06-12 07:15:04 -04:00
Abhijeet Kasurde
b89cb95609 Fix spelling mistakes (comments only) (#25564)
Original Author : klemens <ka7@github.com>

Taking over previous PR as per
https://github.com/ansible/ansible/pull/23644#issuecomment-307334525

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-06-12 07:55:19 +01:00
Chris Atkinson
62fa2019c6 [cloud] Allow IAM certificate secrets, body, chains to be strings (e.g. sourced from ansible-vault) (#24206)
* Allow cert and private pem information to be passed in via string, eg when obtaining sensitive key details from anisble-vault at runtime

* Allow cert chain body to be passed as a string

* Ensure the new options are set in parameters

* Dont publish the private key in logs

* Set the version_added documentation

* Update documentation inline with review

* Removes file based certificates in favour of string only as suggested in feature review

* Documentation changes as suggested by review
2017-06-09 11:32:22 -04:00
Sloane Hertel
b376bde4d7 [cloud] ec2_eni module - allow attaching a new instance - fixes #19452 (#22377)
* fix bug attaching instances

* pep8
2017-06-09 11:31:42 -04:00
Will Thames
3549391673 New facts module for AWS VPC Peering (#19910)
* New facts module for AWS VPC Peering

* Update documentation and region catch for boto3

* Update ec2_vpc_peering_facts for python3 and flake8

Update version to 2.3
Update for python3 exception handling
Use ansible_dict_to_boto3_filter_list rather than duplicating
implementation
Use camel_dict_to_snake_dict on results for ansible standard results
Tidy up flake8 compliance

* Support check_mode for ec2_vpc_peering_facts

Add supports_check_mode to the module declaration

* Fix latest Ansible standards

Update version to 2.4
Add ANSIBLE_METADATA
Move imports
2017-06-06 14:42:58 -04:00
Sloane Hertel
7bebc83dcf module_utils/ec2: fix boto3 tags to remove (#25277)
* Fix boto3 tags to remove; unable to remove a list of strings, need list of {'Key': key}

* pep8 fix

* cannot bugfix due to differences in AWS APIs - just document

* add more info to guidelines for using compare_aws_tags
2017-06-06 11:02:15 -04:00