Commit graph

689 commits

Author SHA1 Message Date
Brian Coca
960388143f
faster config loading (#48333)
* faster config loading

  - used already loaded module var instead of doc functions
  - add preload to populate config defs cache
  - avoid debug work when not in debug
  - generators, force consumption
2019-01-23 12:06:54 -05:00
Martin Krizek
3dc5ba6186
Remove unnecessary calls to cache_block_tasks noop (#50715) 2019-01-17 08:29:26 +01:00
Brian Coca
119b65f919
Clarify clear facts (#50667)
* Revert "avoid x2 setting of set_fact when 'cacheable' (#50564)"

This reverts commit 207848f354.

* clarify clear_facts with set_fact cacheable

 revert previous 'fix' as it will break playbooks by changing precedence
 opted to leave current behaviour but document it on both plugins to mitigate confusion

 fixes #50556

 also fix grammer, add comment, remove unused e
2019-01-15 15:20:33 -05:00
Abhijeet Kasurde
db8702cdb8 Close all open filehandle (#50544)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-01-11 10:14:08 -05:00
Martin Krizek
4c72ef124b tqm._initialize_notified_handlers() is noop now (#50713) 2019-01-09 10:15:11 -05:00
Brian Coca
207848f354 avoid x2 setting of set_fact when 'cacheable' (#50564)
* avoid x2 setting of set_fact when 'cacheable'

  fixes #50556

* ammend docs to new behaviour

* added period
2019-01-08 10:12:49 -05:00
Abhijeet Kasurde
bc6cd13874
Handle exception raised in recursive_finder API (#49590)
User module can contain Indentation errors or syntax errors.
Handle AST exceptions rather than showing traceback while importing such module.

Fixes: #21707

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-01-04 10:33:37 +05:30
Toshio Kuratomi
afdbb0d9d5 Save the command line arguments into a global context
* Once cli args are parsed, they're constant.  So, save the parsed args
  into the global context for everyone else to use them from now on.
* Port cli scripts to use the CLIARGS in the context
* Refactor call to parse cli args into the run() method
* Fix unittests for changes to the internals of CLI arg parsing
* Port callback plugins to use context.CLIARGS
  * Got rid of the private self._options attribute
  * Use context.CLIARGS in the individual callback plugins instead.
  * Also output positional arguments in default and unixy plugins
  * Code has been simplified since we're now dealing with a dict rather
    than Optparse.Value
2019-01-03 18:12:23 -08:00
Nathaniel Case
1829a72885
Allow persistent connection plugins to queue messages back to ansible-connection (#49977)
* Connections can queue messages to be returned from ansible-connection

* Provide fallback for invalid display level

* Strip display from plugins

* Route messages through helper method to try to avoid improper appends
2018-12-19 10:54:42 -05:00
Toshio Kuratomi
3fba006207 Update bare exceptions to specify Exception.
This will keep us from accidentally catching program-exiting exceptions
like KeyboardInterupt and SystemExit.
2018-12-16 15:03:19 -08:00
Jordan Borean
190d1ed7f1 win become: refactor and add support for passwordless become (#48082)
* win become: refactor and add support for passwordless become

* make tests more stable

* fix up dep message for Load-CommandUtils

* Add further check for System impersonation token

* re-add support for become with accounts that have no password

* doc fixes and slight code improvements

* fix doc sanity issue
2018-12-12 17:15:25 -08:00
Nathaniel Case
02432565cd
Remove cliconf from httpapi connection (#46813)
* Bare minimum rip out cliconf

* nxapi changeover

* Update documentation, move options

* Memoize device_info

* Gratuitous rename to underscore use of local api implementation

Fixup eos module_utils like nxos

* Streamline version and image scans

* Expose get_capabilities through module_utils

* Add load_config to module_utils

* Support rpcs using both args and kwargs

* Add get_config for nxos

* Add get_diff

* module context, pulled from nxapi

We could probably do this correctly later

* Fix eos issues

* Limit connection._sub_plugin to only one plugin
2018-12-11 16:26:59 -05:00
Matt Martz
2a469fd959 Consolidate handler tracking (#49338)
* Consolidate handler tracking
  -  Remove unused code. ci_complete
  - unit test fixes. ci_complete
  - Restore previous behavior of matching a single handler
  - when notifying a host for a handler, return True if it was added, False otherwise, to reduce copied logic
  - rename funcitons for clarity. ci_complete
  - Remove handler logic for static includes which was disabled previously
2018-12-11 10:58:19 -05:00
Matt Martz
9007dbec2f
Add info about loop based on jinja2 loop var (#42134)
* Add info about loop based on jinja2 loop var

* ansible_loop

* Update test count

* Add extended loop_control that defines whether ansible_loop should be added

* Extended needs to be defaulted

* Revert "Update test count"

This reverts commit f1e93ee469825f4cdcd90fb28667d29aa088275c.

* Add docs about loop_control.extended

* Add revindex and revindex0

* Document ansible_loop in special vars

* Add changelog fragment

* Add tests, change items to allitems so that dot notation works, fix logic error with previtem
2018-12-07 13:49:50 -06:00
Matt Martz
69d230f4bf Make squash deprecation messages less misleading (#49551) 2018-12-06 10:49:01 -05:00
Monty Taylor
824c72da5f Set play context on stdout callback plugin (#35910)
The loop on self._callback_plugins does not include the stdout callback,
so the stdout_callback never has set_play_context called.
2018-11-29 23:15:56 -05:00
Matt Davis
8c1f701e6e
split PS wrapper and payload (CVE-2018-16859) (#49142)
* prevent scriptblock logging from logging payload contents
* added tests to verify no payload contents in PS Operational event log
* fix script action to send split-aware wrapper
* fix CLIXML error parser (return to -EncodedCommand exposed problems with it)
2018-11-26 15:28:21 -08:00
Matt Martz
9773a1f289
Add a Singleton metaclass, use it with Display (#48935)
* Add a Singleton class, use it with Display

* update six import

* Move remaining failes to display singleton

* Fix rebase issues

* Singleton improvements

* Add code-smell for 'from __main__ import display'. ci_complete

* s/self/cls/g

* Add docs for no-main-display

* Address linting issues

* Add changelog fragment. ci_complete

* Implement reentrant lock for class instantiation in Singleton

* Add Display singleton porting guide
2018-11-20 17:06:51 -06:00
Matt Martz
64bb54dab4 Catch per loop label templating exceptions. Fixes #48879 (#48896) 2018-11-20 15:29:44 -05:00
Abhijeet Kasurde
a6fde8aef9 Add timeout information into async failure message
Add some extra information to an async failure message to reflect the
actually timeout value of the failure.

Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-11-20 08:46:27 -05:00
Martin Krizek
869d866aa1 display.error on no matching --start-at-task (#45221) 2018-11-15 19:35:51 -05:00
Jordan Borean
4fc013a7fa powershell - fix async for Ansible.Basic (#48404)
* powershell - fix async for Ansible.Basic

* moved old out outside of the try block
2018-11-13 12:08:27 -08:00
Brian Coca
b047591dee fix tagged implicit gathering (#48294)
* fix tagged implicit gathering

 - None is not tHe default anymore, its an empty list

* Update changelogs/fragments/fix_taggged_gather.yml

Co-Authored-By: bcoca <bcoca@users.noreply.github.com>
2018-11-09 13:41:07 +00:00
Matt Martz
0d068f1e3a
Support list of dicts with omit. Fixes #45907 (#45923) 2018-11-07 13:41:22 -06:00
Jordan Borean
501acae5ab
Added basic equivalent to PowerShell modules (#44705)
* Added basic equivalent to PowerShell modules

* changes based on latest review

* Added tests

* ignore sanity test due to how tests are set up

* Changes to work with PSCore

* Added documentation and change updated more modules

* Add some speed optimisations to AddType

* fix some issues in the doc changes

* doc changes
2018-11-07 10:53:17 +10:00
Brian Coca
4ec2423a04
make fact gathering obey play tags (#44717)
fixes #44251
2018-11-06 13:19:44 -05:00
Matt Martz
77d32b8f57
Don't use the task for a cache, return a special cache var (#47243)
* Don't use task to cache loop results, use hostvars. Fixes #47207

* Avoid a race condition, supply _ansible_loop_cache through get_vars directly

* Add tests

* Add changelog fragment

* Remove unnecessary copy

* Remove unnecessary host from _get_delegated_vars signature
2018-10-18 15:25:43 -05:00
Matt Martz
02f4d0a57f
Ensure that an empty literal list with loop skips the task (#47129) 2018-10-16 13:35:10 -05:00
Eitan Adler
063a9dd911 Reduce noise in warnings from ansible (#40754)
When python is compiled in debug mode, or certain command line flags are
passed, python issues helpful warnings to let users know of files opened
by not closed.

This can be fairly spammy on an ansible run.  This change reduces the
number of such warnings by a factor of 10.
2018-10-09 18:53:03 -04:00
Matt Clay
a11f631ee4 Python 3.8 collections compatibility fixes.
Includes a new pylint blacklist plugin to prevent regressions.
2018-10-08 11:26:37 -07:00
Jordan Borean
e972287c35 win_exec: refactor PS exec runner (#45334)
* win_exec: refactor PS exec runner

* more changes for PSCore compatibility

* made some changes based on the recent review

* split up module exec scripts for smaller payload

* removed C# module support to focus on just error msg improvement

* cleaned up c# test classifier code
2018-10-02 15:55:53 -07:00
Nathaniel Case
cdaf5bd806
Add configuration to override location of ansible-connection (#46128)
* Add configuration to override location of ansible-connection

* Check var or argv[0] before $PATH
2018-10-01 14:29:59 -04:00
Nathaniel Case
406b59aeba
Move persistent connections to only use registered variables (#45616)
* Try to intuit proper plugins to send to ansible-connection

* Move sub-plugins to init so that vars will be populated in executor

* Fix connection unit tests
2018-09-20 09:56:43 -04:00
Jordan Borean
5c73d4f4bd
async: use async_dir for the async results file directory (#45461)
* win async: use async_dir for the async results file directory

* tried to unify POSIX and PowerShell async implementations of async_dir

* fix sanity issue
2018-09-20 19:37:54 +10:00
Nathaniel Case
3d0f19d8cd
Exclude the whole ansible_failed_task block (#45587)
It has lots of FieldAttributes that won't serialize.
Also the connection doesn't care.
2018-09-12 21:17:24 -04:00
Jordan Borean
d81249994e
win_script: add support for become and centralise exec wrapper builder (#45582)
* win_script: add support for become and centralise exec wrapper builder

* satisfying the pep8 gods

* do not scan for module dependencies when running as a script
2018-09-13 08:50:13 +10:00
Brian Coca
4dea9d84ed fix issue with no_log in py3
the error percolates differently from undefined vars in jinja2,
so fallback to use play_context as it is not involved in current templating
2018-08-27 11:57:02 -07:00
Toshio Kuratomi
55503004ec Fix recursive_finder so it doesn't re-read files multiple times
This closes a corner case where recursive_finder could read a file
multiple times.
2018-08-24 15:37:13 -07:00
Toshio Kuratomi
14b3ae1120 Fix new AnsiballZ code to always include module_utils/basic
The AnsiballZ optimization which only uses one pyton interpreter
currently monkeypatches the arguments into a global argument in module_utils
so we need to always include basic.py.  In the future we should fix this
so that it monkeypatches its own file.  That way we won't need to always
include basic.py
2018-08-24 15:37:13 -07:00
Ryan Brown
4c8808ec9d
Extend module_defaults by adding default groups for cloud modules (#44127)
Extends `module_defaults` by adding a prefix to defaults `group/` which denotes a builtin list of modules. Initial groups are: `group/aws`, `group/azure`, and `group/gcp`
2018-08-22 21:33:27 -04:00
Brian Coca
bda074d34e
fix tempating issues with no_log and loops (#44468)
* fix tempating issues with no_log and loops

 - task is no log if any item is
 - added test cases

fixes #43294
2018-08-21 21:53:56 -04:00
Matt Martz
9b2baebe64
Don't use copy.deepcopy in high workload areas, use naive_deepcopy (#44337)
* Don't use copy.deepcopy in high workload areas, use deepishcopy. ci_complete

* Add tests

* Add changelog fragment

* rename to naive_deepcopy and add extra docs

* Rename to module_response_deepcopy and move to vars/clean
2018-08-20 15:08:29 -05:00
Matt Davis
6b81c36533
restore task arg splatting (#43798)
* restore task arg splatting

* reverts #41804
* supersedes #41295
* fixes #42192
* after lots of discussion amongst the core team, we decided to preserve this feature, clarify the runtime warnings/docs, and prioritize a path toward fixing the underlying behavior that causes this feature to be insecure (un-namespaced facts).

* update faq text

note that warning is disabled when inject_facts_as_vars is

* wordsmithing FAQ entry
2018-08-14 12:58:00 -07:00
Matt Martz
17c89d1ffa
Remove unused rslt_q, rename the one queue to final_q everywhere (#43894)
* Remove unused rslt_q, rename the one queue to final_q everywhere

* Test update
2018-08-13 13:43:28 -05:00
Sam Doran
6d38167d49
Only template values in vars_prompt rather than all vars (#39304)
* Only template values in vars_prompt rather than all vars

This allows the use of variables in vars_prompt fields but allows variables entered in the prompt to affect play vars rather than throwing an undefined error.

Only post validate if there was a vars_prompt

* Add tests for vars_prompt
2018-08-13 12:54:31 -04:00
Nathaniel Case
f221105882
Prevent data being truncated over persistent connection socket (#43885)
* Change how data is sent to the persistent connection socket.

We can't rely on readline(), so send the size of the data first. We can
then read that many bytes from the stream on the recieving end.

* Set pty to noncanonical mode before sending

* Now that we send data length, we don't need a sentinel anymore

* Copy socket changes to persistent, too

* Use os.write instead of fdopen()ing and using that.

* Follow pickle with sha1sum of pickle

* Swap order of vars and init being passed to ansible-connection
2018-08-10 09:26:58 -04:00
Brian Coca
2e9a7e2564 remove deprecated 2018-08-01 15:17:28 -04:00
Brian Coca
21fc6a0c38 nicer error message on multiprocessing fail
fixes #43090
2018-08-01 15:14:23 -04:00
Nathaniel Case
9169daa326
\r does not round-trip the local socket, escape and restore on the other side (#43507) 2018-08-01 08:44:39 -04:00
Matt Martz
a7f45c0660 Add basedir to ansible_search_path before running loops. Fixes #43302 (#43431) 2018-07-31 11:51:32 -04:00