Commit graph

990 commits

Author SHA1 Message Date
Peter Sprygada
9111efc975 adds two new regex filters for use in playbooks (#14696)
The first filter is regex_search which adds the ability to do a regex
search on a fact.  The filter supports returning either all capture groups
or a set of capture groups (either by index or named).

Example usage of regex_search filter

debug: msg="{{ out.stdout[0] | regex_search('image version. (?P<test>\d+).(\d+)', '\\g<test>', '\\2' ) }}"

The second filter is regex_findall with adds the capability to do a
regex findall on a fact or variable and return all occurances of a pattern.

debug: msg="{{ out.stdout[0] | regex_findall('vlan (\d+)' }}"

The filter supports two addtional keyword arguments, ignorecase=[true, false]
and multiline=[true, false]
2016-04-14 14:54:03 -04:00
Toshio Kuratomi
dcc5dfdf81 Controller-side module caching.
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.

Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data.  Should devise a separate test for the module and caching code
2016-04-12 08:01:07 -07:00
James Cammarata
2e55b3567b Also fix intermittent ssh error using pty's for paramiko
Related to #13876
2016-04-12 08:35:48 -04:00
Toshio Kuratomi
0ced20951f Use /var/tmp is user set remote_tmp below /var/tmp.
We want them to be able to influence this choice in case they want to
keep files out of /tmp (perhaps to conserve RAM if that is a tmpfs).
2016-04-11 21:03:13 -07:00
James Cammarata
44877b7c7e Don't use -tt for ssh connections when sudoable=False
Due to an apparent race condition while using pty's on a heavily loaded
system, rarely a request to create a temp directory returns an empty
string rather than the newly created path, causing an error. Disabling
forced pty's appears to resolve the issue, so this patch modifies the
mkdtemp remote call not use -tt as we're not escalating privileges and
thus no pty is required.

Fixes #13876
2016-04-11 23:22:13 -04:00
Toshio Kuratomi
6a97d6386f Always use system /tmp if system=True.
Otherwise unexpected errors can happen when the user sets remote_tmp.

Fixes #15372
2016-04-11 20:00:10 -07:00
Toshio Kuratomi
54acdd7ead Remove workaround for fixed bug. (#15340)
* Remove workaround for fixed bug.

The bug where PluginLoader required objects to directly inherit from
base_classes has been fixed.  Remove workaround from this strategy
plugin   Also switched to using super so that we don't have to modify
all of hte code anytime something like that happens.

* These should be to_uniocde because they're being sent to display()
2016-04-10 09:24:54 -07:00
Kishin Yagami
e4a6106ea5 Add debug strategy plugin (#15125)
* Add debug strategy plugin

* Fix Python 2-3 compatiblity issue

* Add document for debug strategy
2016-04-08 14:39:08 -04:00
peter.jang
65856f2231 added suport rsync protocol (#12197)
* added suport rsync protocol

* use startswith method for safety
2016-04-08 11:42:07 -04:00
James Cammarata
e5c4f88b71 Fix free strategy, which was incorrectly using _wait_on_pending_results
Broken in 210cf06d
2016-04-08 06:49:16 -04:00
Matt Davis
957b336cc1 skip fixup_perms for Powershell
action plugins will now skip _fixup_perms for Powershell. We'll have to come up with another way to do this at some point, but it's not necessary yet since we don't support become on Windows. Also added NotImplementedError throws to chmod/chown/set_facl operations on Powershell (instead of returning '') in case anyone tries to use them in the future.



fixes #15312
2016-04-07 07:27:01 -07:00
Brian Coca
5225048df9 capture attribute errors for invalid class plugins
also switched to py3 safe to_str
2016-04-07 10:16:35 -04:00
James Cammarata
040893a677 Adding a config option to allow disabling locale settings upon module exec
Fixes #15138
2016-04-06 12:19:22 -04:00
Toshio Kuratomi
edab8d338d Expand tilde after expanding vars 2016-04-05 22:27:07 -07:00
Toshio Kuratomi
4b0aa1214c Ziploader
* Ziploader proof of concept (jimi-c)

* Cleanups to proof of concept ziploader branch:

* python3 compatible base64 encoding
* zipfile compression (still need to enable toggling this off for
  systems without zlib support in python)
* Allow non-wildcard imports (still need to make this recusrsive so that
  we can have module_utils code that imports other module_utils code.)
* Better tracebacks: module filename is kept and module_utils directory
  is kept so that tracebacks show the real filenames that the errors
  appear in.

* Make sure we import modules that are used into the module_utils files that they are used in.

* Set ansible version in a more pythonic way for ziploader than we were doing in module replacer

* Make it possible to set the module compression as an inventory var

This may be necessary on systems where python has been compiled without
zlib compression.

* Refactoring of module_common code:

* module replacer only replaces values that make sense for that type of
  file (example: don't attempt to replace python imports if we're in
  a powershell module).
* Implement configurable shebang support for ziploader wrapper
* Implement client-side constants (for SELINUX_SPECIAL_FS and SYSLOG)
  via environment variable.
* Remove strip_comments param as we're never going to use it (ruins line
  numbering)

* Don't repeat ourselves about detecting REPLACER

* Add an easy way to debug

* Port test-module to the ziploader-aware modify_module()

* strip comments and blank lines from the wrapper so we send less over the wire.

* Comments cleanup

* Remember to output write the module line itself in powershell modules

* for line in lines strips the newlines so we have to add them back in
2016-04-05 11:06:17 -07:00
Matt Davis
6a3670b1f0 Merge pull request #15281 from jpic/fix_plugin_configuration
Regression fix: proper plugin configuration was reset
2016-04-05 10:00:40 -07:00
James Cammarata
0f2b1244d2 Make sure VariableManager has a view of HostVars
Fixes #15261
2016-04-05 11:32:28 -04:00
jpic
da175d0a57 Bugfix: proper plugin configuration was reset
Before this patch, if config was ['/some/path'] then it would enter the
else block and config would be set to [].

The regression this patch fixes was introduced by 700db154.
2016-04-05 14:41:49 +02:00
Toshio Kuratomi
c75b5ba85f Use to_bytes and to_str instead of str as str is unsafe. 2016-04-04 12:57:32 -07:00
Dag Wieers
055aae55f7 Implement own CSVRecoder and CSVReader from documentation
As detailed in the python manual:
https://docs.python.org/2/library/csv.html
2016-04-04 20:32:08 +02:00
Dag Wieers
ac75069ee0 Support different encoding types in csvfile lookup plugin
This fixes #15266.
2016-04-04 14:11:44 -04:00
nitzmahone
700db15452 fix unconfigured plugin path case
We tried to expand paths for None- ensure we're not passing [None] in that case, but [] instead.
2016-04-01 20:37:43 -07:00
Brian Coca
dbc5cc9d7a fix issue with plugin load path inconsistency
now assures it is always a list of paths and not just assumes it
this avoids issues of parsing a 'string path' and picking up '/' as
a valid path for plugin loader
2016-04-01 23:14:56 -04:00
Brian Coca
0bd6f3a5e8 correctly adds searchpath as list 2016-04-01 23:14:56 -04:00
Toshio Kuratomi
4ccea9905b Fix async for old style modules and don't send an args file for new-style modules 2016-04-01 20:04:25 -07:00
Dag Wieers
d46af7cda3 On change in diff-mode we don't need to see file content in task output
I was surprised to see complete file content in the (JSON) task output when
in diff-mode. Since we see the diff anyhow, there's no need to send everything
on screen.
2016-04-01 18:03:51 +02:00
Toshio Kuratomi
f29b8e461b Merge pull request #15151 from mattclay/accelerate-unicode
Add to_bytes to file paths for accelerate plugin.
2016-03-31 21:47:33 -07:00
Matt Davis
3f70d7a70d Merge pull request #15144 from mattclay/winrm-test
Add winrm tests and fix exposed unicode errors.
2016-03-31 16:30:45 -07:00
James Cammarata
0656d1466a Adds two variables to facts when a task fails and a rescue block starts
`ansible_failed_task`:
   Contains the task data, essentially a serialized view of the Task() object.
`ansible_failed_result`:
   Contains the result of the task failure, which is identical in function
   to registering the result. Doing so automatically like this saves the user
   from having to register every result in a block and then trying to figure
   out which result had the failure

Similar to the way try/except/finally work, these variables will not be
available in the `always` portion of a block unless there is a corresponding
`rescue` first.

Fixes #12341
2016-03-31 15:29:33 -04:00
James Cammarata
26a4761d0f Fix logic error in script action plug related to use of shell.exists 2016-03-31 15:21:45 -04:00
Matt Clay
262c341cda Add connection tests for winrm connection plugin.
These are the same tests used for the other connection plugins,
adapted to use winrm modules and Windows friendly paths.
2016-03-31 08:32:41 -07:00
Matt Clay
83e53cbb91 Add to_bytes to file paths for accelerate plugin. 2016-03-30 23:23:04 -07:00
James Cammarata
a8dbd3dee9 Pass the variable manager into load_list_of_blocks for include files
Fixes #15194
2016-03-31 00:44:41 -04:00
Brian Coca
4b3257c4fe Merge pull request #15208 from bcoca/constants_pathlist
centralize path list handling
2016-03-30 19:49:18 -07:00
Brian Coca
5dfc013e18 Merge pull request #14971 from bcoca/clear_facts
meta: clear_facts
2016-03-30 19:09:38 -07:00
James Cammarata
eab4ce19f3 Making the return parsing in ActionBase generic for reuse
Now also correctly parses the return value for the _low_level_execute_command
2016-03-30 12:45:21 -04:00
Brian Coca
92d154f4ef centralize path list handling
move it to constants and avoid duplicating code or misinforming
developers on the nature of the data
2016-03-30 07:16:09 -07:00
Brian Coca
3f6166d2bf always return rc 2016-03-30 07:15:51 -07:00
nitzmahone
05af5c88ea fix Mac chown/chmod -R issue, add error checks
The changes to chown/chmod were broken on Mac (-R was being appended to the end of the command- OSX requires it before the file list).

A number of base action remote setup commands were also blindly proceeding without checking for success. Added error raises for unrecoverable failure cases.
2016-03-29 07:44:09 -07:00
Brian Coca
04610106a3 Merge pull request #15173 from mattclay/issue6072
Support remote_user in jail connection plugin.
2016-03-28 09:22:00 -07:00
Brian Coca
e9a4526251 allow bypassing executable setting
if you set executable to an empty string this avoids passing one to the command constructor
fixes #14813
2016-03-28 08:02:27 -07:00
Brian Coca
c303ed24b5 Merge pull request #15159 from sigio/filter-ip4-hex
Add ip4_hex filter to convert ip-address to hex notation
2016-03-27 14:14:25 -07:00
Brian Coca
363384f517 Merge pull request #15174 from mattclay/conn-msg-host
Fix inconsistent/missing host names in messages.
2016-03-27 14:07:13 -07:00
Toshio Kuratomi
0fffb6c60c Merge pull request #15163 from ansible/synchronize-connection-vs-play_context
Use _connection instead of _play_context for information about the connection
2016-03-27 10:24:46 -07:00
Matt Clay
5fdc29e00f Fix inconsistent/missing host names in messages. 2016-03-25 21:11:53 -07:00
Matt Clay
b60062bdf9 Support remote_user in jail connection plugin.
Resolves #6072.
2016-03-25 20:15:52 -07:00
Brian Coca
28d20dbe53 moved 'path exists' function to shell
now it will work with powershell/winrm
2016-03-25 12:56:48 -07:00
Toshio Kuratomi
0cabd133ba have to always run dwim() on the path to get the full absolute path.
Fixes #14944
2016-03-25 12:09:30 -07:00
Toshio Kuratomi
e2f5762232 Use _connection instead of _play_context for information about the connection
If we're not delegating then we change _connection into a local
connection midway through the file but we don't change
_play_context.connection (no need to alter that).  When we later check
it in process_remote() we need to know the actual connection, not the
connection that we thought we were going to use at the start of run().
So we have to use _connection.transport in process_remote().  The rest
of the places could use either one (because we have not yet changed to
a local connection) but we go ahead and switch those to
_connection.transport as well to avoid confusion in the future.

Fixes https://github.com/ansible/ansible-modules-core/issues/3136
2016-03-25 09:25:19 -07:00
Mark Janssen -- Sig-I/O Automatisering
b1015d897d Re-add accidentaly removed comment 2016-03-25 15:33:35 +01:00