nsbl package

Submodules

nsbl.ansible_extensions module

class nsbl.ansible_extensions.AnsibleFilterExtension(environment)

Bases: jinja2.ext.Extension

identifier = 'nsbl.ansible_extensions.AnsibleFilterExtension'

nsbl.ansible_extensions.to_yaml(var)

nsbl.ansible_extensions.utils

alias of nsbl.ansible_extensions.AnsibleFilterExtension

nsbl.cli module

nsbl.cli.raise_error(exc)

nsbl.defaults module

nsbl.exceptions module

exception nsbl.exceptions.NsblException(message)

Bases: exceptions.Exception

Base exception class for nsbl.

nsbl.inventory module

class nsbl.inventory.NsblInventory(**init_params)

Bases: frkl.callbacks.FrklCallback

add_group(group_name, group_vars)

Add a group to the dynamic inventory.

Parameters:

• group_name (str) – the name of the group
• group_vars (dict) – the variables for this group

add_group_to_group(child, group)

Adds a group as a subgroup of another group.

Parameters:

• child (str) – the name of the sub-group
• group (str) – the name of the parent group

add_host(host_name, host_vars)

Add a host to the dynamic inventory.

Parameters:

• host_name (str) – the name of the host
• host_vars (dict) – the variables for this host

add_host_to_group(host, group)

Adds a host to a group.

Parameters:

• host (str) – the name of the host
• group (str) – the name of the parent group

callback(env)

Adds a new environment, and sorts it into the appropriate internal variable.

static create(config, default_env_type=u'group', pre_chain=None)

Convenience method to create a NsblInventory object out of the configs and a few optional parameters.

Parameters:

• config (list) – list of config items
• default_env_type (str) – the type a environment is if it is not explicitely specified, either ENV_TYPE_HOST or ENV_TYPE_GROUP
• pre_chain (list) – the chain of ConfigProcessors to plug in front of the one that is used internally, needs to return a python list

Returns:

the inventory object, already ‘processed’

Return type:

NsblInventory

extract_vars(inventory_dir)

Writes a folder structure with ‘group_vars’ and ‘host_vars’ folders into the target directory.

Parameters:inventory_dir (str) – the directory the inventory should be written to

finished()

get_inventory_config_string()

Returns a string that can be used to write an ansible hosts file, including hosts, groups and child-groups.

get_vars(env_name)

Returns all variables for the environment with the specified name.

First tries whether the name matches a group, then tries hosts.

Parameters:env_name (str) – the name of the group or host Returns:the variables for the environment Return type:dict

host(host)

Returns the inventory information for the specified host, in the format required for ansible dynamic inventories.

Parameters:host (str) – the name of the host

Returns: dict: all inventory information for this host

list()

Lists all groups in the format that is required for ansible dynamic inventories.

More info: https://docs.ansible.com/ansible/intro_dynamic_inventory.html, http://docs.ansible.com/ansible/dev_guide/developing_inventory.html

Returns: dict: a dict containing all information about all hosts/groups

result()

write_inventory_file_or_script(inventory_dir, extract_vars=False, relative_paths=True)

Writes an ansible hosts file or dynamic inventory script into the provided directory.

Writing a dynamic inventory script is not implemented yet.

Parameters:

• inventory_dir (str) – the target directory
• extract_vars (bool) – whether to extract all vars (True, default) or write a dynamic inventory script
• relative_paths (bool) – only important for when writing dynamic inventory scripts, makes the paths in the script relative to the ansible environment root so its easily copy-able

class nsbl.inventory.WrapTasksIntoHostsProcessor(init_params=None)

Bases: frkl.processors.ConfigProcessor

Wraps a list of tasks into a localhost environment.

Convenience processor to not have to do this manually, keeps configuration files minimal and sweet.

handles_last_call()

process_current_config()

validate_init()

class nsbl.inventory.WrapTasksIntoLocalhostEnvProcessor(init_params=None)

Bases: frkl.processors.ConfigProcessor

Wraps a list of tasks into a localhost environment.

Convenience processor to not have to do this manually, keeps configuration files minimal and sweet.

handles_last_call()

process_current_config()

validate_init()

nsbl.inventory.parse_host_string(host_string)

nsbl.inventory_cli module

nsbl.nsbl module

class nsbl.nsbl.Nsbl(config, base_path=None, context=None, default_env_type=u'group', additional_files=None)

Bases: object

Holds and parses configuration to generate Ansible task lists and inventories.

Parameters:

• config (list) – a list of configuration items
• base_path (str) – the base path to use for relative task list imports etc.
• nsbl_context (NsblContext) – the context for this environment
• default_env_type (str) – the type a environment is if it is not explicitely specified, either ENV_TYPE_HOST or ENV_TYPE_GROUP
• additional_files (dict) – a dict of additional files to copy into the Ansible environment
• allow_external_roles (bool) – whether to allow the downloading of external roles

get_lookup_dict()

render(env_dir, global_vars=None, extract_vars=True, force=False, ask_become_pass=None, password=None, secure_vars=None, ansible_args=u'', callback=u'default', force_update_roles=False, add_timestamp_to_env=False, add_symlink_to_env=False, extra_paths=u'')

Creates the ansible environment in the folder provided.

Parameters:

• env_dir (str) – the folder where the environment should be created
• global_vars (dict) – vars to be rendered as global on top of a playbook
• extract_vars (bool) – whether to extract a hostvars and groupvars directory for the inventory (True), or render a dynamic inventory script for the environment (default, True) – Not supported at the moment
• force (bool) – overwrite environment if already present at the specified location, use with caution because this might delete an important folder if you get the ‘target’ dir wrong
• ask_become_pass (bool) – whether to include the ‘–ask-become-pass’ arg to the ansible-playbook call
• password (str) – if provided, it will be used instead of asking for a password
• secure_vars (dict) – vars to keep in a vault (not implemented yet)
• ansible_args (str) – parameters to give to ansible-playbook (like: “-vvv”)
• callback (str) – name of the callback to use, default: nsbl_internal
• force_update_roles (bool) – whether to overwrite external roles that were already downloaded
• add_timestamp_to_env (bool) – whether to add a timestamp to the env_dir – useful for when this is called from other programs (e.g. freckles)
• add_symlink_to_env (bool) – whether to add a symlink to the current env from a fixed location (useful to archive all runs/logs)
• extra_paths (str) – a colon-separated string of extra paths to be exported before the ansible playbook run

nsbl.nsbl.create_nsbl_env(urls, base_path=None, context=None, default_env_type=u'group', additional_files=None)

nsbl.nsbl_tasklist module

class nsbl.nsbl_tasklist.AugmentingTaskProcessor(**init_params)

Bases: frkl.processors.ConfigProcessor

Processor to augment a basic task list.

This will augment tasks that have a ‘name’ property which can be found in the task aliases list with the content of this task alias entry. Existing task properties won’t be overwritten.

This will also make sure that the task description has a ‘meta/task-name’ and ‘vars’ key/value pair.

process_current_config()

nsbl.nsbl_tasklist.GLOBAL_ENV_ID()

nsbl.nsbl_tasklist.GLOBAL_TASKLIST_ID()

class nsbl.nsbl_tasklist.NsblContext(allow_external_roles=None, allow_external_tasklists=None, **kwargs)

Bases: frkl.frklist.FrklistContext

class nsbl.nsbl_tasklist.NsblTasklist(tasklist, context=None, meta=None, vars=None)

Bases: frkl.frklist.Frklist

create_context(context_params)

ensure_freckles_task_list_format(task_list)

Make sure the task list is in ‘freckles’ format, if not, convert and add additional task file.

Parameters:

• task_list (list) – the task list
• env_id – the id of the environment (used to calculate the variable name)
• task_list_id – the id of the task_list inside the environment (used to calculate the variable name)

Returns:

tuple in the form of (final_task_list, external_files_dict)

Return type:

tuple

expand_and_augment_tasklist(tasklist)

fill_task_type(task_item)

Utility method to guess the type of a task and fill in all necessary properties.

Parameters:task_item (dict) – the task item

generate_nsbl_tasks_format(tasks_format={'child_marker': u'tasks', 'default_leaf': u'task', 'default_leaf_key': u'name', 'key_move_map': {u'*': (u'vars', u'default')}, u'use_context': True})

Utility method to populate the KEY_MOVE_MAP key for the tasks

parse_child_tasklists()

preprocess_tasklist(tasklist, context, meta, vars)

render_tasklist()

Renders the playbook into a file.

nsbl.output module

class nsbl.output.ClickStdOutput(display_sub_tasks=True, display_skipped_tasks=True, display_unchanged_tasks=True, display_ignore_tasks=[])

Bases: object

display_item(ev, current_is_dyn_role)

display_nsbl_item(ev, current_is_dyn_role)

display_result(ev, current_is_dyn_role)

format_error(msgs)

pretty_print_item(item)

print_error(line, error)

print_task_string(task_str)

process_role_changed(failed, skipped, changed, msgs, stdouts, stderrs)

process_task_changed(task_has_items, task_has_nsbl_items, saved_item, current_is_dyn_role)

start_env(env_name)

start_new_line()

start_role(current_role)

start_task(task_name, current_role, current_is_dyn_role)

class nsbl.output.CursorOff

Bases: object

class nsbl.output.NsblLogCallbackAdapter(lookup_dict, display_sub_tasks=True, display_skipped_tasks=True, display_unchanged_tasks=True, display_ignore_tasks=[])

Bases: object

add_error_message(line)

add_log_message(line, omit_errors=False)

finish_up()

flush()

write(line)

class nsbl.output.NsblPrintCallbackAdapter

Bases: object

add_error_message(line)

add_log_message(line)

finish_up()

flush()

write(line)

nsbl.output.get_debug_value_string(value)

nsbl.output.get_terminal_width()

nsbl.output.print_coloured(tokens, nl=True)

nsbl.output.print_debug_value(key, value, indentation)

nsbl.output.print_title(title_string, title_char='#')

nsbl.output.reindent(s, numSpaces)

nsbl.playbook_cli module

nsbl.playbook_cli.raise_error(exc)

nsbl.role_utils module

nsbl.role_utils.calculate_role_repos(role_repos)

Utility method to calculate which role repos to use.

Role repos are folders containing ansible roles, and an (optional) task description file which is used to translate task-names in a task config file into roles or ansible tasks.

Parameters:role_repos (list) – a string or list of strings of local folders containing ansible roles Returns:a list of all local role repos to be used Return type:list

nsbl.role_utils.find_roles_in_repo(role_repo)

Utility function to find all roles in a role_repo.

Parameters:role_repo – the path to the role repo

Returns: dict: a dictionary with the name of the role as key, and the path to the role as value

nsbl.role_utils.find_roles_in_repos(role_repos)

nsbl.runner module

class nsbl.runner.NsblRunner(nsbl)

Bases: object

run(target, global_vars=None, force=True, ansible_args=u'', ask_become_pass=None, password=None, secure_vars=None, callback=None, add_timestamp_to_env=False, add_symlink_to_env=False, no_run=False, display_sub_tasks=True, display_skipped_tasks=True, display_ignore_tasks=[], pre_run_callback=None, extra_paths=u'')

Starts the ansible run, executing all generated playbooks.

By default the ‘nsbl_internal’ ansible callback is used, which outputs easier to read outputs/results. You can, however, also use the callbacks that come with ansible, like ‘default’, ‘skippy’, etc.

Parameters:

• target (str) – the target directory where the ansible environment should be rendered
• global_vars (dict) – vars to be rendered on top of each playbook
• force (bool) – whether to overwrite potentially existing files at the target (most likely an old rendered ansible environment)
• ansible_args (str) – verbosity arguments to ansible-playbook command
• ask_become_pass (bool) – whether to include the ‘–ask-become-pass’ arg to the ansible-playbook call
• password (str) – if provided, it will be used instead of asking for a password
• secure_vars (dict) – other vars to keep secure, not implemented yet
• callback (str) – the callback to use for the ansible run. default is ‘default’
• add_timestamp_to_env (bool) – whether to append a timestamp to the run directory (default: False)
• add_symlink_to_env (str) – whether to add a symlink to the run directory (will be deleted if exists already and force is specified) - default: False, otherwise path to symlink
• no_run (bool) – whether to only render the environment, but not run it
• display_sub_tasks (bool) – whether to display subtasks in the output (not applicable for all callbacks)
• display_skipped_tasks (bool) – whether to display skipped tasks in the output (not applicable for all callbacks)
• display_ignore_tasks (list) – a list of strings that indicate task titles that should be ignored when displaying the task log (using the default nsbl output plugin – this is ignored with other output callbacks)
• pre_run_callback (function) – a callback to execute after the environment is rendered, but before the run is kicked off
• extra_paths (str) – a colon-separated of extra paths that should be exported for the nsbl run

Returns:

the parameters of the run

Return type:

dict

nsbl.task_alias_utils module

class nsbl.task_alias_utils.TaskAliasFolderReader(**kwargs)

Bases: luci.readers.MetadataFolderReader

is_usable_file(path)

Returns all files called ‘task-aliases.yml’

process_content(content, current_metadata, luci_metadata, luci_metadata_key_name=None)

Processes a set of lines in a certain format.

Returns:the values of the processed dictlet Return type:dict

class nsbl.task_alias_utils.TaskAliasLucifier(**kwargs)

Bases: luci.lucify.Lucifier

Class to read and merge task aliases into a single dictionary.

This can read both directories and files. If you add a directory dictlet, it will find and read all files called ‘task-aliases.yml’ recursively under that folder. It is not allowed to for multiple ‘task-aliases.yml’ files to contain the same alias key (with different values) as that would result in unpredictable behaviour depending on which file is read first.

If the path added is a file, it can be named anything, but needs to be in yaml format.

In both cases, the yaml files need to contain a single dictionary with the alias as key, and a metadata dict as value.

get_default_dictlet_finder()

get_default_dictlet_reader()

Returns the default dictlet reader for this lucifier.

process()

Processes all dictlets that were added to this Lucifier, one after the other.

process_dictlet(metadata, dictlet_details=None)

The main method implemented by a lucifier, it executes the task it is written for, using the metadata accrued as configuration.

Depending on the lucifier, this may return a string, an object, or nothing (and just prints out text to stdout).

Parameters:

• metadata – the metadata to process
• dictlet_details – metadata about the dictlet itself

Returns:

the new (processed) metadata

Return type:

object

nsbl.task_alias_utils.assemble_task_aliases(paths)

Helper method to assemble one task-alias dictionary from a list of paths.

Check TaskAliasLucifier for more details.

Parameters:paths (list) – a list of paths to directories or files Returns:the (merged) task aliases Return type:dict

nsbl.task_alias_utils.calculate_task_aliases(task_alias_files_or_repos, add_upper_case_versions=True)

Utility method to calculate which task descriptions to use.

Task descriptions are yaml files that translate task-names in a task config into roles or ansible tasks, optionally with extra default parameters.

If additional role_repos are provided, we will check whether each of them contains a file with the value of TASK_DESC_DEFAULT_FILENAME. If so, those will be added to the beginning of the resulting list.

Parameters:

• task_alias_files_or_repos (list) – a string or list of strings of local files or repos containing ‘task-aliases.yml’ files
• add_upper_case_versions (bool) – if true, will add an upper-case version of every task desc that includes a meta/become = true entry

Returns:

a list of dicts of all task description configs to be used

Return type:

list

nsbl.tasklist_utils module

nsbl.tasklist_utils.get_import_task_item(task_list_name)

Small helper toget the task item for importing a task list.

nsbl.tasklist_utils.get_task_list_format(task_list)

This is a not quite 100% method to check whether a task list is in ansbile format, or freckle.

nsbl.tasklist_utils.get_tasklist_file_format(path)

Module contents

View page source