SQL

Deep Dive into Lookup Plugins in Ansible with Example

June 17, 2020 comments off

Lets understand about Lookup Plugins

  • Lookup plugins allow Ansible to access data from outside sources. This can include reading the filesystem in addition to contacting external datastores and services.
  • Like all templating, these plugins are evaluated on the Ansible control machine, not on the target/remote i.e Lookups occur on the local computer, not on the remote computer.
  • The data returned by a lookup plugin is made available using the standard templating system in Ansible, and are typically used to load variables or templates with information from those systems.
  • Lookups are an Ansible-specific extension to the Jinja2 templating language.
  • They are executed within the directory containing the role or play, as opposed to local tasks which are executed with the directory of the executed script.
  • You can pass wantlist=True to lookups to use in jinja2 template “for” loops.

These are used mainly by the template engine inside Ansible. They’re used in two ways.

First, in a function syntax to load external information:

{{lookup(pipe’,/usr/bin/whoami’)}}
{{lookup(etcd’,somekey’)}} – this allows you to fetch a key out of an NCD store.

Second, lookup plugins are also the source of with loops (with_items loads the items.py lookup plugin). Furthermore, they are always expected to return a list of items, because of their potential use with loops.

Purpose of Ansible Lookups

When it comes to automation, we handle different types of data and files such as csv, txt and sometimes we might even need to read data from a key-value store such as etcd or redis. That where the ansible lookup plugins are useful.

Location of lookup plugins directory

  • Default location: ~/.ansible/plugins/lookup:/usr/share/ansible/plugins/lookup
  • Another Location: ansible/plugins/lookup directory inside current ansible python package
  • Or with ANSIBLE_LOOKUP_PLUGINS environment variable

Enabling Custom Lookup Plugins

You can activate a custom lookup by either dropping it into…

  • A lookup_plugins directory adjacent to your play
  • inside a role, or
  • By putting it in one of the lookup directory sources configured in ansible.cfg.

How to Using Lookup Plugins?

Lookup plugins can be used anywhere you can use templating in Ansible:

  • in a play,
  • in variables file, or
  • in a Jinja2 template for the template module.

Command line to see Plugin List?


To see the list of available plugins.
$ ansible-doc -t lookup -l 

To see specific documents and examples
$ ansible-doc -t lookup <plugin name>

List of Lookup Plugins List

  • aws_account_attribute – Look up AWS account attributes.
  • aws_service_ip_ranges – Look up the IP ranges for services provided in AWS such as EC2 and S3.
  • aws_ssm – Get the value for a SSM parameter or all parameters under a path.
  • cartesian – returns the cartesian product of lists
  • chef_databag – fetches data from a Chef Databag
  • config – Lookup current Ansible configuration values
  • conjur_variable – Fetch credentials from CyberArk Conjur.
  • consul_kv – Fetch metadata from a Consul key value store.
  • credstash – retrieve secrets from Credstash on AWS
  • csvfile – read data from a TSV or CSV file
  • cyberarkpassword – get secrets from CyberArk AIM
  • dict – returns key/value pair items from dictionaries
  • dig – query DNS using the dnspython library
  • dnstxt – query a domain(s)’s DNS txt fields
  • env – read the value of environment variables
  • etcd – get info from etcd server
  • file – read file contents
  • fileglob – list files matching a pattern
  • filetree – recursively match all files in a directory tree
  • first_found – return first file found from list
  • flattened – return single list completely flattened
  • hashi_vault – retrieve secrets from HashiCorp’s vault
  • hiera – get info from hiera data
  • indexed_items – rewrites lists to return ‘indexed items’
  • ini – read data from a ini file
  • inventory_hostnames – list of inventory hosts matching a host pattern
  • items – list of items
  • k8s – Query the K8s API
  • keyring – grab secrets from the OS keyring
  • lastpass – fetch data from lastpass
  • lines – read lines from command
  • list – simply returns what it is given.
  • mongodb – lookup info from MongoDB
  • nested – composes a list with nested elements of other lists
  • nios – Query Infoblox NIOS objects
  • nios_next_ip – Return the next available IP address for a network
  • openshift – Query the OpenShift API
  • password – retrieve or generate a random password, stored in a file
  • passwordstore – manage passwords with passwordstore.org’s pass utility
  • pipe – read output from a command
  • random_choice – return random element from list
  • redis – fetch data from Redis
  • redis_kv – fetch data from Redis
  • sequence – generate a list based on a number sequence
  • shelvefile – read keys from Python shelve file
  • subelements – traverse nested key from a list of dictionaries
  • template – retrieve contents of file after templating with Jinja2
  • together – merges lists into syncronized list
  • url – return contents from URL
  • vars – Lookup templated value of variables

Method to invoke lookup plugins

  • Method 1 – using lookup
  • Method 2 – Using query

In Ansible 2.5, a new jinja2 function called query was added for invoking lookup plugins. The difference between lookup and query is largely that query will always return a list. The default behavior of lookup is to return a string of comma separated values. lookup can be explicitly configured to return a list using wantlist=True.

The following examples are equivalent:

lookup(‘dict’, dict_variable, wantlist=True)
query(‘dict’, dict_variable)

Example of Lookups Plugins

Rajesh Kumar
Follow me
Latest posts by Rajesh Kumar (see all)