{"id":15065,"date":"2020-06-17T21:06:18","date_gmt":"2020-06-17T21:06:18","guid":{"rendered":"https:\/\/www.devopsschool.com\/blog\/?p=15065"},"modified":"2025-07-12T05:54:51","modified_gmt":"2025-07-12T05:54:51","slug":"deep-dive-into-lookup-plugins-in-ansible-with-example","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/blog\/deep-dive-into-lookup-plugins-in-ansible-with-example\/","title":{"rendered":"Deep Dive into Lookup Plugins in Ansible with Example"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\">Lets understand about Lookup Plugins<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li>Lookup plugins allow Ansible to access data from outside sources. This can include reading the filesystem in addition to contacting external datastores and services.<\/li><li>Like all templating, these plugins are evaluated on the Ansible control machine, not on the target\/remote i.e Lookups occur on the local computer, not on the remote computer.<\/li><li>The data returned by a lookup plugin is made available using the standard templating system in Ansible, and are typically used to load variables or templates with information from those systems.<\/li><li>Lookups are an Ansible-specific extension to the Jinja2 templating language.<\/li><li>They are executed within the directory containing the role or play, as opposed to local tasks which are executed with the directory of the executed script.<\/li><li>You can pass wantlist=True to lookups to use in jinja2 template \u201cfor\u201d loops.<\/li><\/ul>\n\n\n\n<p>These are used mainly by the template engine inside Ansible. They\u2019re used in two ways.<\/p>\n\n\n\n<p><strong>First, in a function syntax to load external information:<\/strong><\/p>\n\n\n\n<p>{{lookup(pipe\u2019,\/usr\/bin\/whoami\u2019)}}<br>{{lookup(etcd\u2019,somekey\u2019)}} \u2013 this allows you to fetch a key out of an NCD store.<\/p>\n\n\n\n<p><strong>Second, lookup plugins are also the source of with loops <\/strong>(with_items loads the items.py lookup plugin). Furthermore, they are always expected to return a list of items, because of their potential use with loops.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Purpose of Ansible Lookups<\/h3>\n\n\n\n<p>When it comes to automation, we handle different types of data and files such as csv, txt and sometimes we might even need to read data from a key-value store such as etcd or redis. That where the ansible lookup plugins are useful.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Location of lookup plugins directory<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li>Default location: ~\/.ansible\/plugins\/lookup:\/usr\/share\/ansible\/plugins\/lookup<\/li><li>Another Location: ansible\/plugins\/lookup directory inside current ansible python package<\/li><li>Or with ANSIBLE_LOOKUP_PLUGINS environment variable<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Enabling Custom Lookup Plugins<\/h3>\n\n\n\n<p>You can activate a custom lookup by either dropping it into\u2026<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>A lookup_plugins directory adjacent to your play<\/li><li>inside a role, or<\/li><li>By putting it in one of the lookup directory sources configured in ansible.cfg.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">How to Using Lookup Plugins?<\/h3>\n\n\n\n<p>Lookup plugins can be used anywhere you can use templating in Ansible:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>in a play,<\/li><li>in variables file, or<\/li><li>in a Jinja2 template for the template module.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Command line to see Plugin List?<\/h3>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-1\" data-shcb-language-name=\"HTML, XML\" data-shcb-language-slug=\"xml\"><span><code class=\"hljs language-xml\">\nTo see the list of available plugins.\n$ ansible-doc -t lookup -l \n\nTo see specific documents and examples\n$ ansible-doc -t lookup <span class=\"hljs-tag\">&lt;<span class=\"hljs-name\">plugin<\/span> <span class=\"hljs-attr\">name<\/span>&gt;<\/span><\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-1\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">HTML, XML<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">xml<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">List of Lookup Plugins List<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li>aws_account_attribute &#8211; Look up AWS account attributes.<\/li><li>aws_service_ip_ranges &#8211; Look up the IP ranges for services provided in AWS such as EC2 and S3.<\/li><li>aws_ssm &#8211; Get the value for a SSM parameter or all parameters under a path.<\/li><li>cartesian &#8211; returns the cartesian product of lists<\/li><li>chef_databag &#8211; fetches data from a Chef Databag<\/li><li>config &#8211; Lookup current Ansible configuration values<\/li><li>conjur_variable &#8211; Fetch credentials from CyberArk Conjur.<\/li><li>consul_kv &#8211; Fetch metadata from a Consul key value store.<\/li><li>credstash &#8211; retrieve secrets from Credstash on AWS<\/li><li>csvfile &#8211; read data from a TSV or CSV file<\/li><li>cyberarkpassword &#8211; get secrets from CyberArk AIM<\/li><li>dict &#8211; returns key\/value pair items from dictionaries<\/li><li>dig &#8211; query DNS using the dnspython library<\/li><li>dnstxt &#8211; query a domain(s)\u2019s DNS txt fields<\/li><li>env &#8211; read the value of environment variables<\/li><li>etcd &#8211; get info from etcd server<\/li><li>file &#8211; read file contents<\/li><li>fileglob &#8211; list files matching a pattern<\/li><li>filetree &#8211; recursively match all files in a directory tree<\/li><li>first_found &#8211; return first file found from list<\/li><li>flattened &#8211; return single list completely flattened<\/li><li>hashi_vault &#8211; retrieve secrets from HashiCorp\u2019s vault<\/li><li>hiera &#8211; get info from hiera data<\/li><li>indexed_items &#8211; rewrites lists to return \u2018indexed items\u2019<\/li><li>ini &#8211; read data from a ini file<\/li><li>inventory_hostnames &#8211; list of inventory hosts matching a host pattern<\/li><li>items &#8211; list of items<\/li><li>k8s &#8211; Query the K8s API<\/li><li>keyring &#8211; grab secrets from the OS keyring<\/li><li>lastpass &#8211; fetch data from lastpass<\/li><li>lines &#8211; read lines from command<\/li><li>list &#8211; simply returns what it is given.<\/li><li>mongodb &#8211; lookup info from MongoDB<\/li><li>nested &#8211; composes a list with nested elements of other lists<\/li><li>nios &#8211; Query Infoblox NIOS objects<\/li><li>nios_next_ip &#8211; Return the next available IP address for a network<\/li><li>openshift &#8211; Query the OpenShift API<\/li><li>password &#8211; retrieve or generate a random password, stored in a file<\/li><li>passwordstore &#8211; manage passwords with passwordstore.org\u2019s pass utility<\/li><li>pipe &#8211; read output from a command<\/li><li>random_choice &#8211; return random element from list<\/li><li>redis &#8211; fetch data from Redis<\/li><li>redis_kv &#8211; fetch data from Redis<\/li><li>sequence &#8211; generate a list based on a number sequence<\/li><li>shelvefile &#8211; read keys from Python shelve file<\/li><li>subelements &#8211; traverse nested key from a list of dictionaries<\/li><li>template &#8211; retrieve contents of file after templating with Jinja2<\/li><li>together &#8211; merges lists into syncronized list<\/li><li>url &#8211; return contents from URL<\/li><li>vars &#8211; Lookup templated value of variables<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Method to invoke lookup plugins<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li>Method 1 &#8211; using lookup<\/li><li>Method 2 &#8211; Using query<\/li><\/ul>\n\n\n\n<p>In Ansible 2.5, a new jinja2 function called query was added for invoking lookup plugins. The difference between lookup and query is largely that query will always return a list. The default behavior of lookup is to return a string of comma separated values. lookup can be explicitly configured to return a list using wantlist=True.<\/p>\n\n\n\n<p>The following examples are equivalent:<\/p>\n\n\n\n<p>lookup(&#8216;dict&#8217;, dict_variable, wantlist=True)<br>query(&#8216;dict&#8217;, dict_variable)<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Example of Lookups Plugins<\/h3>\n\n\n\n<script src=\"https:\/\/gist.github.com\/devops-school\/b8c52929b15be7ed4e68bd2c66ebcc74.js\"><\/script>\n\n\n<div class=\"epyt-gallery\" data-currpage=\"1\" id=\"epyt_gallery_16311\"><iframe loading=\"lazy\"  id=\"_ytid_49788\"  width=\"760\" height=\"427\"  data-origwidth=\"760\" data-origheight=\"427\" src=\"https:\/\/www.youtube.com\/embed\/?enablejsapi=1&#038;autoplay=0&#038;cc_load_policy=0&#038;cc_lang_pref=&#038;iv_load_policy=1&#038;loop=0&#038;rel=1&#038;fs=1&#038;playsinline=0&#038;autohide=2&#038;theme=dark&#038;color=red&#038;controls=1&#038;disablekb=0&#038;\" class=\"__youtube_prefs__  no-lazyload\" title=\"YouTube player\"  data-epytgalleryid=\"epyt_gallery_16311\"  allow=\"fullscreen; accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen data-no-lazy=\"1\" data-skipgform_ajax_framebjll=\"\"><\/iframe><div class=\"epyt-gallery-list\"><div>Sorry, there was a YouTube error.<\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>Lets understand about Lookup Plugins Lookup plugins allow Ansible to access data from outside sources. This can include reading the filesystem in addition to contacting external datastores and services. Like&#8230; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_joinchat":[],"footnotes":""},"categories":[5153,1],"tags":[],"class_list":["post-15065","post","type-post","status-publish","format-standard","hentry","category-openshift","category-sql"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/15065","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=15065"}],"version-history":[{"count":3,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/15065\/revisions"}],"predecessor-version":[{"id":24753,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/15065\/revisions\/24753"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=15065"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=15065"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=15065"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}