Ansible - Contributing Python Modules(Part-1)

I am a lazy person, which is why I like open source, for other people to do work for me.

I am very happy to share one of my Open Source Learnings here. I feel really proud of myself, by contributing some of the python modules to ansible open source software.

What is Ansible?

Simplify work; Break down silos; Speed transformation.

Ansible is an open-source IT automation engine, can automate Software Provisioning (setting up various servers that you need in your infrastructure), Configuration Management(change the configuration of an application, OS, enabling or disabling services, implement a security policy, install or update application, and other configuration tasks), and Application Deployment Tool(automating the deployment of internally developed applications).

Why Ansible? Why not other DevOps tools?

Working in IT, you’re likely doing the same tasks over and over. What if you could solve problems once and then automate your solutions going forward?

In this automation world there are many DevOps tools like Chef, Puppet, and Saltstack, below are the main advantages to use Ansible, compared to other DevOps tools,

• Agentless does not require any agents installed at the client machine. The only requirement is Ansible installed machine(Ansible control machine) with access to the server to be managed through SSH(for Linux) or WinRM(for Windows) protocols. Chef, Puppet, and Saltstack have a master and client structure, so whichever tool you are utilizing that needs to be configured on both server and client systems.
• Very low uptime, no agent for clients to be installed, so network up time and running time is low.
• It has a low learning curve everything expressed in YAML (Ain’t Markup Language) syntax.

Contribution Requirements for Ansible

Many hands make light work.

If you want to contribute Python or PowerShell module to ansible, you must have to meet the basic objective and subjective requirements defined by ansible.

• Objective Requirements
• Subjective Requirements
• Windows PowerShell Module Requirements

Python - Ansible Module

PYTHON - Programmers Yearning To Homestead Our Noosphere.

Creating Ansible modules is very easy. The Ansible module is a small piece of Python code, that can be triggered by the Playbook(YAML). Ansible has a lot of documentation and best practice to develop a complete module, It’s very important to follow before contributing your module to ansible. Otherwise, you may end up with many commands😢 or your module gets rejected😸 from the ansible core community. Below is the example format of developing ansible module for more details about developing ansible module from scratch,

#!/usr/bin/python
# -*- coding: utf-8 -*-
# Your License details.
from future import (absolute_import, division, print_function)
__metaclass__ = type
ANSIBLE_METADATA = {'metadata_version': '1.1',
                    'status': ['preview'],
                    'supported_by': 'community'}
DOCUMENTATION = r'''
---
module: your_module_name
short_description: your_module_short_description
version_added: '2.8'
description: your_module_long_description
options:
  attr_name:
    required: True
    type: str
    description: attr_description
requirements:
  - python >= 2.7.5
author:
  - Felix Stephen(@felsen)
'''
EXAMPLES = r'''
---
# Your example playbook 1
# Your example playbook 2
'''
RETURN = r'''
---
msg:
  type: (str/list/dict..)
  description: status of the return details
  returned: always
  sample: (example of what it return)
status:
  type: (str/list/dict..)
  description: status of the return details
  returned: success
  sample: (example of what it return)
'''
# Your ansible module imports
# Your python imports
# Your third party package imports
def main():
    module = AnsibleModule(argument_spec={attr: value,..})
    # All your business logics or create a new function and call
    # Any expected failule, module.fail_json(msg="failure_msg")
    # If success, module.exit_json(msg="success", status="str/list")
if __name__ == '__main__':
    main()

ANSIBLE_META block refers to the ansible module lifecycle, that has four states(preview stableinterface deprecated removed).

Testing the Ansible Module

Quality is never an accident; it is always the result of intelligent effort.

There are four types of tests needs to perform before contributing your ansible module to ansible community,

Sanity Tests is performed whenever new functionality, change in functionality or bug fix is implemented.

$ git clone https://github.com/ansible/ansible.git
$ cd ansible
# copy all your modules into the ansible library folder lib/ansible/modules/your_folder/..
$ pip install ansible-test
$ source hacking/env-setup
running egg_info
creating lib/ansible.egg-info
writing requirements to lib/ansible.egg-info/requires.txt
writing lib/ansible.egg-info/PKG-INFO
writing top-level names to lib/ansible.egg-info/top_level.txt
writing dependency_links to lib/ansible.egg-info/dependency_links.txt
writing manifest file 'lib/ansible.egg-info/SOURCES.txt'
reading manifest file 'lib/ansible.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'SYMLINK_CACHE.json'
writing manifest file 'lib/ansible.egg-info/SOURCES.txt'
Setting up Ansible to run out of checkout...
PATH=/home/username/Projects/ansible_2.0/validation/python-ansible-validation/ansible/bin:/root/.cargo/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
PYTHONPATH=/home/username/Projects/ansible_2.0/validation/python-ansible-validation/ansible/lib
MANPATH=/home/username/Projects/ansible_2.0/validation/python-ansible-validation/ansible/docs/man:/usr/local/share/man:/usr/share/man/overrides:/usr/share/man:/usr/man
Remember, you may wish to specify your host file with -i
Done!
# All sanity checks
$ ansible-test sanity lib/ansible/modules/your_folder/your_file.py
# Specific sanity checks
$ ansible-test sanity --test pep8 lib/../your_folder/your_file.py
# Specific to python version
$ ansible-test sanity --python 2.7 lib/../your_folder/your_file.py
# Specific test for python version
$ ansible-test sanity --test pep8 --python 2.7 lib/../your_file.py

The ansible-test commands will provide the results with ERROR😢WARNINGS😿or in case of success, 😄

...
Sanity check using no-tests-as-filters
Sanity check using no-underscore-variable
Sanity check using no-unicode-literals
Sanity check using pep8
ERROR: Found 3 pep8 issue(s) which need to be resolved:
ERROR: lib/ansible/modules/remote_management/../your_file.py:39:27: W291 trailing whitespace
ERROR: lib/ansible/modules/remote_management/../your_file.py:124:1: E302 expected 2 blank lines, found 1
ERROR: lib/ansible/modules/remote_management/../your_file.py:125:16: E225 missing whitespace around operator
Sanity check using pslint
Sanity check using pylint
ERROR: Found 1 pylint issue(s) which need to be resolved:
ERROR: lib/ansible/modules/remote_management/../your_file.py:125:15: bad-whitespace Exactly one space required around assignment     MSG_CREATED='Ethernet Network created successfully.'
Sanity check using replace-urlopen
Sanity check using required-and-default-attributes
...
WARNING: Cannot perform module comparison against the base branch. Base branch not detected when running locally.
ERROR: Found 1 validate-modules issue(s) which need to be resolved:
ERROR: lib/ansible/modules/remote_management/../your_file.py:0:0: E326 Argument 'state' in argument_spec defines choices as ([...]) but documentation defines choices as ([])
Sanity check using yamllint
...

ansible-test --test {choice} the valid choices are, action-plugin-docs ansible-doc azure-requirements boilerplate botmeta changelog compile configure-remoting-ps1 docs-build empty-init import integration-aliases line-endings no-assert no-basestring no-dict-iteritems no-dict-iterkeys no-dict-itervalues no-get-exception no-illegal-filenames no-main-display no-smart-quotes no-tests-as-filters no-underscore-variable no-unicode-literals pep8 pslint pylint replace-urlopen required-and-default-attributes rstcheck sanity-docs shebang shellcheck symlinks test-constrains use-argspec-type-path use-compact-six validate-modules yamllint

Compile Tests

# For all compile test
$ ansible-test sanity --test compile
$ ansible-test sanity --test compile lineinfile
# Specific to python version
$ ansible-test sanity --python 3.6 --test compile
Sanity check using compile with Python 2.6
Sanity check using compile with Python 2.7
Sanity check using compile with Python 3.5
Sanity check using compile with Python 3.6
Sanity check using compile with Python 3.7
Sanity check using compile with Python 3.8

Integration Tests

# Testing whole codebase
$ ansible-test integration
# Testing with specific modules
$ ansible-test integration -v ping

Units Test

# Testing whole codebase
$ ansible-test unints --tox apt
# Testing with specific python version
$ ansible-test units --tox apt --python 3.6

Once, your module is passed with all tests without any ERROR or WARNINGthat the module is ready for contribution.

Thanks for reading, check out my other blog Python — List elements in Memory.