Skip to content

Linux: Automation and Orchestration

Ansible IaC Core Concepts

Ansible let users automate system configuration and management using clear, repeatable commands. Ansible is agentless. I uses SSH on Linux and WinRM on Windows.

Installing Ansible

# on RHEL-based system
dnf install -y ansible-core

# on Debian-based system
apt install -y ansible

# to test install
ansible --version
ansible localhost -m ping

Inventory

Inventory is a list of all the servers or devices. Inventory can store as simple text file using INI format, as structured YAML file, or dynamically from cloud platforms or CMDBs.

[web] Group servers for easy management

ex inventory file

# ./hosts
[local]
localhost ansible_connection=local

# to install htop on RHEL-based system
ansible -i hosts local -m dnf -a "name=htop state=present update_cache=yes" --become

# to create a new user
ansible -i hosts local -m user -a "name=bob state=present" --become

# to copy a file from the control node to the managed nodes
ansible -i hosts local -m copy -a "src=my_config.conf dest=/apps/myapp.conf" --become

Ad Hoc Mode

Ad Hoc Mode is used to run one-time commands to test settings or apply changes across systems.

ex:

# to ping all hosts listed in the inventory
ansible all -m ping

# to restart the nginx service in the web group
ansible web -m service -a "name=nginx state=restarted"

Module

A module is a built-in tool that handles specific tasks like installing software, restarting services, or managing users.

ex of modules:

  • yum: used to install, update, remove packages on RHEL-based systems
  • apt: used to install, update, remove packages on Debian-based systems
  • user: used to manage user accounts on the system
  • service: used to start, stop, restart, or enable services
  • copy: used to transfer files from the control node to remote machines
  • file: used to create directories, change permissions, or delete files

Playbook

Playbook are complex, repeatable tasks, and structured automation. It is a structured YAML file that defines a set of tasks for Ansible to carry out on managed system.

Facts

Facts allows Ansible to automatically gather information about each machine and make decisions based on that data. Data collected can include IP addresses, operating system, available memory, and disk space. They are gathered at the beginning of playbook execution so it can decide what action to take based on the current setup of the machine. Ansible collects facts only when users run a task, using a direct connection like SSH.

Collections

Collections help manage and reuse tools, making it easier to scale and maintain automation environment over time.

Puppet Core Usage

Puppet helps automate system configuration by letting admins describe what the system should look like. Puppet is agent-based.

The Puppet Agent is responsible for communicating with the Puppet server and applying configurations. It is also responsible of collecting facts.

The Puppet server is called Puppet Master.

Facts

Facts are information Puppet collects on the managed devices such as operating system, hostname, IP addresses, memory, and more. Puppet Agent collects facts on a regular schedule during each check-in with the Puppet server. Puppet is well suited for large-scale enterprise environment because it enforces regular automated configuration.

Classes

Classes group related configuration tasks together into one logical unit. They help apply consistent settings to many systems with minimal duplication of effort.

Modules

A module is a package that includes everything needed to manage a specific task or part of a system. It can include one or many classes, files, templates, or custom facts.

Certificates

Certificates ensure that only authorized machines are allowed to talk to the server and receive configurations. The certificates must be approved and signed by the server before configurations are exchanged.

OpenTofu Core Usage

OpenTofu is an open-source designed to help manage and automate cloud infrastructure with code.

Provider

Provider connects configuration code to the actual cloud platform or service that the user is trying to manage. OpenTofu talks to services like AWS, Azure, and GCP using APIs.

Resources

Resources are the specific pieces of infrastructure user wants to create or manage, such as virtual machine, a firewall rule, or a storage bucket. OpenTofu resources focus on provisioning and configuring cloud services from the ground up.

State

The state is how OpenTofu keeps track of what is already been created in the environment.

Unattended Deployment

It is the automation of installation and initial configuration of systems to avoid manual step-by-step administrations.

Kickstart

Kickstart is commonly used in traditional data center environments with RHEL-based systems. You automate the RHEL-based installation by specifying things like language, disk setup, network settings, package selection in configuration file. The general syntax to start a kickstart install from a boot prompt is linux ks=<LOCATION OF KICKSTART FILE> inst.repo=<INSTALLATION SOURCE>

# to start a kickstart install
linux ks=http://192.168.10.10/kickstart/ks.cfg inst.repo=http://192.168.10.10/rhel8

Cloud-init

Cloud-init is the standard for automating deployments in cloud platforms like AWS, Azure, or OpenStack. It reads a YAML configuration file in order to apply the changes during the fire boot of a cloud instance.

ex:

# to create an install and configure using cloud-init
aws ec2 run-instances --image-id ami-0adfads185141422356 --instance-type t2.micro --user-data file://init-script.yaml

CI/CD Concepts

CI/CD is a system of tools and practices that brings order and automation to modern software development.

Version Control

Version control is a system that tracks changes to files over time, allowing developers to collaborate, review history, and roll back if something goes wrong. Git is the most common version control tool used today.

Pipelines

A pipeline is a sequence of automated steps that take code from commit to deployment. It might include testing, security scanning, building the software, deploying to production.

Modern CI/CD Approaches

Shift Left Testing

Shift left testing moves testing earlier in the development cycle, right alongside coding. The common tools used are Jenkins and GitLab CI.

DevSecOps

DevSecOps = Development, Security, and Operations. It is an approach that builds on CI/CD by embedding security practices throughout the software lifecycle.

GitOps

GitOps is a way of managing infrastructure and deployments using Git as the single source of truth. Common tools used are Argo CD and Flux.

Kubernetes Core Workloads for Deployment Orchestration

Kubernetes is an open-source platform that automates the deployment, scaling and management of containerized applications.

Pods

Pods are where the applications run. They allow users to tightly couple containers that need to work together. Containers that run in the same pods can talk to each other like they are running in the same machine.

Deployments

Deployments make sure the right number of Pods are up and are kept up to date. A deployment acts like a controller that keeps track of the application and ensures the right number of Pods are always running and up to date.

Services

Services ensure the application is reachable by other apps or users. They provide stable endpoint so other applications or users can reliably connect to the app regardless of which Pod is curring running.

Kubernetes Configuration

Variables

Variables are the simplest way to pass configuration settings into the containers.

ex:

# to tell the pod to use production settings
ENVIRONMENT=production

ConfigMaps

ConfigMaps store larger sets of configuration data in a Kubernetes object.

Secrets

Secrets work similarly to ConfigMaps, but are specifically designed to store sensitive data such as passwords, API tokens, SSH keys, or SSL/TLS certificates. The defaults, secrets are encoded in base64. Kubernetes uses RBAC to control access to these secrets.

Volumes

Volumes provide a way for containers to store and access data that needs to persist beyond the life of a single container.

Docker Swarm Core Workloads for Deployment Orchestration

Docker Swarm is a tool that helps orchestrate container deployments, making sure everything runs reliably.

Nodes

A node is a physical or virtual machine that is part of the swarm cluster. A node runs the Docker engine and is classified as Manager Node which make decisions and assign tasks, or Worker Node which carry out the tasks.

Tasks

A task is the actual instance of a container running on a node. Each task maps to exactly one container, and Swarm monitors them all continuously. Pods can host multiple containers, while a swarm task maps one-to-one. Tasks help ensure that the application stays running as expected.

Service

A service is a top-level object in Docker Swarm that defines how the application runs.

Docker Swarm Configuration

Networks

Networks defines how containers communicate within the swarm.

Overlay Networks

Overlay Networks are virtual networks that span across all nodes in the swarm. They enable secure, seamless communication between containers on different nodes.

ex docker-compose.yaml

# define an overlay network called frontend.

networks:

  frontend:

    driver: overlay

Scaling

Scaling refers to how many replicas of a service are running at any given time.

ex docker-compose.yaml:

services:

  web:

    image:nginx

    deploy:

      replicas: 3

Docker/Podman Compose for Deployment

Compose file

docker-compose.yaml

version: "3.8"

services:
  web:
    image:nginx
    ports:
      - "8080:80"

  app:
    image:my-web-app:latest
    environment:
      - ENV=production
    depends_on:
      - db

  db:
    image:postgres
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  db-data:

Podman Compose is more a security-focused container engine.

up and down commands

# to start or bring down containers
docker-compose up
docker-compose down

# to start or bring down containers
podman-compose up
podman-compose down

Viewing Logs

Viewing logs is essential for understanding what's happening in the application.

# to view logs
docker-compose logs
podman-compose logs

# to tail logs
docker-compose logs --follow web