Security Hardening

Overview

Hardening jobs apply CIS benchmark configurations to your servers using Ansible automation. This process modifies system settings to comply with security standards.

Warning: Hardening modifies system configurations. Always test on non-production servers first.

Hardening Profiles

ProfileDescriptionImpact
CIS Level 1 - ServerBasic security hardeningMinimal service impact
CIS Level 2 - ServerEnhanced security hardeningMay affect some services
CIS Level 1 - WorkstationBasic workstation securityFor desktop systems
CIS Level 2 - WorkstationEnhanced workstation securityFor high-security desktops

Running a Hardening Job

Single Host

  1. Go to Hosts page
  2. Find the target host
  3. Click Harden button
  4. Select hardening profile
  5. Set priority (1-10, higher runs first)
  6. Click Start Hardening

Bulk Hardening

  1. Go to Hosts page
  2. Select multiple hosts using checkboxes
  3. Click Harden Selected
  4. Select profile and priority
  5. Click Start Hardening All

Job Priority

Priority determines execution order when multiple jobs are queued:

PriorityDescription
1-3Low priority, runs after higher priority jobs
4-6Normal priority
7-9High priority, runs before lower priority jobs
10Critical, runs immediately

Job Status

StatusDescription
PendingJob is queued
QueuedJob is waiting for worker
RunningAnsible playbook executing
CompletedHardening finished successfully
FailedHardening encountered errors

Viewing Jobs

Jobs Page

The Jobs page displays all hardening jobs with:

  • Host name
  • Profile used
  • Status
  • Duration
  • Start time
  • Completion time

Job Details

Click View Output to see:

  • Job ID
  • Host information
  • Profile and priority
  • Ansible output (full execution log)
  • Error messages (if failed)

Understanding Ansible Output

The Ansible output shows:

  1. PLAY - Playbook execution start
  2. TASK - Individual configuration change
  3. ok - Task completed, no change needed
  4. changed - Task made a configuration change
  5. failed - Task encountered an error
  6. PLAY RECAP - Summary of all changes

Example Output

PLAY [Apply CIS hardening] ****************************************************

TASK [Ensure mounting of cramfs is disabled] **********************************
changed: [192.168.1.100]

TASK [Ensure mounting of freevxfs is disabled] ********************************
ok: [192.168.1.100]

PLAY RECAP ********************************************************************
192.168.1.100              : ok=95   changed=42   failed=0

AI Failure Diagnosis

When a job fails, use AI diagnosis:

  1. Find the failed job
  2. Click Diagnose with AI button
  3. Review the AI analysis:
    • Root cause identification
    • Recommended fix steps
    • Prevention tips

Viewing Diagnosis History

  1. Expand the job row
  2. View the AI diagnosis card
  3. Click View Full Diagnosis for details

Post-Hardening Steps

Verify Services

After hardening, verify critical services:

systemctl status sshd
systemctl status nginx
systemctl status your-application

Run Compliance Scan

  1. Go to Compliance page
  2. Run a new scan on the hardened host
  3. Compare scores before and after

Review Changes

Check the Ansible output for:

  • Configuration files modified
  • Services restarted
  • Packages installed/removed

Rollback

The platform does not provide automatic rollback. To reverse changes:

  1. Restore from backup (if available)
  2. Manually undo configuration changes
  3. Redeploy from configuration management

Best Practice: Snapshots

Before hardening:

  1. Create VM snapshot
  2. Run hardening
  3. Verify application works
  4. If issues, restore snapshot

Best Practices

Test First

  1. Clone production server
  2. Run hardening on clone
  3. Test all applications
  4. If successful, run on production

Schedule During Maintenance

  • Run hardening during maintenance windows
  • Have rollback plan ready
  • Monitor services after completion

Document Exceptions

Some rules may conflict with application requirements:

  • Document why rule is skipped
  • Get security team approval
  • Track in exception register

Progressive Hardening

  1. Start with Level 1
  2. Verify all services work
  3. Run compliance scan
  4. Consider Level 2 if needed

Troubleshooting

Job Stuck in Pending

  1. Check worker service is running
  2. Verify Redis connection
  3. Check worker logs

Job Failed

  1. Review Ansible output
  2. Check error messages
  3. Use AI diagnosis
  4. Common causes:
    • SSH connection lost
    • Sudo permission denied
    • Package installation failed
    • Disk full

Services Not Starting After Hardening

  1. Check systemd logs: journalctl -u service-name
  2. Review configuration changes
  3. Verify ports are not blocked
  4. Check if service was disabled by hardening

SSH Connection Lost After Hardening

If locked out:

  1. Use console access (if available)
  2. Boot into recovery mode
  3. Restore SSH configuration
  4. Consider excluding SSH rules next time