# Troubleshooting

> Troubleshoot common SecureBaseline Cloud issues - SSH connection failures, scan errors, agent connectivity problems, and hardening job diagnostics

Source: https://opennix.org/en/docs/haas/troubleshooting/

## Overview

This guide covers common issues and their solutions when using SecureBaseline Cloud.

## Navigation

This guide is accessible from any system page via the help menu.

## Connection Issues

### Cannot Connect to Host

**Symptoms**: Host shows offline, SSH connection fails

**Solutions**:
1. Verify host is running and reachable
2. Check SSH service is running: `systemctl status sshd`
3. Verify firewall allows SSH: `sudo ufw status` or `sudo firewall-cmd --list-all`
4. Test connection manually: `ssh user@host`
5. Check SSH credential is correct
6. Verify SSH port is correct (default: 22)

### SSH Authentication Failed

**Symptoms**: Permission denied, authentication failure

**Solutions**:
1. Verify username is correct
2. For password auth: confirm password is correct
3. For key auth: verify public key is in `~/.ssh/authorized_keys`
4. Check SSH config allows this authentication method
5. Verify user has shell access
6. Check `/var/log/auth.log` for details

### Host Key Verification Failed

**Symptoms**: Host key mismatch error

**Solutions**:
1. If host was reinstalled, clear old key
2. Verify you're connecting to correct server
3. Check for man-in-the-middle concerns

## Scanning Issues

### Scan Stuck in Pending

**Symptoms**: Scan never starts, stays in pending state

**Solutions**:
1. Verify worker service is running
2. Check Redis connection
3. Review worker logs for errors
4. Restart worker service if needed

### Scan Failed

**Symptoms**: Scan shows failed status

**Solutions**:
1. Check SSH connectivity to host
2. Verify sudo permissions for scan user
3. Check disk space on target host
4. Review scan error message
5. Ensure the scanning engine is accessible

### Low Compliance Score

**Symptoms**: Score lower than expected

**Solutions**:
1. Review failed rules
2. Check if profile matches host type
3. Verify system wasn't recently installed
4. Run hardening and rescan
5. Document false positives

### Missing Scan Results

**Symptoms**: Scan completed but no results shown

**Solutions**:
1. Refresh the page
2. Check scan status is "completed"
3. Verify database connectivity
4. Review API logs for errors

## Hardening Issues

### Job Stuck in Queue

**Symptoms**: Job never starts executing

**Solutions**:
1. Check worker service status
2. Verify Redis connectivity
3. Check worker concurrency setting
4. Review worker logs
5. Restart worker if needed

### Hardening Failed

**Symptoms**: Job shows failed status

**Solutions**:
1. Review Ansible output for errors
2. Use AI diagnosis feature
3. Common causes:
   - SSH connection lost
   - Sudo permission denied
   - Package repository unavailable
   - Disk space full
4. Fix the issue and retry

### Services Down After Hardening

**Symptoms**: Application or service stops working

**Solutions**:
1. Check service status: `systemctl status <service>`
2. Review journal: `journalctl -u <service>`
3. Identify which hardening rule affected it
4. Rollback if needed (restore from backup)
5. Document exception for future

### SSH Locked Out

**Symptoms**: Cannot SSH after hardening

**Solutions**:
1. Use console access (cloud provider console)
2. Boot into recovery mode
3. Restore SSH configuration:
   ```bash
   # Common fixes
   sed -i 's/PermitRootLogin no/PermitRootLogin yes/' /etc/ssh/sshd_config
   systemctl restart sshd
   ```
4. Consider excluding SSH rules

## Schedule Issues

### Schedule Not Running

**Symptoms**: Scheduled jobs don't execute

**Solutions**:
1. Verify schedule is enabled
2. Check cron expression is valid
3. Verify scheduler service is running
4. Check server time/timezone
5. Review scheduler logs

### Wrong Execution Time

**Symptoms**: Jobs run at unexpected times

**Solutions**:
1. Verify cron expression
2. Check server timezone
3. Cron uses server local time
4. Test with simpler expression

## UI Issues

### Page Not Loading

**Symptoms**: Blank page or loading forever

**Solutions**:
1. Refresh the page
2. Clear browser cache
3. Try different browser
4. Check browser console for errors
5. Verify API service is running

### Data Not Updating

**Symptoms**: Stale data displayed

**Solutions**:
1. Click refresh button
2. Check auto-refresh setting
3. Verify network connectivity
4. Clear browser cache

### Export Not Working

**Symptoms**: CSV/JSON download fails

**Solutions**:
1. Check popup blocker
2. Try different browser
3. Check browser downloads folder
4. Verify data exists to export

## Authentication Issues

### Cannot Login

**Symptoms**: Login fails with valid credentials

**Solutions**:
1. Verify email and password
2. Check account is active
3. Try password reset
4. Clear browser cookies
5. Check API service status

### Session Expired

**Symptoms**: Suddenly logged out

**Solutions**:
1. Login again
2. Check token expiration settings
3. Verify time sync between client and server
4. Review authentication logs

## Performance Issues

### Slow API Response

**Symptoms**: Pages load slowly

**Solutions**:
1. Check server resource usage
2. Review database performance
3. Check network latency
4. Adjust rate limiting if needed
5. Scale resources if necessary

### Jobs Processing Slowly

**Symptoms**: Jobs take too long

**Solutions**:
1. Check worker concurrency
2. Review resource usage
3. Check network to managed hosts
4. Consider scaling workers

## AI Issues

### AI Not Responding

**Symptoms**: Chat or diagnosis fails

**Solutions**:
1. Verify AI configuration is active
2. Test configuration connection
3. Check API key validity
4. Verify Yandex Cloud access
5. Review API limits

### Poor AI Responses

**Symptoms**: Unhelpful or incorrect answers

**Solutions**:
1. Rephrase question clearly
2. Provide more context
3. Try different model
4. Adjust temperature setting

## Getting Help

### must-gather Diagnostic Tool

The fastest way to collect all diagnostic information is the `must-gather` CLI tool, available on every SecureBaseline Cloud VM.

```bash
# Run on the VM (uses DATABASE_URL from environment)
/opt/haas/bin/must-gather

# Or specify database URL and output path
/opt/haas/bin/must-gather --db "postgres://haas:pass@localhost/haas?sslmode=disable" --output /tmp/diag.tar.gz

# Limit log lines collected (default: 1000)
/opt/haas/bin/must-gather --log-lines 500
```

The tool produces a tar.gz archive containing:

| Section | Contents |
|---------|----------|
| `system/info.json` | Hostname, OS, CPU, version, build time |
| `services/` | systemd status, docker ps, listening ports, disk usage, memory |
| `logs/` | journalctl and docker logs for API, Worker, Vuls server, nginx |
| `config/` | Environment files (secrets redacted), nginx configuration |
| `scap/` | SCAP content version and file listing |
| `database/stats.json` | Table counts, host status distribution, failed jobs/scans (24h), stale running jobs, LLM configs, recent scans and failures, database size |

Attach the archive when reporting issues.

### Log Locations

| Component | Log Location |
|-----------|--------------|
| API | `journalctl -u haas-api` or `docker logs haas-api` |
| Worker | `journalctl -u haas-worker` or `docker logs haas-worker` |
| Vuls Server | `journalctl -u haas-vuls-server` or `docker logs haas-vuls-server` |
| Frontend | Browser console (F12) |
| Database | PostgreSQL logs |
| Nginx | `/var/log/nginx/access.log`, `/var/log/nginx/error.log` |

### Health Checks

Verify services:

```bash
# API health
curl http://localhost:8080/healthz

# Readiness (checks DB and Redis)
curl http://localhost:8080/readyz

# Liveness
curl http://localhost:8080/livez

# Version
curl http://localhost:8080/version
```

### Reporting Issues

When reporting issues:

1. Run `must-gather` and attach the archive
2. Describe the problem clearly
3. Include steps to reproduce
4. Note expected vs actual behavior

## See Also

- [SSH credential configuration](/en/docs/haas/credentials/) - manage keys and passwords for connecting to servers
- [CIS Benchmark security hardening](/en/docs/haas/hardening/) - apply CIS rules and diagnose Ansible playbook failures
- [SecureBaseline Cloud platform overview](/en/docs/haas/) - full list of features and supported operating systems