Documentation

What is the RaaS Certificate Monitor?

The RaaS Certificate Monitor is a centralised dashboard that provides real-time visibility into the TLS/SSL certificate status across all Rhapsody-as-a-Service (RaaS) customer environments. It monitors certificates on both AWS and Azure deployments, covering Rhapsody engine certificates, ACM-managed load balancer certificates, and DigiCert-issued certificates.

Why does this exist?

• Expired certificates cause immediate service outages for customers’ HL7/FHIR integrations
• Manual tracking across 97+ accounts was error-prone and lacked visibility
• This dashboard automates detection, alerting, renewal, and audit trailing in one place

What does it monitor?

Roles & Permissions

Access is controlled via Microsoft Entra ID (Azure AD) single sign-on. There are two roles:

Dashboard Guide

The main dashboard (index.html) is the fleet overview. Here’s what you’ll see:

Fleet KPI Strip

The top row shows key performance indicators across the entire fleet: percentage of healthy certs, total warnings/critical/expired counts, the nearest upcoming expiry, and stale telemetry count.

Customer Tiles

Certificates are grouped by customer. Each customer tile shows:

• Customer name and product badge (RaaS, Envoy, Edge)
• Summary counts for each status (OK, Warning, Critical, Expired)
• Individual certificate rows with hostname, environment, cert type, expiry date, and days remaining

Search & Filters

• Search box — filters by customer name, hostname, environment, or cert type
• Region pills — filter by AWS region (shown when multiple regions have data)
• Product pills — filter by product type (RaaS, Envoy, Edge)
• Sort by expiry — toggle flat view sorted by nearest expiry

Export

Use the Export CSV or Export JSON buttons to download all warning, critical, and expired certificates from the current view. Useful for offline review or sharing with other teams.

Status Badges

Each certificate row has a status badge indicating its health:

Freshness Badges

Alongside the status, a freshness badge indicates how recent the telemetry is:

Customer 360

Clicking a customer tile on the dashboard opens the Customer 360 view. This provides a complete picture for a single customer:

• Health ring — visual indicator of overall cert health
• All accounts — every AWS/Azure account for this customer with their certs
• Salesforce cases — recent support cases raised for this customer
• Work orders — active Jira work orders related to cert renewals
• Timeline — audit history of renewal actions, restarts, and support cases

The Customer 360 is available to all users (Viewers and Admins).

Work Orders

The Work Orders page shows all Jira-tracked cert renewal work orders. These are created automatically when renewals are triggered, or manually by admins.

• Filter by status: Open, In Progress, Resolved, Closed
• Click a work order to see its full timeline and associated Jira ticket
• Admins can update work order status and add notes

Run History

The Run History page shows recent GitHub Actions workflow executions for rhapsody-certificates.yml. Each run corresponds to a cert renewal or recheck triggered from this dashboard or via schedule.

• See whether each run succeeded or failed
• View which customer/instance was targeted
• Click through to the full GitHub Actions log

How It Works

The monitoring system is fully automated with several components working together:

1. Instance Cert Checks (SSM State Manager)

Every day at 06:00 UTC, an SSM State Manager association runs on every Rhapsody instance (tagged Role=rhapsody) in each customer account. The script reads the certificate files on disk and the live certificate served on port 8444, calculates days until expiry, and writes the results to the central DynamoDB table (raas-cert-status) in the management account.

2. ACM Scanner (Daily Lambda)

A separate Lambda runs daily at 06:00 UTC and scans all AWS Certificate Manager (ACM) certs across every registered customer account. It uses cross-account IAM role assumption to enumerate ACM certificates and writes their status to the same DynamoDB table.

3. DigiCert Scanner

For customers using DigiCert-issued certificates, a scanner queries the DigiCert API to pull current certificate status and expiry information.

4. Alerter (Daily Lambda)

After cert data is collected, the alerter Lambda runs daily and checks for any certificates in warning, critical, or expired state. It sends alerts via:

• SNS — email notifications to subscribed ops team members
• Jira — automatic OPS ticket creation for critical/expired certs
• GitHub Actions — can trigger automated renewal workflows

5. Dashboard (This UI)

This dashboard reads from DynamoDB and presents the data in real time. It also provides the ability to trigger renewals, restarts, and other actions directly.

Certificate Lifecycle

Understanding the full lifecycle of a Rhapsody certificate from issuance to renewal:

1. Issuance — certificates are provisioned during initial deployment (Ansible playbook)
2. Monitoring — daily SSM checks report expiry status to DynamoDB
3. Warning — at 30 days before expiry, the dashboard shows WARNING status
4. Critical — at 14 days, the status escalates to CRITICAL and alerts fire
5. Renewal — an admin triggers rhapsody-certificates.yml from the dashboard
6. Restart Pending — new cert is on disk but the engine still serves the old one
7. Restart — a scheduled engine restart loads the new certificate
8. Verification — the next SSM check confirms the live cert matches the file cert

Workflow: Renewing a Certificate

This is the standard workflow for renewing an expiring certificate (admin only):

Workflow: Engine Restart

An engine restart is required after a cert renewal for the new certificate to be served. This action is admin only and requires explicit confirmation.

Admin Guide

Admins have access to additional pages and actions:

User Management

The Users page (admin only) allows you to:

• View all users — see who has signed in, their last active date, and login count
• Grant admin — promote a viewer to admin (adds to the SSM parameter list)
• Revoke admin — demote an admin back to viewer
• Disable/Enable — temporarily block a user’s access
• Delete — permanently remove a user
• Reset password — trigger a password reset email

Audit Trail

The Audit page shows every action taken on the dashboard: renewals, restarts, user management changes, page views, and more. Filter by user, action type, or customer. Export as CSV or JSON for compliance reporting.

Sanity Check

The Sanity Check page runs data quality diagnostics: missing telemetry, stale data, duplicate accounts, cert count mismatches, and other anomalies. Use this to verify the monitoring system is healthy.

Jira Ticket Creation

Admins can create OPS Cloud Ticket issues directly from the dashboard. These are automatically linked to the customer and cert context. Useful for tracking complex renewal or investigation work.

Request Admin Access

If you need admin access to perform certificate renewals, engine restarts, or user management, you can submit a request below. Choose your preferred method:

FAQ & Troubleshooting

Why does a cert still show WARNING after I renewed it?

The renewal puts the new cert on disk, but the Rhapsody engine still serves the old one until it is restarted. The rhapsody-wmc-live row reflects the live cert on port 8444. After restarting the engine, trigger a Recheck or wait for the next daily SSM run (06:00 UTC) to update the status.

What does RESTART PENDING mean?

This means the file cert (rhapsody-wmc-file) has been updated with a new certificate, but the live cert (rhapsody-wmc-live) hasn’t changed yet. The engine needs a restart to load the new file.

Why is a cert showing STALE?

The STALE badge means the telemetry data is more than 48 hours old. Common causes:

• The EC2 instance is stopped or terminated
• SSM Agent is not running on the instance
• The instance’s IAM role does not have the required permissions
• The SSM association is targeting the wrong tag value

Check the instance health in the AWS Console or contact an admin to investigate.

Can I renew certs as a Viewer?

No. Only admins can trigger renewals, restarts, and user management actions. Viewers can see all cert data, export reports, raise support cases, and view run history. If you need admin access, use the Request Admin Access form above.

How often is cert data refreshed?

The SSM cert check and ACM scanner both run daily at 06:00 UTC. Admins can trigger an immediate recheck for any instance using the Recheck button. The dashboard fetches the latest data from DynamoDB every time you load or refresh the page.

What happens if I accidentally trigger a renewal?

Renewals are safe to re-run. The Ansible playbook is idempotent — running it again will simply replace the cert file with a new one. No engine restart occurs unless you explicitly trigger one. All actions are recorded in the audit trail.

Who can see the audit trail?

Only admins can view the Audit Trail page. However, all users’ actions are recorded (including page views and sign-ins), not just admin actions.

How do I sign out?

Click your email address in the user bar at the top of any page, then click Sign out. This clears your session and redirects you to the Microsoft Entra ID sign-out page.