Business Continuity and
Disaster Recovery plan - Subscriptionflow
Business Continuity & Disaster Recovery Plan
Client-Facing Document — Confidential — Compliance Scope: PCI DSS
1 Introduction
Purpose
This document describes how SubscriptionFlow (SF) maintains service continuity and recovers from disruptions to ensure that client operations are protected. It outlines our preparation, response processes, recovery targets, and communication commitments for any event that affects platform availability.
Scope
This plan covers all components of the SubscriptionFlow production platform, including the multi-tenant web application and API layer, subscription and billing data storage, background job processing, webhook ingestion and delivery, third-party integration services, and supporting infrastructure (DNS, CDN, monitoring).
2 Platform Overview
SubscriptionFlow is hosted entirely on Microsoft Azure (primary region: East US 2).
Web & Application Layer
Azure Virtual Machines (Ubuntu Linux)
Handles all tenant HTTP requests and business logic
Database
Azure MySQL Flexible Server (HA enabled)
Stores all tenant and subscription data
Cache & Queue
Azure Cache for Redis (Premium)
Manages background job queues and application caching
Load Balancer
Azure Application Gateway (WAF v2)
Distributes traffic; provides DDoS and WAF protection
DNS & CDN
Cloudflare
Global DNS, CDN, and DDoS protection layer
Monitoring
Azure Monitor, Grafana, Prometheus, Uptime Kuma
End-to-end observability from infrastructure to availability
3 Service Availability Commitment
99.9%
Platform Availability Target
Measured monthly across all production endpoints. Allowable downtime: ≤ 43.8 min/month.
Full Platform Outage — RTO
< 30 min
RPO: < 1 hour
Partial Outage — RTO
< 2 hrs
RPO: < 4 hours
Non-Critical Degradation — RTO
< 8 hrs
RPO: < 24 hours
4 Incident Classification & Response
| Severity | Definition | Examples |
|---|---|---|
| P1 — Critical | Complete platform unavailability or data integrity risk | All tenants unable to access the platform; suspected data loss |
| P2 — High | Significant feature degradation or single-node failure | Background job processing stopped; one web node down; payment webhooks not delivering |
| P3 — Medium | Limited impact; workarounds available | Ancillary integration degraded; performance degradation without data impact |
| Severity | Time to Acknowledge | Time to First Update | Update Cadence |
|---|---|---|---|
| P1 — Critical | 15 minutes | 30 minutes | Every 30 minutes until resolved |
| P2 — High | 30 minutes | 1 hour | Every hour until resolved |
| P3 — Medium | 4 business hours | 8 business hours | Daily update |
5 Business Continuity Procedures
| Component | Failure Behaviour | Continuity Mechanism |
|---|---|---|
| Single web node failure | Traffic automatically routed to healthy nodes | Azure Application Gateway health checks |
| Database primary failure | Automatic failover to standby (60–120 seconds) | Azure MySQL HA with standby replica |
| Background job workers stopped | Jobs remain queued safely; no data lost | Redis queue; workers restarted automatically |
| Redis unavailable | Async features degraded; sync data operations unaffected | Manual restart; Azure Premium tier SLA |
| Cloudflare / DNS issue | Direct access to origin remains available | Azure Application Gateway PIP as fallback |
6 Disaster Recovery Scenarios
1
Database Loss or Corruption
▼- Detect event via monitoring alerts (Azure Monitor / Uptime Kuma).
- If HA failover has not triggered automatically, initiate via Azure Portal.
- If data corruption is detected, restore to last known clean point using PITR.
- Reconnect application nodes to the restored database.
- Validate data integrity and resume normal operations.
- Notify clients per communication plan (§7).
2
Web Application Node Failure
▼- Azure Application Gateway detects the unhealthy node and removes it from rotation automatically.
- Remaining nodes absorb traffic; capacity is monitored.
- Failed node is restarted or replaced.
- Once healthy, the node is re-added to the load balancer pool.
3
Background Job Processing Failure
▼- Detect via Grafana queue-depth dashboard or monitoring alert.
- Restart worker processes.
- Confirm queue is draining normally.
- For extended outage, trigger time-sensitive jobs (renewals, invoices) manually.
4
Full Region Failure
▼- Activate incident bridge and notify all stakeholders.
- Provision replacement infrastructure in an alternate Azure region.
- Restore database from backup to the new region.
- Update DNS records via Cloudflare to point to new region.
- Validate all services and resume operations.
⚠ Cross-region recovery currently has a longer RTO (estimated 4–8 hours). Cross-region geo-redundant backups are on our infrastructure roadmap.
5
Third-Party Integration Failure (GoCardless, Zoho, Stripe, etc.)
▼- SubscriptionFlow platform continues operating normally.
- Affected webhook deliveries or API calls are retried per provider configuration.
- Clients are notified if integration downtime will affect their operations.
- Resolution is coordinated with the relevant third-party provider.
7 Communication Plan
| Event | Notification Method | Timeline |
|---|---|---|
| P1 incident declared | Email + Status Page | Within 15 minutes of detection |
| P2 incident declared | Email + Status Page | Within 30 minutes of detection |
| Incident update | Email + Status Page | Per cadence in §4.2 |
| Incident resolved | Email + Status Page | Within 30 minutes of resolution |
| Post-incident report | Within 48 hours of resolution | |
| Planned maintenance | Minimum 48 hours in advance |
📧 Incident Communication Template
Subject: [SubscriptionFlow] Service Notification — [Date]
We are currently [investigating / experiencing] a disruption affecting [scope].
Status: Investigating / Identified / Monitoring / Resolved
Impact: [Description of what is affected]
Start time: [UTC timestamp]
Next update: [UTC timestamp]
8 Data Protection & Backup Summary
| Data Type | Backup Method | Frequency | Retention | Recovery Granularity |
|---|---|---|---|---|
| Tenant subscription data | Azure PITR (automated) | Continuous | 30 days | Any second within retention window |
| Tenant subscription data | Daily snapshot backup | Daily | TBD | Per-day restore |
| Application configuration | Version-controlled (Git) | On every change | Indefinite | Per-commit |
| Monitoring & log data | Centralised log aggregation | Continuous | TBD | Per-event |
🔒 All production data is encrypted at rest using Azure-managed encryption and in transit using TLS 1.2 or higher.
9 PCI DSS Compliance
| Area | Status |
|---|---|
| Web application firewall (WAF) | Active — Azure Application Gateway WAF v2 |
| Data encryption in transit | TLS 1.2+ enforced on all client-facing endpoints |
| Data encryption at rest | Azure-managed encryption on all data stores |
| Access control | Role-based access; least privilege applied |
| Audit logging | Centralised log aggregation and monitoring |
| Vulnerability management | Regular patching; Azure Advisor reviewed |
| Network segmentation | Production isolated in dedicated Azure Virtual Network |
| PCI compliance | Fully |
10 Plan Governance & Testing
| Test | Frequency |
|---|---|
| Database PITR restore drill | Quarterly |
| Application Gateway failover test | Quarterly |
| Job worker failure and recovery drill | Monthly |
| Full DR tabletop exercise (team walkthrough) | Annually |
| Communication plan test | Semi-annually |
11 Definitions
RTO — Recovery Time Objective
Maximum acceptable time from incident onset to service restoration
RPO — Recovery Point Objective
Maximum acceptable data loss measured in time
PITR — Point-in-Time Restore
Database recovery feature allowing restore to any specific point within a retention window
WAF — Web Application Firewall
Security layer that inspects and filters web traffic for malicious requests
HA — High Availability
Architecture where a standby component automatically takes over if the primary fails
CDN — Content Delivery Network
Distributed network (Cloudflare) that accelerates delivery and absorbs DDoS attacks
P1 / P2 / P3
Internal severity classification for incidents — P1 is the most severe