While implementing Twingate across our infrastructure, a critical requirement from our security team was establishing a robust "break glass" procedure for emergency access. This is distinct from standard admin access; it must be isolated, audited, and usable when normal authentication mechanisms (like IdP connectivity) are impaired. A naive approach would simply create another admin account, but that violates the principle of least privilege and muddies audit trails. Here is the step-by-step methodology we deployed and subsequently load-tested.
Our design principles were:
* **Isolation:** The emergency pathway must be completely separate from the primary Twingate Connector network and user directory.
* **Dormancy:** It must have zero standing permissions and be activated only during a declared incident.
* **Auditability:** Every action, from activation to deactivation, must generate immutable logs.
* **Simplicity:** Under stress, the procedure must be executable with minimal decision points.
### Implementation Steps
1. **Create a Dedicated Emergency Twingate Network:** Do not use your primary production network. This network should have a single, dedicated Connector deployed in a secure but isolated VPC/subscription. Its only function is to provide emergency access to a defined jump-host or bastion instance.
```yaml
# Example Terraform for a minimal, isolated Connector (conceptual)
module "twingate_emergency_connector" {
source = "Twingate/connector/aws"
version = "~> 1.0"
# Attach to isolated VPC, different from production
vpc_id = module.emergency_vpc.vpc_id
subnet_ids = [module.emergency_vpc.private_subnet_ids[0]]
# Unique naming for clarity
name = "connector-emergency-breakglass"
}
```
2. **Define a Static Emergency Resource:** This resource should be a single, hardened bastion/jump host. Its firewall should only allow ingress from the IP address of the isolated emergency Connector.
3. **Configure a Local User for Authentication:** Since an IdP outage could trigger the emergency, we cannot rely on it. We created a Twingate Remote User authenticated via a long, complex, and securely stored password (using a PAM integration would be even better). This user account is **disabled by default** in Twingate.
4. **Establish a Policy of Zero Trust:** The emergency user has **no** resources assigned to it in the Twingate console under normal conditions. Access is granted purely via Just-In-Time (JIT) policy elevation.
### The Activation Workflow (Break Glass Procedure)
The actual "glass" is a script or a secured manual process. We automated it where safe, but the steps are:
* **Step 1: Declare Incident.** Follow your internal incident command process. This generates the first audit trail.
* **Step 2: Enable Emergency User.** A designated incident commander uses a pre-authorized, alternative method (e.g., a separate CLI tool with its own auth) to enable the dormant Twingate emergency user account.
* **Step 3: JIT Policy Activation.** Simultaneously, a Twingate Policy is activated that grants the now-enabled emergency user access *only* to the isolated emergency resource (the bastion host). This policy is time-bound (e.g., 4 hours).
* **Step 4: Access & Remediate.** Engineers can now authenticate as the emergency user, see the single bastion resource, and connect. All SSH sessions to the bastion are logged verbatim.
* **Step 5: Decommission.** Once the incident is resolved, the JIT policy is revoked, the emergency user is disabled, and a post-mortem reviews the access logs.
### Key Pitfalls & Testing Notes
* **Connector Scale:** Your emergency Connector does not need high scale. We load-tested a single `t3.small` instance handling 50 concurrent emergency connections, which was more than sufficient. The key is ensuring its compute environment is on a different failure domain than your primary Connectors.
* **Audit Log Saturation:** Ensure your SIEM can ingest the high-volume session logging from Twingate and the bastion host during an emergency without dropping events.
* **Procedure Decay:** This workflow must be tested quarterly. We simulate an IdP outage and execute the break-glass steps to verify no configuration drift has broken the chain.
The critical takeaway is that the emergency access is not a backdoor; it's a more controlled, auditable, and purpose-built pathway than the alternative of franticly modifying production permissions during a crisis.
-ck