Deploying Perimeter 81 in a K8s environment - unexpected network conflicts

Having recently overseen the integration of Perimeter 81 into a multi-cloud Kubernetes deployment for a mid-sized fintech organization, I encountered a series of non-trivial network layer conflicts that significantly impacted both application performance and, crucially, our predictable cloud cost structure. The deployment was predicated on achieving secure, zero-trust access to internal services, but the observed behavior introduced latency spikes and egress cost anomalies that necessitated a deep architectural review.

The core environment consisted of EKS clusters (AWS) and GKE clusters (GCP), with the Perimeter 81 connector deployed as a DaemonSet for comprehensive node coverage. The primary conflicts manifested in three distinct areas:

* **Subnet and CIDR Overlap with Existing VPC Peering:** Our existing mesh network for inter-cluster communication utilized specific RFC 1918 ranges. The Perimeter 81 virtual network, upon establishment, created overlapping CIDR blocks (notably `10.128.0.0/9`), which conflicted with our peered VPCs in AWS. This caused routing table chaos, where traffic destined for a peered application service was incorrectly routed through the P81 tunnel, adding latency and unnecessary data processing charges.
* **Egress Traffic Hairpinning and Cost Impact:** Pods requiring public internet egress for API calls (e.g., to payment gateways) were, due to the DaemonSet's `iptables` rules, forcibly hairpinned through the P81 connector. This transformed what should have been standard, potentially Savings Plan-optimized cloud egress into "VPN egress" from Perimeter 81's egress points. This not only added 80-120ms of latency but also obfuscated the true source of cloud costs, breaking our granular cost allocation tags and making it impossible to attribute spend to the correct service or team.
* **Service Mesh Interference (Istio):** The injected `istio-proxy` sidecar and the P81 network interface contended for control over the pod's network namespace. The resulting rule precedence was undefined, leading to sporadic failures in mTLS connections between services within the mesh, while some health checks and metrics traffic leaked outside the intended secure channels.

Our initial, simplified DaemonSet configuration was as follows:
```yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: p81-connector
spec:
template:
spec:
hostNetwork: true
containers:
- name: connector
image: perimeter81/connector:latest
env:
- name: P81_API_KEY
valueFrom:
secretKeyRef:
name: p81-secrets
key: apiKey
securityContext:
privileged: true
capabilities:
add: ["NET_ADMIN", "SYS_ADMIN", "NET_RAW"]
```

The resolution required moving away from the DaemonSet model for specific worker node pools and adopting a more surgical, namespace-isolated deployment via dedicated proxies, alongside explicit route management to exclude peered VPC and public service CIDR blocks from the tunnel. This restored predictable networking and cost visibility.

I am interested in hearing from others who have deployed Perimeter 81 in complex, multi-network Kubernetes environments. Specifically:
* What strategies have you employed to avoid CIDR conflicts with existing cloud provider VPC architectures?
* How do you manage egress traffic flows to maintain clean cost attribution and avoid suboptimal, expensive routing paths?
* Has anyone successfully integrated P81 with a service mesh without agent conflict, perhaps through explicit `traffic.sidecar.istio.io/includeOutboundIPRanges` configuration?

- cost_cutter_ray