Chapter 6
Hybrid and Enterprise Connectivity
Scope. This chapter connects Google Cloud to everything outside it: on-premises data centers, branch offices, and other clouds. It covers Cloud VPN and Cloud Interconnect in all their forms, BGP and Cloud Router design, redundancy topologies and the SLAs that depend on them, hub-and-spoke landing zones, hybrid DNS, and the connectivity monitoring that tells you whether any of it is working. VPC internals are Chapter 5; internet-facing delivery is Chapter 7. Prerequisites. Chapter 5, particularly §5.6 (address planning), §5.9–§5.12 (routes and dynamic routing), §5.21 (Cloud Router), §5.26 (Network Connectivity Center), §5.27 (Shared VPC). Verified against. Google Cloud console and API surface as of 2026-09, Cloud SDK 583.0.0; see sources at end.
Hybrid connectivity is where cloud network design stops being a greenfield exercise. The address plan is already decided by someone who left in 2019, the routers are owned by a team with a change window, and the failure modes are physical. It is also the area where the gap between "it works" and "it survives a failure" is widest, because a single tunnel or a single VLAN attachment works perfectly right up until it does not, and the SLA that would have covered you was contingent on a topology nobody built.
Two facts organize this chapter. The first is that Google's availability SLAs for hybrid connectivity are topology-contingent, not product-contingent. HA VPN is documented at 99.99% for most topologies — but the same product configured with a single active interface carries no availability SLA at all. Dedicated Interconnect offers 99.9% or 99.99% depending entirely on how many connections you buy and where you terminate them. Buying the product does not buy the number.
The second is that hybrid connectivity is a security boundary that most organizations treat as a trust extension. A VPN tunnel to the corporate network makes every on-premises system routable from the VPC and vice versa. If that sentence sounds like the design goal, the design is wrong: the on-premises network is not a trusted zone, and the whole point of Chapter 4 was that network position grants no authority. What crosses the boundary should be a deliberate, advertised, and firewalled list — and administrative access should not be on it at all, because that is what IAP is for (§4.19).
6.1 Hybrid Cloud Networking §
Hybrid networking on Google Cloud means one of three physical arrangements, and the choice is driven by bandwidth, latency, and how much you are willing to depend on the public internet.
| Option | Path | Typical bandwidth | Availability SLA |
|---|---|---|---|
| HA VPN | IPsec over the internet | ~1–3 Gbps per tunnel | 99.99% for supported topologies |
| Partner Interconnect | provider circuit to Google | 50 Mbps – 50 Gbps per attachment | 99.9% or 99.99% by topology |
| Dedicated Interconnect | your circuit into a Google colocation facility | 10 or 100 Gbps ports | 99.9% or 99.99% by topology |
The decision sequence:
- Is the traffic sensitive to internet variability? If yes, Interconnect. VPN throughput and latency depend on paths you do not control.
- Is the required bandwidth above a few Gbps? If yes, Interconnect. Adding VPN tunnels scales, but the operational cost rises quickly.
- Can you terminate a circuit in a Google colocation facility? If yes, Dedicated. If not, Partner.
- Is this temporary, a proof of concept, or a backup path? HA VPN, which provisions in minutes rather than weeks.
Design rules applied throughout this chapter:
- Terminate hybrid connectivity in the Shared VPC host project (§5.28), never in a workload project. One team owns the peering relationship.
- Advertise the minimum. Custom route advertisement on Cloud Router (§6.9) means on-premises learns only what it needs.
- Firewall the boundary explicitly. Traffic arriving over Interconnect or VPN is ingress like any other and gets an explicit rule set, not an allow-all.
- Use VPN as a backup for Interconnect where the SLA matters and a second circuit is not affordable — but understand that its bandwidth is an order of magnitude lower, so failover is a degradation, not a transparent event.
6.2 Cloud VPN §
Cloud VPN establishes IPsec tunnels between a Google Cloud VPN gateway and a peer gateway — an on-premises device, another cloud's VPN, or another Google Cloud VPN gateway.
Two products exist. HA VPN is the current one: a gateway with two interfaces, each with an external IP allocated automatically from separate pools, supporting two tunnels to the same peer gateway in active/active. Classic VPN has a single interface, requires manual IP and forwarding-rule configuration, cannot run two tunnels to the same peer, and supports no IPv6. Classic VPN carries a 99.9% SLA against HA VPN's 99.99% for supported topologies.
Use HA VPN for everything new. Classic VPN appears in this book only because you will encounter it in existing estates.
Protocol support (observed 2026-09): both support IKEv1 and IKEv2, but IKEv2 is required to carry IPv6 traffic on HA VPN. Each HA VPN tunnel handles up to 250,000 packets per second, which works out to roughly 1–3 Gbps depending on packet size.
gcloud compute vpn-gateways create vgw-prod-usc1 \
--project=rc-saas-shared-net-01 \
--network=vpc-prod-global \
--region=us-central1 \
--stack-type=IPV4_ONLY
Security posture. The shared secret is a long-lived symmetric credential. Generate it with a CSPRNG, store it in Secret Manager, never in a ticket or a Terraform variables file committed to Git, and rotate it on the same schedule you rotate other long-lived secrets. Choose IKEv2 with modern phase 1 and phase 2 parameters explicitly rather than accepting whatever the peer proposes.
Pitfall. IPsec provides confidentiality and integrity for the tunnel, not authorization for what travels through it. Every rule in §5.13 and §5.14 still applies to traffic arriving over a tunnel; a VPN is not an allow-list.
6.3 HA VPN §
HA VPN is the supported high-availability topology, and its SLA depends on building it correctly.
The gateway has two interfaces, 0 and 1, each with its own external IP from a separate pool. The SLA arithmetic is straightforward: a single active interface carries no availability SLA; two tunnels from both interfaces to a correctly configured peer reaches 99.99%.
Three supported peer topologies:
| Peer | Requirement for 99.99% |
|---|---|
| Two on-premises devices | one tunnel from each Cloud VPN interface to a different device |
| One on-premises device, two interfaces | one tunnel from each Cloud VPN interface to a different peer interface |
| One on-premises device, one interface | not eligible — this is the common mistake |
gcloud compute external-vpn-gateways create egw-onprem-dc1 \
--project=rc-saas-shared-net-01 \
--interfaces="0=203.0.113.10,1=203.0.113.11"
PSK=$(gcloud secrets versions access latest \
--secret=vpn-psk-dc1-if0 --project=rc-saas-shared-sec-01)
gcloud compute vpn-tunnels create tun-prod-usc1-dc1-if0 \
--project=rc-saas-shared-net-01 \
--region=us-central1 \
--vpn-gateway=vgw-prod-usc1 \
--interface=0 \
--peer-external-gateway=egw-onprem-dc1 \
--peer-external-gateway-interface=0 \
--ike-version=2 \
--router=rtr-prod-usc1 \
--shared-secret="${PSK}"
The second tunnel repeats with --interface=1 and --peer-external-gateway-interface=1, and both are attached to the same Cloud Router with distinct BGP sessions (§6.8).
Routing is BGP only. HA VPN requires Cloud Router; there is no static-route mode. That is a feature: failover is automatic and driven by BGP session state rather than by health-check heuristics.
Pitfall. Building both tunnels to the same peer interface. It looks redundant — two tunnels, two Cloud VPN interfaces — and it is not, because the peer device's single interface is a common failure point. The SLA topologies exist precisely to exclude this arrangement.
6.4 Cloud Interconnect §
Cloud Interconnect provides private physical connectivity between your network and Google's, bypassing the public internet entirely. Three forms exist, distinguished by who owns the physical connection.
| Form | Physical connection owned by | Reaches |
|---|---|---|
| Dedicated Interconnect | you, into a Google colocation facility | your VPC networks |
| Partner Interconnect | a service provider | your VPC networks |
| Cross-Cloud Interconnect | Google, to another cloud provider | another CSP's network |
Two resources in every form:
- The Interconnect connection — the physical circuit or the provider relationship.
- The VLAN attachment — a logical connection from that circuit to one VPC network in one region, terminating on a Cloud Router. Attachments are what actually carry traffic, and one circuit carries many.
MACsec for Cloud Interconnect encrypts traffic between your on-premises router and Google's edge router on supported circuits. Enable it: without it, the physical circuit is private but not encrypted, and "private" is a statement about routing, not about a fiber tap. For attachments, --encryption=IPSEC enables HA VPN over Interconnect for an additional encrypted layer.
Security posture. An Interconnect attachment is a hole in the perimeter that bypasses every internet-facing control you built in Chapter 7. Everything that crosses it is governed by Cloud Router advertisement (§6.9) and firewall policy (§5.14) and nothing else. Treat the attachment's creation as a change requiring the same review as an organization-level IAM binding.
Pitfall. Ordering an Interconnect is a lead-time item measured in weeks, and the ordering process involves a LOA-CFA exchange with the facility. Build the VPN path first so the project is not blocked, and keep it afterward as backup (§6.10).
6.5 Dedicated Interconnect §
Dedicated Interconnect is a direct physical connection between your equipment and Google's in a colocation facility. You order circuits, provide a letter of authorization, and cross-connect to Google's ports.
Capacity. Ports are 10 Gbps or 100 Gbps, and 100 Gbps connections can be requested at any location (observed 2026-09). Multiple circuits form a link aggregation bundle at one location.
gcloud compute interconnects create ic-dc1-primary \
--project=rc-saas-shared-net-01 \
--customer-name="Example Corporation" \
--interconnect-type=DEDICATED \
--link-type=LINK_TYPE_ETHERNET_100G_LR \
--location=iad-zone1-1 \
--requested-link-count=1 \
--noc-contact-email=neteng@rickcollette.domain \
--requested-features=IF_MACSEC \
--admin-enabled
gcloud compute interconnects attachments dedicated create att-prod-usc1-dc1 \
--project=rc-saas-shared-net-01 \
--region=us-central1 \
--interconnect=ic-dc1-primary \
--router=rtr-prod-usc1 \
--bandwidth=10g \
--vlan=100 \
--mtu=1500 \
--stack-type=IPV4_ONLY \
--enable-admin
SLA topology requirements. Google publishes three availability tiers: 99.99% for a critical-production topology, 99.9% for non-critical production, and no SLA for a single connection. The 99.99% topology requires connections in two different metropolitan areas, each with redundant circuits terminating in different edge availability zones, with attachments in two Cloud Routers. The 99.9% topology relaxes this to one metropolitan area with two edge availability zones. Confirm the exact current requirements against Google's topology documentation before committing to a number in a contract — the arrangement, not the product, is what carries the SLA.
Pitfall. --requested-features=IF_MACSEC must be set at connection creation on a supported circuit; adding MACsec later may require a new connection. Decide before ordering.
6.6 Partner Interconnect §
Partner Interconnect reaches Google through a service provider that already has connectivity into Google's network. You buy a circuit from the partner; the partner provisions a VLAN attachment; you activate it.
Capacity. VLAN attachments range from 50 Mbps to 50 Gbps (observed 2026-09), with the available increments depending on the partner and region.
The provisioning flow differs from Dedicated in one important way: you create the attachment first, receive a pairing key, hand it to the partner, and the partner completes the connection.
gcloud compute interconnects attachments partner create att-prod-usc1-partner-a \
--project=rc-saas-shared-net-01 \
--region=us-central1 \
--router=rtr-prod-usc1 \
--edge-availability-domain=availability-domain-1 \
--mtu=1440 \
--description="Partner Interconnect, availability domain 1."
Then retrieve the pairing key and give it to the partner:
gcloud compute interconnects attachments describe att-prod-usc1-partner-a \
--project=rc-saas-shared-net-01 \
--region=us-central1 \
--format="value(pairingKey,state)"
--edge-availability-domain is the redundancy control. For a 99.99% topology, provision two attachments in different availability domains (availability-domain-1 and availability-domain-2), ideally through two different partners or at least two different partner circuits.
Security posture. A partner sits in the path and terminates your Layer 2. That is a supply-chain dependency: the partner's operational security is part of your perimeter. Prefer Layer 3 partner connections only when you understand who holds the BGP session, and encrypt sensitive traffic at the application layer or with HA VPN over the attachment regardless.
Pitfall. The pairing key is a bearer credential for provisioning your attachment. Deliver it through a channel you trust, and confirm the partner activated the attachment you expected — the attachment stays in a pending state until you activate it after the partner completes their side.
6.7 Cross-Cloud Interconnect §
Cross-Cloud Interconnect provides a direct physical connection between Google's network and another cloud provider's, so multi-cloud traffic does not traverse the internet or terminate on equipment you operate.
What it gives you over the alternatives:
| Approach | Path | Operational burden |
|---|---|---|
| VPN over the internet | public internet | you run tunnels on both sides |
| Your own colocation transit | your routers in a facility both clouds reach | you own hardware |
| Cross-Cloud Interconnect | Google-provisioned circuit to the other CSP | Google provisions the physical layer |
Google provisions the physical connection; you configure a VLAN attachment on the Google side and the equivalent construct on the other cloud, then run BGP between them.
When it is worth it: sustained high-bandwidth data movement between clouds, cross-cloud database replication, and analytics that read from another cloud's storage. For occasional or low-volume traffic, HA VPN is cheaper and provisions immediately.
Security posture. A cross-cloud circuit connects two administrative domains that each have their own IAM, their own firewall model, and their own compromise history. Do not treat the far side as trusted:
- Advertise only the specific prefixes the other cloud must reach (§6.9).
- Apply a full firewall rule set to the attachment's traffic, deny-default in both directions.
- Prefer identity federation (§4.10, §4.11) over network reachability wherever the requirement is "workload A needs to call service B" — a federated API call needs no route at all.
- Consider Private Service Connect (§5.19) for service-specific access, which exposes one service rather than a network.
Pitfall. Reaching for network connectivity when the actual requirement is authentication. Most "we need to connect our clouds" requirements are satisfied by Workload Identity Federation and a public API endpoint, with far less attack surface than a circuit.
6.8 BGP §
All dynamic hybrid routing on Google Cloud is BGP, spoken by Cloud Router (§5.21) against your peer device.
The parameters you set on a session:
gcloud compute routers add-interface rtr-prod-usc1 \
--project=rc-saas-shared-net-01 \
--region=us-central1 \
--interface-name=if-tun-dc1-0 \
--vpn-tunnel=tun-prod-usc1-dc1-if0 \
--ip-address=169.254.10.1 \
--mask-length=30
BGP_KEY=$(gcloud secrets versions access latest \
--secret=bgp-md5-dc1 --project=rc-saas-shared-sec-01)
gcloud compute routers add-bgp-peer rtr-prod-usc1 \
--project=rc-saas-shared-net-01 \
--region=us-central1 \
--peer-name=peer-dc1-0 \
--interface=if-tun-dc1-0 \
--peer-asn=65001 \
--peer-ip-address=169.254.10.2 \
--advertised-route-priority=100 \
--md5-authentication-key="${BGP_KEY}" \
--bfd-session-initialization-mode=ACTIVE \
--bfd-min-receive-interval=300 \
--bfd-min-transmit-interval=300 \
--bfd-multiplier=3
ASN planning. Cloud Router uses a private ASN you choose (--asn at router creation, §5.21). Use one ASN per VPC per region consistently, from the 16-bit private range 64512–65534 or the 32-bit private range, and record the allocation. The peer's ASN is --peer-asn.
Three security controls belong on every session:
- MD5 authentication (
--md5-authentication-key, maximum 80 printable ASCII characters) so a device that can reach the link-local address cannot establish a session. - BFD (
--bfd-session-initialization-mode=ACTIVE) for sub-second failure detection. Without it, BGP hold timers mean 30–60 seconds of blackholing on a link failure — long enough to be an incident. - Import and export policies (
--import-policies,--export-policies) to filter what you accept and announce. A peer that advertises0.0.0.0/0or a prefix belonging to someone else should be rejected by policy, not discovered in an outage.
Link-local addressing. BGP sessions use 169.254.0.0/16 addresses, which is why that range must never appear in your VPC address plan (§5.7).
Pitfall. Route priority. --advertised-route-priority sets the MED Google announces; lower is preferred. Using the same priority on both a primary and a backup path produces ECMP across a fast circuit and a slow tunnel, which is almost never what anyone wanted (§6.10).
6.9 Cloud Router Design §
Cloud Router is regional, and its design decisions determine both reachability and blast radius.
One router per region per VPC, holding every hybrid session for that region. Do not create separate routers per tunnel; a single router with multiple BGP sessions is what makes ECMP and graceful failover work.
Custom advertisement is the security control. In default advertisement mode, the router advertises every subnet in scope. In custom mode, you list exactly what to announce:
gcloud compute routers update rtr-prod-usc1 \
--project=rc-saas-shared-net-01 \
--region=us-central1 \
--advertisement-mode=custom \
--set-advertisement-groups=all_subnets \
--set-advertisement-ranges=10.160.0.0/24
The Private Service Access case is the one everyone hits. Private Service Access is implemented as VPC peering and is not transitive (§5.18), so an on-premises host cannot reach a Cloud SQL instance by default. The fix is exactly the command above: advertise the allocated PSA range (10.160.0.0/24) explicitly with --set-advertisement-ranges. The same applies to Private Service Connect endpoint addresses and to GKE control plane ranges.
Advertisement can also be per BGP peer, using --set-advertisement-groups and --set-advertisement-ranges on add-bgp-peer / update-bgp-peer. Use it to give a partner a narrower view than your own data center gets.
Custom learned routes (--set-custom-learned-route-ranges, --custom-learned-route-priority) let you install routes for a peer that cannot or will not advertise them. Treat this as a workaround with an owner and a removal date; a manually installed route does not withdraw when the peer fails.
Pitfall. --set-advertisement-ranges replaces the list rather than appending to it. A change that adds one range and omits the existing ones silently withdraws them, and on-premises loses reachability to whatever they covered. Manage the full list in Terraform.
6.10 Redundant Connectivity §
Redundancy is the difference between the SLA you were quoted and the SLA you have. Build it deliberately at four layers.
| Layer | Failure it survives | How |
|---|---|---|
| Tunnel / attachment | one tunnel or VLAN attachment down | two per gateway or per circuit |
| Device | one peer router down | terminate on two peer devices (§6.3) |
| Circuit / facility | one circuit or colocation facility down | two circuits, two edge availability domains (§6.5, §6.6) |
| Metro | a metropolitan area down | connections in two metros — the 99.99% requirement |
| Product | Interconnect provider outage | HA VPN as a backup path |
The primary-with-VPN-backup pattern, which is the pragmatic choice for most enterprises:
- Interconnect attachment advertised with
--advertised-route-priority=100. - HA VPN tunnels to the same on-premises network advertised with
--advertised-route-priority=200. - Lower MED wins, so Interconnect carries traffic while it is up; BGP withdraws its routes on failure and the VPN takes over automatically.
Test the failover. A backup path that has never carried production traffic is a hypothesis. Schedule a controlled failover — administratively disable the Interconnect attachment during a maintenance window — and measure what actually happens: convergence time, throughput on the backup, and which applications notice. Do this at least annually.
Capacity honesty. An HA VPN tunnel carries roughly 1–3 Gbps. If the primary is a 10 Gbps Interconnect running at 4 Gbps, failover is a 60% capacity reduction, and the applications that degrade first should be known in advance rather than discovered.
Pitfall. Redundant paths that share a fate: two circuits from the same provider in the same building, two tunnels over the same internet uplink, or two peer routers on the same power distribution unit. Ask what is actually independent, not how many objects exist.
6.11 On-Premises Integration §
Connecting the network is the easy half. Integrating the environments raises four questions, and each has a specific answer in this book's design.
1. Addressing. On-premises 172.16.0.0/12 and cloud 10.128.0.0/12 (§5.6) must not overlap, and neither may overlap a range Google-managed services use. Where overlap exists, the options are renumbering or NAT, and NAT breaks any protocol that embeds addresses.
2. DNS. Bidirectional resolution requires inbound forwarding for on-premises to resolve cloud names and alternative name servers or a forwarding zone for cloud to resolve on-premises names (§6.17).
3. Identity. Cloud Identity federated from the on-premises IdP, with provisioning by SCIM or Google Cloud Directory Sync (§2.2, §4.4). This dependency deserves explicit failure analysis (§6.18).
4. Trust. The on-premises network is not a trusted zone. Concretely:
- Traffic arriving over Interconnect or VPN gets an explicit, deny-default firewall rule set (§5.14).
- Administrative access does not travel this path. Cloud resources are administered through IAP, not by reaching them from the corporate LAN (§4.19).
- Advertise only what on-premises needs, and expect only what the cloud needs in return (§6.9).
The migration anti-pattern to avoid. Lifting an on-premises flat network into the cloud, connecting it with a VPN, and allowing everything in both directions "for the migration." That configuration outlives the migration by years and makes the cloud estate exactly as segmented as the data center — which is to say, not.
Pitfall. MTU mismatch. Cloud VPN attachments and Interconnect attachments have configurable MTU (--mtu on the attachment; 1460 default on the VPC, §5.2), and a mismatch produces intermittent failures on large packets while pings succeed — the most time-consuming class of hybrid bug. Set MTU explicitly and consistently end to end.
6.12 Branch Connectivity §
Branch offices and retail sites have different economics from a data center: many sites, low bandwidth each, no on-site engineers, and no colocation presence.
Three viable patterns:
| Pattern | How | When |
|---|---|---|
| HA VPN per site | each branch terminates tunnels to a regional VPN gateway | tens of sites, in-house networking |
| SD-WAN with a cloud gateway | vendor appliance at each branch, virtual appliance or router appliance spoke in GCP | many sites, existing SD-WAN estate |
| Zero-trust, no network path | branches reach applications through IAP over the internet | new deployments, web and SSH workloads |
The third is the one to design for. If branch users need applications rather than networks, publishing those applications through IAP (§4.16, §7.17) means the branch needs only an internet connection. No tunnel, no appliance, no address plan, no route advertisement — and no lateral movement path from a compromised retail POS into the cloud estate.
Where a network path is genuinely required — a device speaking a legacy protocol, a store server replicating to a cloud database — terminate branches on a hub rather than meshing them (§6.13), and put each branch's prefixes in a firewall policy that permits only the specific flows.
Scaling consideration. Per-site VPN gateways and BGP sessions accumulate: each is a tunnel, an external gateway, a router interface, and a peer to monitor. Past a few dozen sites, Network Connectivity Center with router appliance spokes or an SD-WAN provider's integration is less operational load than hand-managed tunnels.
Pitfall. Branch networks frequently reuse the same RFC 1918 range at every site (192.168.1.0/24 everywhere). That is fine when each site is isolated and impossible when they all terminate on one hub. Renumber before connecting, not after.
6.13 Hub-and-Spoke Networks §
Hub-and-spoke centralizes connectivity and inspection so that spokes do not need to know about each other's paths.
Two implementations on Google Cloud:
Network Connectivity Center (§5.26) is the native one. A global hub takes VPC spokes, VPN tunnel spokes, Interconnect attachment spokes, and router appliance spokes, and exchanges routes among them — solving VPC peering's non-transitivity.
gcloud network-connectivity hubs create hub-rc-global \
--project=rc-saas-shared-net-01 \
--description="Hybrid and inter-VPC transit hub." \
--preset-topology=mesh
Shared VPC as the hub is the simpler alternative when spokes are workloads rather than separate networks: one VPC per environment holds all hybrid attachments, and workload projects attach as service projects (§5.27). No transit is needed because there is only one network.
Choose Shared VPC when your spokes are teams and workloads. Choose NCC when your spokes are genuinely separate networks — acquired companies, third-party managed environments, regulated subtrees with their own perimeter.
Security posture. A hub is a transit fabric, and transit is the thing segmentation exists to prevent. Every spoke's routes are visible to every other spoke unless you group them. The controls:
- Hierarchical firewall policy at the organization applying to every spoke VPC (§5.14).
- Spoke grouping and hub policy mode to limit which spokes exchange routes.
- An inspection spoke where flows between spokes must be examined (§5.36).
- Alerting on spoke creation — a new spoke is a new set of reachable networks.
Pitfall. All spokes must have non-overlapping address space. The hub is where an address plan violation from three years ago becomes a blocker (§5.6).
6.14 Enterprise Network Landing Zones §
A network landing zone is the network configuration a new workload lands into, provisioned before the workload exists and identically every time.
What a landing zone provides, per environment:
- A Shared VPC host project with the VPC and its subnets, sized from the address plan (§5.6, §5.28).
- Private Google Access on every subnet, with the
restricted.googleapis.comDNS override (§5.17). - Cloud NAT scoped to the workload subnets, with reserved static egress IPs (§5.20).
- Hierarchical and network firewall policies: deny-default east-west and egress, with the health-check and IAP allows (§5.14).
- Cloud Router and hybrid attachments, with custom advertisement (§6.9).
- Cloud DNS private zones and the DNS policy for hybrid resolution (§6.17).
- VPC Flow Logs and DNS query logging enabled (§5.4, §5.22).
- A subnet-scoped
roles/compute.networkUsergrant per service project (§5.28).
All of it is Terraform, invoked by the project factory (§2.26), so attaching a workload project is a terraform apply and not a ticket.
The enterprise variation. The enterprise reference estate (§2.33) adds a business-unit layer, so the landing zone is parameterized by BU: each BU gets its own subnet allocations out of the environment's supernet, its own networkUser grants, and its own firewall policy rules under the organization baseline it cannot override.
Pitfall. A landing zone that provisions network but not policy. Handing a team a subnet without the deny-default firewall policy, the flow logs, and the NAT scoping means the guardrails are added later by someone who has to negotiate for them. Ship the policy with the network.
6.15 Multi-Region Enterprise Networks §
A multi-region estate has to decide how hybrid traffic reaches each region, and the answer changes both cost and failure behavior.
Two topologies:
| Topology | Hybrid attachment | Dynamic routing mode |
|---|---|---|
| Centralized | one region, or two for redundancy | global (§5.12) |
| Regional | attachments in each region | regional |
Centralized is cheaper and simpler: one pair of Interconnects, global dynamic routing so every region's subnets are advertised and every learned route is installed network-wide. The cost is a hairpin — traffic from europe-west1 to on-premises crosses the backbone to us-central1 and out — plus a shared failure domain.
Regional puts an attachment in each region: lower latency, independent failure domains, and no hairpin, at the cost of more circuits and a more complex advertisement design. With regional mode, each Cloud Router advertises only its own region's subnets, so on-premises sees which region a prefix belongs to and can route accordingly.
The decision rule: centralized until either latency or the blast radius of a single-region hybrid failure becomes unacceptable. Most estates start centralized and add a second region's attachment when the first Interconnect outage takes out hybrid connectivity everywhere.
Route priority across regions. With regional attachments, use --advertised-route-priority to express locality: each region advertises its own prefixes at a low value and, if it advertises others as a backup, at a higher one. Without that, on-premises may reach europe-west1 workloads through the us-central1 circuit.
Pitfall. Switching a production VPC from regional to global routing mode reprograms routes network-wide immediately and can shift traffic paths without warning (§5.12). Check for prefix overlap between regions first, and do it in a window.
6.16 Multi-Cloud Connectivity §
Multi-cloud connectivity has three answers, and the network one should be the last considered.
1. Identity federation, no network path. Workload Identity Federation (§4.10, §4.11) lets an AWS or Azure workload call Google Cloud APIs directly over the internet with no keys and no tunnel. Almost all "we need to connect our clouds" requirements are of this shape. Attack surface: one API endpoint governed by IAM.
2. Private Service Connect. Where a specific service must be reachable privately, publish it via PSC (§5.19) rather than connecting networks. The consumer gets one endpoint; you keep your address space.
3. Network connectivity. HA VPN to the other cloud's VPN gateway for modest bandwidth, or Cross-Cloud Interconnect (§6.7) for sustained volume. This is the option that makes two clouds' networks mutually reachable, and it should require a written justification.
If you do connect networks:
- Address planning across all clouds. A single organization-wide plan covering GCP, AWS, Azure, and on-premises (§5.6). Overlap between two clouds discovered at connection time is a renumbering project.
- Deny-default both ways, with an explicit flow list.
- Independent identity. Do not federate one cloud's admin identity into the other's privileged roles; a compromise in one should not be a compromise in both.
- Separate monitoring, so a failure on the far side is visible without depending on the far side.
Pitfall. Symmetric trust. Teams configure a cross-cloud link and allow everything both directions because troubleshooting asymmetry is annoying. The result is that the security posture of both clouds becomes the weaker of the two.
6.17 DNS Integration §
Hybrid DNS needs resolution in both directions, and they are configured separately.
Cloud resolves on-premises names — a forwarding zone, or a DNS policy with alternative name servers:
gcloud dns managed-zones create zone-fwd-onprem \
--project=rc-saas-shared-net-01 \
--dns-name="corp.rickcollette.domain." \
--visibility=private \
--networks=vpc-prod-global \
--forwarding-targets=172.16.10.10,172.16.10.11 \
--description="Forward corp names to on-premises resolvers."
Use --private-forwarding-targets instead of --forwarding-targets when the resolvers must be reached over private connectivity rather than potentially over the internet — which, for on-premises resolvers reachable over Interconnect, is what you want.
On-premises resolves cloud names — a DNS policy with inbound forwarding, which allocates inbound forwarder IP addresses in each subnet that on-premises resolvers can query:
gcloud dns policies create policy-prod-inbound \
--project=rc-saas-shared-net-01 \
--networks=vpc-prod-global \
--enable-inbound-forwarding \
--enable-logging \
--description="Allow on-premises resolvers to query Cloud DNS private zones."
The on-premises resolver is then configured with a conditional forwarder for the cloud zone pointing at the inbound forwarder addresses, and the firewall must permit UDP and TCP 53 from the on-premises resolvers to those addresses.
A forwarding zone scoped to a subdomain beats a wholesale alternative name server. --alternative-name-servers on a DNS policy (§5.23) sends everything Cloud DNS cannot answer to on-premises, making the on-premises resolver a dependency for all external resolution and a single point of both failure and compromise. A forwarding zone for corp.rickcollette.domain. forwards only what belongs there.
Pitfall. DNS is the dependency that makes a hybrid outage total. If cloud workloads resolve through on-premises and the link fails, they cannot resolve anything — including the Google APIs they need to report the failure. Keep Google API resolution local (the private googleapis.com zone of §5.17), and scope forwarding to the narrowest subdomain that works.
6.18 Hybrid Identity Dependencies §
Federating identity to an on-premises IdP creates a runtime dependency that is easy to miss until it fails.
What actually depends on the IdP being reachable:
| Function | Depends on IdP | Notes |
|---|---|---|
Human console and gcloud sign-in | yes | SSO redirects to the IdP |
| PAM grant requests and approvals | yes | requires an authenticated human |
| Workload access via attached service accounts | no | tokens come from the metadata server |
| Pipeline access via WIF from external CI | no | depends on the CI system's IdP, not yours |
| Existing sessions | until expiry | which is why session length is a resilience parameter |
The failure mode. On-premises IdP or the Interconnect is down, so no human can authenticate to Google Cloud, so nobody can fix the outage. This is a real and recurring incident class, and the mitigations are specific:
- Break-glass identities that do not depend on the external IdP (§3.3). Cloud Identity accounts with their own credentials and hardware keys, excluded from SSO enforcement, monitored on every use.
- The IdP reachable from the internet, not only over the private link, so a circuit failure does not also break authentication.
- A federated IdP with its own redundancy, in two sites or as a SaaS product.
- Session lengths long enough to work through a short outage but short enough to satisfy your revocation requirement — an explicit trade-off, decided rather than defaulted.
- Directory synchronization is not on the critical path. SCIM or Google Cloud Directory Sync failing delays provisioning; it must not prevent existing users from signing in.
Test it. Simulate IdP unavailability and confirm the break-glass path works, from a machine outside the corporate network, with the person who would actually use it. An untested break-glass path is not a path (§3.36).
Pitfall. Break-glass accounts that are themselves in the federated domain and subject to SSO enforcement. They authenticate through the very system that is down.
6.19 Connectivity Monitoring §
Hybrid connectivity fails in ways that are invisible from inside a workload: a tunnel down while its partner carries the load, a BGP session flapping, an attachment at capacity.
What to monitor, and the signal each gives:
| Signal | Metric or source | Alert on |
|---|---|---|
| VPN tunnel state | vpn.googleapis.com/tunnel_established | any tunnel down, even with redundancy |
| VPN tunnel throughput | vpn.googleapis.com/network/sent_bytes_count | sustained near capacity |
| Interconnect attachment operational state | interconnect.googleapis.com/network/attachment/... | not operational |
| Interconnect capacity | attachment sent/received bytes | sustained above 70% of provisioned |
| BGP session state | router.googleapis.com/bgp/session_up | any session down |
| Advertised and learned route counts | router.googleapis.com/bgp/received_routes_count | a sudden change in either direction |
| Cloud NAT allocation failures | router.googleapis.com/nat/port_usage and NAT error logs | any allocation error (§5.20) |
Alert on redundant-component failure. The whole point of redundancy is that a single failure is invisible to users — which means it is also invisible to you unless you alert on the component. A tunnel that has been down for three weeks is a redundancy you no longer have.
Route count changes are a security signal. A jump in learned routes may be a route leak from a peer; a drop in advertised routes may be someone's --set-advertisement-ranges change that replaced the list rather than appending to it (§6.9).
Connectivity Tests (part of Network Intelligence Center, §6.20) verify a path configuration without sending traffic:
gcloud network-management connectivity-tests create test-app-to-onprem \
--project=rc-saas-shared-net-01 \
--source-instance=projects/rc-saas-prod-app-01/zones/us-central1-a/instances/api-01 \
--destination-ip-address=172.16.20.10 \
--destination-port=443 \
--protocol=TCP \
--round-trip
Pitfall. Monitoring only from inside Google Cloud. A path failure that affects on-premises reaching cloud but not the reverse is invisible to cloud-side probes. Run a probe from on-premises toward a cloud endpoint as well.
6.20 Network Intelligence Center §
Network Intelligence Center is the diagnostic suite for Google Cloud networking, and it turns several classes of debugging from guesswork into a query.
The modules and what each answers:
| Module | Question it answers |
|---|---|
| Connectivity Tests | would a packet from A to B be delivered, and if not, which rule or missing route stops it |
| Network Topology | what does my network actually look like, including traffic volumes between regions and to the internet |
| Performance Dashboard | what is the latency and packet loss between my projects' regions, and between them and Google |
| Firewall Insights | which firewall rules are shadowed, unused, or overly permissive |
| Network Analyzer | continuous automated detection of misconfigurations — IP exhaustion, shadowed rules, suboptimal routes, BGP problems |
| Flow Analyzer | interactive analysis over VPC Flow Logs |
Connectivity Tests is the one that pays for itself immediately. It evaluates the configuration — routes, firewall rules and policies, Cloud NAT, load balancers, hybrid attachments — and tells you which specific object drops a packet, without needing a VM at either end or any traffic. --round-trip checks the return path too, which catches asymmetric-routing problems that a one-way test would pass.
Firewall Insights is a security control, not only an operations one. Shadowed rules are rules that can never match because a higher-priority rule already decided, which means someone believes a control exists that does not. Unused allow rules are attack surface with no business justification. Review both on a schedule and remove what the data says is dead.
Network Analyzer runs continuously and surfaces findings into Security Command Center. Its highest-value detections for this book's design: subnet IP exhaustion before it causes an outage, firewall rules that permit more than intended, BGP sessions in a degraded state, and Private Service Access ranges approaching capacity (§5.18).
Console path. Console → Network Intelligence, with each module as a sub-page.
Pitfall. Connectivity Tests evaluates configuration, not liveness. A test can pass while the destination service is down, and it does not test the on-premises side of a hybrid path beyond the attachment. Pair it with an actual probe.
Chapter Summary §
- Availability SLAs for hybrid connectivity are topology-contingent, not product-contingent: HA VPN with a single active interface carries no availability SLA, and Interconnect's 99.99% requires connections in two metropolitan areas.
- HA VPN has two interfaces with automatically allocated external IPs and supports two active/active tunnels to one peer gateway; Classic VPN has one interface, no dual tunnels, no IPv6, and a 99.9% SLA.
- Both VPN types support IKEv1 and IKEv2, but IKEv2 is required to carry IPv6 on HA VPN; each HA VPN tunnel handles up to 250,000 packets per second, roughly 1–3 Gbps.
- HA VPN is BGP-only and requires Cloud Router, which is why failover is driven by session state rather than health-check heuristics.
- Cloud Interconnect has two objects: the connection (physical circuit or provider relationship) and the VLAN attachment (logical link to one VPC in one region, terminating on a Cloud Router).
- Dedicated Interconnect ports are 10 or 100 Gbps; Partner Interconnect attachments range from 50 Mbps to 50 Gbps; MACsec must be requested at connection creation.
- Partner Interconnect provisioning uses a pairing key you hand to the provider;
--edge-availability-domainis the redundancy control. - Cross-Cloud Interconnect connects Google's network directly to another CSP's, but most "connect our clouds" requirements are satisfied by Workload Identity Federation with no route at all.
- Every BGP session should carry MD5 authentication, BFD for sub-second failure detection, and import/export policies; link-local
169.254.0.0/16is why that range must never appear in your address plan. --set-advertisement-rangesreplaces the advertisement list rather than appending; an incomplete update silently withdraws prefixes.- Private Service Access is non-transitive, so reaching a Cloud SQL instance from on-premises requires explicitly advertising the allocated PSA range from Cloud Router.
- Route priority (
--advertised-route-priority, lower wins) is what makes an Interconnect primary and a VPN backup rather than an ECMP pair across wildly different capacities. - A backup path that has never carried production traffic is a hypothesis; schedule a controlled failover annually and measure convergence and capacity.
- The on-premises network is not a trusted zone: deny-default firewall rules on hybrid traffic, minimum advertisement, and administrative access through IAP rather than over the link.
- MTU mismatch between the VPC, the attachment, and on-premises produces intermittent large-packet failures while pings succeed.
- For branches, publishing applications through IAP removes the need for a network path entirely; where a path is required, terminate on a hub rather than meshing.
- Centralized hybrid attachment with
globaldynamic routing is simpler and hairpins traffic; regional attachments cost more and give independent failure domains. - Hybrid DNS needs a forwarding zone for cloud-to-on-premises and a DNS policy with inbound forwarding for the reverse; scope forwarding to a subdomain rather than making on-premises the resolver of last resort.
- Federated identity creates a runtime dependency: if the on-premises IdP is unreachable, nobody can sign in to fix the outage. Break-glass identities must not depend on that IdP.
- Alert on redundant-component failure, because redundancy makes single failures invisible; monitor BGP session state and route counts, where a sudden change is a security signal.
- Network Intelligence Center's Connectivity Tests evaluate configuration rather than liveness, and Firewall Insights finds shadowed and unused rules that represent controls that do not exist.
Security Checklist §
| Control | Why it matters | How to verify (CLI + Console) |
|---|---|---|
| HA VPN uses both gateway interfaces to two peer devices or two peer interfaces | A single active interface carries no availability SLA | gcloud compute vpn-tunnels list --format="table(name,vpnGateway,vpnGatewayInterface,peerExternalGateway,status)"; Console → Hybrid Connectivity → VPN |
| All tunnels use IKEv2 with explicitly chosen phase 1 and phase 2 parameters | Accepting the peer's proposal accepts the peer's weakest option | gcloud compute vpn-tunnels describe TUNNEL --region=REGION --format="value(ikeVersion)" |
| VPN shared secrets stored in Secret Manager, not in Git or tickets | They are long-lived symmetric credentials | inspect Terraform for literal shared_secret values |
| MACsec requested on supported Interconnect circuits | A private circuit is unencrypted without it | gcloud compute interconnects describe INTERCONNECT --format="value(requestedFeatures,availableFeatures)" |
| BGP sessions have MD5 authentication configured | Prevents session hijack by anything reaching the link-local address | gcloud compute routers describe ROUTER --region=REGION --format="value(bgpPeers[].md5AuthenticationKeyName)" (the key value itself is never returned) |
| BFD enabled on every BGP session | Without it, a link failure blackholes for BGP hold-timer duration | gcloud compute routers describe ROUTER --region=REGION --format="value(bgpPeers[].bfd.sessionInitializationMode)" |
| Cloud Router uses custom advertisement, not default | Default advertises every subnet in scope to the peer | gcloud compute routers describe ROUTER --region=REGION --format="value(bgp.advertiseMode,bgp.advertisedGroups,bgp.advertisedIpRanges)" |
| Import/export policies filter what peers may announce | A peer advertising 0.0.0.0/0 should be rejected by policy, not in an outage | gcloud compute routers describe ROUTER --region=REGION --format="value(bgpPeers[].importPolicies,bgpPeers[].exportPolicies)" |
| Hybrid traffic is subject to a deny-default firewall rule set | The on-premises network is not a trusted zone | gcloud compute firewall-rules list --filter="direction=INGRESS AND sourceRanges~172.16"; firewall policies (§5.14) |
| Interconnect and VPN terminate in the Shared VPC host project | One team owns the peering relationship | gcloud compute interconnects attachments list --format="table(name,region,router,network)" |
| Route priorities make the primary path primary | Equal priorities ECMP across a fast circuit and a slow tunnel | gcloud compute routers describe ROUTER --region=REGION --format="value(bgpPeers[].advertisedRoutePriority)" |
| Failover to the backup path tested within the last year | An untested backup is a hypothesis | change record for the last controlled failover |
| Alerting exists on individual tunnel and BGP session state | Redundancy makes single failures invisible | gcloud alpha monitoring policies list or Console → Monitoring → Alerting |
| MTU set explicitly and consistently on attachments and the VPC | Mismatches cause intermittent large-packet failure while pings pass | gcloud compute interconnects attachments describe ATTACHMENT --region=REGION --format="value(mtu)" |
| Break-glass identities do not depend on the on-premises IdP | An IdP or circuit outage otherwise prevents anyone from signing in to fix it | test sign-in from outside the corporate network with SSO enforcement in place |
| Hybrid DNS forwarding is scoped to a subdomain, not a wholesale alternative name server | Otherwise on-premises becomes a dependency for all resolution | gcloud dns policies describe POLICY --format="value(alternativeNameServerConfig)" |
| Firewall Insights reviewed for shadowed and unused rules | A shadowed rule is a control that does not exist | Console → Network Intelligence → Firewall Insights |
| Network Analyzer findings triaged into Security Command Center | Catches IP exhaustion and degraded BGP before they become outages | Console → Network Intelligence → Network Analyzer |
Sources §
- Cloud VPN overview — https://cloud.google.com/network-connectivity/docs/vpn/concepts/overview (last validated 2026-09-03)
- Cloud Interconnect overview — https://cloud.google.com/network-connectivity/docs/interconnect/concepts/overview (last validated 2026-09-03)
- Private services access — https://cloud.google.com/vpc/docs/private-services-access (last validated 2026-09-03)
- Cloud NGFW firewall policies overview — https://cloud.google.com/firewall/docs/firewall-policies-overview (last validated 2026-09-03)
- VPC networks overview — https://cloud.google.com/vpc/docs/vpc (last validated 2026-09-03)
- Cloud SDK command surface resolved offline against Google Cloud SDK 583.0.0 (core 2026.08.31), 2026-09-03