VPC Networking: GCP v.s. AWS

Comparing and contrasting Google Cloud Platform’s (GCP) with Amazon Web Services’ (AWS) virtual private cloud (VPC) networking.

TL:DR: Google Cloud Platform VPCs are relatively flat with controls targeting the instance, whereas Amazon Web Services VPCs are hierarchical with multiple layers of control at the region, zone, subnet, and instance.

The GCP and AWS solutions illustrated in this article are available for download as Terraform configuration files.

The Example Scenario

To explore the commonalities and differences between the GCP and AWS VPC features, we will walk through each one’s solution to a scenario consisting of three instances:

• bastion (or jump host): Used for troubleshooting
• frontend: Serves HTTP to the Internet
• backend: Serves HTTP to the frontend

We can further describe the scenario using network flows in the following color-coded network diagram:

Internet to Frontend and Frontend to Internet (red)
Internet to Bastion and Bastion to Internet (blue)

These flows allow for clients on the Internet to access the frontend instance (using HTTP) and the bastion instance (using ssh). These flows also allow for these instances to access servers on the Internet, e.g, performing software updates.

Backend to the Internet (green)

This flow allows for the backend instance to access servers on the Internet.

Bastion to Frontend (dark purple)
Bastion to Backend (light purple)

These flows allow for the bastion instance to access any instance, including the frontend and backend, using ICMP and ssh.

Frontend to Backend (aqua)

This flows allows for the frontend instance to access the backend instances using HTTP.

Google Cloud Platform (GCP)

The routes and firewall rules details are provided after the flow sections.

Internet to Frontend and Frontend to Internet (red)
Internet to Bastion and Bastion to Internet (blue)

The frontend and bastion instances have both an internal IP address, e.g., 10.128.0.2, and external IP address, e.g., 35.206.100.29.

Please note: The instances’ network interface here is configured with the Standard Tier, i.e., traffic traverses the public Internet to and from the GCP region, e.g., us-central1. GCP’s premium tier networking is another differentiator between GCP and AWS; which is outside the scope of this article.

Inbound traffic to the instances’ external IP address published on the Internet gateway (IGW) is forwarded to the instances with their internal IP address.

The allow-http-target-frontend firewall rule allows HTTP traffic from any address, i.e., 0.0.0.0/0, to instances tagged with frontend; the frontend instance is tagged with frontend.

The allow-ssh-target-bastion firewall rule allows SSL traffic from any address, i.e., 0.0.0.0/0, to instances tagged with bastion; the bastion instance is tagged with bastion.

For outbound traffic, the VPC’s implied egress firewall rule, allows any instance to send most any traffic to any destination, i.e., 0.0.0.0/0.

The route “Default route to the internet” sends Internet-bound traffic to the IGW for instances with an external IP address. Otherwise, it sends it to a regional network address translation gateway (NAT) if it exists.

Backend to the Internet (green)

The backend instance has only an internal IP address; without an external IP address there can be no inbound Internet traffic.

For Internet-bound traffic, the route “Default route to the internet” in this case sends Internet-bound traffic to the regional network address translation gateway (NAT). The NAT uses its own external IP address to send traffic to servers on the Internet

Bastion to Frontend (dark purple)
Bastion to Backend (light purple)

The allow-icmp-source-bastion firewall rule allows ICMP traffic from instances tagged with bastion to any instance.

The allow-ssh-source-bastion firewall rule allows ssh traffic from instances tagged with bastion to any instance.

Frontend to Backend (aqua)

The allow-http-target-backend firewall rule allows HTTP traffic from instances tagged with frontend to instances tagged with backend; the backend is tagged with backend.

Routes

Firewall Rules

Amazon Web Services (AWS)

The routing tables and security group details are provided after the flow sections.

Internet to Frontend and Frontend to Internet (red)
Internet to Bastion and Bastion to Internet (blue)

The frontend and bastion instances have both an internal IP address, e.g., 172.16.0.189, and an external IP address, e.g., 3.81.119.142. The subnet housing these instances is configured to assign instances external IP addresses.

Inbound traffic to the instances’ external IP address published on the Internet gateway (IGW) is forwarded to the instances on their internal IP address.

While subnets’ network access control list (NACL) can be used to further control traffic flow, in this project we leave them with the AWS default VPC behavior; an open NACL.

All of the instances are associated with the security groups:

• project-egress: For outbound traffic, this security group allows any traffic to any destination, i.e., 0.0.0.0/0
• project-instance: We will cover this security group later

The frontend instance is also associated with the project-frontend security group. The project-frontend security group allows HTTP traffic from any address, i.e., 0.0.0.0/0, to instances with it.

The bastion instance is also associated with the project-bastion security group. The project-bastion security group allows HTTP traffic from any address, i.e., 0.0.0.0/0, to instances with it.

Because the frontend and bastion instances are in a public subnet, Internet-bound traffic is sent to the IGW. A public subnet is a subnet with a route table that has a default route to an Internet gateway; in this case the project-public route table. A private subnet is not a public subnet.

Backend to the Internet (green)

The backend instance has only an internal IP address; without an external IP address there can be no inbound Internet traffic.

Because the backend instance is in the subnet associated with the project-private-0 route table, Internet-bound traffic is sent to the NAT in the same zone as the subnet. The NAT uses its own external IP address for traffic to servers on the Internet.

Bastion to Frontend (dark purple)
Bastion to Backend (light purple)

The project-instance security group, associated with all instances, allows both ICMP and ssh traffic from instances associated with the bastion security group.

Frontend to Backend (aqua)

The project-backend security group, also associated with the backend instance, allows HTTP traffic from instances associated with the frontend security group.

Route Tables

project-public (associated with both public subnets)

project-private-0 (associated with one private subnet)

Please note: The third route table is project-private-1 that is associated with the second private subnet; with a different NAT; not used in this scenario.

Security Groups

project-frontend

project-bastion

project-egress

project-instance

project-backend

Compare and Contrast

In general, Google Cloud Platform VPCs are relatively flat with controls targeting the instance where-as Amazon Web Services VPCs are hierarchical with multiple layers of control: at the region, zone, subnet, and instance.

While not relevant to this example scenario:

• GCP: VPCs are global resources; routing traffic between regions is automatic
• AWS: VPCs are regional resources; one has to add additional resources to route traffic between regions, e.g., vpc peering

Also not relevant to this example scenario:

• GCP: Subnets are associated with regions; traffic transparently moves across zones
• AWS: Subnets are associated with (availability) zones; moving traffic across zones requires routing between multiple subnets

One relevant difference:

• GCP: Routes are associated with the VPC. They can be restricted to instances via instance tags (or service accounts)
• AW: Routes reside in a subnet’s routing table. Controlling the routing of an instance’s traffic involves placing it in the appropriate subnet

Another related one:

• GCP: Instances are made public by specifically enabling them with an external IP address; the “Default route to the internet” automatically routes Internet-bound traffic to the Internet gateway or NAT gateway (if exists) based on the existence of the external IP address
• AWS: Instances are made public by placing them into a public subnet and are enabled with an external IP address (either automatically from the subnet’s settings or specifically). Again, a public subnet is a subnet with a route table that has a default route to an Internet gateway

Yet another difference not relevant to this scenario:

• GCP: firewall rules can have both allow and deny rules; we only used allow rules in this scenario
• AWS: security groups have only allow rules. Network access control lists (NACL) associated with subnets have both allow and deny rules. Also, unlike the GCP firewall rules and AWS security groups, NACLs are stateless firewalls.

Lastly, one relevant difference:

• GCP: Firewall rules can be automatically applied to all instances. Also, there is an implied egress firewall rule to allow all egress traffic to all destinations. Likewise, there is an implied ingress firewall rule to deny all ingress traffic from all sources
• AWS: Security groups must be associated with an instance to take effect

Conclusion

Trying to remember two solutions to the same problem (in this case, networking) is always challenging. I am going to guess that I will often come back to this article to remind myself of them.