{"id":772,"date":"2026-04-16T03:15:00","date_gmt":"2026-04-16T03:15:00","guid":{"rendered":"https:\/\/www.devopsschool.com\/tutorials\/google-cloud-vpc-flow-logs-tutorial-architecture-pricing-use-cases-and-hands-on-guide-for-networking\/"},"modified":"2026-04-16T03:15:00","modified_gmt":"2026-04-16T03:15:00","slug":"google-cloud-vpc-flow-logs-tutorial-architecture-pricing-use-cases-and-hands-on-guide-for-networking","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/tutorials\/google-cloud-vpc-flow-logs-tutorial-architecture-pricing-use-cases-and-hands-on-guide-for-networking\/","title":{"rendered":"Google Cloud VPC Flow Logs Tutorial: Architecture, Pricing, Use Cases, and Hands-On Guide for Networking"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Category<\/h2>\n\n\n\n<p>Networking<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. Introduction<\/h2>\n\n\n\n<p>VPC Flow Logs is a Google Cloud Networking feature that records network flow metadata for traffic going to and from resources in your Virtual Private Cloud (VPC) network. These logs help you understand who is talking to whom, over which ports and protocols, how much traffic is flowing, and whether connections are allowed or denied.<\/p>\n\n\n\n<p>In simple terms: <strong>VPC Flow Logs captures a sampled, time-aggregated record of network connections<\/strong> for resources attached to a subnet. You can then search, alert, and analyze this data using <strong>Cloud Logging<\/strong> and downstream tools like <strong>BigQuery<\/strong>.<\/p>\n\n\n\n<p>Technically, VPC Flow Logs generates structured log entries (flow records) for network traffic observed on VM network interfaces in a subnet. Flow records are <strong>sampled<\/strong> (you choose a sampling rate) and <strong>aggregated<\/strong> (you choose an aggregation interval). The logs are delivered to <strong>Cloud Logging<\/strong>, where you can retain them, query them, and route them using the <strong>Log Router<\/strong> to BigQuery, Cloud Storage, Pub\/Sub, or external destinations.<\/p>\n\n\n\n<p>The problem it solves is visibility: without flow logs, teams often struggle to answer questions like:\n&#8211; \u201cWhich source IP is scanning my workloads?\u201d\n&#8211; \u201cWhy can\u2019t service A reach service B?\u201d\n&#8211; \u201cWhich subnets generate the most egress?\u201d\n&#8211; \u201cWhich destinations are my workloads calling over the network?\u201d<\/p>\n\n\n\n<p>VPC Flow Logs gives you a practical and scalable starting point for <strong>network troubleshooting, security investigations, and cost governance<\/strong> in Google Cloud.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. What is VPC Flow Logs?<\/h2>\n\n\n\n<p><strong>Official purpose (what it is for):<\/strong><br\/>\nVPC Flow Logs provides network telemetry by recording metadata about IP traffic flows for Google Cloud VPC networks. It is designed to help with network monitoring, forensics, troubleshooting, and security analysis. Official documentation: https:\/\/cloud.google.com\/vpc\/docs\/using-flow-logs<\/p>\n\n\n\n<p><strong>Core capabilities:<\/strong>\n&#8211; Capture network flow metadata for traffic associated with a subnet.\n&#8211; Control <strong>sampling rate<\/strong> to balance visibility and cost.\n&#8211; Control <strong>aggregation interval<\/strong> to balance granularity and volume.\n&#8211; Include different levels of <strong>metadata<\/strong> (for example, to enrich records for investigations).\n&#8211; Use Cloud Logging for querying and for exporting flow logs to analytics systems.<\/p>\n\n\n\n<p><strong>Major components:<\/strong>\n&#8211; <strong>VPC network \/ Subnet<\/strong>: VPC Flow Logs is enabled and configured at the subnet level.\n&#8211; <strong>Flow log configuration<\/strong>: sampling, aggregation interval, and metadata settings.\n&#8211; <strong>Cloud Logging<\/strong>: the default destination where flow log entries appear.\n&#8211; <strong>Log Router (sinks)<\/strong>: routes logs to BigQuery, Cloud Storage, Pub\/Sub, or other supported destinations.\n&#8211; <strong>Downstream analytics<\/strong>: BigQuery, SIEM, dashboards, anomaly detection, alerting, and long-term retention.<\/p>\n\n\n\n<p><strong>Service type:<\/strong>\n&#8211; It\u2019s not a standalone \u201cservice\u201d you deploy; it\u2019s a <strong>Networking feature<\/strong> of Google Cloud VPC integrated with <strong>Cloud Logging<\/strong>.<\/p>\n\n\n\n<p><strong>Scope (how it is applied):<\/strong>\n&#8211; <strong>Configuration scope:<\/strong> enabled <strong>per subnet<\/strong>.\n&#8211; <strong>Log routing scope:<\/strong> Cloud Logging routing is configured per project (with options for centralized logging across projects via sinks and aggregated logging patterns).\n&#8211; <strong>Data scope:<\/strong> logs represent flows involving network interfaces attached to the subnet (exact coverage depends on workload type and Google Cloud\u2019s supported resources\u2014verify coverage for your specific product like GKE, Cloud Run, etc., in official docs).<\/p>\n\n\n\n<p><strong>How it fits into the Google Cloud ecosystem:<\/strong>\n&#8211; <strong>Google Cloud Networking:<\/strong> complements firewall rules, Cloud NAT, load balancers, VPN\/Interconnect, and network design patterns.\n&#8211; <strong>Cloud Logging:<\/strong> you query flow logs, create log-based metrics, and manage retention.\n&#8211; <strong>Security operations:<\/strong> supports threat detection and investigation workflows when exported to BigQuery or a SIEM.\n&#8211; <strong>FinOps \/ cost governance:<\/strong> helps identify unexpected egress, chatty services, and misrouted traffic.<\/p>\n\n\n\n<p><strong>Service name status (renamed\/deprecated?):<\/strong><br\/>\nAs of the latest official documentation, <strong>\u201cVPC Flow Logs\u201d<\/strong> remains the current product\/feature name in Google Cloud and is actively supported. Always validate any newly introduced log fields or format changes in the official docs because log schemas can evolve over time.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. Why use VPC Flow Logs?<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Business reasons<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Reduce downtime and incident duration:<\/strong> faster root cause analysis for connectivity issues can reduce business impact.<\/li>\n<li><strong>Support compliance and audits:<\/strong> network-level evidence of traffic patterns helps satisfy security and governance requirements (where logging is mandated).<\/li>\n<li><strong>Control cloud spend:<\/strong> visibility into network paths and egress destinations can reveal misconfigurations and unexpected traffic patterns that drive costs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Technical reasons<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Network visibility without packet capture:<\/strong> you get metadata about flows without running tcpdump everywhere.<\/li>\n<li><strong>Supports analytics at scale:<\/strong> Cloud Logging + BigQuery can handle large volumes with strong query capabilities.<\/li>\n<li><strong>Works well with infrastructure-as-code (IaC):<\/strong> subnet flow logs can be managed consistently across environments.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Operational reasons<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Troubleshoot firewall and routing issues:<\/strong> confirm whether traffic is observed, its direction, protocol\/port, and volume.<\/li>\n<li><strong>Baseline network behavior:<\/strong> establish normal patterns for services, subnets, and environments.<\/li>\n<li><strong>Build alerts:<\/strong> create log-based metrics and alerts around unexpected ports, denied flows, or suspicious destinations.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Security\/compliance reasons<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Detect scanning and lateral movement indicators:<\/strong> identify unusual east-west traffic or repeated denied attempts.<\/li>\n<li><strong>Support incident investigations:<\/strong> identify which hosts communicated during an event window.<\/li>\n<li><strong>Improve accountability:<\/strong> logs provide evidence trails (note: logs are sampled and aggregated, so they are not a complete forensic packet record).<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Scalability\/performance reasons<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Sampling and aggregation are built-in controls:<\/strong> you can tune visibility vs. cost\/volume.<\/li>\n<li><strong>Centralized analysis:<\/strong> route logs to a central project\/dataset for cross-project investigations.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">When teams should choose it<\/h3>\n\n\n\n<p>Choose VPC Flow Logs when you need:\n&#8211; Network communication visibility between services\/subnets\/projects.\n&#8211; A scalable logging pipeline using Cloud Logging and BigQuery.\n&#8211; Practical troubleshooting without deploying agents everywhere.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">When teams should not choose it<\/h3>\n\n\n\n<p>Avoid relying on VPC Flow Logs as your only tool when you need:\n&#8211; <strong>Full packet contents<\/strong> (payload inspection): use Packet Mirroring or host-based captures where appropriate.\n&#8211; <strong>Complete, unsampled capture<\/strong> of every connection: VPC Flow Logs is typically <strong>sampled<\/strong> and <strong>aggregated<\/strong>.\n&#8211; <strong>Immediate real-time enforcement:<\/strong> it\u2019s an observability tool, not a policy enforcement system.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. Where is VPC Flow Logs used?<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Industries<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Finance and insurance:<\/strong> audit trails, segmentation validation, incident response.<\/li>\n<li><strong>Healthcare:<\/strong> monitoring communications between regulated workloads (with appropriate access controls).<\/li>\n<li><strong>SaaS and technology:<\/strong> microservice connectivity troubleshooting and performance baselining.<\/li>\n<li><strong>Retail and e-commerce:<\/strong> detecting unexpected outbound calls, validating architecture changes during peak events.<\/li>\n<li><strong>Public sector:<\/strong> governance and visibility requirements across large networks and multiple projects.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Team types<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Network engineers and cloud networking teams<\/li>\n<li>SRE and platform engineering teams<\/li>\n<li>DevOps teams managing connectivity across services<\/li>\n<li>Security engineering and SOC teams<\/li>\n<li>FinOps\/cost governance teams<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Workloads<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Compute Engine VM-based applications<\/li>\n<li>GKE node traffic and east-west patterns (coverage depends on configuration and how traffic is routed; verify in docs for your cluster mode and dataplane)<\/li>\n<li>Multi-tier apps across multiple subnets<\/li>\n<li>Hybrid connectivity via Cloud VPN \/ Cloud Interconnect (visibility depends on traffic path; verify specifics in official docs)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Architectures<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Shared VPC with centralized security\/logging<\/li>\n<li>Multi-project environments with centralized observability<\/li>\n<li>Hub-and-spoke network topologies<\/li>\n<li>Segmented environments (prod\/dev\/test) with consistent telemetry baselines<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Real-world deployment contexts<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Production:<\/strong> usually enabled selectively (critical subnets, high-risk segments), exported to BigQuery\/SIEM with tuned sampling to control costs.<\/li>\n<li><strong>Dev\/test:<\/strong> often enabled temporarily at higher sampling to accelerate troubleshooting, then reduced or disabled.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">5. Top Use Cases and Scenarios<\/h2>\n\n\n\n<p>Below are realistic scenarios where VPC Flow Logs is commonly used.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">1) Investigate \u201cservice cannot connect\u201d incidents<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> Service A times out calling Service B.<\/li>\n<li><strong>Why VPC Flow Logs fits:<\/strong> confirms whether traffic is leaving A, reaching B\u2019s subnet, and whether responses are seen.<\/li>\n<li><strong>Example:<\/strong> A VM in <code>subnet-app<\/code> cannot reach a VM in <code>subnet-db<\/code> on TCP\/5432. Flow logs reveal only SYN packets are seen and responses are missing, narrowing the issue to firewall or routing.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">2) Validate firewall segmentation (least privilege)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> Teams create firewall rules but don\u2019t know if segmentation works as expected.<\/li>\n<li><strong>Why it fits:<\/strong> observe allowed\/denied flows across subnet boundaries.<\/li>\n<li><strong>Example:<\/strong> After tightening firewall rules, use flow logs to confirm no unexpected ports are used between app and admin networks.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">3) Detect and triage port scanning attempts<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> Suspicious sources attempt many ports.<\/li>\n<li><strong>Why it fits:<\/strong> flow records show repeated denied attempts and target port ranges.<\/li>\n<li><strong>Example:<\/strong> A VM shows repeated denied connections to TCP\/22 across multiple instances; SOC investigates potential compromise or misconfiguration.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">4) Identify unexpected outbound (egress) destinations<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> A workload begins sending traffic to unknown IPs\/domains.<\/li>\n<li><strong>Why it fits:<\/strong> flow logs show destination IPs and traffic volumes.<\/li>\n<li><strong>Example:<\/strong> A compromised VM starts calling a command-and-control IP. Flow logs highlight high-frequency outbound flows.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">5) Reduce network egress costs through visibility<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> Network egress costs are higher than expected.<\/li>\n<li><strong>Why it fits:<\/strong> you can attribute egress volume by subnet\/workload and destination patterns (with enrichment).<\/li>\n<li><strong>Example:<\/strong> A data pipeline accidentally egresses to the internet rather than using Private Service Connect or internal endpoints. Flow logs help identify misrouted paths.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">6) Audit lateral movement pathways after an incident<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> Need to understand what a suspected compromised host talked to.<\/li>\n<li><strong>Why it fits:<\/strong> provides network communication history (subject to sampling\/retention).<\/li>\n<li><strong>Example:<\/strong> During incident response, export flow logs to BigQuery and query all flows involving an instance IP for the incident window.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">7) Baseline normal traffic for anomaly detection<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> No baseline of \u201cnormal\u201d east-west traffic exists.<\/li>\n<li><strong>Why it fits:<\/strong> you can build BigQuery models or dashboards on top of flow logs.<\/li>\n<li><strong>Example:<\/strong> A microservice normally talks to 3 backends; flow logs show it suddenly contacts 30 destinations, triggering investigation.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">8) Capacity planning and \u201cchatty service\u201d identification<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> Services create excessive internal traffic, stressing NAT gateways or backends.<\/li>\n<li><strong>Why it fits:<\/strong> byte\/packet counts by flow help identify high-volume pairs.<\/li>\n<li><strong>Example:<\/strong> A service misconfiguration causes repeated retries; flow logs show high packet counts and repeated short-lived connections.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">9) Change validation after network redesign<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> After adding new subnets\/peering\/VPN, you need to ensure traffic uses intended paths.<\/li>\n<li><strong>Why it fits:<\/strong> flow logs validate connectivity and traffic direction at subnet boundaries.<\/li>\n<li><strong>Example:<\/strong> After introducing Shared VPC, verify that workloads in service projects communicate only with approved shared services.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">10) Enrich SIEM detection and correlation<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> SIEM needs network telemetry to correlate with identity and host logs.<\/li>\n<li><strong>Why it fits:<\/strong> export to BigQuery or stream via Pub\/Sub to a SIEM pipeline.<\/li>\n<li><strong>Example:<\/strong> Correlate suspicious authentication events with subsequent outbound connections from the same host IP.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">11) Monitor denied traffic to critical subnets<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> Need to know what\u2019s being blocked (and from where) to tune policy and detect attacks.<\/li>\n<li><strong>Why it fits:<\/strong> provides denied flow metadata (when logged).<\/li>\n<li><strong>Example:<\/strong> A restricted admin subnet sees repeated denied attempts from a dev subnet; investigate policy violations.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">12) Troubleshoot DNS or dependency resolution patterns (indirectly)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> Applications fail because dependencies are unreachable, often triggered by DNS changes.<\/li>\n<li><strong>Why it fits:<\/strong> while it doesn\u2019t log DNS query payloads, it can show traffic patterns to DNS servers or dependency endpoints.<\/li>\n<li><strong>Example:<\/strong> A service suddenly contacts a different IP range for the same dependency; flow logs help confirm the change.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">6. Core Features<\/h2>\n\n\n\n<p>This section summarizes important current capabilities. For exact field lists and configuration flags, always validate against official docs: https:\/\/cloud.google.com\/vpc\/docs\/using-flow-logs<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Feature 1: Subnet-level enablement and configuration<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> enables flow logging on a specific subnet and applies sampling\/aggregation\/metadata settings.<\/li>\n<li><strong>Why it matters:<\/strong> lets you target critical network segments without logging everything.<\/li>\n<li><strong>Practical benefit:<\/strong> reduce noise and cost by enabling logs only where needed (prod subnets, sensitive segments).<\/li>\n<li><strong>Caveats:<\/strong> if you forget to enable logs on a subnet, you will not see flows for resources attached to it.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Feature 2: Traffic sampling (flow sampling rate)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> logs only a fraction of flows based on a configured sampling rate.<\/li>\n<li><strong>Why it matters:<\/strong> reduces log volume and cost.<\/li>\n<li><strong>Practical benefit:<\/strong> you can start at a moderate sampling level and adjust based on incident needs.<\/li>\n<li><strong>Caveats:<\/strong> sampling means you might miss short-lived or low-volume flows; do not treat as a perfect record of all traffic.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Feature 3: Time aggregation interval<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> aggregates observed flow data into time buckets before logging (granularity control).<\/li>\n<li><strong>Why it matters:<\/strong> affects how many log entries are generated and how precisely you can time events.<\/li>\n<li><strong>Practical benefit:<\/strong> shorter intervals provide finer detail; longer intervals reduce log volume.<\/li>\n<li><strong>Caveats:<\/strong> aggregation can blur short spikes; choose based on investigation needs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Feature 4: Metadata inclusion \/ enrichment options<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> includes additional metadata fields in log entries (for example, resource identifiers).<\/li>\n<li><strong>Why it matters:<\/strong> enriched logs are easier to join with asset inventories and incident records.<\/li>\n<li><strong>Practical benefit:<\/strong> improved investigations and BigQuery queries without needing extra lookups.<\/li>\n<li><strong>Caveats:<\/strong> more metadata can increase log size, which can increase Cloud Logging ingestion costs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Feature 5: Default integration with Cloud Logging<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> flow logs appear as log entries in Cloud Logging.<\/li>\n<li><strong>Why it matters:<\/strong> you can search immediately, apply retention, and route logs.<\/li>\n<li><strong>Practical benefit:<\/strong> consistent operational workflow with the rest of Google Cloud logs.<\/li>\n<li><strong>Caveats:<\/strong> Cloud Logging costs and quotas apply.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Feature 6: Log Router export (sinks) to analytics destinations<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> routes flow logs to supported destinations such as BigQuery, Cloud Storage, Pub\/Sub, and third-party integrations.<\/li>\n<li><strong>Why it matters:<\/strong> enables long-term storage and advanced analytics beyond basic log search.<\/li>\n<li><strong>Practical benefit:<\/strong> BigQuery queries for threat hunting; Cloud Storage for archival; Pub\/Sub for streaming pipelines.<\/li>\n<li><strong>Caveats:<\/strong> downstream services have their own costs, IAM, and operational requirements.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Feature 7: Log-based metrics and alerting (via Cloud Logging)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> create metrics derived from logs to drive dashboards\/alerts.<\/li>\n<li><strong>Why it matters:<\/strong> converts raw flow logs into actionable operational signals.<\/li>\n<li><strong>Practical benefit:<\/strong> alert on unusual denied flows, unexpected ports, or spikes in egress bytes.<\/li>\n<li><strong>Caveats:<\/strong> metric design matters; poor filters can create noisy alerts.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Feature 8: Works with centralized logging patterns<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> supports exporting logs to a central project\/dataset for multi-project analysis.<\/li>\n<li><strong>Why it matters:<\/strong> enterprises often need consistent visibility across many projects.<\/li>\n<li><strong>Practical benefit:<\/strong> one BigQuery dataset for organization-wide network telemetry (with proper access controls).<\/li>\n<li><strong>Caveats:<\/strong> ensure governance and least-privilege access; flow logs may contain sensitive internal IP data.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">7. Architecture and How It Works<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">High-level architecture<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li>You enable VPC Flow Logs on a subnet.<\/li>\n<li>Google Cloud observes traffic to\/from network interfaces in that subnet and produces flow records.<\/li>\n<li>Flow records are sampled and aggregated based on your configuration.<\/li>\n<li>The resulting log entries are written into Cloud Logging under the VPC flow logs log stream.<\/li>\n<li>Optionally, you configure Log Router sinks to export these logs to:\n   &#8211; BigQuery (analytics \/ threat hunting)\n   &#8211; Cloud Storage (archive)\n   &#8211; Pub\/Sub (stream processing)\n   &#8211; Other supported destinations (verify in Cloud Logging routing docs)<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Request\/data\/control flow<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Control plane:<\/strong> you configure the subnet logging settings (sampling, aggregation, metadata).<\/li>\n<li><strong>Data plane:<\/strong> traffic flows between workloads; VPC Flow Logs observes metadata for those flows.<\/li>\n<li><strong>Telemetry pipeline:<\/strong> log entries are delivered to Cloud Logging; from there, log routing exports them elsewhere.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Integrations with related services<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Cloud Logging:<\/strong> search, retention, log-based metrics, routing.<\/li>\n<li><strong>BigQuery:<\/strong> large-scale querying, joins with asset inventories, dashboards.<\/li>\n<li><strong>Pub\/Sub + Dataflow:<\/strong> streaming analytics and custom enrichment pipelines.<\/li>\n<li><strong>Cloud Storage:<\/strong> archival and cold storage for compliance.<\/li>\n<li><strong>Cloud Monitoring:<\/strong> alerting often uses metrics derived from logs (log-based metrics) rather than raw logs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Dependency services<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>VPC networking and subnetworks (Compute Engine networking).<\/li>\n<li>Cloud Logging (for storage, indexing, querying, routing).<\/li>\n<li>Optional: BigQuery\/Cloud Storage\/Pub\/Sub for exports.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Security\/authentication model<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Access to logs is controlled via <strong>IAM<\/strong> in Cloud Logging (e.g., Logging Viewer).<\/li>\n<li>Exports use <strong>sink writer identities<\/strong> (service accounts) that must be granted access to the destination (e.g., BigQuery dataset permissions).<\/li>\n<li>Sensitive data handling depends on who can read logs and exports.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Networking model<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>VPC Flow Logs observes network flows associated with a subnet. It is not a firewall and does not change routing.<\/li>\n<li>You typically combine it with:<\/li>\n<li>VPC firewall rules and firewall logging (separate feature)<\/li>\n<li>Cloud NAT logs (for NAT visibility)<\/li>\n<li>Load balancer logs (application-facing traffic)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Monitoring\/logging\/governance considerations<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Log volume can be large. Define:<\/li>\n<li>Which subnets are in scope<\/li>\n<li>Sampling\/aggregation defaults by environment (prod vs dev)<\/li>\n<li>Retention policies in Cloud Logging and downstream storage<\/li>\n<li>Centralized export design and access controls<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Simple architecture diagram<\/h4>\n\n\n\n<pre><code class=\"language-mermaid\">flowchart LR\n  A[VMs \/ workloads in a subnet] --&gt; B[VPC network traffic]\n  B --&gt; C[VPC Flow Logs&lt;br\/&gt;(sampling + aggregation)]\n  C --&gt; D[Cloud Logging]\n  D --&gt; E[Logs Explorer \/ Alerts]\n  D --&gt; F[Log Router Sink]\n  F --&gt; G[BigQuery]\n  F --&gt; H[Cloud Storage]\n  F --&gt; I[Pub\/Sub]\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Production-style architecture diagram (centralized analytics)<\/h4>\n\n\n\n<pre><code class=\"language-mermaid\">flowchart TB\n  subgraph Org[Google Cloud Organization]\n    subgraph NetProject[Network Host Project (Shared VPC)]\n      S1[Prod Subnet A&lt;br\/&gt;Flow Logs enabled]\n      S2[Prod Subnet B&lt;br\/&gt;Flow Logs enabled]\n      WL1[Compute\/GKE workloads]\n      WL1 --&gt; S1\n      WL1 --&gt; S2\n    end\n\n    subgraph AppProjects[Service Projects]\n      P1[Project: Payments]\n      P2[Project: Analytics]\n    end\n\n    subgraph LoggingProject[Central Logging Project]\n      CL[Cloud Logging&lt;br\/&gt;Log Buckets &amp; Retention]\n      LR[Log Router&lt;br\/&gt;Aggregated sinks]\n      BQ[(BigQuery Dataset&lt;br\/&gt;vpc_flow_logs)]\n      CS[(Cloud Storage Archive)]\n    end\n\n    subgraph SecOps[Security Operations]\n      SOC[SIEM \/ Threat Hunting]\n      Dash[Dashboards \/ Looker Studio]\n    end\n  end\n\n  S1 --&gt; CL\n  S2 --&gt; CL\n  CL --&gt; LR\n  LR --&gt; BQ\n  LR --&gt; CS\n  BQ --&gt; SOC\n  BQ --&gt; Dash\n<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">8. Prerequisites<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Account \/ project requirements<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A Google Cloud project with <strong>billing enabled<\/strong>.<\/li>\n<li>Permissions to create and manage VPC networks, subnets, VM instances, and logs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">IAM permissions (common minimums)<\/h3>\n\n\n\n<p>Exact least-privilege varies by environment, but for this tutorial you typically need:\n&#8211; To create networks\/subnets and enable flow logs:\n  &#8211; <code>roles\/compute.networkAdmin<\/code> (or a custom role including <code>compute.subnetworks.create<\/code> and <code>compute.subnetworks.update<\/code>)\n&#8211; To create VM instances:\n  &#8211; <code>roles\/compute.instanceAdmin.v1<\/code> (and possibly <code>roles\/iam.serviceAccountUser<\/code> to attach service accounts)\n&#8211; To view logs:\n  &#8211; <code>roles\/logging.viewer<\/code>\n&#8211; To create log sinks:\n  &#8211; <code>roles\/logging.configWriter<\/code>\n&#8211; If exporting to BigQuery:\n  &#8211; BigQuery dataset create permissions (e.g., <code>roles\/bigquery.admin<\/code> for labs; in production prefer narrower roles)<\/p>\n\n\n\n<p>If you use IAP-based SSH in the lab, you may also need:\n&#8211; <code>roles\/iap.tunnelResourceAccessor<\/code> for the user initiating SSH<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Billing requirements<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Cloud Logging ingestion\/retention beyond free allotments may incur cost.<\/li>\n<li>Compute Engine VM usage may incur cost (free tier may apply depending on region and eligible machine types\u2014verify current free tier details).<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">CLI \/ tools<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Cloud Shell<\/strong> (recommended) includes:<\/li>\n<li><code>gcloud<\/code><\/li>\n<li><code>bq<\/code><\/li>\n<li>Or install the Google Cloud CLI: https:\/\/cloud.google.com\/sdk\/docs\/install<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Region availability<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>VPC and Cloud Logging are global services, but subnets and VM instances are regional\/zonal.  <\/li>\n<li>Choose a region close to you and consistent with your organization\u2019s policies.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Quotas \/ limits<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Cloud Logging quotas and billing apply (log ingestion volume, API usage).<\/li>\n<li>Compute Engine quotas apply (VMs, CPUs, etc.).<\/li>\n<li>BigQuery quotas apply if exporting and running queries.<\/li>\n<\/ul>\n\n\n\n<p>Always check:\n&#8211; Cloud Logging quotas: https:\/\/cloud.google.com\/logging\/quotas\n&#8211; Compute Engine quotas: https:\/\/cloud.google.com\/compute\/quotas<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Prerequisite services\/APIs<\/h3>\n\n\n\n<p>Enable these APIs in your project:\n&#8211; Compute Engine API (<code>compute.googleapis.com<\/code>)\n&#8211; Cloud Logging API (<code>logging.googleapis.com<\/code>) is typically available by default, but verify if restricted by policy\n&#8211; BigQuery API (<code>bigquery.googleapis.com<\/code>) only if you export to BigQuery<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">9. Pricing \/ Cost<\/h2>\n\n\n\n<p>VPC Flow Logs itself is a feature, but <strong>the logs it generates create billable usage in Cloud Logging and any export destination<\/strong>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Pricing dimensions (what you pay for)<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\n<p><strong>Cloud Logging ingestion (log volume):<\/strong>\n   &#8211; You are billed based on the volume of log data ingested beyond any free allotment.\n   &#8211; Flow logs can be high-volume depending on traffic, sampling rate, aggregation interval, and metadata.<\/p>\n<\/li>\n<li>\n<p><strong>Cloud Logging retention:<\/strong>\n   &#8211; Retention beyond included\/default periods can incur cost depending on bucket configuration and retention settings.\n   &#8211; Verify current retention pricing and default retention behavior in Cloud Logging docs.<\/p>\n<\/li>\n<li>\n<p><strong>Log Router exports:<\/strong>\n   &#8211; Exporting logs is supported, but you pay for the destination service:<\/p>\n<ul>\n<li><strong>BigQuery:<\/strong> storage + query costs (and streaming ingestion behavior if applicable to Log Router exports\u2014verify current export mechanics).<\/li>\n<li><strong>Cloud Storage:<\/strong> storage and operations.<\/li>\n<li><strong>Pub\/Sub:<\/strong> message volume and delivery.<\/li>\n<\/ul>\n<\/li>\n<li>\n<p><strong>Network and egress implications:<\/strong>\n   &#8211; Exporting logs across regions or to external destinations can incur network egress costs depending on architecture.\n   &#8211; If you move log data out of Google Cloud (e.g., to a third-party SIEM), egress charges may apply.<\/p>\n<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Free tier \/ free allotment<\/h3>\n\n\n\n<p>Cloud Logging typically provides a free allotment of log ingestion per project per month, but <strong>the amount and rules can change<\/strong>. Do not assume your flow logs are \u201cfree.\u201d<br\/>\nVerify current Cloud Logging pricing here:\n&#8211; Official pricing page: https:\/\/cloud.google.com\/logging\/pricing\n&#8211; Pricing calculator: https:\/\/cloud.google.com\/products\/calculator<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Primary cost drivers for VPC Flow Logs<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Total traffic volume<\/strong> in the subnet(s)<\/li>\n<li><strong>Sampling rate<\/strong> (higher sampling \u2192 more logs)<\/li>\n<li><strong>Aggregation interval<\/strong> (shorter interval \u2192 more entries)<\/li>\n<li><strong>Metadata level<\/strong> (more metadata \u2192 larger entries)<\/li>\n<li><strong>Number of subnets enabled<\/strong> (blast radius of telemetry)<\/li>\n<li><strong>Retention period<\/strong> in Cloud Logging and\/or BigQuery storage duration<\/li>\n<li><strong>Query patterns in BigQuery<\/strong> (frequent large scans can be expensive)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Hidden or indirect costs to plan for<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>BigQuery query costs<\/strong> during threat hunting (large time ranges).<\/li>\n<li><strong>Storage costs<\/strong> for long retention (especially if you keep raw logs for months).<\/li>\n<li><strong>Operational overhead<\/strong>: building dashboards, managing IAM, reviewing alerts.<\/li>\n<li><strong>SIEM licensing\/ingestion costs<\/strong> if exporting outside Google Cloud.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">How to optimize cost (practical guidance)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Enable flow logs only on <strong>subnets that matter<\/strong> (prod, sensitive, boundary networks).<\/li>\n<li>Start with <strong>moderate sampling<\/strong> and adjust:<\/li>\n<li>Increase temporarily during an incident.<\/li>\n<li>Reduce for steady-state operations.<\/li>\n<li>Use <strong>longer aggregation intervals<\/strong> in steady state to reduce volume.<\/li>\n<li>Be selective with <strong>metadata inclusion<\/strong>; only include what you actually use.<\/li>\n<li>Export to BigQuery only if you need advanced analytics; otherwise use Cloud Logging queries and log-based metrics.<\/li>\n<li>Use <strong>partitioned tables<\/strong> (where applicable) and time-bounded queries in BigQuery.<\/li>\n<li>Apply lifecycle policies in Cloud Storage if archiving.<\/li>\n<li>Use log sinks with filters to export only the subset you need (for example, only prod subnets).<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Example low-cost starter estimate (qualitative)<\/h3>\n\n\n\n<p>A small lab setup with:\n&#8211; Flow logs enabled on one subnet\n&#8211; Low traffic\n&#8211; Moderate sampling\n&#8211; Short retention in Cloud Logging\nwill typically generate modest log volume; costs usually remain low, but <strong>the actual bill depends on your traffic and configuration<\/strong>. Use the Cloud Billing reports and Logging usage metrics to validate.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Example production cost considerations<\/h3>\n\n\n\n<p>For production environments with:\n&#8211; Many subnets enabled\n&#8211; High east-west traffic\n&#8211; High sampling and rich metadata\n&#8211; Long retention and BigQuery exports\nyou should expect meaningful Cloud Logging ingestion and BigQuery storage\/query costs. Plan budgets and enforce guardrails (sampling defaults, sink filters, retention policies).<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">10. Step-by-Step Hands-On Tutorial<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Objective<\/h3>\n\n\n\n<p>Enable <strong>VPC Flow Logs<\/strong> on a subnet in Google Cloud, generate real network traffic between two VM instances, view flow logs in <strong>Cloud Logging<\/strong>, and (optionally) export them to <strong>BigQuery<\/strong> for querying.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Lab Overview<\/h3>\n\n\n\n<p>You will:\n1. Create a custom VPC and subnet with VPC Flow Logs enabled.\n2. Create two small VM instances in the subnet.\n3. Generate traffic (ICMP and HTTP) between the VMs.\n4. Verify flow logs in Cloud Logging.\n5. (Optional) Export flow logs to BigQuery and run a basic query.\n6. Clean up resources to avoid ongoing costs.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 1: Set variables and enable required APIs<\/h3>\n\n\n\n<p>Use Cloud Shell (recommended).<\/p>\n\n\n\n<pre><code class=\"language-bash\">PROJECT_ID=\"$(gcloud config get-value project)\"\nREGION=\"us-central1\"\nZONE=\"us-central1-a\"\n\ngcloud services enable compute.googleapis.com logging.googleapis.com\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> APIs enable successfully (may take a minute).<br\/>\n<strong>Verify:<\/strong><\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud services list --enabled --filter=\"name:compute.googleapis.com OR name:logging.googleapis.com\"\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Step 2: Create a VPC network and subnet with VPC Flow Logs enabled<\/h3>\n\n\n\n<p>Create a custom mode VPC:<\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud compute networks create vpc-flowlab --subnet-mode=custom\n<\/code><\/pre>\n\n\n\n<p>Create a subnet and enable flow logs with a reasonable lab configuration.<\/p>\n\n\n\n<blockquote>\n<p>Note: <code>gcloud<\/code> flags and allowed values can evolve. If a flag fails, use <code>gcloud compute networks subnets create --help<\/code> and verify against official docs.<\/p>\n<\/blockquote>\n\n\n\n<pre><code class=\"language-bash\">gcloud compute networks subnets create flow-subnet \\\n  --network=vpc-flowlab \\\n  --region=\"$REGION\" \\\n  --range=10.10.0.0\/24 \\\n  --enable-flow-logs \\\n  --logging-aggregation-interval=INTERVAL_5_SEC \\\n  --logging-flow-sampling=0.5 \\\n  --logging-metadata=INCLUDE_ALL_METADATA\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> subnet is created and flow logs are enabled.<\/p>\n\n\n\n<p><strong>Verify subnet settings:<\/strong><\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud compute networks subnets describe flow-subnet --region=\"$REGION\" \\\n  --format=\"yaml(name,region,enableFlowLogs,logConfig)\"\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Step 3: Create firewall rules for internal traffic and SSH (IAP-based)<\/h3>\n\n\n\n<p>Allow internal traffic inside the subnet (ICMP and TCP\/8080 for the web server test):<\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud compute firewall-rules create allow-internal-icmp-8080 \\\n  --network=vpc-flowlab \\\n  --direction=INGRESS \\\n  --priority=1000 \\\n  --action=ALLOW \\\n  --rules=tcp:8080,icmp \\\n  --source-ranges=10.10.0.0\/24\n<\/code><\/pre>\n\n\n\n<p>For SSH access without public IPs, use IAP tunneling. Create a firewall rule allowing IAP to reach TCP\/22:<\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud compute firewall-rules create allow-iap-ssh \\\n  --network=vpc-flowlab \\\n  --direction=INGRESS \\\n  --priority=1000 \\\n  --action=ALLOW \\\n  --rules=tcp:22 \\\n  --source-ranges=35.235.240.0\/20 \\\n  --target-tags=iap-ssh\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> firewall rules created.<\/p>\n\n\n\n<p><strong>Verify:<\/strong><\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud compute firewall-rules list --filter=\"network:vpc-flowlab\" --format=\"table(name, direction, allowed, sourceRanges, targetTags)\"\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Step 4: Create two VM instances without external IPs<\/h3>\n\n\n\n<p>Create <code>vm-a<\/code> and <code>vm-b<\/code> in the same subnet. Use small machine types to keep costs low (verify eligible free tier in your region if you rely on it).<\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud compute instances create vm-a \\\n  --zone=\"$ZONE\" \\\n  --machine-type=e2-micro \\\n  --subnet=flow-subnet \\\n  --no-address \\\n  --tags=iap-ssh\n\ngcloud compute instances create vm-b \\\n  --zone=\"$ZONE\" \\\n  --machine-type=e2-micro \\\n  --subnet=flow-subnet \\\n  --no-address \\\n  --tags=iap-ssh\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> two VMs are created with only internal IPs.<\/p>\n\n\n\n<p><strong>Verify internal IPs:<\/strong><\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud compute instances list --filter=\"name=(vm-a vm-b)\" --format=\"table(name, zone, networkInterfaces[0].networkIP, networkInterfaces[0].accessConfigs)\"\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Step 5: Generate traffic between the VMs<\/h3>\n\n\n\n<p>Get <code>vm-a<\/code> internal IP:<\/p>\n\n\n\n<pre><code class=\"language-bash\">VM_A_IP=\"$(gcloud compute instances describe vm-a --zone=\"$ZONE\" --format='value(networkInterfaces[0].networkIP)')\"\necho \"vm-a internal IP: $VM_A_IP\"\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">5a) Start a simple HTTP server on vm-a<\/h4>\n\n\n\n<p>SSH into vm-a via IAP and start a server on port 8080:<\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud compute ssh vm-a --zone=\"$ZONE\" --tunnel-through-iap --command \\\n  \"nohup python3 -m http.server 8080 &gt;\/tmp\/http.log 2&gt;&amp;1 &amp; sleep 1; ss -lntp | grep 8080 || true\"\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> port 8080 is listening on vm-a.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">5b) From vm-b, ping and curl vm-a<\/h4>\n\n\n\n<pre><code class=\"language-bash\">gcloud compute ssh vm-b --zone=\"$ZONE\" --tunnel-through-iap --command \\\n  \"ping -c 3 $VM_A_IP; curl -sS --max-time 3 http:\/\/$VM_A_IP:8080 | head\"\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> ping succeeds and curl returns HTML directory listing (or similar output).<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 6: View VPC Flow Logs in Cloud Logging<\/h3>\n\n\n\n<p>Flow logs can take a few minutes to appear. First, confirm you\u2019re filtering the correct log stream.<\/p>\n\n\n\n<p>In Cloud Shell, run:<\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud logging read \\\n  \"logName=\\\"projects\/${PROJECT_ID}\/logs\/compute.googleapis.com%2Fvpc_flows\\\"\" \\\n  --freshness=30m \\\n  --limit=10 \\\n  --format=\"table(timestamp, resource.labels.subnetwork_name, jsonPayload.src_instance.vm_name, jsonPayload.dest_instance.vm_name, jsonPayload.connection.src_ip, jsonPayload.connection.dest_ip, jsonPayload.connection.protocol, jsonPayload.connection.src_port, jsonPayload.connection.dest_port, jsonPayload.bytes_sent, jsonPayload.packets_sent)\"\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> you see log entries for flows between <code>vm-b<\/code> and <code>vm-a<\/code> (and possibly other background traffic).<br\/>\nIf the table format fails due to field name differences, output JSON to inspect the exact schema:<\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud logging read \\\n  \"logName=\\\"projects\/${PROJECT_ID}\/logs\/compute.googleapis.com%2Fvpc_flows\\\"\" \\\n  --freshness=30m \\\n  --limit=2 \\\n  --format=json\n<\/code><\/pre>\n\n\n\n<p><strong>Console verification option (often easier):<\/strong>\n1. Go to Logs Explorer: https:\/\/console.cloud.google.com\/logs\n2. Use this query:\n   <code>logName=\"projects\/PROJECT_ID\/logs\/compute.googleapis.com%2Fvpc_flows\"\n   resource.type=\"gce_subnetwork\"\n   resource.labels.subnetwork_name=\"flow-subnet\"<\/code>\n3. Set time range to \u201cLast 30 minutes\u201d.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 7 (Optional): Export VPC Flow Logs to BigQuery<\/h3>\n\n\n\n<p>This step demonstrates how teams typically analyze flow logs at scale.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">7a) Enable BigQuery API and create a dataset<\/h4>\n\n\n\n<pre><code class=\"language-bash\">gcloud services enable bigquery.googleapis.com\n\nbq --location=US mk -d \"${PROJECT_ID}:vpc_flow_logs\"\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> dataset exists.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">7b) Create a Log Router sink to BigQuery<\/h4>\n\n\n\n<p>Create a sink that routes only VPC flow logs:<\/p>\n\n\n\n<pre><code class=\"language-bash\">gcloud logging sinks create vpc-flows-to-bq \\\n  bigquery.googleapis.com\/projects\/${PROJECT_ID}\/datasets\/vpc_flow_logs \\\n  --log-filter=\"logName=\\\"projects\/${PROJECT_ID}\/logs\/compute.googleapis.com%2Fvpc_flows\\\"\" \\\n  --use-partitioned-tables\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> sink is created.<\/p>\n\n\n\n<p>Get the sink writer identity (a service account managed for the sink):<\/p>\n\n\n\n<pre><code class=\"language-bash\">SINK_WRITER=\"$(gcloud logging sinks describe vpc-flows-to-bq --format='value(writerIdentity)')\"\necho \"$SINK_WRITER\"\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">7c) Grant the sink permission to write to the dataset<\/h4>\n\n\n\n<p>BigQuery dataset permissions are managed at the dataset level.<\/p>\n\n\n\n<p><strong>Console method (recommended for accuracy):<\/strong>\n1. Go to BigQuery: https:\/\/console.cloud.google.com\/bigquery\n2. Find dataset <code>vpc_flow_logs<\/code>\n3. Click <strong>Sharing<\/strong> \u2192 <strong>Permissions<\/strong> (or <strong>Manage permissions<\/strong>, UI varies)\n4. Add principal = the sink writer identity you printed (e.g., <code>serviceAccount:...<\/code>)\n5. Grant role: <strong>BigQuery Data Editor<\/strong> on the dataset (or a least-privilege equivalent that allows table creation and data write; verify in your org)<\/p>\n\n\n\n<p><strong>Expected outcome:<\/strong> sink can create\/write tables.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">7d) Wait for data and query it<\/h4>\n\n\n\n<p>Generate a bit more traffic (repeat Step 5b), wait 2\u20135 minutes, then check for tables:<\/p>\n\n\n\n<pre><code class=\"language-bash\">bq ls \"${PROJECT_ID}:vpc_flow_logs\"\n<\/code><\/pre>\n\n\n\n<p>If you see a table related to VPC flow logs, run a query (table names vary; pick the correct one you see in your dataset UI). Example query pattern:<\/p>\n\n\n\n<pre><code class=\"language-bash\"># Replace TABLE_NAME with the actual table name shown in your dataset.\nTABLE_NAME=\"compute_googleapis_com_vpc_flows\"\n\nbq query --use_legacy_sql=false \"\nSELECT\n  timestamp,\n  jsonPayload.connection.src_ip AS src_ip,\n  jsonPayload.connection.dest_ip AS dest_ip,\n  jsonPayload.connection.dest_port AS dest_port,\n  jsonPayload.bytes_sent AS bytes_sent\nFROM \\`${PROJECT_ID}.vpc_flow_logs.${TABLE_NAME}\\`\nWHERE timestamp &gt;= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 HOUR)\nORDER BY timestamp DESC\nLIMIT 50\n\"\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> you see flow records in query output.<\/p>\n\n\n\n<blockquote>\n<p>If your export schema differs (e.g., fields are not under <code>jsonPayload<\/code>), inspect the table schema in BigQuery and adjust the query accordingly. Google Cloud can evolve exported log schemas\u2014always verify field paths.<\/p>\n<\/blockquote>\n\n\n\n<h3 class=\"wp-block-heading\">Validation<\/h3>\n\n\n\n<p>Use the checklist below to confirm the lab is successful:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\n<p><strong>Subnet shows flow logs enabled:<\/strong>\n<code>bash\n   gcloud compute networks subnets describe flow-subnet --region=\"$REGION\" \\\n     --format=\"get(enableFlowLogs)\"<\/code>\n   Should output <code>True<\/code>.<\/p>\n<\/li>\n<li>\n<p><strong>You generated traffic:<\/strong>\n   &#8211; ping succeeded\n   &#8211; curl returned content from <code>vm-a<\/code><\/p>\n<\/li>\n<li>\n<p><strong>Cloud Logging contains VPC flow logs:<\/strong>\n<code>bash\n   gcloud logging read \\\n     \"logName=\\\"projects\/${PROJECT_ID}\/logs\/compute.googleapis.com%2Fvpc_flows\\\" AND resource.labels.subnetwork_name=\\\"flow-subnet\\\"\" \\\n     --freshness=1h --limit=5 --format=json<\/code>\n   Should return entries.<\/p>\n<\/li>\n<li>\n<p><strong>Optional BigQuery export works:<\/strong>\n   &#8211; dataset exists\n   &#8211; sink exists\n   &#8211; table exists and query returns rows<\/p>\n<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Troubleshooting<\/h3>\n\n\n\n<p>Common issues and fixes:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\n<p><strong>No flow logs appear<\/strong>\n   &#8211; Wait 5\u201310 minutes; flow logs are not always instantaneous.\n   &#8211; Confirm you enabled flow logs on the correct subnet and that VMs are attached to that subnet.\n   &#8211; Verify your Logs Explorer filter uses the correct <code>logName<\/code>.\n   &#8211; Ensure you generated traffic (ping\/curl).\n   &#8211; Check IAM: you need <code>roles\/logging.viewer<\/code> to read logs.<\/p>\n<\/li>\n<li>\n<p><strong><code>gcloud compute ssh --tunnel-through-iap<\/code> fails<\/strong>\n   &#8211; Ensure you created the <code>allow-iap-ssh<\/code> firewall rule for <code>35.235.240.0\/20<\/code> and applied <code>--tags=iap-ssh<\/code> to the instances.\n   &#8211; Ensure your user has <code>roles\/iap.tunnelResourceAccessor<\/code>.\n   &#8211; Verify the instance has access to required OS Login settings if enforced in your org.<\/p>\n<\/li>\n<li>\n<p><strong>BigQuery sink exports but no tables appear<\/strong>\n   &#8211; Confirm dataset permissions granted to the sink writer identity.\n   &#8211; Confirm sink filter matches the log stream exactly.\n   &#8211; Generate more traffic and wait a few minutes.<\/p>\n<\/li>\n<li>\n<p><strong>Field names don\u2019t match the example<\/strong>\n   &#8211; Output JSON from Cloud Logging and inspect actual keys.\n   &#8211; Use BigQuery schema view and adjust queries accordingly.\n   &#8211; Always verify log schema in official docs.<\/p>\n<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Cleanup<\/h3>\n\n\n\n<p>To avoid ongoing charges, delete resources created in this lab.<\/p>\n\n\n\n<pre><code class=\"language-bash\"># Delete VMs\ngcloud compute instances delete vm-a vm-b --zone=\"$ZONE\" --quiet\n\n# Delete firewall rules\ngcloud compute firewall-rules delete allow-internal-icmp-8080 allow-iap-ssh --quiet\n\n# Delete subnet and VPC\ngcloud compute networks subnets delete flow-subnet --region=\"$REGION\" --quiet\ngcloud compute networks delete vpc-flowlab --quiet\n<\/code><\/pre>\n\n\n\n<p>If you created the BigQuery export:<\/p>\n\n\n\n<pre><code class=\"language-bash\"># Delete the logging sink\ngcloud logging sinks delete vpc-flows-to-bq --quiet\n\n# Delete the BigQuery dataset (deletes tables inside)\nbq rm -r -f -d \"${PROJECT_ID}:vpc_flow_logs\"\n<\/code><\/pre>\n\n\n\n<p>Cloud Logging entries may remain according to your project\u2019s retention settings.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">11. Best Practices<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Architecture best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Enable flow logs strategically:<\/strong> start with boundary subnets (ingress\/egress), sensitive segments, and production subnets.<\/li>\n<li><strong>Use a centralized logging\/analytics pattern:<\/strong> export flow logs to a central project\/dataset where security and SRE teams can query consistently.<\/li>\n<li><strong>Combine with other telemetry:<\/strong> VPC Flow Logs + firewall rule logging + load balancer logs + Cloud NAT logs often provides a more complete story than any single signal.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">IAM\/security best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Restrict who can read flow logs:<\/strong> internal IPs and communication patterns are sensitive.<\/li>\n<li><strong>Use least privilege for sinks:<\/strong> grant sink writer identities only dataset-level permissions they need.<\/li>\n<li><strong>Separate duties:<\/strong> network admins configure flow logs; security\/ops teams consume analytics with read-only access.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Cost best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Tune sampling and aggregation:<\/strong> define environment defaults:<\/li>\n<li>Prod: moderate sampling, moderate aggregation<\/li>\n<li>Dev\/test: higher sampling temporarily for troubleshooting<\/li>\n<li><strong>Filter exports:<\/strong> export only the logs you truly need to BigQuery\/SIEM.<\/li>\n<li><strong>Set retention intentionally:<\/strong> keep high-detail logs for a short period; archive longer-term if required.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Performance best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Avoid over-logging:<\/strong> enabling full sampling + short aggregation on many high-traffic subnets can create significant log volume and operational overhead.<\/li>\n<li><strong>Prefer BigQuery for long-range analytics:<\/strong> Cloud Logging is great for search; BigQuery is better for large-scale joins and long time windows.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Reliability best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Treat flow logs as best-effort telemetry:<\/strong> do not build mission-critical logic assuming every flow is logged.<\/li>\n<li><strong>Test in staging:<\/strong> validate sampling\/aggregation values and export pipelines before deploying org-wide.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Operations best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Create standard queries and dashboards:<\/strong> common filters for denied flows, top talkers, unusual ports.<\/li>\n<li><strong>Use log-based metrics for alerting:<\/strong> focus on high-signal conditions to avoid alert fatigue.<\/li>\n<li><strong>Document ownership:<\/strong> who changes sampling, who owns BigQuery datasets, who manages retention.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Governance\/tagging\/naming best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Use consistent naming for:<\/li>\n<li>Subnets: <code>prod-app-uscentral1<\/code>, <code>prod-db-uscentral1<\/code><\/li>\n<li>Sinks: <code>sink-vpcflows-prod-to-bq<\/code><\/li>\n<li>Datasets: <code>net_telemetry_vpcflows<\/code><\/li>\n<li>Label resources (where supported) to support chargeback and ownership mapping.<\/li>\n<li>Use org policies where appropriate to standardize logging posture (verify org policy capabilities relevant to logging and networking in your environment).<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">12. Security Considerations<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Identity and access model<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Cloud Logging access<\/strong> is controlled by IAM. Anyone with permission to read logs can see flow metadata.<\/li>\n<li><strong>Log Router sinks<\/strong> use a dedicated writer identity (service account). You must grant it access to the destination.<\/li>\n<li>In enterprises, consider:<\/li>\n<li>Centralizing logs into a dedicated project with tightly controlled IAM.<\/li>\n<li>Granting analysts access via groups and predefined roles.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Encryption<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Logs in Google Cloud are encrypted at rest by default (Google-managed encryption).  <\/li>\n<li>If you require customer-managed encryption keys (CMEK), verify whether your chosen log storage\/export destination supports it and how to configure it (for example, BigQuery and Cloud Storage support CMEK under specific configurations\u2014verify current capabilities and limitations).<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Network exposure<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>VPC Flow Logs does not expose workloads directly, but the logs can reveal:<\/li>\n<li>Internal IP ranges<\/li>\n<li>Service communication patterns<\/li>\n<li>Potentially sensitive destinations\nTreat flow logs as sensitive operational\/security data.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Secrets handling<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Flow logs are not intended to capture payloads, so application secrets should not appear in them.  <\/li>\n<li>However, flow metadata can still be sensitive (e.g., identifying a database port or a restricted subnet).<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Audit\/logging<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Use <strong>Cloud Audit Logs<\/strong> to track changes to subnet configurations and sink creation.<\/li>\n<li>In regulated environments, keep:<\/li>\n<li>Change management records for flow log configuration<\/li>\n<li>Retention and access reviews for log datasets<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Compliance considerations<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Retention requirements vary (PCI, HIPAA, SOC 2, ISO). Ensure:<\/li>\n<li>Retention meets policy<\/li>\n<li>Access is restricted and reviewed<\/li>\n<li>Data residency needs are considered (region of BigQuery dataset \/ storage bucket)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Common security mistakes<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Enabling flow logs everywhere with broad read access (unnecessary exposure).<\/li>\n<li>Exporting to BigQuery without dataset-level IAM hygiene.<\/li>\n<li>Keeping logs too long without a reason (increases exposure and cost).<\/li>\n<li>Assuming flow logs are complete enough to serve as the only evidence source.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Secure deployment recommendations<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Start with a <strong>centralized logging project<\/strong> and <strong>least privilege<\/strong> access.<\/li>\n<li>Export only required subsets to SIEM; keep raw flow logs access restricted.<\/li>\n<li>Use dataset\/table partitioning and time-bounded queries for both cost and operational safety.<\/li>\n<li>Document and test incident response queries ahead of time.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">13. Limitations and Gotchas<\/h2>\n\n\n\n<p>The exact details can vary by product evolution; always confirm in official docs. Key practical limitations include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Sampling means incomplete visibility:<\/strong> you might not see every short-lived or low-volume connection.<\/li>\n<li><strong>Aggregation reduces timing precision:<\/strong> flows are summarized over intervals; exact per-packet timing is not available.<\/li>\n<li><strong>Not a packet capture tool:<\/strong> no payload, no full headers beyond flow metadata fields.<\/li>\n<li><strong>Log volume can grow quickly:<\/strong> high-traffic subnets + high sampling + short aggregation can create very large ingestion.<\/li>\n<li><strong>Schema evolution:<\/strong> log fields can change over time; build queries defensively and validate schema periodically.<\/li>\n<li><strong>Export pipeline complexity:<\/strong> BigQuery sink permissions are a frequent stumbling block (dataset-level IAM).<\/li>\n<li><strong>Not all \u201cnetwork events\u201d are flows:<\/strong> some failures happen before a flow is established (DNS issues, application errors, or routing misconfigurations outside observed scope).<\/li>\n<li><strong>Multiple telemetry sources may be required:<\/strong> load balancer traffic behavior, NAT translations, and firewall evaluations may require their own logging features for complete context.<\/li>\n<li><strong>Quotas and rate limits:<\/strong> Cloud Logging quotas can impact very high-volume exports; monitor logging usage and quotas.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">14. Comparison with Alternatives<\/h2>\n\n\n\n<p>VPC Flow Logs is one tool in a larger network observability toolkit.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Comparison table<\/h3>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th>Option<\/th>\n<th>Best For<\/th>\n<th>Strengths<\/th>\n<th>Weaknesses<\/th>\n<th>When to Choose<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><strong>Google Cloud VPC Flow Logs<\/strong><\/td>\n<td>Network flow visibility in VPC subnets<\/td>\n<td>Native, scalable, integrates with Cloud Logging\/BigQuery, tunable sampling<\/td>\n<td>Sampled\/aggregated, no payload, can be high-volume<\/td>\n<td>Default choice for subnet-level network telemetry in Google Cloud<\/td>\n<\/tr>\n<tr>\n<td><strong>Firewall Rules Logging (Google Cloud)<\/strong><\/td>\n<td>Understanding firewall allow\/deny decisions<\/td>\n<td>Directly ties to firewall actions, great for policy validation<\/td>\n<td>Not a full picture of all traffic volume; different scope than flow logs<\/td>\n<td>When debugging\/validating firewall policy outcomes<\/td>\n<\/tr>\n<tr>\n<td><strong>Cloud NAT Logging (Google Cloud)<\/strong><\/td>\n<td>Visibility into NAT translations and egress<\/td>\n<td>Helps trace outbound connections through NAT<\/td>\n<td>Not a substitute for subnet-level flow visibility<\/td>\n<td>When troubleshooting egress\/NAT behavior or attribution<\/td>\n<\/tr>\n<tr>\n<td><strong>Load Balancer logging (Google Cloud)<\/strong><\/td>\n<td>HTTP(S)\/TCP proxy behavior and client requests<\/td>\n<td>Application-facing visibility and request logs<\/td>\n<td>Not a replacement for east-west flow visibility<\/td>\n<td>When troubleshooting client traffic and LB behavior<\/td>\n<\/tr>\n<tr>\n<td><strong>Packet Mirroring (Google Cloud)<\/strong><\/td>\n<td>Deep packet inspection \/ payload analysis<\/td>\n<td>Full packet capture capabilities (via collector tools)<\/td>\n<td>Higher complexity, storage\/processing heavy, privacy concerns<\/td>\n<td>When you need payload-level forensics\/IDS-like analysis<\/td>\n<\/tr>\n<tr>\n<td><strong>AWS VPC Flow Logs<\/strong><\/td>\n<td>Similar capability in AWS<\/td>\n<td>Mature flow logging ecosystem<\/td>\n<td>Different schema\/tooling; not applicable to Google Cloud<\/td>\n<td>Choose when operating in AWS environments<\/td>\n<\/tr>\n<tr>\n<td><strong>Azure NSG Flow Logs<\/strong><\/td>\n<td>Similar capability in Azure<\/td>\n<td>Integrates with Azure networking and monitoring<\/td>\n<td>Different schema\/tooling; not applicable to Google Cloud<\/td>\n<td>Choose when operating in Azure environments<\/td>\n<\/tr>\n<tr>\n<td><strong>Self-managed NetFlow\/sFlow\/host logging<\/strong><\/td>\n<td>Custom or on-prem\/hybrid environments<\/td>\n<td>Full control, can integrate with existing tools<\/td>\n<td>Operational burden, scaling complexity, inconsistent coverage<\/td>\n<td>When you must standardize across multi-cloud\/on-prem with custom pipelines<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">15. Real-World Example<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Enterprise example: Centralized threat hunting for a Shared VPC organization<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> A large enterprise runs hundreds of projects using Shared VPC. Security needs to detect lateral movement and unexpected egress, and SRE needs faster network troubleshooting across teams.<\/li>\n<li><strong>Proposed architecture:<\/strong><\/li>\n<li>Enable VPC Flow Logs on production and sensitive subnets in the Shared VPC host project(s).<\/li>\n<li>Route flow logs to a central logging project using Cloud Logging sinks.<\/li>\n<li>Export to BigQuery in a centralized dataset with strict IAM (security analysts, SRE read-only groups).<\/li>\n<li>Build standardized BigQuery views and scheduled queries for:<ul>\n<li>top talkers by subnet<\/li>\n<li>denied flow spikes<\/li>\n<li>unusual ports<\/li>\n<li>outbound to suspicious IP ranges (fed by threat intel lists)<\/li>\n<\/ul>\n<\/li>\n<li><strong>Why VPC Flow Logs was chosen:<\/strong><\/li>\n<li>Native Google Cloud Networking integration.<\/li>\n<li>Works across many projects with centralized export patterns.<\/li>\n<li>Sampling\/aggregation controls keep costs manageable.<\/li>\n<li><strong>Expected outcomes:<\/strong><\/li>\n<li>Faster incident response (one place to query flows).<\/li>\n<li>Reduced mean time to resolution (MTTR) for connectivity tickets.<\/li>\n<li>Improved policy compliance validation and evidence trails.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Startup \/ small-team example: Debugging microservice connectivity and controlling egress<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> A startup runs a small set of services on Compute Engine and sees intermittent timeouts plus unexplained egress spend.<\/li>\n<li><strong>Proposed architecture:<\/strong><\/li>\n<li>Enable VPC Flow Logs only on the primary production subnet.<\/li>\n<li>Keep a short retention in Cloud Logging for quick investigations.<\/li>\n<li>Export only production flow logs to a small BigQuery dataset.<\/li>\n<li>Use a few saved queries:<ul>\n<li>\u201cDenied flows last 1 hour\u201d<\/li>\n<li>\u201cTop egress destinations by bytes\u201d<\/li>\n<li>\u201cTraffic to database port over time\u201d<\/li>\n<\/ul>\n<\/li>\n<li><strong>Why VPC Flow Logs was chosen:<\/strong><\/li>\n<li>Minimal operational setup (no agents).<\/li>\n<li>BigQuery enables quick ad-hoc queries when issues occur.<\/li>\n<li><strong>Expected outcomes:<\/strong><\/li>\n<li>Identify misconfigured dependencies and chatty services quickly.<\/li>\n<li>Find unexpected outbound destinations driving egress.<\/li>\n<li>Maintain low operational overhead with targeted logging.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">16. FAQ<\/h2>\n\n\n\n<ol class=\"wp-block-list\">\n<li>\n<p><strong>Is VPC Flow Logs a separate Google Cloud product I need to deploy?<\/strong><br\/>\n   No. VPC Flow Logs is a <strong>feature of Google Cloud VPC<\/strong>. You enable it on subnets, and logs appear in Cloud Logging.<\/p>\n<\/li>\n<li>\n<p><strong>Where do VPC Flow Logs show up?<\/strong><br\/>\n   In <strong>Cloud Logging<\/strong>, typically under a log name like <code>compute.googleapis.com\/vpc_flows<\/code>.<\/p>\n<\/li>\n<li>\n<p><strong>Is every packet logged?<\/strong><br\/>\n   No. VPC Flow Logs records <strong>flow metadata<\/strong>, usually <strong>sampled<\/strong> and <strong>aggregated<\/strong>.<\/p>\n<\/li>\n<li>\n<p><strong>Can I set sampling to 100%?<\/strong><br\/>\n   Sampling configuration supports higher rates, but exact allowed values and behavior should be verified in official docs and tested in your environment. Even with high sampling, treat logs as telemetry, not perfect capture.<\/p>\n<\/li>\n<li>\n<p><strong>How long does it take for flow logs to appear?<\/strong><br\/>\n   It can take minutes. Expect some delay between generating traffic and seeing log entries.<\/p>\n<\/li>\n<li>\n<p><strong>Can I enable VPC Flow Logs for only one environment (prod) and not dev?<\/strong><br\/>\n   Yes. Because it is configured per subnet, you can enable it only where required.<\/p>\n<\/li>\n<li>\n<p><strong>Do flow logs include application payload data?<\/strong><br\/>\n   No. They include metadata such as IPs, ports, protocol, byte\/packet counts, and timestamps.<\/p>\n<\/li>\n<li>\n<p><strong>How do I analyze flow logs at scale?<\/strong><br\/>\n   Export to <strong>BigQuery<\/strong> using a Log Router sink and query over time ranges, join with CMDB\/asset inventory, and build dashboards.<\/p>\n<\/li>\n<li>\n<p><strong>Why don\u2019t I see logs for certain traffic?<\/strong><br\/>\n   Common reasons: flow logs not enabled on the correct subnet, sampling missed the flow, aggregation\/time window issues, traffic doesn\u2019t traverse the expected network interface, or log filters are incorrect. Verify coverage in official docs for your workload type.<\/p>\n<\/li>\n<li>\n<p><strong>Do VPC Flow Logs help debug firewall rule issues?<\/strong><br\/>\n   They help you see observed flows and metadata, but for direct firewall action logging you may also want <strong>Firewall Rules Logging<\/strong>.<\/p>\n<\/li>\n<li>\n<p><strong>Are there privacy concerns with VPC Flow Logs?<\/strong><br\/>\n   Yes. Internal IPs, communication patterns, and service behavior can be sensitive. Apply strict IAM controls and retention policies.<\/p>\n<\/li>\n<li>\n<p><strong>Can I export VPC Flow Logs to Cloud Storage instead of BigQuery?<\/strong><br\/>\n   Yes. Cloud Logging sinks can route logs to Cloud Storage. This is often used for low-cost archival (with tradeoffs in queryability).<\/p>\n<\/li>\n<li>\n<p><strong>How do I reduce VPC Flow Logs cost?<\/strong><br\/>\n   Reduce sampling, increase aggregation interval, limit metadata, enable only on key subnets, and export only what you need.<\/p>\n<\/li>\n<li>\n<p><strong>Can I build alerts on flow logs?<\/strong><br\/>\n   Yes. Use Cloud Logging queries + log-based metrics + Cloud Monitoring alerting.<\/p>\n<\/li>\n<li>\n<p><strong>What is the difference between VPC Flow Logs and Packet Mirroring?<\/strong><br\/>\n   VPC Flow Logs records <strong>metadata about flows<\/strong>. Packet Mirroring provides <strong>packet-level visibility<\/strong> suitable for deep inspection tools, with higher complexity and cost.<\/p>\n<\/li>\n<li>\n<p><strong>Do flow logs work across projects with Shared VPC?<\/strong><br\/>\n   Yes, but you must design log access and exports carefully. Centralized logging patterns are common.<\/p>\n<\/li>\n<li>\n<p><strong>What\u2019s the best retention strategy?<\/strong><br\/>\n   Keep short retention in Cloud Logging for fast investigations, export selected logs to BigQuery for analytics, and optionally archive to Cloud Storage if compliance requires long retention.<\/p>\n<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">17. Top Online Resources to Learn VPC Flow Logs<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th>Resource Type<\/th>\n<th>Name<\/th>\n<th>Why It Is Useful<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Official documentation<\/td>\n<td>VPC Flow Logs overview and usage<\/td>\n<td>Primary reference for configuration, behavior, and limitations: https:\/\/cloud.google.com\/vpc\/docs\/using-flow-logs<\/td>\n<\/tr>\n<tr>\n<td>Official documentation<\/td>\n<td>Cloud Logging documentation<\/td>\n<td>Understand log storage, querying, retention, and routing: https:\/\/cloud.google.com\/logging\/docs<\/td>\n<\/tr>\n<tr>\n<td>Official documentation<\/td>\n<td>Log Router overview<\/td>\n<td>Learn how to route\/export flow logs to BigQuery\/Storage\/PubSub: https:\/\/cloud.google.com\/logging\/docs\/routing\/overview<\/td>\n<\/tr>\n<tr>\n<td>Official tutorial\/docs<\/td>\n<td>Configure and manage log sinks<\/td>\n<td>Step-by-step sink configuration and permissions: https:\/\/cloud.google.com\/logging\/docs\/export\/configure_export_v2<\/td>\n<\/tr>\n<tr>\n<td>Official pricing<\/td>\n<td>Cloud Logging pricing<\/td>\n<td>Flow logs cost is primarily logging ingestion\/retention: https:\/\/cloud.google.com\/logging\/pricing<\/td>\n<\/tr>\n<tr>\n<td>Official tool<\/td>\n<td>Google Cloud Pricing Calculator<\/td>\n<td>Model Logging, BigQuery, and Storage costs: https:\/\/cloud.google.com\/products\/calculator<\/td>\n<\/tr>\n<tr>\n<td>CLI reference<\/td>\n<td><code>gcloud compute networks subnets<\/code> reference<\/td>\n<td>Verify current flags for enabling flow logs: https:\/\/cloud.google.com\/sdk\/gcloud\/reference\/compute\/networks\/subnets<\/td>\n<\/tr>\n<tr>\n<td>Official quotas<\/td>\n<td>Cloud Logging quotas<\/td>\n<td>Plan high-volume telemetry and avoid throttling: https:\/\/cloud.google.com\/logging\/quotas<\/td>\n<\/tr>\n<tr>\n<td>Official learning<\/td>\n<td>Google Cloud Skills Boost<\/td>\n<td>Hands-on labs often include Logging\/networking analysis (search within): https:\/\/www.cloudskillsboost.google\/<\/td>\n<\/tr>\n<tr>\n<td>Video (official channel)<\/td>\n<td>Google Cloud Tech (YouTube)<\/td>\n<td>Look for Logging and Networking observability videos: https:\/\/www.youtube.com\/@googlecloudtech<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">18. Training and Certification Providers<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th>Institute<\/th>\n<th>Suitable Audience<\/th>\n<th>Likely Learning Focus<\/th>\n<th>Mode<\/th>\n<th>Website URL<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>DevOpsSchool.com<\/td>\n<td>DevOps engineers, SREs, cloud engineers<\/td>\n<td>Google Cloud operations, logging, networking fundamentals, practical labs<\/td>\n<td>check website<\/td>\n<td>https:\/\/www.devopsschool.com\/<\/td>\n<\/tr>\n<tr>\n<td>ScmGalaxy.com<\/td>\n<td>Beginners to intermediate DevOps learners<\/td>\n<td>DevOps tooling, CI\/CD, cloud basics, operational practices<\/td>\n<td>check website<\/td>\n<td>https:\/\/www.scmgalaxy.com\/<\/td>\n<\/tr>\n<tr>\n<td>CLoudOpsNow.in<\/td>\n<td>Cloud operations teams<\/td>\n<td>CloudOps practices, monitoring\/logging, incident response workflows<\/td>\n<td>check website<\/td>\n<td>https:\/\/www.cloudopsnow.in\/<\/td>\n<\/tr>\n<tr>\n<td>SreSchool.com<\/td>\n<td>SREs, platform engineers<\/td>\n<td>Reliability engineering, observability, production operations<\/td>\n<td>check website<\/td>\n<td>https:\/\/www.sreschool.com\/<\/td>\n<\/tr>\n<tr>\n<td>AiOpsSchool.com<\/td>\n<td>Operations and platform teams exploring AIOps<\/td>\n<td>AIOps concepts, monitoring analytics, automation approaches<\/td>\n<td>check website<\/td>\n<td>https:\/\/www.aiopsschool.com\/<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">19. Top Trainers<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th>Platform\/Site<\/th>\n<th>Likely Specialization<\/th>\n<th>Suitable Audience<\/th>\n<th>Website URL<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>RajeshKumar.xyz<\/td>\n<td>DevOps\/cloud training content and guidance (verify specific offerings)<\/td>\n<td>Engineers seeking structured coaching<\/td>\n<td>https:\/\/www.rajeshkumar.xyz\/<\/td>\n<\/tr>\n<tr>\n<td>devopstrainer.in<\/td>\n<td>DevOps training services (verify course list)<\/td>\n<td>Beginners to intermediate DevOps practitioners<\/td>\n<td>https:\/\/www.devopstrainer.in\/<\/td>\n<\/tr>\n<tr>\n<td>devopsfreelancer.com<\/td>\n<td>Freelance DevOps consulting\/training platform (verify services)<\/td>\n<td>Teams needing short-term, hands-on enablement<\/td>\n<td>https:\/\/www.devopsfreelancer.com\/<\/td>\n<\/tr>\n<tr>\n<td>devopssupport.in<\/td>\n<td>DevOps support and training resources (verify offerings)<\/td>\n<td>Ops teams needing practical troubleshooting skills<\/td>\n<td>https:\/\/www.devopssupport.in\/<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">20. Top Consulting Companies<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th>Company Name<\/th>\n<th>Likely Service Area<\/th>\n<th>Where They May Help<\/th>\n<th>Consulting Use Case Examples<\/th>\n<th>Website URL<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>cotocus.com<\/td>\n<td>Cloud\/DevOps consulting (verify exact portfolio)<\/td>\n<td>Architecture reviews, observability setup, cloud operations improvements<\/td>\n<td>Centralized logging design, cost optimization for logging, network telemetry rollout<\/td>\n<td>https:\/\/www.cotocus.com\/<\/td>\n<\/tr>\n<tr>\n<td>DevOpsSchool.com<\/td>\n<td>DevOps and cloud consulting\/training<\/td>\n<td>Cloud migrations, DevOps platform setup, operational excellence<\/td>\n<td>Implementing logging pipelines, creating runbooks for network troubleshooting, building monitoring\/alerting patterns<\/td>\n<td>https:\/\/www.devopsschool.com\/<\/td>\n<\/tr>\n<tr>\n<td>DEVOPSCONSULTING.IN<\/td>\n<td>DevOps consulting services (verify specific capabilities)<\/td>\n<td>DevOps transformation, tooling, cloud operations<\/td>\n<td>Setting up log routing to BigQuery, standardizing IAM for observability data, building dashboards for network flows<\/td>\n<td>https:\/\/www.devopsconsulting.in\/<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">21. Career and Learning Roadmap<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">What to learn before VPC Flow Logs<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Google Cloud VPC fundamentals:<\/li>\n<li>VPCs, subnets, routes, firewall rules<\/li>\n<li>Private vs public IPs, NAT basics<\/li>\n<li>Cloud Logging fundamentals:<\/li>\n<li>Logs Explorer queries<\/li>\n<li>Log buckets, retention, exclusions<\/li>\n<li>Log Router sinks<\/li>\n<li>Basic Linux networking:<\/li>\n<li>TCP\/UDP, ports, ICMP<\/li>\n<li>Tools like <code>curl<\/code>, <code>ping<\/code>, <code>ss<\/code>, <code>netstat<\/code><\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">What to learn after VPC Flow Logs<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>BigQuery for log analytics:<\/li>\n<li>Partitioning, clustering (where applicable)<\/li>\n<li>Cost-aware querying<\/li>\n<li>Security operations on Google Cloud:<\/li>\n<li>Incident response playbooks<\/li>\n<li>Integration patterns with SIEM tooling<\/li>\n<li>Broader Google Cloud Networking observability:<\/li>\n<li>Firewall Rules Logging<\/li>\n<li>Cloud NAT logging<\/li>\n<li>Load balancer logging<\/li>\n<li>Packet Mirroring (for deeper inspection use cases)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Job roles that use it<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Cloud network engineer<\/li>\n<li>Site Reliability Engineer (SRE)<\/li>\n<li>DevOps engineer \/ platform engineer<\/li>\n<li>Security engineer \/ SOC analyst (for cloud telemetry)<\/li>\n<li>Cloud solutions architect<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Certification path (Google Cloud)<\/h3>\n\n\n\n<p>Google Cloud certifications evolve. A practical path often includes:\n&#8211; Associate Cloud Engineer (broad foundation)\n&#8211; Professional Cloud Network Engineer (network specialization)\n&#8211; Professional Cloud Security Engineer (security specialization)<\/p>\n\n\n\n<p>Verify current certification details in official Google Cloud certification pages: https:\/\/cloud.google.com\/learn\/certification<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Project ideas for practice<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Build a \u201ctop talkers\u201d dashboard in BigQuery from VPC Flow Logs.<\/li>\n<li>Create log-based metrics for denied flows and alert when they spike.<\/li>\n<li>Implement centralized logging: export flow logs from multiple projects into a single BigQuery dataset.<\/li>\n<li>Run a controlled \u201cattack simulation\u201d in a lab (port scan within allowed boundaries) and detect it using flow logs.<\/li>\n<li>Compare traffic patterns before and after a firewall policy change.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">22. Glossary<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>VPC (Virtual Private Cloud):<\/strong> A logically isolated network in Google Cloud where you run resources with IP addressing, routing, and firewall policies.<\/li>\n<li><strong>Subnet (subnetwork):<\/strong> A regional IP range within a VPC where VM interfaces are attached.<\/li>\n<li><strong>Flow (network flow):<\/strong> A set of packets sharing common properties (often a 5-tuple) over a period of time.<\/li>\n<li><strong>5-tuple:<\/strong> Source IP, destination IP, source port, destination port, protocol.<\/li>\n<li><strong>Sampling rate:<\/strong> The fraction of flows captured and logged (used to control volume and cost).<\/li>\n<li><strong>Aggregation interval:<\/strong> The time window over which flow observations are summarized into a log entry.<\/li>\n<li><strong>Cloud Logging:<\/strong> Google Cloud service for collecting, storing, querying, and routing logs.<\/li>\n<li><strong>Log Router:<\/strong> Cloud Logging component that routes logs to destinations using sinks and filters.<\/li>\n<li><strong>Log sink:<\/strong> A configuration that exports selected logs to a destination like BigQuery, Cloud Storage, or Pub\/Sub.<\/li>\n<li><strong>BigQuery:<\/strong> Google Cloud\u2019s data warehouse used for large-scale querying and analytics.<\/li>\n<li><strong>Pub\/Sub:<\/strong> Messaging service used for streaming data pipelines.<\/li>\n<li><strong>Cloud Storage:<\/strong> Object storage for archiving and durable storage.<\/li>\n<li><strong>Shared VPC:<\/strong> A Google Cloud design where a host project provides networking to service projects.<\/li>\n<li><strong>IAM (Identity and Access Management):<\/strong> Google Cloud\u2019s authorization system controlling who can do what on which resources.<\/li>\n<li><strong>IAP (Identity-Aware Proxy):<\/strong> A secure access method to reach internal VMs without public IPs, commonly used for SSH via tunneling.<\/li>\n<li><strong>Telemetry:<\/strong> Observability data (logs\/metrics\/traces) used to monitor and understand systems.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">23. Summary<\/h2>\n\n\n\n<p>VPC Flow Logs is a Google Cloud Networking feature that records sampled, time-aggregated <strong>network flow metadata<\/strong> for traffic associated with subnets in your VPC. It matters because it provides practical visibility for troubleshooting connectivity, validating segmentation, investigating incidents, and understanding traffic patterns that affect both security posture and cost.<\/p>\n\n\n\n<p>In Google Cloud, VPC Flow Logs fits naturally into an observability architecture using <strong>Cloud Logging<\/strong> for search and retention, and <strong>Log Router<\/strong> exports to <strong>BigQuery<\/strong> for scalable analytics. The most important cost and security points are:\n&#8211; Costs are primarily driven by <strong>Cloud Logging ingestion\/retention<\/strong> and downstream analytics destinations.\n&#8211; Logs can contain sensitive network metadata, so apply <strong>least-privilege IAM<\/strong> and thoughtful retention.<\/p>\n\n\n\n<p>Use VPC Flow Logs when you need subnet-level network visibility without packet capture, especially in production troubleshooting and security investigations. Next, deepen your skills by exporting to BigQuery, building cost-aware queries, and combining flow logs with firewall, NAT, and load balancer logging for a more complete networking observability strategy.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Networking<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[51,50],"tags":[],"class_list":["post-772","post","type-post","status-publish","format-standard","hentry","category-google-cloud","category-networking"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/posts\/772","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/comments?post=772"}],"version-history":[{"count":0,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/posts\/772\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/media?parent=772"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/categories?post=772"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/tags?post=772"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}