{"id":645,"date":"2026-04-14T21:11:20","date_gmt":"2026-04-14T21:11:20","guid":{"rendered":"https:\/\/www.devopsschool.com\/tutorials\/google-cloud-bigquery-ai-tutorial-architecture-pricing-use-cases-and-hands-on-guide-for-data-analytics-and-pipelines\/"},"modified":"2026-04-14T21:11:20","modified_gmt":"2026-04-14T21:11:20","slug":"google-cloud-bigquery-ai-tutorial-architecture-pricing-use-cases-and-hands-on-guide-for-data-analytics-and-pipelines","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/tutorials\/google-cloud-bigquery-ai-tutorial-architecture-pricing-use-cases-and-hands-on-guide-for-data-analytics-and-pipelines\/","title":{"rendered":"Google Cloud BigQuery AI Tutorial: Architecture, Pricing, Use Cases, and Hands-On Guide for Data analytics and pipelines"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Category<\/h2>\n\n\n\n<p>Data analytics and pipelines<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. Introduction<\/h2>\n\n\n\n<p>BigQuery AI is Google Cloud\u2019s umbrella for building and using AI\/ML capabilities <em>directly inside BigQuery<\/em>\u2014using SQL (and, in some workflows, Python) to train models, run predictions, generate text, create embeddings, and operationalize AI next to your data.<\/p>\n\n\n\n<p>In simple terms: <strong>BigQuery AI lets data teams do AI where the data already lives<\/strong>. Instead of exporting data to separate systems, you can often train and serve models, enrich data, and run AI-powered analysis within BigQuery.<\/p>\n\n\n\n<p>Technically: BigQuery AI is not a single separate product; it is a collection of BigQuery features and integrations\u2014most notably <strong>BigQuery ML (BQML)<\/strong> and <strong>BigQuery + Vertex AI integrations<\/strong>\u2014that allow you to create models, call remote models, and apply AI functions from SQL. These capabilities operate within BigQuery\u2019s job execution model, dataset locations, IAM permissions, and audit\/monitoring surfaces.<\/p>\n\n\n\n<p><strong>What problem it solves:<\/strong> it reduces the friction between analytics and machine learning by minimizing data movement, simplifying operationalization (SQL-first), and making AI\/ML accessible to teams that already standardize on BigQuery for data analytics and pipelines.<\/p>\n\n\n\n<blockquote>\n<p>Naming note (important): Google Cloud documentation and product marketing increasingly use <strong>\u201cBigQuery AI\u201d<\/strong> to refer to AI capabilities in BigQuery, while <strong>BigQuery ML<\/strong> remains the core, long-standing feature set for ML in BigQuery. If you see \u201cBigQuery AI\u201d vs \u201cBigQuery ML\u201d in docs, treat BigQuery AI as the <em>umbrella<\/em>, and BigQuery ML as a <em>key component<\/em>. Verify the latest terminology in official docs: https:\/\/cloud.google.com\/bigquery\/docs<\/p>\n<\/blockquote>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">2. What is BigQuery AI?<\/h2>\n\n\n\n<p><strong>Official purpose (in practice):<\/strong> BigQuery AI enables you to <strong>build, evaluate, and use ML models and AI functions in BigQuery<\/strong>, and to integrate BigQuery data with Google Cloud AI services (commonly Vertex AI) without building a separate ML platform for many common use cases.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Core capabilities<\/h3>\n\n\n\n<p>BigQuery AI typically includes:\n&#8211; <strong>BigQuery ML (BQML):<\/strong> Train and run ML models with SQL (classification, regression, clustering, time series, matrix factorization, etc., depending on current support).\n&#8211; <strong>Inference in SQL:<\/strong> Use <code>ML.PREDICT<\/code>, <code>ML.EVALUATE<\/code>, and related functions.\n&#8211; <strong>Generative AI \/ foundation model integration (where available):<\/strong> Call hosted models through supported functions and \u201cremote model\u201d patterns (often leveraging Vertex AI behind the scenes). Availability can be region- and feature-release-dependent\u2014verify in official docs for your project\/region.\n&#8211; <strong>Embeddings + vector search patterns (where available):<\/strong> Create embeddings for text (and sometimes other modalities) and perform similarity search in BigQuery using vector features. Verify current BigQuery \u201cvector search\u201d docs and limitations.\n&#8211; <strong>Operationalization in BigQuery:<\/strong> Scheduled queries, authorized views, Dataform SQL pipelines, governance, and audit logging.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Major components (conceptual)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>BigQuery datasets and tables<\/strong> (your governed data lakehouse layer)<\/li>\n<li><strong>BigQuery jobs<\/strong> (queries and ML jobs that execute in a location)<\/li>\n<li><strong>BigQuery ML models<\/strong> (stored as BigQuery model resources)<\/li>\n<li><strong>Connections \/ integrations<\/strong> (for calling external services such as Vertex AI, when applicable)<\/li>\n<li><strong>IAM + policy controls<\/strong> (dataset\/table\/model permissions)<\/li>\n<li><strong>Monitoring\/audit surfaces<\/strong> (Cloud Logging audit logs, <code>INFORMATION_SCHEMA<\/code> views)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Service type<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Managed analytics platform feature set<\/strong> within <strong>BigQuery<\/strong> (serverless data warehouse \/ lakehouse).<\/li>\n<li>BigQuery AI capabilities are consumed through <strong>SQL<\/strong>, BigQuery Console\/Studio, APIs, and client libraries.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Resource scope (practical scoping)<\/h3>\n\n\n\n<p>BigQuery itself is a <strong>global service<\/strong> with <strong>data and job execution bound to locations<\/strong>:\n&#8211; <strong>Datasets are created in a location<\/strong> (e.g., <code>US<\/code>, <code>EU<\/code>, or a specific region).\n&#8211; <strong>Jobs execute in the dataset location<\/strong> (location mismatch is a common operational issue).\n&#8211; BigQuery ML models are created in datasets, so they inherit <strong>dataset location constraints<\/strong>.\n&#8211; Integrations (for example, to Vertex AI) are typically <strong>regional<\/strong> and must align with your data location and supported regions. <strong>Verify in official docs<\/strong> for your region.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">How it fits into the Google Cloud ecosystem<\/h3>\n\n\n\n<p>BigQuery AI commonly sits at the center of Google Cloud\u2019s <strong>Data analytics and pipelines<\/strong> stack:\n&#8211; Ingest\/stream: Pub\/Sub, Dataflow, Datastream\n&#8211; Transform: Dataform, BigQuery SQL, Dataproc (Spark), Dataflow (Beam)\n&#8211; Govern: Dataplex, Data Catalog, IAM, DLP\n&#8211; AI: Vertex AI (training\/serving, model endpoints, foundation models)\n&#8211; BI: Looker, Looker Studio\n&#8211; Ops: Cloud Monitoring\/Logging, Cloud KMS, Secret Manager (where needed)<\/p>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">3. Why use BigQuery AI?<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Business reasons<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Faster time-to-insight:<\/strong> Analysts can build predictive features and models using familiar SQL workflows.<\/li>\n<li><strong>Lower platform overhead:<\/strong> Many use cases avoid building\/operating separate ML infrastructure.<\/li>\n<li><strong>Better data leverage:<\/strong> AI\/ML happens next to governed, curated datasets and lineage.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Technical reasons<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Minimize data movement:<\/strong> Training\/inference can often occur within BigQuery\u2019s execution engine.<\/li>\n<li><strong>SQL-first ML:<\/strong> Ideal for teams with strong SQL skills; integrates with existing ELT pipelines.<\/li>\n<li><strong>Integrated governance:<\/strong> Uses BigQuery IAM, dataset scoping, authorized views, and audit logs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Operational reasons<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Repeatable pipelines:<\/strong> Schedule training\/inference jobs (scheduled queries, Dataform).<\/li>\n<li><strong>Centralized monitoring:<\/strong> Query history, job metadata, audit logs, and cost controls are already in BigQuery operational practices.<\/li>\n<li><strong>Simplified deployment:<\/strong> For many classic ML workflows, \u201cdeployment\u201d can be writing predictions to a table consumed by BI tools.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Security\/compliance reasons<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Fine-grained access control:<\/strong> Table-level, column-level, row-level security (where configured) applies to training\/inference inputs and outputs.<\/li>\n<li><strong>Auditability:<\/strong> BigQuery audit logs and job metadata support compliance evidence collection.<\/li>\n<li><strong>Encryption options:<\/strong> Google-managed encryption by default; customer-managed encryption keys (CMEK) available for many BigQuery resources\u2014verify applicability for models and specific features in official docs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Scalability\/performance reasons<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Serverless scaling:<\/strong> BigQuery handles large-scale training\/inference across large datasets (within supported model types and quotas).<\/li>\n<li><strong>Separation of storage\/compute:<\/strong> You can optimize with on-demand vs reservations, clustering\/partitioning, materialized views, etc.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">When teams should choose BigQuery AI<\/h3>\n\n\n\n<p>Choose BigQuery AI when:\n&#8211; Your data is already in BigQuery and you want to <strong>predict, classify, cluster, forecast<\/strong>, or enrich data as part of SQL pipelines.\n&#8211; You want <strong>governed, auditable<\/strong> AI workflows aligned to data warehouse practices.\n&#8211; Your ML needs are aligned with supported BQML model types, or you want to <strong>call external\/hosted models<\/strong> from BigQuery (where supported).<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">When teams should not choose it<\/h3>\n\n\n\n<p>Avoid (or limit) BigQuery AI when:\n&#8211; You need <strong>custom deep learning training<\/strong>, advanced distributed training, custom GPUs\/TPUs, or complex feature pipelines better suited to Vertex AI training pipelines.\n&#8211; You require <strong>real-time low-latency online inference<\/strong> (single-digit milliseconds). BigQuery is optimized for analytics; online serving typically belongs in Vertex AI endpoints or a dedicated serving layer.\n&#8211; You need full MLOps capabilities (complex CI\/CD, model registry policies, canarying, feature store, monitoring) beyond what BigQuery-centric workflows comfortably provide\u2014Vertex AI may be a better \u201csystem of record\u201d for ML ops.<\/p>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">4. Where is BigQuery AI used?<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Industries<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Retail\/e-commerce: propensity models, recommendations, demand forecasting<\/li>\n<li>Financial services: risk scoring, anomaly detection, segmentation<\/li>\n<li>Media\/adtech: churn prediction, audience clustering, attribution modeling<\/li>\n<li>SaaS: product analytics, expansion likelihood, support ticket classification<\/li>\n<li>Healthcare\/life sciences: operations analytics, forecasting, cohort analysis (with strict compliance controls)<\/li>\n<li>Manufacturing\/IoT: anomaly detection, predictive maintenance (often with time series)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Team types<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Analytics engineering teams building SQL pipelines<\/li>\n<li>Data science teams that want fast iteration with warehouse-native ML<\/li>\n<li>BI teams that want predictive metrics in dashboards<\/li>\n<li>Platform teams standardizing governance and cost controls<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Workloads<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Batch scoring into BigQuery tables for BI\/ops consumption<\/li>\n<li>Scheduled retraining workflows driven by new data arrivals<\/li>\n<li>Enrichment pipelines (classification, entity extraction, embeddings) as part of ELT<\/li>\n<li>Segmentation\/clustering as a reusable data product<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Architectures<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Lakehouse-style: raw \u2192 curated \u2192 feature tables \u2192 model \u2192 predictions<\/li>\n<li>Event-driven ingestion + batch scoring (streaming in, scoring in intervals)<\/li>\n<li>Hybrid: BigQuery for features + Vertex AI for custom training\/serving<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Real-world deployment contexts<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Production:<\/strong> scheduled scoring tables feeding Looker dashboards; risk\/ops reports; campaign targeting exports<\/li>\n<li><strong>Dev\/test:<\/strong> model prototyping with public datasets; sandbox models using sample slices of production data<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">5. Top Use Cases and Scenarios<\/h2>\n\n\n\n<p>Below are realistic scenarios where BigQuery AI fits well. Each includes the problem, why BigQuery AI fits, and a short example.<\/p>\n\n\n\n<p>1) <strong>Churn prediction in a SaaS warehouse<\/strong>\n&#8211; <strong>Problem:<\/strong> Identify customers likely to churn based on product usage and support interactions.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> Data already in BigQuery; BQML can train a classification model; scoring writes back to a table consumed by CS teams.\n&#8211; <strong>Example:<\/strong> A daily scheduled query scores all active accounts and populates a \u201cchurn_risk\u201d table used in Looker.<\/p>\n\n\n\n<p>2) <strong>Demand forecasting for inventory planning<\/strong>\n&#8211; <strong>Problem:<\/strong> Forecast product demand per SKU\/store\/week.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> Time series models can be trained in BigQuery and run on partitioned historical sales data.\n&#8211; <strong>Example:<\/strong> Weekly retrain; daily forecast table powers procurement dashboards.<\/p>\n\n\n\n<p>3) <strong>Customer segmentation with clustering<\/strong>\n&#8211; <strong>Problem:<\/strong> Group customers into segments for targeted campaigns.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> BQML clustering (e.g., k-means) on aggregated behavioral features works well in SQL.\n&#8211; <strong>Example:<\/strong> Marketing exports segment labels to activation systems.<\/p>\n\n\n\n<p>4) <strong>Fraud or anomaly signals in transaction analytics<\/strong>\n&#8211; <strong>Problem:<\/strong> Flag unusual transactions based on patterns.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> You can compute features in SQL and apply anomaly detection approaches supported in BigQuery ML (verify current model availability).\n&#8211; <strong>Example:<\/strong> Nightly job scores transactions; analysts investigate top anomalies.<\/p>\n\n\n\n<p>5) <strong>Recommendation candidates via matrix factorization<\/strong>\n&#8211; <strong>Problem:<\/strong> Recommend items based on user-item interactions.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> BQML includes recommendation model patterns (e.g., matrix factorization) in SQL (verify supported options).\n&#8211; <strong>Example:<\/strong> Generate top-N item candidates per user and join with catalog for reporting.<\/p>\n\n\n\n<p>6) <strong>Lead scoring for sales prioritization<\/strong>\n&#8211; <strong>Problem:<\/strong> Rank leads by conversion likelihood using marketing and CRM signals.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> Classic supervised learning fits BQML; scoring table integrates with reporting\/exports.\n&#8211; <strong>Example:<\/strong> Hourly scoring table supports SDR workflows.<\/p>\n\n\n\n<p>7) <strong>Text classification of support tickets (warehouse-native enrichment)<\/strong>\n&#8211; <strong>Problem:<\/strong> Route tickets by category and urgency.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> For structured + text features, you can use BigQuery ML approaches or call out to supported text models where available (verify).\n&#8211; <strong>Example:<\/strong> New tickets ingested into BigQuery are enriched with category labels and stored for downstream tools.<\/p>\n\n\n\n<p>8) <strong>Embedding generation for semantic search analytics<\/strong>\n&#8211; <strong>Problem:<\/strong> Build semantic similarity on product descriptions or documentation.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> Embeddings can be generated (where supported) and stored in BigQuery; vector similarity queries can be done close to the data.\n&#8211; <strong>Example:<\/strong> Analysts run \u201csimilar products\u201d queries and evaluate search performance offline.<\/p>\n\n\n\n<p>9) <strong>Data quality anomaly detection for pipelines<\/strong>\n&#8211; <strong>Problem:<\/strong> Detect shifts in key metrics (null rates, outliers, distribution drift).\n&#8211; <strong>Why BigQuery AI fits:<\/strong> You can compute features and anomaly metrics in SQL; optionally apply ML to detect unusual patterns.\n&#8211; <strong>Example:<\/strong> A monitoring dataset stores daily pipeline health scores; alerts are triggered externally.<\/p>\n\n\n\n<p>10) <strong>Forecasting cloud cost or usage trends<\/strong>\n&#8211; <strong>Problem:<\/strong> Predict growth and plan budgets\/capacity.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> Billing export data lives in BigQuery; forecasting models can run directly on it.\n&#8211; <strong>Example:<\/strong> Monthly forecasts drive finance dashboards and anomaly detection.<\/p>\n\n\n\n<p>11) <strong>Campaign uplift proxy modeling<\/strong>\n&#8211; <strong>Problem:<\/strong> Estimate likely uplift or response based on historical campaign exposure.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> Data is already in BigQuery; you can quickly iterate features and compare model performance.\n&#8211; <strong>Example:<\/strong> Experiment analysis tables feed uplift proxy scores to marketers.<\/p>\n\n\n\n<p>12) <strong>Feature store\u2013like curated feature tables<\/strong>\n&#8211; <strong>Problem:<\/strong> Standardize feature computation and reuse across models\/teams.\n&#8211; <strong>Why BigQuery AI fits:<\/strong> Feature tables can be managed as BigQuery tables\/views with Dataform; models read from standardized features.\n&#8211; <strong>Example:<\/strong> A \u201cfeatures\u201d dataset becomes the governed contract for all ML training jobs.<\/p>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">6. Core Features<\/h2>\n\n\n\n<p>This section focuses on <strong>current, commonly documented BigQuery AI capabilities<\/strong>. Availability can vary by region and release stage\u2014verify in official docs for your environment.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.1 BigQuery ML (BQML): Train models with SQL<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> Create ML models using <code>CREATE MODEL ... AS SELECT ...<\/code> against BigQuery tables\/views.<\/li>\n<li><strong>Why it matters:<\/strong> Removes the need to export data to external notebooks for many standard ML tasks.<\/li>\n<li><strong>Practical benefit:<\/strong> Analysts and data engineers can build models as part of SQL pipelines with familiar tooling and governance.<\/li>\n<li><strong>Limitations\/caveats:<\/strong> Not all model types are supported; some advanced tuning\/training workflows require Vertex AI or custom code. Quotas apply.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">6.2 Inference in SQL: <code>ML.PREDICT<\/code> and batch scoring<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> Run predictions against new data and return scores\/labels.<\/li>\n<li><strong>Why it matters:<\/strong> \u201cDeployment\u201d can be as simple as writing predictions to a table via scheduled queries.<\/li>\n<li><strong>Practical benefit:<\/strong> Batch scoring at warehouse scale; easy integration with BI dashboards.<\/li>\n<li><strong>Limitations\/caveats:<\/strong> This is typically batch-oriented; not designed for ultra-low-latency online serving.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">6.3 Model evaluation and explainability (where supported)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> Evaluate metrics (accuracy, AUC, RMSE, etc.) and explain predictions for supported models (for example, <code>ML.EVALUATE<\/code>, <code>ML.CONFUSION_MATRIX<\/code>, and explainability functions where available).<\/li>\n<li><strong>Why it matters:<\/strong> Governance requires measurable performance and interpretability.<\/li>\n<li><strong>Practical benefit:<\/strong> Store evaluation artifacts in tables; automate model comparisons.<\/li>\n<li><strong>Limitations\/caveats:<\/strong> Explainability support depends on model type; verify current function coverage.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">6.4 Feature engineering in SQL (BigQuery-native)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> Use SQL transformations to build feature tables (aggregations, window functions, joins, text normalization, etc.).<\/li>\n<li><strong>Why it matters:<\/strong> Feature pipelines are often more work than the model; BigQuery excels at scalable feature computation.<\/li>\n<li><strong>Practical benefit:<\/strong> Centralize feature logic with Dataform, views, or scheduled jobs.<\/li>\n<li><strong>Limitations\/caveats:<\/strong> Ensure leakage control and train\/serve consistency; SQL makes it easy to accidentally include future information.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">6.5 Integration patterns with Vertex AI (remote models \/ external AI calls)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> For some BigQuery AI workflows, BigQuery can invoke externally hosted models (commonly through Vertex AI) using supported integration mechanisms.<\/li>\n<li><strong>Why it matters:<\/strong> Lets you use foundation models or custom Vertex AI models while keeping orchestration and data in BigQuery.<\/li>\n<li><strong>Practical benefit:<\/strong> Combine BigQuery governance + Vertex AI model hosting.<\/li>\n<li><strong>Limitations\/caveats:<\/strong> Regional alignment, IAM\/service accounts, API enablement, and additional costs (Vertex AI usage) apply. Feature names and availability can change\u2014verify in official docs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">6.6 Generative AI functions in BigQuery (availability-dependent)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> Where enabled, allows generating text or embeddings via SQL functions that call hosted models.<\/li>\n<li><strong>Why it matters:<\/strong> Enables summarization, classification, extraction, and semantic enrichment inside analytics workflows.<\/li>\n<li><strong>Practical benefit:<\/strong> Enrich rows with summaries\/tags; generate embeddings for similarity search analytics.<\/li>\n<li><strong>Limitations\/caveats:<\/strong> Often billed separately (model usage); subject to quotas; may require Vertex AI and specific regions. Verify the latest \u201cgenerative AI in BigQuery\u201d docs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">6.7 Vector storage and similarity search patterns (availability-dependent)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> Store embeddings in columns and run similarity queries (often with vector distance functions and optional indexing).<\/li>\n<li><strong>Why it matters:<\/strong> Supports semantic search, deduplication, clustering, and retrieval-augmented analytics patterns.<\/li>\n<li><strong>Practical benefit:<\/strong> Keep embeddings and business data together; query with SQL.<\/li>\n<li><strong>Limitations\/caveats:<\/strong> Indexing, function availability, and performance characteristics vary. Validate with your dataset size and region.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">6.8 BigQuery Studio \/ notebooks (workflow feature)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> Provides an integrated environment in the BigQuery UI for SQL development and (in some setups) notebook-style workflows.<\/li>\n<li><strong>Why it matters:<\/strong> Lowers friction for experimentation and collaboration.<\/li>\n<li><strong>Practical benefit:<\/strong> Single place for SQL exploration, model creation, and job tracking.<\/li>\n<li><strong>Limitations\/caveats:<\/strong> Notebook capabilities and integrations evolve; verify current BigQuery Studio features in docs.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">6.9 Governance, lineage, and policy controls (BigQuery platform features used by BigQuery AI)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>What it does:<\/strong> IAM, row\/column-level security, Data Catalog\/Dataplex governance, audit logs.<\/li>\n<li><strong>Why it matters:<\/strong> AI systems amplify data risk; governance needs to be \u201cbuilt-in,\u201d not bolted on.<\/li>\n<li><strong>Practical benefit:<\/strong> Controlled access to training data and predictions; auditable model execution.<\/li>\n<li><strong>Limitations\/caveats:<\/strong> Governance is only as good as your policy design; ensure separation of duties and least privilege.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">7. Architecture and How It Works<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">High-level service architecture<\/h3>\n\n\n\n<p>BigQuery AI workloads typically follow this pattern:\n1. <strong>Data lands in BigQuery<\/strong> (batch loads, streaming, or replication).\n2. <strong>Transformations produce curated tables<\/strong> and feature tables (SQL\/Dataform).\n3. <strong>Model training occurs in BigQuery ML<\/strong> (a BigQuery job that creates a model resource).\n4. <strong>Evaluation is stored<\/strong> (tables with metrics, confusion matrix, etc.).\n5. <strong>Batch inference runs on a schedule<\/strong> and writes results to tables\/views.\n6. <strong>Downstream consumers<\/strong> use predictions (Looker dashboards, exports, activation pipelines).\n7. <strong>Optional external calls<\/strong> (for example to Vertex AI) are invoked via supported integration mechanisms where required.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Request \/ data \/ control flow<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Control plane:<\/strong> You define datasets, models, and permissions; configure connections; set scheduled queries; define Dataform pipelines.<\/li>\n<li><strong>Data plane:<\/strong> BigQuery executes SQL jobs that scan data, compute features, train models, and produce outputs.<\/li>\n<li><strong>Observability plane:<\/strong> Audit logs (Admin Activity\/Data Access), job metadata (<code>INFORMATION_SCHEMA<\/code>), and monitoring dashboards support governance and troubleshooting.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Integrations with related services (common)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Ingestion:<\/strong> Pub\/Sub, Dataflow, Datastream, Storage Transfer Service, BigQuery Data Transfer Service<\/li>\n<li><strong>Transform\/pipelines:<\/strong> Dataform, Cloud Composer (Airflow), Dataflow, Dataproc<\/li>\n<li><strong>AI platform:<\/strong> Vertex AI (for custom training\/serving and foundation model access, where used)<\/li>\n<li><strong>BI:<\/strong> Looker, Looker Studio<\/li>\n<li><strong>Governance:<\/strong> Dataplex, Data Catalog, Cloud DLP, IAM<\/li>\n<li><strong>Security:<\/strong> Cloud KMS (CMEK), VPC Service Controls, Private Service Connect (verify applicability by feature)<\/li>\n<li><strong>Ops:<\/strong> Cloud Logging, Cloud Monitoring, Error Reporting (as applicable)<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Dependency services<\/h3>\n\n\n\n<p>At minimum, BigQuery AI depends on:\n&#8211; BigQuery API enabled\n&#8211; A billing-enabled project (for most real workloads)\nOptionally:\n&#8211; Vertex AI API enabled (for remote model\/generative AI workflows, if used)\n&#8211; Dataform\/Composer\/Dataflow depending on pipeline orchestration choices<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Security\/authentication model<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Authentication:<\/strong> Google Cloud IAM identities (users, groups, service accounts), often with Workload Identity Federation for external CI\/CD.<\/li>\n<li><strong>Authorization:<\/strong> BigQuery IAM roles at project\/dataset\/table\/model levels.<\/li>\n<li><strong>Separation of duties:<\/strong> Common split between platform admins (datasets, IAM, connections) and job runners (query execution).<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Networking model (practical)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>BigQuery is a managed service accessed via Google APIs.<\/li>\n<li>Data access can be controlled by IAM and perimeter controls (VPC Service Controls) for exfiltration protection\u2014verify support for any external model calls used by BigQuery AI.<\/li>\n<li>If you integrate with external services (like Vertex AI), ensure <strong>location alignment<\/strong> and <strong>perimeter policy alignment<\/strong>.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Monitoring\/logging\/governance considerations<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Use <strong>Cloud Logging<\/strong> for audit trails and BigQuery job logs.<\/li>\n<li>Use <code>INFORMATION_SCHEMA.JOBS*<\/code> views to analyze failures, slot usage, bytes processed, and query patterns.<\/li>\n<li>Tag and label datasets\/jobs where supported; enforce naming standards for models and output tables.<\/li>\n<li>Build a \u201cmodel ops\u201d dataset to store evaluation snapshots and drift indicators.<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Simple architecture diagram (conceptual)<\/h4>\n\n\n\n<pre><code class=\"language-mermaid\">flowchart LR\n  A[Sources: Apps \/ SaaS \/ Files] --&gt; B[Ingest: Dataflow \/ Transfers]\n  B --&gt; C[BigQuery Raw Dataset]\n  C --&gt; D[BigQuery Curated + Feature Tables]\n  D --&gt; E[BigQuery AI (BigQuery ML)]\n  E --&gt; F[Predictions Table]\n  F --&gt; G[BI: Looker \/ Dashboards]\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Production-style architecture diagram (with governance and orchestration)<\/h4>\n\n\n\n<pre><code class=\"language-mermaid\">flowchart TB\n  subgraph Ingestion\n    S1[Pub\/Sub Streams] --&gt; DF[Dataflow Streaming]\n    S2[Batch Files in Cloud Storage] --&gt; BQLOAD[BigQuery Load Jobs]\n    DS[Datastream \/ Replication] --&gt; BQLOAD\n  end\n\n  subgraph BigQuery_Lakehouse[BigQuery Datasets (Location-bound)]\n    RAW[Raw Tables]\n    CUR[Curated Tables]\n    FEAT[Feature Tables]\n    MOD[BigQuery ML Models]\n    PRED[Predictions &amp; Embeddings Tables]\n    METR[Model Metrics Tables]\n  end\n\n  DF --&gt; RAW\n  BQLOAD --&gt; RAW\n  RAW --&gt; CUR\n  CUR --&gt; FEAT\n\n  subgraph Orchestration\n    DFm[Dataform SQL Pipelines]\n    AIR[Cloud Composer \/ Airflow]\n    SCH[Scheduled Queries]\n  end\n\n  DFm --&gt; CUR\n  DFm --&gt; FEAT\n  AIR --&gt; SCH\n  SCH --&gt; MOD\n  SCH --&gt; PRED\n  MOD --&gt; PRED\n  MOD --&gt; METR\n\n  subgraph Governance_and_Security\n    IAM[IAM + Dataset Policies]\n    DPX[Dataplex \/ Data Catalog]\n    VSC[VPC Service Controls]\n    KMS[Cloud KMS (CMEK where applicable)]\n    AUD[Cloud Logging Audit Logs]\n  end\n\n  IAM --- BigQuery_Lakehouse\n  DPX --- BigQuery_Lakehouse\n  VSC --- BigQuery_Lakehouse\n  KMS --- BigQuery_Lakehouse\n  AUD --- BigQuery_Lakehouse\n\n  PRED --&gt; LKR[Looker \/ Looker Studio]\n  PRED --&gt; EXP[Exports to Apps (optional)]\n<\/code><\/pre>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">8. Prerequisites<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Account\/project requirements<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A <strong>Google Cloud project<\/strong> with <strong>billing enabled<\/strong> (recommended; some limited BigQuery usage may work in sandbox-like modes, but AI\/ML workflows typically require billing).<\/li>\n<li>BigQuery API enabled:<\/li>\n<li>https:\/\/console.cloud.google.com\/apis\/library\/bigquery.googleapis.com<\/li>\n<\/ul>\n\n\n\n<p>Optional (depending on features you use):\n&#8211; Vertex AI API enabled (for remote model\/generative AI integration):\n  &#8211; https:\/\/console.cloud.google.com\/apis\/library\/aiplatform.googleapis.com<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Permissions \/ IAM roles (minimum practical set)<\/h3>\n\n\n\n<p>Exact permissions vary by workflow. Common roles:\n&#8211; To run queries\/jobs:\n  &#8211; <code>roles\/bigquery.jobUser<\/code> (or equivalent permissions including <code>bigquery.jobs.create<\/code>)\n&#8211; To create datasets\/tables\/models in a dataset:\n  &#8211; <code>roles\/bigquery.dataEditor<\/code> on the dataset (or <code>roles\/bigquery.admin<\/code> for broader control)\n&#8211; To view data:\n  &#8211; <code>roles\/bigquery.dataViewer<\/code> on datasets\/tables\n&#8211; For managing connections (if using external\/remote integrations):\n  &#8211; <code>roles\/bigquery.connectionAdmin<\/code> (verify exact requirements in current docs)\n&#8211; For Vertex AI usage (if calling Vertex AI models):\n  &#8211; Typically <code>roles\/aiplatform.user<\/code> (or more specific roles depending on endpoints\/models). <strong>Verify in official docs<\/strong>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Tools<\/h3>\n\n\n\n<p>Choose one:\n&#8211; Google Cloud Console (BigQuery UI)\n&#8211; <code>bq<\/code> command-line tool (part of Google Cloud CLI): https:\/\/cloud.google.com\/sdk\/docs\/install\n&#8211; Optional: Python with <code>google-cloud-bigquery<\/code> client library for automation<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Region availability and location constraints<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Decide a BigQuery dataset location (e.g., <code>US<\/code> or <code>EU<\/code>) and keep training data, models, and prediction outputs in that location to avoid location mismatch errors.<\/li>\n<li>Some BigQuery AI capabilities (especially generative AI integrations) may be region-limited. <strong>Verify availability in official docs<\/strong>.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Quotas\/limits<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>BigQuery has quotas for query size, concurrent jobs, API requests, and ML model training. Quotas also exist for any external model calls.<\/li>\n<li>Always check:<\/li>\n<li>BigQuery quotas: https:\/\/cloud.google.com\/bigquery\/quotas<\/li>\n<li>Any feature-specific quotas (BQML\/generative AI) in the relevant docs<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Prerequisite services (optional but common)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Dataform (for SQL pipeline management): https:\/\/cloud.google.com\/dataform<\/li>\n<li>Cloud Composer (Airflow orchestration): https:\/\/cloud.google.com\/composer<\/li>\n<li>Dataplex (governance): https:\/\/cloud.google.com\/dataplex<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">9. Pricing \/ Cost<\/h2>\n\n\n\n<p>BigQuery AI cost is mainly the sum of:\n1. <strong>BigQuery storage<\/strong> (data stored in tables, plus any additional storage such as materialized views or model artifacts where applicable)\n2. <strong>BigQuery compute<\/strong> (queries, including model training and inference jobs)\n3. <strong>Optional external AI costs<\/strong> (for example, Vertex AI model inference\/training charges if BigQuery calls remote models)\n4. <strong>Data ingestion and pipeline costs<\/strong> (Dataflow, Pub\/Sub, Datastream, etc., if used)<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Pricing dimensions (BigQuery)<\/h3>\n\n\n\n<p>BigQuery pricing changes over time and varies by edition\/model and region. In general, you should expect these dimensions:\n&#8211; <strong>Query compute<\/strong>\n  &#8211; On-demand (charged by data processed) or capacity-based (slot reservations \/ editions)\n&#8211; <strong>Storage<\/strong>\n  &#8211; Active storage and long-term storage (prices differ; verify current policy)\n&#8211; <strong>Streaming inserts \/ ingestion<\/strong>\n  &#8211; Streaming has separate pricing considerations (verify current BigQuery streaming pricing)\n&#8211; <strong>ML workloads<\/strong>\n  &#8211; BigQuery ML training and inference generally consume BigQuery compute (queries\/slots)\n  &#8211; Some advanced integrations may introduce additional SKUs<\/p>\n\n\n\n<p>Official pricing:\n&#8211; BigQuery pricing page: https:\/\/cloud.google.com\/bigquery\/pricing\n&#8211; Google Cloud Pricing Calculator: https:\/\/cloud.google.com\/products\/calculator<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">BigQuery AI\u2013specific cost drivers<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Training frequency:<\/strong> retraining daily vs weekly can multiply compute cost.<\/li>\n<li><strong>Feature table complexity:<\/strong> heavy joins\/window functions can dominate spend (often more than the model training itself).<\/li>\n<li><strong>Batch scoring volume:<\/strong> scoring large tables on tight schedules can be expensive.<\/li>\n<li><strong>Generative AI \/ remote model calls:<\/strong> calling hosted models can add per-request or per-token costs (often billed through Vertex AI or related SKUs). <strong>Do not assume it\u2019s \u201cjust a query.\u201d<\/strong><\/li>\n<li><strong>Embedding storage growth:<\/strong> embeddings increase table size (vectors per row), affecting storage and query costs.<\/li>\n<li><strong>Cross-region data movement:<\/strong> avoid moving data across locations; it can add egress costs and complicate compliance.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Hidden or indirect costs<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Orchestration:<\/strong> Cloud Composer environments and Dataflow jobs have their own costs.<\/li>\n<li><strong>Logs:<\/strong> very high query volume can generate significant logs (usually modest, but not always).<\/li>\n<li><strong>Exports:<\/strong> exporting large prediction tables out of Google Cloud can incur network egress.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Cost optimization strategies (practical)<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Use <strong>partitioning and clustering<\/strong> on large training\/scoring tables.<\/li>\n<li>Materialize stable feature sets into <strong>feature tables<\/strong> to avoid recomputing expensive joins for every training run.<\/li>\n<li>Prefer <strong>incremental scoring<\/strong> (only new\/changed records) instead of rescoring everything.<\/li>\n<li>Use <strong>reservations\/editions<\/strong> if you have predictable heavy workloads; otherwise on-demand might be simpler. Evaluate with real usage.<\/li>\n<li>Set <strong>budgets and alerts<\/strong> (Cloud Billing).<\/li>\n<li>Use <strong>job labels<\/strong> and query audit analysis to attribute cost by team\/pipeline\/model.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Example low-cost starter estimate (model, not numbers)<\/h3>\n\n\n\n<p>A realistic \u201cstarter\u201d BigQuery AI lab cost profile:\n&#8211; Use <strong>public datasets<\/strong> (no storage cost in your project).\n&#8211; Create a small dataset and train a simple BQML model on a limited number of rows\/columns.\n&#8211; Run a few evaluation and prediction queries.\nCosts depend on:\n&#8211; bytes processed (on-demand) or slots consumed (capacity)\n&#8211; region\n&#8211; current BigQuery pricing model<br\/>\n<strong>Result:<\/strong> typically low, but you must verify in the pricing calculator for your region and expected query sizes.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Example production cost considerations (what to model)<\/h3>\n\n\n\n<p>For a production pipeline:\n&#8211; Daily ingestion: X GB\/day\n&#8211; Feature computation: multiple large joins over Y TB\n&#8211; Daily retraining: training query scans Z TB\n&#8211; Hourly scoring: rescoring N million rows\n&#8211; Optional: foundation model calls for summarization\/embedding at M requests\/day<br\/>\nYou should:\n&#8211; create a spreadsheet of query bytes processed (or slot-hours)\n&#8211; simulate partitions scanned\n&#8211; add storage growth (especially embeddings)\n&#8211; add external model usage (Vertex AI)<br\/>\nThen validate with:\n&#8211; BigQuery query plan and job statistics\n&#8211; Pricing calculator<\/p>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">10. Step-by-Step Hands-On Tutorial<\/h2>\n\n\n\n<p>This lab uses <strong>BigQuery ML (part of BigQuery AI)<\/strong> to train and evaluate a classifier using a <strong>public dataset<\/strong>, then run batch predictions and persist results.<\/p>\n\n\n\n<p>It is designed to be:\n&#8211; beginner-friendly\n&#8211; executable in under ~30\u201360 minutes\n&#8211; low-cost (public data, limited queries)<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Objective<\/h3>\n\n\n\n<p>Train a BigQuery ML classification model using SQL, evaluate it, generate predictions, and write the predictions to a BigQuery table suitable for dashboards and downstream pipelines.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Lab Overview<\/h3>\n\n\n\n<p>You will:\n1. Create a dataset for the lab.\n2. Create a training view from a public dataset.\n3. Train a logistic regression classifier with <code>CREATE MODEL<\/code>.\n4. Evaluate the model.\n5. Run predictions and store them in a table.\n6. Validate outputs, troubleshoot common issues, and clean up.<\/p>\n\n\n\n<blockquote>\n<p>Dataset choice: This lab uses the public penguins dataset (<code>bigquery-public-data.ml_datasets.penguins<\/code>). Public datasets can change; if this dataset is unavailable in your region\/org policy, substitute with another BigQuery public dataset (for example <code>bigquery-public-data.ml_datasets.iris<\/code>). Verify in BigQuery Public Datasets if needed.<\/p>\n<\/blockquote>\n\n\n\n<h3 class=\"wp-block-heading\">Step 1: Create a BigQuery dataset for the lab<\/h3>\n\n\n\n<p><strong>Console (recommended for beginners)<\/strong>\n1. Open BigQuery in the Google Cloud Console:\n   &#8211; https:\/\/console.cloud.google.com\/bigquery\n2. In the Explorer pane, select your project.\n3. Click <strong>More actions (\u22ee)<\/strong> \u2192 <strong>Create dataset<\/strong>.\n4. Set:\n   &#8211; Dataset ID: <code>bqai_lab<\/code>\n   &#8211; Data location: choose <code>US<\/code> (or <code>EU<\/code>, but be consistent throughout the lab)\n5. Click <strong>Create dataset<\/strong>.<\/p>\n\n\n\n<p><strong>Expected outcome:<\/strong> A dataset <code>bqai_lab<\/code> appears under your project in the Explorer.<\/p>\n\n\n\n<p><strong>Verification query:<\/strong>\nRun in BigQuery:<\/p>\n\n\n\n<pre><code class=\"language-sql\">SELECT \"dataset_ready\" AS status;\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Step 2: Create a clean training view from the public dataset<\/h3>\n\n\n\n<p>We\u2019ll create a view that:\n&#8211; selects numeric features\n&#8211; filters out rows with NULLs\n&#8211; keeps the label column (<code>species<\/code>)<\/p>\n\n\n\n<p>Run:<\/p>\n\n\n\n<pre><code class=\"language-sql\">CREATE OR REPLACE VIEW `bqai_lab.penguins_train_v` AS\nSELECT\n  species,\n  island,\n  sex,\n  culmen_length_mm,\n  culmen_depth_mm,\n  flipper_length_mm,\n  body_mass_g\nFROM `bigquery-public-data.ml_datasets.penguins`\nWHERE\n  species IS NOT NULL\n  AND island IS NOT NULL\n  AND sex IS NOT NULL\n  AND culmen_length_mm IS NOT NULL\n  AND culmen_depth_mm IS NOT NULL\n  AND flipper_length_mm IS NOT NULL\n  AND body_mass_g IS NOT NULL;\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> A view <code>bqai_lab.penguins_train_v<\/code> exists and returns rows without NULLs in key fields.<\/p>\n\n\n\n<p><strong>Verify row count:<\/strong><\/p>\n\n\n\n<pre><code class=\"language-sql\">SELECT COUNT(*) AS rows\nFROM `bqai_lab.penguins_train_v`;\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Step 3: Train a BigQuery ML classifier model (logistic regression)<\/h3>\n\n\n\n<p>Train a multi-class classifier predicting <code>species<\/code> from features.<\/p>\n\n\n\n<p>Run:<\/p>\n\n\n\n<pre><code class=\"language-sql\">CREATE OR REPLACE MODEL `bqai_lab.penguins_species_clf`\nOPTIONS(\n  model_type = 'logistic_reg',\n  input_label_cols = ['species'],\n  auto_class_weights = TRUE\n) AS\nSELECT\n  species,\n  island,\n  sex,\n  culmen_length_mm,\n  culmen_depth_mm,\n  flipper_length_mm,\n  body_mass_g\nFROM `bqai_lab.penguins_train_v`;\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> BigQuery creates a model resource <code>bqai_lab.penguins_species_clf<\/code>.<\/p>\n\n\n\n<p><strong>Verify the model exists:<\/strong>\n&#8211; In the Explorer, under <code>bqai_lab<\/code>, you should see <strong>Models<\/strong> \u2192 <code>penguins_species_clf<\/code>.<\/p>\n\n\n\n<p><strong>Optional verification query (model metadata):<\/strong><\/p>\n\n\n\n<pre><code class=\"language-sql\">SELECT *\nFROM ML.TRAINING_INFO(MODEL `bqai_lab.penguins_species_clf`);\n<\/code><\/pre>\n\n\n\n<blockquote>\n<p>If <code>ML.TRAINING_INFO<\/code> is not available for your model type or permissions, verify via the UI model details page instead. Function availability can vary\u2014verify in official docs.<\/p>\n<\/blockquote>\n\n\n\n<h3 class=\"wp-block-heading\">Step 4: Evaluate the model<\/h3>\n\n\n\n<p>Run:<\/p>\n\n\n\n<pre><code class=\"language-sql\">SELECT *\nFROM ML.EVALUATE(MODEL `bqai_lab.penguins_species_clf`,\n  (\n    SELECT\n      species,\n      island,\n      sex,\n      culmen_length_mm,\n      culmen_depth_mm,\n      flipper_length_mm,\n      body_mass_g\n    FROM `bqai_lab.penguins_train_v`\n  )\n);\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> You get evaluation metrics (for classification, often including log loss, accuracy, precision\/recall or similar metrics depending on current output).<\/p>\n\n\n\n<p><strong>Practical interpretation:<\/strong>\n&#8211; Use evaluation metrics as a baseline.\n&#8211; In production, evaluate on a proper holdout set and track metrics over time.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 5: Generate predictions and write them to a table<\/h3>\n\n\n\n<p>Create a small \u201cscoring\u201d dataset by taking a sample from the view (in real life, this would be \u201cnew\u201d data).<\/p>\n\n\n\n<p>Run:<\/p>\n\n\n\n<pre><code class=\"language-sql\">CREATE OR REPLACE TABLE `bqai_lab.penguins_scoring_input` AS\nSELECT *\nFROM `bqai_lab.penguins_train_v`\nORDER BY RAND()\nLIMIT 50;\n<\/code><\/pre>\n\n\n\n<p>Now run predictions and store results:<\/p>\n\n\n\n<pre><code class=\"language-sql\">CREATE OR REPLACE TABLE `bqai_lab.penguins_predictions` AS\nSELECT\n  *\nFROM ML.PREDICT(MODEL `bqai_lab.penguins_species_clf`,\n  (\n    SELECT\n      species,\n      island,\n      sex,\n      culmen_length_mm,\n      culmen_depth_mm,\n      flipper_length_mm,\n      body_mass_g\n    FROM `bqai_lab.penguins_scoring_input`\n  )\n);\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> A table <code>bqai_lab.penguins_predictions<\/code> exists containing prediction outputs (predicted label and probabilities\/scores, depending on current BigQuery ML output schema).<\/p>\n\n\n\n<p><strong>Verify prediction output:<\/strong><\/p>\n\n\n\n<pre><code class=\"language-sql\">SELECT *\nFROM `bqai_lab.penguins_predictions`\nLIMIT 10;\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Step 6 (Optional): Create a simple \u201cBI-ready\u201d view of predictions<\/h3>\n\n\n\n<p>Many teams create a view that flattens and standardizes prediction columns for Looker\/BI.<\/p>\n\n\n\n<p>You may need to adjust field names based on the output schema you see in Step 5.<\/p>\n\n\n\n<p>Example pattern (edit to match your columns):<\/p>\n\n\n\n<pre><code class=\"language-sql\">CREATE OR REPLACE VIEW `bqai_lab.penguins_predictions_bi_v` AS\nSELECT\n  species AS actual_species,\n  predicted_species,\n  island,\n  sex,\n  culmen_length_mm,\n  culmen_depth_mm,\n  flipper_length_mm,\n  body_mass_g\nFROM `bqai_lab.penguins_predictions`;\n<\/code><\/pre>\n\n\n\n<p><strong>Expected outcome:<\/strong> A view usable directly in BI tools.<\/p>\n\n\n\n<blockquote>\n<p>If the prediction output uses a nested field name (common in some BQML outputs), inspect the schema and rewrite the view accordingly.<\/p>\n<\/blockquote>\n\n\n\n<h3 class=\"wp-block-heading\">Validation<\/h3>\n\n\n\n<p>Run these checks:<\/p>\n\n\n\n<p>1) Confirm tables\/views exist:<\/p>\n\n\n\n<pre><code class=\"language-sql\">SELECT table_name, table_type\nFROM `bqai_lab.INFORMATION_SCHEMA.TABLES`\nORDER BY table_name;\n<\/code><\/pre>\n\n\n\n<p>2) Confirm model exists:<\/p>\n\n\n\n<pre><code class=\"language-sql\">SELECT model_name, model_type\nFROM `bqai_lab.INFORMATION_SCHEMA.MODELS`;\n<\/code><\/pre>\n\n\n\n<p>3) Quick sanity check: compare predicted vs actual counts<br\/>\n(Adjust column names to match your prediction output.)<\/p>\n\n\n\n<pre><code class=\"language-sql\">SELECT\n  actual_species,\n  predicted_species,\n  COUNT(*) AS n\nFROM `bqai_lab.penguins_predictions_bi_v`\nGROUP BY 1,2\nORDER BY n DESC;\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Troubleshooting<\/h3>\n\n\n\n<p>Common issues and fixes:<\/p>\n\n\n\n<p>1) <strong>Access Denied: <code>bigquery.jobs.create<\/code><\/strong>\n&#8211; <strong>Cause:<\/strong> Missing permission to run queries.\n&#8211; <strong>Fix:<\/strong> Grant <code>roles\/bigquery.jobUser<\/code> (or a higher role) to your user\/service account.<\/p>\n\n\n\n<p>2) <strong>Location mismatch \/ \u201cNot found in location\u201d<\/strong>\n&#8211; <strong>Cause:<\/strong> Your dataset is in <code>EU<\/code> but you\u2019re querying or creating resources assuming <code>US<\/code>, or using a public dataset with a different location behavior.\n&#8211; <strong>Fix:<\/strong> Keep all resources in the same location. Recreate <code>bqai_lab<\/code> in <code>US<\/code> (or consistent location). Ensure job location matches.<\/p>\n\n\n\n<p>3) <strong>Public dataset access blocked<\/strong>\n&#8211; <strong>Cause:<\/strong> Organization policy restrictions or restricted public dataset access.\n&#8211; <strong>Fix:<\/strong> Use a dataset your org allows, or load a small CSV into your project and repeat the workflow.<\/p>\n\n\n\n<p>4) <strong>Schema mismatch in prediction output<\/strong>\n&#8211; <strong>Cause:<\/strong> BQML output can contain nested fields depending on model type and settings.\n&#8211; <strong>Fix:<\/strong> Inspect the output schema in the BigQuery UI and adapt your BI view accordingly.<\/p>\n\n\n\n<p>5) <strong>Quota errors<\/strong>\n&#8211; <strong>Cause:<\/strong> Project-level quotas or concurrency limits reached.\n&#8211; <strong>Fix:<\/strong> Retry later, reduce data scanned, or request quota increases.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Cleanup<\/h3>\n\n\n\n<p>To avoid ongoing costs (storage), delete the lab dataset and everything inside it:<\/p>\n\n\n\n<p><strong>Console:<\/strong>\n&#8211; In BigQuery Explorer \u2192 <code>bqai_lab<\/code> \u2192 <strong>More actions (\u22ee)<\/strong> \u2192 <strong>Delete<\/strong> \u2192 confirm \u201cDelete contents\u201d.<\/p>\n\n\n\n<p><strong>Or run:<\/strong><\/p>\n\n\n\n<pre><code class=\"language-bash\">bq rm -r -f -d bqai_lab\n<\/code><\/pre>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">11. Best Practices<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Architecture best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Keep AI close to curated data:<\/strong> Train and score from curated\/feature tables, not raw ingestion tables.<\/li>\n<li><strong>Separate datasets by purpose:<\/strong> <code>raw<\/code>, <code>curated<\/code>, <code>features<\/code>, <code>models<\/code>, <code>predictions<\/code>, <code>metrics<\/code>.<\/li>\n<li><strong>Use Dataform for SQL pipelines:<\/strong> Version control feature logic and scoring logic; standardize environments.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">IAM\/security best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Least privilege:<\/strong> Separate job runners (<code>bigquery.jobUser<\/code>) from dataset owners\/admins.<\/li>\n<li><strong>Dataset-level boundaries:<\/strong> Put sensitive sources in separate datasets with tighter policies.<\/li>\n<li><strong>Authorized views:<\/strong> Let consumers query safe prediction outputs without accessing raw sensitive features.<\/li>\n<li><strong>Service accounts for automation:<\/strong> Use dedicated service accounts for scheduled scoring\/training.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Cost best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Partition and cluster<\/strong> feature and scoring tables on typical query filters (e.g., date, tenant, region).<\/li>\n<li><strong>Incremental scoring:<\/strong> Only score new records since last run.<\/li>\n<li><strong>Materialize expensive features:<\/strong> Avoid repeating large joins in every job.<\/li>\n<li><strong>Watch bytes scanned:<\/strong> Use query plans and job stats; optimize joins and filters.<\/li>\n<li><strong>Consider capacity<\/strong> if workloads are steady and heavy; keep on-demand for spiky\/adhoc (validate with your usage).<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Performance best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Avoid cross joins and unbounded window functions<\/strong> in feature pipelines.<\/li>\n<li><strong>Use approximate aggregates<\/strong> where appropriate for exploration.<\/li>\n<li><strong>Right-size training data:<\/strong> Start with representative samples, then scale.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Reliability best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Idempotent pipelines:<\/strong> Use <code>CREATE OR REPLACE<\/code> carefully; consider writing to temp tables then swapping.<\/li>\n<li><strong>Backfill strategy:<\/strong> Keep a mechanism for retraining\/scoring backfills without disrupting production tables.<\/li>\n<li><strong>Data validation gates:<\/strong> Validate row counts, null rates, and feature distributions before training.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Operations best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Centralize job metadata:<\/strong> Use <code>INFORMATION_SCHEMA.JOBS*<\/code> to build dashboards for failure rate, runtime, bytes processed.<\/li>\n<li><strong>Label jobs:<\/strong> Where supported, label by <code>pipeline<\/code>, <code>model<\/code>, <code>environment<\/code>, <code>owner<\/code>.<\/li>\n<li><strong>Alerting:<\/strong> Trigger alerts on job failures, anomalous bytes processed, or missing output partitions.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Governance\/tagging\/naming best practices<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Naming conventions:<\/li>\n<li>datasets: <code>raw_*<\/code>, <code>curated_*<\/code>, <code>feat_*<\/code>, <code>ml_*<\/code>, <code>ops_*<\/code><\/li>\n<li>models: <code>clf_*<\/code>, <code>reg_*<\/code>, <code>forecast_*<\/code><\/li>\n<li>outputs: <code>pred_*<\/code>, <code>metrics_*<\/code><\/li>\n<li>Document feature definitions and training windows in a data catalog (Dataplex\/Data Catalog).<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">12. Security Considerations<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Identity and access model<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>BigQuery uses <strong>IAM<\/strong> at project and dataset\/resource levels.<\/li>\n<li>For BigQuery AI:<\/li>\n<li>training\/inference requires query job permissions<\/li>\n<li>creating models requires write permissions in the dataset<\/li>\n<li>accessing predictions requires read permissions on output tables\/views<\/li>\n<li>If calling external models (Vertex AI), ensure:<\/li>\n<li>correct service identity permissions<\/li>\n<li>least privilege for the calling principal<\/li>\n<li>audited access paths<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Encryption<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Encryption at rest<\/strong> is enabled by default (Google-managed keys).<\/li>\n<li><strong>CMEK (Customer-Managed Encryption Keys)<\/strong> via Cloud KMS may be available for datasets and some resources\u2014verify the current BigQuery CMEK documentation and whether it covers your model artifacts and any external integrations:<\/li>\n<li>https:\/\/cloud.google.com\/bigquery\/docs\/customer-managed-encryption<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Network exposure and exfiltration controls<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>BigQuery is accessed via Google APIs; you control access primarily through IAM and organizational policies.<\/li>\n<li>Consider <strong>VPC Service Controls<\/strong> to reduce data exfiltration risk (especially in regulated environments). Verify whether any BigQuery AI external calls are compatible with your perimeter design:<\/li>\n<li>https:\/\/cloud.google.com\/vpc-service-controls<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Secrets handling<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Prefer <strong>service accounts<\/strong> + IAM over embedded credentials.<\/li>\n<li>If any pipeline needs secrets (for example, calling external APIs outside Google-managed integrations), store them in <strong>Secret Manager<\/strong>, not in SQL scripts or notebooks:<\/li>\n<li>https:\/\/cloud.google.com\/secret-manager<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Audit\/logging<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Enable and retain:<\/li>\n<li>BigQuery audit logs (Admin Activity and Data Access as appropriate)<\/li>\n<li>job history retention aligned to compliance needs<\/li>\n<li>Review who can:<\/li>\n<li>create models<\/li>\n<li>run training jobs on sensitive datasets<\/li>\n<li>export data\/predictions<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Compliance considerations<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Data residency: keep datasets and jobs in approved locations (<code>US<\/code>, <code>EU<\/code>, region).<\/li>\n<li>Apply DLP policies where needed (Cloud DLP + Dataplex governance).<\/li>\n<li>Ensure prediction outputs don\u2019t leak sensitive attributes (for example, proxies for protected classes).<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Common security mistakes<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Training models on PII without proper governance or minimization.<\/li>\n<li>Allowing broad dataset access so that model outputs expose sensitive features.<\/li>\n<li>Ignoring audit logs for model training\/scoring jobs.<\/li>\n<li>Mixing dev\/test and prod data in the same datasets with the same permissions.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Secure deployment recommendations<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Use separate projects or folders for dev\/test\/prod.<\/li>\n<li>Implement policy-as-code for IAM where possible.<\/li>\n<li>Use authorized views for serving predictions to broad audiences.<\/li>\n<li>Document model purpose, input features, and acceptable use.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">13. Limitations and Gotchas<\/h2>\n\n\n\n<p>BigQuery AI is production-capable for many workloads, but you should plan around these realities:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Feature availability varies:<\/strong> Some BigQuery AI features (especially generative AI integrations) can be region-limited or in preview. <strong>Verify in official docs<\/strong> for your region.<\/li>\n<li><strong>Location constraints are strict:<\/strong> Dataset location must match model location and job execution. Cross-location workflows are a common source of failures.<\/li>\n<li><strong>Not an online inference service:<\/strong> BigQuery batch scoring is not a replacement for low-latency online serving.<\/li>\n<li><strong>Quotas apply:<\/strong> BigQuery query quotas, concurrent job limits, and model-specific quotas can affect large-scale retraining\/scoring.<\/li>\n<li><strong>Cost surprises from feature queries:<\/strong> Feature engineering SQL can be far more expensive than model training.<\/li>\n<li><strong>Embedding storage bloat:<\/strong> Storing vectors for many rows increases storage and scan costs.<\/li>\n<li><strong>Governance gaps if not designed:<\/strong> Without careful dataset separation and authorized views, predictions can unintentionally expose sensitive inputs.<\/li>\n<li><strong>Schema drift:<\/strong> If upstream pipelines change feature columns, training\/scoring queries may fail or degrade silently.<\/li>\n<li><strong>Operational maturity required:<\/strong> Monitoring, job retries, and backfills must be engineered like any other production data pipeline.<\/li>\n<li><strong>Vertex AI integration complexity (if used):<\/strong> Requires additional IAM, API enablement, and cost tracking across services.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">14. Comparison with Alternatives<\/h2>\n\n\n\n<p>BigQuery AI sits in a spectrum: warehouse-native ML vs full ML platforms vs other cloud warehouses.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Comparison table<\/h3>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th>Option<\/th>\n<th>Best For<\/th>\n<th>Strengths<\/th>\n<th>Weaknesses<\/th>\n<th>When to Choose<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><strong>BigQuery AI (BigQuery ML + integrations)<\/strong><\/td>\n<td>SQL-first ML, batch scoring, analytics-native AI<\/td>\n<td>Minimal data movement; strong governance; easy operationalization into tables<\/td>\n<td>Not ideal for low-latency online serving; limited model types vs full platforms; some features region-limited<\/td>\n<td>Your data is in BigQuery and you want ML\/AI in pipelines and BI<\/td>\n<\/tr>\n<tr>\n<td><strong>Vertex AI (Google Cloud)<\/strong><\/td>\n<td>Full MLOps, custom training, online serving<\/td>\n<td>Custom models, GPUs\/TPUs, pipelines, model registry\/monitoring, endpoints<\/td>\n<td>More setup and operational complexity; feature pipelines often external<\/td>\n<td>You need custom training\/serving, real-time inference, full MLOps<\/td>\n<\/tr>\n<tr>\n<td><strong>Dataflow + Vertex AI<\/strong><\/td>\n<td>Streaming feature pipelines + ML<\/td>\n<td>Strong streaming; robust pipeline control<\/td>\n<td>Complexity; more moving parts<\/td>\n<td>You need event-driven ML features and near-real-time scoring<\/td>\n<\/tr>\n<tr>\n<td><strong>Dataproc (Spark) + ML libraries<\/strong><\/td>\n<td>Spark-native ML at scale<\/td>\n<td>Familiar Spark ecosystem; flexible<\/td>\n<td>Cluster management (even if managed); governance and cost control require discipline<\/td>\n<td>You already standardize on Spark for feature engineering<\/td>\n<\/tr>\n<tr>\n<td><strong>AWS Redshift ML<\/strong><\/td>\n<td>Warehouse-native ML on AWS<\/td>\n<td>Integrated into Redshift; simpler for AWS-native stacks<\/td>\n<td>Service-specific limitations; ecosystem differences<\/td>\n<td>You\u2019re AWS-first and data lives in Redshift<\/td>\n<\/tr>\n<tr>\n<td><strong>Azure Synapse + Azure ML<\/strong><\/td>\n<td>Analytics + ML on Azure<\/td>\n<td>Integrated Azure ecosystem<\/td>\n<td>Can require more plumbing; service boundaries<\/td>\n<td>You\u2019re Azure-first and need integrated ML<\/td>\n<\/tr>\n<tr>\n<td><strong>Snowflake (Snowpark \/ Cortex, etc.)<\/strong><\/td>\n<td>Warehouse-native analytics + AI<\/td>\n<td>Strong warehouse UX; ecosystem features<\/td>\n<td>Cost model and features differ; portability concerns<\/td>\n<td>You are Snowflake-first and want in-warehouse AI features<\/td>\n<\/tr>\n<tr>\n<td><strong>Databricks (Lakehouse AI)<\/strong><\/td>\n<td>Unified data engineering + ML + notebooks<\/td>\n<td>Strong DS workflows; MLflow\/MLOps; flexible compute<\/td>\n<td>Requires platform adoption; cost and governance differ<\/td>\n<td>Your org is notebook\/DS-heavy and wants unified lakehouse ML<\/td>\n<\/tr>\n<tr>\n<td><strong>Self-managed ML (Kubernetes + open-source)<\/strong><\/td>\n<td>Maximum control<\/td>\n<td>Full customization<\/td>\n<td>High ops burden; governance complexity<\/td>\n<td>You have strict constraints and strong platform engineering capacity<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">15. Real-World Example<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Enterprise example: Retail demand + promotion forecasting in BigQuery<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> A retailer needs weekly demand forecasts by SKU\/store and wants to measure promotion impact while keeping data governed in a central warehouse.<\/li>\n<li><strong>Proposed architecture:<\/strong><\/li>\n<li>Ingest POS and inventory data via Dataflow\/transfer \u2192 BigQuery raw<\/li>\n<li>Transform with Dataform into curated sales fact tables<\/li>\n<li>Build feature tables (seasonality, promos, holidays, price changes) in BigQuery<\/li>\n<li>Train forecasting models using BigQuery ML (and evaluate with stored metrics tables)<\/li>\n<li>Batch score weekly forecasts into <code>pred_demand_weekly<\/code><\/li>\n<li>Publish to Looker dashboards; export aggregates to supply chain systems<\/li>\n<li>Use Dataplex\/Data Catalog for governance and lineage; Cloud Logging for audit<\/li>\n<li><strong>Why BigQuery AI was chosen:<\/strong><\/li>\n<li>Data already centralized in BigQuery with strict access controls<\/li>\n<li>SQL-based feature pipelines integrated with existing analytics engineering workflow<\/li>\n<li>Batch forecast outputs naturally consumed by BI and planning systems<\/li>\n<li><strong>Expected outcomes:<\/strong><\/li>\n<li>Faster iteration cycles (feature changes in SQL)<\/li>\n<li>Reduced data movement risk<\/li>\n<li>Repeatable training\/scoring with job-level auditing and cost attribution<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Startup\/small-team example: SaaS churn risk scores for Customer Success<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Problem:<\/strong> A startup wants a churn risk score without hiring a full MLOps team.<\/li>\n<li><strong>Proposed architecture:<\/strong><\/li>\n<li>Product events \u2192 (optional) Pub\/Sub \u2192 Dataflow \u2192 BigQuery<\/li>\n<li>Daily transformation into account-level features (7-day active users, error rates, support ticket counts)<\/li>\n<li>Train a simple classifier in BigQuery ML weekly<\/li>\n<li>Score accounts daily into a <code>pred_churn_risk<\/code> table<\/li>\n<li>Build a Looker dashboard and a weekly CSV export for CS outreach<\/li>\n<li><strong>Why BigQuery AI was chosen:<\/strong><\/li>\n<li>Minimal infrastructure to manage<\/li>\n<li>SQL-only workflows match team skills<\/li>\n<li>Easy integration into dashboards and scheduled jobs<\/li>\n<li><strong>Expected outcomes:<\/strong><\/li>\n<li>Actionable risk ranking quickly<\/li>\n<li>Transparent and auditable pipeline<\/li>\n<li>Controlled costs by limiting scoring volume and using partitions<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">16. FAQ<\/h2>\n\n\n\n<p>1) <strong>Is BigQuery AI a separate Google Cloud product?<\/strong><br\/>\nBigQuery AI is best understood as an umbrella for AI\/ML capabilities in BigQuery (notably BigQuery ML and supported integrations). You typically enable and use it through BigQuery features and APIs rather than a standalone \u201cBigQuery AI\u201d service endpoint.<\/p>\n\n\n\n<p>2) <strong>What\u2019s the difference between BigQuery AI and BigQuery ML?<\/strong><br\/>\nBigQuery ML (BQML) is the core feature that lets you train and run ML models using SQL. BigQuery AI is a broader label that can include BQML plus other AI-related capabilities (for example, calling hosted models where supported). Verify the latest scope in official docs.<\/p>\n\n\n\n<p>3) <strong>Do I need a data science team to use BigQuery AI?<\/strong><br\/>\nNot necessarily for many baseline use cases. Analysts and data engineers can build useful models with SQL. For advanced modeling, experimentation rigor, and governance, data science involvement is still valuable.<\/p>\n\n\n\n<p>4) <strong>Can BigQuery AI do real-time predictions?<\/strong><br\/>\nBigQuery is optimized for analytics and batch workloads. For real-time, low-latency online inference, use Vertex AI endpoints or a dedicated serving layer. BigQuery can still be part of near-real-time pipelines by scoring in micro-batches.<\/p>\n\n\n\n<p>5) <strong>Where are models stored?<\/strong><br\/>\nBigQuery ML models are stored as BigQuery model resources inside datasets (and are subject to dataset location and IAM policies).<\/p>\n\n\n\n<p>6) <strong>Do I pay extra for BigQuery ML training?<\/strong><br\/>\nYou pay for the BigQuery compute used by training\/evaluation\/inference queries (on-demand bytes processed or capacity). If you call external models (for example via Vertex AI), that external usage is typically billed separately. Confirm with the official pricing pages.<\/p>\n\n\n\n<p>7) <strong>How do I schedule retraining and scoring?<\/strong><br\/>\nCommon approaches:\n&#8211; BigQuery Scheduled Queries\n&#8211; Dataform schedules\n&#8211; Cloud Composer (Airflow) for more complex dependencies and retries<\/p>\n\n\n\n<p>8) <strong>How do I version models?<\/strong><br\/>\nA practical approach is to create models with versioned names (for example, <code>model_churn_v2026_04_01<\/code>) and manage a \u201ccurrent model\u201d pointer via views or configuration tables. Some teams store metadata in a model registry table. Verify current BigQuery features for model management in docs.<\/p>\n\n\n\n<p>9) <strong>How do I prevent training on sensitive columns?<\/strong><br\/>\nUse dataset\/table policies, views that exclude sensitive columns, and code reviews for feature SQL. Consider DLP classification and policy tags (where applicable).<\/p>\n\n\n\n<p>10) <strong>Can I explain predictions?<\/strong><br\/>\nBigQuery ML supports explainability for some model types via specific functions\/features. Coverage depends on model type and current BigQuery ML capabilities\u2014verify in official docs.<\/p>\n\n\n\n<p>11) <strong>What\u2019s the best way to track model performance over time?<\/strong><br\/>\nWrite evaluation outputs to a <code>metrics<\/code> table on every retrain, including:\n&#8211; training window\n&#8211; features version\/hash\n&#8211; evaluation metrics\n&#8211; data volume<br\/>\nThen visualize trends and alert on regressions.<\/p>\n\n\n\n<p>12) <strong>How do I handle schema drift in features?<\/strong><br\/>\nUse Dataform (or CI checks) to enforce stable feature schemas. Treat feature tables as contracts. Validate columns and distributions before training.<\/p>\n\n\n\n<p>13) <strong>Can I build embeddings and do vector search in BigQuery?<\/strong><br\/>\nBigQuery supports vector-oriented patterns in some environments, but exact functions and indexing options can vary. Check the latest BigQuery vector search and embeddings documentation for your region and edition.<\/p>\n\n\n\n<p>14) <strong>How do I keep costs under control?<\/strong><br\/>\nPartition\/cluster, avoid rescoring full history, materialize expensive features, and monitor job bytes processed\/slot usage. Set budgets and analyze cost by labels and job metadata.<\/p>\n\n\n\n<p>15) <strong>How do I choose between BigQuery AI and Vertex AI?<\/strong><br\/>\nUse BigQuery AI for SQL-first, warehouse-native batch ML and enrichment. Use Vertex AI when you need custom training, online endpoints, GPUs\/TPUs, full MLOps, or complex pipelines.<\/p>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">17. Top Online Resources to Learn BigQuery AI<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th>Resource Type<\/th>\n<th>Name<\/th>\n<th>Why It Is Useful<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Official documentation<\/td>\n<td>BigQuery documentation<\/td>\n<td>Canonical reference for datasets, jobs, security, and operations: https:\/\/cloud.google.com\/bigquery\/docs<\/td>\n<\/tr>\n<tr>\n<td>Official documentation<\/td>\n<td>BigQuery ML overview<\/td>\n<td>Core ML-in-BigQuery docs and SQL patterns: https:\/\/cloud.google.com\/bigquery\/docs\/bigqueryml-intro<\/td>\n<\/tr>\n<tr>\n<td>Official documentation<\/td>\n<td>BigQuery quotas and limits<\/td>\n<td>Avoid surprises in production planning: https:\/\/cloud.google.com\/bigquery\/quotas<\/td>\n<\/tr>\n<tr>\n<td>Official pricing<\/td>\n<td>BigQuery pricing<\/td>\n<td>Current SKUs and pricing model: https:\/\/cloud.google.com\/bigquery\/pricing<\/td>\n<\/tr>\n<tr>\n<td>Pricing tool<\/td>\n<td>Google Cloud Pricing Calculator<\/td>\n<td>Model costs for storage\/compute\/related services: https:\/\/cloud.google.com\/products\/calculator<\/td>\n<\/tr>\n<tr>\n<td>Architecture center<\/td>\n<td>Google Cloud Architecture Center<\/td>\n<td>Reference architectures for data analytics and pipelines: https:\/\/cloud.google.com\/architecture<\/td>\n<\/tr>\n<tr>\n<td>Product documentation<\/td>\n<td>Dataform documentation<\/td>\n<td>SQL pipeline management integrated with BigQuery: https:\/\/cloud.google.com\/dataform\/docs<\/td>\n<\/tr>\n<tr>\n<td>Product documentation<\/td>\n<td>Vertex AI documentation<\/td>\n<td>For hybrid BigQuery + Vertex AI patterns: https:\/\/cloud.google.com\/vertex-ai\/docs<\/td>\n<\/tr>\n<tr>\n<td>Labs\/tutorials<\/td>\n<td>Google Cloud Skills Boost (BigQuery \/ BigQuery ML labs)<\/td>\n<td>Hands-on labs maintained by Google (search BigQuery ML): https:\/\/www.cloudskillsboost.google\/<\/td>\n<\/tr>\n<tr>\n<td>Videos<\/td>\n<td>Google Cloud Tech YouTube channel<\/td>\n<td>Practical walkthroughs and updates (search BigQuery ML \/ BigQuery): https:\/\/www.youtube.com\/@googlecloudtech<\/td>\n<\/tr>\n<tr>\n<td>Samples<\/td>\n<td>GoogleCloudPlatform GitHub org<\/td>\n<td>Official samples across BigQuery and data tooling: https:\/\/github.com\/GoogleCloudPlatform<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">18. Training and Certification Providers<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th>Institute<\/th>\n<th>Suitable Audience<\/th>\n<th>Likely Learning Focus<\/th>\n<th>Mode<\/th>\n<th>Website URL<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>DevOpsSchool.com<\/td>\n<td>Engineers, DevOps\/SRE, platform and data teams<\/td>\n<td>Cloud\/DevOps\/data platform training programs; verify BigQuery AI coverage on site<\/td>\n<td>Check website<\/td>\n<td>https:\/\/www.devopsschool.com\/<\/td>\n<\/tr>\n<tr>\n<td>ScmGalaxy.com<\/td>\n<td>Students and professionals<\/td>\n<td>Software lifecycle, DevOps, tooling, cloud fundamentals; verify BigQuery content<\/td>\n<td>Check website<\/td>\n<td>https:\/\/www.scmgalaxy.com\/<\/td>\n<\/tr>\n<tr>\n<td>CLoudOpsNow.in<\/td>\n<td>Cloud operations practitioners<\/td>\n<td>Cloud operations and platform practices; verify data\/analytics offerings<\/td>\n<td>Check website<\/td>\n<td>https:\/\/www.cloudopsnow.in\/<\/td>\n<\/tr>\n<tr>\n<td>SreSchool.com<\/td>\n<td>SREs, operations, reliability engineers<\/td>\n<td>Reliability engineering practices; monitoring\/operations that apply to data platforms<\/td>\n<td>Check website<\/td>\n<td>https:\/\/www.sreschool.com\/<\/td>\n<\/tr>\n<tr>\n<td>AiOpsSchool.com<\/td>\n<td>Ops teams adopting AI for operations<\/td>\n<td>AIOps concepts and tooling; verify relevance to Google Cloud data\/AI operations<\/td>\n<td>Check website<\/td>\n<td>https:\/\/www.aiopsschool.com\/<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">19. Top Trainers<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th>Platform\/Site<\/th>\n<th>Likely Specialization<\/th>\n<th>Suitable Audience<\/th>\n<th>Website URL<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>RajeshKumar.xyz<\/td>\n<td>Cloud\/DevOps training content (verify current offerings)<\/td>\n<td>Beginners to intermediate practitioners<\/td>\n<td>https:\/\/www.rajeshkumar.xyz\/<\/td>\n<\/tr>\n<tr>\n<td>devopstrainer.in<\/td>\n<td>DevOps and cloud training (verify course catalog)<\/td>\n<td>Engineers and ops teams<\/td>\n<td>https:\/\/www.devopstrainer.in\/<\/td>\n<\/tr>\n<tr>\n<td>devopsfreelancer.com<\/td>\n<td>Freelance DevOps services\/training platform (verify offerings)<\/td>\n<td>Teams seeking external help or coaching<\/td>\n<td>https:\/\/www.devopsfreelancer.com\/<\/td>\n<\/tr>\n<tr>\n<td>devopssupport.in<\/td>\n<td>DevOps support and training resources (verify offerings)<\/td>\n<td>Practitioners needing guided support<\/td>\n<td>https:\/\/www.devopssupport.in\/<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">20. Top Consulting Companies<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table>\n<thead>\n<tr>\n<th>Company<\/th>\n<th>Likely Service Area<\/th>\n<th>Where They May Help<\/th>\n<th>Consulting Use Case Examples<\/th>\n<th>Website URL<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>cotocus.com<\/td>\n<td>Cloud\/DevOps\/data engineering consulting (verify offerings)<\/td>\n<td>Architecture, delivery support, platform improvements<\/td>\n<td>BigQuery cost optimization review; IAM\/governance hardening; pipeline reliability assessments<\/td>\n<td>https:\/\/cotocus.com\/<\/td>\n<\/tr>\n<tr>\n<td>DevOpsSchool.com<\/td>\n<td>Training + consulting (verify services)<\/td>\n<td>Enablement, platform best practices, implementation support<\/td>\n<td>BigQuery operations runbooks; CI\/CD for Dataform; monitoring and job governance<\/td>\n<td>https:\/\/www.devopsschool.com\/<\/td>\n<\/tr>\n<tr>\n<td>DEVOPSCONSULTING.IN<\/td>\n<td>DevOps\/cloud consulting (verify offerings)<\/td>\n<td>Delivery execution, automation, operational maturity<\/td>\n<td>BigQuery pipeline automation; observability setup; migration planning<\/td>\n<td>https:\/\/www.devopsconsulting.in\/<\/td>\n<\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">21. Career and Learning Roadmap<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">What to learn before BigQuery AI<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>BigQuery fundamentals<\/strong>\n   &#8211; Datasets, tables, views, partitions\/clusters\n   &#8211; Query execution, job history, <code>INFORMATION_SCHEMA<\/code><\/li>\n<li><strong>SQL proficiency<\/strong>\n   &#8211; Joins, window functions, CTEs, query optimization<\/li>\n<li><strong>Data modeling<\/strong>\n   &#8211; Star schemas, event modeling, slowly changing dimensions<\/li>\n<li><strong>Google Cloud IAM basics<\/strong>\n   &#8211; Project vs dataset permissions, service accounts<\/li>\n<li><strong>Data pipeline basics<\/strong>\n   &#8211; ELT patterns, Dataform or scheduled queries, testing<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">What to learn after BigQuery AI<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Vertex AI<\/strong>\n   &#8211; Model training, endpoints, pipelines, evaluation\/monitoring<\/li>\n<li><strong>MLOps practices<\/strong>\n   &#8211; Versioning, reproducibility, CI\/CD, model governance<\/li>\n<li><strong>Advanced governance<\/strong>\n   &#8211; Dataplex, DLP, policy tags, VPC Service Controls<\/li>\n<li><strong>Streaming architectures<\/strong>\n   &#8211; Pub\/Sub + Dataflow for event-time processing<\/li>\n<li><strong>BI integration<\/strong>\n   &#8211; Looker modeling, semantic layers, metric definitions<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Job roles that use it<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Data Engineer \/ Analytics Engineer<\/li>\n<li>Data Scientist (especially for rapid prototyping with warehouse-native ML)<\/li>\n<li>Cloud\/Data Solutions Architect<\/li>\n<li>Platform Engineer for data platforms<\/li>\n<li>SRE\/Operations Engineer supporting data workloads<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Certification path (Google Cloud)<\/h3>\n\n\n\n<p>Google Cloud certifications change; common relevant certifications include:\n&#8211; Professional Data Engineer\n&#8211; Professional Cloud Architect<br\/>\nVerify current certification names and outlines:\n&#8211; https:\/\/cloud.google.com\/learn\/certification<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Project ideas for practice<\/h3>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Build a churn model with BQML and a scheduled scoring pipeline.<\/li>\n<li>Create a forecasting pipeline for demand or cost using partitioned time series.<\/li>\n<li>Implement a governed \u201cpredictions\u201d dataset with authorized views and audit dashboards.<\/li>\n<li>Build embedding + similarity analytics for product descriptions (verify feature availability).<\/li>\n<li>Create a cost attribution dashboard using BigQuery job metadata and labels.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">22. Glossary<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>BigQuery AI:<\/strong> Umbrella term for AI\/ML capabilities in BigQuery, including BigQuery ML and supported AI integrations.<\/li>\n<li><strong>BigQuery ML (BQML):<\/strong> BigQuery feature that lets you train and use ML models using SQL.<\/li>\n<li><strong>Dataset location:<\/strong> Geographic location where BigQuery data and processing occur (e.g., US, EU). Must align across tables\/models\/jobs.<\/li>\n<li><strong>Feature engineering:<\/strong> Transforming raw data into model-ready inputs (features).<\/li>\n<li><strong>Batch scoring:<\/strong> Running inference over a set of rows and writing predictions to a table.<\/li>\n<li><strong><code>CREATE MODEL<\/code>:<\/strong> SQL statement to create and train a BigQuery ML model.<\/li>\n<li><strong><code>ML.PREDICT<\/code>:<\/strong> Function to generate predictions using a trained model.<\/li>\n<li><strong><code>ML.EVALUATE<\/code>:<\/strong> Function to compute evaluation metrics for a model.<\/li>\n<li><strong>IAM:<\/strong> Identity and Access Management; controls who can access and change resources.<\/li>\n<li><strong>Authorized view:<\/strong> A view that allows controlled access to underlying tables without granting direct table permissions.<\/li>\n<li><strong>CMEK:<\/strong> Customer-Managed Encryption Keys, typically via Cloud KMS.<\/li>\n<li><strong>VPC Service Controls:<\/strong> A Google Cloud feature to reduce risk of data exfiltration by defining service perimeters.<\/li>\n<li><strong>Dataform:<\/strong> Google Cloud service for managing SQL transformations and pipelines with version control.<\/li>\n<li><strong>Vertex AI:<\/strong> Google Cloud\u2019s managed ML platform for custom training, deployment, and MLOps.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">23. Summary<\/h2>\n\n\n\n<p>BigQuery AI in Google Cloud brings AI\/ML capabilities into the center of <strong>Data analytics and pipelines<\/strong> by letting you train models, evaluate them, and run inference directly in BigQuery\u2014often using SQL and existing warehouse governance.<\/p>\n\n\n\n<p>It matters because it reduces data movement, shortens iteration cycles, and makes many predictive\/enrichment workflows operationally simple: write outputs to tables, schedule jobs, and connect BI tools. Cost and security require deliberate design: manage bytes processed\/slot usage, avoid expensive feature recomputation, control access to sensitive training data, and audit model jobs and prediction outputs.<\/p>\n\n\n\n<p>Use BigQuery AI when your data is in BigQuery and your use case fits warehouse-native, batch-oriented ML and enrichment. For custom deep learning, real-time online serving, or full MLOps, pair BigQuery with Vertex AI.<\/p>\n\n\n\n<p>Next step: read the BigQuery ML documentation and reproduce the lab with your own curated dataset, then productionize it with Dataform, scheduled scoring, and a metrics table for ongoing model monitoring.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Data analytics and pipelines<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[59,51],"tags":[],"class_list":["post-645","post","type-post","status-publish","format-standard","hentry","category-data-analytics-and-pipelines","category-google-cloud"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/posts\/645","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/comments?post=645"}],"version-history":[{"count":0,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/posts\/645\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/media?parent=645"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/categories?post=645"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/tutorials\/wp-json\/wp\/v2\/tags?post=645"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}