Deployed link: Google Cloud Generative AI Leader Certification Notes
Understanding the structure of Generative AI is key to effective development and deployment. This system is typically broken down into five interconnected layers:
-
1. Infrastructure: 💻
- This is the foundation of generative AI.
- It includes the physical hardware (servers, GPUs, TPUs) and software required to store and run AI models and training data.
-
2. Models: 💡
- Often called the "brain" of an AI agent.
- These are complex algorithms trained on huge amounts of data to learn patterns.
- They enable the AI to generate new content, translate languages, and answer questions.
-
3. Platform: 🔧
- This layer sits between the agents and the models.
- It provides the tools and infrastructure for AI development, such as APIs, data management, and model deployment utilities.
- Its purpose is to simplify the complexities of the underlying infrastructure.
-
4. Agents: 🤖
- These are pieces of software that use models and tools to achieve goals autonomously.
- They can analyze situations, use multiple tools, and handle multi-step tasks without constant human input.
-
5. Gen AI-Powered Applications: 📱
- This is the user-facing layer (the frontend).
- It's what allows users to interact with and leverage the capabilities of generative AI.
- Examples include apps like Gemini and NotebookLM.
- Vital for Leadership: 🧑💼 Understanding these layers is crucial for leaders to guide their teams in successfully adopting and using generative AI.
- Effective Navigation: 🗺️ By understanding this interconnected system, leaders and developers can better navigate the landscape and harness the power of AI for their projects.
| Feature | Retrieval-Augmented Generation (RAG) | Fine-tuning | Grounding |
|---|---|---|---|
| Definition | Augments LLMs by retrieving relevant information from external knowledge bases and adding it to the prompt. | Further trains a pre-trained model on a new dataset to adapt it to a specific task or domain. | Connects an AI model's output to verifiable sources of information. |
| Process | Retrieve relevant information. |
Select a pre-trained model. |
Provide access to data sources. |
| Data sources | External knowledge bases (databases, documents, internet). | Task- or domain-specific datasets. | External knowledge bases or specific datasets. |
| Relationship to Grounding | A specific technique for achieving grounding. | Improves a model's ability to be grounded in specific domains. | The overarching goal, achieved through techniques like RAG and fine-tuning. |
AI agents represent a significant leap beyond standalone models. They are designed to observe, act, and achieve goals by combining two core elements: a reasoning loop and tools.
-
1. Reasoning Loop: 🧠
- This is the agent's "thinking process"—a continuous, iterative cycle that allows agents to analyze a situation, plan actions, and adapt based on outcomes.
- The Four Key Steps of the Loop:
- Observes: 👀 Gathers information from its environment.
- Interprets: 🧠 Processes the information to assess the current situation.
- Plans: 🗺️ Devises a course of action to achieve its goal.
- Acts: ✅ Executes the planned action.
- The cycle continues until the goal is met, ranging from simple rules to complex thought chains.
-
2. Tools: 🔧
- These are functionalities that allow the agent to interact with its environment.
- Tools enable agents to access external data, use software applications, and even communicate with physical robots.
- This capability connects agents with real-world information and services, much like apps on a smartphone.
- Solving Complex Problems: 💡 Agents can analyze information, gather necessary data using tools, and make informed decisions with minimal human intervention.
- Managing Multi-Step Tasks: ➡️ They excel at complex workflows, such as conducting in-depth research, troubleshooting code, and automating tasks across multiple systems.
- Producing Diverse Results: 🌍 Agents move beyond simple text generation to produce results that involve actions in the real world.
- Manual & Tedious Process: 😫 Engineers struggled to get critical information (like material composition and dimensions) from complex 3D CAD models.
- Inefficient Workflow: 📉 This slow, manual process was prone to errors, which severely impacted their ability to efficiently design, analyze, and manufacture jet engine components.
Cymbal Aerospace built a custom generative AI solution on Vertex AI using a two-layered approach:
-
Layer 1: AutoML (The Intelligent Interpreter) 🧠
- They used AutoML to streamline the training process for a machine-learning model that could extract key metadata from the CAD files.
- This meant they did not need deep machine-learning expertise to build a powerful, customized model.
-
Layer 2: Gemini (The Natural Language Interface) 🗣️
- They integrated Gemini to transform the raw metadata from the AutoML model into clear, concise summaries in natural language.
- This allowed engineers to ask simple questions like, "What is the tensile strength of the turbine blade?" and get an immediate, accurate answer.
- The integration was streamlined using MLOps tools like Vertex AI Pipelines.
- Simplified Model Development: 🧑💻 Vertex AI's AutoML handles the complex parts of model training, making it easy to build a high-performing model.
- Customized Solutions: 🎯 The company could train a model that was perfectly tailored to their specific data and needs.
- Efficient Scaling: 🚀 Vertex AI provided the robust infrastructure to deploy and scale the model, allowing them to process huge numbers of CAD files quickly.
- Streamlined Deployment: They could easily deploy their models using a serverless infrastructure, eliminating the need to worry about managing the hardware.
AI Agents are generally categorized by their primary mode of operation. This table highlights the key differences between Conversational Agents and Workflow Agents.
| Feature | 🗣️ Conversational Agents | ⚙️ Workflow Agents |
|---|---|---|
| Primary Goal | To understand and respond to user queries in a natural, human-like way. | To automate and streamline tasks or complex, predefined processes. |
| Main Function | Understanding the meaning and intent behind what the user says. | Executing a series of pre-defined, sequential steps to complete a task. |
| Input Type | Typed or spoken messages from a user (interactive). | A defined task, a system trigger (e.g., submitting a form), or an online order (event-driven). |
| Tool Usage | Calls tools dynamically to gather information or perform actions related to a user's request (e.g., searching the web). | Executes a sequence of actions based on a defined workflow, like transferring data or sending notifications. |
| Output Type | A relevant and natural-sounding response to the user. | A result or output from the completed task, such as a report, a confirmation message, or a data file. |
| Example | A chatbot that answers your questions about a product or service in a dialogue. | An automated system that processes an order, updates inventory, and sends a confirmation email. |
The custom generative AI solution delivered significant, measurable improvements:
- Product Development Accelerated: ⏱️ Development cycles were accelerated by 15%.
- Design Review Time Reduced: Design review times were reduced by 20%.
- Bottlenecks Eliminated: Manufacturing bottlenecks were eliminated, improving overall flow.
- Competitive Advantage: 🏆 This faster time-to-market helped the company respond quickly to customer demands and release innovative products ahead of their competition.
HPC is the powerful backbone required for training and running complex generative AI models.
- GPUs (Graphics Processing Units): 🎮 Originally for graphics, they are essential for AI due to their excellence in parallel processing (doing many things at once).
- TPUs (Tensor Processing Units): 💡 These are Google's custom-designed chips specifically optimized for AI tasks.
- Hypercomputer: 🚀 A supercomputer made by connecting many individual computers (nodes) containing these GPUs and TPUs with high-speed networks, creating one massive computing unit.
- Massive Datasets: Generative AI models are data-hungry, requiring petabytes of data to analyze. 📚
- Speed & Reliability: High-performance storage provides the necessary speed, capacity, and reliability to access these enormous datasets efficiently.
- Optimized Infrastructure: Google Cloud's storage is optimized for AI workloads, ensuring fast access to prevent slow read/write speeds from hindering the training process. ⏱️
- Essential Communication: Fast and efficient communication is essential for coordinating all the processors in an HPC cluster. 🔗
- Global Fiber Network: Google's global network provides high-bandwidth, low-latency connectivity, ensuring smooth data flow throughout the infrastructure. 🛰️
The move to run AI models locally on a device offers key advantages:
- Key Reasons for Local Processing:
- Real-time Responsiveness ⚡
- Increased Data Privacy 🔒
- Ability to Operate Without an Internet Connection 📶
- Example: A drone needs to react instantly to obstacles, making local processing essential for safety. 🚁
- Lite Runtime (LiteRT): ⚙️ Google's platform that helps ML models run efficiently on devices like smartphones.
- It optimizes models by addressing constraints like latency, privacy, and power consumption.
- It supports a wide range of models from frameworks like TensorFlow, PyTorch, and JAX.
- Gemini Nano: 💡 Google's most efficient and compact AI model, specifically designed to run on-device.
- Benefits of Gemini Nano:
- Privacy: Your data stays on your device. 🔒
- Speed: You get fast responses without cloud latency. ⚡
- Offline Access: It can work even without an internet connection. 📶
- Benefits of Gemini Nano:
You can still use Vertex AI to manage the lifecycle of models deployed on the edge:
- Convert Models: Convert models to Lite Runtime (LiteRT) for optimal performance on edge devices. ⚙️
- Package and Deploy: Package and deploy your models and their dependencies into containers for various edge hardware. 📦
- Manage and Monitor: Manage and monitor your edge deployments to track performance and gather insights over time. 📊
It's crucial to have the right people and expertise for your AI project. The AI stack supports a variety of roles:
- Business Leaders: 📊 Use pre-built generative AI solutions (like Gemini for Google Workspace) to improve daily operations.
- Developers: 💻 Build and deploy custom AI agents and integrate AI capabilities into existing apps, using platforms like Vertex AI for orchestration and grounding.
- AI Practitioners: 🧠 Responsible for customizing, deploying, and optimizing generative AI models, including scaling workloads and integrating with other services (like BigQuery).
The cost depends on the product, company, and specific use case, making a realistic budget vital. 💸
- You Pay for Three Main Activities:
- Training the model.
- Deploying the model to an endpoint.
- Using the model for predictions.
- Pricing Models:
- Usage-based: Pay for the amount you use (common for APIs). 📏
- Free tiers: Limited free access for experimentation. 🆓
- Licensing fees: One-time or recurring fees for commercial use. 🏷️
- Subscription-based: Recurring fee for access, often with tiered usage limits.
- Common Usage Metrics:
- Tokens: 🧩 A piece of text (word or part of a word) processed.
- Compute time: ⏱️ The time taken to process your requests.
- Requests: Being charged per request, regardless of complexity.
- Characters: ✍️ The total number of characters processed.
- Model Size and Complexity: Larger and more capable models generally cost more.
- Context Window: A larger context window (the amount of text the model considers) can increase costs.
- Specialized Features: Features like fine-tuning or embeddings may have separate pricing. 🛠️
- Deployment: Where you deploy your model can affect costs based on compute time. ☁️
Choosing the right generative AI solution requires careful consideration of various factors to ensure the most cost-effective solution. 🧐💰
- Evaluate Model Capabilities: 📊 Look at independent benchmarks and research papers to check the quality and performance of different models.
- Compare Pricing: 💸 Understand the pricing models (like usage-based) and metrics (like tokens or characters) of each provider to accurately estimate your long-term costs.
- Factor in Extra Costs: 📝 Always account for potential additional expenses such as data storage, API calls, and the cost associated with fine-tuning a model.
- Read the Fine Print: 📜 Pay close attention to critical details like usage limits, data privacy policies, and other terms of service.
- Provider Websites: 🌐 The most crucial first step is checking the pricing pages on different company websites.
- Research Papers and Benchmarks: 🔬 Seek out independent evaluations and comparisons of different models for objective data.
- Community Forums: 🗣️ Engage with other users and experts to get valuable insights and real-world experiences regarding models and pricing.
- Long-Term Journey: Building a generative AI solution is a long-term commitment, not a one-time project; planning for ongoing maintenance is essential. 🗺️
- Fully Managed Environment: ⚙️ Google Cloud offers a fully managed environment that lets developers focus on building and improving the AI agent, eliminating worries about managing the underlying infrastructure and maintenance.
- Maximizing Value: ✨ This approach helps you get the most value from your investment and avoid costly issues over time.
Grounding is a critical concept for building trustworthy AI systems. ✅
- Definition: Grounding is the ability of an AI model to connect its output to verifiable and specific sources of information. 🧐
- No-Code Grounding: You can achieve grounding without any coding by simply providing the AI with resources like your company's style guide within the prompt or by uploading relevant files. 📁
RAG is a powerful two-step technique for achieving strong grounding:
- Retrieving Relevant Information: 🔎 The AI model first retrieves relevant information from a vast knowledge base (like a database or documents), often using advanced semantic search techniques.
- Generating Output: ✍️ The model then uses this retrieved, factual information to generate the final response.
| Approach | Model Only (without RAG) 🚫 | Model with RAG 💡 |
|---|---|---|
| Knowledge Source | Model provides output based only on its internal knowledge. | Model first queries a backend system to find the most relevant external information. |
| Output Risk | Output might be outdated and is more likely to have hallucinations (false or misleading information). 👻 | Output is based on the latest external knowledge, making it much less likely to have hallucinations. ✨ |
- Improved Accuracy and Relevance: 🎯 By incorporating external knowledge, RAG models produce outputs that are more accurate and informative.
- Improved Explainability: 📜 RAG increases trust by showing the specific sources it used to generate the output, allowing for claim verification.
- Extended LLM Capabilities: 🔒 RAG can constrain the output from an LLM to generate a response only based on the specific context you provided.
- Easy Start: You can start using RAG without any coding or database development with tools like NotebookLM. 💻
Gemini for Google Cloud is a set of AI-powered collaborators embedded across many Google Cloud products. 🤝🔒
- Enterprise Protection: It comes with standard Google Cloud enterprise protections, and your prompts or responses are never used to train models.
| Service | Primary Function | Key Benefit |
|---|---|---|
| Gemini Cloud Assist | Acts as an AI expert for your cloud environment. 💻 | Analyzes resources and assists with troubleshooting, cost optimization, and security best practices. 🛡️ |
| Gemini in BigQuery | Makes data analysis easier and more accessible. 📊 | Helps you write code, understand data, and generate insights automatically, regardless of SQL experience. ⏱️ |
| Gemini Code Assist | An AI pair programmer for developers. 🧑💻 | Provides code suggestions, generates entire code blocks, and offers explanations for 20+ popular languages. |
| Gemini in Colab Enterprise | A feature for interactive notebooks. 📝 | Helps you write Python code by suggesting segments and generating code from your descriptions, streamlining ML workflows. |
| Gemini in Databases | Helps developers and DBAs manage databases. 🗃️ | Simplifies tasks, from building applications with natural language to managing database fleets. |
| Gemini in Looker | An intelligent assistant for data analysis and insights. 📈 | Helps you understand your data, create visualizations, and generate reports intuitively. |
| Gemini in Security | Helps security teams detect, contain, and stop threats. 🚨 | Provides near-instant analysis of security findings and potential attack paths, summarizing threat actor tactics. |
Agent tools are the essential resources that give an agent the skills, connections, and knowledge it needs to achieve its goals. 🎯 They allow agents to access information, perform actions, and interact with various systems. 🤖
- Extensions (APIs): 🔗
- Function: Acts as a bridge between an agent and external APIs.
- Benefit: Provides a standardized way for an agent to use any API, letting the agent focus on the task (e.g., booking a flight) while the extension handles the technical complexities.
✈️
- Functions: 📝
- Function: Specialized tools that represent specific, reusable actions the agent can perform.
- Benefit: An agent's reasoning system will select the correct function based on the task (e.g., calling a
calculate_pricefunction to return the total cost). 💰
- Data Stores: 📚
- Function: Gives agents access to information like real-time, historical data, or knowledge bases.
- Benefit: Ensures that an agent's responses are accurate, relevant, and up-to-date (e.g., accessing real-time stock prices or customer information). 📈
- Plugins: 🔌
- Function: Add new skills or integrations to an agent's capabilities.
- Benefit: Connect an agent to specific external services (e.g., integrating with a calendar app to schedule appointments or a payment gateway to process transactions). 💳
This scenario demonstrates how an agent uses multiple tools to convert unstructured data into a helpful, actionable solution. ✨
- Scenario Goal: 🎯 Suggest the most convenient meeting location for all attendees based on a user-uploaded document.
- Tools Used: Document AI API (for document parsing), Google Maps API (for location analysis), and Custom Logic (for final recommendation).
- Document Processing: 📄 The agent uses the Document AI API to identify all addresses mentioned within the uploaded document.
- Geocoding: 📍 It uses the Google Maps Geocoding API to convert the extracted addresses into latitude and longitude coordinates. This is necessary for distance and time calculations.
- Location Analysis: 🧠 The agent then uses a custom logic (e.g., in a Cloud Function) to analyze the geocoded addresses and determine the most convenient location for all attendees.
Retrieval-Augmented Generation (RAG) enhances an LLM's capability by grounding its responses in external knowledge, leading to more accurate, relevant, and up-to-date responses. ✨
- Retrieval: 🔎 The LLM uses retrieval tools to find relevant information from outside sources. These tools can include:
- Data Stores: Internal databases and sources of structured/unstructured data. 🗃️
- Vector Databases: Find information semantically similar to the user's query using text embeddings.
- Search Engines: Used for relevant web content (articles, pages, etc.). 🌐
- Knowledge Graphs: Structured databases for entities and their relationships. 🔗
- Augmentation: 📝 The retrieved information is added to the original prompt given to the LLM. The augmented prompt now contains both the request and the external context.
- Generation: ✅ The LLM processes the augmented prompt to generate a response that is accurate and contextually appropriate. It can also cite its sources for increased trustworthiness.
- Iteration: 🔄 Advanced systems can refine the retrieval process (e.g., changing the search query or tools) if the initial search is insufficient.
Data stores are the key knowledge bases your agent draws upon. AI Applications allow connection to various types:
- Websites: 🌐 To access public information or stay up-to-date with current events.
- Structured Data: 📊 Information in tables or JSON format, like product catalogs.
- Unstructured Data: 📄 Files like HTML, PDF, and DOCX.
Vertex AI Search helps you create a Google-quality search experience using your own data as the foundation for responses. ✨ This directly implements a RAG approach to reduce "hallucinations" and increase trustworthiness.
- Document Search: 📄 Optimized for searching large repositories of unstructured, text-heavy documents.
- Media Search: 🎬 Specialized for searching within images, videos, and audio files (based on content or spoken words).
- Healthcare Search: ⚕️ Designed for the unique needs of the healthcare industry with regulatory compliance in mind.
- Search for Commerce: 🛍️ Optimized for product discovery and handling complex retail catalog queries.
- Search Summaries: 📝 Generates concise and informative summaries of search results, saving users time.
- Answers and Follow-ups: 💬 Adds AI-generated answers to search results and supports conversational follow-up questions.
- Security: 🔒 Offers granular access controls for robust data security.
- Advanced Analytics: 📈 Provides analytics to help understand search trends and user behavior.
- Scalable Infrastructure: 🚀 Designed to handle large volumes of data and search requests.
- Easy Integration: 🔗 Built to seamlessly integrate with existing enterprise systems using APIs and SDKs.
- Deterministic Agents: ⚙️ Are rule-based and rigid, following a defined system (e.g., if-then logic). They typically require low-to-medium code.
- Generative Agents: 💡 Use LLMs to give a real conversational feel. They can decide what to do autonomously and often require no or low code.
- Hybrid Agents: ✨ Google Cloud's solution combines the strengths of both, offering strict control with the flexibility of Generative AI.
Agent Assist supports live human agents with real-time help. 🤝 It uses AI and Gen AI to:
- Recommend Responses: 💬 Suggest optimal responses to customers.
- Suggest Content: 📚 Recommend relevant content from a knowledge base.
- Transcribe/Translate: Transcribe or translate calls in real time.
- Summarize Conversations: 📈 Helps agents resolve issues faster and ensures consistency.
Conversational Insights analyzes customer conversational data to provide data-driven metrics. 📊 It helps managers:
- Boost Efficiency: Improve agent performance and efficiency.
- Understand Sentiment: Analyze agent and caller sentiment. 🗣️
- Identify Gaps: The Generative FAQ feature helps identify common customer questions and gaps in existing FAQs.
CCaaS is a complete, cloud-based contact center solution that simplifies management. ☁️
- Core Function: Manages infrastructure, integrates with CRMs, and offers omnichannel support for a consistent customer experience.
- Integration: 🛠️ It seamlessly integrates with tools like Agent Assist, Conversational Agents, and Conversational Insights to create a full customer engagement suite.
Agentspace is a central platform for enterprise-ready expert agents tailored for your specific business needs. 💼
- Function: It uses AI agents and unified search to automate tasks and find information across all connected business systems. 💡
- Security: 🔒 Designed as a secure way for businesses to deploy AI agents using Google Cloud’s infrastructure.
- Productivity: 📈 It acts as a personal AI assistant for work, designed to increase productivity across various teams (sales, HR, R&D, etc.).
This table outlines the key distinctions between Agentspace (the comprehensive enterprise assistant) and NotebookLM Enterprise (the specialized research assistant).
| Feature | Agentspace | NotebookLM Enterprise |
|---|---|---|
| Purpose | Your comprehensive enterprise AI assistant that automates tasks and finds information across all business systems. 🤖 | A specialized AI tool for deep research, summarizing, and creating content based only on specific documents you upload. 📚 |
| Knowledge | Draws from all your connected business systems, effectively breaking down information silos. 🌐 | Focuses solely on the sources you provide, such as documents or web pages. 🔎 |
| Actions | Can automate tasks and orchestrate workflows across your connected systems. ⚙️ | Primarily a research and knowledge assistant that helps you understand and create content. ✍️ |
| Integration | A central platform that can connect to NotebookLM Enterprise. 🤝 | A tool for focused analysis that can be integrated as a data source within Agentspace. |