dev0

Testing Local Gemini in Chrome Canary

Testing Google Chromes Window.AI in Canary 128

Introduction

Google is revolutionizing the AI landscape by shipping a Gemini:Nano with Google Chrome Canary. This innovation ensures that your data stays local, providing enhanced privacy and security unless you forget all the telemetry collected by Alphabet ☺️ One of the standout benefits of this local LLM is the speed of responses, offering a seamless and efficient user experience. Planned to be a standard feature in Chrome, I like where the portability of the llms are heading.

The Concept

Simple lab with the window.ai object, and test how we can stream answers to a small frontend app.

Enable Gemini Nano in Chrome Canary

  • Get Chrome Canary
  • Open chrome://flags/
  • Set “Prompt API for Gemini Nano” to Enabled
  • Set “Enables optimization guide on device” to Enable BypassPerfRequirement
  • Restart Chrome
  • Open chrome://components/, check whether the model is successfully downloaded in Optimization Guide On Device Model, and click Check for update
  • Open the console and test if window.ai returns.

An example form

Here’s the complete code for our AI-enhanced web page:

<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <script type="module" src="https://cdn.jsdelivr.net/npm/@ionic/core/dist/ionic/ionic.esm.js"></script>
  <script nomodule src="https://cdn.jsdelivr.net/npm/@ionic/core/dist/ionic/ionic.js"></script>
  <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@ionic/core/css/ionic.bundle.css" />
  <title>Hello Gemini</title>
  <style>
    #canary {
      position: relative;
      height: 100vh; /* Adjust as needed */
      padding-bottom: 50px; /* Adjust to prevent overlap */
    }

    #answerItem {
      position: absolute;
      bottom: 0;
      width: 100%;
    }
  </style>
</head>

<body>
  <div id="canary">
    <ion-list>
      <ion-item>
        <ion-icon slot="start" name="happy-outline" aria-hidden="true"></ion-icon>
        <ion-input placeholder="Ask me anything" clear-input="true" id="user"></ion-input>
        <ion-button id="askbtn">Ask</ion-button>
      </ion-item>
      <ion-item id="answerItem">
        <ion-icon slot="start" name="logo-chrome" aria-hidden="true"></ion-icon>
        <ion-label id="answer">...</ion-label>
      </ion-item>
    </ion-list>
  </div>

  <script>
    /**
     * Checks if the AI is available and ready for use.
     * @returns {Promise<boolean>} - Returns true if AI is available and ready, otherwise false.
     */
    async function isAIAvailable() {
      const hasAI = window.ai != null;
      const isReady = await window.ai?.canCreateTextSession();
      return hasAI && isReady === "readily";
    }

    /**
     * Handles the question and answer process using the AI session.
     */
    async function handleQA() {
      try {
        if (!(await isAIAvailable())) {
          console.log("AI is not available or not ready");
          return;
        }

        const session = await window.ai.createTextSession();
        const inputText = document.getElementById('user').value;
        const stream = session.promptStreaming(inputText);
        await updateDom(stream);
      } catch (err) {
        console.error("Error in handleQA:", err);
      }
    }

    /**
     * Updates the DOM with streamed responses from the AI session.
     * @param {AsyncIterable<string>} stream - The stream of AI responses.
     */
    async function updateDom(stream) {
      const outputItem = document.getElementById('answer');
      outputItem.textContent = ''; // Clear previous content

      try {
        for await (const chunk of stream) {
          outputItem.textContent += chunk;
        }
      } catch (err) {
        console.error("Error in updateDom:", err);
      }
    }

    /**
     * Initializes event listeners for the page.
     */
    function initializeEventListeners() {
      document.addEventListener('keydown', async function(event) {
        if (event.ctrlKey && event.key === 'Enter') {
          event.preventDefault(); // Prevent default behavior
          document.getElementById('askbtn').dispatchEvent(new MouseEvent('mousedown'));
        }
      });

      document.getElementById('askbtn').addEventListener('mousedown', async function(event) {
        event.preventDefault(); // Prevent default mousedown behavior
        await handleQA();
      });
    }

    // Initialize event listeners when the page is loaded
    document.addEventListener('DOMContentLoaded', initializeEventListeners);
  </script>
</body>

</html>

Explanation

HTML Structure
  • ion-list and ion-item: We use Ionic components to create a modern and responsive UI. The ion-list holds our items, including the input field and the response display area.
  • Icons and Labels: We use ion-icon to add visual cues and ion-label to display dynamic content.
CSS Styling
  • Positioning: The #canary div is styled to be a container with a relative position, and we use absolute positioning for the #answerItem to keep it at the bottom of the container.
JavaScript Logic
  • isAIAvailable Function: Checks if the AI model is available and ready for use.
  • handleQA Function: Handles the user input, initiates a session with the AI model, and processes the response stream.
  • updateDom Function: Updates the DOM with the streamed responses from the AI model.
  • Event Listeners:
    • A keydown listener triggers the AI response when the user presses Ctrl + Enter.
    • A mousedown listener on the Ask button initiates the AI response process.

Conclusion

By leveraging the Gemini model in Google Chrome Canary and the Ionic framework, we can create an interactive web page that provides a seamless AI-driven user experience. This example showcases the potential of integrating AI capabilities into web applications, making them more dynamic and responsive to user inputs.