TLDR:
– Chrome now includes 6 high-level functions for calling Gemini Nano: Translator, Language Detector, Summarizer, Writer/Rewriter, Proofreader, and Prompt APIs.
– Some are available in Chrome 138 stable, with the Prompt API initially for Extensions.
– Microsoft Edge is also introducing similar on-device AI features.
Several months ago, Google launched Built-in AI in Chrome, enabling developers to access the in-browser model via JavaScript functions. Back then, we could only use prompts. However, over time, Google shifted its approach from solely providing prompts to offering more ready-to-use functions. Today, let’s explore what these ready-to-use functions are. The benefit of Built-in AI is that we don’t have to incur costs by directly calling the Gemini API every time, and simple tasks can be completed directly in the browser on the frontend.
Translator API
This Translator API helps us avoid the need to manually call a Cloud API for language translation because we can call it directly from the browser. However, we first need to check if our language pair is supported. This feature is a relief for web developers who need to support multiple languages, as previously, implementing such a feature required considering how to manage cloud costs.
Language Detector API
Besides being able to translate languages, we can also detect languages. Sometimes, we need this to check which API to call or to detect the language that needs to be translated in subsequent processes. Normally, we would have to implement this with an API or use a paid service.
Summarizer API
Coming to another API that I like, the Summarizer API. This API helps us transform long articles into simple summaries. In the options, we can specify the desired summary style: TLDR, teaser, key points, or headline. For all of these, we can define the length and also specify whether we want the output as Markdown or plain text.
If you ask what the results are like, I tried using it to summarize an article from this website, and this is roughly what it came out with. It’s quite usable for creating a TLDR feature on various websites or for summarizing long articles to be used in a chatbot to converse with users.
* Meta is actively developing and testing an official Instagram app for iPad, with a release expected this year.
* The delay in releasing an iPad-optimized Instagram app was partly due to technical limitations and perceived petty reasons.
* The decision to develop the iPad app comes as Instagram's user base has grown, the device has become more powerful, and the platform's use cases have evolved.
For these 3 APIs, we will be able to use them on Chrome version 138, both in Extensions and via Web APIs.
Writer / Rewriter API
These two APIs have quite similar functions: the Writer API helps us write articles or emails from a prompt, while the Re-writer API is used when we already have an email or article but want the AI to help change the writing style a bit. These two APIs are used almost 100% identically; the only difference is the function name.
From trying it out, the AI helps rewrite quite well. Plus, if we specify that it should be written as an email, it comes out in a very good email format.
These two APIs are currently in an Origin Trial status, so anyone who wants to use them can go register first. However, it’s thought that they will likely arrive in Chrome 142 at the earliest.
What’s interesting is that Microsoft itself has also embedded its own model, like Phi, into Microsoft Edge. This allows Microsoft Edge to use Summarizer API and Writer/Rewriter API functionalities just like Google Chrome. Importantly, the API interface is 100% identical.
Proofreader API
Regarding this API, it must be said that there’s very little information available (“it’s very empty”). There was a demo in one of the Google I/O sessions, but not many details have been released yet. However, from what I understand, this API will help tell us which word is misspelled by providing the text’s position, allowing us to highlight it ourselves. But one can guess that the API interface probably won’t be much different from the others. It will likely be available for us to try in Chrome 140 by enabling flags.
Prompt API
Okay, here’s the English translation of those two paragraphs:
Coming to the last one, this API is a normal prompt API where we can write anything, just like writing normal prompts for LLMs (Large Language Models). This feature has been available since the very first day of Built-in AI, but there have been many developments and improvements since 2024.
The current API interface looks more standard and is much easier to write/use (after having been changed three times).
In this version, the prompt API supports multimodal input, including both audio and images. We can send prompts with more than just text.
The output structure can be defined to be JSON or a Specific Type to make programming easier for us.
https://developer.chrome.com/docs/ai/structured-output-for-prompt-api
It’s clear that the built-in API is now becoming quite similar to the Cloud API. This doesn’t even include Stream output, which has been supported from the beginning. It probably won’t be long before we see Built-in AI performing on par with Cloud APIs. However, it’s a bit unfortunate that for now, the Prompt API will initially only be supported on Chrome extensions. If you want to use it on the web, you must enable a flag in the browser.
But another piece of good news about the Prompt API is that Microsoft has decided to adopt it as well, with a 100% identical API interface. In the future, even if using different browsers, there will be no need to worry about compatibility anymore.
Summary
It’s evident that the standards for all these functions are now aligning for both Chrome and Edge. I believe that in the future, even if each provider develops their own models, they probably won’t deviate much from this current direction. Essentially, the main workflow involves first calling an “availability” method to check if a model exists and if it needs to be downloaded. If a download is completed, you then call a “create” method. The specific methods for each function will then differ slightly.
The question on everyone’s mind likely concerns device requirements. It must be said that currently, this feature is only supported on desktops with at least 4GB of RAM and 22GB of available storage. We’ll have to see if a new, smaller version of Gemini Nano will be released. If one does become available, I think it could easily be incorporated into mobile devices or machines with lower specifications.
As for whether anyone is using this in production yet, the answer is “not yet.” We’ll likely have to wait a while longer before we see widespread actual implementation.
Reference
- https://developer.chrome.com/docs/ai/get-started
- https://learn.microsoft.com/en-us/microsoft-edge/web-platform/prompt-api
- https://learn.microsoft.com/en-us/microsoft-edge/web-platform/writing-assistance-apis
Leave a Reply