Gemini on Mac Can Now Click and Type in Your Apps With Computer Use

Share

The Gemini app for Mac can now use your computer for you. Google announced computer use on November 19, a feature that lets Gemini click, type, open menus and fill in forms across your local files, desktop apps and the web. It works in a separate background window, so your own screen and cursor stay free. It is rolling out now in English in the US for Google AI Plus, Pro and Ultra subscribers.

Key Takeaways: Gemini computer use on Mac
What it does: you describe a multi-step task and Gemini does the clicking and typing in your Mac apps and on the web.
How it runs: in an isolated background window with a live preview in your chat, so you can keep working.
You stay in control: you choose which apps it can use, watch every step, pause any time and approve key actions.
Who gets it: Google AI Plus, Pro and Ultra users in the US, in English, on Apple Silicon Macs with macOS Sequoia 15 or later.
Chromebooks and Windows: Google has not announced computer use for either yet.

1. What computer use can do

Desktop apps, not only the web: until now, Gemini’s Task feature on Mac (once called Gemini Spark) could read local files and use connected apps like Gmail and Google Calendar. Computer use goes further. Gemini can now work inside real desktop apps, navigating menus and filling in forms the way you would. We covered how Gemini tasks went free earlier this week.

Google’s own examples:

  • Batch color-grade photos in a design app, or set up 3D scenes.
  • Pull weights, sizes and addresses out of local PDFs to fill in shipping quote forms.
  • Grab numbers and screenshots from live dashboards and build a Google Slides deck.
  • Look at clothing photos in a folder, research prices and stage resale listings.
Gemini for Mac computer use prompt asking to model a butterfly sketch in Blender
In Google’s demo, a single prompt ending in @Computer use asks Gemini to model and rig a child’s butterfly sketch in Blender

2. How it works

Just add @Computer use: according to 9to5Google, you add @Computer use to your prompt. Under Customize, then Connectors, new Desktop apps such as Blender and Autodesk Fusion join the existing Git and Obsidian connections.

A window of its own: Gemini works in an isolated background window and streams a live preview into your chat. Your main screen and cursor stay yours, so you can keep typing or take a call while it works.

Safety checks: you pick which apps Gemini can touch, you can stop it with one click, and it asks you to approve key actions before they happen. 9to5Google adds that it pauses for logins and payments.

Butterfly 3D model rigged in Blender by Gemini
The finished result in Blender: Gemini built and rigged the butterfly model on its own

3. Who can use it

RequirementStatus
Google AI Plus, Pro or Ultra on a personal account (18+)Rolling out
US, English languageRolling out
Apple Silicon Mac with macOS Sequoia 15 or laterYes
Free Gemini planNo
Intel MacsNo
Windows PCsNot announced
Chromebooks and GooglebooksNot announced

Get the app: computer use lives in the Gemini app for Mac, which Google offers at gemini.google/mac.

Gemini opening a live AR web app preview in Google Chrome
Later in the same demo, Gemini turns the model into an AR web app and opens a live preview in Google Chrome

4. What about Chromebooks?

Not yet: Google only announced computer use for the Mac app. It said nothing about Windows, ChromeOS or Googlebook.

What you have today: on a Chromebook Plus, Gemini in Chrome can already complete multi-step actions on websites for you, through the Let Gemini browse for you setting. The difference is that it stays inside the browser. Computer use on Mac can work in any desktop app you allow.

Timing on the Mac side: The Verge notes that Apple is preparing new limits on apps that ask for full disk access, partly because of AI agents like this one, with a preview in the macOS 27.3 beta.

Our take

This is Google’s most capable desktop agent so far, and the background window is a clever touch, since you do not have to sit and watch your mouse move on its own. The obvious gap is that it starts on Mac, not on Google’s own Chromebooks and Googlebooks. Most of what Chromebook users do already happens in Chrome, where Gemini can browse for you, but we would love to see this reach Android apps on Googlebook next.


Discover more from Chrome Geek

Subscribe to get the latest posts sent to your email.

Sophie is a Chicago-based editor for Chrome Geek covering Chromebooks, Gemini, and the Google ecosystem. When she is offline, you can usually find her watching Blackhawks hockey or grabbing iced coffee in the city.