At I/O 2024, Google’s teaser for gave us a glimpse at the place AI assistants are going sooner or later. It’s a multi-modal function that mixes the smarts of Gemini with the sort of picture recognition talents you get in Google Lens, in addition to highly effective pure language responses. Nevertheless, whereas the promo video was slick, after attending to attempt it out in particular person, it is clear there’s an extended method to go earlier than one thing like Astra lands in your cellphone. So listed below are three takeaways from our first expertise with Google’s next-gen AI.
Sam’s take:
Presently, most individuals work together with digital assistants utilizing their voice, so straight away Astra’s multi-modality (i.e. utilizing sight and sound along with textual content/speech) to speak with an AI is comparatively novel. In concept, it permits computer-based entities to work and behave extra like an actual assistant or agent – which was one in every of Google’s massive buzzwords for the present – as a substitute of one thing extra robotic that merely responds to spoken instructions.
In our demo, we had the choice of asking Astra to inform a narrative based mostly on some objects we positioned in entrance of digicam, after which it advised us a beautiful story a few dinosaur and its trusty baguette making an attempt to flee an ominous purple gentle. It was enjoyable and the story was cute, and the AI labored about in addition to you’ll anticipate. However on the identical time, it was removed from the seemingly all-knowing assistant we noticed in Google’s teaser. And other than possibly entertaining a toddler with an unique bedtime story, it didn’t really feel like Astra was doing as a lot with the information as you may want.
Then my colleague Karissa drew a bucolic scene on a touchscreen, at which level Astra appropriately recognized the flower and solar she painted. However probably the most partaking demo was after we circled again for a second go along with Astra working on a Pixel 8 Professional. This allowed us to level its cameras at a set of objects whereas it tracked and remembered each’s location. It was even sensible sufficient to acknowledge my clothes and the place I had stashed my sun shades regardless that these objects weren’t initially a part of the demo.
In some methods, our expertise highlighted the potential highs and lows of AI. Simply the power for a digital assistant to inform you the place you might need left your keys or what number of apples have been in your fruit bowl earlier than you left for the grocery retailer might enable you avoid wasting actual time. However after speaking to a number of the researchers behind Astra, there are nonetheless numerous hurdles to beat.
In contrast to numerous Google’s current AI options, Astra (which is described by Google as a “research preview”) nonetheless wants assist from the cloud as a substitute of having the ability to run on-device. And whereas it does assist some stage of object permanence, these “memories” solely final for a single session, which at present solely spans a couple of minutes. And even when Astra might keep in mind issues for longer, there are issues like storage and latency to contemplate, as a result of for each object Astra remembers, you danger slowing down the AI, leading to a extra stilted expertise. So whereas it’s clear Astra has numerous potential, my pleasure was weighed down with the information that it will likely be a while earlier than we are able to get extra full-feature performance.
Karissa’s take:
Of all of the generative AI developments, multimodal AI has been the one I’m most intrigued by. As highly effective as the most recent fashions are, I’ve a tough time getting excited for iterative updates to text-based chatbots. However the concept of AI that may acknowledge and reply to queries about your environment in real-time appears like one thing out of a sci-fi film. It additionally offers a a lot clearer sense of how the most recent wave of AI developments will discover their means into new units like sensible glasses.
Google supplied a touch of that with Challenge Astra, which can at some point have a glasses element, however for now could be principally experimental (the video through the I/O keynote have been apparently a “research prototype.”) In particular person, although, Challenge Astra didn’t precisely really feel like one thing out of sci-fi flick.
It was in a position to precisely acknowledge objects that had been positioned across the room and reply to nuanced questions on them, like “which of these toys should a 2-year-old play with.” It might acknowledge what was in my doodle and make up tales about totally different toys we confirmed it.
However most of Astra’s capabilities appeared on-par with what Meta has accessible with its sensible glasses. Meta’s multimodal AI may also acknowledge your environment and do a little bit of artistic writing in your behalf. And whereas Meta additionally payments the options as experimental, they’re at the least broadly accessible.
The Astra function that will set Google’s strategy aside is the truth that it has a built-in “memory.” After scanning a bunch of objects, it might nonetheless “remember” the place particular gadgets have been positioned. For now, it appears Astra’s reminiscence is restricted to a comparatively quick window of time, however members of the analysis crew advised us that it might theoretically be expanded. That may clearly open up much more prospects for the tech, making Astra appear extra like an precise assistant. I don’t must know the place I left my glasses 30 seconds in the past, however when you might keep in mind the place I left them final evening, that will really really feel like sci-fi come to life.
However, like a lot of generative AI, probably the most thrilling prospects are those that haven’t fairly occurred but. Astra may get there finally, however proper now it appears like Google nonetheless has numerous work to do to get there.
Compensate for all of the information from Google I/O 2024 proper right here!
