Apple Intelligence: new ideas in AI UX

Apple intelligence is here, the long awaited AI play from Apple. In a sea of GPT wrappers, Apple has stayed true to its UX prowess and delivered new, innovative user experiences around LLM capabilities. Here's what I thought was very interesting:

Smart reply

Fusing LLMs into email clients is nothing new, but Apple has a few new tricks up its sleeve. I wonder why the Gmail app hasn't introduced similar features – maybe nobody trusts them on privacy?

(image credits: Apple)

Apple says,

use a Smart Reply in Mail to quickly draft an email response with all the right details. Apple Intelligence can identify questions you were asked in an email and offer relevant selections to include in your response. With a few taps you’re ready to send a reply with key questions answered.

Identifying questions is familiar ground, similar to many of the 'next step extraction' features in email and communication platforms. This capability aligns with existing functionality that users are already accustomed to, making it a natural extension of current tools and interfaces.

Predicting possible answers and structuring them into a quick questionnaire is a genius move. This reduces the frustration of typing out an entire question to instruct a LLM, by taking it down to the press of a button. It streamlines the interaction process, minimizing the effort required from the user while still allowing them to guide the conversation effectively.

Taking these answers back to then generate a reply reduces the chance of a user saying something confusing to the LLM, and increases the control for the user while also making sure that the quality of the output is trustworthy and high quality. By incorporating the user's input in a structured manner, the system can produce responses that are tailored to the user's needs while maintaining a high standard of accuracy and relevance.

Math notes

An exciting new development in productivity software has emerged, featuring a user experience that crosses the line between Word and Excel.

(image credits: Apple)

Apple says,

Crunch numbers, evaluate expressions, and assign variables — right from Notes. The app solves expressions inline as soon as you type an equals sign.

This innovative approach blends the familiarity of text-based interfaces with the computational power of spreadsheets, paving the way for more language-based interactions and enabling the integration of large language models (LLMs).

The text-based interface is particularly meaningful, as it facilitates the use of LLMs, which thrive on language inputs. Notably, the equals sign serves as a neat, definite 'button' to initiate computations, streamlining the process of invoking the system's capabilities.

In a sense, users are 'chatting' with the machine through their notes, but this approach circumvents the common challenge of having to formulate a specific question in a blank chat box.

While this development does not necessarily address the issues of accuracy and hallucinations inherent in LLMs, it presents a neat introduction of LLM capability into a productive workflow.

Rather than attempting a full-blown effort to generate complex outputs akin to Shakespeare's works, this integration offers a more focused and practical application of LLM technology within a familiar productivity context.