Apple has an Xcode plan for genAI — and that’s just the beginning

In the wake of Microsoft’s release of its GitHub Copilot tool, Apple could soon follow suit with a new version of Xcode that includes generative AI (genAI) capabilities designed to help developers write code.

The feature is expected to be capable of generating code in response to natural language commands. This should be really helpful for Apple developers, particularly as Apple’s Xcode Pro should also be able to verify code, check for flaws, and more.

Developers, developers, state-sponsored hackers

It’s possible even the most security-conscious developers will be able to use the tool, as it seems likely Apple will develop it to be as private as possible; that means your code and your requests don’t end up stored on cloud servers somewhere.

It is also interesting to consider how this software might work beside some of the other innovative genAI technologies Apple and others are working on. Think how virtual worlds for gaming apps can be created on the fly and how these virtual spaces can also be used within software-based experiences. (There was speculation last summer that this kind of large language model (LLM)-based virtual world creation was already on Apple’s radar.)

Hopefully, of course, the move won’t inspire state-sponsored hackers such as (presumably) the insidious NSO Group to use these automated tools to brush up their hacks, as Microsoft has already confirmed some groups have been doing with Copilot.

GenAI in the Spotlight

The latest report also suggests Apple might choose to improve Spotlight search with genAI. Spotlight is the on-device search available within every device that lets you sift through your digital “stuff.”

Making Spotlight more contextually aware, fluid, and responsive through generative AI is potentially powerful, as it means achieving almost anything you can do with your device — including groups of actions reliant on different apps — might be possible just by describing what you need.

That’s like taking AppleScript or Shortcuts and turning the volume up to 11, though Cupertino’s developer teams aren’t sure yet whether it will be ready this year.

Cupertino rising

The accelerating pace of well-timed reports from inside Apple’s AI teams represent two key modalities: Apple is making it clear that it is not going to be left behind in the AI arms race, while sending a message to its audiences about what’s coming.

We’ve looked at quite a lot of this activity even this week, but it is now clear that where it makes sense, Apple’s teams are weaving generative AI within strategic domains all through the company’s platform ecosystems.

Bloomberg reports that Apple’s software vice president, Craig Federighi, has requested employees build as many new AI features as they can for Apple’s devices this year, though the company is also building more complex articulations that may not be ready for some time yet. 

The latter implies a road map of genAI deployment across the Apple ecosystem, likely defined at least to an extent by hardware, software, and ethical constraints.

The real everything machine

All the same, the continued acceleration of Apple’s efforts speaks volumes and promises much more. Think back to the launch of the original iPhone and all you had was music, browsing, and communication; now, you’ll be able to add automated creation of almost anything you can speak to the list. The ceiling of the possible remains unmet.

Signing off, here are the most recent revelations concerning Apple’s work around AI at the moment:

• Apple has built Keyframer, a tool that creates animations for 2D images.
• Apple has developed MLLM-Guided Image Editing to perform image edits using text commands.
• The company last year unveiled tech that lets genAI models work on edge devices such as iPhones, no server required.
• There’s the ML Explore machine learning framework for Apple Silicon.
• Apple Ferret helps optimize machine learning.
• Apple built a tool to create avatars from video.
• Apple has published numerous research papers and made dozens of AI-related acquisitions in the last year, with more coming.

What are the secrets Apple’s not leaking?

Bear in mind that, when considering an enterprise as secretive as Apple, the fact that so many in development projects have now been leaked or revealed simply means the company has a lot going on that we don’t yet know about.

And now that development of visionOS 1 is complete (if such development ever really is complete), the company’s software teams have a little more leeway to seek out those domains in which LLM intelligence might make the most difference to people’s lives.

This is a work in progress, and the beginning of that attempt is nowhere near its end.

Please follow me on Mastodon, or join me in the AppleHolic’s bar & grill and Apple Discussions groups on MeWe.