More than a dozen current and former employees of some major AI companies have set up a website and published an open letter to companies in the industry on it.

Titled “A Right to Warn about Advanced Artificial Intelligence,” the letter acknowledges AI’s potential benefits but seeks to underscore its “serious risks,” which for the authors range from the “entrenchment of existing inequalities” to “manipulation and misinformation” and all the way to “human extinction.”

AI companies, they explain, have “strong financial incentives” to evade regulation and possess “substantial non-public information” on capacities and limits that they have little obligation to share with governments and no obligation to share with civil society. Moreover, humanity cannot count on whistleblowers to come forward because whistleblowers are protected only when the law backs their revelations, and much that would need revelation falls under no law.

The authors are therefore calling on the companies:

  • Not to enter into or enforce any agreement that prohibits “disparagement” or criticism of the company for risk-related concerns or retaliate for criticism of risky tech
  • To provide for a “verifiably anonymous” channel through which current and former employees can raise concerns to management, regulators or a suitable independent organization
  • To permit open criticism and public employee discussion of risky tech – without disclosure of trade secrets
  • Not to retaliate against current and former employees who publicly disclose confidential information on risky tech if internal warnings have failed

The signatories are Jacob Hilton, Daniel Kokotajlo, William Saunders, Carroll Wainwright, Daniel Ziegler and six anonymous persons, all formerly of OpenAI; Ramana Kumar, formerly of Google DeepMind; and Neel Nanda, currently of Google DeepMind and formerly of Anthropic. The letter is endorsed by Yoshua Bengio, Geoffrey Hinton and Stuart Russell.

AI OS

Meanwhile, the world’s biggest operating systems for desktop and mobile computing are all incorporating AI. This fact has become public awareness through Elon Musk’s social media posts. On June 11, Musk said he would ban iPhones from the premises of his companies if Apple, in particular, proceeds with its AI project.

Incidentally, Musk, too, worries about human extinction through powerful AI:

Apple announced its project on June 10. It calls the project Apple Intelligence (another way to spell out AI) and describes it as “the personal intelligence system that puts powerful generative models at the core of iPhone, iPad, and Mac.” The system combines “generative models” of AI with “personal context” and is “deeply integrated” into all three versions of the forthcoming edition of Apple’s operating system: iOS 18, iPad OS 18 and macOS Sequoia.

It appears that some part of most of the computing on any Apple device will now occur off the device’s own processor and on the cloud. For instance, the OS will have “systemwide Writing Tools” to rewrite, proofread and summarize text wherever it might be. In other words, whatever the user writes – a chat, an email, an essay – will be run through Apple’s servers.

This has already been the case for a few years with Apple’s built-in dictation software, which used to run on a Mac’s processor.

Another example: “In the Notes and Phone apps, users can now record, transcribe, and summarize audio. When a recording is initiated while on a call, participants are automatically notified, and once the call ends, Apple Intelligence generates a summary to help recall key points.” Presumably, then, Apple and its partner, OpenAI, will be listening in on your conversations – even if you’re working for a competitor: say, Musk’s xAI.

Apple does address this. Apple Intelligence, it says, is “aware of your personal information without collecting your personal information.” The system uses something called Private Cloud Compute, which “can draw on larger server-based models, running on Apple silicon, to handle more complex requests for you while protecting your privacy.” Apple promises that “your data is never stored,” is “used only for your requests” and comes with a “verifiable privacy promise.”

The others

Microsoft has added something called Copilot to Windows 11, but there appears to be a difference. Microsoft’s AI tool is “a keystroke away.” The user turns it on.

That said, Microsoft has been under fire for an AI system called Recall, which takes snapshots of the computer’s screen every five seconds if the screen has changed from the previous snapshot. These snapshots, Microsoft says, are stored and analyzed on the computer, not on the cloud, and enable the user to search for text or images with “natural language,” as opposed to file names. As critics have pointed out, however, this could provide hackers with snapshots of data (passwords, bank statements, etc.).

According to the feature’s support page, the user must open Recall to use it. According to The Information, plans to make it operative by default on new Copilot Plus PCs have been scrapped. Users will now be given the option to enable it when setting up their computer.

Microsoft’s service is otherwise similar to Apple’s: help with writing, summaries, inspiration, image generation and so on.

Android has added the same sort of thing, but the marketing materials provide less detail. The developer page, however, has a section on Gemini Nano (Gemini is Google’s AI), whose operational priorities are “privacy, offline functionality, low latency, and cost.”

Topics