Chip race: Microsoft, Meta, Google, and Nvidia battle it out for AI chip supremacy

Apple — like Google, Meta, Microsoft, OpenAI and everyone else this side of Nvidia — is reportedly working on custom server hardware to power AI models as it prepares to introduce a slew of new features.

Over the past decade, Apple has emerged as a leading player designing chips for iPhones, iPads, Apple Watch and Mac computers. The server project, which is internally code-named Project ACDC—for Apple Chips in Data Center—will bring this talent to bear for the company’s servers, according to people familiar with the matter.

Apple watcher Mark Gurman followed up saying a similar-sounding project was canceled and it doesn’t make sense anyway: it would be too expensive, lack differentiation, and Apple prefers on-device AI.

Update: Added Gurman’s rebuttal.

AMD also says an AI PC refresh cycle will help PCs return to growth in 2024, and that 150 software vendors will be developing for AMD AI PCs by year’s end. The company’s top priority is ramping AI data center GPUs, though, which are “tight on supply.” New AI chips are coming “later this year into 2025,” too.

OpenAI’s Batch API now lets users upload a file of bulk queries to the AI model, like categorizing data or tagging images, with the understanding that they won’t need immediate attention. Promising results within 24 hours lets them run when there is unused compute power, and keeps those pricey GPUs humming around the clock.

Intel first introduced its Gaudi 3 AI accelerator last year, but now the company has revealed more details on performance. When compared to the H100 GPU, Intel says its Gaudi 3 accelerator can deliver “50% faster time-to-train on average across the Llama2 models” with better efficiency.

The company also says the Gaudi 3 AI Accelerator will be a “fraction of the cost” of Nvidia’s pricey H100. It will become available to companies like Dell, HPE, and Lenovo in the second quarter of this year.

Reuters reports the list could strengthen the Commerce Department’s existing restrictions on US tech shipments to Chinese chip factories. The US government has voiced national security concerns about letting China access US technology to grow its own capabilities.

US companies have complained it’s difficult to know which Chinese factories produce advance chips and are subject to the restrictions, Reuters says.

CNN gained entry into the training facility where TSMC teaches engineers to design and operate the machines that build semiconductors. The hope is to train engineers to “seed” factories it’s building in the US, Japan, and Germany.

TSMC told CNN that it needs “to hire thousands more” employees to staff facilities around the world as demand for advanced chips grows.

That’s what this NYT report says a16z and Saudi Arabia’s Public Investment Fund are considering, which may explain the rumors that Elon Musk (who denied them), Sam Altman, and others looking to the Middle East to fund their AI dreams.

It also recalls last fall’s news that the US government added an “additional licensing requirement” for Nvidia and AMD AI shipments to unspecified Middle Eastern countries.

JEDEC Solid State Technology Association has released details about its new standard, which it says is better positioned to handle the demands of gaming, networking, and AI.

JESD239 GDDR7 is the first JEDEC standard DRAM to use the Pulse Amplitude Modulation (PAM) interface for high frequency operations. Its PAM3 interface improves the signal to noise ratio (SNR) for high frequency operation while enhancing energy efficiency. It also says GDDR7 has double the bandwidth over GDDR6, up to 192 GB/s per device, and double the number of channels.

We don’t expect to see GDDR7 out in the world until Nvidia and AMD release next-gen GPUs, which could come before the end of 2024 but is a long way from being confirmed.

Intel vice president David Feng said during Mobile World Congress that as part of the push to put AI into everything it builds, it will produce 40 million CPUs for AI PCs this year and 60 million in 2025, reports Nikkei Asia.

The “AI PC” concept includes Microsoft’s new CoPilot button plus Intel Core Ultra processors with built-in GPUs and neural processing units for AI models, which are now available as part of its vPro platform for business laptops.

With over 600 statements of interest received, Commerce Secretary Gina Raimondo acknowledged today that the amount requested is more than twice the $28 billion the government has budgeted to invest.

We have decided to prioritize projects that will be operational by 2030. There are worthy proposals with plans to come online after 2030 that we say no to in order to maximize our impact in this decade...We anticipate that our investments in leading-edge logic chip manufacturing will put us on track to produce roughly 20% of the world’s leading-edge logic chips by 2030, up from the zero percent we produce today.

The CHIPS Act originally had $52 billion in subsidies to boost US semiconductor manufacturing, but it’s not nearly enough to catch up by itself — industry leader Taiwan Semiconductor Manufacturing Company (TSMC) earmarked $44 billion in 2022 just to expand its existing capacity.

As Nvidia reports its Q4 2023 earnings, CEO Jensen Huang says:

Accelerated computing and generative AI have hit the tipping point. Demand is surging worldwide across companies, industries and nations

That demand, and Nvidia’s dominance in AI chips, powered the company’s record earnings of $60.9 billion for the full year 2023 — a 126 percent increase from last year. Its Q4 revenue of $22.1 billion is an increase of an astounding 265 percent over the same period last year.

The open-source Gemma models are optimized for “the installed base of over 100 million Nvidia RTX GPUs,” installed in PCs around the world, in addition to Nvidia’s ubiquitous AI chips like the H100.

The models will also be part of Nvidia’s Chat with RTX demo, which lets AI models run locally and access users’ files to generate answers to prompts.

Intel has said 2025 is the year it leads the world in chips again (TSMC begs to differ).

Beyond that lies Intel 14A — the company’s smallest node yet, thanks to High-NA EUV. But Intel is hedging its bets with a tick-tock 18A successor, where the P in “18A-P” stands for performance jump. What does 14A stand for? Intel didn’t say. “We think this is the next leadership technology and we don’t want to give anyone something to shoot at,” Intel VP Craig Orr tells me.

It’s code-named Izanagi and is designed to combat Nvidia, sources are telling Bloomberg. OpenAI’s Sam Altman had been courting Softbank and other companies to potentially bankroll a chipmaking arm already — but the Izanagi project is reportedly not connected to that.

If successful, Izanagi may become the largest investment in AI since Microsoft put a rumored $10 billion wager on OpenAI.

As first reported by The Information, which says he’d been working on an AI assistant. Karpathy’s exit comes a year after rejoining OpenAI, where he was a founding member, with stops at Apple and Tesla in between.

Karpathy’s path has taken him from Google’s If I Had Glass competition to obtaining a Vision Pro, and tonight, he tweeted, “My immediate plan is to work on my personal projects and see what happens.”

The outlay is part of the CHIPS and Science Act signed in 2022. The CHIPS Manufacturing USA Institute will see investments of “at least $200 million” in experiments to lower the cost of US-based chip production and development.

And around $300 million will fund research into advanced substrate packaging, which can help improve performance, avoid shortages, and enable new uses for semiconductors, like chips that process data with light instead of electrons.

Reuters reports on Nvidia’s new unit that will design custom AI chips, even as its customers plan for a future less reliant on Nvidia’s dominant H100 AI chips. Those can start at $16k each, and power the majority of large language models in use today, which has pushed its market value well beyond $1 trillion.

Nvidia officials reportedly met with OpenAI, Meta, Google, Amazon, and Microsoft to offer its “bespoke” AI chip services, as well as businesses in other industries like gaming and telecom.