MacroXX

MacroXX

Share this post

MacroXX
MacroXX
NVDA

NVDA

From training to inference

MacroXX's avatar
MacroXX
Mar 18, 2025
∙ Paid
5

Share this post

MacroXX
MacroXX
NVDA
1
Share

NVIDIA's CEO Jensen Huang will deliver the keynote speech for GTC 2025 on Tuesday, March 18, 2025, at 1 PM ET at the SAP Center in San Jose, California. Here are the key details:

  1. Streaming: The keynote will be available for free streaming on NVIDIA's official website and YouTube channel.

    Expected announcements:

    • New Blackwell Ultra GPUs with improved performance and increased memory (288GB)

    • Updates on the upcoming Rubin GPU series, set for release in 2026

    • Possible revelations about post-Rubin products

    • Advancements in AI, including applications in robotics, healthcare, and autonomous vehicles

  2. Focus areas: The keynote is expected to cover quantum computing, "physical AI," robotics, healthcare, and Agentic AI.

  3. Industry context: NVIDIA faces challenges due to recent overheating issues with early Blackwell cards, U.S. export controls, and competition from Chinese AI labs. This keynote presents an opportunity for Huang to address these concerns and showcase NVIDIA's latest innovations.

  4. Conference scope: GTC 2025 will feature about 2,000 speakers, 400 exhibitors, and 1,000 sessions, making it a premier event for AI and technology professionals.

NVIDIA currently dominates the AI chip market, but faces increasing competition from several key players:

Major Competitors

  1. AMD: AMD is NVIDIA's main rival in the GPU market. They launched the MI300 series for AI workloads in 2023 and are releasing the MI350 series to compete with NVIDIA's H200. AMD claims their MI325X chip has market-leading inference performance.

  2. Intel: Intel is playing catch-up in the GPU market but remains a significant competitor. They are working on their Gaudi 3 AI chips to compete in the data center market.

  3. Cloud Providers: Major cloud companies are developing their own AI chips:

    • AWS: Developing Trainium3 chips

    • Google (Alphabet): Creating Trillium chips

    • Microsoft Azure: Producing Maia 100 chips

Emerging Competitors

Several AI startups and tech companies are also entering the market:

  • Groq: Focusing on fast LLM inference

  • SambaNova Systems: Offering high-capacity hardware solutions

  • Cerebras: Developing the WFE-3 chip

  • Apple: Working on the M4 chip

  • Meta: Developing MTIA v2 chips

Market Dynamics

While NVIDIA controls around 80% of the add-on GPU market, the increasing demand for AI chips is creating opportunities for competitors. The software ecosystem remains a critical factor, with NVIDIA's CUDA platform still having an advantage in terms of usability compared to competitors like AMD.

As the AI hardware market continues to evolve rapidly, with performance improvements multiple times annually, competition is likely to intensify, potentially leading to more choices and better pricing for consumers and businesses in the AI chip market.

Training vs. Inference

AI training and inference are two distinct but interconnected processes in the development and deployment of artificial intelligence models. Here's a comparison of these two crucial phases:

Training

Training is the process of teaching an AI model to perform specific tasks by exposing it to large datasets. During this phase:

  • The model learns patterns and relationships within the data.

  • It's computationally intensive, often requiring powerful GPUs or TPUs.

  • It's typically a one-time or periodic process, depending on the need for updates.

  • The goal is to minimize prediction errors and optimize model parameters.

NVIDIA has established a dominant position in the AI training market, with its leadership expected to continue into 2025. Here's an overview of NVIDIA's dominance and the challenges it faces:

Market Share and Growth

  • NVIDIA held 70-95% of the advanced AI chip market share in 2024, according to Mizuho Securities.

  • The company's market cap reached $2.9 trillion in July 2024, making it the world's third most valuable company.

  • NVIDIA is projected to consume 77% of wafers used for AI processors in 2025, up from 51% in 2024.

Factors Contributing to Dominance

  1. Superior Technology: NVIDIA's GPUs consistently outperform competitors in AI training benchmarks.

  2. Software Ecosystem: The CUDA toolkit and extensive pre-trained models give NVIDIA a significant advantage over competitors.

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 MacroXX
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share