NVIDIA/acc
GPUs, Chipsets, Energy, CUDA and cool stuff.
Introduction
Welcome to the computing power era:
Welcome to the computing era, where the most valuable assets are data, energy, and computing power. NVIDIA currently stands as the most valuable company in the world, driving this new era with its groundbreaking technology and innovation.
Energy
On this previous post “AGI is coming”, we talked about computing power and the most probable solutions for the future energy supply problem.
The most probable solution? Nuclear Fusion.
Who already place his bets on this? Right Sam Altman.
Here some words on Helion his investment targeting the nuclear fusion solution for the energy dilemma.
Data
Data, currently is part of the tech companies is the most valuable asset of them. They run ads targeted to users using data, sell data, improve products with data, and most importantly, train models with data.
This chart show the data-landscape this 5 players hold the most amount of data on users. Maybe a part of the explanation of why Google is doing so well.
In the future Google plans to use their browser market share to instead of Google something up you just use Gemini and interact with shops, calendar, gmail and all of Google products. Kind of a everything app.
Computing power
So now we are going to talk about computing power and why this companies are paying NVIDIA so much for it.
AI Systems and NVIDIA's Role
As a reminder, AI systems operate through two core stages:
🎓 Training: AI learns from vast amounts of data, developing intelligence and pattern recognition. NVIDIA's powerful GPUs dominate this phase.
🧠 Inference: AI applies its knowledge to real-world tasks and decision-making. While facing stiffer competition here, NVIDIA is making significant progress.
First, this is the current landscape of NVIDA Customer shipments.
Maybe you wonder why Open AI isnt in this customer list taking into account that they recieved shipments with H100 GPUs
Well, the most probable explanation is that OpenAI buys all their compute via Microsoft Azure part of the partnership and 13$Billion investment made by Microsoft.
(Don’t double check the amount of the Microsoft investment is literally 13 Billion dollars)
At this point of the article, I realized that is going to be long. So first thank you and we are going to wrap this part up.
Basically, we are talking about that one single company is providing a whole bunch of whales computing power and recieveing their money. NVIDIA has in revenue the equal gdp of countries like Germany for example.
CUDA the game changer: Transforming GPU Programming
Now we are going to talk about CUDA and why its so important to understand NVIDIA.
CUDA (Compute Unified Device Architecture) has fundamentally transformed the way we utilize GPUs for complex computational tasks. Prior to CUDA, leveraging the power of GPUs for purposes such as model training required navigating through cumbersome graphical APIs. This process was similar to communicating with an artist who would then interpret and execute the instructions, often leading to inefficiencies and complications.
The introduction of CUDA brought about a significant shift. It provided developers with a more accessible and intuitive way to program GPUs, removing the need for intermediary graphical APIs. With CUDA, developers can write instructions directly for the GPU, streamlining the process and maximizing computational efficiency.
This transformation is akin to replacing the artist with a direct line of communication between the programmer and the GPU, represented by the NVIDIA RTX 4090. The result is a more efficient, powerful, and user-friendly approach to harnessing GPU capabilities for a wide range of complex tasks.
Here a full video about it:
CUDA is probably one of the things that made NVIDA outperform AMD for example.
Why? Pretty simple CUDA is compatible with NVIDIA GPUs so if you are working for example with a AMD 6700xt you will face so many headaches that a 3090 will solve only for being a NVIDIA GPU.
More simple? Well this means that is better to have a NVIDIA GPU for other purposes than gaming and that all the datacenters in the world are built with NVIDA GPUs for easier compatibilty with CUDA.
NVIDA Financials
For a detailed financial analysis of NVIDIA, I recommend reading this article, which provides excellent financial data and a deep understanding of why NVIDIA is outperforming other players.
NVIDIA Q1 FY25 (January 2024) Income Statement
NVIDIA's Q1 FY25 income statement shows impressive financial performance, demonstrating strong growth across various segments. The company's total revenue for the quarter is $26.0 billion, reflecting an 18% increase quarter-over-quarter (Q/Q). Wich is fucking crazy.
Data Center:
Revenue: $22.6 billion
Growth: 23% Q/Q
Data Center is the largest revenue contributor, accounting for a significant portion of NVIDIA's total revenue. The substantial growth indicates strong demand for NVIDIA's data center solutions, driven by the increasing need for AI and cloud computing capabilities.
Here some data about the revenue amount of NVIDIA Datacenters vs Gaming:
What’s next?
For NVIDIA
Nvidia's announced Blackwell architecture a significant advancement in AI and high-performance computing. Designed to push the boundaries of AI training and inference, Blackwell GPUs offer unprecedented computational power and efficiency. These GPUs feature cutting-edge innovations in architecture, interconnects, and memory, enabling faster data processing and lower latency. Blackwell aims to meet the growing demands of AI models, making it a cornerstone for future AI developments in various industries.
For more details, you can visit the Nvidia Blackwell Architecture page.
A New Class of AI Superchip
Built with 208 billion transistors, more than 2.5x the amount of transistors in NVIDIA
Hopper GPUs, and using TSMC’s 4NP process tailored for NVIDIA, Blackwell is the largest
GPU ever built. NVIDIA Blackwell achieves the highest compute ever on a single chip, 20
petaFLOPS.
Well, recently some starups came up with alternatives to GPU powered AI. Developing chipsets that are way powerful than NVIDIA GPUs.
This are some projects in this space:
Etched
Etched is pioneering the development of transformer ASIC chips designed to run AI models significantly faster and cheaper than traditional GPUs. Their flagship product, Sohu, is capable of handling advanced AI tasks such as real-time voice agents, coding enhancements with tree search, and multicast speculative decoding. With an open-source software stack and scalable architecture, Sohu supports models up to 100 trillion parameters, providing a robust platform for future AI innovations. Learn more on their website.
China
Enflame
China's artificial intelligence (AI) chip startup Enflame, backed by tech giant Tencent, raised 2 billion yuan ($273.68 million) from investors including funds linked to a government authority in Shanghai.
Tencent, which has cooperated with Enflame in developing an AI chip named Zixiao and contributed to the startup's earlier fundraising.
Chinese companies like Baidu Kunlunxin, T-Head (Alibaba), Tencent, Huawei, Cambricon Technologies, Biren Technology, Horizon Robotics, and Enflame Technology are investing heavily in self-development to reduce dependence on Nvidia, focusing on innovations in AI chip technology to enhance their competitive edge in the global market.
Thats all for now, thank you for reading.