Mobile Tutorials, Satellite And IPTV tutorials, PC And Automotive Tips

AI computing: Is The GPU Relevant in The Field Of AI Computing?

Home » Technology » AI computing: Is The GPU Relevant in The Field Of AI Computing?

Published on at > Last Updated by .- Leave a Comment

Semiconductor chips are usually divided into digital and analog chips in the industry. Among them, digital chips account for a relatively large market share, reaching about 70%.
 Digital chips can be subdivided into logic, memory, and micro-control units (MCU). Our focus today is on logic chips and the relevance of the GPU in AI computing. 
Logic chips are computing chips, to put it bluntly. It contains various logic gate circuits, which can realize operational and logical functions. It is one of the most common chips.
 The CPU, GPU, FPGA, and application-specific integrated circuits (ASIC) that everyone often hears about are all logic chips. And the so-called “AI chips ” used in AI, which are particularly popular now, mainly refer to them.

CPU (Central Processing Unit)

CPU (Central Processing Unit)

Everyone knows that the CPU is the heart of the computer.
 Modern computers are based on the von Neumann architecture, born in the 1940s. This architecture includes an arithmetic unit (logical operation unit, ALU), control unit(CU), memory, input device, and output device.
Von Neumann Architecture"s diagram
 When data is processed, it will be put into the memory first. Then, the control unit will get the corresponding data from the memory and give it to the arithmetic unit for calculation.  After the operation is completed, the result is returned to the memory.
 This process also has a more exciting name: “Fetch – Decode – Execute – Memory Access – Write Back.”
As you can see, the CPU is responsible for the two core functions of the arithmetic and control units.
 Specifically, arithmetic units ( including addition, subtractors, multipliers, and dividers ) perform arithmetic and logical operations and are the actual work.
 The control unit is responsible for reading instructions from memory, decoding instructions, and executing instructions.
 In addition to the arithmetic unit and controller, the CPU includes clock modules and registers (cache) components.

CPU Time And Registers

The clock module is responsible for managing the CPU time and providing a stable time base for the CPU. It drives all operations in the CPU and schedules the work of each module by periodically sending out signals.
 Registers are high-speed memories in the CPU that temporarily store instructions and data. Its “buffer” between the CPU and memory (RAM) is faster than ordinary memory, preventing the memory from “dragging” the CPU’s work.
 The capacity and access performance of the register can affect the number of accesses from the CPU to the memory, affecting the entire system’s efficiency. We will mention it later when we talk about memory chips.

CPU Architecture Classifications

CPUs are generally classified based on instruction set architecture, including x86 and non-x86 architecture. x86 is a complex instruction set ( CISC ), while non-x86 is a reduced instruction set ( RISC).
PCs and most servers use the x86 architecture, dominated by Intel and AMD. Many non-x86 architectures have risen rapidly in recent years, including ARM, MIPS, Power, RISC-V, Alpha, etc. 

GPU (graphics processing unit)

Graphic cards

GPU is the core component of the graphics card. Its full English name is Graphics Processing Unit, a graphics processing unit (graphics processor).
GPU cannot be equated with the graphics card. In addition to the GPU, the graphics card includes video memory, a VRAM voltage stabilizing module, an MRAM chip, a bus, a fan, a peripheral device interface, etc.
 In 1999, NVIDIA took the lead in proposing the concept of GPU.
 The reason why GPUs are proposed is because the game and multimedia businesses developed rapidly in the 1990s.
 These businesses put higher requirements for the computer’s 3D graphics processing and rendering capabilities. Traditional CPUs couldn’t handle it, so GPUs were introduced to share the work.
 According to the form, GPU can be divided into independent GPU (dGPU, discrete/dedicated GPU) and integrated GPU (iGPU, integrated GPU), often called independent graphics and integrated graphics.
GPUs are also computing chips. Therefore, like the CPU, it includes components such as arithmetic units, control units, and registers.
 However, because the GPU is mainly responsible for graphics processing tasks, its internal architecture differs from the CPU’s.
CPU vs GPU components comparison
 As shown in the figure above, the number of CPU cores (including ALU) is relatively small, only a few dozen at most. However, the CPU has a large cache (Cache) and a complex control unit (CU).
 The reason for this design is because the CPU is a general-purpose processor. As the central core of the computer, its tasks are very complex. It must deal with different types of data calculations and respond to human-computer interaction.
 Complex conditions and branches, as well as synchronization and coordination between tasks, will bring a lot of branch jumps and interrupt processing work. It requires a larger cache to save various task states to reduce the delay during task switching. It also requires more complex controllers for logic control and scheduling.
 The strength of the CPU is management and scheduling. The real work function is not strong (ALU accounts for about 5% to 20%).
 If we think of the processor as a restaurant, the CPU is like an all-around restaurant with dozens of senior chefs. This restaurant can cook all kinds of cuisines, but because there are so many cuisines, it takes a lot of time to coordinate and prepare the dishes, and the serving speed is relatively slow.
 

How A GPU Works

GPUs are designed for graphics processing, and their tasks are very clear. What it does is graphics rendering. Graphics are composed of massive pixels large-scale data with highly unified types and no dependence on each other.
 Therefore, the task of the GPU is to complete parallel operations on a large amount of homogeneous data in the shortest possible time. The so-called “cores” of scheduling and coordination are actually very few.

Parallel computing, of course, requires more cores.

As shown in the previous figure, the number of cores of the GPU far exceeds that of the CPU and can reach thousands or even tens of thousands (so it is called “multicores”).
 The core of the GPU, called Stream Multi-processor (SM ), is an independent task processing unit.
 The entire GPU is divided into multiple streaming processing areas. Each processing area contains hundreds of cores. Each core is equivalent to a simplified version of the CPU, with the functions of integer operations, floating point operations, and queuing and result collection functions.
 The GPU control unit has simple functions and relatively few caches. Its ALU ratio can reach more than 80%.
 Although the processing power of a single GPU core is weaker than that of a CPU, its large number makes it very suitable for high-intensity parallel computing. Under the same transistor scale, its computing power is stronger than the CPU’s.
 Let’s take the restaurant as an example. A GPU is like a monolithic restaurant with thousands of junior chefs. It is only suitable for specific cuisines. However, because there are many chefs and the side dishes are simple, everyone cooks together, and the food is served faster.

GPU and AI computing: Relevance of the GPU in AI Computing

As we all know, everyone is rushing to buy GPUs for AI computing now. Nvidia also made a lot of money from this. Why is this so?
 The reason is simple: AI computing, like graphics computing, also contains many high-intensity parallel computing tasks.
relevance of the gpu in ai computing
 
Deep learning is currently the most mainstream artificial intelligence algorithm. From a process perspective, it includes two links: training and inference.
 In the training process, a complex neural network model is trained by feeding a large amount of data. In the reasoning process, the trained model is used to infer various conclusions using a large amount of data.
 Since the training process involves massive training data and complex deep neural network structures, it requires a vast calculation scale. It requires a relatively high computing power performance of the chip. The reasoning process requires simple, specified repeated calculations and low latency.
 The specific algorithms they use, including matrix multiplication, convolution, loop layer, gradient operation, etc., are decomposed into many parallel tasks, which can effectively shorten the time to complete the task.
 With its powerful parallel computing capabilities and memory bandwidth, GPU can handle training and inference tasks well and has become the industry’s preferred solution in deep learning.
 Currently, most companies use NVIDIA GPU clusters for AI training. If adequately optimized, a GPU card can provide computing power equivalent to dozens to hundreds of CPU servers.
 However, in the inference segment, the market share of GPU is not that high. We will talk about the specific reasons later.

Origin Of Applying GPU To Calculations Other Than Graphics Rendering

 Applying GPU to calculations other than graphics first originated in 2003.
 That year, the concept of GPGPU (General Purpose computing on GPU, GPU-based general computing) was first proposed. 
It means using the computing power of GPU to perform more general and extensive scientific calculations in non-graphics processing fields.
 Based on traditional GPU, GPGPU has been further optimized and designed to make it more suitable for high-performance parallel computing.
 In 2009, several Stanford scholars demonstrated the results of using GPUs to train deep neural networks, which caused a sensation for the first time.
 
A few years later, in 2012, two students of Geoffrey Hinton, the father of neural networks — Alex Krizhevsky and (Ilya Sutskever ), using the “deep learning + GPU ” solution, proposed the deep neural network AlexNet, which increased the recognition success rate from 74% to 85%, winning the Image Net Challenge in one fell swoop.
 This ultimately detonated the “AI+GPU” wave. Nvidia quickly followed up and invested a lot of resources, improving GPU performance by 65 times in three years.
 In addition to hard-core computing power, they are also actively building a development ecosystem around GPUs. They have established a CUDA (Compute Unified Device Architecture) ecosystem based on their GPUs, providing a complete development environment and solutions to help developers more easily use GPUs for deep learning development or high-performance computing.
 These early, careful layouts ultimately helped Nvidia reap huge dividends when AIGC broke out. Their market value is as high as US$1.22 trillion (nearly six times that of Intel), making them the veritable “uncrowned king of AI.”

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.