AI Acceleration in Every Core, 5th Gen Intel® Xeon®

by Chuck Brooks

Earlier this year, I wrote a blog highlighting the enhanced cybersecurity and computing performance capabilities of the 4th Gen Intel® Xeon® processor. (please see (14) A NEW PARADIGM OF PERFORMANCE AND CYBERSECURITY WITH INTEL’S NEW 4th GEN XEON PROCESSOR | LinkedIn) Since then, I have closely followed the research and development efforts of the next-gen processors Intel has been developing with a special focus on artificial intelligence. I am excited to share some of my insights below on the computing impact of their newest 5th Gen Intel® Xeon® processor product.

 We are now in a budding era of sophisticated classical computing that is pushing the digital transformation of most industries and government agencies. To optimize computing performance, associated with the emergence of technologies such as artificial intelligence (AI) and 5G, both the hardware and software will need to keep pace with the acceleration. Intel’s 5th Gen Intel® Xeon® processors are leading in that pathway.

 It is no secret that artificial intelligence is beginning to change the world as we know it. Virtually every industry will be significantly impacted by the networked computer components of the digital ecosystem, which are enabled by artificial intelligence and machine learning. New horizons in industries as diverse as genetic engineering, augmented reality, robotics, renewable energy, big data, and more may be made possible by these combined AI and computing capabilities.

 AI is transforming how we work. AI has the power to improve cognitive computing functions and produce economic rewards by automating knowledge job tasks from structured and unstructured commands. The advent of AI tools has impacted every business, small, medium, and large. The reality is that computing combined with AI drives performance in the dynamic commercial environment of today by improving an organization's ability to operate securely and efficiently. 

 With AI, power, and performance can be optimized to enable multitasking on workloads that are essential to the operation of the business. For employees to examine data swiftly, it is also necessary to maximize their productivity.

 With AI acceleration in every core, 5th Gen Intel Xeon processors fulfill that business mission and handle any company’s demanding AI workloads. Classical AI machine learning functions optimally on Intel Xeon CPUs. Other GPUs are not as good at machine learning training and inference as Intel Xeon processors, which have larger memory, faster cores, and Intel® Advanced Vector Extensions 512 (Intel® AVX-512).

 5th Gen Intel® Xeon® processors are equipped with AI acceleration in every core, meaning that businesses won't need to install separate discrete accelerators to tackle your demanding AI tasks, which can include inference and fine-tuning on models with up to 20 billion parameters.

 The capabilities of Intel’s 5th Gen Intel Xeon processors will have a revolutionary effect on corporate performance. Unfortunately, a lot of AI systems never reach the deployment stage because companies rapidly encounter issues with hardware cost, availability, integration, and scalability. Moreover, even though greater processing power is needed for certain tasks. Intel has remedied those common challenges with this new processor product line.

 Thanks to the company's integrated AI accelerators and Intel Advanced Matrix Extensions (Intel AMX), Intel's first integrated neural processing unit, or NPU, for power-efficient AI acceleration, has significantly improved AI performance. With up to 2.7x the AI performance of any other CPU1, 5th Gen Intel Xeon Scalable processors with Intel AI Engines let businesses add intelligence to more of their processes and products.

 The 5th generation of Intel Xeon processors outperforms all other CPUs, offering training and inference performance increases of up to 29% and 42%, respectively, over their predecessors' generation and up to 14x over 3-year-old processors.

 Discrete accelerator performance is available without the need for additional hardware or complexity thanks to Intel® Advanced Matrix Extensions (Intel® AMX). Because Intel AMX is so good at transfer learning and retraining, you can maintain the currency of your models.

 Moreover, Intel is developing platforms and solutions that enable clients to reliably assist in securing a variety of AI workloads across the data center, cloud, PC, and edge. This will help protect AI models through confidential computing. Even when working with sensitive or regulated data, data and AI models are protected with the use of trusted execution environments (TEEs) and confidential computing. In fact, the majority of data center AI inference deployments today run on Intel Xeon processors.

In addition, Intel’s 5th Gen Intel Xeon processors operate under an open-standards framework that is very important to AI adaptation of business computing applications. When it comes to AI in particular, Intel is one of the major contributors to the open-source community. As a matter of fact, 90% of developers use software that Intel has developed or optimized.

 Working with open-source models is preferred by many development teams over creating their own models from scratch. Numerous efficient, pre-trained AI models from Intel is readily available and simple to modify right out of the box. Additionally, there is no need to wait for hardware because of the world's largest processor ecosystem and wide available supply. An open-standards architecture enables the addition of accelerators and GPUs to Intel Xeon CPUs for more demanding applications.

 For companies, the benefits to the total cost of ownership of 5th generation Intel Xeon processors balance price and performance on technology that most companies already use for other tasks. Intel Xeon processors offer the broadest ecosystem and supply availability, a recognizable toolkit, and the capacity to expand across cloud, data center, and edge, to quickly implement AI.

 Intel Xeon processors are also pre-installed with software tools and the ecosystem to enable AI everywhere, including with the most widely used libraries and frameworks, giving businesses industry-leading AI performance. Numerous AI models that are ready for usage have been trained and refined by Intel. With the help of Intel Software Development Tools, developers can freely move code between various hardware architectures and suppliers without encountering expensive or time-consuming difficulties.

 In summary, the out-of-the-box built-in AI accelerators at every core generated by 5th Gen Intel Xeon processors will run AI everywhere and faster leading to a performance capability necessary for any business to thrive in this era of digital transformation. 

Please see link for more info on the Intel product announcement: AI Everywhere (intel.com)

This blog was sponsored by Intel  #5thGenIntelXeon, #IntelAmbassador #advancedcomputing #artificialintelligence

Chuck Brooks is President of Brooks Consulting International and is a Consultant with over 25 years of experience in cybersecurity, emerging technologies, marketing, business development, and government relations. He also serves as an Adjunct Professor at Georgetown University, where he teaches graduate courses on risk management, homeland security, and cybersecurity.

 As a thought leader, blogger, and event speaker, Chuck has briefed the G20 on energy cybersecurity, The US Embassy to the Holy See and Vatican on global cybersecurity cooperation. He has served on two National Academy of Science Advisory groups, including one on digitalizing the USAF, and another on securing BioTech.  He has also addressed USTRANSCOM on cybersecurity and served on an industry/government Working group for CISA focused on security space systems.

In his career, Chuck has received presidential appointments for executive service by two U.S. Presidents and served as the first Director of Legislative Affairs at the DHS Science & Technology Directorate. He has been named "Cybersecurity Person of the Year" by Cyber Express, Cybersecurity Marketer of the Year, a Top Cybersecurity SME to Follow, and a "Top 5 Tech person to follow" by LinkedIn where he has 112,500 followers on his profile.

Chuck has an MA from the University of Chicago, a BA from DePauw University, and a Certificate in International Law from The Hague Academy of International Law