Scality’s Post

Hot off the presses from Scality! Check out the details on #ScalityRINGXP - the first and only all-flash #objectstorage to deliver 500-microsecond response-time latencies for small object data. 🔥🔥🔥

Unlock the value of data to speed innovation. Do you need to feed your hungry NVIDIA GPUs with data? Lightening fast?⚡️ Years of co-development, testing, and data services for NVIDIA GPUs - and other accelerated clusters - come together in a leadership platform of the leading scalable storage software, a full throughput NDR support, and future flexibility. The IBM Storage Scale System 6000: Announcement ➡️ ibm.co/40lM3QO Website: ➡️ https://lnkd.in/dSwt7nBQ Data Sheet IBM SSS6000: ➡️ https://lnkd.in/dkutpk3P Accelerate business growth and AI innovation. Get the eBook here: ➡️ https://lnkd.in/d6GjXVAz 👁🐝Ⓜ️‬ #IBM #IBMStorage #GlobalDataPlatform #ArtificialIntelligence #IBMStorageScale #IBMStorageScaleSystem #TechnologicalSuperiority #InnovationLeader #IBMtechnology #technology #IBMStorageRocks🚀

Western Digital Introduces New 64TB SSD, 32TB HDD for the AI Data Cycle High-Performance PCIe Gen5 SSD: High-speed AI training and inference. High Capacity 64TB SSD: Designed for fast AI data lakes, enabling rapid access to large datasets. UltraSMR 32TB HDD: Cost-effective storage at scale for massive data management. In my school days, I used 8" floppy disks with a 1MB capacity. Back then, it would have been hard to imagine using 1TB of data. #WD #HDD #SDD #Storage https://lnkd.in/gkt4gUX3

Why IBM Scale is the Best Platform for NVIDIA DGX BasePOD & SuperPOD 💡 It's offering unparalleled scalability, high-performance capabilities, and seamless integration with NVIDIA's GPU-accelerated computing technology. - Scalability: IBM Scale ensures effortless scaling to meet the demands of NVIDIA SuperPOD, ensuring that your infrastructure grows in alignment with your business needs. - High-Performance: Benefit from IBM Scale's optimized architecture and advanced cooling systems, empowering NVIDIA Superpod to operate at peak performance, delivering faster results and enhanced productivity. - Seamless Integration: IBM Scale is tailored to work seamlessly with NVIDIA Superpod, offering a unified and efficient computing environment that streamlines workflows and reduces complexity. Storage Scale System 6000 is designed for extreme performance ▶ up to 310 gigabytes per second (GB/S) throughput with low latency; ▶ up to 13 million IOPS with the Hyper-Store NVMeoF performance tier; and ▶ up to 3.4PBe (effective capacity) in a standard 4U rack space. https://lnkd.in/eh2PtUK7 #NVIDIADGX #AI #IBMSCALE

𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝘀: 𝟲 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗶𝗻𝗴 𝘁𝗶𝗽𝘀 I learned the hard way. Save hours of trial and error and 𝗰𝘂𝘁 𝗰𝗼𝘀𝘁𝘀 𝗯𝘆 𝟴𝟬%. ⌨ 💰 𝘋𝘪𝘴𝘤𝘭𝘢𝘪𝘮𝘦𝘳: 𝘐 𝘥𝘰 𝘧𝘶𝘭𝘭 𝘧𝘪𝘯𝘦-𝘵𝘶𝘯𝘪𝘯𝘨 𝘶𝘴𝘪𝘯𝘨 𝘵𝘩𝘦 𝘛𝘳𝘢𝘯𝘴𝘧𝘰𝘳𝘮𝘦𝘳𝘴 𝘭𝘪𝘣𝘳𝘢𝘳𝘺. 𝘚𝘰𝘮𝘦 𝘵𝘩𝘪𝘯𝘨𝘴 𝘮𝘪𝘨𝘩𝘵 𝘣𝘦 𝘥𝘪𝘧𝘧𝘦𝘳𝘦𝘯𝘵 𝘪𝘯 𝘰𝘵𝘩𝘦𝘳 𝘴𝘦𝘵𝘶𝘱𝘴. Let's begin 🤓: 1️⃣ Remember to 𝗮𝗱𝗱 𝘁𝗵𝗲 𝗲𝗼𝘀 (end of sentence) token.  Make sure this is added at the end of every example when you format the dataset to create the "text" column. If you have ever trained a model that kept going and didn't stop, that could have been it. I know I did. 🙋♂️ 2️⃣ If you add new tokens to the tokenizer, make sure you 𝗿𝗲𝘀𝗶𝘇𝗲 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝗲𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴𝘀. The first layer of the LLM is where each token gets embedded to a corresponding vector. The number of embeddings in this layer has to match the vocabulary size of the tokenizer. If the layer is not updated, your new tokens will never get trained because they effectively don't exist inside the model. 3️⃣ 𝗨𝘀𝗲 "𝗽𝗮𝗰𝗸𝗶𝗻𝗴" instead of padding to length. Traditionally, to make every example the same length, we add padding, but that is quite wasteful. Instead, tokenize the dataset, concat everything and then chunk it into equal size sequences of tokens. This should be defined by your sequence length. This shaves 𝟯𝟬% of your training time easily. 4️⃣ 𝗨𝘀𝗲 𝗳𝗹𝗮𝘀𝗵 𝗮𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻. This is a very easy and quick one to save further 𝟮𝟬-𝟮𝟱% off the training time. 5️⃣ If you use deepspeed with ZeRO-3 on multiple GPUs, 𝗱𝗼 𝗻𝗼𝘁 𝗼𝗳𝗳𝗹𝗼𝗮𝗱 𝘁𝗼 𝘁𝗵𝗲 𝗖𝗣𝗨. Even if it gives you a larger batch size, it's an illusion as your training speed will be much slower. The CPU itself and transfer speed are the bottleneck. Offload only when you don't have enough GPUs to load the model. Running only on GPU cuts the training time by 𝟱𝟬% 🤯 6️⃣ 𝗔𝟭𝟬𝟬 𝗦𝗫𝗠 𝗶𝘀 𝘁𝗵𝗲 𝗺𝗼𝘀𝘁 𝗰𝗼𝘀𝘁 𝗲𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲 GPU I have tried and I tried pretty much all of them. The H100 is the crown jewel, but it costs 2x as much per hour and it's "only" 1.5x faster. 𝗪𝗶𝘁𝗵 𝘁𝗵𝗶𝘀 𝘀𝗲𝘁𝘂𝗽, 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗟𝗹𝗮𝗺𝗮-𝟯 𝟴𝗕 𝗼𝗻 𝟴𝘅 𝗔𝟭𝟬𝟬 𝗦𝗫𝗠 𝗼𝗻 𝗮 𝟭𝗕 𝘁𝗼𝗸𝗲𝗻𝘀 𝗱𝗮𝘁𝗮𝘀𝗲𝘁 𝘁𝗮𝗸𝗲𝘀 𝗿𝗼𝘂𝗴𝗵𝗹𝘆 𝟭𝟰 𝗵𝗼𝘂𝗿𝘀 𝗽𝗲𝗿 𝗲𝗽𝗼𝗰𝗵. Not too bad... Happy fun-tuning 👋 ------------------------ I share AI content from my direct, hands-on experience. Please 𝗰𝗼𝗺𝗺𝗲𝗻𝘁 💬 , 𝗹𝗶𝗸𝗲 👍 𝗼𝗿 𝗿𝗲𝘀𝗵𝗮𝗿𝗲 ♻ if you think others would find it useful. 🙏 Hit the 🔔 to stay updated.

My last article published by the CFA UK "A Data Scientist's Guide to Hardware" serves a double purpose: 1. Local hardware Guide: for readers interested in understanding hardware specs, especially those planning to purchase a new laptop or desktop. 2. Remote GPUs Guide: overview of the current offering of remote GPU capabilities for experienced data scientists with computing-intensive workflows. Based on the fast-paced innovation of industry, let's hope the article doesn't become ancient history before you even get through it. You can read it here: https://lnkd.in/dFTbSA97 #hardware #gpus #datascience #machinelearning #quantitativeresearch #cfainstitute #cfauk #ml #laptop #investments

#AMD needs to catch up to CUDA, so they need the chips obviously but the software ecosystem to deploy training and inference models is the key. https://lnkd.in/dCXbAt5i

Nicely outlined model fine tuning touch points and insights. Works like a charm. #llms #llmops #aiml #mlops

Enterprises can achieve high levels of inference performance and take full advantage of hardware acceleration on Intel Corporation hardware by integrating the #opensource Triton Inference Server with the OpenVINO backend. Learn more about this powerful solution for deploying and serving #machinelearning models.  https://lnkd.in/gPNHq-JX