Ire Sun

When working with an unfamiliar JSON string and needing to identify all keys, using a JSON_KEYS extractor can be extremely useful. #bigquery https://lnkd.in/eY3nKTE6

6

𝗘𝘅𝗰𝗶𝘁𝗶𝗻𝗴 𝗡𝗲𝘄𝘀 𝘄𝗶𝘁𝗵𝗶𝗻 𝘁𝗵𝗲 𝗟𝗮𝗿𝗴𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀 (𝗟𝗟𝗠𝘀) 𝗦𝗽𝗮𝗰𝗲! PyTorch has officially released TorchTune, a new tool designed to fine-tune Large Language Models (LLMs) with ease and efficiency. This groundbreaking tool promises to streamline the fine-tuning process, making it more accessible to researchers and practitioners alike. https://lnkd.in/eMf6dNZJ 𝗧𝗼𝗿𝗰𝗵𝘁𝘂𝗻𝗲 𝗢𝗳𝗳𝗲𝗿𝘀: ▪ 𝗣𝘆𝗧𝗼𝗿𝗰𝗵-𝗻𝗮𝘁𝗶𝘃𝗲 𝗟𝗟𝗠 𝗶𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻𝘀: Build on familiar ground with composable building blocks for popular LLMs. ️ ▪ 𝗠𝗲𝗺𝗼𝗿𝘆-𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗶𝗻𝗴 𝗿𝗲𝗰𝗶𝗽𝗲𝘀: Train on consumer-grade GPUs with 24GB VRAM – no beefy machines needed! ▪ 𝗘𝘅𝘁𝗲𝗻𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆 𝗮𝗻𝗱 𝗲𝗮𝘀𝗲 𝗼𝗳 𝘂𝘀𝗲:  YAML configs and support for common dataset formats streamline your workflow. 𝗙𝗼𝗿 𝘁𝗵𝗼𝘀𝗲 𝗶𝗻𝘁𝗲𝗿𝗲𝘀𝘁𝗲𝗱 𝗶𝗻 𝗱𝗶𝘃𝗶𝗻𝗴 𝗱𝗲𝗲𝗽𝗲𝗿, 𝗵𝗲𝗿𝗲 𝗮𝗿𝗲 𝘀𝗼𝗺𝗲 𝗿𝗲𝘀𝗼𝘂𝗿𝗰𝗲𝘀: Code: https://lnkd.in/gEgETe5t Blog: https://lnkd.in/gP3mctA6 Tutorials: https://lnkd.in/gvh2eEsU This is a fantastic opportunity to explore and innovate, I'd be very interested in hearing your thoughts. ▪ 𝗪𝗵𝗮𝘁’𝘀 𝘆𝗼𝘂𝗿 𝘁𝗮𝗸𝗲 𝗼𝗻 𝘁𝗵𝗶𝘀?  ▪ 𝗛𝗼𝘄 𝗱𝗼 𝘆𝗼𝘂 𝘀𝗲𝗲 𝗧𝗼𝗿𝗰𝗵𝗧𝘂𝗻𝗲 𝗶𝗺𝗽𝗮𝗰𝘁𝗶𝗻𝗴 𝘆𝗼𝘂𝗿 𝗰𝘂𝗿𝗿𝗲𝗻𝘁 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 𝗼𝗿 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵? ▪ 𝗛𝗼𝘄 𝗱𝗼 𝘆𝗼𝘂 𝘁𝗵𝗶𝗻𝗸 𝗧𝗼𝗿𝗰𝗵𝗧𝘂𝗻𝗲 𝘄𝗶𝗹𝗹 𝗶𝗺𝗽𝗮𝗰𝘁 𝘁𝗵𝗲 𝗳𝘂𝘁𝘂𝗿𝗲 𝗼𝗳 𝗺𝗮𝗰𝗵𝗶𝗻𝗲 𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗔𝗜 𝗱𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁? #AI #PyTorch #TorchTune #LLMs #MachineLearning

24

1 Comment

In context of debate if large context #LLM models will eventually negate the need for #RAG, an article from Databricks has some good insights - • Longer contexts boost RAG accuracy... to a point • Performance often peaks at 32k-64k tokens, then declines • Top models (GPT-4, Claude 3.5) maintain consistency at scale • Others show unique failure modes (e.g., copyright concerns, summarization instead of Q&A) • Optimal context size varies by model and task • Lack of long-context training data may explain some issues Key takeaway: Long context and RAG are synergistic, but LLMs need refinement to fully leverage extended contexts. https://lnkd.in/eehdKtCm

17

🌟 Unveiling the Magic of PySpark RDDs! 🚀🔥 🌟 Are you diving into the world of big data and distributed computing? PySpark’s Resilient Distributed Datasets (RDDs) are the ultimate game-changers. 💡 In my latest Medium blog, I’ve broken down the fundamentals of RDDs, covering: 🔸 Partitioning for parallelism 🧩 🔸 Fault tolerance with lineage 🛠️ 🔸 Lazy evaluation for optimization ⚙️ 🔸 In-memory computation for speed ⚡ Plus, I’ve compared RDDs to DataFrames to help you choose the right tool for your next data project. Whether you’re handling unstructured data or need low-level transformations, RDDs have got you covered! 👉 https://lnkd.in/g3xxMxcr Let me know your thoughts and favorite PySpark use cases in the comments below. Let’s spark the conversation! 💬✨ #PySpark #DataEngineering #BigData #RDDs #DistributedComputing #TechBlog #Medium

18

1 Comment

Pyspark Architecture Inner Working! Interviewer: Today, let's delve into the inner workings of PySpark. Specifically, I'd like to discuss stages, DAGs (Directed Acyclic Graphs), and tasks. Can you explain what happens when a PySpark program is executed? Candidate: Of course. When a PySpark program is submitted, the first step is to construct a logical execution plan, represented as a DAG. This DAG captures the sequence of transformations and actions defined in the program. Interviewer: Great. Can you elaborate on what a stage is within the context of PySpark? Candidate: Certainly. A stage is a collection of tasks that can be executed in parallel. PySpark optimizes the execution of transformations by dividing them into stages based on data dependencies. Stages are typically delineated by shuffle operations, such as joins or aggregations, where data needs to be exchanged between partitions. Interviewer: I see. So, how does PySpark determine the execution plan and break it down into stages? Candidate: PySpark's Catalyst optimizer analyzes the logical execution plan and applies optimizations to generate an optimized physical execution plan. This plan is then broken down into stages based on data dependencies and transformations that can be performed in parallel. Interviewer: Fascinating. Now, let's talk about tasks. What role do tasks play in the execution of a PySpark program? Candidate: Tasks are the smallest units of work in PySpark. Each stage consists of one or more tasks that are executed on partitions of the input data. Tasks perform transformations or actions on the data and are executed in parallel across the cluster. Interviewer: Excellent explanation. Now, can you walk me through the sequence of events that occur when a PySpark program is submitted? Candidate: Sure. When a PySpark program is submitted, the driver node creates a SparkContext, which coordinates the execution of the program. The program defines a series of transformations and actions on RDDs or DataFrames, which are translated into a logical execution plan represented as a DAG. This DAG is optimized by the Catalyst optimizer to generate an optimized physical execution plan. The plan is then broken down into stages, each consisting of tasks that are executed in parallel across the cluster. Interviewer: Thank you for that overview. #bigdata #pyspark #etl #dataengineering

86

Pyspark Architecture Inner Working! Interviewer: Today, let's delve into the inner workings of PySpark. Specifically, I'd like to discuss stages, DAGs (Directed Acyclic Graphs), and tasks. Can you explain what happens when a PySpark program is executed? Candidate: Of course. When a PySpark program is submitted, the first step is to construct a logical execution plan, represented as a DAG. This DAG captures the sequence of transformations and actions defined in the program. Interviewer: Great. Can you elaborate on what a stage is within the context of PySpark? Candidate: Certainly. A stage is a collection of tasks that can be executed in parallel. PySpark optimizes the execution of transformations by dividing them into stages based on data dependencies. Stages are typically delineated by shuffle operations, such as joins or aggregations, where data needs to be exchanged between partitions. Interviewer: I see. So, how does PySpark determine the execution plan and break it down into stages? Candidate: PySpark's Catalyst optimizer analyzes the logical execution plan and applies optimizations to generate an optimized physical execution plan. This plan is then broken down into stages based on data dependencies and transformations that can be performed in parallel. Interviewer: Fascinating. Now, let's talk about tasks. What role do tasks play in the execution of a PySpark program? Candidate: Tasks are the smallest units of work in PySpark. Each stage consists of one or more tasks that are executed on partitions of the input data. Tasks perform transformations or actions on the data and are executed in parallel across the cluster. Interviewer: Excellent explanation. Now, can you walk me through the sequence of events that occur when a PySpark program is submitted? Candidate: Sure. When a PySpark program is submitted, the driver node creates a SparkContext, which coordinates the execution of the program. The program defines a series of transformations and actions on RDDs or DataFrames, which are translated into a logical execution plan represented as a DAG. This DAG is optimized by the Catalyst optimizer to generate an optimized physical execution plan. The plan is then broken down into stages, each consisting of tasks that are executed in parallel across the cluster. Interviewer: Thank you for that overview. #bigdata #pyspark #etl #dataengineering

26

1 Comment

🚀 [New Article] 🚀 : An overview of classifier-free guidance for diffusion models (part 2) On their new blog post, Nikolas Adaloglou and Tim Kaiser expand on part 1 and review different classifier-free guidance (CFG) approaches to a diffusion model trained without conditioning dropout. In these cases, CFG cannot be applied. So how what are the alternatives in such purely unconditional generative setups ? Article : https://lnkd.in/dqigagjN Revisit part 1 here: https://lnkd.in/dVucTJXe

43

Here is the blog post explains how to write to OneLake from Azure Databricks as well as how to write to ADLS Gen2 from Fabric Spark. https://lnkd.in/gb4kac5m #onelake #AzureDatabricks #ADLSGen2

9

LLMs have become more capable with better training and data. But they haven’t figured out how to “think” through problems at test-time. The latest research from Scale finds that simply scaling inference compute–meaning, giving models more time or attempts to solve a problem–is not effective because the attempts are not diverse enough from each other. 👉 Enter PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language to encourage response diversity. PlanSearch enables the model to “think” through various strategies before generating code, making it more likely to solve the problem correctly. The Scale team tested PlanSearch on major coding benchmarks (HumanEval+, MBPP+, and LiveCodeBench) and found it consistently outperforms baselines, particularly in extended search scenarios. Overall performance improves by over 16% on LiveCodeBench from 60.6% to 77%. Here’s how it works: ✅ PlanSearch first generates high-level strategies, or "plans," in natural language before proceeding to code generation. ✅ These plans are then further broken down into structured observations and solution sketches, allowing for a wider exploration of possible solutions. This increases diversity, reducing the chance of the model recycling similar ideas. ✅ These plans are then combined before settling on the final idea and implementing the solution in code. Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. When PlanSearch is paired with filtering techniques—such as submitting only solutions that pass initial tests—we can get better results overall and achieve the top score of 77% with only 10 submission attempts. Big thanks to all collaborators on this paper including: Evan Wang, Hugh Zhang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen H., Sean Hendryx, Summer Yue 👉 Read the full paper here: arxiv.org/abs/2409.03733

179

2 Comments

In a world where large langue models, Deep learning, Neural Networks, Cloud based Data Platforms & Data Mesh take the headlines, Let’s not forget the humble Analyst. I love a good old fashioned ‘Analyst’ role. In many cases they work on projects where the rubber really meets the road in organisations. And deep sector experience is often required. Really good Analysts are few & far between. And AI won’t put them out of a job anytime soon. #analyst #data

24

4 Comments

Snowflake blog on How the Arctic LLM was trained from Scratch is amazing and even better than the Model :-) - Arctic uses a Dense-MoE hybrid architecture that combines a dense transformer of ~10 billion parameters with residual sparse MoE FFN modules. "The MoE module has 128 experts, each with about 3.66 billion parameters, associated with a top-2 gating function. In total the model consists of 17 billion active parameters and 480 billion total parameters" - Model states need a huge 7.5 TB memory and needs aggregate GPU memory over multiple devices. It uses ZeRO DP and EP parallelization, ZeRO Stage-2 across all GPUs for the dense param & used expert parallelism to distribute experts among E devices and ZeRO-Stage 2 to further scale to all the remaining GPUs. "if the total number of GPUs is N, and expert-parallelism degree is E, then the dense parameters are scaled to all N devices using ZeRO Stage 2 while the experts are distributed among E experts, and each expert is scaled to the rest of the N/E GPUs using ZeRO Stage-2" - Profiling was done using DeepSpeed. Along with forward-backward compute latencies & end-to-end training latency. Activation-Checkpointing is used to save activation memory by recomputing them in the backward pass. The performance Bottlenecks explaination is very detailed and covered many interesting details.

22

- SwiftKV is a novel model transformation and distillation procedure (by folks at Snowflake) designed to optimize large language model (LLM) inference. Very cool findings in the paper (as presented in the vLLM meetup on 11/13). - SingleInputKV: Prefills later layers' KV cache using earlier layer outputs, reducing computation on prompt tokens. The key point is that the first few layers of attention in LLMs are similar (nice find), allowing SwiftKV's SingleInputKV mechanism to skip computations in later layers during inference. By using a single earlier layer's output to compute the KV cache for subsequent layers, it significantly reduces both computation (by 50%) and memory usage (by 62.5%) without sacrificing model performance. - AcrossKV: Merges KV caches of neighboring layers to minimize memory usage and increase batch size for higher throughput. - For models like Llama-3.1-8B and 70B, SwiftKV reduces prefill compute by 50%, memory usage by 62.5%, and achieves up to 2× higher throughput and 60% lower time per token. https://lnkd.in/guhEBW5G

62

Checkpointing Large models in Production Systems? Asynchronous Checkpointing is quickly becoming a requirement for anyone serious about optimizing large model training - check out this blog post for a quick getting started guide, and see how we measured ~10-20x improvements to training latency using PyTorch's new Distributed Asynchronous Checkpointing API's in close collaboration with IBM.

109

I’ve put together a Python implementation of the Dunn Index and uploaded it to GitHub. If you’re into cluster analysis, you might find this project useful. 🔍 What Sets It Apart? Unlike many existing implementations, mine includes: - two cluster diameter computation strategies: the mean distance between all points inside a cluster and the traditional approach of finding the two farthest points inside the cluster. - two inter-cluster distance strategies, considering the two nearest or farthest points between clusters. 🔗 Feel free to check out the code, suggest improvements, or just move along: https://lnkd.in/dPhAbs7X This is an old project I had on a GitHub Gist that got too much attention, so I decided to move it to a proper repo. #DataScience #MachineLearning #ClusterAnalysis #Python #GitHub #DunnIndex

32

1 Comment

Read on to see how Faire is improving search through fine-tuned LLMs!

7

🚀 New Video Alert: Data Engineer Aces Meta Mock Interview (ft. Jonathan Exume) 🎥 I'm thrilled to share my latest YouTube video where Jonathan Exume asks me challenging data engineering questions in this Meta mock interview. Whether you're preparing for a tech interview or simply curious about the process, this video is packed with valuable insights and practical tips. 🔍 What You'll Learn: Key strategies for acing a data engineer interview at top tech companies Real-world problem-solving approaches Expert advice on interview preparation 📈 Don't miss out on this opportunity to elevate your interview skills and learn from the best in the field! 👉 Watch the video now! #dataengineer #mockinterview #Meta #JonathanExume #interviewtips #techinterview #careeradvice #engineeringjobs #techcareers #jobinterview #interviewprep #techindustry #dataengineering #careergoals #successstory

34

8 Comments