I keep reading that “AI progress is slowing down”. But it was 6 years from the paper “Attention is all you need” until ChatGPT was released. That’s 6 years of incremental updates to get to an AI model that regular consumers would find useful. ChatGPT is now 2 years old. Maybe it only seems like incremental improvements in that time. But I’ve seen plenty of use cases where the models finally get good enough to be useful. Not only that but we’re getting better at using them. “But aren’t we running out of training data?” No. No we are not. The big players in AI have all sorts of ways of getting more data. That’s not the biggest problem right now. The biggest problem is that additional data is only giving incremental gains. But everyone knows that. It’s not some secret. New papers come out every week about how to get more out of these models. It also seems to take more researchers, more engineers, and more compute to make each incremental update. That’s because everyone is focused on one paradigm and we keep finding ways to squeeze more juice out of it. Paradigm shifts happen on slower time scales while incrementalism gets less and less effective. But incrementalism is how we discover paradigm shifts. “Attention is all you need” was an insight that came from trying to squeeze more performance out of the last paradigm.
Tarik Dzekman’s Post
More Relevant Posts
-
🚀 Boosting Efficiency with Formula Optimization! 🚀 In my current role, I faced a challenge with our Annual Performance Review (APR) process. The formula I initially wrote was causing significant slowdowns: =IFERROR(XLOOKUP($A3 & TEXT($J$1, "mm-dd-yyyy"), DUMP!$D:$D & TEXT(DUMP!$AD:$AD, "mm-dd-yyyy"), DUMP!$AE:$AE), 0) This formula worked, but it was making my system incredibly slow due to its complexity and the large dataset it was handling. Determined to find a better solution, I sought assistance and discovered a more efficient approach: =ARRAY_CONSTRAIN(ARRAYFORMULA(IFERROR(INDEX(DUMP!$AE:$AE, MATCH($A3 & TEXT($J$1, "mm-dd-yyyy"), DUMP!$D:$D & TEXT(DUMP!$AD:$AD, "mm-dd-yyyy"), 0)),0)), 1, 1) This new formula, suggested with the help of ChatGPT, dramatically improved the processing speed of our APR system! It’s a great example of how leveraging AI and innovative thinking can lead to substantial performance enhancements. 💡 Key Takeaway: Never stop exploring new solutions and tools. Sometimes, a little optimization can make a huge difference in productivity. #Efficiency #DataProcessing #ExcelTips #AI #Productivity #ContinuousImprovement #Innovation #PerformanceReview
To view or add a comment, sign in
-
Muse for today. I've been contemplating sharing my thoughts on the two rules of AI, considering ChatGPT as a large language model trained on web data. The number one rule is that any AI tool should save time. Sometimes, these AI tools give biased information, but despite their limited knowledge of the world, they make you productive and increase your value. Also, AI tools should not add unnecessary complexity. As a senior engineer, you know what you want before AI writes it, and you can detect when it is giving you bad guidance. That is one differentiating factor between senior and junior engineers. Companies value these sets of people any day as they prevent complexities of a system, business, or program. What are your thoughts on this?
To view or add a comment, sign in
-
I recently came across Paul Graham's post on 'Delve' in a pitch deck and how that is a giveaway that AI was used, which got me thinking about our perceptions of AI tools like ChatGPT. While I'm jesting (should I rather use "joking" to avoid AI flag? 😄 ) that I've never used "delve" in my posts to avoid suspicions of AI assistance, this raises serious questions about our reliance on technology. For instance, when calculators were first introduced, were users viewed as less skilled in math? It's worth pondering whether using AI detracts from human intelligence, or if it's simply a new tool in our toolbox. Consider a scenario where two people are solving a complex math problem: one uses a calculator to score 100%, and the other scores 20% without. Who is more intelligent? Should our definition of intelligence evolve to include the use of technology? Moreover, a graphic designer friend mentioned that his clients value his work less when they learn that AI played a role. Does knowing that AI was involved in creating something you love diminish its value to you? These are ongoing questions with no definitive answers, but they are crucial as we navigate the increasing integration of AI in our lives. P.S.: No "delve" allowed in the comments! 😉 What are your thoughts on the use of AI in professional settings?
To view or add a comment, sign in
-
Did you know, that the new marketplace for prompts is here? PromptBase.com is a platform where users can search over 100,000 quality AI prompts created by top prompt engineers. #Creativity #Inspiration #PromptEngineer #OMG #AI
To view or add a comment, sign in
-
AI prompting trick: When learning or relearning a concept, ask ChatGPT to explain the same concept six different ways. As an example: "Please explain [concept here]. Explain it to a 6-year old, then a 12-year old, then an 18-year old, then a 24-year old, then a business professional, and then a data scientist. Also provide 2 URLs to two helpful articles that explain the concept. One should be a short article, and the other should go more in-depth." https://lnkd.in/eRzTtkaX
To view or add a comment, sign in
-
If your AI stack is Chat-GPT + RAG, this one's for you! 👋🏻 You don't own your model. As you grow, your IP is your model and you don't own it. It's like ordering takeout and claiming it as your own food to serve to guests - it doesn't scale. 🥡 🤖 Hallucinations will be a problem. There is no substitute for knowledge injection in your model, and that won't happen with this setup. 💸 Your costs will eventually exceed your value. RAG is expensive. Is RAG needed in all use cases? No. It's there to ensure your most recent data is remembered, but what about historical data? Catastrophic forgetting is real. 📊 Own your model, use your data, and drive the right mix of knowledge injection and remembrance with continual pretraining for your use case. 🔧 There are alternatives. Arcee.ai 🤙🏻 #AI #MachineLearning #TechInnovation #DataManagement #AIModels
To view or add a comment, sign in
-
I recently asked my friend ChatGPT what trends it was seeing for using AI in recognition. The top 5 trends listed were: 1. Personalized Recognition 2. Real-time Feedback 3. Data Analytics for Insights 4. Integration with HR Systems 5. Predictive Analytics While we are currently exploring ways to integrate AI in our flow of work with more robust system integrations, predictive analytics and data insights, we haven’t yet made the proverbial leap into leveraging it holistically in our recognition platforms for writing and creating more personalized recognition experiences. Curious, how are other organizations leveraging it today? What are your thoughts on the use of AI in recognition?
To view or add a comment, sign in
-
Do you use GenAI in your job? As of Feb '24, Pew Research said that only 23% of Americans had used ChatGPT, and 20% of those who work had used it for tasks at work. (source: https://lnkd.in/gMaJVndJ) Meaning 80% of people have never used it at all for their jobs. Certainly there's a lot of jobs where there would be no practical use for ChatGPT, but it's also clear that the hype doesn't match the real-world usage. I personally use it all the time for code generation, debugging, etc. But I don't hardly ever use it for actual analysis. Perhaps I should be? I'm looking forward to finding out more in our latest CBUSDAW talk about AI next week were Nicholas Woo from AlignAI presents on real-world use cases for AI that can actually help our day to day. Join us next Wednesday (May 8th) at Rev1! Free registration at https://lnkd.in/gFbKPh2E
Columbus DAW - May 2024 - Columbus Data & Analytics Wednesdays
https://cbusdaw.com
To view or add a comment, sign in
-
🚀 Exciting times in AI! Ever heard of Chain of Thought (CoT) prompting? It's like giving AI a step-by-step playbook to solve complex problems, much like planning your dream vacation from start to finish. 🌍✈️ For simpler tasks, like sensing if a review is positive or negative, regular AI does the trick. But for anything that needs a sequence of decisions— think of it as setting up dominos to fall just right — that's where CoT shines. Why does this matter for us in business? It means a well-thought serial CoT can handle intricate tasks in your organization with ease. What are the tasks that come straight to mind that you think CoT can help with? Let's discuss!🌟 #AI #Innovation #BusinessTransformation #ChainOfThought
Why CoT Works: Iterative Computation for Serial Problems Ever wondered when to use fancy Chain of Thought (CoT) prompting, and when regular ChatGPT will do? Here's the key: Serial problems NEED CoT. Turns out, without CoT, we can only solve problems that have fast, parallel solutions. Let's get practical: - Travel Planning: Imagine an AI travel agent. CoT lets the AI figure out your destination, considering your preferences and budget. Then, it factors in dates (weather and your schedule), books your stay, and suggests activities based on the destination's attractions and your interests – each step builds on the last, ensuring a perfect trip. This is a classic example of a serial problem – each step depends on the outcome of the previous one, making it impossible to ask five people to do these steps in parallel. - Sentiment Analysis: This one ISN'T serial. This is a parallel task! A regular transformer can analyze a review for positive/negative/neutral vibes all at once. Imagine it like asking multiple people to independently assess the sentiment of the text simultaneously. They don't need to know the results of each other's analysis or follow any specific order. The Tech Deep Dive (Simplified!) CoT basically lets transformers tackle way more complex problems. Without CoT, they're limited to AC0 problems (solvable by constant-depth circuits). But add CoT, and now we're talking problems solvable by log-depth circuits (NC1 complete problems). Want proof? Check out the paper "Chain of Thought Empowers Transformers to Solve Inherently Serial Problems" (https://lnkd.in/giCaSGA5). Author: Zhiyuan Li, Hong Liu, Denny Zhou , and Tengyu Ma The Takeaway: Next time you've got a problem for your transformer, ask: Is this serial (step-by-step), or can it be done in parallel? That'll tell you if you need the power of CoT. #ChainOfThought #Transformers #CoT #LLM #AI
To view or add a comment, sign in