AI is not good software. It is pretty good people. Paradoxical ? We want our software to yield the same outcomes every time. So, we ensure that software systems are reasonably reliable and predictable. Large Language Models are neither of those things, and will absolutely do different things every time. They have a tendency to forget their own abilities, to solve the same problem in different ways, and to hallucinate incorrect answers. There are ways of making results more predictable, by turning down the level of randomness and picking a known “seed” to start, but then you get answers so boring that they are almost useless. Reliability and repeatability will improve, but it is currently very low, which can result in some interesting interactions. We also want to know what our software does, and how it does it, and why it does it. We don’t know any of these things about LLMs. They are also literally inexplicable. When you ask it why it did something, it is making up an answer, not truly reflecting on its own “thoughts.” There is no good way of understanding their decision-making, though, again, researchers are working on it. Finally, we should know how to operate a piece of software. Software projects are often highly documented, and come with training programs and tutorials to explain how people should use it. But there is no operating manual for LLMs, you can’t go to the world’s top consultancies and ask them how to best use LLMs in your organization - no on has any rulebook, we are all learning by experimenting. So the software analogy is a bad one. You should actually treat AI as people since that is, pragmatically, the most effective way to use the AIs available to us today. What tasks are AI best at? Intensely human ones. They do a good job with writing, with analysis, with coding, and with chatting. They make impressive marketers and consultants. They can improve productivity on writing tasks by over 30% and programming tasks by over 50%, by acting as partners to which we outsource the worst work. But they are bad a typical machine tasks like repeating a process consistently and doing math without a calculator . So give it “human” work and it may be able to succeed, give it machine work and you will be frustrated. And, of course, the AI still lies, makes mistakes, and “hallucinates” answers. But, again, so do humans. I would never expect to send out an intern’s work without checking it over, or at least without having worked with the other person enough to understand that their work did not need checking. In the same way, an AI may not be error free, but can save you lots of work by providing a first pass at an annoying task. We need to decide what tasks we are willing to delegate with oversight, what we want to automate completely, and what tasks we should preserve for humans alone. #Genai #LLMs https://lnkd.in/gzEtUqvr
Abhay Johorey’s Post
More Relevant Posts
-
A few people have remarked to me recently that they think software developers are going to be replaced by AI. You may have seen Devin, an AI developer, that tends to play into this idea. However, I think we are a very, very long way from developers being automated out of a job. I just watched a very detailed debunking of Devin which prompted me to write some thoughts. Firstly, the hard part of software development is figuring out what is actually needed. Often customers and managers ask for things that make no sense. A back-and-forward game, figuring out what it is they actually need, is the most valuable part of the development process. Just building what someone asks for is almost always a mistake. They may have badly formulated their request, they may be unaware of some critical details or they may just not realise there are different or better ways to solve their problem. As someone who uses AI in software development I can state that it’s a huge productivity benefit. But it also has big limitations. 50% of the time it’ll take me down the wrong path and it’s my job to realise that and ignore the AI when it happens. It is a useful tool, but it’s nowhere near ready to build entire systems on its own. Things like Devin suggest that AI is further ahead than I personally think it is. The detailed debunking video I just watched shows that Devin is fixing bugs (impressive) but also that the bugs it’s fixing were ones it introduced into a very simple task (not so impressive). This doesn’t surprise me. I see this pattern a lot - the AI does something stupid, I tell it I got an error and it says “oh, you are right - the code I gave you is wrong”. It’s smart enough to suggest a solution to a problem but dumb enough to, half the time, introduce more problems as it fixes the original bug. A human can recognise this death spiral and ignore the AI. But an AI on its own will just keep fixing and introducing more problems. What AI does today is help lessen the need to remember syntax, to automate some things like writing simple utility functions, help auto suggest variable names. It removes the drudgery elements of writing code and it improves productivity. But the idea that someone will be able to just say to the AI “build me a product catalog website” is farcical. For a long time yet it is going to be a tool to help developers. It will get better and help them more over time, but it’s unlikely to replace the developer any time soon IMHO. And if it does replace developers, it’ll probably also replace those making the requests as well! But I think that’s a long way off. So let’s roll back the hype. AI is a useful tool that can be a big productivity benefit. But it’s not about to automate developers out of existance any time soon. And let’s not forget that “yes I know that’s what you asked for, but have you considered…” is often the most valuable conversation a developer ever has.
To view or add a comment, sign in
-
Performing mundane tasks and reading through uncommented, messy code can be quite frustrating for developers 😖. With the advent of AI, why not utilise it to enhance efficiency? Explore the pros and cons of incorporating AI in development; click the link to learn how to protect your data🛡️ while using AI tools🚀.
To view or add a comment, sign in
-
How to Stay Relevant as a Software Developer We’re Drowning in AI… Now What? For years and years, we joked about robots taking our jobs one day. “It’s coming,” some warned. Fast-forward to a few years ago when AI exploded seemingly overnight. “It’s here,” those same people said. Enter budget cuts, mass lay-offs, and software developers the world over echoing a similar sentiment: “What now?” Here’s the good news: Brands are finding that swapping humans for AI might’ve been premature. Here’s the tricky part: Software developers still must choose to evolve or drown. How to Make Yourself Invaluable as a Software Developer Some of us have learned, perhaps the hard way, that while AI is incredible and will only get better, there are some things that will simply turn out better if a human being manages them. However, we’d also be foolish to deny that things have changed tremendously. I work with developers every single day, and I wanted to better understand what this evolution has meant for them. What can software engineers do to make themselves irreplaceable in the age of AI? I did my homework and also spoke with Luce Carter, Developer Advocate at MongoDB and Microsoft MVP, who first told me, “AI is not going away, especially with tools like Copilot. One of the best things you can do is learn a skill called prompt engineering, knowing the best and most effective way to ask the AI for what you want that will produce the most useful results.” Here’s what else I’ve learned. 1. Solve a Problem We’ve seen that AI is sometimes great for taking over monotonous, manual tasks — for instance, sifting through or generating code. Image by Luca Bravo But you, the software developer, still have an upper hand: You see the bigger picture. The end goal in mind. The target you’re aiming for. This is a skill unique to you. There are countless examples of how this can work. One way I’ve seen my teammates elevate the application development process using AI is via vector search, which understands the meaning and context of unstructured data, which it then transforms into numbers. How does this solve a problem? Well, it allows them (and/or their users) to more efficiently query data. This is a great example of humans and AI working together in harmony for the greater good. Remember, AI technology might sound cool but is a total moot point if we don’t use it to solve a problem and make people’s lives easier. If you’re not sure if your work is solving a problem, take a step back, look at what you’re doing, and ask yourself, “So what?” If you can’t come up with a clear answer, there’s more work to be done. 2. Make Way for Strategy Alright, you’re using AI, in some capacity, to solve a problem. You’ve found ways to leverage the technology to save you time. Now, you’re going to use that newly freed up time for… what, exactly? Scrolling on TikTok! Amazon shopping! Getting a snack! No, no, and yes because snacks are life, but then get back...
How to Stay Relevant as a Software Developer We’re Drowning in AI… Now What? For years and years, we joked about robots taking our jobs one day. “It’s coming,” some warned. Fast-forward to a few years ago when AI exploded seemingly overnight. “It’s here,” those same people said. Enter budget cuts, mass lay-offs, and software developers the world over echoing a similar sentiment:...
To view or add a comment, sign in
-
Is AI over-hyped? I think there are two different ways to answer that question: What *can’t* the AI do for us? or What *can* the AI do for us? Let’s look at an example to see how these different questions can yield very different perceptions of how valuable large language models (LLMs) actually are. Software development is one area where AI appears to have a lot of traction right now. If we ask what AI *cannot do* in software development, we might see that AI cannot: - Invent algorithms that were not in the AI’s training data - Design innovative/creative user experiences without lots of human help - Solve problems the AI hasn’t been trained on - And so on… You’re basically left with the conclusion that AI is over-hyped and not having a big impact on software development. If, however, we ask what AI *can do* when it comes to development, we see that AI can: - Suggest routine code completions as developers type - Act like a tutor to help us understand new technologies / languages / frameworks - Write detailed comments and documentation - And so on… When we ask the question that way (what can AI do for us?), there’s really only one conclusion: that AI is actually having a positive impact on how we write code. That shows up in developer productivity data as well: GitHub claims that 88% of developers who used their Copilot product felt they were more productive and completed tasks 55% faster than those that didn’t. 60–75% of users reported they feel more fulfilled with their job, feel less frustrated when coding, and are able to focus on more satisfying work when using Copilot. Software development is just one example. I’m sure we can find others in areas like customer service or sales. So, it really comes down to how we ask the question: can or cannot? I’d encourage us all to ask “can” questions, rather than “cannot” questions.
To view or add a comment, sign in
-
AI Isn't Taking Jobs—It's Supercharging Software Development! I just published my latest Forbes article, in which I challenge the narrative that AI is replacing developers. Instead, AI is amplifying our capabilities and accelerating innovation in the software industry. 👉 Read it here: AI Is Not Stealing Jobs, It Is Supercharging Software Development Let's cut through the noise. #AI #SoftwareDevelopment #Innovation #ForbesTechCouncil #FutureOfWork #Forbes #Sonatafy
A lot of the hype today centers on the idea of developers using AI "to write their code." This is not practical, at least not for complex tasks. Read more: https://hubs.li/Q02ZjmVk0 Post written by Steve Taplin, Forbes Councils Member.
Council Post: AI Is Not Stealing Jobs—It Is Supercharging Software Development
social-www.forbes.com
To view or add a comment, sign in
-
A lot of the hype today centers on the idea of developers using AI "to write their code." This is not practical, at least not for complex tasks. Read more: https://hubs.li/Q02ZjmVk0 Post written by Steve Taplin, Forbes Councils Member.
Council Post: AI Is Not Stealing Jobs—It Is Supercharging Software Development
social-www.forbes.com
To view or add a comment, sign in
-
The emergence of artificial intelligence (AI) continues to transform the technological landscape. Its application in several facets of software development continues to grow. One of the areas of software development where the adoption of AI can advance is software testing. Software testing is crucial in ensuring the release of software products that meet both compliance standards and user demands for quality. However, with many permutations surrounding the use of artificial intelligence, we’ll dive deep into uncovering what AI is in software testing. How does AI in the context of software test automation differ from its broader definition? What do we mean when we talk about AI and its sister term, machine learning? What are the benefits of using AI and machine learning to advance state-of-the-art API testing? Let’s find out. What Is AI & How Is It Changing the Dynamics of Software Testing? Artificial intelligence is one of the most overloaded buzzwords in the digital marketplace. “AI” conjures up images of things like all-powerful supercomputers hell bent on human destruction, voice-control assistance in the way of Alexa or Siri, computer chess opponents, and self-driving cars. Wikipedia defines AI research as “…the study of “intelligent agents”: any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals.” But that’s a little too abstract. I like to think of AI as the ability of a computer program or machine to think (reason for itself) and learn (collect data and modify future behavior in a beneficial way). It’s in this definition that we start to see something more meaningful in the context of what AI means for software development tools and technology. More Software Releases Means More Software Testing As the number of developers worldwide continues to surge, more software releases are expected to hit the software market. A recent report by Statista corroborates this expectation with a projection that suggests that the global developer population is expected to increase from 24.5 million in 2020 to 28.7 million people by 2024. This portends that we’ll continue to see more software launches in the coming years. With this expected growth in the number of software releases comes the need to automate software testing. Software testing is the process of subjecting a software infrastructure to a series of functional and nonfunctional testing scenarios. It’s a process of evaluating software to ensure that it can do what it’s designed to do efficiently. When teams test software, they can discover and resolve runtime defects, scalability issues, security vulnerabilities, and more. The software testing process is usually rigorous, hence the need for automation. However, for software automation to be super efficient and seamless, there is a need to incorporate AI.
To view or add a comment, sign in
-
In this Part 2 analysis of the AI software testing market, Rafael E. Santos explores various AI/ML approaches for automated test design, such as model authoring, user journey monitoring, and NLP requirements ingestion, highlighting their limitations and challenges. Rafael notes that these methods often rely on incomplete or outdated models. Some methods are incremental improvements that fail to address fundamental testing issues. Is there a path forward? Can the software testing industry successfully deal with the enormous challenge of significantly improving software testing through AI? You'll have to catch Part 3 to find out! #artificialintelligence #softwaretesting #qualityassurance #qa #sdlc #softwaredevelopment https://lnkd.in/eAh6B6XC
The AI Software Testing Market, Part II
testaify.com
To view or add a comment, sign in
-
The second part of my blog post series is on the AI/ML testing market, specifically about trying to automate test design and why all the approaches in the marketplace do not meet the need. Enjoy!
In this Part 2 analysis of the AI software testing market, Rafael E. Santos explores various AI/ML approaches for automated test design, such as model authoring, user journey monitoring, and NLP requirements ingestion, highlighting their limitations and challenges. Rafael notes that these methods often rely on incomplete or outdated models. Some methods are incremental improvements that fail to address fundamental testing issues. Is there a path forward? Can the software testing industry successfully deal with the enormous challenge of significantly improving software testing through AI? You'll have to catch Part 3 to find out! #artificialintelligence #softwaretesting #qualityassurance #qa #sdlc #softwaredevelopment https://lnkd.in/eAh6B6XC
The AI Software Testing Market, Part II
testaify.com
To view or add a comment, sign in
-
Updates on #AI : Google Leverages AI to Automate Code Generation, Full article link 👇🏻👇🏻 https://lnkd.in/dAt6FYzt #artificialintelligence #machinelearning #ML
Google Leverages AI to Automate Code Generation
https://www.aimlmag.com
To view or add a comment, sign in
Innovative Marketing and Creative Professional
7mo👍👍👍 Your point about deciding which tasks to delegate, automate, or reserve for humans is crucial. As we navigate this evolving landscape, thoughtful integration of AI into our workflows can lead to impressive outcomes.