The hidden complexity of A/B testing. 🧪 By Ruby Stoddart “While model performance metrics are valuable, nothing speaks louder than a successful A/B test.” But real-world testing isn't always straightforward. To learn how to… → Design truly independent tests → Use cluster-based randomization → Implement sequential testing correctly …see our complete guide 👉 https://bit.ly/4dkROmF #DataScience
Datasparq’s Post
More Relevant Posts
-
Is my unit test a good one? Probably every developer asked themselves that question one day in their lives. So here’s a checklist to determine that: - Is the test clearly named? I usually like the convention: function_being_tested_scenario_expected_behaviour - Are the inputs simple enough so quickly understand what’s being tested? If not, simplify them to the bear minimum to execute and produce what the test need. - Are the assertions clear and directly linked to the expected behaviour? - Does the test runs fast? - Do you mock everything else apart from the function being tested? - Is your test reproducible (same inputs return the same outputs)? Using this checklist will ensure you write very good tests. What else would you add?
To view or add a comment, sign in
-
Hello LinkedIn peers, I’m excited to share my latest article on Medium where I dive into the world of software testing. Far beyond just finding bugs, software testing is a comprehensive process that ensures quality, functionality, performance, and user satisfaction. 🔗 Read the full article here: [https://lnkd.in/gTwiH768] Neural Web
To view or add a comment, sign in
-
This is always contentious topic - to test or not to test, what about what to test and what is the actual ROI of testing? Interesting thought in this article https://lnkd.in/g6FNkJ9A
To view or add a comment, sign in
-
The Coco team is happy to announce the release of Coco 7.2.1! Based on our testing and customer feedback, this maintenance release includes over 20 bug fixes and several improvements: 🟣 Added Chinese translation for the Coverage Browser. 🟣 Improved command line analysis performance for incoherent instrumentation. 🟣 Function Profiler data now available in HTML reports. 🟣 CSV reports include detailed condition coverage info. 🟣 Uninstrumented source files now appear in reports. 🟣 New --cs-record-call command line argument to allow recording coverage for functions that might break the execution flow. Learn more and get started here: https://hubs.li/Q02ZGLWG0 #SoftwareQualityAssurance #SQA #CodeCoverage #SoftwareTesting #CodeAnalysis
To view or add a comment, sign in
-
Today I identified a classic automated test failure for my team. Friday for us is a no-meeting-day and is also often used for the "10% Learn and Be Curious" budget. I noticed yesterday that there was an automated test case that used to pass consistently and started failing recently. So for my "Learn and Be Curious" today, I decided to put on my software testing hat and poke around our codebase. 1. Was there a change to the code recently? No. 2. Was there a change to the dependencies recently? Doesn't seem like it. 3. What does the error code really mean? A failure in completing the action. 4. Can the action be done manually? Nope (see screen capture) ----> Why can't it be done manually? Automated tests hit the upper boundary of what's allowed. Ah. Classic. So I identified (and executed) short-term workarounds to get the automated tests into a good working state, created a TODO for the team to update the automated test case to account for the boundary condition and update the test cleanup step. Not bad for a 15-minute deep (?) dive. Now to learn and be curious about more things... :)
To view or add a comment, sign in
-
Ideas for testing improvement are everywhere - write some tooling, test in production, shift testing left, "sprinkle some AI on it", and so on. But how will we know if they work? Over the years, we've come up with a few ways to measure test improvement. These are not bug counts, but instead measurements of system performance. Before trying any change, we can ask "what should this change do to these measures?" If the answer is "it will make them worse", we can pause and think. Likewise, we can check before and after and see if the measures improve. Of course, the real outcome we want is "less complaining customers", "more free time to add new features", "less arguing in circles" and so on. Specifically, we are talking about cycle time, lead time, and touch time, along with coverage measures. Want to learn how to do them in twenty minutes? I'll give you some ideas at the SDTimes Improve testing series on March 20 at 11AM Eastern. The event is at no cost, remote, and will run about a half hour.
You’ve heard it said that without a location, a map doesn’t help. But even if you have a roadmap to improvement, that doesn’t tell you where you are, nor measure your progress. Classic techniques such as bug counts, are sophomoric at best or even irrelevant in some modern environments. Attend this session led by Matt Heusser at Improve: Testing next week to learn new ideas for measuring release cadence. This is one of many sessions designed to help you improve your testing process. Register for Improve: Testing today! We hope to see you on 3/20. https://lnkd.in/dtVvxZGb
Improve: Testing 2024
assets.sdtimes.com
To view or add a comment, sign in
-
Automated tests should be readable and serve as the specification of the behaviors present in a codebase. One pattern I’ve come across while looking at unit tests in the wild is the Object Mother Pattern (seen on the left). It sounds like a good idea at first…let’s bury away all of this setup code to make the test less cluttered. However, important pieces of the puzzle get buried away as well. In xUnit Test Patterns by Gerard Meszaros, the author talks about the testing anti-pattern of the Mystery Guest. The mystery guest hides away important details that leave the reader of the test confused as to how the assertion relates to the obscure setup. One easy way to resolve this test smell is to use the Test Data Builders pattern instead (seen on the right). This way you can still hide away irrelevant and easily duplicated details in a test fixture, while also presenting the critical inputs that make your test expressive.
To view or add a comment, sign in
-
Our colleague Sebastiaan Koot has written a blog about testing: "Beyond the illusion of test coverage". "Have you ever spent hours on creating unit tests in order to reach a unit test coverage above 80%, only to discover in production that your software still harbours critical bugs?" "Discover the profound shift in unit testing philosophy that’s turning heads in the front-end and back-end communities." Read the full blog post here: https://lnkd.in/eXdznSuj #QualityAssurance #Testing #Blog
To view or add a comment, sign in
-
[A/B Testing fundamental steps] ✨ An A/B test is an experiment designed to test which version is better. 1️⃣ Specify the goal and designs/experiences 2️⃣ Randomly sample users for enrollment 3️⃣ Randomly assign users to: control variant (current state) treatment/test variant(s): new design 4️⃣ Log user actions and compute metrics 5️⃣ Test for statistically significant differences
To view or add a comment, sign in
2,757 followers