API Testing vs Monitoring: What's The Difference?

We’ve already outlined why API performance matters and what aspects of APIs to test, but what is the difference between API testing and monitoring?

As with most things, context matters. The use cases for testing and monitoring are different because the objectives are different. The ultimate goal is to verify that your APIs are functioning properly, but staging environments vary significantly from production environments.

Why API Testing

Identifying a performance or functional problem in production is less than ideal. Testing should be integrated throughout the dev process because testing for code defects in pre-production helps minimize risk, thereby saving you time and money.

In pre-production, you want more flexibility because the use case of what you are trying to accomplish is different than it is in production. For instance, you don’t need to know if an API is stable and reliable. Rather, you need to know that you didn’t create a defect or slow down a workflow with a new update. Testing seeks to validate that performance and functionality haven’t been impacted by updates.

With API testing you want to:

• Test key junctions and milestones.
  When you make a change, check in new code, or deploy to staging, it’s important to ensure that you don’t deploy bugs.
• Know immediately if something is broken.
  You don’t want to have to wait for the next monitoring test run to know if there is an error. With testing, you can simply push a button, run the test, and determine if you are good to go. Similarly, in dev or QA environments, you want to know if a test passes or fails, and the context about the nature of a failure.

The value of testing lies in identifying the deltas that occur once changes to the environment are introduced. As a result, tests should be run around these changes, but don’t need to be ongoing.

Why API Monitoring

Testing in pre-production, however, doesn’t negate the need to monitor production or live API performance on an ongoing basis. Performance monitoring verifies that APIs are functional, secure, and fast. This is especially critical to both identify and troubleshoot outages and errors in production environments that are often erratic.

API monitoring helps you do a few important things such as:

• Ensure performance.
  Put simply, API Monitoring ensures the continued performance and availability of APIs on which you rely. And, if anything goes awry, it alerts you immediately so you can respond. Splunk Synthetic Monitoring’s API Checks pinpoint the root cause of failure, even in service-oriented environments, so you can address any defects quickly and accurately.
• Improve over time.
  It is one thing to know that your APIs are performing properly, but it is entirely different to know if their performance is improving or degrading over time. With API monitoring, you can set alerting thresholds that reflect your performance goals.  

API Monitoring can help pinpoint failure

While API monitoring solutions could be used for testing, they aren’t fit for purpose. Pre-production environments are more controlled than production environments and failures likely won’t result in lost income. And, aspects of these products that are beneficial in production monitoring can actually be a hindrance in pre-production.

Testing and Monitoring Together

Both performance testing and monitoring are essential to approaching performance in a holistic way. You likely already leverage some form of functional testing, but if an aspect of your digital user experience–including your APIs–isn’t fast then we argue that they aren’t functional. For instance, if a critical API flow goes from taking half a second to complete to taking ten seconds, the API isn’t delivering a good user experience, even if it is technically functional. Finding that out in pre-production is ideal, so it can be addressed.

API testing and monitoring go hand-in-hand and comprise a comprehensive approach to delivering the best possible user experience. Performance testing seeks to increase the speed, transparency, and confidence by which you deliver user experiences. Monitoring helps ensure that those user experiences stay fast and reliable.