Workato Ipaas Architecture Deep Dive
Workato Ipaas Architecture Deep Dive
Workato Ipaas Architecture Deep Dive
Introduction 3
Conclusion 22
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
Introduction
Integration requirements, and associated complexity, have evolved from the basic
need for connectivity: “how can I connect my on-prem CRM with my on-prem DB?” to
“how can I integrate the plethora of SaaS applications and in-house applications
while automating all the steps associated with any given business process that
crosses over these systems in a consistent, reliable, maintainable, and scalable
way?”
To address these new and ongoing challenges, and through the convergence of
existing integration technologies and the cloud, the Integration
Platform-as-a-Service (iPaaS) market emerged. However, “how” each iPaaS vendor
decided to adopt the cloud has limited the potential value of their platforms.
Fully leveraging the cloud and understanding its possibilities is usually what the
industry calls “cloud-native”. In this paper, we’ll discuss why cloud-native is critical
for an iPaaS platform.
But before digging into why cloud-native is the answer to commoditizing the
ever-growing integration complexity, let's level set on what cloud-native means.
workato.com | 2
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
The best way to understand “cloud-native” is by comparing it with its close cousin:
"cloud-optimized".
As the cloud became mainstream, organizations from every industry decided to look
at the cloud from one of two perspectives: either as a cost-saving mechanism or as
an opportunity for disruption and transformation. This is both from the perspective of
how they leverage the cloud for internal IT operations and also how they leverage the
cloud in shaping the product or service they deliver to customers.
In the former (and most common) group, organizations are drawn to the cloud as a
way to save money on CapEx and operational costs. To achieve this goal, the focus is
on porting their existing on-prem loads to the cloud as close to their "as-is" state as
possible to minimize disruption and risk.
workato.com | 3
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
On the other hand, a smaller set of organizations saw the cloud as a way to reinvent
themselves and/or their respective markets by realizing the full potential of the cloud:
speed, scalability, resiliency, and agility. Netflix and Zoom are prime examples of
cloud-native products and their potential for market disruption.
workato.com | 4
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
The following diagram describes the different concerns that on-prem integration
platforms imposed on their users before the cloud emerged:
As the cloud became the new Northstar, a new integration space was created:
iPaaS. However, the cloud architectural paradigm employed as a guiding principle
defines the potential impact and benefits of such platforms for the customers
leveraging them.
As mentioned earlier, the iPaaS space is the result of continuous evolution that
emerged from the need for connectivity. The technological underpinnings of these
new platforms were directly affected by how each vendor decided to embrace the
cloud to “build their iPaaS alternative”.
workato.com | 5
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
At a high level, we can generally agree that every iPaaS has 3 main logical
components:
1. Management plane: Usually, the main UI access point to manage user access
controls, application deployment, runtime provisioning, and orchestration (not
the user interface for building integrations & automations)
2. Runtimes: Where the rubber meets the road. Execution engines for processing
the integration/automation job
3. Infrastructure: Underlying resources that support both the management
plane and runtime provisioning
Understanding these basic elements is key because most vendors have focused on
building their cloud management planes with microservices in mind, but with
complete disregard for leveraging microservices for the runtimes that manage the
heavy lifting processing of their platforms.
workato.com | 6
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
By following a cloud-optimized approach, the only concerns that get removed are:
● Server provisioning
● Networking provisioning
While these are great benefits over the on-prem architectures that came before, all
traditional limitations associated with integration platforms remain a customer
concern. Customers still need to worry about non-functional requirements such as
reliability, traceability, version upgrades, peak loads, high availability, etc., thus
hindering the real value that can be derived from a true cloud-native iPaaS.
workato.com | 7
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
For this purpose, we are going to look at an iPaaS platform through the lens of:
● Microservices-based architecture
● Containerization and container orchestration
Management plane
Building a management plane based on a microservices architecture is a
no-brainer. As with virtually any other modern web application, there are
clear-cutting concerns you would typically encapsulate in microservices, such as
security, logging, auditing, as well as UI components. Since microservices in this
context are pretty much aligned with most of the content out there on the web, I
won't delve into specifics. But when in doubt, you can always count on Martin Fowler's
definition of microservices.
Runtimes
Those who have used other integration platforms will know that an iPaaS runtime
usually executes a unit of work (UOW), such as a “flow” or a “process” end-to-end.
These UOWs are typically self-contained integration logic that includes information
about how to receive an event (understanding “event” as a broader concept that
can include synchronous API requests, async messages, etc.), what to do with it (e.g.
business logic) and where to send it to (e.g. target destinations):
workato.com | 8
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
But what if your runtimes followed a microservices architecture and your “units of
work” became distributed rather than monolithic?
workato.com | 9
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
workato.com | 10
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
On the other hand, you could have another set of purpose-specific runtimes that are
only focused on executing the business logic associated with each unit of work with a
different set of concerns, such as:
Infrastructure
While microservices define your solution architecture, your container and
container-orchestration strategy define your deployment architecture. Docker and
Kubernetes are arguably the industry standards when it comes to cloud-native
initiatives.
However, cloud-native is not just about running containers, but leveraging cloud
technologies to their maximum potential. At the end of the day, anything can run on
a container but using containers to change an architectural paradigm is what
promotes change and disruption.
workato.com | 11
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
Similar to the concepts of just-in-case (JIC) and just-in-time (JIT) from the
manufacturing world, the notion of pre-allocating servers or cores to a given
application can resemble the strategy of preemptively assigning resources in case
they are necessary (JIC) which potentially incur in waste and oversizing. In contrast,
dynamically allocating generic runtimes to any type of event from any type of
workato.com | 12
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
The benefits of avoiding server affinity don't stop there. The ability to have the
execution runtimes completely decoupled from the builder's intent (e.g. application)
enables zero downtime and zero impact upgrades.
This level of indirection empowers product teams with new levels of agility as they
can enhance and build new capabilities into the platform without impacting existing
applications.
workato.com | 13
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
Therefore, this unique architecture enables zero-ops as all the platform infrastructure
management overhead becomes a platform concern while letting end users focus
on delivering value.
Now that we understand what a cloud-native iPaaS should look like, let's explore how
Workato not only embraces this architectural paradigm to its fullest but also delivers
new levels of value and productivity.
workato.com | 14
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
Workato is a cloud-native iPaaS that in many ways behaves more like a "Serverless
Integration and Automation Platform" than a traditional iPaaS platform. This is a
direct result of Workato's platform architecture and how it fully embraces the
cloud-native paradigm.
The Workato Recipe (an example is provided in the image above) is the main
building block for integrations and automations. From a logical standpoint, it may be
seen as the equivalent of flows or processes in other platforms as it's where builders
will configure which events to listen to, what to do with them, and where to send
them after. But that's where the similarities end.
workato.com | 15
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
At first glance, it may feel like you are configuring a single unit of work that will be
executed as a whole just like any other runtime. However, the way we execute
recipes is completely different.
● Recipes are descriptors of the "builder's intent" and are decoupled from our
execution runtime
● Recipe logic is evaluated on-demand during execution by any available (idle)
server, thus removing the notion of a “deployed app” and closely aligning with
a serverless execution paradigm
● While recipes look like a single set of instructions, Workato has separate
auto-scalable container infrastructures dedicated to triggers that are
constantly listening to events
● Once these events are captured, they are transcribed (persisted) into our
highly distributed, and highly performant queuing infrastructure (without users
even realizing it)
● As events are loaded in our queues, we have yet another auto-scalable
container infrastructure dedicated to picking up these events and executing
the set of actions defined on each recipe
workato.com | 16
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
This is one example of how our runtime is broken into microservices that can be
independently maintained and scaled as needed to support our entire customer
base in a multi-tenant way. Along those lines, every other major component in our
platform follows this architecture:
workato.com | 17
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
With a traditional iPaaS, the time and effort to build key non-functional requirements
(e.g. in-sequence delivery and traceability of records) is arguably as much as
building the business logic itself. Time is the most precious resource in any business
so any opportunity to win back time to focus on what’s driving the most value for the
business is imperative.
In legacy platforms, conservatively speaking, about 50% of the final solution logic
(and time spent building it) is dedicated to implementing these repetitive,
cumbersome (but highly necessary) requirements to deliver true enterprise-grade
solutions (scalable, fault-tolerant, etc.).
workato.com | 18
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
● Guaranteed delivery
● Message persistency
● Error handling
● Transaction retries
● In-sequence delivery
● Deduplication
● Watermarking
These are a few of the capabilities that, in Workato, are no longer a developer’s
concern but a platform concern. We not only increase the overall quality of the
solutions being built but also increase developers’ productivity by allowing them to
focus on real business problems.
workato.com | 19
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
Yet another example of how microservices and cloud-native can make our
customers' lives better is when we are operating live in production and something
bad happens.
For the longest time, logging and observability have arguably stayed at the bottom
of the "cool tech pyramid" (if there was such a thing). Developers usually not only
dread adding logs across their solutions but also frequently lack proper standards or
practices to do so. Needless to say, by the time someone realizes the logging
capabilities are not enough, something would go wrong, and sadly there is no way to
know what happened (without spending excessive amounts of time and resources
that is...).
workato.com | 20
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
workato.com | 21
OVERCOME INTEGRATION COMPLEXITY WITH CLOUD NATIVE IPAAS
Conclusion
From . . . To . . .
To sum up, the integration and automation space has greatly evolved over the years,
and Workato’s platform is driving innovation in this market forward.
Hopefully, this article helped explain why, as opposed to monolithic runtimes of the
past, cloud-native can help commoditize and abstract integration complexity and
enable our customers to focus on delivering value.
workato.com | 22