How CRIO Consolidated 3 Test Frameworks into 1 Playwright Suite

Clinical trial software · Selenium + Karate + Playwright → unified Playwright TypeScript

At a Glance

Running multiple automation frameworks? Book a framework consolidation assessment.

Book a call

Meet CRIO

Clinical Research IO (CRIO)  builds digital clinical trial software used by sponsors, research sites, and contract research organizations worldwide. Their platform handles the workflows that clinical research runs on: study setup, data capture, source documentation, and the audit trails that come with all of it.

That last part matters for this story. In clinical research software, a defect isn't a UX complaint. It's a data integrity question, and data integrity is the product.

The Problem: Three Frameworks, Three Sets of Problems

CRIO's automation had grown the way most do; one tool at a time, each solving the problem in front of it. Selenium for UI. Karate when API testing became a priority. Playwright when the team started modernizing. Each choice made sense on its own. Together they created five compounding issues:

• Three frameworks, three maintenance bills. Every tool needed its own utilities, its own patterns, its own upkeep. Framework maintenance was consuming hours the team wanted to spend on coverage.
• Builds nobody trusted. Flaky and unreliable tests caused recurring failures. QA was spending 40+ hours a month investigating automation problems instead of testing features - the most expensive kind of busywork.
• Duplicated code everywhere. The assessment found roughly 35% duplicated code across the three repos - the same utilities written three times, in three styles, aging in three directions.
• Reporting scattered across systems. Results lived in multiple places, so root-cause analysis meant assembling the picture manually. Average bug resolution lagged by about three business days, and in a regulated environment, slow diagnosis is a risk, not just an annoyance.
• Onboarding took too long. New automation engineers had to learn three setups and three sets of conventions before they could contribute.

The pattern underneath all five: the team wasn't slow because they lacked skill. They were slow because they were maintaining an architecture instead of using one.

What ThinkSys Recommended and Why

Consolidation was the obvious move. Which framework to consolidate onto was the real decision, and it came down to three things:

1. Why Playwright, not Selenium. CRIO already had Playwright in the stack for partial UI coverage, so the team had a foothold. More importantly, Playwright's auto-waiting removes the entire class of explicit-wait code that was generating most of the flake — a problem consolidation alone wouldn't have fixed.
2. Why TypeScript. It let UI and API automation live in one language, one repo, one set of conventions. The alternative Java for Selenium's sake  would have kept the API and UI worlds separate, which was the problem we were solving.
3. Why Playwright could absorb Karate's job. This was the question that decided whether one framework was even possible. Playwright's APIRequestContext handles REST API testing natively: request construction, response validation, and mocking. Once we validated that CRIO's API test contracts could move over without losing assertion depth, the three-into-one plan was viable.

The rest followed: shared cross-platform utilities and stronger locator strategies to kill the duplication, centralized configuration and one reporting dashboard for visibility, parallel execution for faster CI feedback, and documented standards so the framework stayed consistent after we left.

How the Migration Ran Without Pausing Releases

This is the part that matters most for a regulated platform, and it's the question every engineering leader asks: what happens to our release train while you're rebuilding the thing that gates it?

The answer was to never let the old suite go dark.

1. We built bridging tooling so both suites ran together in CI. Legacy Selenium and Karate tests kept executing alongside the new Playwright suite throughout the migration. That meant existing coverage stayed live and gating releases while new coverage was being proven. At no point did CRIO ship against a partial safety net.
2. We migrated by risk, not by convenience. The assessment mapped which tests protected the highest-impact workflows and which failures recurred most - more than 170 recurring test failures per quarter were traced to identifiable patterns. Those areas moved first, so the earliest wins were also the most valuable ones.
3. We retired the old frameworks in phases, not in one cut. Selenium and Karate were decommissioned gradually, as the Playwright equivalents proved they caught the same failures. Phased rollout also gave the team time to give feedback while the framework was still malleable.
4. Validation before rollout, not after. After migration, the suite went through unit, integration, and regression testing against internal and production-like data sets. Stability was tracked through an integrated dashboard so failure patterns were visible and fixable before the framework became the release gate.

Once the framework was stable and validated, it went live through CRIO's CI/CD pipeline in a single deployment. 

Built for a Regulated Environment

Clinical trial software carries obligations most SaaS products don't. Consolidating the framework was also an opportunity to make quality evidence easier to produce, not harder.

• Traceable test evidence. Consolidating three reporting systems into one Allure dashboard meant test execution records stopped being scattered across tools. One place to answer "what was tested, when, with what result",  which is the shape of the question an auditor asks.
• Deterministic, reproducible runs. Reducing flaky failures by 90%+ isn't only a productivity win in a regulated context. Evidence that a test passed is worth less when the same test fails on rerun for no reason. Stability is what makes test results usable as quality records.
• Documented, reviewable standards. Code review practices, documentation, and consistent conventions were part of the deliverable - the difference between a framework that produces defensible evidence and one that produces output.

Results

Want your automation consolidated onto one framework? Book a 30-minute assessment - we'll map your frameworks and the migration path.

Book a call

What This Engagement Teaches About Framework Consolidation

Three lessons that generalize beyond CRIO:

1. Multiple frameworks cost more than the sum of their maintenance. The obvious cost is upkeep on each tool. The hidden cost is duplication, inconsistent conventions, scattered reporting, and onboarding that takes three times as long. Teams usually underestimate the second category, because it doesn't show up on any invoice.
2. Consolidation only pays if it also fixes the flake. Merging three unstable suites into one gets you a single unstable suite. The reason Playwright was the right target wasn't just consolidation, it was that auto-waiting eliminated the explicit-wait code generating most of the failures. Pick the destination that solves the underlying problem, not just the tooling count.
3. Never let the old suite go dark. Bridging tooling that runs legacy and new suites together in CI is what makes a migration safe. It costs some CI time for the duration. It buys you the ability to migrate a release-critical system without ever shipping against reduced coverage, which, on a regulated platform, isn't optional.

This is the same framework-first approach behind our Playwright automation practice, and the same discipline from our Selenium to Playwright migration work. If flaky tests are your starting problem rather than framework sprawl, start with how to reduce flaky tests.

Conclusion

CRIO didn't have a tooling problem. They had three tooling problems, plus the overhead of holding them together. Consolidating UI and API automation into one Playwright TypeScript framework cut primary scenario execution to under two minutes, removed the flaky failures that were burning 40+ hours a month, and made onboarding a matter of learning one system instead of three.

The framework that replaced them is the part that lasts: shared utilities, one reporting view, documented standards, and a foundation their team can extend without rebuilding.

Tell us how many frameworks you're maintaining - we'll show you what consolidating them looks like.

Book a call

Frequently Asked Questions

Why consolidate multiple test automation frameworks into one?

Running separate frameworks for UI and API testing multiplies maintenance, duplicates utilities, splits reporting, and slows onboarding -  engineers have to learn every stack before contributing. Consolidating onto one framework removes that overhead. In CRIO's case, moving from Selenium, Karate, and Playwright to a single Playwright TypeScript framework cut maintenance effort by roughly 60% and onboarding time by about the same.

Can Playwright replace Karate for API testing?

Yes. Playwright's APIRequestContext handles REST API testing natively: request construction, response validation, and mocking, which lets UI and API automation share one framework, one language, and one reporting layer. The migration question to validate first is whether your existing API assertions can move over without losing depth; once that's confirmed, consolidation is viable.

How do you migrate test automation without losing coverage?

Run both suites in parallel. Bridging tooling lets the legacy and new frameworks execute together in CI, so existing coverage keeps gating releases while the new suite proves itself. Retire the old framework in phases  by test category, as each Playwright equivalent demonstrates it catches the same failures,  rather than cutting over all at once.

Does framework consolidation help with flaky tests?

Only if the destination framework addresses the cause. Merging unstable suites produces one unstable suite. Playwright's auto-waiting eliminates the explicit-wait and sleep code that generates most Selenium-era flake, which is why CRIO's consolidation cut flaky failures by more than 90% rather than simply relocating them.

Is Playwright suitable for regulated or clinical software testing?

Yes, with the right architecture around it. What regulated environments need is traceable, reproducible test evidence: consistent execution records, deterministic runs, and documented standards. Consolidating reporting into a single dashboard and eliminating flaky failures makes test results usable as quality records, because a result that changes on rerun is weak evidence regardless of the tool that produced it.

How long does a test framework consolidation take?

It depends on script volume and how much of the existing suite survives the audit. CRIO's engagement covered 1,400+ migrated and reworked scripts. The phases that drive the timeline are assessment, framework design, migration, and a parallel-run validation window - the migration itself is rarely the longest part.