How FreshTracks Canada Cut 8,000 Test Cases to 2,000 with Playwright Automation
Summarize With:
Gaurav Mehta
TL;DR
FreshTracks Canada was running 8,000+ manual test cases. Every regression cycle took 2-3 weeks, and earlier automation attempts had already failed. ThinkSys cut the suite down to ~2,000 journey-based cases, built a custom Playwright framework, and automated the riskiest flows first, wired into CI/CD. The suite now catches real bugs (broken exports, bad sorting, confusing error messages) before customers see them. And the team trusts automation again.
At the Glance
Client | FreshTracks Canada: custom Canadian travel experiences for North American travelers. |
|---|---|
Industry | Travel / eCommerce. |
Challenge | 8,000+ manual test cases, 2-3 week regression cycles, previous automation attempts had failed. |
Approach | Test-suite consolidation, custom Playwright framework, risk-first automation, CI/CD integration. |
Headline results | Test suite consolidated 8,000 → ~2,000 cases · critical bugs caught before production · automation running in CI on a risk-first rollout. |
Buried under manual regression? Talk to a Playwright automation specialist.
Meet FreshTracks Canada
FreshTracks Canada plans custom Canadian trips for travelers across North America. The trips are analog. Everything behind them isn't; booking, logistics, and customer support all run on software. So a broken export or a mis-sorted search result isn't a cosmetic bug. It's someone who can't book their trip.
The Problem: 8,000 Test Cases and a Three-Week Regression Cycle
FreshTracks' test suite grew the way most do. One case at a time, for years. Then the numbers stopped working:
- 8,000+ manual test cases in the regression suite. Many overlapped, many were outdated, and plenty tested the same flow in fragments.
- Every regression cycle took 2–3 weeks. Releases queued behind testing. Features shipped late.
- Bugs still reached production. Coverage looked thorough on paper. Real risks slipped through anyway.
- They'd already tried automation. It failed. So the team was skeptical this time would be any different. Fair enough.
That last point is the whole story. FreshTracks didn't lack effort, they'd automated before. But automating a bloated suite just gets you bloated automation. It goes flaky, people stop trusting it, and it quietly gets abandoned. That's the cycle ThinkSys came in to break.
What ThinkSys Did
- Cleaned the suite before automating anything. This is the step failed automation projects skip. We went through all 8,000+ cases, killed the duplicates, dropped the dead ones, and merged fragments into real end-to-end scenarios built around user journeys, not UI details. What came out the other side: roughly 2,000 well-structured test cases covering the same ground. A 75% smaller suite, before we'd written a single line of automation.
- Picked Playwright for reasons. It's fast. It covers every major browser from one codebase. It tests APIs too, which matters when booking data flows behind the UI. And when a test fails, the traces and inspector tell you why instead of leaving you guessing. For a travel platform where one flow crosses search, booking, payment, and confirmation? Those things beat any feature checklist.
- Built a framework, not a script pile. Reusable components. Real waits instead of hard-coded sleeps. Selectors that survive a redesign. Reports the whole team can actually read. That's the difference between automation that compounds and automation that rots, and FreshTracks had already lived through the rotting kind.
- Automated by risk, not by list order. Critical flows first, the ones where a production bug costs bookings. Phase one put 10% of the suite under automation, and that number was a choice, not a shortfall. The riskiest 10% buys you most of the protection while the framework proves it can be trusted. Scaling coverage on an unproven framework? That's exactly how the previous attempts died.
- Wired it into CI/CD. Working with FreshTracks' DevOps team, the suite now runs in the pipeline. Regressions show up when the code changes, not three weeks later in a manual cycle.
Results and Future Opportunities
When all the changes were implemented, the difference was evident:
Area | Before | After |
|---|---|---|
Test suite | 8,000+ fragmented manual cases. | ~2,000 consolidated, journey-based cases. |
Regression execution | Entirely manual, 2–3 weeks per cycle. | Highest-risk flows automated in CI; manual effort focused on exploratory testing. |
Automation trust | Previous attempts failed and were abandoned. | Stable Playwright framework the team builds on. |
Production escapes | Critical bugs reaching users. | Broken export feature, incorrect sorting logic, and inconsistent error messaging all caught before release. |
Worth pausing on those three catches. Each one was found by the automated suite before a customer ever saw it. A broken export. A wrong sort order. On a booking platform, that's exactly the kind of bug that used to slip through 8,000 manual cases, and now gets stopped by 200 well-chosen automated ones.
Want your regression suite consolidated and automated the right way? Book a 30-minute assessment.
Why Automation Projects Fail, and What This One Did Differently
You've probably seen this movie: a company tries automation, it goes flaky, trust evaporates, and everyone drifts back to manual testing muttering "automation doesn't work for us." It's rarely true. Three lessons from this engagement travel well:
- Consolidate before you automate. Automate a messy suite and you get a fast-running mess. The 8,000 → 2,000 cleanup wasn't prep work, it was the work. Even the still-manual portion takes a fraction of the effort now, because the suite tests journeys instead of fragments.
- Start with the riskiest 10%, not the easiest 50%. Coverage percentage is a vanity metric. Protection is what counts, and protection lives in the critical paths. A stable 10% that catches real escapes earns the trust that funds phase two. A flaky 50% kills the whole program.
- The framework is the product. Scripts are disposable. The framework; selector strategy, wait discipline, reporting, CI hooks decides whether automation compounds or rots. It's also why "let's just record some tests" never survives contact with reality.
This is the same framework-first approach behind our Playwright automation testing services. Still picking a framework? Our Playwright vs Selenium vs Cypress comparison covers when Playwright is the right call, and when it isn't.
Conclusion
FreshTracks didn't need more testing. It needed less; fewer, better test cases, with the riskiest flows automated on a framework built to last. Cutting 8,000 cases to 2,000 and putting critical paths under stable Playwright automation turned a three-week manual bottleneck into a safety net that runs on every code change. Export bugs, sorting bugs, messaging bugs, caught before customers see them. And maybe the part that matters most: a team that had been burned by automation trusts it again.
Tell us about your regression bottleneck - we'll show you the consolidation-first plan.