Nithyakarma Quality Lab
The same 39 test cases, written twice: Playwright (TypeScript) and Selenium (Java + TestNG), against the live app.
Playwright
37 / 39 passed
- Failed
- 0
- Skipped
- 2
- Wall time
- 69s
Selenium
37 / 39 passed
- Failed
- 0
- Skipped
- 2
- Wall time
- 123s
Parity
39 / 39 agree
- Test cases
- 39
- Same result in both
- 100%
Test cases
| ID | Test case | Playwright | Selenium |
|---|---|---|---|
| Login page | |||
| TC-01 | Login page renders all elements | passed1.2s | passed2.4s |
| TC-02 | Validation: bad email format | passed0.6s | passed1.1s |
| TC-03 | Validation: email missing domain | passed0.7s | passed1.1s |
| TC-04 | Validation: password 7 chars | passed0.5s | passed1.2s |
| TC-05 | Validation: valid email | passed0.5s | passed0.6s |
| TC-06 | Validation: password exactly 8 chars | passed0.5s | passed1.2s |
| TC-07 | Validation: empty email | passed0.5s | passed1.3s |
| TC-08 | Validation: empty password | passed0.5s | passed1.1s |
| Security (negative) | |||
| TC-09 | Captcha blocks an automated browser login | passed12s | passed14s |
| TC-10 | API rejects login without a captcha token | passed0.4s | passed2.1s |
| Today | |||
| TC-11 | Header shows logo, tabs, streak pill and logout | passed1.2s | passed0.5s |
| TC-12 | Every tab navigates to its page | passed1.8s | passed3.1s |
| TC-13 | Greeting shows today (IST) and the user name | passed1.6s | passed0.6s |
| TC-14 | Panchangam box and three kalams | passed1.0s | passed1.1s |
| TC-15 | Family switcher shows Me and Add child | passed0.6s | passed0.0s |
| TC-16 | Add child opens the profile family section | passed0.7s | passed0.1s |
| TC-17 | Streak card: current, best, punya, tier | passed0.7s | passed0.0s |
| TC-18 | Anushtanams list: sandhya slots and Mark Done | passed1.4s | passed0.6s |
| TC-19 | Mark one sandhya slot (opt-in write) | skipped0.7s | skipped0.0s |
| Add anushtanam | |||
| TC-20 | Picker lists options, tracked ones disabled | passed1.8s | passed0.6s |
| TC-21 | Search filters the picker | passed1.5s | passed0.7s |
| TC-22 | Add an untracked anushtanam (opt-in write) | skipped1.2s | skipped0.0s |
| Learning | |||
| TC-23 | Lists every read-along practice | passed1.2s | passed0.3s |
| TC-24 | Hanuman Chalisa reader and language switch | passed1.3s | passed0.2s |
| History | |||
| TC-25 | Family switcher and dated entries | passed2.0s | passed1.3s |
| Sabha | |||
| TC-26 | Defaults to this week and shows my row | passed1.3s | passed0.1s |
| TC-27 | Week / Month / Kids switch the board | passed1.2s | passed0.2s |
| Calendar | |||
| TC-28 | Day view: panchangam details and kalams | passed1.7s | passed0.2s |
| TC-29 | Next and Previous move a day and back | passed1.9s | passed0.2s |
| TC-30 | Week and Month views | passed1.9s | passed0.2s |
| Profile | |||
| TC-31 | Name, tier progress and stats | passed1.6s | passed0.1s |
| TC-32 | Save enabled only after the name changes | passed1.8s | passed0.2s |
| TC-33 | Delete account disabled without the email | passed1.5s | passed0.1s |
| TC-34 | Footer links | passed1.6s | passed0.1s |
| Settings round-trips | |||
| TC-35 | Add a child and remove it | passed2.9s | passed1.5s |
| TC-36 | Tradition switches and switches back | passed2.7s | passed1.9s |
| TC-37 | Marital status switches and switches back | passed2.4s | passed2.0s |
| TC-38 | Leaderboard opt-in toggles and toggles back | passed2.7s | passed2.1s |
| Logout | |||
| TC-39 | Logout returns to the login page | passed2.1s | passed1.1s |
How the two suites differ
| Playwright (TS) | Selenium (Java) | |
|---|---|---|
| Login | Setup project logs in once via Supabase magic link, saves storageState | Same magic link, session injected into localStorage per test |
| Waiting | Auto-waiting locators and web-first assertions | Explicit WebDriverWait conditions in page objects |
| Locators | Role and label based (getByRole, getByLabel) | CSS and XPath (no accessibility-tree locators) |
| Execution | Parallel workers, projects for ordering | Sequential, TestNG <test> blocks for ordering |
| Network waits | page.waitForResponse, built in (~210 ms per save) | WebDriver BiDi listener + CountDownLatch (~228 ms per save; ~533 ms with default WebDriverWait polling) |
| Session start | ~70 ms browser launch | ~880 ms ChromeDriver session, ~110 ms more with BiDi |
| Data-driven | Loop over a cases array | TestNG @DataProvider |
| API tests | Built-in request fixture | REST Assured |
| Captcha | Not bypassed in the UI: kept as a negative test. Logged-in tests use the admin magic link instead. | |