// NOTE: The element exists on the original form but is hidden and gets rerendered, which leads to intermittent detached DOM issues cy.contains('Next').click().wait(4000)

Thus, by adding system tests, we increase the maintenance costs for development and CI environments and introduce potential points of failures or instability: due to the complex setup, flakiness is the most common problem with end-to-end testing. And most of this flakiness comes from communication with a browser.

What’s a flaky test? It’s a test that sometimes fails, but if you retry it enough times, it passes, eventually.

Quarantined tests are run on the CI in dedicated jobs that are allowed to fail

The test is being marked as skipped because it has randomly failed. How much confidence do we have in that test and feature in the first place.

This policy allows the test suite to stay green while letting individual teams decide when they would like to put in the effort to write more deterministic tests. They may choose to do so right away, or delay until they work on the feature again.

All tests that fail on the master branch will be considered flakes. These tests will be marked as skipped.