5 Lessons learned from re-writing end-to-end tests with Cypress

Photo by Nicole Wolf on Unsplash

For anyone with experience writing interactive tests, they will know all too well these types of tests are temperamental at best. So many factors can affect them, that when they fail the first reaction is the test is faulty, not the code it’s testing. This was exactly my experience with our original Cypress test suite, not the fault of the tools (honest) but the way the tests were written.

Why did we decide to re-write our test suite?

As a bit of perspective, the test suite we had was very small, written over a Christmas break period, and poorly maintained. Not a great start! However, after a post-release bug was found by a user, not us, and realizing we could have easily caught this if we had better test coverage. We committed to invest more in our test suite.

On top of this, the tests we had were flakey and failing often because of timing issues. Adding the quick dirty hack cy.wait() all over the place to try and keep them alive was a naive approach to solving the problem and bound to fall over eventually.

Principles around the rewrite

After some heavy persuasion from my colleague, we talked through a few key aspects these tests needed to meet for us to continue to support them. Part of this was to persuade me you can write reliable end-to-end tests as well. We came up with:

• Avoid using cy.wait, unless there is a legitimate constraint such as avoiding polling a service for example.
• Split our tests into two suites, smoke and full. Not uncommon but something we hadn’t introduced previously.
• Reliability and confidence they will be green unless a legit failure and not commit a test that will occasionally fail because of a timing issue.
• Have the smoke suite run in a ‘short’ period of time. Anything over 10–15 minutes seems very excessive for the scale of our app at the moment.
• A logical structure, I’ll talk about this more but having a base that can scale was the key here.

Lessons learned

1. Overcoming ‘detached from DOM’ error.

This was a major cause of our initial set of tests unreliability so solving this was fundamental to us continuing this investment with Cypress. If you’ve never seen this error before, it occurs when you select an element on the page and try to perform an action or assertion against it but in-between time the DOM has changed and the node no longer exists.

A really common problem for apps that use the hydration technique, documented in many issues on the Cypress GitHub account. There are a lot of suggestions on how to overcome these but the one we found most reliable is cy.waitUntil() , a simple retry function to get your element if it's not been found, or anything you want to wait on, not just DOM nodes. I have no affiliation with the team that made this but I’m incredibly thankful I could regain some sanity after using this helper. I highly recommend you take a look (https://github.com/NoriSte/cypress-wait-until).

2. Using `cy.route()` correctly and more frequently

Our key functionality, ordering a product, relies on a bunch of API calls for different aspects of this process. Fetching a list of products, buyers, suppliers, favorited products. The list goes on and will grow as we enhance the app. Because of this, we need to assert on a DOM element when we know it’s available, not just when assume it is. Listening to the API calls in a sequence meant we could step through a complex process with confidence the data is there when we want to assert on the UI. Something this simple took me a wee while to realize and has enhanced the stability of our tests tenfold.

3. Testing our Redux state alongside our UI

Taking inspirations from a Cypress blog post around testing a vuex data store, it was obvious there was a lot of benefit in testing our redux store alongside the UI. Namely:

• You can verify all the details of the data you need, without them having to be in the UI. For example, I have a test checking an order just placed has a created timestamp within the last minute. We don’t expose this data to the UI, but we can use it to verify we have the order in our state.
• Being able to find something in a list with a certain attribute. Another order related example, I want to find an order which is ‘Overdue’ in my list of orders. This is much faster to do in our redux store than fetching DOM nodes and looping over them.
• A secondary check on list length. I use this a lot, searching for something and want to check only one is found not just one item matching your DOM node. We can be sure of this by using the redux store to assert over.
• For pagination, want to be sure we’ve got a new set of data to load? Your UI will likely not display that until it needs to but our state knows there’s a new page we can load if needed.
• If our tests fail on the UI but not the data, we know this could be a rendering issue or regression in the React component but not in our redux state. This can be incredibly helpful for more complex interactions which might involve animations or costly re-renders within a component and debugging issues as they come up.

These are just a select few common examples, but I have no doubt there will be many many more.

4. Using our Redux store to perform clean up and set up tasks

Closely related to point three, but I think it’s a valid point on its own. Being able to dispatch events from our store has been incredibly useful. It’s meant we can fetch data from the API to set up the state in our app if we need to, or clean up data after a test which we do a lot.

The last thing we wanted the tests to do is to navigate through the UI for these tasks. It’s slow and unnecessary. It also means we don’t have to do separate API calls specific to the test saving time and code duplication. Sure, we do need to write our API calls outside the app for certain tasks but not every time, which is fantastic.

5. Structuring tests so they demonstrate common flows in our app

Something we should have considered from the beginning but worth noting a lesson learned from not doing this. We’ve decided to approach our tests based on a top-level of user role. For example an unauthenticated user, user acting as a buyer or user acting as a supplier. They all have different workflows and experiences in the app dictated by this so made sense to make it our top-level structure. From here we split out into more specific flows such as ‘ordering’ or ‘adding products’ etc. We can share common functionality outside of this or within each role. It works well for us as a simple solution to keeping our Cypress house in order.

In Conclusion

We were able to start building a robust, fast, and reliable test suite with Cypress by following a few key principles and learning a few lessons along the way. I hope you find some of these useful in your test suite and looking forward to seeing more in the comments.

Cheers 👋