Rainforest: Crowdsourcing Quality Assurance
Rainforest is seeking to take the pain out of software quality assurance, so businesses can focus on building great products instead of hunting down bugs.
The digital transformation of the modern economy in recent years is obvious to even the most casual observer – having a web presence is now table stakes for even the most traditional businesses, and many new ventures are entirely web-based. At the same time, web technologies have continued to advance such that it has become progressively easier to spin up a website or application relatively quickly, and to make continuous changes to it. One consequence of this is that the quality and performance of the websites and smartphone apps on offer can vary quite dramatically. Even the most reliable software is not immune to bugs in an environment where businesses are rapidly iterating to meet user needs, and these bugs can present significant costs in labor time, eroded user trust, and lost revenue. The tension between speed and quality is a challenge that all software development must account for, and one that companies like Rainforest are trying to help solve.
Rainforest offers an online platform that provides thorough, fast, crowd-sourced quality assurance (QA) testing for websites and applications. Their business model is built around the mission “to make QA easy and painless for everyone.” [1] While software is typically built with a set of automated tests that can be run to ensure that code is functioning as expected, many companies also incorporate an element of manual, human testing for new features before releasing them to the public. Rainforest creates value for its customers by allowing them to dramatically reduce the time they spend on this manual testing by outsourcing it to Rainforest’s thousands of testers around the world. Additionally, Rainforest offers testing across combinations of browsers, operating systems and devices. Speaking from personal experience, coding for the idiosyncrasies of all of these possible combinations is an enormous headache, and yet typically a highly important business requirement, and thus Rainforest’s management of this complexity is a significant part of their value proposition. The price of this service will vary depending on how many tests a business wants to run, and with what frequency. Although the price can turn out to be substantial, Rainforest makes economic sense for many businesses that are consequently able to pare down or even eliminate their dedicated QA teams.
At a tactical level, Rainforest’s operating model is structured around its network of human testers around the world. These testers follow testing instructions, written in plain english and provided by the customer, and log whether the test cases “passed” or “failed” into the online platform for customers to view. [2] The platform allows businesses to easily store, organize, and update their test cases, and to provide all stakeholders with visibility into test runs that are in progress and outstanding bugs that need to be addressed. Because Rainforest recruits testers from all over the world, customers can have their website tested at all hours of the night. Additionally, the testing is fairly low-skill work that can be easily done remotely via the use of virtual machines (which can mimic different operating systems and browsers), reducing the labor costs for Rainforest.
Looking to the future, Rainforest has an opportunity to continue to expand on its QA offerings. Given their existing large base of hired testers, they are well positioned to offer more qualitative, user experience feedback, in addition to simple “pass/fail” assessments. For example, if a company is testing a new design for their homepage, they can currently submit tests to Rainforest that might say something like “Do you see a button that says ‘Log in’ in the top right hand corner?” But what if it could also ask Rainforest testers to provide feedback on how easy the “Log in” button was to find? Or on whether the “Log in” button felt too small to easily click on a smartphone screen? Given that businesses often do user research to obtain feedback before releasing new features, Rainforest could give them easy and fast access to a broad array users and to streamline and organize their feedback. As they continue to scale and build out new features, an extension into broader categories of “testing” seems to be a natural fit with both their business model and their operating model.
(697 words)
[1] Rainforest, “About Rainforest QA.”, https://www.rainforestqa.com/about/, accessed November 2016.
[2] Kyle Russel, “Rainforest Raises $4M To Build Scalable QA”, techcrunch.com, February 10, 2015, https://techcrunch.com/2015/02/10/rainforest-raises-4m-to-build-scalable-qa/, accessed November 2016.
As an entrepreneur building cloud based software I learned the hard way how important testing is. Often in the West, we assume wrongly that everyone runs the last version of Windows or mac OS whereas many people are still surfing the web with an outdated operating system. As a fact, a quick search on Google shows that about 9% of people still surf the web with a 15 year old OS, Windows XP. For developers this creates a massive challenge as it can take weeks or even months before finding out that a software is not performing as it should be under certain conditions. As a result, if a company is not performing its QA testing thoroughly, competitors that have better testing capabilities can move in quickly and take market shares. Startups that have generally less resources in their earlier years often overlook the testing part of the development process and could therefore profit the most from Rainforest’s platform.
Laure, this article was great and gave me brilliant insight into something I know little about. I imagine one of the complexities of the business would be selecting testers who have the “right” level of computer literacy for beta testing, i.e. a level of literacy commensurate with the potential customer base depending on the product, which might be significantly older and less computer literate than average. More signigifcantly perhaps, it would also be interesting to understand how they manage the risk of plagiarism / poaching by testers for more advanced features? I imagine this could present significant commercial risk.
Laure – Rainforest’s approach to crowdsource QA makes a lot of sense. I’m surprised this hasn’t been done earlier and I wonder who the other competitors are in this space. I would assume that testers would be filtered by language and skill background to ensure the testers put the website through its paces and the customer can provide feedback through reviews. However, I wonder if the business model that Rainforest has is sustainable? There are now so many templates available to create websites like wix.com and part of the allure is that it looks professional, is very easy to set up and virtually bug free, a website in a box. Given the rise of chatbots and machine learning it seems as though human based testing could very rapidly become obsolete. Perhaps Rainforest needs to actually to start having their expert users train algorithms to be able to complete this QA almost like automatic spell check. This would be a way for a user/customer to run a QA test real time and then check the results themselves.