When Should a Startup Test Software Manually?

No company today can afford to ship broken software-based products. Even if you're not in a highly regulated industry, bad quality assurance (QA) can decimate sales and prompt job losses, reputational damage, customer loss, and even loss of life.

QA sounds like a given, but software testing is growing more complicated, thanks to the expansion from just desktop applications to web, mobile, and embedded applications. Startups ordinarily test a large amount of their software code manually. After enough expansion, they're usually unable to keep up with the volume. This is where software test automation does wonders, and conventional wisdom suggests you might as well always use it. And why not, right?

But as impressive as test automation is, does a startup always need automated testing? Are there, in fact, times when manual testing is more prudent? And if you decide to switch to automation as a startup, how do you do it? When is the right time?

What's Your Company's Maturity?

Ask yourself: Are you a startup whose product is still in flux? You may not be ready for automated testing if underlying frameworks and target platforms are changing by the week.

Startups move extremely quickly when starting from zero, often pushing function to the max while they attract investment, introducing a colossal amount of new code weekly. Doing this will likely bring numerous foundational changes to features, UI, and the like.

Related:What Are Software Testing and Quality Assurance?

Automated testing, in this case, doesn't necessarily help because the process of creating build automation test scripts takes time. If you must do this from scratch every time your app undergoes significant change, you're wasting developer resources.

Conversely, relying on manual testing will only prove possible if your app development stabilizes and you gain enough scale. For example, suppose one of Europe's most prominent industrial players has had a product on the market for over a decade. In that case, its enormity probably means nobody wants to change much under the hood out of fear it will break. Automated testing makes lots of sense in that case because it reduces the risk of human error.

Manual and Automated Software Testing Require Different Skill Sets

The other thing to understand is that automated and manual testing are not necessarily mutually exclusive. Each has its merits, but they also require vastly different skill sets.

Suppose a startup company is launching a food delivery app. The QA process would look something like this:

In a scenario like this, manual testers would typically involve non-scripters, non-programming savvy individuals. And that's a good thing. You don't want those testers to make assumptions a programmer would make about how things run. You want them to look at a GUI, not overthink it, and use it like a "normal" human consumer. This approach reveals entirely different problems.

Automated testing, on the other hand, uses scripted sequences to execute a test case scenario, hunting for bugs/defects — the goal being to simplify manual labor into scripts that can be executed repetitively anytime.

What Should You Test Manually vs. Automatically?

Any startup dipping into automated testing must first figure out which test cases to automate.

As a rule of thumb, if you want to maximize test automation, you should generally automate these:

More specifically, these are better left automated to save time and effort:

That said, some test cases are more effectively executed manually:

How Would You Transition from Manual to Automated Testing?

What about when your startup's software reaches a scale that makes manual testing impractical? Well, the transition isn't an on/off switch. You have to investigate tools early (and believe me, some organizations start this process far too late).

The good news is you don't have to wait to get started on testing approaches that complement automation like static code analysis, which reviews code in programming without executing it. Many free alternatives are a great way to dip into automation until you adopt more advanced tools.

You should generally consider these factors when switching to test automation:

Design the right framework for your team

Your test suite's effectiveness relies on your framework. The right framework depends on your programmers' skill set, your software development procedures, and nature of the software.

For example, when Specsavers built out an extensive portfolio of in-house Java/web applications on Linux and Windows, they needed a GUI testing tool that required no scripting experience from staff. They chose an automated GUI testing tool that let them specify tests that needed performing by simply entering data into spreadsheets that could then drive GUI tests.

Choose the right tools

Think about future objectives when your startup scales up, not just current goals. Will the tool still exist in five years? Are new releases issued frequently? What does support look like?

It might be tempting to pick low-code/no-code toolkits, aka "the easy tools." Many companies initially gravitate toward these tools because they're easy to learn, but some revert back within a year because they lack the flexibility and robustness to help them achieve sufficient coverage testing.

And while lots of free, open source test automation tools exist, be cautious. While significant, they don't necessarily provide troubleshooting support, and should you need to comply with certification standards, these tools won't be compliant with regulations out of the box. Getting this wrong could result in a costly and lengthy time-to-market process for your product.

Set manageable goals and a quick learning curve

Moving from manual to automation isn't easy, so start with digestible objectives. Start with automating the most-used regression test suite or the longest and most tedious one. Identify and eliminate recurring bugs, then gradually improve automation to cover more testing areas.

Measure the automated testing tool's performance

When you start using an automated testing tool, measure its performance! You don't want to use a tool that doesn't meet your testing goals. You can begin by determining:

What Quality Assurance Certifications Do I Need?

Whether you're in a regulated industry (like safety-critical sectors, e.g., automotive, manufacturing, healthcare) or you want to demonstrate to customers that you meet industry standards and quality practices, there are several certifications you can earn. They're usually a great way to evaluate software vendors.

Software testing can be daunting for a startup — after all, it's a critical factor in the market success of a product and all future products a company might release. But a good software quality assurance management strategy that follows best practices, with a strong focus on testing, can alleviate a lot of the stress and ensure on-time, repeated releases that meet and exceed your users' expectations.

Jan Aarsaether is QA Tools Advisor at Qt Group.