5 considerations you need to take when picking a test management solution
Management Agile Auditability
12 mins read
January 9, 2024

5 considerations you need to take when picking a test management solution

There are many test management solutions, and they share essential functionality.

Martin Koch
Denis Matusovskiy

There are many test management solutions, and they share essential functionality. They all offer test cases & scenarios, reporting, and integrations with other QA and/or software development tools. How do you make the right test tool selection in software testing and not regret it later? Simple: go over less obvious features.

We’ve created this list as people doing QA ourselves and helping others. Our own solution was launched in 2013. We, an already high-profile consulting company andagon, tried to find the perfect test management solution for our software development clients.

The few available options at the time had too many drawbacks, so we had to walk this road ourselves.

Almost a decade later, we still talk to in-house quality assurance specialists, clients, and clients’ software testers to see what they miss in modern test management solutions. Below are just some of their musings on how to choose a test management tool.

On-Premise vs Cloud Versions

On-Premise vs Cloud Versions

The deployment model may not be your primary criterion, but it can become a deal-breaker. In most cases, you will be just fine running a test management solution on cloud. In fact, it is preferable for the following reasons:

  • Upkeep (and associated man-hour costs) lie with the solution provider.
  • Prices start low and can be scaled depending on the project.
  • Online availability enables work-from-home, both as a policy and emergency Covid containment measure.

Unfortunately, it’s the online availability that can be a problem. FinTech companies and banks, yet alone government agencies can’t risk personal data getting leaked online. Sensitive industries have to use on-premise software, which entails:

  • Restricted access (office-only or corporate VPN).
  • Higher start fee for an enterprise license.
  • Various maintenance options inevitably involve your IT resources.
  • Tailored solution to address your company’s unique needs.

Large companies can often use cloud software for some projects yet be forced to on-premise products for others. Even if you’re not a large company (yet!), the safest option is to pick a test management solution that runs both on cloud and on-premise. Surprisingly, Jira + a test management plugin for it is not the answer: they are retiring the on-premise offering.

Now that you know the extra criteria for picking the right test management solution, it’s time to scout them. Some companies even allow you to try their software as soon as you sign up for a trial. We are such a company: create a no-obligation demo account below.

Try AI-powered TMS

Try aqua

Desktop Client vs Web

Desktop Client vs Web

The client vs web choice is about both preference and performance. Some people prefer to have everything they use at work as separate apps. After all, it makes sense to split tools where you actually work, talk about work, and show your progress. Desktop apps also have better offline functionality, although some web applications handle internet outages well.

The desktop convenience, however, is a double-edged sword. Some test management solutions may lack versions for individual operating systems, such as QTest lacking a Linux version. Desktop-only products can also frustrate management: you can’t simply use your phone to glance at unresolved issues or stability dashboards.

Performance-wise, both desktop and web apps can get things done at scale via a performance test tool. Modern software architecture offloads as many calculations from the user as possible.

When either cloud or on-premise server does the hard labor, web apps run virtually as well as desktop apps.

On the other hand, some solutions can become slower (no matter how you load them) due to architectural compromises. Here’s a review of Polarion from a seasoned user. 

As for web solutions that run fast, you can see one for yourself – get demo access to aqua and see how it handles a real-life QA project.

Neglecting User Permissions

Neglecting User Permissions

Modern test case management solutions target more than just testers making selecting a test management tool a little bit more complicated. You have the project manager who checks if things are going well. Sometimes, a product owner might get curious if any bugs would delay a new feature. Finally, you have software developers that need to know what bugs to fix first.

The use scenarios go beyond basic “read/write permissions”, and things only get harder when you run multiple projects.

Luckily, dedicated test case management solutions handle user permissions much better than general-purpose task boards. Advanced products come with more than just several presets of permissions (e.g. Developer, Tester, Manager).

When the functionality is available at all, you can usually create roles with more granular permissions or even change them for each individual user. Here are just some scenarios:

  • Junior Project Manager can see/comment on bug reports, but not edit them.
  • Senior Developer can mark a bug as done, but only Testers can verify that it is fixed and archive the ticket.
  • Members of the Agile team can access issues and tests from their project only.

If you would appreciate such control, run possible scenarios in your head before picking a tool. Here’s an abstract from a review on Tuleap from a user who didn’t enjoy this particular aspect:

customer review

Apart from granularity, the difference in user permission often comes down to price. Some cloud solutions allow role-based access only if you buy a more expensive package (Qase’s charge at least 50% extra for that). Individual user permissions might be restricted to enterprise (see TestRail) and/or on-premise licenses only. aqua offers custom user permissions on all cloud and on-premise plans. 

creating and removing defects

Underdeveloped Filters

Underdeveloped Filters

Now that you’ve decided what users can access, let’s see how they can access it. Even for a QA specialist, test management solutions give more information than one can handle or even need at the moment.

This is an even bigger issue for managers.

The common answer is functionality to see only items assigned to somebody and/or having a certain deadline. Such filters, however, usually lack flexibility and require too much clicking. Even when you usually cycle through similar sets of filters, you have to set them up every time.

So, it would be great to have all items in one place, but also have a neat and fast way to slice through them. Our solution is Views, which are advanced filters and presets of filters.

You can choose a number of filters that you would like to apply at the same time, save them as a View, and switch to it whenever you like. They save a few minutes every day, which adds up to hours and workdays.

Views are also a powerful management tool to assess an individual’s workload and mid-sprint progress. Here are some examples:

  • QA Lead can see which Software Tester has the least tickets to assign them reproduction of a recently reported critical bug.
  • QA Lead can verify whether the Software Tester re-assigned items before leaving for vacation.
  • Product Owner can see if there are any unresolved defects that caused a recent spike of the abandonment rate.

sorting in qa tool

Last but not least, Views can be a great collaborative tool.

QA Leads can set them up for testers to help them see their area of responsibility. Testers can share their views to speed up sprint planning and daily standups.

You save time by creating views and increase transparency by sharing them — looks perfect to me!

Subpar Logging

Subpar Logging

In industries with mandatory external audits, software development and testing should be traceable. Regulatory compliances require that you know who deployed a new version of the software and how well it was tested beforehand.

This functionality is often what separates test management solutions from regular task boards.

On the surface, logging is a non-issue. Practically every cloud tool shows you the history of changes to individual items/tickets. This information helps even if you are not legally required to store it.

But what happens if the cloud solution shuts down? How do you access the changes if a card was permanently deleted?

Simple: you decouple public change history from regulatory-compliant logging.

log of changes in qa tool

In our case, aqua logs all actions of users. You have information on how all present and past employees interacted with your workspace, projects in it, and even individual items.

The record is unaffected by items and even entire projects removed from the workspace. The log cannot be edited, so it always displays a genuine timeline of changes.

Now that you know extra criteria for picking the right test management solution, it’s time to scout them. Some companies even allow you to try their software as soon as you sign up for a trial. We are such a company: create a no-obligation demo account below.

Try AI-powered TMS

Try aqua
On this page:
See more
Speed up your releases x2 with aqua
Start for free
How to choose a test management tool?

To choose a test management tool, define your needs, research available options, evaluate shortlisted tools by using trial versions or demos, consider integration with existing tools and processes, assess ease of use, consider cost including upfront and ongoing costs, evaluate support and documentation, and make a decision based on the evaluation of the different options to select the tool that best fits your requirements and budget.

What are the types of test management tools?

There are several types of test management tools:

  • Test Case Management Tools for creating and organising test cases:
  • Test Execution Tools for tracking test results
  • Test Automation Tools for reducing manual effort and increasing testing speed
  • Performance Testing Tools for testing application performance
  • Regression Testing Tools for retesting after changes
  • Test Management & ALM Tools providing a comprehensive suite of tools for the entire software development lifecycle
  • Defect Tracking Tools for tracking and managing bugs and issues found during testing


What is test management?

Test management is the process of planning, designing, executing, and reporting on tests used to evaluate the quality of software applications. It involves coordinating and organising the efforts of different teams involved in the software development process, including developers, testers, and stakeholders.

closed icon