There are many test management solutions, and they share essential functionality. They all offer test cases & scenarios, reporting, and integrations with other QA and/or software development tools. How do you make the right test tool selection in software testing and not regret it later? Simple: go over less obvious features.
Weāve created this list as people doing QA ourselves and helping others. Our own solution was launched in 2013. We, an already high-profile consulting company andagon, tried to find the perfect test management solution for our software development clients.
The few available options at the time had too many drawbacks, so we had to walk this road ourselves.
Almost a decade later, we still talk to in-house quality assurance specialists, clients, and clientsā software testers to see what they miss in modern test management solutions. Below are just some of their musings on how to choose a test management tool.
On-Premise vs Cloud Versions
The deployment model may not be your primary criterion, but it can become a deal-breaker. In most cases, you will be just fine running a test management solution on cloud. In fact, it is preferable for the following reasons:
- Upkeep (and associated man-hour costs) lie with the solution provider.
- Prices start low and can be scaled depending on the project.
- Online availability enables work-from-home, both as a policy and emergency Covid containment measure.
Unfortunately, itās the online availability that can be a problem. FinTech companies and banks, yet alone government agencies canāt risk personal data getting leaked online. Sensitive industries have to use on-premise software, which entails:
- Restricted access (office-only or corporate VPN).
- Higher start fee for an enterprise license.
- Various maintenance options inevitably involve your IT resources.
- Tailored solution to address your companyās unique needs.
Large companies can often use cloud software for some projects yet be forced to on-premise products for others. Even if youāre not a large company (yet!), the safest option is to pick a test management solution that runs both on cloud and on-premise. Surprisingly, Jira + a test management plugin for it is not the answer: they are retiring the on-premise offering.
Now that you know the extra criteria for picking the right test management solution, itās time to scout them. Some companies even allow you to try their software as soon as you sign up for a trial. We are such a company: create a no-obligation demo account below.
Try AI-powered TMS
Desktop Client vs Web
The client vs web choice is about both preference and performance. Some people prefer to have everything they use at work as separate apps. After all, it makes sense to split tools where you actually work, talk about work, and show your progress. Desktop apps also have better offline functionality, although some web applications handle internet outages well.
The desktop convenience, however, is a double-edged sword. Some test management solutions may lack versions for individual operating systems, such as QTest lacking a Linux version. Desktop-only products can also frustrate management: you canāt simply use your phone to glance at unresolved issues or stability dashboards.
Performance-wise, both desktop and web apps can get things done at scale via a performance test tool. Modern software architecture offloads as many calculations from the user as possible.
When either cloud or on-premise server does the hard labor, web apps run virtually as well as desktop apps.
On the other hand, some solutions can become slower (no matter how you load them) due to architectural compromises. Hereās a review of Polarion from a seasoned user.Ā
As for web solutions that run fast, you can see one for yourself – get demo access to aqua and see how it handles a real-life QA project.
Neglecting User Permissions
Modern test case management solutions target more than just testers making selecting a test management tool a little bit more complicated. You have the project manager who checks if things are going well. Sometimes, a product owner might get curious if any bugs would delay a new feature. Finally, you have software developers that need to know what bugs to fix first.
The use scenarios go beyond basic āread/write permissionsā, and things only get harder when you run multiple projects.
Luckily, dedicated test case management solutions handle user permissions much better than general-purpose task boards. Advanced products come with more than just several presets of permissions (e.g. Developer, Tester, Manager).
When the functionality is available at all, you can usually create roles with more granular permissions or even change them for each individual user. Here are just some scenarios:
- Junior Project Manager can see/comment on bug reports, but not edit them.
- Senior Developer can mark a bug as done, but only Testers can verify that it is fixed and archive the ticket.
- Members of the Agile team can access issues and tests from their project only.
If you would appreciate such control, run possible scenarios in your head before picking a tool. Hereās an abstract from a review on Tuleap from a user who didnāt enjoy this particular aspect:
Apart from granularity, the difference in user permission often comes down to price. Some cloud solutions allow role-based access only if you buy a more expensive package (Qaseās charge at least 50% extra for that). Individual user permissions might be restricted to enterprise (see TestRail) and/or on-premise licenses only. aqua offers custom user permissions on all cloud and on-premise plans.Ā
Underdeveloped Filters
Now that you’ve decided what users can access, letās see how they can access it. Even for a QA specialist, test management solutions give more information than one can handle or even need at the moment.
This is an even bigger issue for managers.
The common answer is functionality to see only items assigned to somebody and/or having a certain deadline. Such filters, however, usually lack flexibility and require too much clicking. Even when you usually cycle through similar sets of filters, you have to set them up every time.
So, it would be great to have all items in one place, but also have a neat and fast way to slice through them. Our solution is Views, which are advanced filters and presets of filters.
You can choose a number of filters that you would like to apply at the same time, save them as a View, and switch to it whenever you like. They save a few minutes every day, which adds up to hours and workdays.
Views are also a powerful management tool to assess an individual’s workload and mid-sprint progress. Here are some examples:
- QA Lead can see which Software Tester has the least tickets to assign them reproduction of a recently reported critical bug.
- QA Lead can verify whether the Software Tester re-assigned items before leaving for vacation.
- Product Owner can see if there are any unresolved defects that caused a recent spike of the abandonment rate.
Last but not least, Views can be a great collaborative tool.
QA Leads can set them up for testers to help them see their area of responsibility. Testers can share their views to speed up sprint planning and daily standups.
You save time by creating views and increase transparency by sharing them ā looks perfect to me!
Subpar Logging
In industries with mandatory external audits, software development and testing should be traceable. Regulatory compliances require that you know who deployed a new version of the software and how well it was tested beforehand.
This functionality is often what separates test management solutions from regular task boards.
On the surface, logging is a non-issue. Practically every cloud tool shows you the history of changes to individual items/tickets. This information helps even if you are not legally required to store it.
But what happens if the cloud solution shuts down? How do you access the changes if a card was permanently deleted?
Simple: you decouple public change history from regulatory-compliant logging.
In our case, aqua logs all actions of users. You have information on how all present and past employees interacted with your workspace, projects in it, and even individual items.
The record is unaffected by items and even entire projects removed from the workspace. The log cannot be edited, so it always displays a genuine timeline of changes.
Now that you know extra criteria for picking the right test management solution, itās time to scout them. Some companies even allow you to try their software as soon as you sign up for a trial. We are such a company: create a no-obligation demo account below.
Try AI-powered TMS
