What’s new in Katalon Studio 5.5: Advanced recording and more

1. Introduction

Writing and maintaining test scripts are two important tasks directly affecting the effectiveness and efficiency of test automation. One dilemma in test automation is that if scripts are quickly generated using tools, they may not be executable or playable when the application under test (AUT) changes. This is exactly the problem with the recording capability offered in many existing test automation tools. The cost of maintaining test cases and scripts generated by recording is too high that makes benefits of test automation unattainable.

To help address this problem, Katalon introduced the new recording capabilities with several key benefits to help automation engineers improve their working processes and the quality of tests. This advanced recording is the feature not only useful for test automation beginners but also for experienced engineers.

Katalon’s advanced recording is designed to generate effective, efficient, and maintainable tests that can be collaborated among team members. With this enhancement, the recording becomes an essential part of Katalon Studio.Advanced Recording-KS-5.5

2. Why is advanced recording needed?

Record & Playback is incredibly useful for beginners to start within their automation career as it helps generate and execute automation test easily and quickly. However, this approach has several major drawbacks when it comes to using in professional contexts, including :

  • Robustness

The recording capability depends heavily on a capture engine such as Selenium Webdriver which does not fully address the complexity of modern user interfaces. The generated test scripts may not be playable and fragile to changes in the AUT. For example, modern UI framework such as AngularJS has dynamically generated ID for HTML components. Therefore, scripts created by ID locator approach which is popular in recording will be more likely not playable under a small interface change.

  • Reusability

Test cases generated from recording are linear, and they are difficult to be reused across test cases and suites. Moreover, the recording process is usually not designed in such a way that encourages automation engineer to leverage existing scripts.

  • Maintainability

In automation testing, maintainability is always the main concern in any long-term projects as the maintenance cost of test scripts can be enormous if the AUT is changing.

➤ Learn basic “Record & Playback” using Katalon Studio here: Creating test case using Record & Playback

3. Recording in Katalon Studio 5.5

Fortunately, Katalon’s Advanced Recording offers an answer to address the dilemma in generating tests quickly while ensuring them to be maintained easily. With Advanced Recording, automation engineers are able to enjoy fast test generation while still being able to maintain and reuse test cases easily. Furthermore, in a  team with diversified skill sets, it’s essential to allow team members to collaborate effectively in creating test cases. With Advanced Recording, this goal can be achieved.

In Katalon Studio 5.5, the recording capability has been improved with the spirit of Advanced Recording: efficiency, effectiveness, maintainability, and collaborativeness.

Continue to record from an existing test case

Users are now able to load all steps of a test case to the recording window for continuing to record and generate new test steps for the test case. As shown in the figure below, when users select a test case and click on Record, a prompt will be shown to confirm if they want to record for the test case.

Katalon advanced recording

Test steps and corresponding dependencies of the selected test case are loaded to the recording window as shown below.Katalon advanced recording


Be able to fully edit the test case while recording

During recording, most test step types can be added. The Variables section is available for editing. Test objects in the object repository can also be customized throughout the recording process and are synched with the objects in the test steps.

Katalon advanced recording

Be able to execute specific steps while recording. Multiple options for executing test steps such as Run All Steps, Run Selected Steps or Run From a Selected Step are available. Logs and execution status are also tracked properly.

Katalon advanced recording

Merge duplicate test objects after recording

After recording, there’s a default option to merge recorded test objects with the existing ones if their names match.

Katalon advanced recording

Benefits of the new recording

With the above functionalities, the new recording provides the following benefits:

  • The capability to run selected test steps during recording helps users to assure that all recorded steps are playable. It’s very useful when dealing with long and complex test cases. If a test case is solid from the beginning, maintenance effort can be reduced significantly. Moreover, the run feature during recording also opens the capability to add advanced test steps such as custom keywords or call test cases. For instance, when an automation engineer is recording a new test case, and she or he knows there is a common test case which can be used as part of the new one. Instead of recording those steps again, the engineer can add that test case in then execute it to go through its steps as if they were recorded. This helps encourage automation engineers to reuse the existing tests which eventually improves the effectiveness of the whole project.
  • The new recording capability unites three previously separated functions including spying, recording, and executing. Without this integration, it is time-consuming to fix failed captured test objects in test cases. Also with the new function to load the existing test steps of a test case to the recording mode, users are able to easily reproduce issues then fix them and verify the fix immediately. All of those debug and fix operations are taken efficiently on just one screen.
  • In automation testing, besides test cases, test objects should be taken seriously. Merging test objects is important to improve the consistency and maintainability of test artifacts.
  • Collaboration is a crucial focus of Advanced Recording as in a development team nowadays, not all testers are automation experts. Most of them might be manual testers or have less than one year of test automation who definitely prefer the easy and quick manner of recording approach. Meanwhile, the experts in the team are normally in charge of building the framework and prepare common materials. Therefore, in Katalon Studio, advanced features are gradually introduced into the test generation process to better leverage existing test materials.

It’s just the beginning…

Katalon Studio continues improving its features to be the best that a test automation solution can offer. More features are under development to be released in the near future. Below are some:

  • Smart object locator strategies: Object locator strategies are often a black box in Selenium-based products of which Katalon Studio is not an exception. However, going beyond Selenium-based object locator strategies, Katalon Studio will introduce smart object locator strategies that are more maintainable and resilient to changes in the AUT. This feature is expected to reduce maintenance cost significantly.
  • Object Type: As mentioned above, test objects are treated as an important asset of automation projects in Katalon Studio. A future release will include Object Types with behaviors and semantics. This feature can help users group a set of common or repetitive actions to a single item then use it in multiple places; therefore, it is expected to improve reusability and maintainability of test assets considerably. For example, in order to record a searchable combobox, an automation engineer needs to go through several steps: focus on the search text box, fill in the search content, wait for the value to appear in the dropdown list then select the appropriate item. Those steps are normally fragile due to the nature of the recording engine, looks not so meaningful to readers and also not highly reusable as well. With Object Type, they can be grouped into one unique step like set combobox value with one input so-called combobox test object. It will look much more meaningful, be more robustness and much easier to be reused.

Download Katalon Studio to explore the powerful features of Katalon Studio v5.5 which help your automation testing easier than ever before.

How to select the right automation testing tool for your project?

How do People Select Test Automation Tools?

A large majority of experienced testers and managers prioritize licensing and support costs and good test reports when deciding which test automation tools to use for their projects. In the Test Automation Challenges survey conducted with 2,000+ test professionals, more than 60% of respondents with 3+ years of experience in test automation agreed that these criteria are the most important priorities in deciding a test automation tool. Moreover, respondents with less experience in test automation prioritize training and support materials over other criteria.

How do People Select Test Automation Tools?

The top six priorities considered by more than 50% of respondents with 3+ years of experience are:

(1) Licensing and support costs

(2) Good test reports

(3) Training, documentation, tutorials, guidelines

(4) CI, DevOps support

(5) Level of programming skills required

(6) Level of skills and experience required

Suggested Tools:

Open-source tools (coding experience required)

Free tools (limited community’s support)

Commercial tools (high support & license cost)

Software automation tools and frameworks are crucial for the success of test automation projects. But it is a difficult task to select a right set of test automation tools for a new software project. There are many tools with each having several unique good features. One single tool may not satisfy all project requirements and constraints, resulting in multiple automation tools to be used. Chosen tools seem to be working at the beginning, but troubles occur later as hidden problems are overlooked at the time of tool selection.

A tool vendor often provides a list of key features and characteristics when comparing their tool with others in the market. They tend to highlight their tool’s strengths while ignoring important features that may pinpoint its weaknesses. This practice confuses not only new test automation adopters but also experienced ones in making decision about which tools and frameworks for new test automation projects.

To select a suitable tool, one can base on a set of top priorities for which every interested tool is checked. Such checklist may come from experience of professionals as well as specific project requirements and constraints.

Understanding the priorities in determining automation tools is part of our survey which questioned software test professionals about their experience with test automation. The survey’s questionnaire was posted on several software testing websites and shared via several mailing lists including that of Katalon Studio’s.

We received answers from 2,291 respondents of which 1,877 (82%) have ever applied test automation on their projects. Of all respondents, 970 (or 42%) and 484 (22%) have at least one and more than three years of experience in test automation, respectively. The results reported here exclude the respondents who said to have yet to apply test automation.

Tool Selection Priorities by all Respondents

The chart below shows all attributes suggested as possible top priorities and the corresponding percentages of respondents who agreed to. One respondent could choose one or more attributes.Automation tool selection criteria

The top four attributes chosen by at least 50% of respondents as their most important priorities in tool selection include:

(1) Training, documentation, tutorials, guidelines: the highest percentage of respondents chose this attribute, which is concerned with training and materials for learning and using the tool, as a priority for their tool selection.

(2) Good test reports: tool generated test documents and logs for status report, test execution analysis, and defect diagnosis.

(3) Licensing and support costs: all costs associated with acquiring, maintenance, and support the use of tools.

(4) Level of programming skills required: if testers on the team do not possess good programming skills, learning programming to use a tool is a real concern for them.

The highest priority is identified by 56% of all respondents. Many respondents, 58%, have less than 1 year of experience in test automation. So, it is reasonable that training and materials for learning and using tools is the most important attribute for choosing a tool.

The fifth highest priority chosen by almost 50% of respondents is the level of skills and experience required by tools. This attribute, which is related to the 4th highest priority or the level of programming skills required, refers to the need of skills and experience to use the tool effectively. For example, Selenium is a popular test automation framework but it requires adopters to possess a high level of technical skills and experience to get started.

It is quite surprising to find that a majority of respondents did not consider user interface/user experience (UI/UX), support services, long-term commitments, and application lifecycle management (ALM) tool integration as the priorities in selecting an automation tool. Noticeably, less than 20% of respondents chose the ability to integrate with the ALM tools as an important priority.

It is worth noting that although functionality/feature-rich of a tool is often cited as a strength, it is not appreciated by many respondents with just 46% of them using it as a priority.

Tool Selection Priorities Evaluated by Experienced Respondents

We considered two respondent groups, one with at least 3 years of experience in test automation and the other with 1 year or less. It is intuitive to believe that the feedback on the top priorities for selecting tools by experienced test automation professionals is more reasonable than that by less experienced ones.

➤➤➤ Continue reading How do People Select Test Automation Tools?

Author: Vu Nguyen
Dr. Vu Nguyen, Advisor at Katalon, LLC,  Director of Software Engineering – KMS Technology, Lecturer at the University of Science, Vietnam National University.
1 2 3 4 5 6