Katalon team is excited to announce the release of Katalon Studio 5.8 to bringing to you an all-new experience for your API/Web Service testing project:
Instantly import Web service requests with Quick Start Wizard
The Quick Start wizard allows you to quickly import Swagger 2.0/WSDL definition, create draft REST/SOAP requests, or try out with different testing requirements. When importing Web service definition, Katalon Studio will load specified information and populate into your project so you will never have to spend time converting them manually. This wizard can be launched and relaunched anytime in Help > Quick Start.
Retrieve Request History at any time
All Web service requests sent in Katalon Studio are stored in the ‘Request History’ panel, which you can retrieve anytime.
Add Web service request to new/existing Test Case
Web service request objects can be added directly to a test case from the ‘Object Details’ view with two available options: ‘Add to new Test Case’ or ‘Add to existing Test Case.’
Support PATCH HTTP method
Katalon Studio now supports the PATCH HTTP method to help you apply a partial modification to resources.
This release also includes new bugs fixes to advance the convenience and quality of your test automation:
Fix an issue in which local URL cannot be used for SOAP requests.
Remove an error message displayed when users create a new sample project.
We hope you enjoy your API testing experience with Katalon Studio v5.8 and don’t forget to provide your feedback so that we can make Katalon Studio more suitable for your needs.
Before releasing any application, the most important thing that needs to be seriously implemented is ensuring every performance characteristics. This means you then must do a series of test, which is called Soak Testing, to help you discover immediate performance-related problems—such as memory leaks or resources leaks—before the release of the product on a particular platform.
What is Soak Testing?
Soak testing (otherwise known as endurance testing, capacity testing, or longevity testing) involves testing the system to detect the performance-related issues such as stability and response time by requesting the designed load on a system. The system is then evaluated to see whether it could perform well under a significant load for an extended period, thereby measuring its reaction and analyzing its behavior under sustained use. Soak testing is a type of load testing.
There is a myriad of soak testing tools these days, such as LoadStorm, LoadRunner, LoadUI, OpenSTA, Apache JMeter, Appvance, and WebLoad.
Ideally, a soak test should run as long as possible, though it will depend on the system for you to determine the duration of your test. Below are some examples when soak testings can be considered:
When a bank announces that it will be closing, its system is expected to handle a large number of transactions during the closing days of the bank. This event is rare and unexpected, but your system has to handle this unlikely situation anyway.
A system needs to withstand the period of no on-call staff. Usually, this will happen when the staff doesn’t work on the weekend, so the soak test would prove the system’s ability to function for a period longer than a weekends-worth of non-stop operation.
Your system is required to process 1000 transactions in 1 hour. You may need to test not only this capability but also go beyond this requirement: testing 1000 transactions in half an hour or 2000 in 1 hour.
Several Considerations for Soak Testing
A soak test duration is often decided by the system’s availability.
Every application must run without interruption.
Each system normally has a frequent maintenance window time; such period will be the key factor in deciding the scope of a soak test.
Why Should You Perform a Soak Test?
Soak testing is mainly used to identify and optimize potential problems, such as memory leaks, resource leaks, or degradation that could happen over time, to avoid impaired performance or system errors. While stress tests will help the development team to test the system to its limits, soak testing takes the system to its limits over a sustained period of use. In other words, soak testing allows the team to mimic the real world usage, in which the users will constantly need access to the system.
Advantages that Soak Testing Brings
It guarantees the application’s fitness.
It detects bugs that could not be found by other performance tests.
It spots the deterioration in performance, which could happen under high continuous load, and helps to fix those issues so that the application will be more robust.
It shows how sustainably the system runs over time.
The test result could be used to validate or improve customers’ infrastructure demands.
Challenges of Soak Testing?
The test is a time-consuming process, and it would be inconvenient to be conducted on a project with very strict time constraints.
You need to use an automation tool together with technical expertise since soak tests run over an extended period of time and consume a lot of data.
It is often difficult to determine how much test is worth applying.
If the test environment is not correctly detached from the live production environment, the errors found by the soak test could negatively impact the whole working system, causing data loss or data corruption permanently.
When to do Soak Testing?
Before a system is released on a particular platform to clients, it must successfully undergo a series of load tests at higher or similar traffic levels. Then you execute the soak test. Soak testing will help you to see how the system runs for a continuous availability period. If there are any issues like memory leaks or memory corruption during the test, it should be reported immediately.
Since the process could take time as long as a day, soak testing should be done over the weekends, although it also depends entirely on the test situation’s restrictions.
Soak testing is now one of the most vital requirements that any team must strictly comply with for a successful deployment.
Soak Testing Strategies
Before performing a soak test, you need to take a bird-eye view of the system and determine:
Figure out which hardware, software, database, and operating system to perform the soak test.
Since soak testing is an extensive process, it is important to design, review, and finalize the test scenarios before performing any tests. Here, the development team also needs to determine how heavy the load that will be put on the system.
The duration of the soak test is essential depending on the scope of the system. It also depends on the involvement of the development team and clients, actual production use, and more.
Consider some questions, such as: How the soak test remains consistent over use? Are there any bugs that have yet to be detected? Are there any external interferences that will cause the systems to go down?
What are the Common Issues that Soak Testing Detects?
Memory allocation (memory leaks which finally lead to a memory crisis or rounding failures that only display over time).
Database resource usage (errors when closing the database cursors under certain conditions that would eventually cause the entire system to come to a standstill).
It can also bring about a deterioration in performance, that is, to make sure that the response time after an extended period is as good as it is when the test starts.
Errors when closing connections between tiers of a multilayer system under certain circumstances that could block some or all of its modules.
The gradual deterioration in the response time of some tasks as internal data-structures are less organized over an extended test period.
Before doing any soak testing, it is important to ensure that your system works properly under normal conditions. Katalon Studio provides a simple solution for the software testing system, helping the development cycle run smoothly and guaranteeing a successful deployment.