Comparing Karate Framework with Others for Automated Regression Testing: A Case Study of PT Fliptech Lentera Inspirasi Pertiwi
Abstract
In the rapidly evolving digital era, applications, and software systems increasingly rely on Application Programming Interfaces (APIs) to enable interaction, integration, and functionality extension. However, manual testing of APIs is often inefficient and challenging to reuse when changes occur. To address this, automation testing has become a more effective choice, where test scripts can verify and execute tests repeatedly, easily adapting to API changes. Essentially, automation testing plays a vital role in software maintenance, particularly in regression testing, which tests modified or upgraded software versions to ensure that their core functions remain unchanged and unaffected. One approach to automation testing is employing the Software Testing Life Cycle (STLC), which follows a systematic series of stages conducted by the testing team to ensure software product quality. This paper utilizes PT Fliptech Lentera Inspirasi Pertiwi’s public API to conduct testing on 25 scenarios from two modules. The objective is to utilize the Karate Framework to conduct these automated regression tests, resulting in an impressively short testing duration, averaging only 42.645 seconds, or approximately 1.706 seconds per scenario. A comparison with the Behave framework, using the same scenarios but with differences in steps, reveals that Behave achieves a duration of 18.762 seconds, or 0.750 seconds per scenario, making it 127.295% faster than Karate. However, in terms of the number of steps, Behave covers only 188, while Karate includes 543. This means that Behave requires 0.100 seconds per step, while Karate necessitates 0.079 seconds per occurrence. Karate provides more detailed results by 188.830% per step or 26.582% in terms of step duration. The primary goal is to enhance testing efficiency, expedite issue identification and resolution, provide a clearer testing process, and potentially improve overall software quality.
Downloads
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike International License (CC-BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
Copyright without Restrictions
The journal allows the author(s) to hold the copyright without restrictions and will retain publishing rights without restrictions.
The submitted papers are assumed to contain no proprietary material unprotected by patent or patent application; responsibility for technical content and for protection of proprietary material rests solely with the author(s) and their organizations and is not the responsibility of the ULTIMATICS or its Editorial Staff. The main (first/corresponding) author is responsible for ensuring that the article has been seen and approved by all the other authors. It is the responsibility of the author to obtain all necessary copyright release permissions for the use of any copyrighted materials in the manuscript prior to the submission.
