Karate is an open-source general-purpose test-automation framework that can script calls to HTTP end-points and assert that the JSON or XML responses are as expected. Karate also has support for service-virtualization where it can bring up "mock" (or stub) servers which can substitute for web-services that need to participate in an integration-test. Karate's capabilities include being able to run tests in parallel, HTML reports and compatibility with Continuous Integration tools.
This page explains how to setup the API testing framework built with Java and Karate.
Set an environment variable named
BASE_URL (e.g. if testing locally set it to http://localhost:9000) or use the
export BASE_URL=http://localhost:9000 command.
Running the tests
To run all functional tests please navigate to
./api-tests-karate path and execute the following command:
To run the Smoke tests independently, use:
Run Functional tests on the specific environment, use:
Run the specific feature by tag option
Karate allows running tests by a specific tag. To do that please set your tag to the specific feature or test, and then specify it in the Runner Class as follows:
Switching the Environment
There is only one thing you need to do to switch the environment - which is to set a Java system property. By default, the value of karate.env when you access it within karate-config.js - is base_url value taken from the environment variables
Running tests in parallel
Choosing the right forking strategy and parallel execution settings can have a substantial impact on the memory requirements and the execution time of the build system. The tests can be easily grouped in features and tags to run and compose test-suites in a very flexible manner.
In rare cases you may want to suppress the default of Scenario-s executing in parallel and the special tag @parallel=false can be used. If you place it above the Feature keyword, it will apply to all Scenario-s. And if you just want one or two Scenario-s to NOT run in parallel, you can place this tag above only those Scenario-s.
For convenience, some stats are logged to the console when execution completes, which should look something like this:
Any *.feature file tagged as @ignore will be skipped - as the ~ prefix means a "NOT" operation. You can also specify tags on the command-line. The tags() method from test runner class also takes multiple arguments,for e.g.
this is an "AND" operation: tags("@customer", "@smoke", "~@ignoredTags")
and this is an "OR" operation: tags("@customer, @smoke, ~@ignoredTags")
The more threads are used, the higher the chances of sporadic, hard-to-reproduce test failures due to timeouts, test data clean up, and other related issues. The added performance gain of each thread also tends to drop off for higher numbers of threads.
As a result of the test execution - karate will automatically generate the test report -