Hands-On Testing the Circuit Breaker Pattern in Microservices

by

Introduction

Microservices involve multiple services working together. Sometimes, a service might fail, and repeated calls to it can cause bigger problems. The Circuit Breaker Pattern helps stop calls to a failing service and gives it time to recover. It makes systems more stable and reliable by preventing overloading, reducing delays, and optimizing resource usage. However, to ensure that the circuit breaker is functioning as expected, it is important to validate its behavior using various testing approaches.

Problem Statement

Microservices systems often face:

  • Overloading: Failing services can be overwhelmed by repeated requests.
  • Delays: Slow responses can make the whole system seem unresponsive.
  • Resource Waste: Retrying failed calls can waste resources like CPU and memory.
  • Cascading Failures: When one service fails, it can affect multiple dependent services.

Solution: Circuit Breaker Pattern

The Circuit Breaker pattern addresses these challenges by monitoring failures and temporarily halting requests to a failing service until it recovers.

How Circuit Breakers Work:

  1. Normal State (Closed): Calls go to the service as usual.
  2. Open State: Calls are stopped if the service is failing.
  3. Half-Open State: A few test calls are made to check if the service has recovered.

By using circuit breakers, services can prevent cascading failures, improve system stability, and allow faster recovery.

A full working example is available here

1. Service Setup (Spring Boot Example)

PaymentServiceApplication.java (Main Class)

@SpringBootApplication
public class PaymentServiceApplication {
    public static void main(String[] args) {
        SpringApplication.run(PaymentServiceApplication.class, args);
    }
}

PaymentController.java (Service Controller)

@RestController
public class PaymentController {

    private final PaymentService paymentService;

    public PaymentController(PaymentService paymentService) {
        this.paymentService = paymentService;
    }

    @GetMapping("/process-payment")
    public String processPayment(@RequestParam double amount) {
        return paymentService.makePayment(amount);
    }
}

2. Implementing Circuit Breaker

PaymentService.java (Circuit Breaker Implementation)

@Service
public class PaymentService {

    private final RestTemplate restTemplate;
    private final CircuitBreaker circuitBreaker;

    public PaymentService() {
        this.restTemplate = new RestTemplate();

        CircuitBreakerConfig config = CircuitBreakerConfig.custom()
                .failureRateThreshold(50)  // Open circuit if 50% failures occur
                .waitDurationInOpenState(Duration.ofSeconds(5))  // Wait 5 seconds before retry
                .slidingWindowSize(10)  // Evaluate last 10 calls
                .build();

        this.circuitBreaker = CircuitBreaker.of("paymentServiceCB", config);
    }

    public String makePayment(double amount) {
        Supplier<String> paymentSupplier = CircuitBreaker.decorateSupplier(circuitBreaker, () -> callExternalPaymentApi(amount));
        return paymentSupplier.get();
    }

    private String callExternalPaymentApi(double amount) {
        String paymentServiceUrl = "http://localhost:8080/external-payment?amount=" + amount;
        String response = restTemplate.getForObject(paymentServiceUrl, String.class);
        if (amount > 1000) {
            throw new RuntimeException("Transaction limit exceeded");
        }
        return response;
    }
}

Validating Circuit Breaker Functionality

Example of Unit Testing

PaymentServiceTest.java

@SpringBootTest
public class PaymentServiceTest {

@Autowired
    private PaymentService paymentService;
    @Test
    public void testCircuitBreakerOpensOnFailures() {
        for (int i = 0; i < 10; i++) {
            try {
                paymentService.makePayment(1500);
            } catch (Exception e) {
                System.out.println("Expected Failure: " + e.getMessage());
            }
        }
        assertEquals(CircuitBreaker.State.OPEN, paymentService.getCircuitBreakerState());
    }
}

Example of Integration Testing

PaymentControllerTest.java

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@AutoConfigureMockMvc
public class PaymentControllerTest {

    @Autowired
    private MockMvc mockMvc;

    @Test
    public void testProcessPaymentEndpoint() throws Exception {
        mockMvc.perform(get("/process-payment?amount=500"))
                .andExpect(status().isOk())
                .andExpect(content().string(containsString("Payment Processed Successfully")));
    }
}

Example of Functional Testing (Using Postman)

  1. Open Postman and make a GET request to:
http://localhost:8080/process-payment

2. Observe the response for successful or fallback messages.

3. Use Postman’s Runner feature to simulate multiple requests and verify circuit breaker behavior.

Example of Load Testing (Using Gatling)

Example of Gatling Script

import io.gatling.core.scenario.Simulation
import io.gatling.core.Predef._
import io.gatling.http.Predef._

class CircuitBreakerSimulation extends Simulation {

  val httpProtocol = http.baseUrl("http://localhost:8080")

  val scn = scenario("CircuitBreakerTest")
    .exec(http("Process Payment")
    .get("/process-payment?amount=500")
    .check(status.is(200)))

  setUp(
    scn.inject(atOnceUsers(100))
  ).protocols(httpProtocol)
}

Run the above script using Gatling and analyze the generated reports to verify circuit breaker behavior.

Chaos Engineering with Chaos Monkey

Step 1: Add Chaos Monkey Dependency

Add the following dependency in pom.xml:

<dependency>
    <groupId>de.codecentric</groupId>
    <artifactId>chaos-monkey-spring-boot</artifactId>
    <version>2.5.0</version>
</dependency>

Step 2: Configure Chaos Monkey

Add the following to application.properties:

management.endpoints.web.exposure.include=*
management.endpoint.chaosmonkey.enabled=true
chaos.monkey.enabled=true
chaos.monkey.assaults.latency-active=true
chaos.monkey.assaults.latency-range-start=1000
chaos.monkey.assaults.latency-range-end=3000
chaos.monkey.assaults.exceptions-active=true
chaos.monkey.assaults.exception-rate=50
chaos.monkey.watcher.controller=true
chaos.monkey.watcher.service=true

Step 3: Running Chaos Monkey

Start the application:

mvn spring-boot:run

Enable Chaos Monkey attack via:

curl -X POST "http://localhost:8080/actuator/chaosmonkey/enable"

Observe latency and fallback responses.

Step 4: Monitoring Chaos Impact

Access actuator metrics:

curl "http://localhost:8080/actuator/metrics/resilience4j.circuitbreaker"

Conclusion

The Circuit Breaker Pattern plays a crucial role in ensuring system stability by preventing repeated calls to a failing service. By implementing robust testing strategies right from the development face is critical in ensuring the reliability of the system Unit, Integration and using tools like Postman, Gatling, Resilience4j, and Chaos Monkey, we can verify that the circuit breaker is functioning correctly, ensuring improved system reliability and performance under failure conditions.

Publication & Disclaimer : This article is published on both [My Blog] and [Medium]. The views expressed in this article are my own and do not reflect the views of my employer.

Leave a Comment