Introduction to the Circuit Breaker Pattern

Consider that you’re running a web service that requires input and delivers it to another backend service. If the backend service is not available for some time, then what kind of fail-proof system should you implement? This is where the Circuit Breaker design pattern comes in.

Let’s think about a situation where a request goes to a middleware application and you need to call another remote backend service. If everything goes fine and smooth, the application can forward the request to the backend service and send the response back to the client. But, if the backend service was down, the request could not be completed. The next few requests from the client also try to call the backend service and fail. Here, we can use the Circuit Breaker model to manage backend errors. According to the Circuit Breaker pattern, the middleware may be in the following possible states.

• Closed State: Just like in electronic circuits, the closed state is the operating state. Requesting a flow through the specified route works and results in no system failures. This status means that the application is up and running. Here, the client’s application was sent back to the backend and the reply was sent back to the client.
• Open State: The client request goes to the middleware application, but the backend server is not available. The circuit is in the open state, the same as in electronic circuits. Once the application enters this state, no upcoming requests will be sent to the backend server for a specified timeout. Instead of attempting to call the backend service, the application responds to the client with an error message.
• Half-Open: The state after the application is in the Open State for a given timeout. In this state, the application attempts to send the request back to the backend again and to verify whether the backend service is accessible or not. If it is accessible, the request will move to the closed state. If the backend is still inaccessible, it will go back to the open state.

Why the Circuit Breaker Pattern?

Consider that you have a backend application that executes a database query and sends the outcome back to the client. Imagine clients are sending 10,000 TPS (transaction per second) to your system. The backend server crashes and waits for recovery, but the client is still trying to send a huge amount of traffic to the backend server that is still trying to recover. In this type of situation, the system should be designed in such a way that it waits for some time until the service is available. Once the backend service is available, requests can be sent again.

This pattern is particularly helpful in situations such as database migration and software updates.

Circuit Breakers in Microservices

Containers/VMs are dynamic things in microservice architecture. The container could vanish, and in the meantime, another container could be launched. The whole concept of microservices is that there might be a component failure, and the Microservice architecture itself handles all of these failures. Expecting errors will make your software architecture more resilient.

Here are some practical complex scenarios where NGINX load balancing circuit breakers are used. NGINX Microservices Reference Architecture has a service called “Resizer” that can resize, rotate, and shrink an image when the image is uploaded to the system. This resizer operation is a CPU- and memory-intensive job that can cause out of memory errors. Here, we can place the Circuit Breaker between the Resizer service and the Image Uploader service.

Here, the Uploader service examines the health status of the Resizer service and checks whether Resizer has enough memory to execute operations. If NGINX has discovered the service to be defective, the Uploader will distribute loads. Once the crashed Resizer service becomes healthy, NGINX slowly ramps back to the restarted Resizer service.

Implementing the Circuit Breaker Pattern

Here is the practical application of the WSO2 Integrator Circuit Breaker Patten. WSO2 Integrator is an open-source middleware platform used to connect distinct web services. The integrator is between two endpoints. One endpoint (client) sends a request to the other endpoint (backend) and the integrator performs the required conversion through a sequence of mediators before sending it to the backend endpoint. This scenario can be implemented with the following configurations on the WSO2 Integrator.

xxxxxxxxxx

<api name="ServiceAPI" context="/cal">  

   <resource methods="GET">  

    <inSequence>  

      <send>  

       <endpoint key="CircuitBreakerEP"/>  

      </send>  

    </inSequence>  

    <outSequence>  

      <send/>  

    </outSequence>  

    <faultSequence>  

      <header name="To" action="remove"></header>  

      <property name="RESPONSE" value="true"></property>  

      <property name="NO_ENTITY_BODY" scope="axis2" action="remove"></property>  

      <log level="full"></log>  

      <payloadFactory media-type="xml">  

       <format>  

         <ns:MyResponse xmlns:ns="http://services.samples">  

          <ns:Error>We can't response you at this time, we will reponse through E-mail soon</ns:Error>  

         </ns:MyResponse>  

       </format>  

      </payloadFactory>  

      <send/>  

    </faultSequence>  

   </resource>  

 </api>

Here we identify the ServiceAPI API that forwards the incoming application to the backend endpoint. Incoming requests are processed by the mediator `<inSequence>`. Inside this mediator, redirect to the endpoint name “TimeoutEP.” Response to this endpoint will be directed to the `<outSequence>` mediator. Here, the backend reply is sent back to the client. Here, the `<faultSequence>` section is used to record the failure information.

The endpoint definition would be as follows.

xxxxxxxxxx

<endpoint name="CircuitBreakerEP">  

   <address uri="http://localhost:9764/CalculatorService-war_1.0.0/services/calculator_service/call">  

    <suspendOnFailure>  

      <initialDuration>40000</initialDuration>  

    </suspendOnFailure>  

    <markForSuspension>  

      <errorCodes>101507,101508,101505,101506,101509,101500,101510,101001,101000,101503,101504,101501</errorCodes>  

      <retriesBeforeSuspension>3</retriesBeforeSuspension>  

      <retryDelay>400</retryDelay>  

    </markForSuspension>  

    <timeout>  

      <duration>200</duration>  

      <responseAction>fault</responseAction>  

    </timeout>  

   </address>  

 </endpoint>

In the endpoint section, we can define timeout parameters for the Circuit Breaker pattern. Address URI is the backend URI where the request redirects.

• initialDuration: This property used to define how long (in milliseconds) to wait until retrying the backend service.
• retriesBeforeSuspension: Number of retries before going into the suspended state (Open Circuit state).
• retryDelay: delay between two failure calls.

Conclusion

The Circuit Breaker design pattern is a basic pattern used in both monolithic- and microservice-based deployments. It helps the system prevent sending unnecessary loads to a failed backend service. It provides some delay to the backend service in order to recover from errors. It plays a main part in a microservice architecture, as the containers are dynamic and could die at any time. The middleware platform that is used to manage traffic should be prepared to manage the application when the backend server goes down. In short, Circuit Breaker patterns provide an elegant way to handle errors.