An introduction to Microservices

author
Talend Documentation Team
EnrichVersion
6.5
EnrichProdName
Talend MDM Platform
Talend Data Services Platform
Talend ESB
Talend Real-Time Big Data Platform
Talend Data Fabric
task
Administration and Monitoring
Deployment
Design and Development > Designing Routes
EnrichPlatform
Talend Studio
Talend Administration Center
Talend ESB
Talend Runtime
Talend CommandLine

An introduction to Microservices

Microservices are an architecture style used by many organizations to go beyond Service Oriented Architecture. Microservices are designed to achieve a high degree of agility, fast delivery, and to scale.

Overview

In this article, we will go through a step-by-step guide to develop a microservice with Talend Studio.

Pre-requisites:

You will need Talend ESB 6.2 or later to be able to follow this guide. Talend introduced the ability to build microservices since version 6.2. Also, due to the nature of microservices, you can only build microservices with Mediation routes components in Talend 6.2 as you will see later in this guide.

Microservice

Microservices are an architectural style or an approach to building IT systems. These are considered as a set of business capabilities that are autonomous, self-contained, and loosely coupled.

As for communication or transport among microservices, there is no agreed standard. In general, microservices communicate with each other using widely adopted lightweight protocols, such as HTTP and REST, or messaging protocols, such as JMS or AMQP.

As microservices are more aligned to business capabilities and have independently manageable life cycles, they are the ideal choice for enterprises embarking on DevOps and cloud. DevOps and cloud are two facets of microservices.

Scenario

In this example, we will build three Microservices using Talend Studio and will monitor them using Talend Runtime and Talend Administration Center. We will expose a rest interface that will deliver customer data.

An Audit Service listening on a queue will display message on the console. The third service will call the rest interface to retrieve the customer data. Normally, microservices should only be limited to be "Provider" services.

However, we are using third service to demonstrate rest communication between two microservices.

Designing Microservices with Talend Studio

It is very easy to build microservices using Talend Studio. Talend leverages the Spring Boot features to export any service as an independent and self-managed service. Before we start programming microservices, we need to set the maven preferences in Talend studio.

Under Maven section of Preferences in studio, uncheck Offline. This option will allow the studio to go online and download all necessary jar files during the build of the microservice.

Please make sure that your system has Java 1.8 installed and JAVA_HOME variable is set correctly.

Audit Service

The Audit Service will listen to a JMS queue and will simply log the message on the console.

Procedure

  1. Create a new route and place the components as displayed below.
    You have cMQConnectionFactory, cJMS, cProcessor and cMessagingEndpoint components in the designer canvas.

    You need to configure the Broker URL of the MQ Connection Factory to your Active MQ Broker Server which is tcp://localhost:61616, in this example.

    You are running a local instance of ActiveMQ which is part of Talend ESB. The cJMS component is configured to listen the queue audit.

  2. Use the cProcessor component to enrich the camel Exchange.
    String audit = "Audit Message Received: "+ exchange.getIn().getBody(String.class);                    
  3. Configure the cMessagingEndpoint component as a Camel Log component.
    "log:AuditLog?showAll=true&multiline=true"

    The Audit Service will now listen to the Queue audit and will log it on the console. This message can be saved to the database for auditing purposes. You can run the route to verify that it compiles and runs correctly at this point.

  4. To export the Audit Service as a Microservice, right click on the route and select Build Route.
  5. Change build type to ESB Microservices (Spring Boot) and save it.
  6. To run Microservice, locate the 'Audit****.jar file where you have saved it and run it using java -jar command.
    java -jar AuditService_0.1.jar --server.port=8066  

    By default, the monitoring port is 8605 which can be overwritten by passing simple parameter --server.port=port to the java command starting the service.

    This port is used to monitor the status of the service and is discussed in next section.

    The microservice can be exported as a zip file that contains the Windows and Linux scripts (.bat and .sh) to help with starting up the microservice. This functionality is available from Talend 6.3.1.

Customer Service Provider

This service will expose a rest interface which will return random customers list as a JSON string.

Procedure

  1. Place the following components as shown in the screenshot below:
    • cBeanRegister - cMQConnectionFactory - cConfig

    • cCXFRS - cSetBody - cWireTap

    • cSEDA - cJavaDSLProcessor - cJMS - cJavaDSLProcessor

  2. Configure the cBeanRegister component to customers as a Bean Id. In the code section, we are going to populate some random data.
    beans.Customers customers = new beans.Customers();
                            beans.Customer customer = new beans.Customer();
                            customer.setFirstName(TalendDataGenerator.getFirstName());
                            customer.setLastName(TalendDataGenerator.getLastName());
                            customer.setCity(TalendDataGenerator.getUsCity());
                            customers.addCustomer(customer);
                            customer = new beans.Customer();
                            customer.setFirstName(TalendDataGenerator.getFirstName());
                            customer.setLastName(TalendDataGenerator.getLastName());
                            customer.setCity(TalendDataGenerator.getUsCity());
                            customers.addCustomer(customer);
                            beanInstance = customers;    

    You have created two Beans namely Customer and Customers.

    The following is the code for the Customers Bean:

    @XmlRootElement
    public class Customers {
                        ArrayList<Customer> customers = new ArrayList<Customer>();
                        @XmlElement
                        public ArrayList<Customer> getCustomers() {
                                            return customers;
                        }
                         public void setCustomers(ArrayList<Customer> customers) {		
                                    this.customers = customers;
                        }
                          public void addCustomer(Customer customer) {
                                    if (this.customers == null) {
                                             customers = new ArrayList<Customer>();
                                    }
                                    this.getCustomers().add(customer);
                                    }
                                    }

    and the following code is for the Customer Bean:

    @XmlRootElement
    @XmlType(propOrder = { "firstName", "lastName", "city" })
    public class Customer {
           private String firstName;
           private String lastName;
           private String city;
    @XmlElement(name = "firstName")
    public String getFirstName() {
           return firstName;
           }
    public void setFirstName(String firstName) {
            this.firstName = firstName;
                            }
    @XmlElement(name = "lastName")
    public String getLastName() {
           return lastName;
    }
    public void setLastName(String lastName) {
            this.lastName = lastName;
                            }
    @XmlElement(name = "city")
    public String getCity() {
            return city;
    }
    public void setCity(String city) {
      this.city = city;
      }
    }
  3. We need to convert the data to XML format. Configure a cConfig component to include the required jar files and instantiate the Jaxb context.
    import org.apache.camel.spi.DataFormat; 
    import org.apache.camel.converter.jaxb.JaxbDataFormat;
    import javax.xml.bind.*;
    
    JAXBContext jc = JAXBContext.newInstance( beans.Customers.class);
    DataFormat jaxb = new JaxbDataFormat(jc);
  4. Configure the cCxFRS component to listen on the endpoint http://127.0.0.1:8041/services/customers. Select the option Produces JSON in Rest API Mapping.
  5. Configure the cSetBody component to call the customers Bean as shown in the screenshot below.
  6. Connect the cSetBody to the cWireTap component to send an audit message to the seda component. Configure the cWireTap with URI seda:toAudit.
  7. Connect the cSEDA component with the cJavaDSLProcessor and configure it with the code .marshal(jaxb) as shown below. Jaxb is the DataFormat handler you already defined in the cConfig component.
  8. Connect with the cJMS and again with the cJavaDSLProcessor with Code .end(). Configure the Queue name audit.
  9. Run the Route and test it with a browser. You should get a customer list in JSON format as shown below.

    Now instead of deploying service to a Talend Runtime server, we will export it as a microservice.

  10. Right click on the route and select Build Route.
  11. In Build type, select ESB Microservices (Spring Boot) and click Finish.
  12. Locate the jar file and run it with a java command.
    Java -jar CustomerProducer***.jar 

Customer Service Consumer

In this example, you will consume a rest service to demonstrate how to call a rest service.

Procedure

  1. Place the cTimer, cCXFRS and cLog components on the studio canvas.
  2. Configure the cCXFRS component to call the service you just executed as a microservice.
  3. Just like before, build and export the service as a microservice. Start the service by calling the java -Jar command. The service will log customer data on consumer console as shown below.
    As the consumer service calls the provider service, the Audit service will log an audit message on the console.

Context Variables

Context variable options is also available from Talend 6.3.1.

They can be set in context file or call via script with the --context_param <param-name>=<param-value>’ as inline parameters.