Developing a component using Talend Component Kit

author
Talend Documentation Team
EnrichVersion
7.0
EnrichProdName
Talend Big Data
Talend Real-Time Big Data Platform
Talend Open Studio for Data Integration
Talend MDM Platform
Talend Data Management Platform
Talend Open Studio for Big Data
Talend Open Studio for ESB
Talend Data Integration
Talend Data Fabric
Talend Data Services Platform
Talend Big Data Platform
Talend ESB
Talend Open Studio for MDM
task
Design and Development > Designing Components
EnrichPlatform
Talend Studio

Developing a component using Talend Component Kit

Talend Component Kit provides a scalable component development framework with a rich set of pre-defined classes, methods and functions, to streamline your component development process and make your component easily portable across Talend products.

This framework is required to develop components that can obtain optimal performance in different Talend tools.

In this tutorial, a component is developed, for demonstration purposes, to read data from the map data structure of a Hazelcast database.

Once the component is created using Talend Component Kit, you are able to seamlessly use it with the components provided along with the Studio. This allows you to graphically develop Jobs to read data from a Hazelcast database in a distributed manner and visually handle the data in the unified Talend tools.

Hazelcast is a distributed, in-memory computing platform. If you want to learn further information about this company and their products, see Hazelcast.

Designing a component

Analyze the application API of Hazelcast to design the component
You need to analyze the application API from the following aspects:
  • What kind of data entities are accessible for processing?

  • What properties should be defined to support performing operations on these data entities? A common principal is all potentially user-defined properties must be part of the component configuration interface. These properties are often split into a Basic settings group for the frequently used properties and an Advanced settings group for the other properties.

  • You also need to ensure that the code structure fits your back-end configuration structure.

As a result of this analysis, you can find that this component needs the following properties to read data from Hazelcast:
  1. Basic settings:
    • hazelcastXml to read the configuration XML file of Hazelcast.

    • mapName to read the Hazelcast Map to be used.

  2. Advanced settings:

    • executorService to indicate the name of the executor service to be used to launch threads.

    • timeout to define the execution timeout of a Hazelcast task.

You can then imagine that this component could look as is displayed in the following image:

To implement this design, Talend Component Kit provides the following tools:
  • Component Kit Starter (often referenced as the Starter in this tutorial). This is a graphical interface to help you create your component development project, declare the properties of the component and generate the skeleton for further development.

  • Component configuration templates. They are the configuration classes generated by the Starter for you to register the family name, the icon information of the component and define the layout of the component properties.

  • Component runtime templates. They are the mapper class and the producer class generated by the Starter for you to define how the component accesses the data to be handled using the component configuration, splits its work in parallel tasks and thus efficiently produces input data to a processing flow.

  • Component test template. This is a test class generated by the Starter for you to define a test for your component.

All these templates contain annotations to guide you through the development of the methods required by a Talend component.

Creating the skeleton of your component

Use the Component Kit Starter to automatically create the skeleton of a Talend component so that you can simply update this skeleton to create your component.

Before you begin

Ensure the following is setup:
  • Java 1.8.x has been properly installed in your machine.

  • Maven 3.5.x has been properly installed in your machine.

  • A Java Integrated Development Environment has been properly set up such as Eclipse or IntelliJ.

  • The version of your Studio is 7.0 and onwards.

Procedure

  1. Create a folder to store the resource files of the component you want to create. For example, C:/my_components.
  2. In your Internet browser, enter component starter to open the Talend Component Kit Starter in the browser.
  3. In the first step of the Starter, enter the metadata for your component development project.
    1. Select Maven to display the corresponding configuration interface.
    2. Click the Select a facet to add to your project field to display the facet list and then select the Talend Component Kit Testing facet.
    3. In the Component Family field, enter the family name the most specific to the component you want to develop. This name must be a valid java name and is recommended to be capitalized, for example Hazelcast.

      This name will not only be displayed in the Palette of your Studio to group all of the Hazelcast related components you could develop, but also be a part of your component name.

    4. In the Category field, select the category to which your specific component family should belong. As Hazelcast is a kind of database, select Databases in this tutorial.

      This Databases category will be used and displayed as the parent family of the Hazelcast group in the Palette of the Studio.

    5. In the Project Metadata area, enter the Maven required metadata for your component project. You can enter the same metadata as is presented in the image above.
    6. Click ADD A COMPONENT to start to create your component.
  4. In the second step of the Starter, define how your component is generated by the Starter.
    1. Click the INPUT button to create an input component to read data from Hazelcast.
    2. In the Name field, enter the technical name of the component to be developed. In this tutorial, enter Input.

      Talend Component Kit concatenates the specific family name you entered in the previous step of the Starter and this technical name to create the eventual component name to be displayed in the Studio.

      Using the information you have entered, the component name in this tutorial will be HazelcastInput.

      This technical name Input is also used to generate the classes of your component.

    3. Click CONFIGURATION MODEL to display the configuration blade of this component to the right. This blade is used to declare the properties your component needs to use to perform actions.
    4. In the blade, click the [+] button four times to add four entries.
    5. Click the first row and in the field that is displayed, enter hazelcastXml and leave the default type: String.
    6. Click the tick button to save the changes.
    7. Do the same on the other rows to add mapName, executorService and timeout, but select the Integer type for timeout.
    8. Leave the Stream toggle to be off because this HazelcastInput component is developed to run in a traditional data integration Job.
    9. From the Record Type drop-down list, select Generic to be the type of the records your component loads in a Job because this component generates JSON records.
    10. Click GO TO FINISH to save your changes and pass to the Finish step.
  5. In the Finish step, click DOWNLOAD AS ZIP to download your project as a zip file to C:/my_components.
  6. Unzip your project and from the root of your project, open the pom file and append the following dependency to the other dependencies in this file:
    <dependency>
        <groupId>com.hazelcast</groupId>
        <artifactId>hazelcast-client</artifactId>
        <version>3.9.3</version>
    </dependency>

Defining component properties

The component uses these properties to read data from Hazelcast.

Procedure

  1. Import the your component skeleton into Eclipse or IntelliJ and if being asked, allow your IDE to import the dependencies.

    As the skeleton is hosted in the Maven project, you need to import this Maven project from its parent level, that is to say, from the directory in which the pom file is stored.

  2. Open package-info.java to verify the family registration of your component. This class contains the family information generated by the Starter. Leave the lines as they are because they should reflect exactly the information you provided to the Starter in the previous steps.
    @Components(family = "Hazelcast", categories = "Databases")
    @Icon(value = Icon.IconType.CUSTOM, custom = "Hazelcast")
  3. Open org.talend.components.hazelcast.source.InputMapperConfiguration to define the component properties: hazelcastXml, mapName, executorService and timeout.
    1. Set the layout of the properties. In this example, move mapName and hazelcastXml to the same row to make them appear on the same row in the user interface of your component. Then move the other properties to a new @GridLayout line in which these properties are defined as advanced properties.
      @GridLayout({
              // the generated layout put one configuration entry per line,
              // customize it as much as needed
              @GridLayout.Row({"hazelcastXml", "mapName"}),
      })
      
      @GridLayout(names = GridLayout.FormType.ADVANCED, value = {
              @GridLayout.Row({"executorService"}),
              @GridLayout.Row({"timeout"}),
      })

      You need to add names = GridLayout.FormType.ADVANCED to the second @GridLayout line to declare the properties in it as advanced properties.

    2. Verify that these properties are properly declared and give default values to the executorService and the timeout properties.
      @Option
      @Documentation("TODO fill the documentation for this parameter")
      private String hazelcastXml;
      
      @Option
      @Documentation("TODO fill the documentation for this parameter")
      private String mapName;
      
      @Option
      @Documentation("TODO fill the documentation for this parameter")
      private String executorService = "default";
      
      @Option
      @Documentation("TODO fill the documentation for this parameter")
      private Integer timeout = 5000;
      • Talend Component Kit recognizes all the properties annotated with @Option as configuration elements of a component, and binds them with pre-defined widgets to determine, depending on the types of these properties, how these properties are displayed in the user interface of the component.

        For details about the available pre-defined widgets, see the Gallery of Talend widgets in the Component Kit documentation on GitHub.

      • The @Documentation lines are generated by the Starter for you to add documentation in English for your code. If you do not want to add any documentation, leave these lines as they are . Do not remove these lines as the @Documentation lines are required by Talend Component Kit.

        For further explanations about @Documentation and how it is used by the framework to generate the documentation for your component, see the Component Kit documentation on GitHub.

        The same rule applies on all the code you develop using Talend Component Kit.

    3. Add a utility method to convert the configuration with these properties to a Hazelcast client configuration object.
      ClientConfig newConfig() throws IOException { 
             final ClientConfig newconfig = hazelcastXml == null ? new XmlClientConfigBuilder().build() :
                 new XmlClientConfigBuilder(hazelcastXml).build();
      
             newconfig.setInstanceName(getClass().getSimpleName() + "_" + UUID.randomUUID().toString());
             newconfig.setClassLoader(Thread.currentThread().getContextClassLoader());
             return newconfig;
      }
    4. Leave the other lines of this class as they are.
      These lines are about the getters and setters generated by the Starter.
  4. Open resources/org/talend/components/hazelcast/source/Messages.properties to define display name of these properties on the user interface of the component.
    InputMapperConfiguration.executorService._displayName=Executor service
    InputMapperConfiguration.hazelcastXml._displayName=Hazelcast XML
    InputMapperConfiguration.mapName._displayName=Map name
    InputMapperConfiguration.timeout._displayName=Execution timeout
    configuration._displayName=Configuration
    Hazelcast.Input._displayName=Reads data from Hazelcast.

Results

The component properties are defined. Then you need to define a partition mapper class for the component to read data in a distributed manner.

Developing the component partition mapper

A Talend Input component requires a partition mapper class. A properly defined partition mapper class allows your component to efficiently work in a distributed manner.

At runtime, this class heuristically estimates the size of the data to be handled and based on this estimation and the capacity of the execution engine of your component, it divides the work of your component to reduce the overall execution time.

For more information about this partition mapper, see the Component Kit documentation on GitHub.

Defining the mapper class

The Starter has generated the template of this partition mapper. Fill this template with your own code.

Procedure

  1. Open org.talend.components.hazelcast.source.InputMapper.
  2. Verify the name of the partition mapper the Starter has generated in the @PartitionMapper line. Although you can change it as you want, leave it as is in this tutorial.
    @PartitionMapper(name = "Input")

    By default, this name is the same as the technical name of the component, Input in this tutorial, you gave in the Starter.

  3. If you have created a custom icon for your component, name this icon as follows: custom_name_icon32.png, for example, hazelcastinput_icon32.png, place this icon in resources/icons and change the @Icon line to read as follows:
    @Icon(value = Icon.IconType.CUSTOM, custom = "hazelcastinput")

    The value of custom must be identical with the custom_name part of your icon name in order for Talend Component Kit to recognize your custom icon. If you leave the default line generated by the Starter, a default icon is used for your component in the Studio.

  4. Add a Jsonb property and accept the proposal of your IDE to import the corresponding libraries. This Jsonb tool is used to transform the Hazelcast records to the JSON format.
    private final Jsonb jsonb;
  5. Add a Collection property to the constructor of this mapper and accept the proposal of your IDE to import the corresponding libraries. This attribute is used to construct a list of Hazelcast cluster members to be used by the mapper later.
    private final Collection<String> members;

    For further information about Hazelcast cluster members, see Discovering Cluster Members.

  6. Add the Jsonb and the Collection properties to the public constructor of this mapper.
    1. Update the public constructor of this mapper: InputMapper(@Option("configuration")...) to make this constructor read as follows:

      The default constructor is generated by the Starter. It provides for the mapper the configuration, which is annotated with @Option, of your component and the services that could be either developed locally in your code or pre-defined by Talend Component Kit.

      public InputMapper(@Option("configuration") final InputMapperConfiguration configuration,
              final JsonBuilderFactory jsonBuilderFactory,
              final Jsonb jsonb,
              final Hazelcast_componentService service) {
           this(configuration, jsonBuilderFactory, service, emptyList());
      }
    2. Add an internal constructor to get the collection of the Hazelcast cluster members.
      // internal
      protected InputMapper(final InputMapperConfiguration configuration,
              final JsonBuilderFactory jsonBuilderFactory,
              final Jsonb jsonb,
              final Hazelcast_componentService service, 
              final Collection<String> members) {
          this.configuration = configuration;
          this.jsonBuilderFactory = jsonBuilderFactory;
          this.jsonb = jsonb;
          this.service = service;
          this.members = members;
      }

    At this step, the Hazelcast_componentService service is still a Starter-generated empty class but you will fill it in the following steps.

  7. Implement the method annotated with @PostConstruct to initialize a Hazelcast client instance.
    1. Add a property to the mapper for a Hazelcast client instance.
       private transient HazelcastInstance instance; 
    2. Open the org.talend.components.hazelcast.services.Hazelcast_componentService class to be used by @PostConstruct and define the logic for the creation of this Hazelcast client instance.
      import com.hazelcast.client.HazelcastClient;
      import com.hazelcast.client.config.ClientConfig;
      import com.hazelcast.core.HazelcastInstance;
      import org.talend.sdk.component.api.service.Service;
                                          
      @Service
      public class Hazelcast_componentService {
          public HazelcastInstance findInstance(final ClientConfig config) {
              return HazelcastClient.newHazelcastClient(config);
          }
      }

      A Service class is used by Talend Component Kit to load the component configuration that is not directly defined by the users of your component but is developed in the code. When you develop Service classes yourself, always annotate these classes with @Service so that Talend Component Kit recognizes these services to perform related actions.

    3. Use this Service class of your component to eventually create this Hazelcast client instance.
      @PostConstruct
      public void init() throws IOException {
          instance = service.findInstance(configuration.newConfig());
      } 
  8. Add the method annotated with @PreDestroy to clean up the resource at the end of the execution of the partition mapper. In this tutorial, this method is used to shut down the Hazelcast client instance initialized by the PostConstruct() method.
    1. Declare an execution service property for the mapper.
      private transient IExecutorService executorService;
    2. Shut down the Hazelcast client instance created by the method annotated with @PostConstruct and thus free the executorService reference.
      @PreDestroy
      public void close() { 
          instance.getLifecycleService().shutdown();
          executorService = null;
      }
  9. Add the method annotated with @Accessor to estimate the volume of the dataset to be handled. In this tutorial, this method is used to estimate the volume of all the Hazelcast members.
    @Assessor
    public long estimateSize() {
        return getSizeByMembers() 
                        .values().stream()
                        .mapToLong(this::getFutureValue) 
                        .sum(); 
    }
    
    private long getFutureValue(final Future<Long> future) {
       try {
           return future.get(configuration.getTimeout(), SECONDS);
       } catch (final InterruptedException e) {
           Thread.currentThread().interrupt();
           throw new IllegalStateException(e);
       } catch (final ExecutionException | TimeoutException e) {
           throw new IllegalArgumentException(e);
       }
    }
    • The getSizeByMembers() method submits a task to each Hazelcast member to calculate the member size locally and asynchronously.

    • The mapToLong() method gets the size calculated by that task.
    • The sum() method calculates the total size of all the members.

  10. Implement the methods annotated with @PreDestroy and the methods annotated with @Accessor to create a map of the volumes of the Hazelcast cluster members.
    1. Create the org.talend.components.hazelcast.source.SerializableTask class
      import java.io.Serializable;
      import java.util.concurrent.Callable;
      
      import com.hazelcast.core.HazelcastInstance;
      import com.hazelcast.core.HazelcastInstanceAware;
      
      public abstract class SerializableTask<T> implements Callable<T>, Serializable, HazelcastInstanceAware {
          protected transient HazelcastInstance localInstance;
      
          @Override
          public void setHazelcastInstance(final HazelcastInstance hazelcastInstance) {
              localInstance = hazelcastInstance;
          }
      }
    2. Back in the org.talend.components.hazelcast.source.InputMapper class, add the methods to create the volume map.
      private Map<Member, Future<Long>> getSizeByMembers() {
          final IExecutorService executorService = getExecutorService();
          final SerializableTask<Long> sizeComputation = new SerializableTask<Long>() {
                      
               @Override
               public Long call() throws Exception {
                      
                   return localInstance.getMap(configuration.getMapName()).getLocalMapStats().getHeapCost();
               }
          };
          if (members.isEmpty()) { // == if no specific members defined, apply on all the cluster
              return executorService.submitToAllMembers(sizeComputation);
          }
          final Set<Member> members = instance.getCluster().getMembers().stream()
                   .filter(m -> this.members.contains(m.getUuid()))
                   .collect(toSet());
          return executorService.submitToMembers(sizeComputation, members);
      }
                      
                      
      private IExecutorService getExecutorService() {
          return executorService == null ?
                  executorService = instance.getExecutorService(configuration.getExecutorService()) :
                  executorService;
      }
  11. Implement the method annotated with @Split method to dynamically split the partition mapper into a collection of sub-mappers according to the bundle size required by the execution engine to be used and the volume of the dataset to be handled.
    public List<InputMapper> split(@PartitionSize final long bundleSize) {
        final List<InputMapper> partitions = new ArrayList<>();
        final Collection<Member> members = new ArrayList<>();
        long current = 0;
        for (final Map.Entry<Member, Future<Long>> entries : getSizeByMembers().entrySet()) {
            final long memberSize = getFutureValue(entries.getValue());
            if (members.isEmpty()) {
                members.add(entries.getKey());
                current += memberSize;
            } else if (current + memberSize > bundleSize) {
                partitions.add(
                       new InputMapper(configuration, jsonBuilderFactory, jsonb, service, toIdentifiers(members)));
                // reset current iteration
                members.clear();
                current = 0;
            }
        }
        if (!members.isEmpty()) {
            partitions.add(new InputMapper(configuration, jsonBuilderFactory, jsonb, service, toIdentifiers(members)));
        }
    
        if (partitions.isEmpty()) { // just execute this if no plan (= no distribution)
            partitions.add(this);
        }
        return partitions;
    }
    
    private Set<String> toIdentifiers(final Collection<Member> members) {
        return members.stream().map(Member::getUuid).collect(toSet());
    }
  12. Implement the method annotated with @Emitter to generate producers using the component configuration to load the data to be handled to the processing flow of a Talend Job, each sub-mapper generating one producer.
    @Emitter
    public InputSource createSource() {
        return new InputSource(configuration, service, jsonBuilderFactory, jsonb, members);
    }
    Your @Emitter method is not valid until you will have properly defined the producer class in the next procedure.

Defining the producer class

The producer class use the component configuration to read the actual data from a source system, Hazelcast in this tutorial.

The Starter has generated the template of this producer class, named InputSource in this tutorial. Fill this template with your own code.

Procedure

  1. Open org.talend.components.hazelcast.source.InputSource.
  2. Add the Jsonb property and the Collection property as you did for the InputMapper class and accept the proposal of your IDE to import the corresponding libraries.
    private final Jsonb jsonb;
    private final Collection<String> members;
  3. Add the Jsonb and the Collection properties to the public constructor of this producer. Update the public constructor: public InputSource(@Option("configuration")...) to make this constructor read as follows:

    The default constructor is generated by the Starter.

     public InputSource(@Option("configuration") final InputMapperConfiguration configuration,
              final Hazelcast_componentService service,
              final JsonBuilderFactory jsonBuilderFactory,
              final Jsonb jsonb,
              final Collection<String> members) {
         this.configuration = configuration;
         this.service = service;
         this.jsonBuilderFactory = jsonBuilderFactory;
         this.jsonb = jsonb;
         this.members = members;
     }
  4. Implement a method annotated with @PostConstruct to initialize a buffer of the data to be handled. In this tutorial, this method creates a buffer of the Hazelcast records using a utility class named BufferizedProducerSupport.
    1. Add a property to the mapper for a Hazelcast client instance.
      private transient HazelcastInstance instance; 
    2. Add a buffer property.
      private transient BufferizedProducerSupport<JsonObject> buffer;

      This BufferizedProducerSupport class encapsulates the buffering logic to be iterated over the data to be handled. This way, you need only to define how to provide the data to this buffer. In this tutorial, the buffer is created in @PostConstruct and loaded once to produce the records one by one.

    3. Implement the createInstance() method annotated with @PostConstruct using the above-mentioned two properties to create the buffer.
      @PostConstruct
      public void createInstance() throws IOException {
          instance = service.findInstance(configuration.newConfig());
          final Iterator<Member> memberIterators = instance.getCluster().getMembers().stream()
                  .filter(m -> members.isEmpty() || members.contains(m.getUuid()))
                  .collect(toSet())
                  .iterator();
      
          buffer = new BufferizedProducerSupport<>(() -> {
             if (!memberIterators.hasNext()) {
                      return null;
             }
             final Member member = memberIterators.next();
             // note: this works if this jar is deployed on the hz cluster
             try {
                 return instance.getExecutorService(configuration.getExecutorService())
                          .submitToMember(new SerializableTask<Map<String, String>>() {
      
                              @Override
                              public Map<String, String> call() throws Exception {
                                 final IMap<Object, Object> map = localInstance.getMap(configuration.getMapName());
                                 final Set<?> keys = map.localKeySet();
                                 return keys.stream().collect(toMap(jsonb::toJson, e -> jsonb.toJson(map.get(e))));
                              }
                           }, member).get(configuration.getTimeout(), SECONDS).entrySet().stream()
                           .map(entry -> {
                              final JsonObjectBuilder builder = jsonBuilderFactory.createObjectBuilder();
                              if (entry.getKey().startsWith("{")) {
                                  builder.add("key", jsonb.fromJson(entry.getKey(), JsonObject.class));
                              } else { // plain string
                                 builder.add("key", entry.getKey());
                              }
                              if (entry.getValue().startsWith("{")) {
                                 builder.add("value", jsonb.fromJson(entry.getValue(), JsonObject.class));
                              } else { // plain string
                                 builder.add("value", entry.getValue());
                              }
                              return builder.build();
                           })
                            .collect(toList())
                            .iterator();
             } catch (final InterruptedException e) {
                 Thread.currentThread().interrupt();
                 throw new IllegalStateException(e);
             } catch (final ExecutionException | TimeoutException e) {
                 throw new IllegalArgumentException(e);
             }
          });
      }
  5. Add a method annotated with @Producer to load the actual data to the processing flow. @Producer is required in a producer class. This method returns null when no records are found.
    @Producer
    public JsonObject next() {
        return buffer.next();
    }
  6. Add a method annotated with @PreDestroy to clean up all the resources used in this class. In this tutorial, this method shuts down the Hazelcast instance created by @PostConstruct.
    @PreDestroy
    public void destroyInstance() {
        //We shutdown the hazelcast instance
        instance.getLifecycleService().shutdown();
    }
    Now that you have properly defined the InputSource class, the method annotated with @Emittershould become valid now in the InputMapper class. If not, ensure that you place the properties in the right order in this method with @Emitter.

Writing a test unit

Once your component is developed, write a test unit to test your component.

The Starter has generated the template of a test unit for your component. In this tutorial, this test unit is org.talend.components.hazelcast.source.InputMapperTest. Fill this template with your own code.

Procedure

  1. Open org.talend.components.hazelcast.source.InputSource.
  2. Update the default JUnit rule annotated with @ClassRule to load your component from its package.
    @ClassRule
    public static final SimpleComponentRule COMPONENT_FACTORY = new SimpleComponentRule(InputMapperTest.class
       .getPackage().getName());

    Talend Component Kit uses JUnit rules to perform this action. For further information about JUnit rules, see the Component Kit documentation on GitHub.

  3. Add an embedded Hazelcast instance to be used to test your component and accept the proposal to import the corresponding libraries.
    private static HazelcastInstance instance;
  4. Implement a startInstanceWithData() method annotated with @BeforeClass to create a Hazelcast instance and provide some test data to this instance.
    @BeforeClass
    public static void startInstanceWithData() {
       instance = Hazelcast.newHazelcastInstance();
       final IMap<Object, Object> map = instance.getMap(InputMapperTest.class.getSimpleName());
       IntStream.range(0, 100).forEach(i -> map.put("test_" + i, "value #" + i));
    }
  5. Implement a stopInstance() method annotated with @AfterClass to clean up the instance after the test.
    @AfterClass
    public static void stopInstance() { 
       instance.getLifecycleService().shutdown();
    } 
  6. Implement a testHazelcastInput() method annotated @Test to replace the default template also annotated with @test.
    @Test
    public void testHazelcastInput() {
       Job.components()
            .component("source", "Hazelcast://Input?configuration.mapName=" + InputMapperTest.class.getSimpleName())
            .component("output", "test://collector")
            .connections()
            .from("source").to("output")
            .build()
            .run();
    
        final List<JsonObject> outputs = COMPONENT_FACTORY.getCollectedData(JsonObject.class);
        assertEquals(100, outputs.size());
     }
    • The Job.component() method is provided by Talend Component Kit. This method uses the Job pipeline builder of Talend Component Kit to create a job to connect your input component to a test collector component.

      The Job pipeline builder requires a given format of URI to identify the components to be connected. In this tutorial, the URI to be used to find your component is given as follows:
      .component("source", "Hazelcast://Input?configuration.mapName=" + InputMapperTest.class.getSimpleName())

      In this line of code, Hazelcast is the family name you defined in the first step with the Starter and Input is the technical name of the component you defined in the second step with the Starter and thus is also the name of the partition mapper as is defined in the @PatitionMapper line in the InputMapper class.

      For further information about the Job pipeline builder of Talend Component Kit, see the Component Kit documentation on GitHub.

    • Once this job finishes running, the COMPONENT_FACTORY JUnit rule instance is used to get data from the collector component and evaluate the result against a given assertion.
    • The default @ignore annotation is removed. Otherwise, your test is ignored by Talend Component Kit.

Results

Once the test is done, build your component and add it to the Studio by following Building the component in the Studio.

Building the component in the Studio

Build your component in the Studio and create documentation for it.

Procedure

  1. From the root folder of your project, that is to say, the folder in which the pom file is present, run the following command.
    mvn package talend-component:deploy-in-studio -Dtalend.component.studioHome=/path/to/studio
    • Before executing this command, verify that you have updated the -Dtalend.component.studioHome parameter using the path to the root folder of your Studio.

    • If you need to skip the test phase when building the component, add -DskipTests to this command or keep the default @ignore annotation in the InputMapperTest test class.
  2. Launch your Studio. The new component is displayed under Databases > Hazelcast in the Palette and it retrieves the display information you defined in resources/org/talend/components/hazelcast/source/Messages.properties.
  3. Test it by running it in a Job.
  4. If you want to generate the documentation for this component, build it by following the explanations in the Component Kit documentation on GitHub.