Set up an HCatalog table and partition - 7.0

Big Data Job Examples

author
Talend Documentation Team
EnrichVersion
7.0
EnrichProdName
Talend Big Data
Talend Big Data Platform
Talend Data Fabric
Talend Open Studio for Big Data
Talend Real-Time Big Data Platform
task
Design and Development > Designing Jobs
Design and Development > Designing Jobs > Hadoop distributions
Design and Development > Designing Jobs > Job Frameworks > Standard
EnrichPlatform
Talend Studio

Procedure

  1. Double-click the second tHCatalogOperation component, labelled HCatalog_CreateTable in this example, to open its Basic settings view on the Component tab.
  2. Define the same HCatalog connection details using the same procedure as for the first tHCatalogOperation component.
  3. Click the Schema list box and select Repository, then click the [...] button next to the field that appears to open the [Repository Content] dialog box, expand Metadata > Generic schemas > access_log and select schema. Click OK to confirm your choice and close the dialog box. The generic schema of access_log is automatically applied to the component.

    Alternatively, you can directly select the generic schema of access_log from the Repository tree view and then drag and drop it onto this component to apply the schema.

  4. From the Operation on list, select Table; from the Operation list, select Drop if exist and create.
  5. In the Table field, enter a name for the table to be created, weblog in this example.
  6. Select the Set partitions check box and click the [...] button next to Edit schema to set a partition and partition schema.

    The partition schema must not contain any column name defined in the table schema. In this example, the partition schema column is named ipaddresses.

  7. Upon completion of the component settings, press Ctrl+S to save your Job configurations.