You can create a semantic type based on a regular expression in Talend Dictionary Service and add it to the list of recognized data types in Talend Data Stewardship.
In Talend Dictionary Service, not every type of data can currently be matched with and validated against one of the predefined semantic types. Italian social security numbers, also known as codice fiscale, are currently not recognized for example.
Let's say that you work for an Italian company, only dealing with Italian customers. In this example, you need to intervene and manage some customer data, such as their names, email address, or their social security number.
When defining the data model in Talend Data Stewardship, you will be obliged to
set the semantic type for the column containing the social security number to
text as there is no predefined semantic type for Italian social
security number. This is a bit disappointing and you would like to create a more
specific category in order to match this type of data: a
codice_fiscale semantic type in this case.
You will create this new semantic type in Talend Dictionary Service, and it will be automatically available in Talend Data Stewardship so that your data can be matched with and validated against a proper type.
Create a .txt file containing the following regular
expression and save it as REGEX_CODICE_FISCALE.txt.
This regular expression is designed to match the Italian codice fiscale, which is an alphanumeric code of 16 characters.
Add this file to the
This folder is used for the sake of this example, but you can save it to your preferred location.
Open a command prompt window and use the
cdcommand to go to the <Dictionary_Service_Path>/command-line folder.
To create the new
codice_fiscalesemantic type in Talend Dictionary Service and configure its different parameters, put the following command in one single line and execute it according to your operating system:
You are prompted for your Talend Administration Center credentials. The command is executed after you enter a valid login and password.
category_manager.bat -c -name codice_fiscale -type REGEX -desc "Italian social security number" -src samples\source\REGEX_codice_fiscale.txtfor Windows.
./category_manager.sh -c -name codice_fiscale -type REGEX -desc "Italian social security number" -src samples/source/REGEX_codice_fiscale.txtfor Linux.
codice_fiscalesemantic type is now added to the list of categories in Talend Dictionary Service.
Go back to Talend Data Stewardship and create the data model for the Italian customers data.
The new semantic category
codice_fiscaleis available now in the list of semantic types and you can set it for the column containing the social security number.
codice_fiscalesemantic type, that you created in Talend Dictionary Service.
category_manager.bat -hcommand for Windows.
./category_manager.sh -hfor Linux.