Calculating the weight of a word in each message
Procedure
-
Double-click the tModelEncoder component
labeled tf to open its Component view.
-
Repeat the operations described previously over the tModelEncoder labeled Tokenizer to add the sms_tf_vect column of the Vector type to the output schema and define the
transformation as displayed in the image above.
In this transformation, tModelEncoder uses HashingTF to convert the already tokenized SMS messages into fixed-length (15 in this scenario) feature vectors to reflect the importance of a word in each SMS message.
Did this page help you?
If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!