Capturing Salesforce object changes in real time with Talend ESB
This article shows how to capture and propagate real time changes happening in Salesforce using Talend ESB.
Salesforce provides on-demand customer relationship management (CRM) software services and many Talend users have chosen Salesforce as their cloud CRM provider.
Some users want to capture real time changes occurring in Salesforce on their CRM data in order to propagate or synchronize those changes on third-party applications. It is somewhat similar to Change Data Capture with Relational Databases, but for Salesforce.
In order to capture and propagate real time changes happening in Salesforce, you need the following technologies:
- Salesforce Notification and Outbound Messaging
- Talend ESB and Web Services technologies
- Apache ActiveMQ
Outbound Messaging is part of the workflow rule functionality in Salesforce. Workflow rules watch for specific kinds of field changes and trigger automatic Salesforce actions, such as sending an outbound message to external applications.
To understand this article, you need some knowledge of Salesforce and Outbound Messaging, Talend ESB and Mediation routes, Web Services technologies, and Apache ActiveMQ.
This article was written using the commercial version of Talend ESB 6.2 which provides more features than the open source version.
This article is also valid for previous commercial versions of Talend ESB.
- You have a valid Salesforce account: a free Developer Edition account can be created for testing purposes on the Salesforce website at https://developer.salesforce.com/signup.
- You have installedTalend Studio 6.2 installed.
- You have a installed Apache ActiveMQ: ActiveMQ is shipped with the Talend ESB installation. Alternatively, it can be downloaded from the Apache ActiveMQ website.
For more information on the procedure to install and configure both Talend ESB and ActiveMQ, see Talend Help Center.
Architecture of the flow to capture changes
The diagram below shows the architecture for this example:
- Salesforce User updates a Salesforce object.
- Salesforce sends an outbound message to Talend ESB to notify object update.
- Talend ESB persists the notification message in an ActiveMQ queue.
- Talend ESB receives acknowledgement of successful persistence in ActiveMQ.
- Talend ESB answers with a successful acknowledgement back to Salesforce.
Configuring Salesforce to send changes notifications to Talend
This section explains how to setup Salesforce to send outbound notifications on objects changes.
It focuses on the standard Salesforce Account object, but the procedure remains the same for all other Salesforce standard or custom objects.
- Connect to your existing Salesforce account, or create a free Developer Edition account if you do not have one.
Navigate to Build > Create > Workflow & Approvals > Workflow Rules in the left panel.
Alternatively, you can search for workflow rules in the search area.If you do not see or do not have access to this menu, check your account permissions with your Salesforce Administrator.
Click the Workflow Rules menu to open a page where you
can manage your organization’s workflow rules.
Click New rule to create a workflow rule which will be triggered every time an account is created or updated in your Salesforce organization.
Select the object on which the workflow rule will apply. In this case, select
Account, then click
This should also work with any other Salesforce standard or custom object.
Configure the workflow rule. Since the workflow must be triggered each time an
account is created or updated, configure the rule as follows:
Field Value Rule name UpdateOrCreateAccountRule Rule criteria Account name not equal to ""
Use double quotes in the rule criteria to indicate that Account name should not be empty.
Since the account name is mandatory for Account objects, the clause Account name not equal to "" should be always true.
Click Save & next to open a page with the rule criteria you defined.
Click Add Workflow Action, then choose New Outbound Message to send a new Outbound Message to Talend ESB.
This option is used to specify one or more actions to be executed by the workflow when triggered. You have the choice between Immediate Workflow Actions and Time-dependant Workflow Actions. In this case, choose to create an Immediate Worflow Action to send a new outbound message to Talend ESB.
Configure Outbound Message as follows.
Field Value Name Account Outbound Message Unique Name This field should be auto-filled. You can also update it by entering a unique name. Endpoint URL http://52.28.xxx.xxx:8765/services/accountThis is the endpoint on which Talend ESB is listening for Salesforce Outbound Messages.Note: At this stage, you can put any dummy IP and come back later to replace it with the correct endpoint after the Talend Web Service has been developed and deployed. If your are using an EC2 instance, you need to update your instance security group to allow inbound traffic from Salesforce. User to send as Keep the default user. Account fields to send Choose any available fields that you want to send along with the outbound message, according to your use case. Good practice would be to send only the ID and the minimum relevant information to keep the notification message as light as possible.
You should see the following screen.
Click Done, then click Activate on the Workflow Rule Detail page.
The flag Active is checked, indicating that the workflow is activated.
Navigate to Administer > Security Controls > Remote Site Settings and click Remote Site Settings.
By default, Salesforce does not authorize outbound traffic towards remote hosts. In order to send outbound messages to Talend ESB, you need to whitelist (authorize) the remote URL of Talend ESB. To do this, you have to add the remote URL information in the whitelisted sites.
Click New Remote Site to add a new remote site and configure it as follows.
Field Value Remote Site Name WsTalendAccountNotificationServer Remote Site URL http://52.28.xxx.xxx:8765/This is the Web Service URL with the IP Address of the server hosting the Web Service and the PORT on which the Web Service is listening. It is not mandatory to put the relative path, for example services/account. Choose not to mention the relative path in order to be able to handle multiple Web Services on the same host.
- Click Save.
Salesforce is now setup and ready to send notifications to Talend ESB every time an account is created or updated in your Salesforce organization. The next step is to implement the Talend Web Service which will be listening for these notifications coming from Salesforce.
It is important to handle these upcoming notifications in a reliable manner. You must not lose any of these notifications once you receive and acknowledge them, even if a failure or crash occurs on Talend ESB, because Salesforce is not going to send again any acknowledged notification.
Downloading the Salesforce notification WSDL document
- Navigate to Build > Create > Workflow & Approvals > Workflow Rules and click Workflow Rules in the left menu.
Click the workflow rule UpdateOrCreateAccountRule on the All Workflow Rules page.
The details of the workflow rule are displayed.
- Click Account Outbound Message.
- Click Click for WSDL next to the Endpoint WSDL label. This should open a new tab on your browser displaying the WSDL.
- Download and save the WSDL file on your computer as workflowOutboundMessage.wsdl for example.
Configuring Talend ESB to listen for and process Salesforce notifications using the Notification WSDL
Download and install Talend ESB and Apache ActiveMQ.
ActiveMQ is shipped with the Talend ESB installation if you subscribe to the commercial version.
If you are not familiar with it, see Talend Help Center for more information on how to install and start ActiveMQ.
Launch your Talend Studio, then open the Mediation perspective.
Starting from Talend 6.2, the Mediation and Integration perspectives are merged in Talend Studio. Therefore, you need to open the Integration perspective.
Create a new mediation route named
salesforce_account_synchro for example.
Drag and drop the following components to the design workspace:
The route should look like this before you connect the components to each other:
- Right-click cCxf, select Row > Route from the contextual menu and click cExchangePattern.
- Repeat the previous step to connect cExchangePattern to cJms, cJms to cSetBody and cSetBody to cLog.
Rename the components as follows for more clarity.
Component Name cCxf webservice_salesforce.notifcation.account cExchangePattern InOnly cJms activemq_salesforce.notification.account cSetBody set_webservice_response cLog log_response cMQConnectionFactory activemq_connection_factory
The route salesforce_account_synchro should now look like this: