Troubleshooting: Resolving Databricks runtime incompatibilities - Cloud

Talend Cloud Management Console for Pipelines User Guide

Talend Documentation Team
Talend Cloud
Administration and Monitoring > Managing projects
Administration and Monitoring > Managing users
Deployment > Deploying > Executing Tasks
Deployment > Scheduling > Scheduling Tasks
Talend Management Console
When upgrading to a newer Remote Engine Gen2 that uses a Databricks cluster, you might have compatibility issues during pipeline executions and you might see this error message in the logs:
Incompatible runtime found : Cluster runtime version =/= Engine runtime version


This error is due to the fact that Talend Cloud Pipeline Designer requires a Databricks cluster with compatible libraries in order to submit pipelines from a Remote Engine Gen2.



  1. In order to resolve this issue, log in to your Databricks account and go to the Clusters page.
  2. Select your existing cluster and go to the Libraries tab to uninstall the old libraries.

    By removing these old dependencies the compatibility issue should be solved and you should be able to execute pipelines on your new engine with the same Databricks cluster.