Securing the connections between Talend Data Preparation, Streams Runner and Spark Job Server requires editing their
corresponding configuration files.
Any security configuration in the Streams Runner configuration file should be done
at the end of the file, in the Append section, after the
Include section, to avoid being overwritten.
The first step will be to configure Spark Job Server as a service in HTTPS. Then, you
will need to enable SSL communication between Streams Runner and Spark Job Server running in HTTPS. After that, you
will configure Streams Runner as a
service in HTTPS, and finally, enable SSL communication between Talend Data Preparation and Streams Runner running in HTTPS.
Before you begin
-
Talend Data Preparation has been configured as a service in
HTTPS. For more information, see Configuring an HTTPS connection for Talend Data Preparation.
- You have generated a certificate for Talend Data Preparation and added it to your Web
browser truststore.
-
Spark Job Server and Streams Runner are installed and
running.
Procedure
-
To secure the Spark Job Server
service in HTTPS, open the
<Spark_Job_Server_installation_path>/settings.sh
configuration file.
-
Set the value of the
security_ssl_enabled
parameter to
on
.
-
Edit the values of the
security_path_to_keystore
and
security_keystore_password
parameters to set the path and
password of your keystore
file containing the certificate for
Spark Job Server.
security_ssl_enabled=on
# DO NOT CHANGE
# SECURITY PATH TO KEYSTORE
# Required : No
# Env variable : SECURITY_PATH_TO_KEYSTORE
security_path_to_keystore=<path_to_keystore>
# DO NOT CHANGE
# SECURITY KEYSTORE PASSWORD
# Required : No
# Env variable : SECURITY_KEYSTORE_PASSWORD
security_keystore_password=<password>
After restarting the service, Spark Job Server will be running in
HTTPS.
-
To enable SSL communication between Streams Runner and Spark Job Server running in HTTPS, you can
either:
- Use the
JOBSERVER_TRANSPORT_PROTOCOL
environment
variable with the following command: export
JOBSERVER_TRANSPORT_PROTOCOL=https
.
- Edit the
<Streams_Runner_installation_path>/conf/application.conf
configuration file and set the value of the
app.svc.jobserver.protocol
parameter to
https
.
Using the environment variable will override the
application.conf configuration.
-
To add the Spark Job Server
certificate, or its Certificate Authority to the Streams Runner truststore, add the
following lines to the
<Streams_Runner_installation_path>/conf/application.conf
file, according to the file format used for your truststore.
- For .pem
files:
play.ws.ssl {
trustManager = {
stores = [
{ type = "PEM", path = "/path/to/pem_file" }
]
}
}
- For .jks
files:
play.ws.ssl {
trustManager = {
stores = [
{ type="JKS", path="/path/to/truststore", password="<password>"}
]
}
}
-
To secure the Streams Runner
service in HTTPS, define the path and password of its certificate by editing the
following lines of the
<Streams_Runner_installation_path>/conf/application.conf
file:
play.server.https.keyStore.path = <path_to_keystore>
play.server.https.keyStore.password = <password>
-
Edit the two following lines to set the HTTPS port and disable the HTTP
port:
https.port=9443
http.port=disabled
9443
is the default port value for the HTTPS connection.
Warning: Any play.server.http.port=<port>
configuration will conflict with the http.port=disabled
configuration and the port will not be disabled.
After restarting the service, Streams Runner will be running in
HTTPS.
-
To enable SSL communication between Talend Data Preparation and Streams Runner running in HTTPS, retrieve
the Streams Runner certificate,
or its Certificate Authority, and add it to the Talend Data Preparation truststore using the
following command:
keytool -import -trustcacerts -alias <cert-alias> -file
<streams_runner_certificate.crt> -keystore
<truststore.jks>
-
In the
<Data_Preparation_Path>/config/application.properties
file, add the following properties to set the truststore:
tls.trust-store=/path/to/<truststore.jks>
tls.trust-store-password=<trust-store_password>
false to disable hostname verification
tls.verify-hostname=true
-
Restart Talend Data Preparation.
Results
Your Talend Data Preparation instance running
in HTTPS can now communicate with Streams Runner and Spark Job Server, also running with a secured
HTTPS connection.