List of configuration parameters for Talend Data Preparation

Parameter	Description
`dataprep.locale`	Setting for the application interface language.
`public.ip=<local machine ip>` `server.port=9999` `async-runtime.contextPath=/api`	IP of the server hosting Talend Data Preparation and server port
`server.compression.enabled=true` `server.compression.mime-types=text/plain,text/html,text/css,application/json,application/x-javascript,text/xml,application/xml,application/xml+rss,text/javascript,application/javascript,text/x-js`	Response compression parameters
`iam.ip=<local machine ip>` `iam.uri=http://${iam.ip}:9080` `iam.api.uri=${iam.uri}`	IP of the server hosting Talend Identity and Access Management, used for SSO, and server port
`spring.mvc.async.request-timeout=600000`	Timeout setting for Asynchronous executions. Do not change this value, unless you were told so by Talend.
`dataprep.event.listener=spring`	Event propagation parameter. Can be Spring or Kafka.
`live.dataset.location=tac` `live.dataset.url=http://<local machine ip>:8080/org.talend.administrator/`	Parameters related to the Live dataset feature. URL to the Talend Administration Center instance used to list execution tasks as dataset sources.
`live.dataset.task-prefix=dataprep_`	Prefix used to list Talend Administration Center tasks in the Talend Data Preparation interface and create Live datasets. Only the tasks with this prefix will be listed when importing data with the Talend Job option.
`mongodb.host=<local machine ip>` `mongodb.port=27017` `mongodb.database=dataprep` `mongodb.user=dataprep-user` `mongodb.password=<randomly generated password>` `multi-tenancy.mongodb.active=true`	MongoDB settings
`mongodb.uri=`	For more complex use cases, `mongo.*` configurations can be overwritten by specifying a URI directly.
`mongodb.ssl=true` `mongodb.ssl.trust-store=/path/to/trust-store.jks` `mongodb.ssl.trust-store-password=trust-store-password`	Uncomment these parameters to set up a secured connexion with MongoDB
`tls.key-store=/path/to/key-store.jks` `tls.key-store-password=key-store_password` `tls.trust-store=/path/to/trust-store.jks` `tls.trust-store-password=trust-store_password` `tls.verify-hostname=false`	Uncomment these parameters to set up a secured `https` connexion for Talend Data Preparation
`security.provider=oauth2` `security.token.secret=encrypted password`	Authentication parameters
`spring.profiles.active=server-standalone` `spring.mvc.favicon.enabled=false` `spring.http.multipart.maxFileSize=200000000` `spring.http.multipart.maxRequestSize=200000000`	Spring parameters. Do not change these values, unless you were told so by Talend.
`service.documentation.name=Talend Data Preparation - API` `service.documentation.description=This service exposes high level services that may involve services orchestration.` `service.paths=api` `springfox.documentation.swagger.v2.host=${public.ip}:${server.port}${gateway-api.service.path}`	Set these parameters to enable access to Swagger
`dataset.records.limit=10000` `dataset.local.file.size.limit=2000000000` `dataset.imports=local,job,tcomp-JDBCDatastore,tcomp-SimpleFileIoDatastore,tcomp-SalesforceDatastore,tcomp-S3Datastore` `dataset.list.limit=10`	Size limit and display parameters for your datasets
`dataset.service.url=http://${public.ip}:${server.port}` `transformation.service.url=http://${public.ip}:${server.port}` `preparation.service.url=http://${public.ip}:${server.port}` `fullrun.service.url=http://${public.ip}:${server.port}` `async_store.service.url=http://${public.ip}:${server.port}` `gateway.service.url=http://${public.ip}:${server.port}`	Address of the dataset service
`dataset.metadata.store=mongodb` `preparation.store=mongodb` `user.data.store=mongodb` `folder.store=mongodb` `upgrade.store=mongodb`	File storage service configuration parameters. Do not change these values, unless you were told so by Talend.
`content-service.store=local` `content-service.store.local.path=data/` `content-service.journalized=true`	Location for cache and content storage
`preparation.store.remove.hours=24`	Preparation service configuration. Do not change these values, unless you were told so by Talend.
`lock.preparation.store=mongodb` `lock.preparation.delay=600`	Lock duration parameter in seconds, when working on shared preparations
`luceneIndexStrategy=singleton`	Lucene index configuration. Do not change these values, unless you were told so by Talend.
`execution.store=mongodb` `async.operation.concurrent.run=5`	Parameters for asynchronous full run and sampling operations, namely storage and number of allowed concurrent runs. Do not change the `mongodb` value, unless you were told so by Talend. Regarding asynchronous operations, if there are more full run operations than the parameter's value running in parallel, the operations will be queued, and will resume when there is an available slot. You can increase the value of this parameter, according to your machine's power.
`tcomp.server.url=http://<local machine ip>:8989/tcomp`	URL of the server hosting the Components Catalog, used to configure self service connectors
`tcomp-SimpleFileIoDatastore.kerberosPrincipal.default=${streams.kerberos.principal}` `tcomp-SimpleFileIoDatastore.kerberosKeytab.default=${streams.kerberos.keytab_path}` `tcomp-SimpleFileIoDataset.path.default=${streams.hdfs.server.url}`	Components Catalog configuration properties. Allows you to set your kerberos configuration automatically when importing datasets from HDFS.
`tcomp-SimpleFileIoDatastore.test_connection.visible=false`	Parameter to remove the test connection step from the Talend component form. Do not change this parameter, unless you were told so by Talend
`async.operation.watcher.ttl=3600000`	Maximum execution time for full runs, in milliseconds
`receivers.timeout=3600000`	Maximum waiting time for Live datasets input
`dataquality.indexes.file.location=data/data-quality/org.talend.dataquality.semantic`	Data quality indexes storage location. If you change this value, Talend Data Preparation will automatically recreate the indexes at startup, but only for the default ones. In order to retrieve your custom semantic types, you need to copy the content of your old directory, and paste it to the new location.
`dataquality.semantic.list.enable=true` `dataquality.server.url=http://<local machine ip>:8187/`	Parameter to activate the semantic type edition in the Talend Data Preparation interface, and url of the server hosting Talend Dictionary Service
`tsd.consumer.enabled=true` `tsd.consumer.semantic-topic-content=raw` `dataquality.event.store=mongodb` `spring.cloud.stream.kafka.binder.brokers=tal-rd44.talend.lan`	Data quality updates parameters
`streams.enable=false` `streams.flow.runner.url=http://<local machine ip>:<Big data preparation port>/` `streams.kerberos.principal=<principal>` `streams.kerberos.keytab_path=<keytab path>` `streams.hdfs.server.url=hdfs://<host>:<port>/<filepath>`	Streams Runner configuration parameters Enable these parameters to configure Talend Data Preparation with Big Data
`security.basic.enabled=false` `security.oidc.client.expectedIssuer=http://tal-rd44.talend.lan:9080/oidc` `iam.license.url=http://${iam.ip}:9080/oidc/api` `security.oidc.client.keyUri=${iam.uri}/oidc/jwk/keys` `security.oauth2.client.clientId=64xIVPxviKWSog` `security.oauth2.client.clientSecret=9C0zCjp8yS-eZBqEi-KhBQ` `security.oidc.client.claimIssueAtTolerance=120` `# security.oauth2.resource.serviceId=${PREFIX:}resource` `security.oauth2.resource.tokenInfoUri=${iam.uri}/oidc/oauth2/introspect` `security.oauth2.resource.uri=/v2/api-docs,/api/,/folders/,/datasets/,/dataset/,/preparations/,/transform/,/version/,/acl/,/apply/,/export,/export/,/aggregate,/sampling/,/receivers/,/error,/docs,/datastores/,/preparation/,/actions/,/suggest/,/dictionary/**` `security.oauth2.resource.filter-order=3` `security.scim.enabled=true` `security.oauth2.client.access-token-uri=${iam.uri}/oidc/oauth2/token` `security.oauth2.client.scope=openid refreshToken` `security.oauth2.client.user-authorization-uri=${iam.uri}/oidc/idp/authorize` `security.oauth2.sso.login-use-forward=false` `server.session.cookie.name=TDPSESSION` `spring.session.store-type=hash_map` `security.sessions=stateless` `security.user.password=none`	Single Sign-On security configuration parameters
`security.oidc.client.endSessionEndpoint=${iam.uri}/oidc/idp/logout` `security.oidc.client.logoutSuccessUrl=http://${public.ip}:${server.port}` `security.oauth2.logout.uri=/signOut` `security.oauth2.sso.login-path=/signIn` `iam.scim.url=http://${iam.ip}:9080/scim/` `security.oauth2.resource.tokenInfoUriCache.enabled=true` `tenant.account.cache.enabled=true`	Single Sign-On properties for Talend Data Preparation API and Gateway
`gateway-api.service.url=http://${public.ip}:${server.port}` `gateway-api.service.path=/gateway` `zuul.servletPath=/gateway/upload` `zuul.routes.dq.path=/gateway/dq/semanticservice/` `zuul.routes.dq.sensitiveHeaders=${zuul.sensitiveHeaders}` `zuul.routes.dq.url=${dataquality.server.url}/` `proxy.auth.routes.dq=oauth2` `zuul.routes.api.path=/gateway/api/` `zuul.routes.api.sensitiveHeaders=${zuul.sensitiveHeaders}` `zuul.routes.api.url=http://${public.ip}:${server.port}/api` `proxy.auth.routes.api=oauth2` `zuul.sensitiveHeaders=Cookie,Set-Cookie,Expires,X-Content-Type-Options,X-Xss-Protection,Cookie,X-Frame-Options,Cache-control,Pragma` `zuul.host.socket-timeout-millis=300000` `zuul.host.connect-timeout-millis=5000`	Single Sign-On configuration parameters. Do not change these values, unless you were told so by Talend.
`logging.file=data/logs/app.log`	Path of the log file storage folder
`logging.pattern.level=%5p [user %X{user}]`	Level output pattern for the log file
`logging.pattern.file=%d{yyyy-MM-dd HH:mm:ss.SSS} %5p --- [%t] %-40.40logger{39} : %m%n%wEx`	Uncomment this parameter to enable log pattern configuration
`logging.level=WARN` `logging.level.org.talend.dataprep=INFO` `logging.level.org.talend.dataprep.api=INFO` `logging.level.org.talend.dataprep.dataset=INFO` `logging.level.org.talend.dataprep.preparation=INFO` `logging.level.org.talend.dataprep.transformation=INFO` `logging.level.org.talend.dataprep.fullrun=INFO` `logging.level.org.talend.dataprep.api.dataquality=INFO` `logging.level.org.talend.dataprep.configuration=INFO` `logging.level.org.talend.dataquality.semantic=INFO`	Talend Data Preparation loggers parameters
`logging.pattern.console=%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(%5p) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n%wEx`	Uncomment this parameter to enable console logging pattern configuration
`spring.output.ansi.enabled=always`	Uncomment this parameter to configure ansi coloration in console output
`logging.config=logback.xml`	Uncomment this parameter to configure the Talend Data Preparation logging with a custom logback file. Enter the path to your logback file
`audit.log.enabled=true` `talend.logging.audit.config=config/audit.properties`	Audit logs parameters
`default.text.enclosure="` `default.text.escape="` `default.text.encoding=UTF-8`	Configurable values for the default enclosure and escape characters for CSV exports, as well as the default text encoding. The default separator can be semicolon `";"`, tab `"\t"`, space `" "`, comma `","` or pipe `"\|"`
`default.import.text.enclosure="` `default.import.text.escape=`	Configurable values for the default enclosure and escape characters for CSV imports
`app.products[0].id=TDS` `app.products[0].name=Data Stewardship` `app.products[0].url=<place_your_tds_url_here>`	When Talend Data Preparation and Talend Data Stewardship are both installed, you have the possibility to switch between the two applications. Configure the URL to your Talend Data Stewardship instance so that you can reach it.

dataprep.locale

Setting for the application interface language.

public.ip=<local machine
                            ip>

server.port=9999

async-runtime.contextPath=/api

IP of the server hosting Talend Data Preparation and server port

server.compression.enabled=true

server.compression.mime-types=text/plain,text/html,text/css,application/json,application/x-javascript,text/xml,application/xml,application/xml+rss,text/javascript,application/javascript,text/x-js

Response compression parameters

iam.ip=<local machine
                            ip>

iam.uri=http://${iam.ip}:9080

iam.api.uri=${iam.uri}

IP of the server hosting Talend Identity and Access Management, used for SSO, and server port

spring.mvc.async.request-timeout=600000

Timeout setting for Asynchronous executions. Do not change this value, unless you were told so by Talend.

dataprep.event.listener=spring

Event propagation parameter. Can be Spring or Kafka.

live.dataset.location=tac

live.dataset.url=http://<local machine ip>:8080/org.talend.administrator/

Parameters related to the Live dataset feature. URL to the Talend Administration Center instance used to list execution tasks as dataset sources.

live.dataset.task-prefix=dataprep_

Prefix used to list Talend Administration Center tasks in the Talend Data Preparation interface and create Live datasets. Only the tasks with this prefix will be listed when importing data with the Talend Job option.

mongodb.host=<local machine
                            ip>

mongodb.port=27017

mongodb.database=dataprep

mongodb.user=dataprep-user

mongodb.password=<randomly generated password>

multi-tenancy.mongodb.active=true

MongoDB settings

mongodb.uri=

For more complex use cases, mongo.* configurations can be overwritten by specifying a URI directly.

mongodb.ssl=true

mongodb.ssl.trust-store=/path/to/trust-store.jks

mongodb.ssl.trust-store-password=trust-store-password

Uncomment these parameters to set up a secured connexion with MongoDB


                  tls.key-store=/path/to/key-store.jks

tls.key-store-password=key-store_password

tls.trust-store=/path/to/trust-store.jks

tls.trust-store-password=trust-store_password

tls.verify-hostname=false

Uncomment these parameters to set up a secured https connexion for Talend Data Preparation

security.provider=oauth2

security.token.secret=encrypted password

Authentication parameters

spring.profiles.active=server-standalone

spring.mvc.favicon.enabled=false

spring.http.multipart.maxFileSize=200000000

spring.http.multipart.maxRequestSize=200000000

Spring parameters. Do not change these values, unless you were told so by Talend.

service.documentation.name=Talend Data Preparation -
                            API

service.documentation.description=This service exposes high level services that may involve services orchestration.

service.paths=api

springfox.documentation.swagger.v2.host=${public.ip}:${server.port}${gateway-api.service.path}

Set these parameters to enable access to Swagger

dataset.records.limit=10000

dataset.local.file.size.limit=2000000000

dataset.imports=local,job,tcomp-JDBCDatastore,tcomp-SimpleFileIoDatastore,tcomp-SalesforceDatastore,tcomp-S3Datastore

dataset.list.limit=10

Size limit and display parameters for your datasets

dataset.service.url=http://${public.ip}:${server.port}

transformation.service.url=http://${public.ip}:${server.port}

preparation.service.url=http://${public.ip}:${server.port}

fullrun.service.url=http://${public.ip}:${server.port}

async_store.service.url=http://${public.ip}:${server.port}

gateway.service.url=http://${public.ip}:${server.port}

Address of the dataset service

dataset.metadata.store=mongodb

preparation.store=mongodb

user.data.store=mongodb

folder.store=mongodb

upgrade.store=mongodb

File storage service configuration parameters. Do not change these values, unless you were told so by Talend.

content-service.store=local

content-service.store.local.path=data/

content-service.journalized=true

Location for cache and content storage

preparation.store.remove.hours=24

Preparation service configuration. Do not change these values, unless you were told so by Talend.

lock.preparation.store=mongodb

lock.preparation.delay=600

Lock duration parameter in seconds, when working on shared preparations

luceneIndexStrategy=singleton

Lucene index configuration. Do not change these values, unless you were told so by Talend.

execution.store=mongodb

async.operation.concurrent.run=5

Parameters for asynchronous full run and sampling operations, namely storage and number of allowed concurrent runs. Do not change the mongodb value, unless you were told so by Talend. Regarding asynchronous operations, if there are more full run operations than the parameter's value running in parallel, the operations will be queued, and will resume when there is an available slot. You can increase the value of this parameter, according to your machine's power.

tcomp.server.url=http://<local machine
                    ip>:8989/tcomp

URL of the server hosting the Components Catalog, used to configure self service connectors

tcomp-SimpleFileIoDatastore.kerberosPrincipal.default=${streams.kerberos.principal}

tcomp-SimpleFileIoDatastore.kerberosKeytab.default=${streams.kerberos.keytab_path}

tcomp-SimpleFileIoDataset.path.default=${streams.hdfs.server.url}

Components Catalog configuration properties. Allows you to set your kerberos configuration automatically when importing datasets from HDFS.

tcomp-SimpleFileIoDatastore.test_connection.visible=false

Parameter to remove the test connection step from the Talend component form. Do not change this parameter, unless you were told so by Talend

async.operation.watcher.ttl=3600000

Maximum execution time for full runs, in milliseconds

receivers.timeout=3600000

Maximum waiting time for Live datasets input

dataquality.indexes.file.location=data/data-quality/org.talend.dataquality.semantic

Data quality indexes storage location. If you change this value, Talend Data Preparation will automatically recreate the indexes at startup, but only for the default ones. In order to retrieve your custom semantic types, you need to copy the content of your old directory, and paste it to the new location.

dataquality.semantic.list.enable=true

dataquality.server.url=http://<local machine ip>:8187/

Parameter to activate the semantic type edition in the Talend Data Preparation interface, and url of the server hosting Talend Dictionary Service

tsd.consumer.enabled=true

tsd.consumer.semantic-topic-content=raw

dataquality.event.store=mongodb

spring.cloud.stream.kafka.binder.brokers=tal-rd44.talend.lan

Data quality updates parameters

streams.enable=false

streams.flow.runner.url=http://<local machine ip>:<Big data preparation port>/

streams.kerberos.principal=<principal>

streams.kerberos.keytab_path=<keytab path>

streams.hdfs.server.url=hdfs://<host>:<port>/<filepath>

Streams Runner configuration parameters

Enable these parameters to configure Talend Data Preparation with Big Data

security.basic.enabled=false

security.oidc.client.expectedIssuer=http://tal-rd44.talend.lan:9080/oidc

iam.license.url=http://${iam.ip}:9080/oidc/api

security.oidc.client.keyUri=${iam.uri}/oidc/jwk/keys

security.oauth2.client.clientId=64xIVPxviKWSog

security.oauth2.client.clientSecret=9C0zCjp8yS-eZBqEi-KhBQ

security.oidc.client.claimIssueAtTolerance=120

# security.oauth2.resource.serviceId=${PREFIX:}resource

security.oauth2.resource.tokenInfoUri=${iam.uri}/oidc/oauth2/introspect

security.oauth2.resource.uri=/v2/api-docs,/api/**,/folders/**,/datasets/**,/dataset/**,/preparations/**,/transform/**,/version/**,/acl/**,/apply/**,/export,/export/**,/aggregate,/sampling/**,/receivers/**,/error,/docs,/datastores/**,/preparation/**,/actions/**,/suggest/**,/dictionary/**

security.oauth2.resource.filter-order=3

security.scim.enabled=true

security.oauth2.client.access-token-uri=${iam.uri}/oidc/oauth2/token

security.oauth2.client.scope=openid refreshToken

security.oauth2.client.user-authorization-uri=${iam.uri}/oidc/idp/authorize

security.oauth2.sso.login-use-forward=false

server.session.cookie.name=TDPSESSION

spring.session.store-type=hash_map

security.sessions=stateless

security.user.password=none

Single Sign-On security configuration parameters

security.oidc.client.endSessionEndpoint=${iam.uri}/oidc/idp/logout

security.oidc.client.logoutSuccessUrl=http://${public.ip}:${server.port}

security.oauth2.logout.uri=/signOut

security.oauth2.sso.login-path=/signIn

iam.scim.url=http://${iam.ip}:9080/scim/

security.oauth2.resource.tokenInfoUriCache.enabled=true

tenant.account.cache.enabled=true

Single Sign-On properties for Talend Data Preparation API and Gateway

gateway-api.service.url=http://${public.ip}:${server.port}

gateway-api.service.path=/gateway

zuul.servletPath=/gateway/upload

zuul.routes.dq.path=/gateway/dq/semanticservice/**

zuul.routes.dq.sensitiveHeaders=${zuul.sensitiveHeaders}

zuul.routes.dq.url=${dataquality.server.url}/

proxy.auth.routes.dq=oauth2

zuul.routes.api.path=/gateway/api/**

zuul.routes.api.sensitiveHeaders=${zuul.sensitiveHeaders}

zuul.routes.api.url=http://${public.ip}:${server.port}/api

proxy.auth.routes.api=oauth2

zuul.sensitiveHeaders=Cookie,Set-Cookie,Expires,X-Content-Type-Options,X-Xss-Protection,Cookie,X-Frame-Options,Cache-control,Pragma

zuul.host.socket-timeout-millis=300000

zuul.host.connect-timeout-millis=5000

Single Sign-On configuration parameters. Do not change these values, unless you were told so by Talend.

logging.file=data/logs/app.log

Path of the log file storage folder

logging.pattern.level=%5p [user %X{user}]

Level output pattern for the log file

logging.pattern.file=%d{yyyy-MM-dd HH:mm:ss.SSS} %5p --- [%t]
                        %-40.40logger{39} : %m%n%wEx

Uncomment this parameter to enable log pattern configuration

logging.level=WARN

logging.level.org.talend.dataprep=INFO

logging.level.org.talend.dataprep.api=INFO

logging.level.org.talend.dataprep.dataset=INFO

logging.level.org.talend.dataprep.preparation=INFO

logging.level.org.talend.dataprep.transformation=INFO

logging.level.org.talend.dataprep.fullrun=INFO

logging.level.org.talend.dataprep.api.dataquality=INFO

logging.level.org.talend.dataprep.configuration=INFO

logging.level.org.talend.dataquality.semantic=INFO

Talend Data Preparation loggers parameters

logging.pattern.console=%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint}
                        %clr(%5p) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint}
                        %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n%wEx

Uncomment this parameter to enable console logging pattern configuration

spring.output.ansi.enabled=always

Uncomment this parameter to configure ansi coloration in console output

logging.config=logback.xml

Uncomment this parameter to configure the Talend Data Preparation logging with a custom logback file.

Enter the path to your logback file

audit.log.enabled=true

talend.logging.audit.config=config/audit.properties

Audit logs parameters

default.text.enclosure="

default.text.escape="

default.text.encoding=UTF-8

Configurable values for the default enclosure and escape characters for CSV exports, as well as the default text encoding. The default separator can be semicolon ";", tab "\t", space

"
                        "

, comma "," or pipe "|"

default.import.text.enclosure="

default.import.text.escape=

Configurable values for the default enclosure and escape characters for CSV imports

app.products[0].id=TDS

app.products[0].name=Data Stewardship

app.products[0].url=<place_your_tds_url_here>

When Talend Data Preparation and Talend Data Stewardship are both installed, you have the possibility to switch between the two applications. Configure the URL to your Talend Data Stewardship instance so that you can reach it.

List of configuration parameters for Talend Data Preparation - 7.3

Talend Data Preparation User Guide