TPS-5359 (cumulative patch) - 7.3

This patch is cumulative. It includes all previous generally available patches for Talend Jobserver 7.3.1.

NOTE: For information on how to obtain this patch, reach out to your Support contact at Talend.

This patch contains the following fixes:

• TPS-3872: Tasks redeployed after each restart of the jobserver (TPSVC-13658)
• TPSVC-14107: Parameter Delimiter tab (\t) treated as string in tFileOuputDelimited if artifact published fromTalend Studio 7.2 and task published in 7.1 updated
• TPSVC-14689: override JVM parameters specified for Job in Studio with ones provided during Job execution
• TPSVC-15112: Allow dynamically switching JobServer to use native context values
• TPS-4318: JobServer memory leak related to ZeroMQ mailbox (TPSVC-12728)
• TESB-29867: Avoid org.talend.libraries.jmx export META-INF.services
• TPSVC-15569: Allow jobserver clients to detect cache cleanups
• TPS-4381: (7.3.1) Jobserver dont stop on CTRL+C (TPSVC-15729)
• TPSVC-16265: (7.3.1) Move CONTEXT_PASSTHROUGH_PROPERTY to jobCmdPrms
• TPSVC-15923 TAC / Jobserver resiliency issue with flaky network
• TPSVC-16463 Upgrade jobserver dependency libraries version
• TPSVC-15823 Remote Engine automatic purge doesn't work
• TPSVC-13908 JobServer lifecycle broken in OSGi environments
• TPSVC-16934 Update BeanUtils to 1.9.4 and Bouncy Castle Provider to 1.68
• TPSVC-16969 Update Commons IO to 2.8.0
• TPSVC-16967 Update HttpClient version to 4.5.13
• TPSVC-16933 Update Jackson to 2.11.4 or exclude if not needed
• TPRUN-326 Change JobServer encryption to use aesGCM
• TPRUN-392 Update vulnerable ANT version in 7.3 JobServer
• TPRUN-858 Instrument the CE used by the customer for TMC-24785 / TPRUN-861
• TPRUN-861 Task job_tmcRestApi_00_Orchestration_Parent is intermittently failing on CE.
• TPRUN-1088 Allow passing predefined execution ids to JobServer
• TPRUN-643 Improvements to ExecuteCreateTmpDir
• TPRUN-641 Cleanup JS monitoring server
• TPRUN-1762 .cfg file doesn't contain all the info
• TPRUN-1106 Support usage of classpath.jar for RemoteEngine Gen1
• TPS-4994 file triggers fired more than once on new file creation (TPRUN-2340)
• TPRUN-2532 Support for bundle refresh on feature installation failure.
• TPS-5040 Mitigate / fix JobServer log4j2 vulnerabilities ( CVE-2021-44228 ) (TPRUN-2701)
• TPRUN-2801 Typo in Jobserver patch doc link for TPS-5040
• TPRUN-2543 Fix compatibility statement logged at JobServer startup
• TPS-5075 [7.3.1]including the possibility to define the certificate password when defining the SSL on jobserver and runtime (TPRUN-1805)
• TPRUN-3050 Upgrade Ant dependency in JobServer to avoid known vulnerabilities
• TPS-5110 [7.3.1] JMX port 8888 is inactive for runtime from TAC while enabling SSL (TPRUN-2948)
• TPRUN-3106 When archive was deleted, wrong job execution state will be returned.
• TPS-5122 [7.3.1] Error "java.lang.NoClassDefFoundError: org/apache/log4j/Logger" on restarting jobserver after removing log4j-1.2.16.jar (TPRUN-2864)
• TPRUN-3152 JobServer secure mode is off by default
• TPRUN-1294 Restrict impersonation users by default.
• TPRUN-2345 Harden message deserialization ( backport to 7.3 ).
• TPRUN-1293 Backport changes for 'CommandServerSocket thread safety issue' to version 7.3.
• TPRUN-2214 JobServer package should include a NOTICE file with licenses.
• TPRUN-3515 java.lang.ClassNotFoundException: org.talend.remote.commons.msg.client.request.CheckServerMessage cannot be found by org.talend.remote.jobserver.server_7.3.1.20220321_1200_patch
• TPRUN-3405 The FileListener does not jail the path to the jobserver deploy directory.
• TPRUN-1296 Backport 'Prevent path manipulation attack in the FileServer' to 7.3.
• TPRUN-3450 JobServer should not weaken TLS in the TACClient (backport to 7.3)
• TPRUN-3451 CommandServer Denial of Service vulnerability (backport to 7.3)
• TPRUN-3508 AuthorizationKey is logged
• TPRUN-3697 JobServer should close stream of temporary context.
• TPRUN-3604 Unzipper Incorrect size limit check and created files not deleted in case of error
• TPRUN-3777 Non thread safe ClasspathJar writing
• TPRUN-3679 Modularize function required for user impersonation.
• TPS-5285 [7.3.1] Code cleanup & deprecation of 'launchFromShellScript' (TPRUN-3775)
• TPRUN-3605 Unzipper add limits for nesting and path length.
• TPRUN-3784 Update JobServer configuration/docs related to TLS version
• TPRUN-3948 Align versions of JAVA source/target, dependencies and plugins on pom(s).xml
• TPS-5359 [7.3.1] JobServer File server has no authentication. (TPRUN-3518)
• TPRUN-4022 Update patch creation process
• TPRUN-3916 Use RockyLinux as base image for JobServer docker in tests
• TPRUN-4131 Check Zip Slip and Zip Symlink vulnerabilities
• TPRUN-4126 Upgrade to OSHI 6.2.2
• TPRUN-3836 Improve error message in case Job archive checks fail
• TPRUN-3523 Add ability to disable the monitoring service
• TPRUN-1740 Simplify approach to let users install patches and (windows) services
• TPRUN-4023 Reduce merging pain between active branches due to different logging framework
• TPRUN-4267 Folder name length check not working for ZIP without folder entries
• TPRUN-4238 Attempt to publish a large job (while FileServer authentication is available?) causes a command server timeout
• TPRUN-4400 JobServer client checkServer returns wrong compatibility info
• TPRUN-4255 Do not log warnings when properties are not set but default value exists
• TPRUN-4355 Ensure Copyright is up-to-date for JAVA classes with UnitTesting
• TPRUN-4269 After Unzipper Exception partially unzipped file remain
• TPRUN-3519 Add constraints on jobs to prevent DoS attacks
• TPS-5372 [7.3.1] Adding File path traversal guard (TPRUN-4050)
• TPRUN-4515 Delete deployedJobPath directory before re-deploying
• TPRUN-4486 JobServer - Cleanings
• TPRUN-4447 JobServer start_jconsole.bat script has wrong classpath
• TPRUN-4761 Issue with FileEventsPacket

• CVE-2021-4104 ( reload4j JMSAppender )
• CVE-2022-23302 ( reload4j JMSSink )
• CVE-2019-17571 ( reload4j SocketServer )
• CVE-2020-9493 ( reload4j Chainsaw )
• CVE-2022-23307 ( reload4j Chainsaw )
• CVE-2022-23305 ( reload4j JDBCAppender )
• CVE-2020-9488 ( reload4j SMTPAppender )
• CVE-2021-36373 ( Ant TAR )
• CVE-2022-42889 ( Apache Commons Text variable interpolation)

Consider the following requirements for your system:

• Talend Jobserver 7.3.1 must be installed.

1. Create a backup for the patched files in <jobserver_home>/lib and <jobserver_home>/conf.
2. Stop Jobserver

3. Remove files from <jobserver_home>/lib :

4. commons-beanutils-*

5. commons-codec-*.jar
6. commons-configuration2-*.jar
7. commons-io-*.jar
8. commons-lang3-*.jar
9. commons-text-*.jar
10. crypto-utils-*.jar
11. daikon-*.jar
12. daikon-exception-*.jar
13. daikon-signature-verifier-*.jar
14. httpclient-*.jar
15. httpcore-*.jar
16. log4j-*.jar
17. slf4j-api-*.jar
18. slf4j-log4j12-*.jar
19. jackson-core-asl-*.jar
20. org.talend.monitoring-7.3.1*.jar
21. org.talend.monitoring.server-7.3.1*.jar
22. org.talend.remote.commons-7.3.1*.jar
23. org.talend.remote.jobserver.commons-7.3.1*.jar
24. org.talend.remote.jobserver.server.standalone-7.3.1*.jar
25. org.talend.remote.server-7.3.1*.jar
26. org.talend.utils*-7.3.1.jar
27. oshi-core-4.0.0.jar
28. jna-5.4.0.jar

29. jna-platform-5.4.0.jar

30. And replace them with their patched counterparts:

31. commons-codec-1.14.jar

32. commons-io-2.8.0.jar
33. commons-lang3-3.11.jar
34. commons-text-1.10.0.jar
35. crypto-utils-5.12.0.jar
36. daikon-5.12.0.jar
37. daikon-exception-5.12.0.jar
38. daikon-signature-verifier-5.12.0.jar
39. httpclient-4.5.13.jar
40. httpcore-4.4.13.jar
41. jackson-core-asl-1.9.16-TALEND.jar
42. reload4j-1.2.18.5.jar
43. slf4j-api-1.7.33.jar
44. slf4j-reload4j-1.7.33.jar
45. org.talend.monitoring-7.3.1.20221110_1952_patch.jar
46. org.talend.monitoring.server-7.3.1.20221110_1952_patch.jar
47. org.talend.remote.commons-7.3.1.20221110_1952_patch.jar
48. org.talend.remote.jobserver.commons-7.3.1.20221110_1952_patch.jar
49. org.talend.remote.jobserver.server.standalone-7.3.1.20221110_1952_patch.jar
50. org.talend.remote.server-7.3.1.20221110_1952_patch.jar
51. org.talend.utils-7.3.1-PATCH.jar
52. org.talend.utils.minimal-7.3.1-PATCH.jar
53. oshi-core-6.2.2.jar
54. jna-5.12.1.jar

55. jna-platform-5.12.1.jar

56. Remove files from <jobserver_home> to replace them with their patched counterparts:

57. start_rs.bat

58. start_rs.sh

59. start_jconsole.bat

60. Recommended change of following configuration properties in /conf/TalendJobserver.properties in case you use SSL:

# Enabled protocols for JobServer socket communication
org.talend.remote.server.ssl.enabled.protocols=TLSv1.2,TLSv1.3

# Enabled protocols for JMX management server
org.talend.jmxmp.ssl.enabled.protocols=TLSv1.2,TLSv1.3

1. Recommended to set following configuration property to false in case you want to disable the Monitoring Port 8888:

# Enable the Monitoring port or not. true by default
org.talend.remote.jobserver.server.TalendJobServer.ENABLE_MONITORING_PORT=true

1. Add the following new configuration properties to <jobserver_home>/conf/TalendJobserver.properties:

# Set to retrieve the job table after restart to avoid redeployment of already deployed jobs
org.talend.remote.jobserver.commons.config.JobServerConfiguration.RESTORE_JOBTABLE=true

# Set to true to unescape context parameters. 
org.talend.remote.jobserver.commons.config.JobServerConfiguration.UNESCAPE_CONTEXTS=true

# Max size in bytes that an unzipped archive is allowed to be. The default is 1G.
org.talend.remote.jobserver.commons.config.JobServerConfiguration.MAX_UNZIPPED_SIZE=1073741824
# Max number of entries allowed in a zipped archive
org.talend.remote.jobserver.commons.config.JobServerConfiguration.MAX_ZIPPED_ENTRIES=2048

If your server side SSL keys are protected by passwords different from the keystore password, you can configure those with the following settings:

# Set password of server side ssl key (command and file server) - optional
org.talend.remote.server.ssl.keyPassword=<jobserver_key_password>

# Set password of server side ssl key (monitoring server) - optional
org.talend.jmxmp.ssl.keyPassword=<monitoring_server_key_password>

It is recommended to set the following configuration property to true:

# Set to true to enable authorization for all jobserver commands (recommended)
org.talend.remote.jobserver.commons.config.JobServerConfiguration.SECURITY_MODE=true

New possibility to disable hostname configuration for the TACClient:

# Set this to true to disable hostname verification for the TACClient
#org.talend.remote.jobserver.commons.config.JobServerConfiguration.TLS_DISABLE_CN_CHECK=true

The following configuration property enables authorization for all job file deployments. This requires that on client-side ( TAC, Studio ) support for file server authorization must be available and the system property org.talend.remote.jobserver.client.old has be set to false. For more details refer to the documentation.

# Set to true to enable authorization for all job file deployments.  
( Requires additional configuration for TAC and Studio. )
org.talend.remote.jobserver.commons.config.JobServerConfiguration.FILESERVER_AUTHORIZATION=true

The following configuration have been added to add constraints on job to prevent Denial Of Service attacks. High default have been defined, it is recommended to adapt these to your environment.

# Maximum number of file listeners, 0 = No limit
org.talend.remote.jobserver.commons.config.JobServerConfiguration.MAX_FILE_LISTENERS=6000

# Maximum number of library dependencies embedded in a job
org.talend.remote.jobserver.commons.config.JobServerConfiguration.MAX_LIBRARY_DESC_NB=1000
# Maximum size of all library dependency names for a job
org.talend.remote.jobserver.commons.config.JobServerConfiguration.MAX_LIBRARY_DESC_SIZE=100KB

# Maximum number of deployed jobs, 0 = No Limit
org.talend.remote.jobserver.commons.config.JobServerConfiguration.MAX_JOB_NB=6000

# Max size that a job archive is allowed to be. The default is 1G, 0 = No limit
org.talend.remote.jobserver.commons.config.JobServerConfiguration.MAX_JOB_FILE_SIZE=1G

# Maximum size of TalendJobServersFiles/archiveJobs folder, 0 = No limit, 0 = No Limit
org.talend.remote.jobserver.commons.config.JobServerConfiguration.MAX_ARCHIVES_DIR_SIZE=100G

1. An empty allow list for impersonation users is not supported anymore. If you want to allow impersonation for anyone and support jobs running without impersonation, you need to explicitly set:

org.talend.remote.jobserver.server.TalendJobServer.RUN_AS_ALLOWLIST=anybody

Please refer to the following examples to understand this setting:

1. Check Read, Write & Execution rights on files

Per default, the rights should be: - rw-r--r-- (i.e. 0664) per default, meaning everybody can read, but only the owner can overwrite the file - rwxr-xr-x (i.e. 0655) for *.sh files, meaning everybody can read & execute, but only the owner can overwrite the file

If needed, feel free to update these rights to match your current installation environment.
Especially to allow further patch updates by a non-owner user, you might want to allow other users to have Write access.

1. Start Jobserver

This requires that the general patch installation ( which also includes the backup of wrapper.conf modified here ) has been performed already.

1. Stop the Windows Service

2. Adapt the wrapper.conf configuration file wrapper.java.classpath. entries to point to the jar files patched with this fix:

3. commons-codec-1.14.jar

4. commons-io-2.8.0.jar
5. commons-lang3-3.11.jar
6. commons-text-1.10.0.jar
7. crypto-utils-5.12.0.jar
8. daikon-5.12.0.jar
9. daikon-exception-5.12.0.jar
10. daikon-signature-verifier-5.12.0.jar
11. httpclient-4.5.13.jar
12. httpcore-4.4.13.jar
13. jackson-core-asl-1.9.16-TALEND.jar
14. reload4j-1.2.18.5.jar
15. slf4j-api-1.7.33.jar
16. slf4j-reload4j-1.7.33.jar
17. org.talend.monitoring-7.3.1.20221110_1952_patch.jar
18. org.talend.monitoring.server-7.3.1.20221110_1952_patch.jar
19. org.talend.remote.commons-7.3.1.20221110_1952_patch.jar
20. org.talend.remote.jobserver.commons-7.3.1.20221110_1952_patch.jar
21. org.talend.remote.jobserver.server.standalone-7.3.1.20221110_1952_patch.jar
22. org.talend.remote.server-7.3.1.20221110_1952_patch.jar
23. org.talend.utils-7.3.1-PATCH.jar
24. org.talend.utils.minimal-7.3.1-PATCH.jar
25. oshi-core-6.2.2.jar
26. jna-5.12.1.jar

27. jna-platform-5.12.1.jar

28. Remove entries for

29. commons-beanutils-1.9.3

30. commons-configuration2-*.jar

( Includes changes possibly done for JobServer running as Windows service. )

1. Stop Jobserver.

2. Remove the following files

3. lib/commons-codec-1.14.jar

4. lib/commons-io-2.8.0.jar
5. lib/commons-lang3-3.11.jar
6. lib/commons-text-1.10.0.jar
7. lib/crypto-utils-5.12.0.jar
8. lib/daikon-5.12.0.jar
9. lib/daikon-exception-5.12.0.jar
10. lib/daikon-signature-verifier-5.12.0.jar
11. lib/httpcore-4.4.13.jar
12. lib/httpclient-4.5.13.jar
13. lib/jackson-core-asl-1.9.16-TALEND.jar
14. lib/reload4j-1.2.18.5.jar
15. lib/slf4j-api-1.7.33.jar
16. lib/slf4j-reload4j-1.7.33.jar
17. lib/org.talend.monitoring-7.3.1.20221110_1952_patch.jar
18. lib/org.talend.monitoring.server-7.3.1.20221110_1952_patch.jar
19. lib/org.talend.remote.commons-7.3.1.20221110_1952_patch.jar
20. lib/org.talend.remote.jobserver.commons-7.3.1.20221110_1952_patch.jar
21. lib/org.talend.remote.jobserver.server.standalone-7.3.1.20221110_1952_patch.jar
22. lib/org.talend.remote.server-7.3.1.20221110_1952_patch.jar
23. lib/org.talend.utils-7.3.1-PATCH.jar
24. lib/org.talend.utils.minimal-7.3.1-PATCH.jar
25. lib/oshi-core-6.2.2.jar
26. lib/jna-5.12.1.jar
27. lib/jna-platform-5.12.1.jar
28. conf/TalendJobserver.properties
29. conf/wrapper.conf

and restore the unpatched counterparts from your backup

• lib/commons-beanutils-*.jar
• lib/commons-codec-*.jar
• lib/commons-configuration2-*.jar
• lib/commons-io-*.jar
• lib/commons-lang3-*.jar
• lib/commons-text-*.jar
• lib/crypto-utils-*.jar
• lib/daikon-*.jar
• lib/daikon-exception-*.jar
• lib/daikon-signature-verifier-*.jar
• lib/httpclient-*.jar
• lib/httpcore-*.jar
• lib/jackson-core-asl-*.jar
• lib/log4j-*.jar
• lib/slf4j-api-*.jar
• lib/slf4j-log4j12-*.jar
• lib/org.talend.monitoring-7.3.1*.jar
• lib/org.talend.monitoring.server-7.3.1*.jar
• lib/org.talend.remote.commons-7.3.1*.jar
• lib/org.talend.remote.jobserver.commons-7.3.1*.jar
• lib/org.talend.remote.jobserver.server.standalone-7.3.1*.jar
• lib/org.talend.remote.server-7.3.1*.jar
• lib/org.talend.utils*-7.3.1.jar
• lib/oshi-core-*.jar
• lib/jna-*.jar
• lib/jna-platform-*.jar
• conf/TalendJobserver.properties

• conf/wrapper.conf

• Remove the following files and restore the unpatched counterparts from your backup

• start_rs.bat

• start_rs.sh

• start_jconsole.bat

• Start Jobserver

The following files are installed into <jobserver_home>/lib folder by this patch:

• commons-codec-1.14.jar
• commons-io-2.8.0.jar
• commons-lang3-3.11.jar
• commons-text-1.10.0.jar
• crypto-utils-5.12.0.jar
• daikon-5.12.0.jar
• daikon-exception-5.12.0.jar
• daikon-signature-verifier-5.12.0.jar
• httpclient-4.5.13.jar
• httpcore-4.4.13.jar
• jackson-core-asl-1.9.16-TALEND.jar
• reload4j-1.2.18.5.jar
• slf4j-api-1.7.33.jar
• slf4j-reload4j-1.7.33.jar
• org.talend.monitoring-7.3.1.20221110_1952_patch.jar
• org.talend.monitoring.server-7.3.1.20221110_1952_patch.jar
• org.talend.remote.commons-7.3.1.20221110_1952_patch.jar
• org.talend.remote.jobserver.commons-7.3.1.20221110_1952_patch.jar
• org.talend.remote.jobserver.server.standalone-7.3.1.20221110_1952_patch.jar
• org.talend.remote.server-7.3.1.20221110_1952_patch.jar
• org.talend.utils-7.3.1-PATCH.jar
• org.talend.utils.minimal-7.3.1-PATCH.jar
• oshi-core-6.2.2.jar
• jna-5.12.1.jar
• jna-platform-5.12.1.jar

The following files are installed into <jobserver_home> folder by this patch:

• start_rs.bat
• start_rs.sh
• start_jconsole.bat

When the option org.talend.remote.jobserver.commons.config.JobServerConfiguration.LAUNCH_SHELL_SCRIPT was set to false (which is the default value), a script file was generated in : - deployedJobPath/[jobName]/[jobName]_run.bat for Windows - deployedJobPath/[jobName]/[jobName]_run.sh for UNIX

This file will no longer be generated.
Instead, to see executed command please use the debug level log.

The possibility to launch from shell script using option org.talend.remote.jobserver.commons.config.JobServerConfiguration.LAUNCH_SHELL_SCRIPT set to true is deprecated and will be removed in end 2022.