TPS-5424 (cumulative patch)

Introduction

This patch is cumulative. It includes all previous generally available patches for Talend Administration Center 8.0.1.

NOTE: To download this patch, liaise with your Support contact at Talend.

Prerequisites

Consider the following requirements for your system:

• Talend Administration Center 8.0.1 must be installed.

Installation

1. Log in to TAC and switch to Configuration-> Software Update, then enter the correct values and save. Follow the procedure described in the documentation: https://help.talend.com/r/en-US/8.0/installation-guide-big-data-linux/config-update-repo
2. Switch to Software update page, where the new patch will be listed. The patch can be downloaded from here into the nexus repository.
3. Login to local Nexus, and download the patch file.
4. Stop all TAC instances. Repeat the following steps for each instance.
5. Please backup your database (if you meet issues with new patch, you can change to old one with this backup)
6. Create a patch directory (eg: <Talend>/TAC_Patch).
7. Unzip patch file you received from support into this directory, then unzip the org.talend.administrator.war file as org.talend.administrator folder. (Note: Please rename org.talend.administrator-8.0.1.war if your old TAC application folder has a different name. Set the same name as your old TAC application name.)
8. Create a backup directory (eg: <Talend>/TAC_Backup).
9. Copy folder <Tomcat>/webapps/org.talend.administrator into the backup directory. DO NOT place org.talend.administrator backup folder into webapps directory.
10. In <Tomcat>/webapps/ directory, remove the previous org.talend.administrator folder, then copy the org.talend.administrator folder unzipped at step 6 and paste in the current directory.

11. Restore TAC configuration by replacing <Tomcat>/webapps/org.talend.administrator/WEB-INF/classes/configuration.properties and quartz.properties with the same files that are stored in your backup directory.

    Note:

    • Make sure that no other instances of TAC webapp are deployed into Tomcat's webapps folder. Make sure your TAC backup folder has NOT been stored in <Tomcat>/webapps folder.
    • Restore DB driver by copying driver to <Tomcat>/webapps/org.talend.administrator/WEB-INF/lib (available in backup directory <Talend>/TAC_Backup).
    • If your TAC database is H2 db and embedded in TAC web folder (<Tomcat>/webapps/org.talend.administrator/WEB-INF/database by default), don't forget to restore H2 db by replacing this folder with the exact corresponding folder from your backup directory.
    • H2 version in this patch is updated due to security reasons. To migrate to new version of H2, please follow the documentation: https://help.talend.com/r/en-US/8.0/migration-upgrade-guide-big-data/upgrading-the-h2-database-after-changing-h2-driver-to-21210 .
    • If your TAC works with SSO, you should restore the IDP Metadata file (<Tomcat>/webapps/org.talend.administrator/WEB-INF/classes/IDPMetadata.xml) from your backup directory.
    • After the step 9, log4j 1.x libraries should have been removed from the folder: <Tomcat>/webapps/org.talend.administrator/WEB-INF/lib.

12. Restart TAC.

    Note:

    • It's recommended to clear browser cache after TAC patch has been applied.
    • Log4j CVE-2021-44228 & CVE-2021-45046 fixed on Patch_20211223_TPS-5053_v1: please rebuild the jobs with latest Studio patch.
    • New LDAP connection timeout parameter: ldap.config.timeout. You can change it by editing the value of the ldap.config.timeout property in milliseconds in the database configuration table.
    • In case of patch rollback, only the backup database can be used

TPS-5424

CVEs fixed in TPS-5424

• TAC-17558 [8.0.1] Update CXF library to version 3.5.5
• TAC-17489 [8.0.1] CVE: CVE-2022-40154 com.thoughtworks.xstream:xstream:1.4.19(to 1.4.20)
• TAC-17596 [8.0.1] CVE-2022-45693: Vulnerable lib Jettison 1.5.1 found in TAC
• TAC-17591 [8.0.1] CVE-2022-1471: Vulnerability was found in library SnakeYAML version 1.32
• TAC-17594 [8.0.1] CVE-2022-40152: com.fasterxml.woodstox:woodstox-core:6.​2.​7

Other issues fixed in TPS-5424

• TAC-16293 [8.0.1] TAC will hang up if a task with a specific job enabling Statistics on the task is running
• TAC-17353 [8.0.1] Issue adding a new LDAP user in TAC
• TAC-17555 [8.0.1] Add a description into TAC cumulative patch Release Note
• TAC-17560 [8.0.1] "Storage period for generated Jobs" is not working.
• TAC-17569 [8.0.1] Add index for some table in MSSQL, Postgre, Oracle
• TAC-17608 [8.0.1] Metaservlet TAC database migration from Postgres to SQL Server is not working
• TPS-5420 [8.0.1] Update CXF library to version 3.5.5 (TAC-17558)
• TPS-5424 [8.0.1] Cumulative Patch - 20230119

Fixed issues

This patch is cumulative and contains the following fixes:

• TAC-14830 [8.0.1] Consolidate InetUtil RunIfConfigCommand methods
• TAC-15654 [8.0.1] Improve the error handle and print necessary error message
• TAC-14895 [8.0.1] Irrelevant warning when edit user group
• TAC-15954 [8.0.1] URL returned blank when adding administrator at the end of TAC URL
• TAC-15910 [8.0.1] NPE when saving LDAP user with non-existing DN
• TAC-15898 [8.0.1] TAC continues to work though set auditlog.failure.stopActivity to true
• TAC-14907 [8.0.1] error accessing runtime page, via a reverse proxy (F5)
• TAC-15899 [8.0.1] Error when undeploying ESB task
• TAC-15951 [8.0.1] migrate libraries : not all artifacts from org.talend.libraries are migrated from old to new nexus
• TAC-15967 [8.0.1] edit user group which have user assigned will throw 500 error
• TAC-15992 [8.0.1] Forgot password should be executed for existing and not existing user for the same time
• TAC-15897 [8.0.1] A task running by a plan with a custom context will run with default context at times
• TAC-15823 [8.0.1] Default context is not changed though removed from later version
• TAC-15894 [8.0.1] Task status in execution details are always in running when job server host ip is unavailable
• TAC-15878 [8.0.1] metaservlet projectExist didn't work as expected
• TAC-15778 [8.0.1] Add missing reset context audit log
• TPS-5028 [8.0.1] DBConfig page show username and password is not correct and license can not be imported (TAC-15880)
• TAC-16001 [8.0.1] Context parameters not displaying in TAC
• TAC-16022 [8.0.1] RemoteDataRetriver never shutdown for execution when jobserver is unreachable
• TPS-5053 [8.0.1] Log4j CVE-2021-44228/CVE-2021-45046 on TAC (TAC-16076)
• TAC-15962 [8.0.1] TAC upgraded to TPS-4989 then startup too long time
• TAC-16060 [8.0.1] Execution log is not immediately displayed though task has finished running
• TAC-16065 [8.0.1] Upper / Lower Panels in the ERROR RECOVERY MANAGEMENT page not "synchronized"
• TAC-16127 [8.0.1] Cannot see context in one of TAC in a cluster
• TAC-16121 [8.0.1] TAC patch list does not manage continuation_token from nexus
• TAC-16126 [8.0.1] FileNotFoundException error when deploy a task which enabled "Use Latest Version"
• TAC-15776 [8.0.1] Delete task/plan print details in business log regarding task/plan deleted
• TAC-15917 [8.0.1] Null Pointer exception while browsing through the tasks in Job Conductor Tab
• TAC-16148 [8.0.1] ExecutionPlan Page refresh has the 500 client error
• TAC-16190 [8.0.1] Faild to execute metaservlet with the error 'password for Db config is incorrect.
• TPS-5079 [8.0.1] TAC Log4j CVE-2021-44832: update to Log4j 2.17.1 (TAC-16203)
• TPS-5089 [8.0.1] CVE-2021-42392 - Disable Remote H2 Console Access (TAC-16214)
• TAC-15513 [8.0.1] "scheduler.conf.retryRestartTaskWhenConnectionServerFailed" to be used by Tasks in Execution Plans
• TAC-16300 [8.0.1] Jobconductor task hanging on "1 awaiting exec"
• TAC-16282 [8.0.1] after login tac via SSO, cannot see full properties
• TAC-16245 [8.0.1] Metaservlet 'removeServerProjectAuthorization' faild with 'Cannot commit transaction'
• TAC-16246 [8.0.1] "String index out of range: -1" for MetaServlet-> runTask with empty context {}
• TAC-16280 [8.0.1] DB Migration failure from 721, 731 to 801 regarding DeprecatedFeaturesOn801Migration
• TAC-16277 [8.0.1] TAC's DB issue when deploying ESB Tasks after patch
• TAC-16249 [8.0.1] Cannot update a task when task name and plan name are the same
• TAC-13275 [8.0.1] Unable to import user with xml file
• TPS-5129 [8.0.1] TAC v801 Migration Failed, all data has been deleted on executionplanpart table by TAC migration (TAC-16341)
• TAC-16284 [8.0.1] No errors thrown on all migration Operations
• TAC-16343 [8.0.1] Message need update when add one new longer license on License page
• TAC-16202 [8.0.1] Too many segment logs when debug threshold is set
• TPS-5135 [8.0.1] TAC task duration is at least 10 seconds greater than job duration (TAC-16198)
• TAC-16413 [8.0.1] Configuration page showing endless Refresh
• TAC-16400 [8.0.1] jgit hangs/sleep in FS.FileStoreAttributeCache step on Git Project Connection checking
• TAC-16304 [8.0.1] Customer doesn't see his admin users
• TAC-13275 [8.0.1] Unable to import user with xml file
• TAC-16335 [8.0.1] Job running on Jobserver is killed unexpectedly
• TAC-16198 [8.0.1] TAC task duration is at least 10 seconds greater than job duration
• TAC-16460 [8.0.1] java.lang.NoSuchMethodError: org.apache.log4j.MDC.put error when upload license
• TAC-15911 [8.0.1] Apply schema change automatically
• TAC-16442 [8.0.1] Cannot edit TAC projects with empty credential
• TAC-16474 [8.0.1] TAC latest patch v8.0.1 with log2 doen`t log events
• TAC-16368 [8.0.1] Investigate "Trigger-Runner" what is he used for
• TAC-16468 [8.0.1] Change in behavior for getTaskIdByName metaservlet call
• TAC-16497 [8.0.1] Migration failed when upgrading Postgres DB to TAC 8.0
• TAC-16333 [8.0.1] Update default value for ldap connection timeout to 30s
• TAC-16420 [8.0.1] Talend2 - 02 - Database authentication testing endpoint is not authenticated
• TAC-16516 [8.0.1] Use default value jobserver.useCache=true when having DB connection problem
• TAC-16546 [8.0.1] Fix TAC name error in MetaServlet command help
• TAC-16513 [8.0.1] TAC 731 - H2 DB to Oracle Migration not recognizing the License in the Oracle Database
• TAC-16555 [8.0.1] Attribute:'svnid' not present while adding users in TAC using LDAP with SVN as storage
• TAC-16147 [8.0.1] TAC role don't sync when update tac role from sso
• TAC-16370 [8.0.1] "DBException: task not found exception" when tasked deleted from metaservlet ->runTask and Jobconductor UI is still refreshing on it
• TAC-16494 [8.0.1] The trigger info on plan is lost
• TAC-16561 [8.0.1] Trigger name left ' is lost in File trigger
• TPS-5189 [8.0.1] Talend2 - 01 - XXE processing vulnerability (TAC-16390)
• TAC-16598 [8.0.1] Metaservlet command failed for createSandboxProject
• TAC-16610 [8.0.1] Find possibility to enable hibernate.generate_statistics in TAC hibernate
• TAC-16327 [8.0.1] Migration failed on executionplanpartcontextprmsid column from mysql to postgresql executionplanpartcontextprmsid using Metaservelet-> migrateDatabase
• TAC-16626 [8.0.1] Metaservlet command "listUsers" doesn`t show users ldap parameters
• TAC-16309 [8.0.1] When Set business log limit by: Time, it can happen that all business log files are deleted and no new file created
• TAC-16519 [8.0.1] SSO - Support for keycloak
• TAC-15771 [8.0.1] Generate a Personal Access Token from TAC metaservlet
• TAC-16313 [8.0.1] Skip Backup option during TAC-Migration
• TAC-16536 [8.0.1] cannot deploy and run normal task deployed as zip after jobserver reboot
• TAC-16683 [8.0.1] Stop & start features in ESBConductor are not working
• TPS-5233 [8.0.1] CVE-2022-31648: SSOUtils.buildErrorPage doesn't escape the error message (TAC-16644)
• TPS-5245 [8.0.1] TAC connection to Nexus behind proxy(TAC-16445)
• TAC-16704 [8.0.1] Fix ConcurrentModificationException in RealtimeDataParser
• TAC-16695 [8.0.1] missing realtime statistics from older executions
• TAC-15218 [8.0.1] add checksum in software update for the download of patch
• TAC-16554 [8.0.1] Add innodbstrictmode=OFF setting in DB config file
• TPS-5255 [8.0.1] Transaction deadlocked with SQL Server (TAC-16738)
• TAC-16801 [8.0.1] Notification isn't send for 'On user deletion' event when deleting user with metaservlet
• TAC-16834 [8.0.1] Reset password: typo in error message
• TAC-16743 [8.0.1] org.hibernate.HibernateException: Illegal attempt to associate a collection with two open sessions
• TAC-16303 [8.0.1] TAC real time statistics do not work sometimes.
• TAC-16858 [8.0.1] Not all connection results are visible in real time statistics
• TAC-16856 [8.0.1] Execution Plan Name not available in Triggered by Section in Job Conductor
• TAC-16703 [8.0.1] No error message when project is NPA and role is admin when login from SSO
• TAC-16770 [8.0.1] Limit the number of patches on SoftwareUpdate page
• TAC-16643 [8.0.1] TAC is updating completed tasks after service restart and triggering misfire notifications
• TPS-5281 [8.0.1] The interaction between tds and scim takes more time than 721 in 801 (TAC-16753)
• TAC-16495 [8.0.1] TAC Execution Plan stuck in Status "Killing"
• TAC-16621 [8.0.1] Add in Audit logs actions on Personal Tokens for TAC
• TAC-16761 [8.0.1] use Long for execution task parameter id
• TAC-16897 [8.0.1] Unable to display/update context parameter using API while publishing a new version of job
• TAC-16909 [8.0.1] No token set error on TAC DB config page
• TAC-16958 [8.0.1] New added context in jobconductor will disappears after running artifact task
• TAC-16982 [8.0.1] Plan: delete parameter in plan, but it is still referenced in context parameter
• TAC-17009 [8.0.1] The EP status should be interrupted when EP is not parallel execution
• TAC-17021 [8.0.1] Create task failed when artifact with context (H2 db)
• TPS-5297 [8.0.1] The job always keep "running" when stop jobserver(TAC-16988)
• TAC-17014 [8.0.1] Delete custom context parameter need a extra refresh to see parameter disappear
• TAC-17026 [8.0.1] Metaservlet help all for revokePersonalAccessTokenOfUser need update
• TAC-17035 [8.0.1] Rollback does not work when EP is killed by timeout
• TAC-17057 [8.0.1] Contains the multiple repeat keys when export the config parameters
• TPS-5324 [8.0.1] Metaservlet: 'Cannot flush and commit transaction' when deleting ESB task(TAC-16884)
• TAC-15432 [8.0.1] add the ability to include or not the logs in attachment (or, at least, zip the attachment)
• TAC-17044 [8.0.1] Facing issue in servers page of TAC when trying to edit the name in the label section.
• TAC-17076 [8.0.1] Migration faild from 72 to 73/801
• TAC-16790 [8.0.1] task status set to "Ended with Warning" and could not be triggered anymore
• TPS-5329 [8.0.1] Convert @ when Artifact Repository user name contains this symbol(TAC-17121)
• TAC-17056 [8.0.1] Integrate with authentication feature for JobServer's FileServer
• TAC-15590 [8.0.1] Proxy server authentication not working
• TAC-17157 [8.0.1] Authorization Resource/Role assignments not properly refreshed
• TAC-17184 [8.0.1] Update context from default and custom save it will show error after deploy a new version.
• TAC-17265 [8.0.1] Unable to create tasks with TPS-5329
• TPS-5344 [8.0.1] "use latest version" : the latest artifact version is not always selected - continuation token (TAC-17158)
• TAC-17176 [8.0.1] Master key encoded wrongly when running service in Japanese locale
• TAC-17177 [8.0.1] "use latest version" is not the latest job for the job order in jfrog is not same as studio
• TAC-17181 [8.0.1] migrateDatabase command Source=Oracle Target=PostgreSQL : creates empty tables in postgreSQL DB
• TPS-5357 [8.0.1] job server high availability via virtual job server does not work (TAC-17249)
• TAC-17295 [8.0.1] Version: 500 The call failed on the server after apply the latest TAC patch (SqlServer with jtds driver)
• TPS-5358 [8.0.1] Metaservlet migratedatabase action does not work between mysql and mssql (TAC-17248)
• TAC-17304 [8.0.1] Old context parameter names not removed when updating task manually in TAC or using contextParamsRefresh=false with MetaServlet
• TAC-17362 [8.0.1] reset context parameter result in emply context (blank) with TPS-5343 if generatedJobs folder path is non canonical
• TAC-17373 [8.0.1] Wrong unit for maxDurationBeforeCleaningOldJobs maxDurationBeforeCleaningOldExecutionsLogs, but doc showing days as unit
• TAC-17389 [8.0.1] job status stuck "running" if using postgres DB, and job generating "null" in job logs
• TAC-17393 [8.0.1] Duplicate entry XXX for key 'executiontaskjobprm.PRIMARY'
• TAC-17443 [8.0.1] execution plan doesn't show in ui after creating with oracle database
• TPS-5384 [8.0.1] CVE-2022-42889: Update lib apache.commons-text (TAC-17340)
• TAC-17443 [8.0.1] execution plan doesn't show in ui after creating with oracle database
• TAC-17475 [8.0.1] Task Status are not updating in TAC UI
• TAC-17500 [8.0.1] Deadlock when reset task on jobserver timeout (Postgres Sql)
• TPS-5393 [8.0.1] Talend jobs getting killed automatically in 7.3.1(TAC-17432)
• TPS-5408 [8.0.1] Big data streaming Conductor could not list, deploy and run(TAC-17433,TAC-17474)
• TAC-17525 [8.0.1] Metaservlet not able to read context from Artifact
• TPS-5411 [8.0.1] Update jobserver client version for issue TPRUN-4892 and TPRUN-4898

Security fixes

This patch includes the security fixes:

• TAC-15950 [8.0.1] Vulnerability in "forgot password" functionality in TAC
• TAC-16115 [8.0.1] TAC - Log4j2 CVE-2021-45105 DOS attack Fix - Version (2.17.0 update)
• TAC-15298 [8.0.1] Talend - 01 - OTG-INFO-005 - Review Webpage Comments and Metadata for Information Leakage
• TAC-16213 [8.0.1] Update H2 dependency to 2.0.206
• TAC-16344 [8.0.1] Update H2 dependency to 2.1.210
• TAC-16286 [8.0.1] Migration from log4j1 to log4j2 (update to 2.17.1v)
• TAC-16390 [8.0.1] CVE-2022-29943: Talend2 - 01 - XXE
• TAC-16407 [8.0.1] CVE-2022-29942: Talend2 - 03 - SSRF
• TAC-16486 [8.0.1] Vulnerable library Liquibase
• TAC-16487 [8.0.1] Vulnerable library JDOM
• TAC-16567 [8.0.1] CVE-2021-43859: Vulnerable library XStream Core 1.4.18
• TAC-16568 [8.0.1] CVE-2020-36518: Vulnerable library jackson-databind 2.12.2
• TAC-16644 [8.0.1] CVE-2022-31648: SSOUtils.buildErrorPage doesn't escape the error message
• TAC-16668 [8.0.1] Update to Apache CXF 3.5.2 for TAC
• TAC-16792 [8.0.1] Session creation is insecure
• TAC-16794 [8.0.1] For cookie "dbadminsession" HttpOnly needs to be added
• TAC-14807 [8.0.1] Fix possible SQL Injection issues
• TAC-16833 [8.0.1] Update studio-utils to 1.0.8 version
• TAC-16855 [8.0.1] CVE-2021-41303: Vulnerable library Apache Shiro update to v 1.9.0
• TAC-16870 [8.0.1] CVE-295: Insecure HostnameVerifier implementation on NetIQ plugin
• TAC-16977 [8.0.1] CVE-2022-32532: Update apache shiro to 1.9.1 version
• TAC-16978 [8.0.1] CVE-2022-25647: Update Gson lib to version 2.9.0
• TAC-16979 [8.0.1] CVE-2022-23221: Update H2 Database Engine to version 2.1.214
• TAC-16980 [8.0.1] CVE-2021-26291: Update Maven Core to version 3.8.6
• TAC-17017 [8.0.1] CVE-2022-33980: Update Apache Commons Configuration to version 2.8.0
• TAC-16985 [8.0.1] Implement file path traversal guards
• TAC-15749 [8.0.1] Make sure CRLF characters are removed from MailSender
• TAC-16959 [8.0.1] Ensure output is encoded
• TAC-17205 [8.0.1] CVE-2018-5382: Update Bouncy Castle Provider to version 1.69
• TAC-17227 [8.0.1] Remove default credentials to nexus and artifactory
• TAC-17270 [8.0.1] Fix Veracode SAST Output Log Neutralization issues
• TAC-17331 [8.0.1] CVE-2022-23437: Vulnerable lib Xerces 2.12.0 found in TAC
• TAC-17329 [8.0.1] CVE-2021-37136: Vulnerable lib netty-codec 4.1.54.Final found in TAC
• TAC-17330 [8.0.1] CVE-2022-40150: Vulnerable lib Jettison 1.4.0 found in TAC
• TAC-17332 [8.0.1] CVE-2022-40664: Vulnerable lib shiro-web found in TAC
• TAC-17340 [8.0.1] CVE-2022-42889: Update lib apache.commons-text
• TAC-17352 [8.0.1] CVE-2022-30973: Vulnerability found in org.apache.tika:tika version1.24.1
• TAC-17354 [8.0.1] CVE-2022-42003: Vulnerable library jackson-databind was found in TAC
• TAC-17424 [8.0.1] CVE-2022-25857: Vulnerability was found in library SnakeYAML version 1.26
• TAC-17426 [8.0.1] CVE-2021-20293: Vulnerability was found in library RestEasy core version 4.5.10.Final
• TAC-17482 [8.0.1] CVE: commons-codec:commons-codec:1.11(to 1.15)
• TAC-17483 [8.0.1] CVE: CVE-2022-36033 org.jsoup:jsoup:1.14.2 (to 1.15.3)
• TAC-17542 [8.0.1] CVE-2021-33813: Remove vulnerable jdom-1.1 from project
• TAC-17549 [8.0.1] Vulnerability found in org.json:org.json:20120509 and org.json:json:20140107
• TAC-17553 [8.0.1] Update Pax URL Aether
• TAC-17541 [8.0.1] Update CXF library to version 3.5.2
• TAC-17546 [8.0.1] CVE-2019-7611: Vulnerability found old in org.elasticsearch:elasticsearch 2.4.3