URI format and Options - 6.3

Talend ESB Mediation Developer Guide

EnrichVersion
6.3
EnrichProdName
Talend Data Fabric
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
task
Design and Development
EnrichPlatform
Talend ESB
ftp://[username@]hostname[:port]/directoryname[?options]
sftp://[username@]hostname[:port]/directoryname[?options]
ftps://[username@]hostname[:port]/directoryname[?options]

Where directoryname represents the underlying directory, which can contain nested folders.

If no username is provided, then anonymous login is attempted using no password. If no port number is provided, Camel will provide default values according to the protocol (ftp = 21, sftp = 22, ftps = 2222).

You can append query options to the URI in the following format, ?option=value&option=value&..., where option can be:

Table 9. 

Name

Default Value

Description

username

null

Specifies the username to use to log into the remote file system.

password

null

Specifies the password to use to log into the remote file system.

binary

false

Specifies the file transfer mode, BINARY or ASCII. Default is ASCII ( false ).

disconnect

false

Whether or not to disconnect from remote FTP server right after use. Can be used for both consumer and producer. Disconnect will only disconnect the current connection to the FTP server. If you have a consumer which you want to stop, then you need to stop the consumer/route instead.

localWorkDirectory

null

When consuming, a local work directory can be used to store the remote file content directly in local files, to avoid loading the content into memory. This is beneficial if you consume a very big remote file and thus can conserve memory. See below for more details.

passiveMode

false

FTP and FTPS only : Specifies whether to use passive mode connections. Default is active mode ( false ).

securityProtocol

TLS

FTPS only: Sets the underlying security protocol. The following values are defined: TLS : Transport Layer Security SSL : Secure Sockets Layer

disableSecureData-ChannelDefaults

false

FTPS only : Whether or not to disable using default values for execPbsz and execProt when using secure data transfer. You can set this option to true if you want to be in absolute full control what the options execPbsz and execProt should be used.

download

true

Starting with Camel 2.11, whether the FTP consumer should download the file. If this option is set to false, then the message body will be null, but the consumer will still trigger a Camel Exchange that has details about the file such as file name, file size, etc. It's just that the file will not be downloaded.

streamDownload

false

Starting with Camel 2.11, whether the consumer should download the entire file up front, the default behavior, or if it should pass an InputStream read from the remote resource rather than an in-memory array as the in-body of the Camel Exchange. This option is ignored if download is false or if localWorkDirectory is provided. This option is useful for working with large remote files.

execProt

null

FTPS only : This will use option P by default, if secure data channel defaults hasn't been disabled. Possible values are: C : Clear S : Safe (SSL protocol only) E : Confidential (SSL protocol only) P : Private

execPbsz

null

FTPS only : This option specifies the buffer size of the secure data channel. If option useSecureDataChannel has been enabled and this option has not been explicit set, then value 0 is used.

isImplicit

false

FTPS only: Sets the security mode(implicit/explicit). Default is explicit ( false ).

knownHostsFile

null

SFTP only: Sets the known_hosts file, so that the SFTP endpoint can do host key verification.

knownHostsUri

null

SFTP only: Sets the known_hosts file (loaded from classpath by default), so that the SFTP endpoint can do host key verification.

keyPair

null

SFTP only: Sets the Java KeyPair for SSH public key authentication, it supports DSA or RSA keys.

privateKeyFile

null

SFTP only: Set the private key file to that the SFTP endpoint can do private key verification.

privateKeyUri

null

SFTP only: Set the private key file (loaded from classpath by default) to that the SFTP endpoint can do private key verification.

privateKey

null

SFTP only: Set the private key as byte[] to that the SFTP endpoint can do private key verification.

privateKeyFilePassphrase

null

SFTP only: Deprecated: use privateKeyPassphrase instead. Set the private key file passphrase to that the SFTP endpoint can do private key verification.

privateKeyPassphrase

null

SFTP only: Set the private key file passphrase to that the SFTP endpoint can do private key verification.

preferredAuthentications

null

SFTP only: Set the preferred authentications which SFTP endpoint will used. Some example include:password,publickey. If not specified the default list from JSCH will be used.

ciphers

null

A comma separated list of ciphers that will be used in order of preference. Possible cipher names are defined by JCraft JSCH. Some examples include: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-cbc,aes256-cbc. If not specified the default list from JSCH will be used.

fastExistsCheck

false

If true, camel-ftp will use the list file directly to check if the file exists. Since some FTP servers may not support listing the file directly, if the option is false, camel-ftp will use the old way to list the directory and check if the file exists. Note this option also influences readLock=changed to control whether it performs a fast check to update file information or not. This can be used to speed up the process if the FTP server has a lot of files.

strictHostKeyChecking

no

SFTP only: Sets whether to use strict host key checking. Possible values are: no, yes and ask. Note: ask does not make sense to use as Camel cannot answer the question for you as it is meant for human intervention.

maximumReconnectAttempts

3

Specifies the maximum reconnect attempts Camel performs when it tries to connect to the remote FTP server. Use 0 to disable this behavior.

reconnectDelay

1000

Delay in milliseconds Camel will wait before performing a reconnect attempt.

connectTimeout

10000

the connect timeout in milliseconds. This corresponds to using ftpClient.connectTimeout for the FTP/FTPS. For SFTP this option is also used when attempting to connect.

soTimeout

null

FTP and FTPS Only: the SocketOptions.SO_TIMEOUT value in milliseconds. Note SFTP will automatic use the connectTimeout as the soTimeout .

timeout

30000

FTP and FTPS Only: the data timeout in milliseconds. This corresponds to using ftpClient.dataTimeout for the FTP/FTPS. For SFTP there is no data timeout.

throwExceptionOnConnect-Failed

false

Whether or not to throw an exception if a successful connection and login could not be established. This allows a custom pollStrategy to deal with the exception, for example to stop the consumer.

siteCommand

null

FTP and FTPS Only: To execute site commands after successful login. Multiple site commands can be separated using a new line character (\n). Use help site to see which site commands your FTP server supports.

stepwise

true

Whether or not stepwise traversing directories should be used or not. Stepwise means that it will 'cd' one directory at a time. See more details below. You can disable this in case you can't use this approach.

separator

Auto

Dictates what path separator char to use when uploading files. Auto = Use the path provided without altering it. UNIX = Use unix style path separators. Windows = Use Windows style path separators.

ftpClient

null

FTP and FTPS Only: Allows you to use a custom org.apache.commons.net.ftp.FTPClient instance.

ftpClientConfig

null

FTP and FTPS Only: Allows you to use a custom org.apache.commons. net.ftp.FTPClientConfig instance.

ftpClient.trustStore.file

null

FTPS Only: Sets the trust store file, so that the FTPS client can look up for trusted certificates.

ftpClient.trustStore.type

JKS

FTPS Only: Sets the trust store type.

ftpClient.trustStore. algorithm

SunX509

FTPS Only: Sets the trust store algorithm.

ftpClient.trustStore. password

null

FTPS Only: Sets the trust store password.

ftpClient.keyStore.file

null

FTPS Only: Sets the key store file, so that the FTPS client can look up for the private certificate.

ftpClient.keyStore.type

JKS

FTPS Only: Sets the key store type.

ftpClient.keyStore. algorithm

SunX509

FTPS Only: Sets the key store algorithm.

ftpClient.keyStore. password

null

FTPS Only: Sets the key store password.

ftpClient.keyStore. keyPassword

null

FTPS Only: Sets the private key password.

sslContextParameters

null

FTPS Only: Reference to a org.apache.camel.util.jsse.SSLContext

Parameters in the Registry.  This reference overrides any configured SSL related options on ftpClient as well as the securityProtocol (SSL, TLS, etc.) set on FtpsConfiguration.  See Using the JSSE Configuration Utility.

proxy

null

SFTP Only: Reference to a com.jcraft.jsch.Proxy in the Registry.  This proxy is used to consume/send messages from the target SFTP host.

useList

true

FTP/FTPS Only: Whether the consumer should use FTP LIST command to retrieve directory listing to see which files exists. If this option is set to false, then stepwise=false must be configured, and also fileName must be configured to a fixed name, so the consumer knows the name of the file to retrieve. When doing this only that single file can be retrieved. See further below for more details.

ignoreFileNotFoundOr

PermissionError

false

Whether the consumer should ignore when a file was attempted to be retrieved but did not exist (for some reason), or failure due insufficient file permission error.


Note besides those listed below, all options from the File are inherited and hence available to the FTP component.

Important

By default, the FTPS component trust store accepts all certificates. If you only want to trust selective certificates, you have to configure the trust store with the ftpClient.trustStore.xxx options or by configuring a custom ftpClient.

You can configure additional options on the ftpClient and ftpClientConfig from the URI directly by using the ftpClient. or ftpClientConfig. prefix.

For example to set the setDataTimeout on the FTPClient to 30 seconds you can do:

from("ftp://foo@myserver?password=secret&ftpClient.dataTimeout=30000")
   .to("bean:foo");

You can mix and match and have use both prefixes, for example to configure date format or timezones.

from("ftp://foo@myserver?password=secret&" +
   "ftpClient.dataTimeout=30000&ftpClientConfig.serverLanguageCode=fr")
   .to("bean:foo");

You can have as many of these options as you like.

See the documentation of the Apache Commons FTP FTPClientConfig for possible options and more details, and also Apache Commons FTP FTPClient.

If you do not like having complex configurations inserted in the url you can use ftpClient or ftpClientConfig by letting Camel look in the Registry for it. For example:

<bean id="myConfig" class="org.apache.commons.net.ftp.FTPClientConfig">
   <property name="lenientFutureDates" value="true"/>
   <property name="serverLanguageCode" value="fr"/>
</bean>

And then let Camel lookup this bean when you use the # notation in the url.

from("ftp://foo@myserver?password=secret&ftpClientConfig=#myConfig")
   .to("bean:foo");