Camel Component: HTTP4 - 6.3

Talend ESB Mediation Developer Guide

EnrichVersion
6.3
EnrichProdName
Talend Data Fabric
Talend Data Services Platform
Talend ESB
Talend MDM Platform
Talend Open Studio for ESB
Talend Real-Time Big Data Platform
task
Design and Development
EnrichPlatform
Talend ESB

The http4: component provides HTTP based endpoints for calling external HTTP resources (as a client to call external servers using HTTP).

Maven users will need to add the following dependency to their pom.xml for this component:

<dependency>
   <groupId>org.apache.camel</groupId>
   <artifactId>camel-http4</artifactId>
   <!-- use the same version as your Camel core version -->
   <version>x.x.x</version>
</dependency>

URI format

http4:hostname[:port][/resourceUri][?options]

This will by default use port 80 for HTTP and 443 for HTTPS.

You can append query options to the URI in the following format, ?option=value&option=value&...

Important

Should you use camel-http4 or camel-jetty? You can produce only to endpoints generated by the HTTP4 component. Therefore it should never be used as input into your Camel routes. To bind/expose an HTTP endpoint via a HTTP server as input to a Camel route, use the Jetty Component instead.

HttpComponent Options

Table 10. 

Name

Default Value

Description

maxTotalConnections

200

The maximum number of connections.

connectionsPerRoute

20

The maximum number of connections per route.

cookiestore

null

Camel 2.11.2/2.12.0: To use a custom org.apache.

http.client.CookieStore. By default the org.apache.http.impl.

client.BasicCookieStore is used which is an in-memory only cookie store. Notice if bridgeEndpoint=true then the cookie store is forced to be a noop cookie store as cookies shouldn't be stored as we are just bridging (eg acting as a proxy).

httpClientConfigurer

null

Reference to a org.apache.camel.component.

http.HttpClientConfigurer in the Registry.

clientConnectionManager

null

To use a custom org.apache.http.

conn.ClientConnectionManager .

httpBinding

null

To use a custom org.apache.camel.

component.http.HttpBinding.

httpContext

null

To use a custom org.apache.http.

protocol.HttpContext when executing requests.

sslContextParameters

null

To use a custom org.apache.camel.util.

jsse.SSLContextParameters. See Using the JSSE Configuration Utility. Important: Only one instance of org.apache.camel.util.

jsse.SSLContextParameters is supported per HttpComponent. If you need to use 2 or more different instances, you need to define a new HttpComponent per instance you need. See further below for more details.

x509HostnameVerifier

BrowserCompatHost

nameVerifier

Camel 2.7You can refer to a different org.apache.http.conn.

ssl.X509HostnameVerifier

instance in the Registry such as org.apache.http.conn.

ssl.StrictHostnameVerifier

or org.apache.http.conn.ssl.

AllowAllHostnameVerifier.

connectionTimeToLive

-1

Camel 2.11.0: The time for connection to live, the time unit is millisecond, the default value is always keep alive.

authenticationPreemptive

false

Camel 2.11.3/2.12.2: If this option is true, camel-http4 sends preemptive basic authentication to the server.


HttpEndpoint Options

Table 11. 

Name

Default Value

Description

throwExceptionOnFailure

true

Option to disable throwing the HttpOperationFailedException in case of failed responses from the remote server. This allows you to get all responses regardless of the HTTP status code.

bridgeEndpoint

false

If true, HttpProducer will ignore the Exchange.HTTP_URI header, and use the endpoint's URI for requests. You may also set the throwExceptionOnFailure to be false to let the HttpProducer send all the fault response back. Also if set to true HttpProducer and CamelServlet will skip the gzip processing if the content-encoding is "gzip".

clearExpiredCookies

true

Camel 2.11.2/2.12.0: Whether to clear expired cookies before sending the HTTP request. This ensures the cookies store does not keep growing by adding new cookies which is newer removed when they are expired.

cookieStore

null

Camel 2.11.2/2.12.0: To use a custom org.apache.http.

client.CookieStore. By default the org.apache.http.impl.

client.BasicCookieStore is used which is an in-memory only cookie store. Notice if bridgeEndpoint=true then the cookie store is forced to be a noop cookie store as cookies shouldn't be stored as we are just bridging (eg acting as a proxy).

disableStreamCache

false

DefaultHttpBinding will copy the request input stream into a stream cache and put it into message body if this option is false to support multiple reads, otherwise DefaultHttpBinding will set the request input stream directly in the message body.

headerFilterStrategy

null

Reference to a instance of org.apache.camel.spi.

HeaderFilterStrategy in the Registry. It will be used to apply the custom headerFilterStrategy on the new create HttpEndpoint.

httpBindingRef

null

Deprecated and will be removed in Camel 3.0: Reference to a org.apache.camel.component.

http.HttpClientConfigurer

in the Registry. Use the httpClientConfigurer option instead.

httpBinding

null

To use a custom org.apache.camel.

component.http.HttpBinding.

httpClientConfigurerRef

null

Deprecated and will be removed in Camel 3.0: Reference to a

org.apache.camel.component.

http.HttpClientConfigurer in the Registry. Use the httpClientConfigurer option instead.

httpClientConfigurer

null

Reference to a org.apache.camel.component.

http.HttpClientConfigurer in the Registry.

httpContextRef

null

Deprecated and will be removed in Camel 3.0:Camel 2.9.2: Reference to a custom org.apache.http.

protocol.HttpContext in the Registry. Use the httpContext instead.

httpContext

null

Camel 2.9.2: To use a custom org.apache.http.

protocol.HttpContext when executing requests.

httpClient.XXX

null

Setting options on the BasicHttpParams. For instance httpClient.soTimeout=5000 will set the SO_TIMEOUT to 5 seconds. Look on the setter methods of the following parameter beans for a complete reference: AuthParamBean, ClientParamBean, ConnConnectionParamBean, ConnRouteParamBean, CookieSpecParamBean, HttpConnectionParamBean and HttpProtocolParamBean

Since Camel 2.13.0: httpClient is changed to configure the HttpClientBuilder and RequestConfig.Builder, please check out API document for a complete reference.

clientConnectionManager

null

To use a custom org.apache.http.

conn.ClientConnectionManager.

transferException

false

If enabled and an Exchange failed processing on the consumer side, and if the caused Exception was send back serialized in the response as a application/x-java-serialized-object content type (for example using Camel Component: Jetty or Camel Component: Servlet Camel components). On the producer side the exception will be deserialized and thrown as is, instead of the HttpOperationFailedException . The caused exception is required to be serialized.

sslContextParametersRef

null

Deprecated and will be removed in Camel 3.0:Camel 2.8: Reference to a org.apache.camel.util.

jsse.SSLContextParameters

in the Registry. Important: Only one instance of org.apache.camel.util.jsse.

SSLContextParameters is supported per HttpComponent. If you need to use 2 or more different instances, you need to define a new HttpComponent per instance you need. See further below for more details. See Using the JSSE Configuration Utility. Use the sslContextParameters option instead.

sslContextParameters

null

Camel 2.11.1: Reference to a org.apache.camel.util.jsse.

SSLContextParameters in the Registry. Important: Only one instance of org.apache.camel.util.jsse.

SSLContextParameters is supported per HttpComponent. If you need to use 2 or more different instances, you need to define a new HttpComponent per instance you need. See further below for more details. See Using the JSSE Configuration Utility.

x509HostnameVerifier

BrowserCompatHost

nameVerifier

Camel 2.7: You can refer to a different org.apache.http.conn.ssl.

X509HostnameVerifier instance in the Registry such as org.apache.http.conn.ssl.

StrictHostnameVerifier or org.apache.http.conn.ssl.

AllowAllHostnameVerifier.

urlRewrite

null

Camel 2.11:Producer only Refers to a custom org.apache.camel.component.

http4.UrlRewrite which allows you to rewrite urls when you bridge/proxy endpoints. See more details at UrlRewrite and How to use Camel as a HTTP proxy between a client and server.

maxTotalConnections

null

Camel 2.14: The maximum number of total connections that the connection manager has. If this option is not set, camel will use the component's setting instead.

connectionsPerRoute

null

Camel 2.14: The maximum number of connections per route. If this option is not set, camel will use the component's setting instead.


The following authentication options can also be set on the HttpEndpoint:

Setting Basic Authentication and Proxy

Before Camel 2.8.0

Name

Default Value

Description

authUsername

null

Username for authentication.

authPassword

null

Password for authentication.

authDomain

null

The domain name for authentication.

authHost

null

The host name authentication.

proxyAuthHost

null

The proxy host name

proxyAuthPort

null

The proxy port number

proxyAuthScheme

null

The proxy scheme, will fallback and use the scheme from the endpoint if not configured.

proxyAuthUsername

null

Username for proxy authentication

proxyAuthPassword

null

Password for proxy authentication

proxyAuthDomain

null

The proxy domain name

proxyAuthNtHost

null

The proxy Nt host name

Since Camel 2.8.0

Name

Default Value

Description

username

null

Username for authentication.

password

null

Password for authentication.

domain

null

The domain name for authentication.

host

null

The host name authentication.

proxyHost

null

The proxy host name

proxyPort

null

The proxy port number

proxyUsername

null

Username for proxy authentication

proxyPassword

null

Password for proxy authentication

proxyDomain

null

The proxy domain name

proxyNtHost

null

The proxy Nt host name

Message Headers

Name

Type

Description

Exchange.HTTP_URI

String

URI to call. This will override existing URI set directly on the endpoint.

Exchange.HTTP_PATH

String

Request URI's path, the header will be used to build the request URI with the HTTP_URI.

Exchange.HTTP_QUERY

String

URI parameters. This will override existing URI parameters set directly on the endpoint.

Exchange. HTTP_RESPONSE_CODE

int

The HTTP response code from the external server. Is 200 for OK.

Exchange. HTTP_CHARACTER_ENCODING

String

Character encoding.

Exchange.CONTENT_TYPE

String

The HTTP content type. Is set on both the IN and OUT message to provide a content type, such as text/html.

Exchange. CONTENT_ENCODING

String

The HTTP content encoding. Is set on both the IN and OUT message to provide a content encoding, such as gzip.

Message Body

Camel will store the HTTP response from the external server on the OUT body. All headers from the IN message will be copied to the OUT message, so headers are preserved during routing. Additionally Camel will add the HTTP response headers as well to the OUT message headers.

Response code

Camel will handle according to the HTTP response code:

  • Response code is in the range 100..299, Camel regards it as a success response.

  • Response code is in the range 300..399, Camel regards it as a redirection response and will throw a HttpOperationFailedException with the information.

  • Response code is 400+, Camel regards it as an external server failure and will throw a HttpOperationFailedException with the information.

    Note

    The option, throwExceptionOnFailure, can be set to false to prevent the HttpOperationFailedException from being thrown for failed response codes. This allows you to get any response from the remote server. There is a sample below demonstrating this.

HttpOperationFailedException

This exception contains the following information:

  • The HTTP status code

  • The HTTP status line (text of the status code)

  • Redirect location, if server returned a redirect

  • Response body as a java.lang.String, if server provided a body as response

Calling using GET or POST

The following algorithm is used to determine whether the GET or POST HTTP method should be used: 1. Use method provided in header. 2. GET if query string is provided in header. 3. GET if endpoint is configured with a query string. 4. POST if there is data to send (body is not null). 5. GET otherwise.

How to get access to HttpServletRequest and HttpServletResponse

You can get access to these two using the Camel type converter system using NOTE You can get the request and response not just from the processor after the camel-jetty or camel-cxf endpoint.

HttpServletRequest request = exchange.getIn().getBody(
   HttpServletRequest.class);
HttpServletRequest response = 
   exchange.getIn().getBody(HttpServletResponse.class);

Configuring URI to call

You can set the HTTP producer's URI directly form the endpoint URI. In the route below, Camel will call out to the external server, oldhost, using HTTP.

from("direct:start").to("http4://oldhost");

And the equivalent Spring sample:

<camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
   <route>
      <from uri="direct:start"/>
      <to uri="http4://oldhost"/>
   </route>
</camelContext>

You can override the HTTP endpoint URI by adding a header with the key Exchange.HTTP_URI on the message.

from("direct:start")
   .setHeader(Exchange.HTTP_URI, constant("http://newhost"))
   .to("http4://oldhost");

In the sample above Camel will call the http://newhost despite the fact the endpoint is configured with http4://oldhost. where Constants is the class, org.apache.camel.component.http4.Constants. If the http4 endpoint is working in bridge mode, it will ignore the Exchange.HTTP_URI message header.

Configuring URI Parameters

The http producer supports URI parameters to be sent to the HTTP server. The URI parameters can either be set directly on the endpoint URI or as a header with the key Exchange.HTTP_QUERY on the message.

from("direct:start").to("http4://oldhost?order=123&detail=short");

Or options provided in a header:

from("direct:start")
   .setHeader(Exchange.HTTP_QUERY, constant("order=123&detail=short"))
   .to("http4://oldhost");

How to set the http method (GET/POST/PUT/DELETE/HEAD/OPTIONS/TRACE) to the HTTP producer

Note: The http PATCH method is supported starting with Camel 2.11.3 / 2.12.1.

The HTTP4 component provides a way to set the HTTP request method by setting the message header. Here is an example;

from("direct:start")
   .setHeader(Exchange.HTTP_METHOD, 
       constant(org.apache.camel.component.http4.HttpMethods.POST))
   .to("http4://www.google.com")
   .to("mock:results");

The method can be written a bit shorter using the string constants:

.setHeader("CamelHttpMethod", constant("POST"))

And the equivalent Spring sample:

<camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
   <route>
      <from uri="direct:start"/>
      <setHeader headerName="CamelHttpMethod">
         <constant>POST</constant>
      </setHeader>
      <to uri="http4://www.google.com"/>
      <to uri="mock:results"/>
   </route>
</camelContext>

Configuring a Proxy

The HTTP4 component provides a way to configure a proxy.

from("direct:start")
   .to("http4://oldhost?proxyHost=www.myproxy.com&proxyPort=80");

There is also support for proxy authentication via the proxyUsername and proxyPassword options.

Using proxy settings outside of URI

To avoid System properties conflicts, you can set proxy configuration only from the CamelContext or URI. Java DSL:

context.getProperties().put("http.proxyHost", "172.168.18.9");
context.getProperties().put("http.proxyPort" "8080");

Spring XML

<camelContext>
   <properties>
      <property key="http.proxyHost" value="172.168.18.9"/>
      <property key="http.proxyPort" value="8080"/>
   </properties>
</camelContext>

Camel will first set the settings from Java System or CamelContext Properties and then the endpoint proxy options if provided. So you can override the system properties with the endpoint options.

Configuring charset

If you are using POST to send data you can configure the charset using the Exchange property:

exchange.setProperty(Exchange.CHARSET_NAME, "ISO-8859-1");

Sample with scheduled poll

This sample polls the Google homepage every 10 seconds and write the page to the file message.html:

from("timer://foo?fixedRate=true&delay=0&period=10000")
   .to("http4://www.google.com")
   .setHeader(FileComponent.HEADER_FILE_NAME, "message.html")
   .to("file:target/google");

URI Parameters from the endpoint URI

In this sample we have the complete URI endpoint that is just what you would have typed in a web browser. Multiple URI parameters can of course be set using the & character as separator, just as you would in the web browser. Camel does no tricks here.

// we query for Camel at the Google page
template.sendBody("http4://www.google.com/search?q=Camel", null);

URI Parameters from the Message

Map headers = new HashMap();
headers.put(Exchange.HTTP_QUERY, "q=Camel&lr=lang_en");
// we query for Camel and English language at Google
template.sendBody("http4://www.google.com/search", null, headers);

In the header value above notice that it should not be prefixed with ? and you can separate parameters as usual with the & char.

Getting the Response Code

You can get the HTTP response code from the HTTP4 component by getting the value from the Out message header with Exchange.HTTP_RESPONSE_CODE .

Exchange exchange = 
   template.send("http4://www.google.com/search", new Processor() {
      public void process(Exchange exchange) throws Exception {
         exchange.getIn().setHeader(
            Exchange.HTTP_QUERY, constant("hl=en&q=activemq"));
      }
   });
Message out = exchange.getOut();
int responseCode = out.getHeader(Exchange.HTTP_RESPONSE_CODE, 
   Integer.class);

Disabling Cookies

To disable cookies you can set the HTTP Client to ignore cookies by adding this URI option: httpClient.cookiePolicy=ignoreCookies

Advanced Usage

If you need more control over the HTTP producer you should use the HttpComponent where you can set various classes to give you custom behavior.

Setting up SSL for HTTP Client

Basically camel-http4 component is built on the top of Apache HTTP client. Please refer to SSL/TLS customization for details or have a look into the org.apache.camel.component.http4.HttpsServerTestSupport unit test base class. You can also implement a custom org.apache.camel.component.http4.HttpClientConfigurer to do some configuration on the http client if you need full control of it.

However if you just want to specify the keystore and truststore you can do this with Apache HTTP HttpClientConfigurer, for example:

KeyStore keystore = ...;
KeyStore truststore = ...;

SchemeRegistry registry = new SchemeRegistry();
registry.register(new Scheme("https", 443, new SSLSocketFactory(
   keystore, "mypassword", truststore)));

And then you need to create a class that implements HttpClientConfigurer, and registers https protocol providing a keystore or truststore per example above. Then, from your Camel Route designer class you can hook it up like so:

HttpComponent httpComponent = getContext().getComponent(
   "http4", HttpComponent.class);
httpComponent.setHttpClientConfigurer(new MyHttpClientConfigurer());

If you are doing this using the Spring DSL, you can specify your HttpClientConfigurer using the URI. For example:

<bean id="myHttpClientConfigurer"
   class="my.https.HttpClientConfigurer">
</bean>

<to uri="https4://myhostname.com:443/myURL?httpClientConfigurer=   \\
   myHttpClientConfigurer"/>

As long as you implement the HttpClientConfigurer and configure your keystore and truststore as described above, it will work fine.