Additional information about how to extract data with HTTP Client

You can configure the HTTP Client connector in order to return a specific part of the response body instead of the whole response payload using the Extract a sub-part of the response and Output key/value pairs parameters.

The following examples are based on this JSON file that contains records about user ages, addresses, cars, etc.:

[
  {
    "name": "Peter",
    "age": 40,
    "male": true,
    "addresses": {
       "city": "Nantes",
       "zip": "44000",
       "street": "bd prairie au duc"
    },
    "cars": [
       {"brand": "Ford", "model": "Transit", "km": 123456},
       {"brand": "Renault", "model": "Clio", "km": 87234}
    ]
  },
   {
    "name": "Emma",
    "age": 34,
    "male": true,
    "addresses": {
       "city": "Paris",
       "zip": "75000",
       "street": "bd Saint-Germain"
    },
    "cars": [
       {"brand": "Tesla", "model": "Model 3", "km": 63456},
       {"brand": "Ford", "model": "Mustang Mach-E", "km": 32543},
       {"brand": "Volkswagen", "model": "Golf 8", "km": 43876},
    ]
  }
]

'Extract a sub-part of the response'

This configuration processes the whole response payload and let you extract the sub-element you want to return. If the selector points to an array, it will generate one output record per item.

Value of 'Extract a sub-part of the response'	Result
<empty> or `.` or `.root` (the three values will return the same result)	Returns the entire response payload. In this example, the full records for Peter and Emma are returned.
`.root[1]` If the document is an array (it starts with [ and ends with ]), and you want to extract only one item from it, you can use `.root` to be able to define the index. Indeed `.[1]` and `[1]` are not valid, but `.root[1]` will extract the second element of the document (index starts at 0).	Returns the second response payload. In this example, the full record for Emma is returned.
`.root[1].cars` Allows you to select a sub-array and loop over.	Returns a part of the second response payload. In this example, the record containing the cars owned by Emma is returned.
`.root[1].cars{.km > 40000}` Allows you to filter returned elements.	Returns a part of the second response payload. In this example, the record containing the cars owned by Emma that have more than 40,000 kms is returned.

'Output key/Value pairs'

Once the main data is extracted with the Extract a sub-part of the response parameter, you can also extract some values to create a flat record.

To do so, enable the Output key/value pairs and add some Name / Value entries. Name will be the field's name of the output record, and Value can contain some DSSL selectors to retrieve values from the response of the HTTP query configured in the connector.

In this example, the Extract a sub-part of the response value is '.', so it will loop over the two records 'Peter' and 'Emma':

Name / Value	Result
`name` / `{.response.name}` Allows you to retrieve names `age` / `{.response.age}` Allows you to retrieve ages `address1` / `{.response.addresses.street}` Allows you to set the 'street' attribute of the 'address' in the 'address1' field `address2` / `{.response.addresses.zip} {.response.addresses.city}` Allows you to concatenate the 'zip' and 'city' attributes, from the 'addresses' sub-object, in the 'address2' field `first_car_brand` / `{.response.cars[0].brand}` Allows you to extract the 'brand' attribute of the first item of the 'car' array	Extract the data from the payload contained by the Name/Value entries and put them in columns. In this example, the records containing the name, age, adress and first car brand of Peter and Emma are extracted.

Name / Value

Result

name / {.response.name}
Allows you to retrieve names
age / {.response.age}
Allows you to retrieve ages
address1 / {.response.addresses.street}
Allows you to set the 'street' attribute of the 'address' in the 'address1' field
address2 / {.response.addresses.zip} {.response.addresses.city}
Allows you to concatenate the 'zip' and 'city' attributes, from the 'addresses' sub-object, in the 'address2' field
first_car_brand / {.response.cars[0].brand}
Allows you to extract the 'brand' attribute of the first item of the 'car' array

Extract the data from the payload contained by the Name/Value entries and put them in columns. In this example, the records containing the name, age, adress and first car brand of Peter and Emma are extracted.

Did this page help you?

If you find any issues with this page or its content – a typo, a missing step, or a technical error – let us know how we can improve!

Leave your feedback here