JSON Handler details

The JSON Handler channel consumes a JSON response and loops over the data.

All responses that start with { at root level are called Objects. For example, this is a kind of expected JSON response (EXAMPLE 1):

# And endpoint which returns Rep information and Rep's contacts
# GET /reps/{rep_id}/details
{
"name": "Rep user name",
"email": "rep-user-name@gmail.com",
"contacts": [
{"full_name": "Vladimir Korichkov", "age": 32},
{"full_name": "Anton Petrov", "age": 46},
{"full_name": "Simeon Popov", "age": 25}
]
}

If they start with [ they are called Arrays. An example of a response with Array at root level (EXAMPLE 2):

# We might have an endpoint which returns list of Rep's contacts
# GET /reps/{rep_id}/contacts
[
{"full_name": "Vladimir Korichkov", "age": 32},
{"full_name": "Anton Petrov", "age": 46},
{"full_name": "Simeon Popov", "age": 25}
]

Both EXAMPLE 1 and 2 from above can be fetched/consumed by JSON Handler channel use it to iterate over the records (by record we mean this part of the json {"full_name": "Vladimir Korichkov", "age": 32}).

JSON Sources / Fetch JSON (Action)

There are two ways to start with fetching or consuming JSON payload, two different JSON sources, two different steps to start with - Incoming JSON and Fetch JSON.

Fields

Authentication Schema - supports Basic and Digest Authentication and Quick Base Usertoken, based on selected Auth Schema username and password or token fields are marked as required.

Notes: If you are going to use Quick Base API (https://api.quickbase.com/v1/) in combination with Quickbase Usertoken Authentication schema you will need to specify this header QB-Realm-Hostname and set your realm hostname.

Outgoing request's method type - select the type of the request from a list of supported ones

Headers - use this field to add extra headers to the fetching request

Request Body - some API endpoints are expecting to receive a specific request body like this one https://developer.quickbase.com/operation/runQuery

JSON URL - API endpoint or URL which points to a JSON file. The link should be publicly available or behind Authentication schema which the channels supports. File channels are also supported (pipelines:// protocol) and JSON files can be fetch using file_transfer_url.

Force Content Encoding - Force the encoding used to decode the JSON document. If left blank - Content-type header is taken into account, or if header is not presented utf-8 encoding is assumed. If specified - we will try to decode JSON payload using the selected one.

Limits

There is a payload size limitation (max 1MB) and because of that Incoming JSON and Fetch JSON have limited export fields. You can see that the raw JSON body is missing and the reason for that is that we are doing an optimization and we are compressing it so that we can have a larger payload (more than 1MB, technically it should be < 0.95MB and 0.05MB for the other exported fields like headers, query params, etc.).

Difference between JSON handler and Webhook Handler

If you want to use raw JSON body (without compression and consuming it with Jinja) you can use Webhook channel step Make Request. Both channels are similar but with some extra feature and limitations:

• Webhook/Make Request has support for OAuth 2.0 Client Credentials, JSON-Handler does not have it for now

• Webhook/Make Request can access the raw JSON payload using special field json, {{b.json['contacts'][0]['full_name']}}, JSON Handler is not exposing that without the usage of Iterate over JSON Records step

• Webhook/Make Request will fail if the payload is larger than 1MB raw JSON payload + the extra export fields

• Webhook/Make Rrequest does not have easy way to iterate over the records stored under {{b.json['contacts']}} field.

Incoming JSON (Trigger)

Triggers when you make an HTTP request (with a JSON payload) to an auto-generated webhook endpoint.

Authentication Schema - supports JWT Token auth and Basic Auth and we can restrict (set username and password option fields) who can hit the autogenerated endpoint.

Incoming request's method type - we can ignore incoming request different from what we are expecting - POST, PUT, PATCH or ANY BELOW (this filtration won't trigger the pipeline)
Notes: if we choose ANY BELOW, we could Branch the logic based on the export field Method and for example do create logic for method POST and update logic for method PUT.

JWT auth

The Incoming JSON step supports authentication through JWT token.

When the JWT Token option is selected, the user needs to provide a public key and also select a signing algorithm.

The pipeline gets triggered upon successful verification of the token. The output of the step includes the JWT payload in the form of an object that can be used in subsequent steps.

If verification fails, the pipeline does NOT get triggered, instead we return response with status code 401 (Unauthorized).

Iterate over JSON Records

Iterate over JSON records. Use this step after fetch or receiving a JSON file from one of these step(s) in JSON Sources.

JSON Schema Sample (optional) - used to generate user friendly fields used in subsequent steps during design time. If not specified, by default the record structure will be stored in field raw_record.

JSON Records Path (optional) - used for nested arrays within JSON object on root level (check above).

Loop over records (array of JSON objects, see EXAMPLE 1 and 2). There are some options which are helpful for JSON record processing.

Let’s see EXAMPLE 1

(A) JSON payload (Incoming or Fetch JSON step) → (B) Iterate over JSON Records loop options:

1. JSON Source target can point to a previous step, but limited to Incoming or Fetch JSON step

2. JSON Records Path field which is used to point to the field where the records are located (in our case, EXAMPLE 1, we have an object and the records are store under field contacts), /contacts.

loop over /contacts:
record: {"full_name": "Vladimir Korichkov", "age": 32}, etc.

If the root object is an array, EXAMPLE 2, we don’t need to specify any JSON Records Path.

loop over [ ... ]:
record: {"full_name": "Vladimir Korichkov", "age": 32}, etc.

Note: if the root payload is an object, like EXAMPLE 1, and we don’t specify JSON Records Paththen the whole object will be counted and interpreted as a single record.

loop over [{"name": "Rep user name", ... etc.}]
record: {"name": "Rep user name", ... etc.}

JSON Schema sample optional field - used for generating user friendly fields (based on the JSON structure) for easier usages in next steps in the designer

1. If the option is not specified, then the records will be stored in {{b.raw_record}} and can be accessible only by writing manually a Jinja expression with prefix raw_record

(B) loop over /contacts:
b.raw_record will contain {"full_name": "Vladimir Korichkov", "age": 32}, etc.

# If we want to get for example the full_name, age, etc.
# in the next steps within the loop then we need to use this Jinja expression
{{b.raw_record['full_name']}}, {{b.raw_record['age']}}, etc.

(B) loop over [{"name": "Rep user name", ... etc.}]
b.raw_record will contain {"name": "Rep user name", ... etc.}

# If we want to get for example the full_name, age, etc.
# in the next steps within the loop then we need to use this Jinja expression
{{b.raw_record['contacts'][0]['full_name']}},
{{b.raw_record['contacts'][0]['age']}}, 
{{b.raw_record['name']}},
etc.

If the option is configured, then autogenerated fields will be available for the next steps in the right designer window. You can use a sample with fields you are interested in or all of them:

# If we put in JSON Schema sample this sample JSON:
{"full_name": "Vladimir Korichkov", "age": 32}

# then based on that we will parse and generate the fields 
# (with the correct field type) and use them by 
# drag-and-dropping functionality.
# If we are interesting only in one field then we just need to uset this
# sample JSON:
{"full_name": "Vladimir Korichkov"}

EXAMPLE 2 should be a standard and easy one (it is an array of objects/records)

The only thing we need to configure, but it is not important is to specified the JSON Schema Sample. We don’t need to use and specified something in JSON Records Path because at root level we have an array.

EXAMPLE 3 (NOT SUPPORTED)

{"name": "Gilbert", "wins": [["straight", "7♣"], ["one pair", "10♥"]]}
{"name": "Alexa", "wins": [["two pair", "4♠"], ["two pair", "9♠"]]}
{"name": "May", "wins": []}
{"name": "Deloise", "wins": [["three of a kind", "5♣"]]}