1 min read

Lease DataFilter Spec

Data Harbor allows for flexibility in client's ability to filter data moving through the platform. The following doc provides examples and guidance to achieve that. 

filters:
    - target: ''
      filterType: 
    - target: ''
      filterType:

At the core, a Filter Spec contains a list of targets and their respective filter goals. Points of note:

  1. The Filter Spec adheres to strict YAML syntax.
  2. Targets are required to appear in the payload. If payload is missing the target, DataHarbor will refuse to process the entity. Note: Targets must be contained within single quotes (').
  3. Passthrough is considered to be implied. If a target appears in the payload API data but is not represented in the filter spec, it is passed in it's current state.

DataFilters allow for a combination of the following four scenarios.

Direct Access

Consider the following JSON

{
  "name": "All of our addresses",
  "description": "Sue's family and friends",
}

The following Filter Spec Syntax sample would allow filtering at the top level:

filters:
    - target: 'name'
      filterType: REDACT

Arrays

Arrays come in two forms : Implicit and "named"/Object based. DataHarbor can work with both scenarios.

Named Array Access

Consider the following JSON

{
  "name": "All of our addresses",
  "description": "Sue's family and friends",
  "metadata": {
    "tag": "Sue Henderson Family",
    "lastUpdateYear": "2023",
    "viewCount": 15
  },
  "addresses": [
    {
      "name": "Home Address",
      "addressLine1": "2100 Anywhere Road",
      "addressLine2": "Unit 402",
      "city": "Mountain View",
      "state": "California",
      "postalCode": "94043",
      "country": "United States"
    },
    {
      "name": "Bobs House",
      "addressLine1": "3100 NW Stream Rd",
      "addressLine2": null,
      "city": "Seattle",
      "state": "Washington",
      "postalCode": "93103",
      "country": "United States"
    }
  ]
}

This array is "named" meaning that it belongs to an object. DataHarbor can accommodate this with the following Filter Spec Syntax sample. 

filters:
    - target: '[addresses]/name'
      filterType: REDACT

Implicit Array Access

Consider the following JSON

[
    {
      "name": "Home Address",
      "addressLine1": "2100 Anywhere Road",
      "addressLine2": "Unit 402",
      "city": "Mountain View",
      "state": "California",
      "postalCode": "94043",
      "country": "United States"
    },
    {
      "name": "Bobs House",
      "addressLine1": "3100 NW Stream Rd",
      "addressLine2": null,
      "city": "Seattle",
      "state": "Washington",
      "postalCode": "93103",
      "country": "United States"
    }
]

DataHarbor can process this array with the following Filter Spec Syntax sample.

filters:
    - target: '[]/name'
      filterType: REDACT

Object Access

Consider the following JSON

{
  "name": "All of our addresses",
  "description": "Sue's family and friends",
  "metadata": {
    "tag": "Sue Henderson Family",
    "lastUpdateYear": "2023",
    "viewCount": 15
  }
}

The above JSON payload contains a sub-object "metadata." The following Filter Spec Syntax sample would accommodate this

filters:
    - target: 'metadata/lastUpdateYear'
      filterType: REDACT
Haven Admin

Haven Admin