Skip to main content

Creating master data

Golden records combine the records of several sources into one trusted record per real-world entity — one customer, one supplier, one employee — no matter how many systems know it and how differently they spell it.

A golden record with its three source records

How it works​

A refresh runs three stages, every time on all source records of the rule (or only the changed ones):

  1. Matching decides which source records describe the same entity: equal keys such as the VAT ID, then similar names within the same country. → Matching
  2. Survivorship decides, property by property, which source value goes into the golden record: the most trusted source, the most recent value, or your own function. → Survivorship rules
  3. Writing creates, updates or retires the golden records under the Master Data node, keeps their UIDs stable and links each one to its source records. → Master data records

The building blocks​

NodeWhereWhat it defines
Golden record type (a NodeType)Model › NodeTypesThe properties of the golden record: Legal Name, VAT ID, Country …
Survivorship RuleModel › RulesThe sources, the matching settings, and one Property Rule per golden property
Master Data nodeMaster DataWhere the golden records live; points to the rule
Table nodesReference DataThe loaded source records — see Importing data

The demo has three complete setups to learn from: Customer (3 sources), Supplier (2) and Employee (2). See Demo on Free Edition.

Step by step​

  1. Load the sources — every source is a Table node with loaded records. → Importing data
  2. Define the golden record type — below.
  3. Create the survivorship rule, with its matching settings and property rules. → Matching, Survivorship rules
  4. Create the Master Data node and refresh it. → Master data records

Define the golden record type​

  1. Go to Model › NodeTypes, click Add Child and switch to the NodeType tab.

  2. Name it, e.g. Customer Golden Record, and pick an icon.

  3. Add the properties the golden record should have, for example:

    PropertyPropertyTypeGroup
    Legal NameStringGolden Record
    VAT IDStringGolden Record
    CountryEnum(Countries)Golden Record
    CityStringGolden Record
    EmailStringGolden Record
    PhoneStringGolden Record
    Survivorship RuleLink(Survivorship Rule)Survivorship
    Match ScoreFloatSurvivorship
  4. Click Create NodeType.

A few names have a meaning for the refresh:

  • Legal Name (or else Name) is the property compared by fuzzy name matching.
  • Country separates the fuzzy comparison: only records of the same country are compared.
  • Survivorship Rule and Match Score are filled by the refresh when the type has them.

Values that a survivorship rule picks are checked against the property's PropertyType — a Country value must resolve to a value of the Countries enum (see Enum, Link and Array); a value that does not is reported as a conflict and not stored.