Metadata architecture

Centralized, distributed, hybrid and bi-directional metadata architectures, each with an interactive diagram you can stress-test.

1 min read

Metadata has a lifecycle with layers: creation/sourcing → storage → integration → delivery → usage → control/management.

  1. 01
    Creation / sourcing
  2. 02
    Storage
  3. 03
    Integration
  4. 04
    Delivery
  5. 05
    Usage
  6. 06
    Control / management

The four architectures

ArchitectureHow it worksKey trait
CentralizedSingle repository holding copies of Metadata from sourcesHigh consistency; global search possible
DistributedSingle access point, no persistent repository; retrieves from sources in real time (via object request broker)Always current; no manual entries possible
HybridMetadata moves source→central repo, but repo only holds user-added/critical/standardized/manual itemsNear-real-time + enhanced; doesn't improve availability
Bi-DirectionalMetadata can change in any part (source, integration, UI); feedback coordinated back to sourceAdvanced; repo must manage source changes

Explore each architecture

Single repository holding copies of Metadata from sources

Key traitHigh consistency; global search possible

Databasesv1ETL pipelinesv1BI toolsv1Metadata repositorystores copies from sourcesDB v1✓ETL v1✓BI v1✓Manual entries: 0Users & appssearch, lineage, glossary
  • Metadata
  • Manual entries
  • Dashed links: batch replication
  • Click a source to target it
Try it on ETL pipelines
What users see
  • Databasesv1 · copy, up to date
  • ETL pipelinesv1 · copy, up to date
  • BI toolsv1 · copy, up to date
  • Manual entries0 saved
What happened
  1. Pick an action above to stress-test the centralized architecture.
Pros
  • High availability (independent of sources)
  • Quick retrieval
  • Resolved structures (unaffected by proprietary systems)
  • Can be transformed/enhanced
Cons
  • Complex replication of source changes
  • Costly maintenance
  • Custom extraction modules/middleware