Introduction
Video Overview
This video introduces data governance and outlines the problems that led to its need, the various areas within it, and future trends.
Series Context
It is the introductory video in a series that explains different data governance areas in detail.
Challenges Driving Data Governance
Data Chaos Before Governance
Organizations faced no single source of truth, absent data owners, undefined data purpose, and missing documentation, leading to conflicting data versions and poor decisions.
Four Critical Challenges
- Multiple versions of truth: same data elements differed across systems due to sync issues.
- No data owners: despite application owners, no one owned data quality or issue resolution.
- Undefined data purpose: without documented importance, critical decisions were delayed.
- Missing standardized documentation: reliance on individuals instead of enterprise-wide understanding.
This is called multiple versions of truth.
Defining Data Governance
In simple terms, data governance is a process of establishing accountability and authority in all aspects in and around data.
Rapid Adoption
Data governance has seen rapid adoption by large organizations over the past decade as they understood the high cost of maintaining bad data.
Overview of Governance Areas
Interconnected Governance Areas
The various areas within data governance are often interconnected and overlapping, which reflects how they are structured in practice.
MDM Placement for Learning
Master Data Management (MDM) is commonly placed under data governance, but here it is categorized under data warehousing concepts to facilitate an effective learning experience.
Data Stewardship
Data stewards are basically data owners.
Data Stewardship: Accountability for Data
Data stewardship is the accountability that ensures data is managed properly. It assigns ownership to data stewards who oversee data assets.
From Silos to Holistic Ownership
Traditionally, ownership was siloed at the application, business, or product level. But data is an overarching entity, requiring a dedicated ownership approach.
The Need for a Data Stewardship Program
Establishing data ownership leads to the data stewardship program, which is essential for effective data management across the organization.
Data Policy
Data Policy Formulation
Data policy provides guidelines for managing data and is typically created by a data governance council with representation from all lines of business.
Data policy is usually formulated by a data governance council, which consists of data-related executives and key personnel in an organization with adequate representation from all lines of businesses.
Data Standards & Metadata Management
Data Standards
Data standards ensure a consistent way of capturing, recording, and maintaining data across an enterprise. Example standards will be discussed in later videos on data lineage and metadata management.
Metadata Management
Metadata management deals with capturing, recording, and maintaining metadata to make data locatable. This topic is covered in a separate video.
Data Lineage
Data lineage is the process of tracking data movement back to its origin while recording all transformations.
Data Lineage & Data Cataloging
Data Lineage
Data lineage tracks data movement and transformations back to its origin. A dedicated video with illustrations explains how it is captured and ways to build a lineage system.
Data Cataloging
Data cataloging involves business glossary creation and links technical metadata with business metadata, enabling better data understanding.
data cataloging involves business glossary creation and most importantly, it links technical metadata and business metadata.
Data Quality & Data Security
Data quality is basically a measure of data's usability and is very important to ensure data can serve its purpose.
Data Quality
Data quality measures how usable data is, ensuring it can fulfill its intended purpose. This topic will be explored in depth in a separate video.
Data Security
Data security handles access management and the detection of personally identifiable information (PII), which is critical for regulatory compliance like GDPR.
Future Trends in Data Governance
Vendors Embrace AI for Data Governance
Collibra, Informatica, ASG, IBM and others are actively advancing their data governance products with artificial intelligence to tackle challenges like data cataloging and quality.
Three Key AI Applications Emerging
- Data cataloging using NLP (Natural Language Processing)
- Linking technical and business metadata with machine learning and NLP
- AI-driven data quality checks
If data is defined, ownership established, and there will be a single source of truth, return on investment on data-related initiatives can increase.