Blog > July 2021 > The “Where” of Lineage in 2021

The “Where” of Lineage in 2021

According to Gartner, it takes a community of analytics users at different levels between producer and consumer to deliver the most relevant and timely insights. As a result, “central versus decentral” organization models take a back seat to the possibility of viewing the organization from the sum of its collective intelligence.  Gartner predicts that by 2023, 30% of organizations will harness the collective intelligence of their analytics communities, outperforming competitors that rely solely on centralized analytics or self-service.*

As a leading metadata management vendor, we at ASG Technologies are hearing this same need from our clients. In fact, the #1 use case for data lineage this year is to create a marketplace of trusted, easily discovered and well-understood data to support their data and analytics (D&A) strategies.

Data leaders are “freeing the data”, as well as providing the needed transparency across enterprise BI domains for this collective intelligence. Traditionally, D&A requirements have been managed and centralized by IT departments, but this is no longer the case. Every organization has multiple BI departments, with a BI reporting environment associated with nearly every domain within an organization. Clients need to classify, protect, validate and provide transparency behind this data from an enterprise perspective, thus the Marketplace concept.

In 2021, data has come of age as the primary means to deliver agility for change and disruption. From a data management perspective, we hear from customers on a weekly basis about their need for capabilities to help them with Data Literacy, Data Scoring (see picture 1a), Active Insights and Monetization features that allow users to quickly digest, use and integrate data. In any integration or migration project, the ability to re-pipe (see picture 1b) the data quickly and accurately accelerates the business outcome.

Embedding lineage into the data fabric

Data Lineage is deployed behind each of these Marketplace features to accurately attest to the data insights. Lineage pipes provide tracing and tracking individual critical data elements and PI  data quickly.  Metadata and lineage permeate throughout the data fabric, integrating and joining metadata using the application inventory, transformation code and insights from the BI environments. Data integrations are more accurately executed and examined holistically across domains.

Data scores empower data quality and acceptance

The snapshot below  illustrates how ASG’s Intelligent Marketplace Insights offering enriches your enterprise data by scoring and validating it. When metadata is accessible throughout the organization, like water, you can start to action it based on your threshold of data quality and acceptance. Users can then “group think” around this classified data to further curate it and add to its usability. with explanations of the data, the purpose of the reports, quality ratings, privacy tags, policies and other useful context. With this approach, data can be continually measured and monitored  to drive data “value”.

Intelligence Marketplace Insights

Fig. 1a ASG Intelligent Marketplace Insights: Data Scoring

Application Lineage After Migration

Fig. 1b

From static reports to dynamic insights

Clients don’t want static reports and dashboards anymore, they want to make their own correlations and insights based on what data is available to them at their fingertips. Traditional business users are now performing a hybrid function of aligning business purpose to data to create and rapidly use the insights on demand. The push for data vendors is to construct composable business applications to support these bespoke use cases.

User adoption accelerates with online familiarity features that access the metadata fabric. When the features are intuitive, users begin using the metadata without realizing it.  In order to align the data to the business task at hand metadata must  be presented in a simplified, non-technical manner. The easier it becomes for users to consume the marketplace datasets and insights, the faster they can react to the insights they glean from the data. Embedding Data Lineage into the back end will further accelerate these insights.

According to Gartner*, by 2024, organizations that utilize active metadata to enrich and deliver a dynamic data fabric will reduce time to integrated data delivery by 50% and improve the productivity of data teams by 20%. Lineage can be just as pervasive as metadata and is absolutely an integral part of the metadata fabric. Increased visibility of the data lineage brings with it an increase in trust, leading to growth in data consumption and usage. The “where” of lineage is the “where” of metadata, throughout your data environment. To fast-track successful D&A strategies, consider Metadata Pipelines that cross business and technology borders for real impact to change.

*Gartner: Over 100 Data and Analytics Predictions Through 2025
Published 18 March 2021 - ID G00744238 - By Alan D. Duncan
Posted: 7/8/2021 8:30:00 AM by Susan Laine - Global DI Evangelist
Filed under :Data_Intelligence, DI, Lineage

Click here to request a demo of data lineage or the Intelligent Marketplace of Insights

July 21, 2021 | 11 AM ET

Inside Data Lineage: Results Driven Intelligent Lineage

Join the third session of this webinar panel series for a discussion with ASG data lineage partners, Accenture and Daugherty to gain new insights into how you can accelerate your data journey with automated, intelligent data li...

On-Demand