A number of changes in the contemporary data landscape have affected the implementation of data governance. The normalization of big data has resulted in a situation in which such deployments are so common that theyâre merely considered a standard part of data management. The confluence of technologies largely predicated on big dataâcloud, mobile and socialâare gaining similar prominence, transforming the expectations of not only customers but business consumers of data.
Consequently, the demands for big data governance are greater than ever, as organizations attempt to implement policies to reflect their corporate values and sate customer needs in a world in which increased regulatory consequences and security breaches are not aberrations.
The most pressing developments for big data governance in 2016 include three dominant themes. Organizations need to enforce it outside the corporate firewalls via the cloud, democratize the level of data stewardship requisite for the burgeoning self-service movement, and provide metadata and semantic consistency to negate the impact of silos while promoting sharing of data across the enterprise.
These objectives are best achieved with a degree of foresight and stringency that provides a renewed emphasis on modeling in its myriad forms. According to TopQuadrant co-founder, executive VP and director of TopBraid Technologies Ralph Hodgson, âWhat you find is the meaning of data governance is shifting. I sometimes get criticized for saying this, but itâs shifting towards a sense of modeling the enterprise.â
In the Cloud
Perhaps the single most formidable challenge facing big data governance is accounting for the plethora of use cases involving the cloud, which appears tailored for the storage and availability demands of big data deployments. These factors, in conjunction with the analytics options available from third-party providers, make utilizing the cloud more attractive than ever. However, cloud architecture challenges data governance in a number of ways including:
- Semantic modeling: Each cloud application has its own semantic model. Without dedicated governance measures on the part of an organization, integrating those different models can hinder dataâs meaning and its reusability.
- Service provider models: Additionally, each cloud service provider has its own model which may or may not be congruent with enterprise models for data. Organizations have to account for these models as well as those at the application level.
- Metadata: Applications and cloud providers also have disparate metadata standards which need to be reconciled. According to Tamr Global Head of Strategy, Operations and Marketing Nidhi Aggarwal, âSeeing the metadata is important from a governance standpoint because you donât want the data available to anybody. You want the metadata about the data transparent.â Vendor lock-in in the form of proprietary metadata issued by providers and their applications can be a problem tooâespecially since such metadata can encompass an organizationâs so that it effectively belongs to the provider.
Rectifying these issues requires a substantial degree of planning prior to entering into service level agreements. Organizations should consider both current and future integration plans and their ramifications for semantics and metadata, which is part of the basic needs assessment that accompanies any competent governance program. Business input is vital to this process. Methods for addressing these cloud-based points of inconsistency include transformation and writing code, or adopting enterprise-wide semantic models via ontologies, taxonomies, and RDF graphs. The critical element is doing so in a way that involves the provider prior to establishing service.
The Democratization of Data Stewardship
The democratization of big data is responsible for an emergence of what Gartner refers to as âcitizen stewardshipâ in two capital ways. The popularity of data lakes and the availability of data preparation tools with cognitive computing capabilities are empowering end users to assert more control over their data. The result is a shifting from the centralized model of data stewardship (which typically encompassed stewards from both the business and IT, the former in accordance to domains) to a decentralized one in which virtually everyone actually using data plays a role in its stewardship.
Both preparation tools and data lakes herald this movement by giving end users the opportunity to perform data integration. Machine learning technologies inform the former and can identify which data is best integrated with others on an application or domain-wide basis. The celerity of this self-service access and integration to data necessitates that the onus of integrating data in accordance to governance policy falls on the end user. Preparation tools can augment that process by facilitating ETL and other forms of action with machine learning algorithms, which can maintain semantic consistency.
Data lakes equipped with semantic capabilities can facilitate a number of preparation functions from initial data discovery to integration while ensuring the sort of metadata and semantic consistency for proper data governance. Regardless, âif you put data in a data lake, there still has to be some metadata associated with it,â MapR Chief Marketing Officer Jack Norris explained. âYou need some sort of schema thatâs defined so you can accomplish self-service.â
Metadata and Semantic Consistency
No matter what type of architecture is employed (either cloud or on-premise), consistent metadata and semantics represent the foundation of secure governance once enterprise wide policies based on business objectives are formulated. As noted by Franz CEO Jans Aasman, âThatâs usually how people define data governance: all the processes that enable you to have more consistent dataâ. Perhaps the most thorough means of ensuring consistency in these two aspects of governance involves leveraging a data lake or single repository enriched with semantic technologies. The visual representation of data elements on an RDF graph is accessible for end user consumption, while semantic models based on ontological descriptions of data elements clarify their individual meanings. These models can be mapped to metadata to grant uniformity in this vital aspect of governance and provide semantic consistency on diverse sets of big data.
Alternatively, it is possible to achieve metadata consistency via processes instead of technologies. Doing so is more tenuous, yet perhaps preferable to organizations still utilizing a silo approach among different business domains. Sharing and integrating that data is possible through the means of an enterprise-wide governance council with business membership across those domains, which rigorously defines and monitors metadata attributes so that there is still a common semantic model across units. This approach might behoove less technologically savvy organizations, although the sustainment of such councils could become difficult. Still, this approach results in consistent metadata and semantic models on disparate sets of big data.
The emphasis on modeling that is reflected in all of these trends substantiates the viewpoint that effective big data governance requires strident modeling. Moreover, it is important to implement at a granular level so that data is able to be reused and maintain its meaning across different technologies, applications, business units, and personnel changes. The degree of prescience and planning required to successfully model the enterprise to ensure governance objectives are met will be at the forefront of governance concerns in 2016, whether organizations are seeking new data management solutions or refining established ones. In this respect, governance is actually the foundation upon which data management rests. According to Cambridge Semantics president Alok Prasad, âEven if you are the CEO, you will not go against your IT department in terms of security and governance. Even if you can get a huge ROI, if the governance and security are not there you will not adopt a solution.â
Originally Posted at: 2016 Trends in Big Data Governance: Modeling the Enterprise