The notion of a data fabric encompasses several fundamental aspects of contemporary data management. It provides a singular means of data modeling, a centralized method of enforcing data governance, and an inimitable propensity for facilitating data discovery.
Its overarching significance to the enterprise, however, supersedes these individual facets of managing data. At its core, the data fabric tenet effectively signifies a transition in the way data is accessed and, to a lesser extent, even deployed.
According to Denodo Chief Marketing Officer Ravi Shankar, in a big data driven world with increasing infrastructure costs and regulatory repercussions, it has become all but necessary: âto be able to connect to the data instead of collect the data.â
Incorporating various facets of data lakes, virtualization technologies, data preparation, and data ingestion tools, a unified data fabric enables organizations to access data from any variety of locations including on-premise, in the cloud, from remote devices, or from traditional ones. Moreover, it does so in a manner which reinforces traditional centralization benefits such as consistency, uniformity, and oversight vital to regulatory compliance and data governance.
âThe world is going connected,â Shankar noted. âThereâs connected cars, connected devices. All of these are generating a lot of different data that is all being used to analyze information.â
Analyzing that data in times commensurate with consumer expectationsâand with the self-service reporting tools that business users have increasingly become accustomed toâis possible today with a data fabric. According to Cambridge Semantics Chief Technology Officer Sean Martin, when properly implemented a data fabric facilitates âexposing all of the processing and information assets of the business to some sort of portal that has a way of exchanging data between the different sourcesâ, which effectively de-silos the enterprise in the process.
Heterogeneous Data, Single Repository
The quintessential driver for the emergence of the enterprise data fabric concept is the ubiquity of big data and its multiple manifestations. The amounts and diversity of the types of data ingested test the limits of traditional data warehousing methods, which were not explicitly designed to account for the challenges of big data. Instead, organizations began turning to the cloud more and more frequently, while options such as Hadoop (renowned for its cheap storage) became increasingly viable. Consequently, âcompanies have moved away from a single consuming model in the sense that it used to be standardized for [platforms such as] BusinessObjects,â Shankar explained. âNow with the oncoming of Tableau and QlikView, there are multiple different reporting solutions through the use of the cloud. IT now wants to provide an independence to use any reporting tool.â The freedom to select the tool of choice for data analysis largely hinges on the modeling benefits of a data fabric, which helps to âconnect to all the different sources,â Shankar stated. âIt could be data warehousing, which many people have. It could be a big data system, cloud systems, and also other on-premises systems. The data fabric stitches all of these things together into a virtual repository and makes it available to the consumers.â
From a data modeling perspective, a data fabric helps to reconcile the individual semantics involved with proprietary tools accessed through the cloud. Virtually all platforms accessed through the cloud (and many on-premise ones) have respective semantics and taxonomies which can quickly lead to vendor lock-in. âQlikView, Tableau, BusinessObjects, Cognos, all of these have semantic models that cater to their applications,â Shankar said. âNow, if you want to report with all these different forms you have to create different semantic models.â The alternative is to use the virtualization capabilities of a data fabric for effectively âunifying the semantic models within the data fabric,â Shankar said.
One of the principal advantages of this approach is to do so with semantics tailored for an organizationâs own business needs, as opposed to those of a particular application. What Shankar referred to as the âhigh level logical data modelâ of a data fabric provides a single place for definitions, terms, and mapping which is applicable across the enterpriseâs data. Subsequently, the individual semantic models of application tools are used in conjunction with that overlying logical business model, which provides the basis for the interchange of tools, data types, and data sources. âWhen the dataâs in a data store itâs usually in a pretty obscure form,â Martin commented. âIf you want to use it universally to make it available across your enterprise you need to convert that into a form that makes it meaningful. Typically the way we do that is by mapping it to an ontology.â
The defining characteristic of a data fabric is the aforementioned virtual repository for all data sources, which is one of the ways in which it builds upon the data lake concept. In addition to the uniform modeling it enables, it also supplies a singular place in which to store the necessary metadata for all sources and data types. That metadata, in turn, is one of the main ways users can create intelligent action for data discovery or search. âSince this single virtual repository actually stores all of this metadata information, the data fabric has evolved to support other functions like data discovery and search because this is one place where you can see all the enterprise data,â Shankar observed. Another benefit is the enhanced governance and security facilitated by this centralized approach in which the metadata about the data and the action created from the data is stored.
âThe data fabric stores what we call metadata information,â Shankar said. âIt stores information about the data, where to go find it, what type of data, what type of association and so on. It contains a bridge of the data.â This information is invaluable for determining data lineage, which becomes pivotal for effecting regulatory compliance. It can also function as a means of implementing role-based access to data âat the data fabric layer,â Shankar commented. âSince you check the source systems directly, if it comes through the data fabric it will make sure it only gives you the data you have access to.â Mapping the data to requisite business glossaries helps to buttress the enterprise-wide definitions and usage of terminology which are hallmarks of effective governance.
The data fabric tenet is also a critical means of implementing data preparation quickly and relatively painlesslyâparticularly when compared to conventional ETL methods. According to Shankar: âConnecting to the data is much easier than collecting, since collecting requires moving the data, replicating it, and transforming it, all of which takes time.â Interestingly enough, those temporal benefits also translate into advantages for resources. Shankar estimated that for every four IT personnel required to enact ETL, only one person is needed to connect data with virtualization technologies. These temporal and resource advantages naturally translate to a greater sense of agility, which is critical for swiftly incorporating new data sources and satisfying customers in the age of real-time. In this regard, the business value of a data fabric directly relates to the abstraction capabilities of its virtualization technologies. According to Martin, âYou start to build those abstractions that give you an agility with your data and your processing. Right now, how easy is it for you to move all your things from Amazon [Web Services] to Google? Itâs a big effort. How about if the enterprise data fabric was pretty much doing that for you? Thatâs a value to you; you donât get locked in with any particular infrastructure provider, so you get better pricing.â