Business face huge competition and should be in a position to utilize information and create new knowledge to lead their business. innovations in technology and interconnectivity of things are generating enoromous data every day leading to creating and managing 'Modern Data Systems'.
Developing data systems overall is hard, its really hard. Enterprises spend a lot of time and effort in talking to business teams, IT, IT vendors in selecting the software components for their data infrastructure. However often is overlooked is the tools and components of choice that are valuable.
Across all industries, an ever-expanding number of companies have started to focus on building / scaling the IT capabilities / colloborations, eventhough many are not software companies in building Data-Driven business systems for a Digital Transformation. These companies will sensibly start with by using packaged software or platforms built around which will indeed be a great piece of software and will provide a great deal of needed functionality. However, at some point of time the need for the internal users, customers, external users and your IT teams will outmatch these pre-built solutions. simply integrating number of components or disperate systems together might not fullfill a data-driven organisation. Bluntly if the path is of being a data-driven company, the need have to be on the path to being a development-enable company, either by building internal capabilities or with vendors.
I think there are 6 fundamental capabilities that define a 'Modern Data System' :
Open Data Access : Data must be consumable / ingestible with an Industry standard protocols like 'REST', 'API' and not just propreitary connectors. A Modern Data System should be future proof so that organisations have the confidence that they can get the data whenever and however they choose.
Virtual Data Consolidation : data is scattered across multiple locations, systems infrastructures and in silos. It is practically impossible to bring all organisations data together to one location. A modern data system must virtually unite disprate data locations and formats by providing consistant management ,in creating, maintaining and navigating the datasets.
Data Indexing : Ability to handle Metadata is critical in gaining control of the data, without which we end up with we end up with managing blocks. A Modern Data System should have a Metadata handling and must go beyond simple ACLs but into Content, Faceting, Clasiffication and NLP. It should have the capability of programatically accept dynamic and custom attributes to tune the data smarter.
Data Security : A Modern Data System must provide authentication and authorization for individual data objects to prevent data leakages or loss. Data Security must handle complex changing permissions, roles and responsibilities and should be able to integrate with organizations existing directory and security services.
Data Lifecyle : This capability provides opportunity for significant cost savings, risk reduction and operational simplicity. A Modern Data System, should transperantly orchestrate and automate the lifecycle, compliance and governance of data across infrastructure, applications, formats.. etc. A Data Platform is a natural layer to control the underlying storage resources it uses and should be able to work independently and not locking to a specific hardware or platform.
Data value : Data Value is more than Analytics or Visualization, but it is more about its capability to match information of the user's needs. The greater the accuracy a Modern Data platform can match user needs the greater will be its capability of delivering value.
Much of the Data Systems are being build using Open Source Projects. Many of these Open Source Projects which power much of the data systems were originally created as infrastructure-software that provides a generalized functionality for multiple use cases. Focus here is to reap the benefits that detailed insights and processing capabilities of organizations. Building these effective systems will require a close collaboration with developers and the consumers of the infrastructure. As both creator and consumers are dependent on the each other the need here is to create the process within the organization that can effectively enable build and create feedbacks on the data-driven systems.
Development effort is often one of the most precious resources, and the time taken is often overlooked input in the TCO while implementing new project. Providing the right tools and establishing a processes that maximize the potential of collaboration between developers and users while minimizing the amount of re-work is imperative. Business tend to understand this as an in-built quality of a Software companies having development team, however the best practices of the Software has moved beyond traditional software companies. They often manifest themselves as 'Consulting Services', ' Managed Services' , 'PaaS' or other externally provided capabilities. Whether to hold/build these capabilities in-house, or not will be based on organizations specific goals, constraints and the Digital Transformation Roadmap that is built, but the most important thing is that these concerns have to be addressed while building / remodeling an organization to be data-driven.
Comentários