Web Platform for Microscopic Data & Knowledge Management

Web-based scalable platform for generation of hierarchically relational unique identification for gathering relational data and multimedia content in real time

Imagine a modern telescope for viewing distant objects. Yes, you can see that distant planet. You want to know if there is life on it. Could that blinking dot be an alien? You wish you had a microscope to zoom in. And I give you one. You are now face to face with the alien.

You don't want to stop there. You have structured data collection sheets in which you have been collecting data about aliens everywhere. There are columns for recording standardised and structured data. Done. You have space for writing descriptions. Also done. Why not take photos, record an audio of the alien's mutterings and capture a video of the alien and the habitat. You are glad you could do it all.

You see more dots moving in the vicinity. You focus on them. Hey! It's his family. You have more data-sheets to fill giving everyone a unique identification that also says that they belong to the alien's family.

You feel excited. You want to share this information with the other aliens you had met recently. Thankfully, your database has unique coordinates for everyone. You simply send a micro service request with the unique coordinates of this alien to other selected aliens. All done in a few seconds.

How easy was that? An unique identification system mapped on to a menu tree. You can hover across any data node, how ever distant, along the dynamic and auto-contextualising menu scheme. Wow, there is a search tool too. It is so different. Its result sets have no duplicates. And every result has an embedded mapping of its hierarchical lineage. It is so easy pick the the one that you are looking for by going over the mapping. It is fantastic. You can search for any alien using their Unique ID. It brings up the details of the exact alien. By clicking on it, you are taken directly to the page containing all the data and multimedia files that you had collected about that alien. And you can hover over to or search and find any specific alien - no matter where they live.

You want to turn your data into a useful knowledge-base. You want to enable every alien you have contacted - to post data relating to all others in their habitats. You move your system to the web. You give the aliens secure access rights for posting data pertaining to authorised habitats. At the same time, you give them right to view all the data hosted on the platform and also interact with each other through an in-built social media platform.

Your alien friends have been posting so much data. You perceive a gold mine of knowledge, if only you could aggregate and analyse it from multiple perspectives. Options are already available for aggregation and analysis. Since the data is stored in a relational format there is no limit on the types of analysis one wishes to apply in future. How useful this innovative knowledge-base be?

It includes all. It is a new process with no precedent but involves new ways of applying existing open source relational database technologies along with independently developed new technology components. As a completely web based platform providing end to end customisable solutions, it uses several technologies. The prototype demonstrates how our own back-end database solutions and innovative and never-seen-before user interfaces work in tandem with a host of existing technologies to deliver unique solutions. For example, by combining existing relational and hierarchical database approaches, we have created a hierarchically relational schema that works inside RDBMS databases. 

Privacy design includes 1) unique identification of not just entities and records but every data piece at the row/column intersection for controlling access to individual data pieces; 2) data is distributed inside the database into multiple tables using internal algorithms, which serves as a barrier to unauthorised breach of privacy. Every data is entered only once and thereafter, internally referenced with the help of its UID. All structured data is stored and transported in binary format or Json objects to minimise device/network load. User interfaces use icons alongside textual menus. Web token based authentication and micro-service architecture are in-built.

The solution is built as a modularised web platform with 100% data-driven front-ends, which receive data from independent back-ends – designed to work in tandem both in automated and manual data exchange modes. There are four different back-ends. 1) Taxonomy/Classification Generator, 2) Data Collection Form Creator, 3) Data Distributor & Databases Manager 4) Front-End & Back-end Inter-linker. The current architecture permits deployment Micro-services for all interconnections. While public data interconnections would be only through micro-service, customised back-end data connections can be built as necessary. The front end can be a mobile app or a web browser. We have simplified integration with digital identification system by introducing two new data types for pulling hierarchically relational UIDs and filtering data based on multiple parameters. These new data types make it easy for external applications to pull data through APIs. We have tested the concept but would need to write more code to implement our 'Data As Service' (DAS) concept fully. The entire system has been designed to be cost-effective since our distributed database architecture enables us to achieve both speed and scalability by using commodity servers and the Internet based connectivity and thus creating a system with low set-up and operational costs.

MicroKM's platform has three different users – 1) individuals who submit and update their data, 2) Knowledge-base creators who build and manage knowledge-bases and knowledge networks and 3) micro service users. Innovative user interfaces like auto-contextualising dynamic menu trees with no limits on drop-downs, search tools producing no-duplicate results, real time structured data submission and social media posting tools. Knowledge-base/networks get a menu-based management interface. Micro-service clients would get APIs and custom-built service request tools. Its greatest convenience is its web based access with easy extensibility to mobile applications. It performs with speeds comparable to Google, Facebook or LinkedIn

For security and privacy reasons, the back-end databases work in a black-box mode. Interoperability is through micro services only except for customised gateways - to be created for bulk data exchange using the same security framework. External users have no direct access to the database since the internal algorithms controlling databases would never be exposed to the outside world. At the same the data ownership would belong to those who provide the data and they would get interfaces through which they can retrieve their data in desired formats such as XML, CSV, etc. under legally binding contracts.

There are two specific ways in which this platform keeps the data transactions between the client and server minimal. First, it eliminates sending redundant codes. It uses standardised, templated and data-driven user interfaces with reusable Javascript functions that get executed on the clients device. Mostly the page remains the same and only the data keeps changing. Secondly, it minimises the amount of content dispatched by transmitting only the precisely relevant data. Google for example, pumps out humongous search results containing irrelevant, duplicate and unclear items. MicroKM's search results will be smaller and precise with nothing duplicate, irrelevant or misleading. 

Our ultimate vision is to transform the web into a data-driven knowledge-base that would support knowledge production factories everywhere as opposed to the document laden and most disorganized information warehouse it has become. In the next five years, we aim to become the world's leading and most cost-effective platform for building and managing taxonomy based global knowledge-bases and networks in areas such as e-governance, international trade, healthcare, etc. In the next three years, the focus is on setting up 'data as service' operations for governments, international organisations, institutions and businesses alternative tools for managing large volumes of structured data.

I am not currently working with any organization. I have invested my time and resources in the technology solution that I have prototyped. We are looking for social sector investors to join hands with us in taking the technology into production because of the beneficial impact it would have on the society.  

We have demonstrated our skills in the prototype we have developed. Since it is a disruptive innovation, we would need to draw attention to the concept. The entire team is now actively engaged in popularising the concept by demonstrating the prototype. We are not interested in attracting funding for the sake of funding. We are scouting for investors who believe in the solution

Our solution is a platform for building projects. Each project would have a different revenue model. We are focused on creating a new category called data as a service to corporates, governments and NGO's. The long term potential is embedded in the solution. Our platform would be also ideal for international organisations which use taxonomies. We recognise that it would be difficult for them to look at our solution as an immediate alternative because of large funds already sunk on existing solutions. We are looking at green field projects for demonstrating the benefits of our solution. The truth is no one else has or can offer our solution - not now and not in the near future. That is both an advantage and disadvantage that we would need to cope with as of now.

The very purpose of our solution is to give people the real benefit of web technologies. Our solution can make a difference to the lives of people in rich and poor countries. But, people must know about the solution.  Sane voices are getting lost in the din of marketing. We believe by showcasing the prototype in events that promise visibility, our solution could draw the attention of people who still believe in substance. We believe that we have the solution to the data challenge of Mission Billion and look forward to this challenge giving us the break.

Swimming against the tide is what our problem is. There is too much of marketing. People hardly have the time to think about long term challenges. But we recognise that it has always been like this. We need to be just that much more committed to cross the humps. We are identifying opportunities such as the Mission billion challenge to demonstrate our solution. We are constantly scouting for opportunities.