Open Content Delivery Network

1. Digital services are not reaching enough of the world's population in an affordable, deployable, sustainable and equitable way.

2. Open Content Delivery Networks are a new approach to infrastructure that creates a platform that combines storage, networking and computation resources to deliver the services that people actually need without being encumbered by the costs of demanding and unnecessary features.

3. OCDNs allow the basic digital services required for e-government and critical services such as health, education and disaster recovery to be delivered ubiquitously and at minimal cost.

The Internet was never developed to meet the challenge of universal service to infrastructure-challenged environments. In the commercial media market, high end Content Delivery Networks have been developed to meet the needs of large for-profit corporations. The technology we have developed, OCDNs makes it possible to similarly build an infrastructure that can affordably scale to provide universal service for necessary digital engagement.

End-to-end packet delivery, which is the fundamental mechanism for providing digital services in today's Internet, is a direct solution which is viewed as being easy to deploy because of its weak requirements. However it is not well adapted to extreme connectivity-challenged environments because data must flow from sender to receiver without interruption. 

Current approaches to delivering digital services to connectivity-challenged environments focus on creating continuous paths along which packets can flow. The most common approach is to use optical fiber to reach radio broadcast towers which connect to personal mobile devices. However this approach requires relatively dense deployment of radio towers and is often not capable of high speed data transfer. Another approach uses satellites or arial drones, but these approaches are both expensive and fragile. Many people throughout the world do not have connectivity adequate to basic digital inclusion.

It is possible to deliver critical and necessary digital services adequate to digital inclusion using approaches that are cheaper and more resilient than end-to-end packet delivery. However these approaches do not easily support some of the most profitable and high-growth consumer applications that drive the growth of broadband Internet connectivity in wealthy communities. These alternatives are not being pursued.

Many digital services can be delivered without requiring continuous end-to-end connectivity by leveraging storage and processing embedded in the network. Content Delivery Networks (CDNs) are an approach that is widely used to manage high value network services, but the current implementation requires expensive hardware and specialized operations. Our solution uses a different way of implementing CDN functionality to provide basic digital services cheaply and reliably. It uses inexpensive hardware and can be operated as easily as current networks. 

We augment routers with a new type of network appliance which makes storage and processing resources available in a standard form which can be easily used and shared. This infrastructure is used as a platform for implementing basic CDN services such as file and streaming media distribution. The use of local storage and processing enables these enhanced appliances to function in environments where end-to-end paths do not exist at any moment but must be constructed over time.

The key innovation which makes our solution possible is the definition of the service that controls this new type of network appliance. It is designed specifically to be easily deployed and safely shared in providing basic digital services, not for peak performance.

The target population that is most directly served by our solution consists of individuals who have the means to afford inexpensive consumer grade personal devices such as mobile phones and tablets but who either are in communities that are not served by broadband, are not adequately and reliably served or who cannot afford to pay the recurring cost of broadband connectivity. This includes communities in economically disadvantaged rural or urban areas with limited connectivity, those who face discrimination or other exceptional hardship or who are in parts of the world where broadband is not widely implemented.

The critical functions of modern society are moving online at a rapid rate. In the rich and developed world activities such as interacting with government, obtaining education, interacting with employers, being aware of current events and obtaining required information require a fundamental level of Internet connectivity. However the definition of broadband service is based on the idea that end-to-end packet delivery is the only way to interact with such applications. Historically telephone and text messaging services have acted as a more universally available alternative to the Internet, but these suffer from high personnel costs and/or lack of expressiveness and flexibility and are disappearing. The underserved are increasingly isolated or burdened with unreasonable costs.

Basic broadband services that can be implemented without end-to-end packet delivery are capable of being used to meet almost all of the critical needs of this population. They can provide highly flexible and media-rich services using techniques including data caching, replication, asynchronous routing and edge processing. The devices that make this possible are inexpensive and can be operated with minimal training. Our solution, if widely deployed, would provide a means to make necessary connectivity truly universal at a cost and convenience that is practical. It has the potential to reduce isolation and the high recurring costs of connectivity.

Low-income, remote, and refugee communities are often blocked from  accessing to digital services by a lack of community infrastructure and the recurring cost of connectivity adequate to support required applications. Our solution enables access to many critical services using commodity consumer devices. It reduces the recurring cost of backbone connectivity, enabling the use of affordable transmission technologies. It augments Internet connectivity, is incrementally deployable and can deliver many high-value services. Adaptation to this technology is almost entirely a burden on infrastructure operators and application developers, not requiring new skills, training or substantial change in work habits among end users.

We have deployed our solution in multiple continent-level infrastructures (eg. Internet2 Distributed Storage Infrastructure, the National Logistical Networking Testbed, the NSF-funded multi-petabyte Research and Education Data Depot Network) and also in demonstrations in application to resilient communication in wildfire scenarios (a project that was located at Michigan Technological University) and in Kenya (during Beck's Fulbright year).

One way to understand the state of development and readiness of our solution is to think of it as analogous to an "orphan drug" developed and ready to go into production, but not having penetrated its target market.

Our solution is based on a reconsideration of a fundamental aspect of the design of the Internet, namely the reliance on end-to-end packet delivery as the common means of accessing digital services. Similar ideas have been pursued in specialized forms, implemented as part of an "overlay" infrastructure. They have also been the basis of the development of Content Delivery Networks and Cloud infrastructure, but in those cases specialized to support easily monetized high traffic services. Efforts to apply this technique to a general model of networking have run into the barrier created by the universal adoption of the Internet. These approaches have not been affordable and easy to install and manage.

Our solution leverages new deign principles in the management of storage and processing that limits and weakens it to enable affordable and widespread deployment. This enables the creation of a number of different services that use these resources. The implementation of these user-oriented services need not be as widely distributed or as shared as the network resources, and can reside within existing endpoint systems (personal devices, servers, etc) as well as in "service management" dedicated nodes operated specifically for that purpose.

Our solution leverages the current Internet to provide resources, creating an overlay.  It requires adaptation by service developers, but is compatible with existing infrastructure. It can be used to solve many other challenges, generalizing it much as the Internet generalized the telephone system. There is huge potential for disruption and growth in other applications of our fundamental technology.

The core technology that powers our solution is a network appliance called a "depot" that acts as an intermediate node (like an Internet router) but also provides shared access to primitive storage and processing resources. What makes this appliance unique is that the service (called the Internet Backplane Protocol or IBP) that provides access to storage and processing is not expressed in terms of traditional operating system files and processes but is *weaker* in the sense that it provides less structure and fewer guarantees. This design approach, which mimics successful aspects of the design of both the Internet and the Unix operating system, enables it to be more widely and cheaply deployed for a greater variety of purposes and using a greater variety of devices.

IBP implements a "buffer management" service which can be used to build services which are equivalent to current storage, networking and processing and are compatible with current environments. It can also be used to implement current applications in new ways, including the Open Content Delivery Network which implements access to common Web functionality using asynchronous networking and edge processing. It can also be used to implement "hybrid functionality" that is difficult to implement using only end-to-end packet delivery.

IBP breaks Internet conventions in safe ways to implement resource scheduling and routing algorithms. These techniques are used in the implementation of Content Delivery Networks and Cloud infrastructure but are not accessible to end users. Our solution applies these techniques to achieving universal access.

Our solution is implemented as a stack of services, with depots implementing the Internet Backplane Protocol (IBP) at its foundation. Higher levels of the stack are progressively more specialized. The entire stack comprises the Data Logistics Toolkit, a mature open source project that is hosted on Github and is coordinated by Martin Swany's group at Indiana University.

The IBP depot has had many interoperable implementations. The current DLT depot implementation, developed and maintained by Alan Tacket of Vanderbilt University's Advanced Computing Center for Research and Education has been seen wide use, in production facilities, in pilot projects and in experimentation. Tackett has developed a stack based on its depot implementation that supports an enterprise storage solution called Lstore that is has been in production use for over 10 years. Lstore was used to implement the Research and Education Data Depot Network, an NSF-sponsored petascale national storage infrastructure which operated from 2012 to 2017.

The same IBP depot implementation is used by the other component of the DLT stack, developed and maintained by Swany's group centered on his Intelligent Data Management System. It has been applied in collaboration with Nancy French's group at the Michigan Technological University to develop a prototype Wildfire Data Logistics Network for resilient data distribution under NIST funding for wildfire fighters working in disconnected environments.

There is a 20 year history of other components of the stack that were used experimentally or in pilot deployments.

"Interoperable Convergence of Storage, Networking and Computation" Micah Beck, Terry Moore, Piotr Luszczek, Anthony Danalis Future of Information and Communication Conference, 14-15 March 2019, San Francisco. https://arxiv.org/abs/1706.07519

"Data Logistics: Toolkit and Applications" Micah Beck, Nancy French, Ezra Kissel, Terry Moore, Martin Swany GOODTECHS 2019 - 5th EAI International Conference on Smart Objects and Technologies for Social Good, 9/25-27/2019, Valencia. http://web.eecs.utk.edu/~mbeck/GoodTechs-DLT.pdf

Privacy and security are important concerns in any shared information and communication technology (ICT) infrastructure. The IBP protocol uses only cryptographically secure  keys to identify the buffers that it manages. The lack of user-specified name service prevents the depot from being used to implement rendezvous, as keys must be shared by other means.This provides a measure of isolation and security, as long a those keys are transferred securely between client and depot and managed securely by the client. The current implementation of the depot uses standard operating system security to protect the contents of buffers, treating the stored data much like a router treats packets in transit. As with Internet communication, the primary way that clients can ensure security is through end-to-end encryption. There is a risk of intrusion if network communication between client and depot is snooped, which can be addressed either by encrypting such communication (at a potential performance cost) or in application design, by keeping such network communication local whenever possible and using encryption over risky public links.

Every distributed infrastructure intersects with freedom of expression and the exercise of control for political or economic purposes. Communication networks have claimed immunity from responsibility for data by maintaining neutrality and claiming that applying policy fairly is impractical.Transient storage such as Web caches are similarly immune, but the increased generality of our solution raises the possibility that immunity will not be granted. This is an issue that invites a regulatory solution, which may interfere with openness.

As a technology solution that has not achieved "successful launch" as a persistent service, our solution has in the past been used by communities served by our prototype and pilot deployments, but many of these are not active. The population of users who currently benefit from use of IBP depots are the users of the Lstore enterprise storage network on the Vanderbilt campus, who number in the hundreds, but are not the underserved population that is the target of the Open Content Delivery Network. 

As an project mainly operating in academia, our currently pending funding proposals to the NSF and the DOE may fund converged platforms based on the Logistical Networking stack to support big data science collaborative analysis associated with Physics experiments and supercomputing simulations. We also have a pending proposal to the NSF to initiate a major proposal to implement Open Content Delivery Network to support rural schools in Tennessee, which if successful would serve hundreds in two years and thousands of students during the next five years. We are seeking seed funding to continue the work that Beck initiated during his Fulbright year in Kenya to deploy an Open Content Delivery Network in rural areas that are very poorly served by communication services, which could serve hundreds within two years and thousands within five years.

As an open source technology, our pilot projects could stimulate adoption that could reach millions (Internet scale), assuming broad acceptance. The potential for adaptation to address a variety of problems is huge.

Our models are technologies that have grown to dominate the information technology field, specifically the Unix operating system kernel and the Internet Protocol Suite. Both of these cases benefitted from early support by insitutions with deep pockets, (AT&T Bell Telephone Labs and the US Defense Advanced Research Projects Agency, respectively).

These foundational technologies attracted tool and application developers who built stacks that leveraged the generality of their design. These early developers served limited numbers of early adopters who took decades to convince established industries to adopt them, to the point that today they are almost universal.

The experience of these layered systems has been studied and mimiced in the design of the Data Logistics Toolkit, and it is positioned to be validated at a larger scale and potentially adopted widely in today's highly dynamic technological development environment.

Fundamental steps in building community acceptance of our solution is the appropriate involvement of standards bodies such as the IETF, open source consortia such as the Linux Foundation or influential industry leaders such as Google, AWS, Microsoft, VMware, Akamai or others who may see a benefit in expanding the potential market for digital services through the adoption of a new architectural model.

As academics, publication, pilot projects and community outreach are our primary tools for disseminating new ideas, tools and projects. The next step is establishing interoperable standards that can be widely adopted by stakeholders, some of which we can collaborate with for the public good, others we may partner with commercially.

The foundational nature of our solution makes the interest of and adoption by applied researchers and developers the first metric that indicates impact. These are notoriously difficult to measure, given the competitive environment for innovation in information technologies and the huge potential payoffs.

When we have operated pilot infrastructures we monitor deployment metrics such as the number of adopting institutions, the number of sites, and the scale of nodes at each site. We also seek to measure the size of end user populations by metrics such as the number of user accounts, the traffic on user support forums and trouble tickets, attendance at tutorial workshops, accesses to documentation and software and attention from the media and general public.

One way to understand the state of development and readiness of our solution is to think of it as analogous to an "orphan drug" developed and ready to go into production, but not having penetrated its target market. An analogy in technology might be the state of the Oak project before it became Java. Development and adaptation is required, but there are also individuals and projects who are prepared to adopt if non-technical factors including the plausibility of wide acceptance and long-term viability made it possible.

The question of how to measure the inherent design strength of our solution and its potential to "go viral" is very difficult. We could hold focus groups or engaging in other studies of potential, but we are not familiar with these strategic planning and marketing techniques.

Our primary team members are faculty and staff members at research universities: University of Tennessee, Indiana University and Michigan Technological University. Team members at UT and IU are tenured faculty members whose institutional salaries include support for research, which they have leveraged to develop our solution. Team members at Michigan Technological University and one at UT Institute of Agriculture are research faculty, whose involvement is funded by research contracts, with less institutional support. Students and postdoctoral researchers and faculty at other institutions participate when funding is available. Application researchers collaborate with our team to propose projects that use our solution.

Our solution is an open source project with no dedicated full time staff. Currently four faculty members who are the primary investigators. In the past there have been as many as 6 full time programmers and several part time graduate students.

Micah Beck has worked in distributed systems for 40 years. He led the Internet2 Distributed Storage Infrastructure and his research in Content Delivery was used as the basis for a startup company. He co-directed a research lab at the University of Tennessee where Logistical Networking originated. He received a Fulbright Scholarship to investigate application of Logistical Networking to rural Kenyan schools and has served as a National Science Foundation Program Director in the Office of Advanced Cyberinfrastructure. 

Martin Swany was an experienced cluster computing and network engineer, Web services manager and operated of an Internet Service Provider before completing his graduate studies in Network Measurement and Architecture and embarking on a successful academic career. He now Professor and Chair of Intelligent Systems Architecture at Indiana University in Bloomington. Research and development of the Data Logistics Toolkit software stack is currently led by his research lab.

Nancy French is Senior Research Scientist and Adjunct Professor at Michigan Technological University with research interests involving large geospatial datasets, remote sensing, and spatial data analysis. Her expertise  includes Wildland Fuels, Fire, and Carbon Emissions From Fire and Remote Sensing of Wildland Fire and its Effects. She led the Wildland Fire Data Logistics Network project.

Sreedhar Upendram is Assistant Professor of Agriculture and Resource Economics at University of Tennessee. He has launched an initiative that studies and brings broadband internet to underserved rural communities across Tennessee. He has received the 2021 Bonnie Teater Community Development Early Career Achievement Award from the Southern Rural Development Center.

Our leadership team is diverse in gender, ethnicity and national origin. We are faculty at public institutions that themselves serve diverse populations. We individually work with the programs focused community outreach, extension and equity without our institutions. We are committed to bringing into our project leadership individuals from the groups and communities we seek to serve as our deployment and service work expands.

We can benefit from even relatively small amounts of unrestricted funding that can be used to help in the development of grants, travel for the purpose of outreach and partnership-building and other activities not covered by research contracts.

We seek exchange of ideas and challenging interactions with intellectual and business innovators and leaders who believe in the possibility of fundamental change and are not primarily defending the status quo.

We seek mentorship, coaching, and strategic advice from experts who are motivated by both individual achievement and the common good. 

We need to reorient our activities toward impact on target communities rather than academic or institutional goals and gain exposure that builds confidence in our changes of large scale success.

We believe that our technology has the ability to enable the delivery of digital services in a way that is more scalable, affordable, sustainable and equitable than the current Internet model. One major impediment to the acceptance of our solution is that the current model is seen as unalterable - that we are at "the end of history" in wide area digital services or that only the largest corporations can have any impact. 

Our solution cannot be fully evaluated without being implemented and invested in. Being seen as a "plausible" solution is a social and communication challenge as much as technical one. Our hope is that Solve could help us to address these aspects of our project through contact with leaders, by attracting funding and through improved communications.

Indiana University and Michigan Technological University are currently partnering to implement a demonstration of our solution in collaboration with Federally funded wildfire control organizations in Colorado. This demonstration involves deployment of "data ferrying" solutions in simulated fire scenarios and integration of the communication technology with tools and systems currently in use by firefighters.

Our solution has been developed mainly within academic research, with some application to institutional IT within the university environment. As such our emphasis has been on design concepts, laboratory experimentation, prototype building and pilot deployments funded by research contracts. We have little experience or capability in business development or in the marketing of our solution, whether for profit or in a not-for-profit model. Our users are other software developers who have a high degree of technical competence, tolerance for risk, and who are mainly themselves part of the academic research or IT environments funded by government research contracts.

It is unusual for any solution to gain widespread acceptance in the absence of technical participants who are personally invested in it. We are open to partnerships of all kinds, with other academics who might have a business orientation, with individuals who seek to found start-up companies, with established companies looking for innovative ideas, with governmental or private entities who seek solutions to local or global problems, or others who can help us bridge the gap between what we have and continue to accomplish and what we see as its potential to create positive change. 

Our default path has been to concentrate on academic publication and dissemination of our ideas and leave the pursuit of social and economic impact to others who may consume our results. We seek means of taking a more proactive role in this process through assistance and partnerships.

AOur experience with promulgation of our ideas is that few are willing to comment on them publicly. In one-on-one conversations with research and industry leaders, the uniform reaction has been that the solution we are suggesting, regardless of its merits, is too radical to gain serious consideration. Individuals at the highest level advise us on how to pursue the ideas in less ambitious ways. Our hope is that MIT Solve will seek to meet those challenges in order to create change on the largest scale rather than avoid them.

Eric Schmidt is someone who has participated in epochal change in the area we are addressing - convergence of networking with storage and computation. Several people whose work intersects with our are affiliated with Alphabet companies, including Vint Cerf, Ken Thompson and David Patterson. Working with any of them or similar architects in a collaborative mode could be an enabling step.

Our solution is also directly related to the legacy of the Multics project at MIT, and intersects with the seminal work of Gerald Saltzer, David Clark and David Reed on the Internet, as well as that of Arvind and his students Greg Papadopolous and David Culler, as well as that of Dan Hillis. The input and evaluation of any of these leaders would be invaluable.

We have had some conversations with Cerf, Saltzer, Clark and Reed and Patterson.

Possible organizations to partner with would be the IETF, W3C, Linux and Mozilla Foundations.