In recent years, decentralization in distributed computing systems, such as Grids and Clouds has been widely explored in order to improve system performance in terms of scalability and reliability. However, the decentralized nature of the system also raises some serious challenges. This chapter discusses the major challenges of designing and implementing decentralization in Grid and Cloud systems. It also presents a survey of some existing decentralized distributed systems and technologies regarding how these systems have addressed the challenges.
TopIntroduction
A distributed computing system enables the sharing, selection, and aggregation of distributed heterogeneous computational and storage resources, which are under the control of different sites or domains. The key applications of the computational distributed systems is to provide solutions to the complex scientific or engineering problems, such as weather forecasting, stock portfolio management, medical diagnoses.
The configuration of a distributed system is considered as decentralized if none of the participants in the system are more important than the others, in case that one of the participants fails, then it is neither more nor less harmful to the system than caused by the failure of any other participant in the system. Thus, in a Decentralized distributed system, management services, such as application scheduling, resource discovery are distributed over the sites so that if one site is failed, another site can take over its responsibility autonomously. Moreover, decentralized systems are highly scalable as they can seamlessly add or remove the components or resource pool in order to accommodate varying workload. On the other hand, in a centralized distributed system, the central servers play the role of scheduling and resource discovery services. In Figure 1, we present example application runtime scenarios in case of both centralized and decentralized distributed system.
Figure 1. Application runtime environment in centralized and decentralized distributed systems: (a) centralized system, (b) decentralized system
Decentralization of distributed computing systems based on Peer-to-Peer (P2P) network model can certainly overcome the limitations of centralized and hierarchical model in terms of scalability, single point failure, autonomy, and trust-worthiness. However, complete decentralized nature of the system raises other serious challenges in domains of application scheduling, resource allocation, coordination, resource discovery, security, trust, and reputation management between participants.
In this chapter, we aim to identify the basic challenges of decentralized distributed systems and survey some existing decentralized distributed systems and technologies along with a case study. Specifically, we describe the basic functionalities and important features of these systems and technologies, as well as compare them in the context of addressing the challenges. Finally, we outline some opportunities or future directions in this research discipline.
TopChallenges Of Decentralized Distributed Systems
Scheduling
In centralized scheduling approach, all the system-wide decision makings are coordinated by a central controller. Centralized scheduler organization is simple to implement, easy to deploy, and presents few management hassles. However, this scheme raises serious concerns when subjected to larger system size.
The decentralized scheduler organization negates the limitations of centralized organization with respect to fault-tolerance, scalability, and autonomy (facilitating domain specific resource allocation policies). This approach scales well for both, a small scale resource sharing environment (e.g. resource sharing under same administrative domain) to a large scale environment (e.g. the Internet). However, this approach raises serious challenges in the domain of distributed information management, enforcing system wide coordination, security, resource consumer authenticity, and resource provider's policy heterogeneity. We can classify decentralized scheduling into two categories.