Article Preview
TopIntroduction
In the SOA architecture, software is packaged as separate Web services, and consumers can combine these services to provide new services to users. Selecting and composing different Web services to meet the increasingly diverse needs of users has become a prominent concern. In practice, many Web services have the same functionality, but their QoS attributes (such as response time, throughput, price, call success rate, and so on) vary differently. How to select optimal Web services to meet both functional requirements and non-functional QoS attributes is a key research problem (i.e., constraint-satisðed services composition, CSSC in short) in the service community (Hassine, Matsubara, & Ishida, 2006; Yu, Zhang, & Lin, 2007; Lecue & Mehandjiev, 2010; Sun & Zhao, 2012; Alrifai, Risse, & Nejdl, 2012; Zhao, Shen, Peng, & Zhao, 2015).
Due to the dynamic nature of Internet, the availability of Web services and their QoS attributes may change from time to time. On the other hand, the functional requirements and QoS constraints for SOA applications are becoming more and more complex. This introduces great difficulties in addressing the service selection and composition problem. Traditional solutions usually address the CSSC via two phases: service selection and service execution (Ardagna & Pernici, 2007; Kritikos & Plexousakis, 2009; Li & Lin, 2011; Klein, Ishikawa, & Honiden, 2011; He, Yan, Jin, & Yang, 2014; Llinás & Nagi, 2014). In the selection stage, services satisfying the functional requirements and QoS constraints are selected and composed. In the execution stage, the composed services are invoked to handle users’ requests. An assumption is made in such solutions: once a service is selected and composed in a service composition, this service and its QoS values should not be changed. However, this assumption does not always hold in practice due to the volatility of QoS attributes and the uncertainty of the host server behavior. Therefore, an optimal service selected for a service composition may not be optimal or even not available in the service execution stage. As a result, users need to re-select all the candidate services from the scratch, which undoubtedly consumes extra resources and may lead to poor user experience.
Recently, many research efforts have been devoted to addressing the CSSC problem in the dynamic environments, including random model (Chattopadhyay & Banerjee, 2018), combination of MDP model and HTN programming model (Xu, Chen, & Reiff-Marganiec, 2011), game theory (Wang et al., 2014), to name a few. However, these methods have some shortcomings such as high cost and slow speed. Some researchers proposed to use the Q-learning model to address the dynamic CSSC problem (Wang, Zhou, Zhou, Liu, & Li, 2010; Ren, Wang, & Xu, 2017). The major limitation of this solution lies in the state explosion problem for the Q-table. To reduce the large number of states, the authors classify all the continuous states into a small given number of discretized state levels. As a result, the QoS attributes are defined in a coarse-grained way.
To address this issue, we propose in this paper a DQN-based model to handle the state explosion problem for dynamic CSSC. In our model, a neural network is designed and trained to output the Q values from the current state and the candidate services. This network is used to replace the Q-table in our model, and therefore our model can be used to handle the CSSC problem with fine-grained and continuous QoS attributes. In particular, we train an agent for a service composition using a DQN-based reinforcement learning approach. During the service execution stage, the agent is able to pick up suitable services and execute them on the fly from the available candidate services based on the current state and the QoS constraints. In this way, our solution is able to handle the dynamic service composition scenarios in real industrial production where the QoS values may fluctuate or the candidate services may be added or removed.