Ambient Intelligence for Eldercare – the Nuts and Bolts: Sensor Data Acquisition, Processing and Activity Recognition Under Resource Constraints

I. Introduction

With environmental sensors and wearable sensors gathering data continuously (Figure 1), and with software algorithms working behind the scene, making sense (intelligence) of the data, continuously processing the data, the awareness of the context and situation of elderly can be achieved. In this manner the elderly can be assisted at points of need and thereby helped to live independently at home.

Figure 1.

Activities of elderly as ambient phenomena

Distributed sensors have long been used to sense the status of environmental parameters such as humidity, temperature and light intensity and aggregate / record these parameters from a distance. Sensors have also long been used for surveillance and military applications, for example in the identification and tracking of targets such as aircrafts and battle tanks. Environmental changes, or aircraft / objects in motion may be thought of as causal agents for phenomena changes in a so-called phenomena aware system. The activities and behavioral patterns of the elderly may also be regarded as phenomena in an ambient space which is aware of various phenomena unfolding in the space.

Section II discusses sensor data acquisition and the notion of micro-context as a paradigm for building systems that support diverse applications which build upon knowledge captured from ambient spaces. The notion of activities as phenomena is presented in section III. In section IV is presented one of the most important aspects of any sensor data acquisition system, namely, feature extraction. It is important to note that feature extraction and further operational processes that follow, such as classification and reasoning are essentially independent of sensing modality. We illustrate this with a variety of examples in this chapter, ranging from video camera based agitation detection to pressure sensor based sleeping posture detection (both in section IV), and other modalities such as ultrasound sensors and accelerometers in subsequent sections. Algorithmic techniques are based on micro-context and are essentially agnostic of sensing modality. With the help of an example, section V introduces one of the key challenges of ambient intelligence – the challenge of satisfying the needs of applications through information quality. Each of these sections discusses real-life application case studies either from deployment in hospitals or nursing homes or from advanced laboratory prototypes. In order to put these applications in perspective we also include in section VI, a discussion on resources in terms of costs of computation and communication and the inherent benefits of carrying out judicious sensor selection in the presence of resource constraints. The mathematical tool employed to carry out the sensor selection is a dynamic Bayesian network. A few remarks on related work especially pertaining to higher level artificial intelligence techniques are presented in Section VII. The chapter ends with a short summary and conclusions in section VIII.

Ii. Sensor Data Acquisition And Micro-Context

To build a platform upon which ambient intelligence applications can be built, some core technologies and capabilities are needed. First, a wireless sensor network comprising a variety of ambient and wearable sensors must be incorporated into the smart home for the elderly. Data acquisition, filtering, segmentation and classification must be carried out at a low level, in order to extract features from sensor produced data into meaningful micro-context information (see definition below) which is stored in appropriate data archives. This data then becomes the basis for activity recognition, behavior understanding and learning. Algorithms must be flexibly mixed and matched, and therefore need to access micro-context stored in a generic format. Our approach relies on multi-modal information fusion and activity primitive recognition at the low level, and context aware reasoning at the high level (Figure 2). Central to our approach is the notion of micro-context which is defined as follows:

Figure 2.

Micro-context as a tool for multi-modal recognition