Multi-modal Convolutional Neural Networks for Activity Recognition
- 1 October 2015
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3017-3022
- https://doi.org/10.1109/smc.2015.525
Abstract
Convolutional neural network (CNN), which comprises one or more convolutional and pooling layers followed by one or more fully-connected layers, has gained popularity due to its ability to learn fruitful representations from images or speeches, capturing local dependency and slight-distortion invariance. CNN has recently been applied to the problem of activity recognition, where 1D kernels are applied to capture local dependency over time in a series of observations measured at inertial sensors (3-axis accelerometers and gyroscopes). In this paper we present a multi-modal CNN where we use 2D kernels in both convolutional and pooling layers, to capture local dependency over time as well as spatial dependency over sensors. Experiments on benchmark datasets demonstrate the high performance of our multi-modal CNN, compared to several state of the art methods.Keywords
This publication has 10 references indexed in Scilit:
- 3D gesture classification with convolutional neural networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- Convolutional Neural Networks for Human Activity Recognition using Mobile SensorsPublished by European Alliance for Innovation n.o. ,2014
- mHealthDroid: A Novel Framework for Agile Development of Mobile Health ApplicationsLecture Notes in Computer Science, 2014
- Improving deep neural networks for LVCSR using rectified linear units and dropoutPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Activity recognition from acceleration data based on discrete consine transform and SVMPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2009
- Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor SelectionPublished by Springer Science and Business Media LLC ,2008
- Activity Recognition from User-Annotated Acceleration DataLecture Notes in Computer Science, 2004
- Multi-sensor context aware clothingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- A Survey of Computer Vision-Based Human Motion CaptureComputer Vision and Image Understanding, 2001
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998