WEAR
Marius Bock, Hilde Kuehne, Kristof Van Laerhoven, Michael Moeller

Description

Research has shown the complementarity of camera- and inertial-based data for modeling human activities, yet datasets with both egocentric video and inertial-based sensor data remain scarce. In this paper, we introduce WEAR, an outdoor sports dataset for both vision- and inertial-based human activity recognition (HAR). Data from 22 participants performing a total of 18 different workout activities was collected with synchronized inertial (acceleration) and camera (egocentric video) data recorded at 11 different outside locations. WEAR provides a challenging prediction scenario in changing outdoor environments using a sensor placement, in line with recent trends in real-world applications. Benchmark results show that through our sensor placement, each modality interestingly offers complementary strengths and weaknesses in their prediction performance. Further, in light of the recent success of single-stage Temporal Action Localization (TAL) models, we demonstrate their versatility of not only being trained using visual data, but also using raw inertial data and being capable to fuse both modalities by means of simple concatenation. The dataset and code to reproduce experiments is publicly available via: mariusbock.github.io/wear/

Preview / Browse the dataset

An interactive, fast, but lower-resolution version of the dataset can be inpected here: [click here].

Download

The full dataset can be downloaded via the [WEAR datasetpage] and [here]. The download folder is divided into 3 subdirectories:

• annotations (> 1MB): JSON-files containing annotations per-subject using the THUMOS14-style
• (2) processed (15GB): precomputed I3D, inertial and combined per-subject features,
• (3) raw (130GB): Raw, per-subject video and inertial data.

Please follow instructions mentioned in the README.md file in the data creation subfolder.

Citation

WEAR: An Outdoor Sports for Wearable and Egocentric Activity Recognition. Bock, Marius and Kuehne, Hilde and Van Laerhoven, Kristof and Moeller, Michael. In Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. (IMWUT), vol.8(4), ACM Press, 2024.

Disclaimer

WEAR is offered under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. You are free to use, copy, and redistribute the material for non-commercial purposes provided you give appropriate credit, provide a link to the license, and indicate if changes were made. If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. You may not use the material for commercial purposes.