Archive > Volume 19 Issue 10 > 2023,19(10):629-634. DOI:https://doi.org/10.1007/s11801-023-3028-7 CSTR:[cstr] Prev Next
Memory-boosting RNN with dynamic graph for event-based action recognition
- Article
- Figures
- Metrics
- Preview PDF
- Reference
- Related
- Cited by
- Materials
Abstract:
Existing action recognition methods based on event cameras have not fully exploited the advantages of event cameras, such as compressing event streams into frames for subsequent calculation, which greatly sacrifices the time information of event streams. Meanwhile, the conventional PointCloud-based methods suffer from large computational complexity while processing event data, which make it difficult to handle long-term actions. To tackle the above problems, we propose a dynamic graph memory-boosting recurrent neural network (DG-MBRNN). The proposed DG-MBRNN splits the event stream into sequential graph data for preserving structural information, then uses the recurrent neural network (RNN) with boosting spatiotemporal memory to handle long-term sequences of actions. In addition, the proposed method introduces a dynamic reorganization mechanism for the graph based on the distances of features, which can effectively increase the ability to extract local features. In order to cope with the situation that the existing datasets have too simple actions and too limited categories, we propose a new event-based dataset containing 36 complex actions. This dataset will greatly promote the development of event-based action recognition research. Experimental results show the effectiveness of the proposed method in completing the event-based action recognition task.
Keywords:
Project Supported:
Clc Number:
