Background: In this paper, I propose the use of various deep learning and artificial intelligence techniques to evaluate the detection and possible prediction of an individual falling taken from images or video. Falls are a serious issue within the medical sector [1]. While falls are often associated with senior centers and retirement homes, falls are responsible for approximately 37.3 million injuries severe enough for medical attention [2]. The goal of this thesis is to utilize human pose estimation and its output into another network or networks in an attempt to detect and predict falls that may occur. In addition, there is a number of additional possibilities such as utilizing angles between the joints or attempting to predict the next sequence of key points as an indicator of chance to fall which may be useful in an effort to catch individuals before they fall and thus ameliorate the risk to health that falls cause. Objectives: The primary purpose of this thesis is utilizing various techniques and deep learning models to determine if falls can be predicted or determined before they occur. To develop such a process, a literature review will be undertaken to understand the best possible methodology to achieve this objective. In addition, the creation of a new dataset with joint key points will be created and also tested solely for the purpose of determining the efficacy of detecting falls purely from key points. Methods: This thesis utilizes a fall detection dataset whereby 70 videos of subjects falling and not falling were recorded. These videos have been hand labeled on a pure frame basis as to whether the subject was falling or not falling. By applying a general-purpose human pose estimation system to these videos, we can gather key points of joint locations. From these various calculations, models can be made to achieve the objectives of this thesis to detect and predict falls. Results: At the end of this thesis, it was determined that human pose estimation has gathered enough general capacity to accurately create key points for videos and images. In order to evaluate performance, F1 score is utilized. F1 is the currently accepted best single measure of performance of a classification model with a score of 1 being perfect. From these aforementioned key points, falls could be accurately detected with a high F1 score of .88. In addition, falls could be predicted one frame ahead of time at a decent performance, consisting of an F1 score of .80. Conclusions: From the results gathered, it was concluded that falls can be detected from human pose estimation and that human pose estimation may provide additional benefit from simply evaluating the image as a whole. In addition, it was concluded that falls could be predicted ahead of time, however, more exploration is needed in this area in order to determine how far in the future this prediction could be made and how accurate the prediction could become.

Recommended Citation: Kurpiewski E., Dogan G., Ricanek K., Modaresnezhad M., Song Y., (2022). Pose Estimation for Future Prediction of Falling. UNCW MS CSIS Proceedings. V. 16 , N. 0 .