If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
We have previously developed DIGITS, a platform for remote evaluation of range of motion, dexterity, and swelling of fingers for reducing barriers to accessing clinical resources. The current study was aimed at evaluating DIGITS across different devices with varied operating systems and camera resolutions using a single person’s hands.
Methods
Our team has now developed a web application version of the DIGITS platform, which makes it accessible on any device that is equipped with a camera, including computers, tablets, and smartphones. In the present study, we aimed to validate this web application by comparing flexion and extension measurements on the same person’s hands using three different devices with cameras of different resolutions. The absolute difference, SD, standard mean error, and intraclass correlation coefficient were calculated. Additionally, equivalency testing was performed using the confidence interval approach.
Results
Our findings indicated that the differences in degree measured between the devices ranged from 2° to 3° when digit extension was assessed (all hand landmarks are visible in the camera’s direct view) and from 3° to 8° when digit flexion was assessed (some of the hand landmarks are hidden from view). The intraclass correlation coefficient of individual trials ranged from 0.82 to 0.96 for extension and 0.77 to 0.87 for flexion across all devices. Additionally, within a 90% confidence interval, our data showed equivalency with measurements using three different devices.
Conclusions
The absolute differences were within an acceptable 9° tolerance for measurements taken between devices for flexion and extension. Equivalency was observed for measurements of finger range of motion taken using DIGITS, regardless of devices, platforms, or camera resolutions.
Clinical relevance
In summary, the DIGITS web application has good test-retest reliability to generate data on finger range of motion for hand telerehabilitation. DIGITS can reduce costs to patients, providers, and health care facilities for conducting postoperative follow-up assessments.
Patient engagement in hand therapy programs is invaluable in the restoration of hand function following trauma, surgery, or degenerative conditions. Clinical evaluation and therapy with trained hand therapists is the current standard of care in hand practice.
The goal of hand therapy is to rehabilitate patients’ injured hands such that they return to baseline function. The assessment of small-joint range of motion (ROM) of the hands is a critical component of evaluating patients’ progress during hand rehabilitation. However, access to hand therapy can be limited for patients because of either geographic or financial constraints.
The coronavirus disease 2019 pandemic prompted rapid adoption of virtual care, remote monitoring, and remote assessment of patients.
Wearable hand module and real-time tracking algorithms for measuring finger joint angles of different hand sizes with high accuracy using FBG strain sensor.
Advancements in software and artificial intelligence have prompted the evolution of novel assessments of joint ROM in the context of hand surgery.
In an attempt to reduce barriers to access and implementation costs of assessment tools, our team has developed “DIGITS,” a cross-platform remote assessment tool designed to track small-joint ROM of the hands (Fig. 1). Initially developed as a smartphone application, DIGITS is a useful tool to remotely assess ROM compared with the current gold standard of in-person finger ROM measurement using a goniometer. A comparison of joint ROM measurements between in-clinic goniometry with trained hand therapists and the DIGITS smartphone application yielded a difference within 9°,
Although effective for remote measurements of joint ROM, the smartphone application version of DIGITS limited the accessibility of our platform to Android smartphone users. To resolve this limitation, we developed the DIGITS web application, which can be accessed through internet browsers on any device with internet capability, including laptops, personal computers (PCs), and tablets. This study is proof of concept that aimed to assess the test-retest reliability of the newly developed DIGITS web application across different devices (laptops, PCs, and smartphones), operating systems, and camera resolutions in a healthy hand. The null hypothesis of this study that that the true difference between means is outside of the prespecified equivalency region, either below −9° or above +9°.
Materials and Methods
DIGITS web application development
Finger ROM data were collected using the DIGITS web application, a custom web application, using the Chrome bowser.
The DIGITS web application was written in JavaScript using the React JS framework and incorporated the ReactWebCam package.
The web application uses the open-sourced machine learning framework MediaPipe Hands pipeline from Google. The pipeline combines two convolutional network models. These networks were implemented and trained using Tensorflow.
MediaPipe Hands: On-Device Real-Time Hand Tracking. cvpr Workshop on Computer Vision for Augmented and Virtual Reality, Seattle, WA, arXiv:2006.10214 [cs].
Liu W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector. Paper presented at: 14th European Conference; October 11-14, 2016; Amsterdam, The Netherlands. https://doi.org/10.1007/978-3-319-46448-0_2
TensorFlow: a system for large-scale machine learning. Paper presented at: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16).
Using the device’s camera, the “MediaPipe Hands” framework detects the location of surface landmarks on the palmar surface of the hand and extrapolates x, y, and, relative depth coordinates, z. The hand pose estimation tracks 21 distinct points on the hand (ie, landmarks) and generates an output consisting of time-stamped spatial coordinates.
The hand landmark coordinates and time stamps were logged in real time into a comma-separated value text file on the device. The vectors between each hand landmark coordinate were used to calculate the angle between each finger and hand segment, yielding the corresponding joint angle. For example, the vectors generated by distal interphalangeal (DIP) crease landmarks to proximal interphalangeal (PIP) crease landmarks and PIP crease landmarks to metacarpophalangeal (MCP) crease landmarks yield the joint angle at the PIP joint. The calculations converting coordinates to vectors and then angles is performed using R Studio but can be programmed into the DIGITS software. Given the two adjacent segments and , the angle θ (in radians, converted to degrees in a later step) was simply calculated as follows:
The devices used for data collection were a Huawei P30Lite smartphone with a 24-mega pixel front-facing camera behind f/2.0 lens (maximum supported resolution, 2,312 × 1,080), an ASUS VivoBook laptop with Realtek USB 2.0 HD UVC WebCam (resolution, 1,280 × 720), and a custom-built desktop PC with Saimspunhgone USB 2.0 Webcam (resolution, 640 × 480).
ROM measurement
In this proof-of-concept study, the research participant’s hand was placed 15 inches away from the screen to allow full visibility by the camera. The assessment included the following fingers: index, middle, ring, and little. We measured the angle across the MCP, PIP, and DIP joints in a right-handed man. The test subject was instructed to either hold a full fist (ie, flexion) or keep all joints extended (ie, extension). The landmarks of each finger joint of the hand were identified and tracked by our system in real time, and the angles between adjacent segments were calculated to estimate the ROM end points across the different joints of the hand.
The dataset had a sampling frequency of 15 new full sets of recorded data per second, and the sampling time for each data set was 30 seconds. This resulted in an average of 450 data entries of whole-hand ROM measurements per data set. We controlled the face orientation toward the camera (ulnar, palmar, or rotating views between ulnar and palmar, as demonstrated in Fig. 2) at a previously determined optimal object distance of 15 inches and bright-light environment. Three identical repeats of data sets at each distinct experimental condition were collected. In total, 108 sets of data (three repeats of each condition, three different face orientations, three devices, left and right hands, extension, and flexion), each set containing approximately 450 entries of whole-hand joint angle (total 48,600 entries for each joint angle), were collected for analysis.
Figure 2Illustration of the different hand positions from which the data were collected: A extension and palmar facing as well as B extension and ulnar facing. C Flexion and palmar facing as well as D flexion and ulnar facing.
The landmark coordinates were recorded in the comma-separated value file, which was then used to calculate the vector angle. Vector angle calculation was performed as described above. The reproducibility of DIGITS angle measurements for each device was assessed based on the mean absolute error in degrees and percent error of all three trials on each device. Intraclass correlation coefficients (ICCs) were calculated separately for each device to determine the test-retest reliability of the DIGITS web-based application on each device.
To assess the equivalence of the DIGITS software across the three devices (laptop, PC, and smartphone), we performed equivalency testing using the confidence interval (CI) approach.
This method is used to calculate 90% CIs around the mean differences of the groups being equated; these CIs are then compared with an a priori determined equivalency interval. If the CIs fall within the equivalency interval, the groups are said to be equivalent. The equivalency interval is determined using the minimum meaningful difference that would warrant considering the groups nonequivalent. This determination should be based on criteria that are relevant to the research focus.
Thus, the equivalency interval for the purpose of this study was set at ±9°.
A total of 12 CIs were calculated using repeated measures t tests. Given the immense number of data points produced by the DIGITS application, two joints were selected at random and assessed for statistical equivalency. The PIP joint of the left index finger and the DIP joint of the left middle finger were selected, and the outcome of the equivalency tests were extrapolated to all 20 joints. For each joint, CIs were calculated for the laptop versus PC, laptop versus smartphone, and PC versus smartphone for both flexion and extension views. Analyses were performed using the SAS software, version 9.4 (SAS Institute Inc).
Results
Figure 3 shows the average measurements of flexion and extension data for both hands of the participant across the three different devices. The absolute difference and the standard error measurement for each comparison (laptop vs smartphone, PC vs smartphone, and PC vs laptop) for both the left and right hands can be found in Table 1. All comparisons were within the acceptable degree of difference range. The average SD, standard error of the mean, and ICC values for each device are presented in Table 2. The ICC values ranged from good to excellent for all the devices.
Figure 3The average degrees recorded by the DIGITS web application of the left or right hand at full flexion or extension positions using the following three devices: laptop, PC, and smartphone (phone).
Table 2Average SD and Standard Error of the Mean as Well as ICCs for Joint ROM Measurements for Both Right and Left Hands Across the Different Device Types
Equivalency testing was performed for the PIP joint of the index finger and DIP joint of the middle finger of the left hand across all three devices. Table 3 summarizes the overall means and SDs for flexion and extension of both the joints. When the ROM measures were compared between the laptop and PC, equivalency was shown for flexion and extension of each joint and all 90% CIs were within the 9° interval (Table 4). Equivalence was shown for all flexion and extension comparisons between the laptop and smartphone and between the PC and smartphone.
Table 3Mean Joint Angles of PIP Joint of the Index Finger (Single View) and DIP Joint of the Middle Finger (Three View), With SD, Captured by the DIGITS Application Across the Different Devices
This study showed the test-retest reliability of the DIGITS web application across different devices with different camera resolutions. Additionally, the ICC values indicated strong test-retest reliability of each of the devices. Lastly, our equivalency testing using 90% CIs showed equivalence of the ROM data collected through three different devices. In our previous work, we demonstrated that the DIGITS smartphone application differs from in-person goniometry within the acceptable margins of error.
In our current study, we were able to show that the newly developed DIGITS web application also demonstrates test-retest reliability when accessed on devices that have different operating systems and camera resolutions.
assessed finger ROM using a smartphone as an alternative goniometer to subsequently measure each joint separately and semimanually, which was comparable with standard finger goniometry. In contrast, the DIGITS framework approached the assessment of the whole hand in augmented reality and computer vision, improving efficiency in generating a set of ROM measurements for both researchers and patients. “Computer vision” is defined as artificial intelligence that can derive meaningful information from visual input and has been widely incorporated into medicine, retail, and gaming.
By comparing the ROM angle measurements recorded using the three different devices at flexion and extension of the same test subject’s hands, we were able to demonstrate the reliability of augmented reality and the computer vision network across these platforms. This proof of concept provides promising initial evidence for the potential use of this technology as a valuable tool for remote hand therapy assessment.
As for limitations of our study, although our findings were encouraging, they revealed a much lower value in the extension data (SD, 1.68–3.07) than in the flexion data (SD, 4.51–7.91), which is consistent with our previous study.
This is largely attributed to the fact that in a flexed position, the landmarks of the hand that the pipeline seeks to recognize are not directly visible to the camera. Therefore, their three-dimensional locations are largely extrapolated with computer vision models using previous patterns and movements. We can improve the overall accuracy and precision of the application by expanding our data collection. This will be performed by collecting additional measurements in the intrinsic minus hand position (MCP joint in extension and DIP and PIP joints in flexion) to provide more varied views and by compiling data from a larger sample of hands.
Another key limitation of our initial DIGITS smartphone application (distinct from the current DIGITS web application) was that it was only available on Android smartphone devices. The DIGITS web application was developed by recreating and adapting the original smartphone coding to a different platform. The web application increases accessibility by catering to a diverse range of electronic devices (laptops, desktop computers, and phones) with different camera resolutions, operating systems (eg, Android and Windows), and software, thus ensuring that the technology is adapted to patients’ resources rather than the other way around. The only hardware requirements are a camera and internet connectivity.
Our findings indicate that the DIGITS application functions favorably across different devices and platforms, increasing the accessibility of this technology for end users. In the current clinical gold standard, patients need to arrange transport to a health care facility with a trained hand therapist, which are limited by resource availability outside the urban setting. The previously mentioned wearable devices currently used to automate this process are often limited by the cost of implementation and require extensive training for clinical staff. DIGITS will greatly reduce the cost of implementation compared with those of these other emerging devices.
The limitations of our study will be addressed with future research on this technology. The present study demonstrated the reliability of DIGITS across different devices in a single subject’s hand. As such, one key future direction of this research program includes the use of the DIGITS application to assess bimanual ROM in a much larger sample of both healthy and pathologic hands. Further studies must also be performed to demonstrate both the accuracy and reliability of DIGITS in a larger sample of pathologic hands.
Our eventual plan is to demonstrate DIGITS to be an effective tool for tracking recovery to baseline bimanual function in injured hands. Our ultimate goal for the DIGITS web application is to develop dexterity, strength, and artificial intelligence-volumetric swelling analysis as additional metrics of our remote assessment of hand pathologies. Pattern recognition and digital prognostics will help with automation and standardization of subjective clinical assessments of the hands. In summary, with continued development and further validation, the DIGITS application is a promising emerging remote assessment and therapy tool. This tool has the potential to improve accessibility and reimagine the clinical implementation of the standard of care for patients in hand clinics.
Acknowledgments
This project is currently being funded by New Frontiers in Research Fund – 2021 Exploration (Social Sciences and Humanities Research Council (SSHRC), Le counseil de recherches en sciences (CRSH), and Government of Canada) and Lawson Internal Research Fund, London Health Sciences Center (LHSC).
References
Ellis B.
Bruton A.
A study to compare the reliability of composite finger flexion with goniometry for measurement of range of motion in the hand.
Wearable hand module and real-time tracking algorithms for measuring finger joint angles of different hand sizes with high accuracy using FBG strain sensor.
MediaPipe Hands: On-Device Real-Time Hand Tracking. cvpr Workshop on Computer Vision for Augmented and Virtual Reality, Seattle, WA, arXiv:2006.10214 [cs].
Liu W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector. Paper presented at: 14th European Conference; October 11-14, 2016; Amsterdam, The Netherlands. https://doi.org/10.1007/978-3-319-46448-0_2
TensorFlow: a system for large-scale machine learning. Paper presented at: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16).
00020-8&id=gr1.jpg){kind=link}
00020-8&id=gr2.jpg){kind=link}
00020-8&id=gr3.jpg){kind=link}