Advertisement

Validity and Reliability of a Depth Camera–Based Quantitative Measurement for Joint Motion of the Hand

  • Lulu Lv
    Affiliations
    Department of Microsurgery, Orthopaedic Trauma and Hand Surgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
    Search for articles by this author
  • Jiantao Yang
    Affiliations
    Department of Microsurgery, Orthopaedic Trauma and Hand Surgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China

    Guangdong Province Engineering Laboratory for Soft Tissue Biofabrication, Sun Yat-sen University, Guangzhou, Guangdong, China

    Guangdong Provincial Key Laboratory for Orthopaedics and Traumatology, Guangzhou, Guangdong, China
    Search for articles by this author
  • Fanbin Gu
    Affiliations
    Department of Microsurgery, Orthopaedic Trauma and Hand Surgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
    Search for articles by this author
  • Jingyuan Fan
    Affiliations
    Department of Microsurgery, Orthopaedic Trauma and Hand Surgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China
    Search for articles by this author
  • Qingtang Zhu
    Affiliations
    Department of Microsurgery, Orthopaedic Trauma and Hand Surgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China

    Guangdong Province Engineering Laboratory for Soft Tissue Biofabrication, Sun Yat-sen University, Guangzhou, Guangdong, China

    Guangdong Provincial Key Laboratory for Orthopaedics and Traumatology, Guangzhou, Guangdong, China
    Search for articles by this author
  • Xiaolin Liu
    Correspondence
    Corresponding author: Xiaolin Liu, MD, Department of Microsurgery, Orthopaedic Trauma and Hand Surgery, The First Affiliated Hospital, Sun Yat-sen University, No. 58, Zhong Shan Er Lu, Guangzhou, Guangdong 510080, China.
    Affiliations
    Department of Microsurgery, Orthopaedic Trauma and Hand Surgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, Guangdong, China

    Guangdong Province Engineering Laboratory for Soft Tissue Biofabrication, Sun Yat-sen University, Guangzhou, Guangdong, China

    Guangdong Provincial Key Laboratory for Orthopaedics and Traumatology, Guangzhou, Guangdong, China
    Search for articles by this author
Open AccessPublished:October 21, 2022DOI:https://doi.org/10.1016/j.jhsg.2022.08.011

      Purpose

      Quantitative measurement of hand motion is essential in evaluating hand function. This study aimed to investigate the validity and reliability of a novel depth camera–based contactless automatic measurement system to assess hand range of motion and its potential benefits in clinical applications.

      Methods

      Five hand gestures were designed to evaluate the hand range of motion using a depth camera–based measurement system. Seventy-one volunteers were enrolled in performing the designed hand gestures. Then, the hand range of motion was measured with the depth camera and manual procedures. System validity was evaluated based on 3 dimensions: repeatability, within-laboratory precision, and reproducibility. For system reliability, linear evaluation, the intraclass correlation coefficient, paired t-test and bias were employed to test the consistency and difference between the depth camera and manual procedures.

      Results

      When measuring phalangeal length, repeatability, within-laboratory precision, and reproducibility were 2.63%, 12.87%, and 27.15%, respectively. When measuring angles of hand motion, the mean repeatability and within-laboratory precision were 1.2° and 3.3° for extension of 5 digits, 2.7° and 10.2° for flexion of 4 fingers, and 3.1° and 5.3° for abduction of 4 metacarpophalangeal joints, respectively. For system reliability, the results showed excellent consistency (intraclass correlation coefficient = 0.823; P < .05) and good linearity with the manual procedures (r = 0.909–0.982, approximately; P < .001). Besides, 78.3% of the measurements were clinically acceptable.

      Conclusions

      Our depth camera–based evaluation system provides acceptable validity and reliability in measuring hand range of motion and offers potential benefits for clinical care and research in hand surgery. However, further studies are required before clinical application.

      Clinical relevance

      This study suggests a depth camera–based contactless automatic measurement system holds promise for assessing hand range of motion in hand function evaluation, diagnosis, and rehabilitation for medical staff. However, it is currently not adequate for all clinical applications.

      Key words

      Quantitative evaluation of hand range of motion after impairment is essential in many fields, such as medical care, rehabilitation, or appraisals of working capability.
      • Sollerman C.
      • Ejeskär A.
      Sollerman hand function test. A standardised method and its use in tetraplegic patients.
      ,
      • Rondinelli R.D.
      • Genovese E.
      • Katz R.T.
      • et al.
      AMA Guides® to the Evaluation of Permanent Impairment.
      Specialists often use hand-held goniometers to assess hand range of motion, which is the most commonly used and widely accepted method in clinical work.
      • Macionis V.
      Reliability of the standard goniometry and diagrammatic recording of finger joint angles: a comparative study with healthy subjects and non-professional raters.
      ,
      • Bain G.I.
      • Polites N.
      • Higgs B.G.
      • Heptinstall R.J.
      • McGrath A.M.
      The functional range of motion of the finger joints.
      However, manual measurement techniques are time consuming and skill-dependent. Over the last 2 decades, automated devices that use optical, electronic, or magnetic technologies to measure hand motion have shown rapid development.
      • Henderson J.
      • Condell J.
      • Connolly J.
      • Kelly D.
      • Curran K.
      Reliability and validity of clinically accessible smart glove technologies to measure joint range of motion.
      • Lin B.S.
      • Hsiao P.C.
      • Yang S.Y.
      • Su C.S.
      • Lee I.J.
      Data glove system embedded with inertial measurement units for hand function evaluation in stroke patients.
      • Fang Q.
      • Mahmoud S.S.
      • Gu X.
      • Fu J.
      A novel multistandard compliant hand function assessment method using an infrared imaging device.
      Most of these devices are categorized under wearable and contactless devices.
      • Chen W.
      • Yu C.
      • Tu C.
      • et al.
      A survey on hand pose estimation with wearable sensors and computer-vision-based methods.
      ,
      • Sama A.J.
      • Hillstrom H.
      • Daluiski A.
      • Wolff A.
      Reliability and agreement between two wearable inertial sensor devices for measurement of arm activity during walking and running gait.
      However, wearable devices, such as data gloves, might be incompatible and inconvenient because the possibility of measuring various impaired hands with malformation, open injury, or muscle weakness is high.
      • Chen W.
      • Yu C.
      • Tu C.
      • et al.
      A survey on hand pose estimation with wearable sensors and computer-vision-based methods.
      ,
      • Dipietro L.
      • Sabatini A.M.
      • Dario P.
      Evaluation of an instrumented glove for hand-movement acquisition.
      Unlike wearable devices, contactless automatic measuring devices could solve these issues because hand motion can be measured automatically by optical cameras without direct contact.
      • Zabatani A.
      • Surazhsky V.
      • Sperling E.
      • et al.
      Intel® RealSense™ SR300 coded light depth camera.
      ,
      • Lee A.R.
      • Cho Y.
      • Jin S.
      • Kim N.
      Enhancement of surgical hand gesture recognition using a capsule network for a contactless interface in the operating room.
      Various methods have been proposed for contactless human activity measurement with the development of hardware and artificial intelligence (AI) algorithms.
      • Zhu Y.
      • Lu W.
      • Gan W.
      • Hou W.
      A contactless method to measure real-time finger motion using depth-based pose estimation.
      ,
      • Seo N.J.
      • Fathi M.F.
      • Hur P.
      • Crocher V.
      Modifying Kinect placement to improve upper limb joint angle measurement accuracy.
      Many studies have also yielded encouraging data on capturing and recognizing full-body movement with virtual reality.
      • Alagha M.A.
      • Alagha M.A.
      • Dunstan E.
      • Sperwer O.
      • Timmins K.A.
      • Boszczyk B.M.
      Development of a new assessment tool for cervical myelopathy using hand-tracking sensor: part 1: validity and reliability.
      • Colombini G.
      • Duradoni M.
      • Carpi F.
      • Vagnoli L.
      • Guazzini A.
      LEAP Motion technology and psychology: a mini-review on hand movements sensing for neurodevelopmental and neurocognitive disorders.
      • Wright T.
      • de Ribaupierre S.
      • Eagleson R.
      Design and evaluation of an augmented reality simulator using leap motion.
      • Cai L.
      • Ma Y.
      • Xiong S.
      • Zhang Y.
      Validity and reliability of upper limb functional assessment using the Microsoft Kinect V2 sensor.
      • Capecci M.
      • Ceravolo M.G.
      • Ferracuti F.
      • et al.
      A hidden semi-Markov model based approach for rehabilitation exercise assessment.
      • Seo N.J.
      • Arun Kumar J.
      • Hur P.
      • Crocher V.
      • Motawar B.
      • Lakshminarayanan K.
      Usability evaluation of low-cost virtual reality hand and arm rehabilitation games.
      • Leal A.F.
      • da Silva T.D.
      • Lopes P.B.
      • et al.
      The use of a task through virtual reality in cerebral palsy using two different interaction devices (concrete and abstract) – a cross-sectional randomized study.
      • Windolf M.
      • Götzen N.
      • Morlock M.
      Systematic accuracy and precision analysis of video motion capturing systems—exemplified on the Vicon-460 system.
      • Merriaux P.
      • Dupuis Y.
      • Boutteau R.
      • Vasseur P.
      • Savatier X.
      A study of Vicon system positioning performance.
      • Siena F.L.
      • Byrom B.
      • Watts P.
      • Breedon P.
      Utilising the Intel RealSense camera for measuring health outcomes in clinical research.
      There is a notable limitation in the simultaneous evaluation of 14 hand joints using optical motion capture tools, such as Leap motion (Leap Motion Inc) and Kinect (Microsoft Inc).
      • Alagha M.A.
      • Alagha M.A.
      • Dunstan E.
      • Sperwer O.
      • Timmins K.A.
      • Boszczyk B.M.
      Development of a new assessment tool for cervical myelopathy using hand-tracking sensor: part 1: validity and reliability.
      • Colombini G.
      • Duradoni M.
      • Carpi F.
      • Vagnoli L.
      • Guazzini A.
      LEAP Motion technology and psychology: a mini-review on hand movements sensing for neurodevelopmental and neurocognitive disorders.
      • Wright T.
      • de Ribaupierre S.
      • Eagleson R.
      Design and evaluation of an augmented reality simulator using leap motion.
      • Cai L.
      • Ma Y.
      • Xiong S.
      • Zhang Y.
      Validity and reliability of upper limb functional assessment using the Microsoft Kinect V2 sensor.
      • Capecci M.
      • Ceravolo M.G.
      • Ferracuti F.
      • et al.
      A hidden semi-Markov model based approach for rehabilitation exercise assessment.
      • Seo N.J.
      • Arun Kumar J.
      • Hur P.
      • Crocher V.
      • Motawar B.
      • Lakshminarayanan K.
      Usability evaluation of low-cost virtual reality hand and arm rehabilitation games.
      • Leal A.F.
      • da Silva T.D.
      • Lopes P.B.
      • et al.
      The use of a task through virtual reality in cerebral palsy using two different interaction devices (concrete and abstract) – a cross-sectional randomized study.
      Vicon (Vicon) is another optical system that can capture extreme dexterous motions.
      • Windolf M.
      • Götzen N.
      • Morlock M.
      Systematic accuracy and precision analysis of video motion capturing systems—exemplified on the Vicon-460 system.
      However, its multicamera system and marker-dependent accuracy are costly and inconvenient.
      • Merriaux P.
      • Dupuis Y.
      • Boutteau R.
      • Vasseur P.
      • Savatier X.
      A study of Vicon system positioning performance.
      ,
      • Siena F.L.
      • Byrom B.
      • Watts P.
      • Breedon P.
      Utilising the Intel RealSense camera for measuring health outcomes in clinical research.
      Previous studies have focused more on hardware performance than on clinical needs, limiting the use of such instruments in clinical applications.
      • Colombini G.
      • Duradoni M.
      • Carpi F.
      • Vagnoli L.
      • Guazzini A.
      LEAP Motion technology and psychology: a mini-review on hand movements sensing for neurodevelopmental and neurocognitive disorders.
      ,
      • Napoli A.
      • Glass S.
      • Ward C.
      • Tucker C.
      • Obeid I.
      Performance analysis of a generalized motion capture system using Microsoft Kinect 2.0.
      ,
      • Guna J.
      • Jakus G.
      • Pogačnik M.
      • Tomažič S.
      • Sodnik J.
      An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking.
      Besides, the anatomy of hand motion is still poorly understood. Thus, most studies experienced setbacks because of the lack of a supportive theoretical framework of medical knowledge.
      To facilitate rehabilitation and treatment of patients and research in hand surgery, we proposed a depth camera–based quantitative hand range of motion evaluation system. Intel RealSense SR300 (Intel) is based on a triangulation algorithm in which coded light is projected and captured by an infrared sensor that generates the position information of hand motion. The cost of the camera is low (the price of a depth camera was approximately $223); it is portable, with high resolution and efficiency, which may be suitable for clinical application. In this study, we analyzed the validity and reliability of the proposed evaluation system against hand-held finger goniometers. We also explored the system’s potential benefits for clinical care and research in hand surgery.

      Materials and methods

      Participants

      This was a cross-sectional study. The enrollment criteria for volunteers were as follows: (1) the participants could perform hand gestures in front of a depth camera independently and (2) the hand could be recognized as a “hand” by the built-in AI algorithms. In this study, we excluded people with a history of congenital deformity of the hand and those with space-occupying lesions in the hand that might make it difficult to perform tasks as instructed or might not be recognized by the AI. We also excluded people with more than 1 peripheral nerve involved and those with hand amputation and brachial plexus injury or central nervous system injury. Of the 96 volunteers reviewed, 71 participants met these criteria. We included 21 healthy adults and 50 patients who were confirmed to have upper-limb peripheral nerve injury (Fig. 1). Then, patients were further divided into 3 groups: (1) those with median nerve injury, (2) those with radial nerve injury, and (3) those with ulnar nerve injury. The demographic characteristics of participants are described in Table 1.
      Figure thumbnail gr1
      Figure 1A flowchart of the participant enrollment process. A The domains of hand motion. B-D The systematic setup and traditional goniometers. E The 5 hand skeletons obtained using the depth camera.
      Table 1Demographic Characteristics of Participants


      Group
      Healthy Control Group (N = 14)Median Nerve Injury Group (N = 20)Ulnar Nerve Injury Group (N = 20)Radial Nerve Injury Group (N = 10)
      Sex, n (%)
       Male7 (50.0)6 (30.0)13 (65.0)9 (90.0)
       Female7 (50.0)14 (70.0)7 (35.0)1 (10.0)
      Age (y), x¯ ± SD37.5 ± 14.550.3 ± 15.636.55 ± 13.236.3 ± 7.3
      Height (cm), x¯ ± SD170.9 ± 8.4163.5 ± 7.1166.2 ± 22.7173.2 ± 3.1
      Weight (kg), x¯ ± SD63.6 ± 10.658.6 ± 4.664.6 ± 8.063.5 ± 5.5
      HandN = 28N = 25N = 20N = 10
       Left hand, n (%)14 (50.0)13 (52.0)8 (40.0)6 (60.0)
       Right hand, n (%)14 (50.0)12 (48.0)12 (60.0)4 (40.0)
      Injury site, n (%)
       HandNone1 (4)00
       WristNone23 (92)4 (20)0
       ElbowNone1 (4)12 (60)1 (10)
       ForearmNone04 (20)3 (30)
       Upper armNone006 (30)
      Participants were instructed to sit with their arm and elbow on the table to support the hand, 90° flexing elbow followed by neutral pronation of the forearm and wrist. Then, the palmar side of the hand was placed 30–40cm in front of the camera and parallel to the camera plane until it was recognized and a steady hand skeleton image was generated (Fig. 1B1, B2). The hand skeleton consists of 22 measuring points representing the motion center of the hand joint (Fig. 2A ). Three-dimensional spatial coordinates for the 22 measuring points were collected for 10 seconds (3000 values and 125 measuring time points) at a resting interval of 2 minutes. Then, the mean values of the 125 measurements were analyzed as 1 set of replicates.
      Figure thumbnail gr2
      Figure 2Key points of the hand and how to evaluate the hand range of motion using a combination of 5 hand gestures. A The hand skeleton that consists of 22 key points. B Flat hand type 2 gesture with complete extension/hyperextension and adduction of all digital joints. C Flat hand type 1 gesture with complete extension/hyperextension and abduction of all digital joints. D Evaluating the thumb radial abduction using the thumb-up hand gesture. The total active motion function of finger joints was collected using the flat hand type 1 and thumb-up gestures, as represented in E and F. G The gradual enlargement of the thumb and index aperture. H The gradual enlargement of the thumb opposing distance. MCP, metacarpophalangeal.
      Manual measurement using a hand-held goniometer was carried out following the recommendations of a traditional dorsal technique by 2 trained senior hand surgeons in a blinded manner (Q.Z. and J.Y.). Measurement was taken after 2 minutes of rest. The mean of 3 readings for each measurement was considered as 1 replicate. Data acquired by manual procedures served as the gold standard in this study. The depth camera and manual procedures were coded and analyzed by another observer blindly.
      To facilitate automatic measurement of hand range of motion using a depth camera, it is necessary to design a series of standard hand gestures (Fig. 1A, C). Our preliminary experiment revealed that the depth camera had difficulties in recognizing severely deformed or amputated hands and capturing all designed gestures, especially thumb flexion and adduction. Finally, 3 domains of hand function relating to 34 quantitative measurement items were assessed using 5 hand gestures in this study (Table 2; Fig. 2B– H). The 3 domains were finger extension/flexion, finger adduction/abduction and thumb radial abduction (extension), and thumb palmar opposition. The corresponding 5 hand gestures were as follows: (1) flat hand type 1 (complete extension and abduction of fingers), (2) flat hand type 2 (complete extension and adduction of fingers), (3) “thumb up,” (4) thumb-index pinch, and (5) thumb opposition.
      Table 2Hand Gestures and Corresponding Expert Knowledge
      Hand Function ModuleHand GestureBiomechanics
      AgonistFunction SiteNerve
      Amputation(−)Flat hand1/flat hand 2---
      (+)----
      FingerExtension and hyperextensionFlat hand 1EDC

      Lum, Int
      MCP joint, DIP joint

      PIP joint
      RN

      UN
      FlexionThumb-upLum, Int

      FDS

      FDP
      MCP joint

      PIP joint

      DIP joint
      UN

      MN

      MN, UN
      AbductionFlat hand 1DI, ADMFingersUN
      AdductionFlat hand 2PIFingersUN
      ThumbExtension and hyperextensionThumb-upEPB

      EPL
      MCP joint

      IP joint
      RN

      RN
      Flexion-FPB

      FPL
      MCP joint

      IP joint
      MN, UN

      MN
      Radial abductionThumb-upAPLThumbRN
      Adduction-APThumbUN
      Palmar abduction/oppositionThumb-index pinch

      Thumb opposition
      APB

      OP
      Thumb

      Thumb
      MN

      MN
      ADM, abductor digiti minimi; AP, adductor pollicis; APB, abductor pollicis brevis; APL, abductor pollicis brevis; DI, dorsal interossei; EDC, extensor digitorum communis; EPB, extensor pollicis brevis; EPL, extensor pollicis longus; FDP, flexor digitorum profundus; FDS, flexor digitorum superficialis; FPB, flexor pollicis brevis; FPL, flexor pollicis longus; Int, interossei; IP, interphalangeal; Lum, Lumbricals; MCP, metacarpophalangeal; MN, median nerve; OP, opponens pollicis; PI, palmar interossei; RN, radial nerve; UN, ulnar nerve.

      System setup

      The quantitative evaluation system for hand range of motion consisted of a RealSense SR300 device equipped with an red, green and blue camera and an infrared camera.
      Intel
      Intel® RealSense™ Camera SR300 Embedded Coded Light 3D Imaging System with Full High Definition Color Camera Product Datasheet.
      The camera was supported with a tripod and connected to a computer via a Universal Serial Bus 3.0 cable. A software development kit was initially installed, and three-dimensional spatial coordinates for the 22 measuring points of a hand were obtained at 60 frames per second (Fig. 1B2). All items related to the hand range of motion involve angles or distances. They were calculated following algorithms, as described in Appendix 1 (available on the Journal’s website at www.jhsgo.org).

      Experimental protocol

      Validity

      The experimental protocol was divided into 4 experiments. For the first part, a within-subject repeated measuring study was employed. A healthy volunteer was enrolled to perform all of the 5 hand gestures, each of which received 10 seconds of measurement per replicate. The independent variables of the experiment consisted of 3 replicates per run, 2 runs a day, with 2 depth cameras running for 5 days (3×2×2×5 protocol). The length of all 14 phalanges per hand gesture for each hand was measured using the depth camera.
      For the second part, 5 healthy volunteers were invited to perform 3 hand gestures—flat hand type 1, flat hand type 2, and thumb-up. The independent variables of the experiment consisted of 3×10-second replicates at 2 runs a day for 5 days (3×2×5 protocol). Both depth camera and manual procedures collected the extension of 14 hand joints and flexion of 12 hand joints (joint flexion of the thumb was unavailable), and the abduction or adduction of 4 webs was collected separately.

      Reliability

      The third part was carried out to explore the linearity using the hand gestures of thumb-index pinch and thumb opposition. Thirteen pieces of transparent bars in preassigned length were used. The length of the bars increased gradually from 0.5, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 to 12 cm. For the thumb-index pinch (“OK” gesture), the bars were pinched by the thumb and index finger pulp from 0 to 12 cm. For the thumb opposition gesture, bars were pinched from the flexor crease of the thumb interphalangeal joint to the distal palmar crease over the third metacarpophalangeal joint. The distance increased gradually from 0 to 8 cm. Each test had 3 replicates per hand.
      The fourth part was carried out in a clinical setting by 64 volunteers. Of the 83 hands studied, 28 belonged to 14 healthy volunteers and 55 belonged to 50 patients. All the volunteers were instructed to perform the following 3 hand gestures with 1 replicate protocol: flat hand type 1, flat hand type 2, and thumb-up. These hand gestures captured the extension of 14 hand joints, the flexion of 12 hand joints, and the abduction and adduction of the 4 webs using the depth camera and manual procedures.

      Statistical analysis

      A general linear model was used to conduct an analysis of variance (ANOVA) for the data from the first and second parts of the experiment to determine their main effects and interactions. The mean absolute error was the dependent variable. Before performing ANOVA, the sphericity assumption was identified using the Mauchly test or adjusted using the Greenhouse-Geisser test. For the first part, factors that influence the variation of the phalangeal length were analyzed, focusing on independent variables, such as the depth cameras, hand gestures, days, runs, and replicates. On the basis of the results of the ANOVA, system validity was evaluated on the basis of 3 dimensions: repeatability, within-laboratory precision, and reproducibility. For the second part, system validity was assessed with repeatability and within-laboratory precision. The system reliability of repeated measures was also analyzed using the intraclass correlation coefficient, Pearson’s product-moment correlation, and bias. For the third part, system linearity was determined using linear correlation and regression between the measurements. For the fourth part, bias was employed to evaluate system reliability, and the differences between measurements were tested using the paired t test. Five degrees were considered as an acceptable clinical threshold for repeatability, within-laboratory precision, and bias.
      • Pham T.
      • Pathirana P.N.
      • Trinh H.
      • Fay P.
      A non-contact measurement system for the range of motion of the hand.
      ,
      • Swanson A.B.
      • Hagert C.G.
      • Swanson G.D.G.
      Evaluation of impairment of hand function.
      All comparisons were performed at a 2-tailed significant level of .05. The definitions and calculation equations of the terms used in the statistical analysis can be found in Appendix 2 (available on the Journal’s website at www.jhsgo.org).

      Results

      Internal validity of the depth camera measurements

      The results indicated that the type of hand gesture may have an important effect on the measured length of phalanges (P < .001). Besides, there was a significant interaction between the effects of gesture and the depth camera (P < .05). When measuring the phalangeal length, data from the overall measurement validity revealed low to moderate dispersal: the mean percentage coefficient of variation of repeatability was 2.63% ± 0.29%, the within-laboratory precision was 12.87% ± 1.87%, and the reproducibility was 27.15% ± 4.70% (Figure 3, Figure 4; Supplementary Table S1, available on the Journal’s website at www.jhsgo.org). Data from the second part of the experiment showed that when measuring hand range of motion, the overall repeatability and within-laboratory precision were 1.8° and 5.8°, respectively. The mean repeatability and within-laboratory precision were 1.2° and 3.3° for extension of 5 digits, 2.7° and 10.2° for flexion of 4 fingers, and 3.1° and 5.3° for abduction of 4 metacarpophalangeal joints, respectively (Fig. 5).
      Figure thumbnail gr3
      Figure 3Repeated measuring results for the length of 14 phalanges for 5 hand gestures of both hands using depth cameras. A–D The left hand. E–H The right hand. A and F The flat hand type 1 gesture. B and G The flat hand type 2 gesture. C and H The thumb-up gesture. D and I The thumb-index pinch gesture. E–J The thumb opposition gesture.
      Figure thumbnail gr4
      Figure 4Validity for the depth camera–based hand joint motion measurement system. System validity was based on 3 dimensions: repeatability, within-laboratory precision, and reproducibility. CV, coefficient of variation; I, index finger; L, little finger; LH, left hand; M, middle finger; R, ring finger; RH, right hand; T, thumb.
      Figure thumbnail gr5
      Figure 5Comparison between the depth camera and manual procedure for measuring hand range of motion. A and B The bias of digital extension, adduction, and repeated measuring validity. C and D The bias of finger flexion, abduction, and repeated measuring validity. Abd, abduction; Add, adduction; CV, coefficient of variation; E, extension; F, flexion; I, index finger; IP, interphalangeal joint; L, little finger; M, middle finger; MCP, metacarpophalangeal; R, ring finger; T, thumb.

      Comparison of depth camera measurements with manual goniometer measurements

      The linear experiment showed that the measured distances of thumb opposition and thumb-index pinch had positive linear correlation with the actual distances. Although these correlations were statistically significant, they could not fully satisfy the evaluation of the total range of motion of thumb opposition/pinch. This was because the measurements for relatively small distances (0–2 cm for thumb-index pinch gesture and 0–4 cm for thumb opposition gesture) were less reliable than those for larger distances (Fig. 6).
      Figure thumbnail gr6
      Figure 6A–D Linear regression analysis between measured length and actual values(A and C left hand; B and D right hand ) for both A and B thumb-index pingching and C and D thumb opposition.
      The repeated measuring experiment revealed that the depth camera provided excellent consistency in evaluating the hand range of motion compared with the consistency provided by the manual procedures (intraclass correlation coefficient = 0.823; P < .05). The overall association between the 2 measurements showed a significant positive correlation (r = 0.970; P < .001) (Figs. S1, S2, available on the Journal’s website at www.jhsgo.org). The repeated measuring experiment showed that 78.3% of the measurements that differed from the manual measurements were <5° (clinically acceptable).
      Data from the fourth experiment of reliability indicated that the overall measured hand range of motion with the depth camera was lesser than that with manual procedures. However, only the differences in the flexion of the proximal interphalangeal (PIP) and distal interphalangeal (DIP) joints between the manual procedures and depth camera were statistically and clinically significant (P < .05). For the 4 groups of participants (healthy, median nerve injury, ulnar nerve injury, and radial nerve injury), a larger hand deformity resulted in an overall reduced hand range of motion, as measured by the depth camera, and a lower measurement reliability (Fig. 7; Supplementary Table S2, available on the Journal’s website at www.jhsgo.org).
      Figure thumbnail gr7
      Figure 7Paired comparison of hand range of motion between 2 measurements in 4 groups of people. A and E Healthy control group. B and F The median nerve injury group. C and G The ulnar nerve injury group. D and H The radial nerve injury group. A–D The extension/hyperextension and adduction for finger joints. E–H The flexion and abduction for finger joints. Andrew’s curve is included in each figure part at the upper right corner, which was triangulated from the 2 measurements and may indicate the overall difference between the 2 measurements. I, index finger; IP, interphalangeal joint; L, little finger; M, middle finger; MCP, metacarpophalangeal; R, ring finger; T, thumb

      Discussion

      Unlike traditional manual procedures, which are highly dependent on professional staff, hand gestures are the key to analyzing hand motion using contactless activity capture devices, such as depth cameras. The participants can take photographs in front of the camera without the help of observers.
      • Fang Q.
      • Mahmoud S.S.
      • Gu X.
      • Fu J.
      A novel multistandard compliant hand function assessment method using an infrared imaging device.
      ,
      • Zhu Y.
      • Lu W.
      • Gan W.
      • Hou W.
      A contactless method to measure real-time finger motion using depth-based pose estimation.
      ,
      • Buffi J.H.
      • Sancho Bru J.L.
      • Crisco J.J.
      • Murray W.M.
      Evaluation of hand motion capture protocol using static computed tomography images: application to an instrumented glove.
      Capturing hand gestures is the first step in hand recognition, only after which the depth camera can generate hand position information. Thus, we analyzed 5 well-designed and clinically accepted hand gestures to facilitate the automatic measuring of hand range of motion.
      • Zhu Y.
      • Lu W.
      • Gan W.
      • Hou W.
      A contactless method to measure real-time finger motion using depth-based pose estimation.
      ,
      • Nizamis K.
      • Rijken N.H.M.
      • Mendes A.
      • Janssen M.
      • Bergsma A.
      • Koopman B.
      A Novel setup and protocol to measure the range of motion of the wrist and the hand.
      Measurements of hand motion were collected by corresponding hand gestures or a combination of different hand gestures. Because most hand gestures are simple and conveniently recognized, depth cameras can be expected to be valid and reliable in evaluating hand range of motion.
      The first and second experiments systematically investigated the internal validity of the depth camera system and its influencing factors. The findings of this study indicated that internal validity was satisfactory when the phalangeal length and hand range of motion were repeatedly measured, based on the analysis of more than 3 million items of data output from the depth camera–based measurement system. It is important to note that the time taken to collect those data using the depth camera was <2 hours, which was highly efficient compared with the manual procedure. The results of the internal validity of the depth camera and ANOVA indicated that all the variances were expected to be derived from the system because the length of each phalanx of the same participant’s hand is unchangeable during the measurement. Among these, hand gestures had the greatest effect on the source of measurement variance. Further analysis showed that thumb opposition and thumb-up gestures had the lowest and highest variance, respectively (Fig. 3). Our data also agreed with the previous finding that classified hand gestures can achieve a more valid measurement of hand range of motion.
      • Anaz A.
      • Skubic M.
      • Bridgeman J.
      • Brogan D.M.
      Classification of therapeutic hand poses using convolutional neural networks.
      Currently available optical cameras have a universal limitation: the camera’s view of the digit of interest might often be blocked by other digits.
      • Chen W.
      • Yu C.
      • Tu C.
      • et al.
      A survey on hand pose estimation with wearable sensors and computer-vision-based methods.
      Thus, a well-designed hand gesture is perhaps a major solution to solve the issues. It would improve the measurement validity by reducing the blocking. The findings of this study provide insights into the hand gestures used and lay the ground for better gestures that are static and intransitive and have maximum extension and abduction while performing step-by-step flexion and adduction, which can be conveniently measured using a single camera from a special angle of view.
      The linear analysis of the third experiment showed that the depth camera loses its reliability when measuring a relatively small or narrow distance (<2 cm for the thumb-index pinch gesture and <4 cm for the thumb opposition gesture). However, the performance of the depth camera becomes acceptable when the tapping distance increases. These findings can be partly explained by the fact that the finger pulps of both the thumb and the index finger/palm were not rigid.
      • Cutkosky M.R.
      On grasp choice, grasp models, and the design of hands for manufacturing tasks.
      The objects pinched by the thumb and index finger/palm might exert a force on the soft tissue and cause a certain level of elastic deformation, which might have substantially shorten the measured distances compared with the actual distances.
      • Elliott J.M.
      • Connolly K.J.
      A classification of manipulative hand movements.
      As mentioned earlier, the thumb might be self-blocking when pinching or opposing to the minimum extent. Thus, further development is required to improve the depth camera’s performance in measuring short distances.
      One of the most important aspects of this study was to quantitatively analyze the repeated measurement reliability of a depth camera–based measurement system for hand range of motion. Most of the angles obtained using the depth camera were generally smaller than those obtained using manual procedures. This result might be because the manual hand measurement is commonly obtained using dorsal techniques, whereas the depth camera usually obtains hand motion from the palmar view.
      • Cave E.F.
      • Roberts S.M.
      A method for measuring and recording joint function.
      Therefore, the depth camera’s views might be another potential source of measurement variance. Although the paired differences between the measurements were stable, part of the variance might derive from systematic errors and can be rectified through an improved system. After comparing with the standard method of manual procedures, the hand range of motion information collected by the depth camera can be further calculated and analyzed for hand physiopathologic status classification or prediction intelligently. These analyses cannot be conducted with the information gathered from manual procedures.
      In addition, measurement reliability was assessed in clinical applications, especially in patients with upper-limb peripheral nerve injury. Data on hand range of motion showed that most of the measurements were reliable, except for the flexion of PIP and DIP joints that were below the clinical assessment requirement. This outcome differs from previous findings that verified the system reliability for the flexion of PIP and DIP joints using digital image capture devices, such as Leap motion and Creative Senz3D (Intel).
      • Pham T.
      • Pathirana P.N.
      • Trinh H.
      • Fay P.
      A non-contact measurement system for the range of motion of the hand.
      ,
      • Arman N.
      • Oktay A.B.
      • Tarakci D.
      • Tarakci E.
      • Akgul Y.S.
      The validity of an objective measurement method using the Leap Motion Controller for fingers wrist, and forearm ranges of motion.
      However, these studies were broadly consistent with earlier studies, which indicated similar measurement bias with the Leap motion device.
      • Fang Q.
      • Mahmoud S.S.
      • Gu X.
      • Fu J.
      A novel multistandard compliant hand function assessment method using an infrared imaging device.
      ,
      • Nizamis K.
      • Rijken N.H.M.
      • Mendes A.
      • Janssen M.
      • Bergsma A.
      • Koopman B.
      A Novel setup and protocol to measure the range of motion of the wrist and the hand.
      In addition, the system bias increased as the degree of hand deformity changed from median nerve injury through ulnar nerve injury to radial nerve injury. The radial nerve injury group showed the smallest hand range of motion and had the worst system reliability, which has been rarely reported in previous studies. These findings might help prioritize the system with more accurate hand motion analysis in complicated clinical applications.
      When measuring the 3 domains of hand amputation, thumb flexion, and adduction, our preliminary experiment revealed that the AI algorithms of the depth camera could neither recognize the hand nor provide output position information for the whole hand. Amputation of finger or phalanges is related to impairment percentage of hand function; however, it has nothing to do with the range of motion of a healthy hand. Flexion and adduction are only part of the thumb’s range of motion. Thus, the absence of these 3 domains of hand range of motion has a limited impact on the overall reliability and validity of this modality.
      Another limitation of this study was not choosing other optical hand motion analysis devices but choosing hand-held goniometers as the gold standard. Despite the disadvantages of the traditional hand-held finger goniometer, it is the clinical standard, and its reliability and validity have been established. Thus, the depth camera–based measurement system should be compared with the clinical standard before using it in clinical applications.
      In conclusion, this study investigated the validity and reliability of a depth camera–based quantitative measurement system for hand range of motion, especially in clinical settings. The proposed system provides acceptable measurement validity and reliability for evaluating most hand range of motion, potentially benefiting clinical care and research in hand surgery. Before clinical application, there is a need to improve the measurement reliability for flexion of DIP and PIP joints and system validity in recognizing severely deformed hands (including amputated hands) because the system performance reduced with more severe pathology of the hand.

      Acknowledgments

      We thank Professor Alan Leng for his continuous encouragement. We also express our gratitude to Ms Zheng Jianwen for her constant assistance.

      Supplementary Data

      Figure thumbnail figs1
      Supplementary Figure S1Five hand gestures captured by traditional camera and depth camera, respectively. Figure 1A/a to 1E/e represent the healthy hand. Figure 2A/a to 2E/e represent the meadian nerve injured hand with thenar muscle atrophy and thumb oppositon disfunction. Figure 3A/a to 3E/e represent the ulner nerve injured hand. Figure 4A/a to 4E/e represent the radial nerve injured hand with dropping fingers and thumb.
      Figure thumbnail figs2
      Supplementary Figure S2Heatmap of Pearson’s product-moment correlation between depth camera and goniometer for measuring extension and adduction of finger joints.
      Figure thumbnail figs3
      Supplementary Figure S3Heatmap of Pearson’s product-moment correlation between depth camera and goniometer for measuring flexion or abduction of finger joints.

      References

        • Sollerman C.
        • Ejeskär A.
        Sollerman hand function test. A standardised method and its use in tetraplegic patients.
        Scand J Plast Reconstr Surg Hand Surg. 1995; 29: 167-176
        • Rondinelli R.D.
        • Genovese E.
        • Katz R.T.
        • et al.
        AMA Guides® to the Evaluation of Permanent Impairment.
        6th ed. American Medical Association, 2008: 432-456
        • Macionis V.
        Reliability of the standard goniometry and diagrammatic recording of finger joint angles: a comparative study with healthy subjects and non-professional raters.
        BMC Musculoskelet Disord. 2013; 14: 17
        • Bain G.I.
        • Polites N.
        • Higgs B.G.
        • Heptinstall R.J.
        • McGrath A.M.
        The functional range of motion of the finger joints.
        J Hand Surg Eur. 2015; 40: 406-411
        • Henderson J.
        • Condell J.
        • Connolly J.
        • Kelly D.
        • Curran K.
        Reliability and validity of clinically accessible smart glove technologies to measure joint range of motion.
        Sensors (Basel). 2021; 21: 1555
        • Lin B.S.
        • Hsiao P.C.
        • Yang S.Y.
        • Su C.S.
        • Lee I.J.
        Data glove system embedded with inertial measurement units for hand function evaluation in stroke patients.
        IEEE Trans Neural Syst Rehabil Eng. 2017; 25: 2204-2213
        • Fang Q.
        • Mahmoud S.S.
        • Gu X.
        • Fu J.
        A novel multistandard compliant hand function assessment method using an infrared imaging device.
        IEEE J Biomed Health Inform. 2019; 23: 758-765
        • Chen W.
        • Yu C.
        • Tu C.
        • et al.
        A survey on hand pose estimation with wearable sensors and computer-vision-based methods.
        Sensors (Basel). 2020; 20: 1074
        • Sama A.J.
        • Hillstrom H.
        • Daluiski A.
        • Wolff A.
        Reliability and agreement between two wearable inertial sensor devices for measurement of arm activity during walking and running gait.
        J Hand Ther. 2022; 35: 151-154
        • Dipietro L.
        • Sabatini A.M.
        • Dario P.
        Evaluation of an instrumented glove for hand-movement acquisition.
        J Rehabil Res Dev. 2003; 40: 179-189
        • Zabatani A.
        • Surazhsky V.
        • Sperling E.
        • et al.
        Intel® RealSense™ SR300 coded light depth camera.
        IEEE Trans Pattern Anal Mach Intell. 2020; 42: 2333-2345
        • Lee A.R.
        • Cho Y.
        • Jin S.
        • Kim N.
        Enhancement of surgical hand gesture recognition using a capsule network for a contactless interface in the operating room.
        Comput Methods Programs Biomed. 2020; 190105385
        • Zhu Y.
        • Lu W.
        • Gan W.
        • Hou W.
        A contactless method to measure real-time finger motion using depth-based pose estimation.
        Comput Biol Med. 2021; 131104282
        • Seo N.J.
        • Fathi M.F.
        • Hur P.
        • Crocher V.
        Modifying Kinect placement to improve upper limb joint angle measurement accuracy.
        J Hand Ther. 2016; 29: 465-473
        • Alagha M.A.
        • Alagha M.A.
        • Dunstan E.
        • Sperwer O.
        • Timmins K.A.
        • Boszczyk B.M.
        Development of a new assessment tool for cervical myelopathy using hand-tracking sensor: part 1: validity and reliability.
        Eur Spine J. 2017; 26: 1291-1297
        • Colombini G.
        • Duradoni M.
        • Carpi F.
        • Vagnoli L.
        • Guazzini A.
        LEAP Motion technology and psychology: a mini-review on hand movements sensing for neurodevelopmental and neurocognitive disorders.
        Int J Environ Res Public Health. 2021; 18: 4006
        • Wright T.
        • de Ribaupierre S.
        • Eagleson R.
        Design and evaluation of an augmented reality simulator using leap motion.
        Healthc Technol Lett. 2017; 4: 210-215
        • Cai L.
        • Ma Y.
        • Xiong S.
        • Zhang Y.
        Validity and reliability of upper limb functional assessment using the Microsoft Kinect V2 sensor.
        Appl Bionics Biomech. 2019; 20197175240
        • Capecci M.
        • Ceravolo M.G.
        • Ferracuti F.
        • et al.
        A hidden semi-Markov model based approach for rehabilitation exercise assessment.
        J Biomed Inform. 2018; 78: 1-11
        • Seo N.J.
        • Arun Kumar J.
        • Hur P.
        • Crocher V.
        • Motawar B.
        • Lakshminarayanan K.
        Usability evaluation of low-cost virtual reality hand and arm rehabilitation games.
        J Rehabil Res Dev. 2016; 53: 321-334
        • Leal A.F.
        • da Silva T.D.
        • Lopes P.B.
        • et al.
        The use of a task through virtual reality in cerebral palsy using two different interaction devices (concrete and abstract) – a cross-sectional randomized study.
        J Neuroeng Rehabil. 2020; 17: 59
        • Windolf M.
        • Götzen N.
        • Morlock M.
        Systematic accuracy and precision analysis of video motion capturing systems—exemplified on the Vicon-460 system.
        J Biomech. 2008; 41: 2776-2780
        • Merriaux P.
        • Dupuis Y.
        • Boutteau R.
        • Vasseur P.
        • Savatier X.
        A study of Vicon system positioning performance.
        Sensors (Basel). 2017; 17: 1591
        • Siena F.L.
        • Byrom B.
        • Watts P.
        • Breedon P.
        Utilising the Intel RealSense camera for measuring health outcomes in clinical research.
        J Med Syst. 2018; 42: 53
        • Napoli A.
        • Glass S.
        • Ward C.
        • Tucker C.
        • Obeid I.
        Performance analysis of a generalized motion capture system using Microsoft Kinect 2.0.
        Biomed Signal Process Control. 2017; 38: 265-280
        • Guna J.
        • Jakus G.
        • Pogačnik M.
        • Tomažič S.
        • Sodnik J.
        An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking.
        Sensors (Basel). 2014; 14: 3702-3720
        • Intel
        Intel® RealSense™ Camera SR300 Embedded Coded Light 3D Imaging System with Full High Definition Color Camera Product Datasheet.
        • Pham T.
        • Pathirana P.N.
        • Trinh H.
        • Fay P.
        A non-contact measurement system for the range of motion of the hand.
        Sensors (Basel). 2015; 15: 18315-18333
        • Swanson A.B.
        • Hagert C.G.
        • Swanson G.D.G.
        Evaluation of impairment of hand function.
        J Hand Surg Am. 1983; 8: 709-722
        • Buffi J.H.
        • Sancho Bru J.L.
        • Crisco J.J.
        • Murray W.M.
        Evaluation of hand motion capture protocol using static computed tomography images: application to an instrumented glove.
        J Biomech Eng. 2014; 136124501
        • Nizamis K.
        • Rijken N.H.M.
        • Mendes A.
        • Janssen M.
        • Bergsma A.
        • Koopman B.
        A Novel setup and protocol to measure the range of motion of the wrist and the hand.
        Sensors (Basel). 2018; 18: 3230
        • Anaz A.
        • Skubic M.
        • Bridgeman J.
        • Brogan D.M.
        Classification of therapeutic hand poses using convolutional neural networks.
        Annu Int Conf IEEE Eng Med Biol Soc. 2018; 2018: 3874-3877
        • Cutkosky M.R.
        On grasp choice, grasp models, and the design of hands for manufacturing tasks.
        IEEE Trans Rob Autom. 1989; 5: 269-279
        • Elliott J.M.
        • Connolly K.J.
        A classification of manipulative hand movements.
        Dev Med Child Neurol. 1984; 26: 283-296
        • Cave E.F.
        • Roberts S.M.
        A method for measuring and recording joint function.
        J Bone Joint Surg Am. 1936; 18: 455-465
        • Arman N.
        • Oktay A.B.
        • Tarakci D.
        • Tarakci E.
        • Akgul Y.S.
        The validity of an objective measurement method using the Leap Motion Controller for fingers wrist, and forearm ranges of motion.
        Hand Surg Rehabil. 2021; 40: 394-399