Intra- and Inter- Rater Reliability Measurements of Kinematic and Temporo-Spatial Parameters of Gait Using a Simple Video Technique

Gait analysis is an effective way of assessing the walking ability of individuals. From a clinical perspective clinical gait assessment plays a significant role in physical medicine and rehabilitation [1]. From a physical medicine and rehabilitation perspective patient rehabilitation is routinely prescribed to improve mobility and walking performance. As a functional assessment measure, Brand [2,3] and Baker [4] suggest that clinical gait analysis should be performed primarily to investigate:


Introduction
Gait analysis is an effective way of assessing the walking ability of individuals. From a clinical perspective clinical gait assessment plays a significant role in physical medicine and rehabilitation [1]. From a physical medicine and rehabilitation perspective patient rehabilitation is routinely prescribed to improve mobility and walking performance. As a functional assessment measure, Brand [2,3] and Baker [4] suggest that clinical gait analysis should be performed primarily to investigate: a) The differential diagnosis between disease entities, b) The assessment of the severity, extent or nature of a disease or injury, c) Monitoring progress in the presence or absence of intervention d) The prediction of the outcome of intervention (or absence of intervention).
Recently rehabilitation and gait research have focused on outcome measures such as: qualitative observational methods [5], the Functional Ambulatory Categories (FAC) [6], the Modified Rivermead Mobility Index [7], the semi quantitative clinical scales such as the Timed-Up-and-Go test (TUG) [8], the Postural Assessment Scale for Stroke Patients (PASS) [9] and the Berg Balance Scale (BBS) [10]. These clinical assessments provide useful information about gait and posture. However, a simple and accessible gait assessment technique which could quantify gait directly would be of great value to clinical research. Three-dimensional movement analysis has been used to do this for many years but it is complex, costly, time consuming and beyond the means and capacity of most rehabilitation services. Recently there have been attempts to develop simpler motion analysis systems with strong potential for clinical use. These clinical systems often have high accuracies, operate in a fast manner and are extremely reliable. Examples of such systems include: the Clinical Stride Analyser ® [11,12]; GAITRite ® Electronic Walkway Mat [13][14][15][16][17][18]; Biometrics Electrogoniometer [12,19]; and XSense Motion Tracker [20]. These systems have been used in conjunction with other devices such as the SMTEC ® Foot Switches system [21], Trunk Tri-Axial Accelerometer System [22], and the Peak Performance Modus ® video based system [14].
Despite the high reliability of these modern clinical motion analysis systems they remain expensive, require technical expertise to operate and clinicians can find them difficult to understand and interpret. To overcome these limitations the study reported here investigated the intra-rater and inter-rater reliability of a new low cost portable and reliable system with potential for use in rehabilitation settings as well as gait laboratories.
To date a new system that consists of a low cost video based portable tool has been developed with the ability to conduct a kinematic analysis of gait using dedicated software that automatically tracks human gait [23]. Although this system was validated using a healthy subject more research is necessary to show that the data acquired by the system is reliable. Within our laboratory, we have developed a new video gait assessment that incorporates techniques developed by Wall [24] and applied by Rowe and others [25,26] in which the participant walks across a 6 metre long, high-contrast-grid vinyl mat while being video-filmed in the sagittal plane. This new simplified and cost effective technique of gait analysis is portable and useful within rehabilitation clinics and gait laboratories. Furthermore, the simple portable system is also capable of bi-directional temporo-spatial gait analysis and the production of sagittal plane estimates of joint kinematics. When the grid mat was placed on top of the GAITrite ® Electronic Walkway Mat a high level of concurrent validity was found [26]. The results also demonstrated a high level of clinical applicability [26]. Also, unpublished pilot data have shown a strong concurrent validity for lower limb kinematic variables when compared against a known three dimensional motion analysis system (Vicon MX Giganet, Oxford Metrics Ltd., UK) as the gold standard. In addition, more unpublished pilot studies performed by Rowe and colleagues suggest this simplified method of gait assessment can be applied to neuro-and musculoskeletal rehabilitation. Although most of the unpublished research findings suggest this simplified method of gait assessment is reliable and valid, the need has arisen to validate this technique. Thus, to reduce the likelihood of erroneous data reliable measures of assessment are essential for the establishment of effective treatment outcomes.
The purpose of this study was therefore to determine intra-and inter-rater reliability of kinematic and temporo-spatial gait parameters using the commercially available ProTrainer system (Sports Motion Inc ® , Cardiff, CA) software, a walkway grid mat and a video camera.

Method Design and ethics
The study was approved by the Bioengineering Unit, University of Strathclyde departmental ethics committee. All participants read the study information sheets and completed an informed consent form. The participants were advised of their rights i.e. to withdraw from the test (for any reasons) at any time before or during the test. They were also given the option to ask any questions about the test.

Participants
Twelve healthy participants (4 females and 8 males), aged 27.4 ± 3.9 years, height 1.71 ± 0.09 m and mass 66.75 ± 10.7 kg were recruited from the university community to participate in the study. All healthy  Emerging Technology for Use in Rehabilitation participants had the ability to walk without an aid. In addition, the reliability of the video gait system was tested using different categories of stroke patients. Four stroke patients with different levels of stroke severity were recruited. These stroke survivors (three females and one male) were aged 55.3 ± 9.5 years, with a height of 1.67 ± 0.07 m and mass of 78.67 ± 21.2 kg. One of the stroke patients had the ability to walk with the aid of an assistant, the second patient used a Zimmer Frame, the third stroke patient used a walking stick and the fourth patient walked independently without an aid. Our sample size of twelve healthy participants and four stroke patients was based on evidence from the literature. To achieve a Power of 80% for reliability values of 0.7 to 0.9 with five repeated measurements, six participants are required [27]. The four stroke patients were included in the study to show the capability of the video gait system being able to assess pathological movement during neuro-rehabilitation.

Raters
Three raters participated in the study. The reliability of the measures obtained using the video gait system was measured by two raters with biomechanical processing experience, and one rater, a practicing clinician. One of the raters with biomechanical processing expertise was a postgraduate student with three years experience in biomechanics data processing / analysis, and the second was a biomechanics researcher with over ten years experience in biomechanics data processing / analysis. Since the ultimate goal was to introduce this tool into clinical practice, it was pertinent to test the reliability of measures when used by clinicians. Thus, the third rater recruited to the study was a physiotherapist with over twelve years experience as a practising clinician.
Prior to recording measurements all raters underwent a standard training session, that provided an introduction of how to use the ProTrainer system (Sports Motion Inc ® , Cardiff, CA) software to extract temporo-spatial parameters. The training lasted for approximately an hour and was concluded once the trainer (who also was a rater) was satisfied the raters had understood how to extract the desired measurements. Each rater extracted the temporo-spatial parameter measurements independently of each other without sight of earlier versions. Identical gait cycles were processed and analysed. Two ratings were made by the same rater with respect to each walking trial. The length of time elapsed between ratings was approximately one hour.

Equipment
The equipment used and how it was set out in space is illustrated in The Linoleum Walkway Grid Mat was designed and manufactured to provide bi-directional measurements to an accuracy of less than 0.01 m. The surface of the Linoleum Walkway Grid Mat was colour coded in neutral colours, black and white checkered patterns, and different shades of grey. Visually, the colour theme combination ensured temporo-spatial measurements were obtained with less difficulty when analysed using the ProTrainer system (Sports Motion Inc ® , Cardiff, CA) software. The width of the grid mat was divided into two halves by a dark grey vertical line. The length of the grid mat was divided into six sub-divisions of 1 m set of dimensions. Each 1 m dimension along the direction of progression was colour coded in black and white checkered patterns, white, and black colours. Each 1 m dimension was sub-divided into units of 0.1 m and represented by a lighter shade of grey and white colours. Each 0.1 m dimension was further sub-divided into 0.05 m dimensions and graduated such that measurements to the nearest centimetre could be obtained.
The E3JM photoswitches (Emitter / Receiver) (OMRON, IL, USA) were assembled and connected to a black box light bulb indicator. The custom made black box light bulb indicator was wired to function as a flash light triggered by the photoswitches. Two sets of receiver and emitter photoswitch beams at about 1.2 m in height were placed directly opposite each other at both ends of the grid mat and approximately 1.5 m from the sides of the grid mat. These photoswitch beams were positioned at right angles to the mat so they were aligned with the end of the grid mat and 6 m apart in the direction of progression. The camera was also placed at right angles to the grid mat and about 4 m

Procedure for data collection
Participants were advised to place each limb on either side of the vertical line. The vertical line also acted as a visual aid that guided the participant in the direction of progression during the gait assessment. The video camera and photoswitches were switched on. The functionality of the photoswitches and black box light indicator were checked and then activated. The black box light bulb indicator was off (dark) initially and came on when the infra-red light beam was broken and went off again when the infra-red light beam was restored. The area of capture was set using the video camera zoom facility. The video camera zoom facility was adjusted to ensure at least two-thirds of the grid mat was within the camera's field of view. The focus was also adjusted so that a clear image of the grid mat in the centre of the image was achieved. Additional adjustments were made to ensure the black box light bulb indicator was facing the video camera and in its field of view. After confirmation that the bull's eye markers were visible on the video screen, prior to capturing video data, a trial run was performed to ascertain the equipment was working.
All participants wore comfortable shoes. The workspace volume and test area was cleared and free of any obstacles. Bull's eye markers ( Figure 2) were made as sticky labels.
Precautions were observed to ensure the grid mat was flat and firmly secured to the floor. If need be tape was used to secure the grid mat so as to avoid volunteer's tripping or the mat slipping. When the participant arrived comfortable clothing in the form of a Lycra ® Suit was provided. The bull's eye markers were then placed on the subject by peeling the bull's eye marker sticky labels from the template sheet. In the sagittal plane facing the view of the video camera the bull's eye marker sticky labels were placed on the patient's hip joint (greater trochanter), knee joint (lateral epicondyle), ankle joint (lateral malleolus), toe (first and fifth metatarsals) and the heel (medial and lateral sides) of both the left and right limbs ( Figure 3).
The hip joint marker position was identified by palpation. The Anterior Superior Iliac Spine (ASIS) and Posterior Superior Iliac Spine (PSIS) positions on the pelvis were identified on each side and a sticky label positioned mid way between them to denote the hip. This position was in line with the greater trochanter. The knee joint marker position was identified by passively flexing and extending the knee joint while observing the skin surface on the lateral aspect of the knee joint. The subject was instructed to stand as the lateral epicondyle was identified. The ankle joint marker position was identified by dorsiflexing and plantarflexing the ankle joint. All participants stood upright with their feet plantargrade (flat on the floor). Bull's eye markers were placed on the lateral malleolus of the subject's left and right legs. The forefoot (left toe and right toe) markers were placed in the sagittal plane and in line with the front edge of the subject's shoe. The outer black border line of the bull's eye marker placed so that it was in line with the front edge of the subject's shoe. The heel marker was also placed in the sagittal plane and in line with the edge of the heel of the subject's shoe. Precautions were observed to ensure the outer black border line of the bull's eye marker was flush with the edge of the subject's shoe. Care was also taken to ensure both the toe and heel markers were on the same level and parallel to the long axis of the foot. The video data were then captured at 210 Hz and recorded.

Data processing
The ProTrainer system (Sports Motion Inc ® , Cardiff, CA) software has user friendly playback features as well as video analysis tools. After data collection, the videos were played back and analysed. Gait events were identified using the ProTrainer system (Sports Motion Inc ® , Cardiff, CA) software bookmark feature. The gait events measured were the position of the lower limb at initial contact, foot flat, mid stance  and terminal contact. The software zoom features were used to clearly identify the gait events before recording the output measurements in Microsoft Excel. Temporal features were also recorded at specific time points. Spatial features were measured using the grid mat dimensions as well as the software drawing tools. The drawing tools had straight line and angle measurement capabilities. Additionally, the drawing tools also had calibration settings with equivalent unit dimension settings. The unit of measurement was set to SI units (m) and up to an accuracy of three decimal places. The calibration procedure and settings were evaluated based on pixel values which were then converted and represented by equivalent vertical and horizontal length measurements.
Regarding the task each participant performed six walking trials (three gait trials each up and down the grid mat). All gait trials were processed and analysed in the sagittal plane. The gait outcome measurements were determined from each of the walks performed across the 6 m long grid mat. The primary outcome measure walking speed was determined from the video data acquired using the infrared light beams and the black box with light indicator. As the video camera data was played, the first frame the light came on was recorded as the initial time. The second time the subject broke the infra-red light beam was recorded as the final time. Walking speed was determined by dividing the distance covered by the subject (6 m) by the duration of the walk (Final time -Initial time). The process was repeated for each trial. The primary outcome measure was walking speed. Gait events from the video recordings were identified and using the ProTrainer system (Sports Motion Inc ® , Cardiff, CA) software toolbar the following measurements were obtained: Other outcome measures were gait speed symmetry measurements, temporal symmetry measurements, spatial symmetry measurements, and the angle of the tibia with respect to the vertical (tibia angle of inclination) at initial contact, foot flat, mid stance and terminal contact as a measure of smooth forward progression of the lower limb.
Gait speed symmetry measurement was expressed as a function of walking speed. This measurement parameter represented a relationship between the ipsilateral (right) limb walking speed and the contralateral (left) limb walking speed kinematic variables. Gait speed symmetry was defined as the ratio of the walking speeds for the right and left limbs.   Temporal (step time) and spatial (step length) symmetry measurements was determined by a frame by frame video analysis aimed at identifying the spatial location of the feet with their corresponding temporal parameters. Temporal symmetry was defined as the ratio of step times of the right and left limbs. Spatial symmetry was expressed as a ratio of step lengths of the right and left limbs.
Angle of tibia measurement with respect to the vertical (tibia angle of inclination) at initial contact, foot flat, mid stance and terminal contact was measured from the captured video for each specific gait event. The tibia angle of inclination was calculated as the angle of the tibia with respect to the vertical for each specific gait event.

Data analysis
Descriptive statistics were used to summarise the results. Means and standard deviations for each of the temporo-spatial parameters and kinematic measurements with respect to the trials and between each rater were compared. Mean differences between the three raters were determined. Correlations between the raters were examined. Intraclass correlation coefficients (ICC) for intra-rater reliability were obtained with corresponding 95% Confidence Intervals. A two way mixed effect model using an absolute agreement definition where rater effects are random and measured effects are fixed was used. IBM SPSS Statistics 19.0 software was used for the statistical analysis. The reliability of the measurements was interpreted based on the classifications of reliability coefficients developed by Landis and Koch [28]. In accordance with the classification the ICC interpretation scale was represented as follows: poor to fair (below 0.4), moderate (0.41 -0.60), excellent (0.61 -0.80), and almost perfect (0.81 -1.0). Cronbach's coefficient alpha was used to quantify inter-rater reliability. Significance level was set to P = 0.05.

Results
Reliability study in healthy participants With respect to kinematic variables all raters showed similar mean values with low standard deviations ( Table 1). The walking speed mean differences were small and ranged from 0 to 0.02 m/s. For each rater, the temporospatial variables with respect to the trials were close as reflected by the size of their standard deviation ( Table 1). The mean difference for the temporo-spatial variables were also low and ranged from 0.01 m to 0.86 m for the spatial variables and 0.01 s to 0.09 s for the temporal variables. The tibia angle of inclination with respect to the gait events produced very small mean differences between raters. Also within the trials for each rater the tibia angle of inclination were similar as indicated by the size of standard deviations ( Table 1).
The results showed a high intra-and inter-rater reliability for both the kinematic and temporo-spatial parameters. The ICC value for the intra-rater reliability test was 0.993 for the kinematic variables, and ranged from 0.941 to 0.956 for the temporo-spatial variables and 0.731 to 0.954 for the tibia inclination angle with respect to gait events ( Table  2). Based on the ICC interpretation scale, the results were classed as either excellent or almost perfect. Between raters there was a high inter-rater correlation for the kinematic and temporo-spatial variables. In particular, only the tibia inclination angle at initial contact for the right limb and at foot flat for the right limb produced low correlation values for Rater 1 versus Rater 2 and Rater 2 versus Rater 3 respectively. Also, it is worth noting that the smallest range of intra-rater correlation values were observed for the tibia inclination angle at initial contact for the left limb. However, the tibia inclination angle at initial contact for the left limb produced the largest 95% confidence interval ( Table 2). In particular, the inter-rater reliability results produced ICC values greater than 0.822 for all the measurement parameters measured (Table 4).

Reliability study in stroke survivors
The results for the kinematic variables were very close with the exception of the gait speed symmetry data produced by Rater 3 ( Table 1). The results show that Rater 3 produced a mean difference > 0.04 m/s when compared to Rater 1 and Rater 2. There was no statistical differences between Rater 1 and Rater 2 (P = 0.147), Rater 2 and Rater 3 (P = 0.12) and, Rater 1 and Rater 3 (P = 0.224) for the temporo-spatial parameters. All the kinematic and temporo-spatial variables produced very high intra-rater ICC values. Also between raters a strong correlation was also observed ( Table 3). The inter-rater reliability results for the kinematic and temporo-spatial parameters produced ICC values greater than 0.971. High ICC values were also observed at the tibia inclination angle with respect to gait events for both the left (range: 0.986 -0.999) and right (range: 0.971 -1.000) limbs (Table 4).

Discussion
Simple gait assessment systems are ideal and preferred within   [4,29]. Our results suggest there is a good level of agreement with respect to each trial and between raters for the kinematic and temporospatial measurement parameters. This data is supported by the mean differences, sizes of the standard deviations as well as the ICC intraand inter-rater reliability. The data acquisition and analysis process was simple to learn by all raters. In general, all other variables produced excellent measurements within a good range (ICC > 0.73) for both the healthy participant and stroke survivor assessments. The results also indicate the level of repeatability and reliability across raters (in terms of digitising the bull eye marker during the process of data acquisition and analysis). The small differences between the raters could be attributed to the final placement and position of the goniometric tool on the bull's eye marker during the process of digitisation.
The results generated from the current study are comparable to the literature. On average our healthy participants produced a walking speed of 1.37 m/s. This kinematic parameter measurement agrees well with the results obtained from earlier studies [10,30,32]. This value is within the walking speed range of 1.30 m/s to 1.50 m/s reported by Reid [30], Steffen [32], Bohannon [31] and Öberg et al. 1993. Despite the walking speed kinematic parameter showing a strong agreement with the literature, our spatial parameter, step length was on average lower than those previously reported by Reid [30], and Öberg et al. 1993. While the walkway grid mat and kit is suitable for a simple biomechanical evaluation of gait, the parameters generated from this study have strong clinical implications. Specifically with respect to the derived parameters such as gait speed symmetry, temporal symmetry and spatial symmetry, these gait parameters could be used as outcome measures to examine patients with neuromuscular deficits such as stroke patients. With respect to stroke patients both paretic and non-paretic lower limbs can be examined. Furthermore, these gait parameters could operate as functional outcome measures where definitive features from pathological knees can be distinguished from non-pathological knees. There is a lot of scope for the use of this kit both within and outside the clinical environment.
Our study used a sample size comprising of only twelve healthy participants and four stroke survivors. This sample size may be small and could be considered as a limitation. A total of twelve healthy participants were justified as supported by the power analysis. Considering that this study was designed to primarily strengthen the already acquired preliminary data, choosing twelve healthy participants and four stroke patients was adequate for the intra-and inter-rater reliability study. Also, twelve participants and four stroke patients was considered more than ideal for our preliminary test because it showed the functionality of the system as a rehabilitation tool designed to evaluate both healthy people and patients with neurological and musculoskeletal deficits. As a rehabilitation tool it is cost effective and easy to operate. Processing and analysing the data using the ProTrainer system (Sports Motion Inc ® , Cardiff, CA) software is not complicated but user friendly. Currently, the data processing and analysis can be lengthy and is manual, thus an automatic system will be preferable to a manual system. Various trial runs of this simplified system and cost effective technique of gait analysis have been executed within the clinical setting among physiotherapists and nurses. Within the Bioengineering Unit, University of Strathclyde, the system has also been used by students carrying out gait analysis studies among a healthy population and among patients. At present there is a growing demand for the use of this video gait assessment system particularly among our research students and clinical collaborators. In fact all user groups have found running and implementing the designed user guide and standard operating protocol with little or no difficulties. This has been encouraging feedback for both biomechanical and clinical researchers who have tested the applicability of the kit within a rehabilitation and gait laboratory environment.
In addition, although the system is capable of bi-directional measurement, for the purpose of this study the step width data was not incorporated as a measurement parameter. The future of the kit is promising. Data analysis improvements are underway with the ultimate goal leading towards reducing the data processing duration. Pattern recognition algorithms are being developed that ultimately will be interfaced with the system. While this may sound complex it is not and will not change the user friendliness of the data processing procedure instead it will reduce the data processing and analysing time significantly. More developments in terms of analysing asymmetric gait patterns, shuffled gait patterns and other functional activities of daily living are currently being investigated.

Conclusions
The reliability of a new simplified video technique of gait analysis has been investigated and can be considered as an objective measurement technique of gait analysis. The walkway grid mat and kit is affordable and cost effective as all components and accessories can locally be obtained. Both the intra-rater ICC values and interrater reliability results for the healthy participants and stroke survivors were high. The intra-rater results when interpreted showed excellent to almost perfect intra-rater reliability according to the classifications outlined by Landis and Koch. The validation data suggest the kit is capable of generating highly reliable and repeatable data. Also the kit can be used as a simplistic assessment for screening gait as well as targeted rehabilitation. While our results support the reliability of the video gait system, efforts are currently being made to use the kit as a clinical rehabilitation functional outcome measuring evaluation kit within stroke unit centres in Norwich, UK, and the West of Scotland, UK.