Gait Reconstruction From Motion Artefact Corrupted Fabric-Embedded Sensors

‘‘Fabric-embedded sensors” are of growing interest in clinical diagnostics and rehabilitation studies that desire the measurement and analysis of human movement outside the laboratory environment. A major issue limiting their usage is the undesired effect of fabric motion artefacts corrupting movement signals. While supervised calibration methods can be used to eliminate these artefacts, these methods make assumptions on the fabric motion, and are unable to address changes in user motion (e.g., locomotion speed) or clothing deformation. In this letter, an unsupervised latent space regression method is presented for learning body movements from fabric motion corrupted sensors, while simultaneously allowing for automatic recalibration. Experiments in this letter show that unsupervised gait learning performs equally as well as supervised learning when removing motion artefacts. This allows for the implementation of adaptive motion artefact methods in real-world sensor-embedded clothing.


I. INTRODUCTION
Measuring and analysing human gait is important in a wide range of fields including medical diagnostics [1] and post-stroke rehabilitation [2].In these, gait analysis is often performed with the aim of informing clinical decisions for an individual, or to make prognoses and select treatment options [3].Accurate gait information is also vital for controlling robotic prosthetics and exoskeletons, for problems such as determining user-intention [4] for assisting movement [5].To perform this, gait analysis involves first capturing kinematic and dynamic information about an individual, often in the form of joint kinematics and electromyography (EMG) [6].
In the clinical setting, these measurements can be obtained from a variety of devices, including sophisticated inertial motion capture systems [7], camera based methods [8], or other strap-on sensors [9].However gait analysis is often underutilised for examining individuals, due to the high cost of the procedure, which can take at least two hours to complete and requires both a trained physiotherapist and technician [6], with an average examination cost of $1,500, and annual facility running costs of $160,000 [10].
Due to this high cost, there has been recent interest in developing human motion sensing systems that operate outside the laboratory (i.e., collecting data independently without a operator), allowing long-term data collection in everyday environments (such as the home).A natural solution has been to incorporate sensors into items already in use by patients, with fabric-embedded sensors [11], also known as e-textiles, being a promising emerging technology.These low-price small scale sensors (costing between $10 to $200 [12], and measuring in the order of millimetres in diameter) United Kingdom brendan.michael@kcl.ac.uk Fig. 1: (A) Prediction of wearer movement (e.g., knee angle Y = (y 1 , y 2 ) ) based on sensor readings.Fabric motion introduces artefacts into the e-textile sensor readings W, compared to sensors fixed rigidly to the body X. (B) Predictive models can be calibrated to estimate Y from W, however variations in user locomotion (e.g., running versus walking) varies the fabric dynamics, creating poorly calibrated models.can include gyroscopes and accelerometers (e.g., ITG3205 gyroscope and ADXL335 accelerometer) for measuring body kinematics.
These new, soft sensing technologies offer significant potential for inexpensive and unobtrusive capture of human movement data, by minimising both physical and visual invasiveness.However, there remain a number of problems in their use, most significantly, how to deal with motion artefacts corrupting data recordings, as caused by the unpredictable motion of fabric sensors with respect to the body (Figure 1).A successful approach to eliminating these motion artefacts has been to use non-parametric statistical learning methods to model body movements, while viewing the motion artefacts as stochastic perturbations to the sensed motion [13].However, this approach requires that, in order to correctly calibrate the fabric-embedded sensors to account for motion artefacts, a baseline (noiseless) data set of the same motions be collected in a calibration stage prior to first-use (e.g., through the use of a high fidelity motion capture system).This requirement limits applicability in realworld motion capture tasks as, due to the large space of possible human motions (e.g., walking at different speeds), the calibration approach would require the computation and switching between multiple fabric-body interaction models.
To solve this problem, this paper proposes to use unsupervised learning methods to learn models of human motion solely from motion artefact corrupted signals, without the need of either a clinical calibration stage, nor the requirement to define discrete motions (e.g., walking, running) prior to use.In this paper, the learning approach is used successfully to extract dependency relationships from motion artefact corrupted signals observed from fabric-embedded sensors.The quality of these artefact free estimates are comparable to the results obtained from a calibrated model, showing the unsupervised learning model's generalisation provides superior usability for continuous real-world gait analysis.

II. RELATED WORK
The problem of undesired fabric motion in e-textile data acquisition is generally minimised by ensuring that there is a strong coupling between the body and the clothing (i.e., by wearing tightly fitting garments [11]).This approach, however, is unsuitable if sensors are to be incorporated into everyday items of clothing, which exhibit features such as slack, stretch, or folding.As such, methods must be applied to remove motion artefacts from sensors embedded in ordinary clothing.
One approach to deal with fabric motion in everyday garments, is to explicitly model wearer/fabric interaction dynamics, for example, by using animation techniques [14], physics-based models [15], or statistical machine learning to estimate constrained cloth states [16].However, these are complex procedures, often requiring a detailed analysis of the interaction mechanics.Coupled with the large number of fabric parameters that affect the dynamics of clothing (e.g., thickness, weave pattern, looseness, and flexibility), these procedures can be prohibitively complex to use.
An alternative solution is to use statistical signal filtering to obtain noise-free estimates of the observed signal.This can be performed in a number of ways, such as using (i) a differential filter [17] in combination with multiple sensors placed on the garment, to remove local artefacts in each sensor, or (ii) an adaptive filter [18] to detect artefacts by using a secondary sensor that is independent of the desired (noise-free) signal, but correlated with the artefact.
However, the key drawback in filtering approaches is the need to have multiple sensors situated on the garment.This presents a problem for fabric-embedded sensors, as (i) all sensors are coupled with each other due to their placement on the deformable surface of the fabric (i.e., all sensors will observe similar noise, making differential filtering unsuitable), (ii) the motion of the fabric (and thereby the artefact) is correlated with the motion of the body (i.e., there is no reference signal available to adaptive filters that is decoupled from the noise-free signal).As such, the standard correlation requirements do not hold in fabric-embedded systems, and more complex and computationally costly implementations are required to address this [19].Even filters that do not require multiple sensors, but make assumptions on the motions and artefacts (such as the Kalman filter [20]), require additional optimisation for dealing with this correlation, on top of the standard tuning for noise assumption and state dynamics [21], limiting the generalisability and increasing user maintenance.

III. PROBLEM DEFINITION
In a standard body sensing task, one would have access to data corresponding to body motions, consisting of N measurements X = (x 1 . . .x n ) ∈ R P×N from P sensors (e.g., limb acceleration), which can be used directly for diagnostic or control purposes.However, in the case of sensing from e-textiles, one does not have access to the measurements X, but measurements W ∈ R P×N obtained from a sensor mounted onto an item of clothing (e.g., an accelerometer embedded in a sleeve).This results in a loose coupling between the wearer motion and sensor readings, introducing motion artefacts, and causing a significant discrepancy between the sensor readings W and the underlying motion of the wearer X.
Specifically, the latter are assumed to be subject to zeromean, additive noise E, that corrupts the sensor readings: (1) To solve this motion artefact problem, it is desired to estimate noise free measurements X ∈ R P×N .To make these predictions, one may learn a generalised linear model of the form: (2) between the measurements W, and the desired prediction X.In this, Φ(•) ∈ R J ×N is a suitable feature matrix or set of basis functions (such as Gaussian radial basis functions or polynomials), and Θ ∈ R J ×N is the parameter matrix, often learnt through error minimisation (e.g., least squares [22]).
To solve this motion artefact problem, it has been shown [13] that artefacts can be eliminated by using statistical errors-in-variables learning techniques, to form a predictive model of the wearer's motion, while explicitly accounting for stochasticity in the measurements W during learning and prediction.Instead of minimising the residual error (as in standard learning methods such as ordinary least squares) the parameter matrix Θ is instead chosen such that squared orthogonal residuals to the predicted curve are minimised, a technique known as Total Least Squares (TLS) fitting (for the full derivation see [13]).This approach has been shown to provide superior estimates X compared to standard methods.
A key step in this approach is that a calibration procedure is performed by first recording motion data from both the fabric sensors and a (noiseless) sensor measuring the target quantity (e.g., a rigidly-attached or optical motion capture sensor).A predictive model that accounts for motion artefacts is then formed, and the rigidly attached sensor may be discarded, in favour of the predictions obtained solely from the fabric sensor readings.
While this learning approach is appealing for dealing with fabric-mounted sensor data, in a practical setting (such as continuous measurement of daily activities), the relationship between the motion of the body, and that of the fabric changes during use.The sources of these changes in fabric motion are various, and can include, (i) a change in wearer locomotion type altering the dynamics of the fabric, changing the frequency and amplitude of clothing movement [24], (ii) clothing being stretched over time, changing the amount of contact between the fabric and body (i.e., greater artefacts), (iii) the wearer themselves varying movement Hip angle (radians) Knee angle (radians) Step Number Mean Squared Error -1.5 -1 -0.5 0 Fig. 2: A) Hip/knee angle/angle plots (from high fidelity motion capture systems), commonly used to represent movements [23]).In this, gait changes between walking and running.B) Mean squared error between noiseless and noise eliminated TLS prediction during walking and running from fabric-sensors, after using artefact elimination methods calibrated solely on walking data.behaviour, either due to gait disorders (e.g., age associated decreased limb strength [25]) or exercise interventions [26].
As such, the assumption of there being static learnt parameters Θ (which defines the relationship between fabric and body motion), is inconsistent with these sources of variation, and can result in poor prediction accuracy.For example, fabric-embedded sensors calibrated in the clinic during a locomotion task (e.g., walking, Figure 1(A)) experiences motion artefacts within a range of amplitudes and frequencies, due to the fabric motion.However, even relatively small changes in a user's gait (e.g., change in locomotion type Figure 2(A)), can result in the behaviour of the fabric also altering (Figure 1(B)), generating additional motion artefacts that are not covered by the initial calibration procedure.This problem is illustrated in Figure 2(B), where models calibrated (TLS fitting) on only one type of motion (walking), predict poorly when the user switches to running.These modalities of motion are not just limited to gait, but can also be seen in other activities (e.g., switching between swings in tennis [27]).
A simple solution to dealing with these modalities, is to compute multiple models of motion, capturing a large range of possible human motions (e.g., different walking modes), then switching to the desired calibration model depending on the user motion.However, not only does such an approach massively increase the clinical time spent capturing all these different motions, but an additional prediction system is required to determine what mode of the motion the user is currently doing.
It can be seen that the lack of ability in this stochastic learning approach to cope with different modalities of motion, limits its applicability in real-world fabric-embedded measurement systems.

IV. METHOD
To deal with these problems, in this paper it is proposed to learn user motion X from the noisy fabric-embedded signals W, without relying on the collection of noiseless observations for calibration.This allows for models of motion to be recalibrated during use, adapting for changes in motion, environment, or fabric dynamics.This is achieved by using: (i) unsupervised learning that exploits existing structure within the motion data to build regression models without user input, and (ii) a lower-dimensional (latent) representation of the motion signals, as an artefact elimination technique in the learning process.
In general, unsupervised non-linear dimensionality reduction techniques are used to discover hidden dependencies between variables within the latent data, thereby learning a manifold that captures relationships between recorded motion parameters (e.g., joint interactions).After learning, new (noisy) measurements are then projected onto the lowerdimensional manifold, and noiseless reconstructions can then be made back in the original data space.
Specifically, this method solves the motion artefact problem by removing components of the noisy measurement that do not have a significant contributory role in the formation of the latent space manifold.An example of this is seen in Figure 3(A), where noisy complex data in a high dimensional space can be seen to mask any relationship between variables.After finding structure within a latent space representation of this data (Figure 3

A. Manifold Learning
There exist a number of dimensionality reduction techniques that may be used to learn latent manifolds.For example, the commonly used Principle Component Analysis (PCA) converts data to a new coordinate system comprised of orthogonal axes (principle components), depending on the variance of the data.Axes with small amounts of variance can be discarded when reconstructing back to the original data space, as they have only limited contribution.However, in the context of human motion, where manifolds are potentially non-linear, many commonly used methods such as PCA or Linear Discriminant Analysis may not be suitable due to their linear assumptions.As such, non-linear methods, such as Kernel PCA [28], or Gaussian processes latent variable models (GPLVM) [29], are more suitable.While the method for motion artefact elimination presented in this paper does not depend on any specific manifold learning technique, a key concern is the the suitability of the method to the lowcost computational constraints of fabric-embedded devices.
To solve this problem, this paper uses Unsupervised Kernel Regression (UKR) [30], due to its low computational cost for learning (O(PN 2 ) [30]) compared to other methods (e.g., GPLVM is O(N 3 ) [29]).In this, latent manifolds are learnt with automatic complexity control, and require minimal a priori specification (only latent space dimensionality and density kernel shape) [31].The UKR computation also contains a number of useful tricks to optimise the training procedure, including making use of the kernel trick to apply learning to general Hilbert spaces, and performing leaveone-out cross-validation with no additional computational cost.In addition, to prevent the algorithm from falling into local minima, candidate solutions (such as linear manifolds obtained from principle component analysis or local linear embeddings) can be used to initialise the training.For further details of the optimisation methods, see [30].
UKR aims to find both a latent representation S = (s 1 . . .s n ) ∈ R Q×N of the observed data W, and a functional relationship W = f (S) between them.In this, Q < P, and Q can be selected by examining criteria such as reconstruction error [32].
In UKR, these reconstructions (also known as the forward mapping) are made by an approximation of the conditional expectation X using smooth kernel regression estimators (such as the Nadaraya-Watson estimator [33], [34]).Kernelbased estimates of the probability space densities are used to compute the prediction: where K is a kernel function.In the second equality, this is written as a weighted set of basis functions.
The latent variables themselves are found via a gradientbased training procedure, that minimises the orthogonal data-space reconstruction error:

B. Motion Prediction from Noisy Sensor Readings
The learnt variables Θ define the optimal latent space manifold for the data.Estimates of the noiseless measurements x can be made by first projecting the noisy measurement w * (e.g., motions captured from fabric-embedded sensors) onto this manifold, and then reprojecting this estimate back into the original data space.Specifically, this is done by finding a latent space estimation s * that minimises the (orthogonal) reconstruction error [35]: (this can be achieved via nonlinear optimisation (e.g., a constrained nonlinear least squares algorithm [36], or backpropagation [30])).From s * , an estimate of the noise free motion x * is then made by application of (3) (i.e., x * = f (s * )).

V. EVALUATION
In this section, the proposed approach is evaluated through a simulation study, and through an experiment on gait learning from fabric-embedded devices. 1

A. Simulation
The goal of the first evaluation is to characterise the performance of the manifold learning approach for learning and predicting movements from noisy sensory inputs.For this, learning is tested on artificial data with additional noise between the sensed input w and the target x.In this, x can represent joint angles as sensed from a high accuracy sensor (e.g., a rigidly attached encoder), and w the fabric sensor reading of the same angles.
In this evaluation, a set of N = 200 two-dimensional coordinates are sampled linearly from the unit circle, to form the matrix of noiseless measurements X ∈ R 2×N .To simulate readings sensed with noise corruption, X is corrupted by additive Gaussian noise E ∼ N ([0, 0], Σ), where Σ = 10 −2 I, to generate the matrix of noisy data W = X + E. The data are then randomly decomposed into independent training (70% of data) and testing data sets (30%) for learning, and shown in Figure 4(A).A one-dimensional (Q = 1) UKR model is then trained using the "UKR toolbox" developed by [30].For this, the noise corrupted training data W is used to fit a manifold in the data-space.No initialisation candidates are presented Further information about the data and conditions of access can be found by emailing research.data@kcl.ac.uk.
to the method, and both the default quartic kernel [37] and default number of back-propagation time-steps (100) are used.For comparison, the supervised errors-in-variables model total least squares (TLS), is also learnt from both W and X using the approach outlined in [13].As in the UKR model, no additional feature space mapping is performed (i.e., the feature matrix is Φ(W) = W).This procedure is repeated on 20 independent data sets.
The prediction results for one data set are shown in Figure 4(B).In this, it is seen that the learnt UKR manifold is a good fit to the noiseless unit circle, and that reconstructions made using UKR have eliminated the additive noise.This is reflected in the low mean squared error (MSE) between the UKR predictions X and the ground truth X, averaging 0.37 ± 0.085 and 0.41 ± 0.098 for the respective axes, over the 20 trials.In comparison, predictions made using the TLS estimator have a larger error (0.64 ± 0.10, and 0.66 ± 0.12) and are seen to have a somewhat poorer fit.
To evaluate the robustness of the method, the experiment is repeated varying the additive noise Σ in the range [0, 0.01]I, with prediction results shown in Figure 4(C).In this it is seen that initially, at low levels of noise, the error in both the TLS and UKR predictions remains low.As Σ increases there is, as expected, a larger error for both TLS and UKR.However, while prediction error for both models increases, the UKR error is less than TLS, indicative greater predictive accuracy at higher noise levels.
The results demonstrate that not only can a statistical model be designed to account for motion artefacts without the need to calibrate with a noiseless data set, but that these predictions can be more accurate than standard errors-invariables techniques.

B. Experiment -Fabric mounted sensor during walking
In this evaluation, the proposed approach is tested in a realworld motion task, with the goal of estimating hip and knee angles during locomotion using fabric-embedded sensors.
The experimental platform for this experiment consists of two independent measuring systems (M 1 and M 2) used to obtain sagittal hip and knee angles, shown in Figure 5(A).These measurements are collected during walking from four healthy male participants, on a treadmill operating at a constant 1.2ms −1 (average speed for men age 20-29 [38]) for 20s.
To obtain hip and knee joint angles, the relative orientation of the two segments forming the joint is computed.In M 1, high-accuracy (noiseless) angular velocity measurements of the shank and thigh are obtained using the full-body Xsens motion capture system [7], which requires a prior calibration procedure involving measuring participant limb segment lengths, and performing poses.Measurements of the sensor angular velocities measurements are sampled at 140Hz, and converted to thigh and shank segment angular velocities through the use of an inbuilt biomechanical model.
Simultaneously, a second data set is collected from M 2, consisting of two ITG3205 tri-angle gyroscopes [39] embedded into the leg of a pair of loose trousers worn by the participant, sampling at a rate of 100Hz.The gyroscopes are located on the mid-points of the upper and lower leg, and are connected to an Arduino Uno via conductive thread wiring sewn into the trousers to a belt pouch.Using conductive thread instead of standard wiring allows for natural motion of the fabric that is not impeded by wiring.Data is streamed via a serial connection to a PC base-station for analysis.Note that unlike the Xsens system, there is no calibration from the sensor to segment coordinate frame, nor filtering, as this is included as part of the unsupervised learning process.
For both systems, a cumulative trapezoidal numerical integration is performed on the limb segment angular velocities, to obtain estimates of the angular position of each limb segment.To determine initial joint orientation, participants momentarily stand still in an upright position at the beginning of each data recording.Integration drift is removed by subtracting the linear trend (found via a least squares fit) from the signal after data collection.Note, in a live-prediction setting, both the determination of initial joint orientation, and the removal of drift, can be solved by using on-sensor absolute orientation computation methods [40], or inexpensive orientation sensors (e.g., the Adafruit BNO055 [41]).The relative orientation between the thigh and shank are then computed to estimate the knee angle.For the hip angle, the relative orientation between the thigh orientation and the vertical (as determined by the upright reference position) is used as a substitute for the pelvic-thigh relative orientation, as this is a reasonable approximation [42].Data from the XSens system (M 1) is then sub-sampled to 100Hz using a least-squares linear phase finite input response filter, forming the matrix X ∈ R 2×2000 for M 1.Data from the fabric system M 2 is stored in the matrix W ∈ R 2×2000 .Motions are also split manually into individual steps for validation and visualisation, but this does not form part of the learning process.
An example of the body angles obtained from both systems is shown in Figure 5(B).In this, it is seen that the sensed motion of the fabric experiences additional noise, making for poor predictions of the underlying true body angles.
For learning, a leave-one-out cross-validation [43] is performed on the recorded steps, exhaustively assessing the accuracy and robustness of the learning method.The proposed UKR approach is then used to learn the latent space representation of the motion-corrupted data W.As a baseline for this comparison, a supervised TLS model is also learnt using both the motion corrupted data W and noiseless samples X.
The predictions for both models are shown in Figure 5(C), for one participant.It is seen that the manifold learnt by UKR exhibits a good fit to the underlying body angle interaction pattern.The prediction results of the UKR and TLS models are shown in Table I.In this, it is seen that the average angle error for one participant remains low for both UKR and TLS modelling, in comparison to the raw noisy data.This low error is also seen across participants, demonstrating the generalisability of the method.These results show that UKR is able to make prediction estimates just as well as learning   It should be noted that there is some error in the segment of the manifold corresponding to the foot-strike (Figure 5(C) lower-right corner), that results in the large standard deviation seen in Table I for participant one for both learnt models.This appears to be due to the high peak ground reaction force [44] causing a higher variability in the fabric motion at this point and, it is seen in Figure 5(B) that the motion of the fabric is shifted to one side of the walking step.This is also illustrated in Figure 5(D), where it is seen that the (noisy) mean walking step observed by the fabric embedded sensors is slightly shifted away from the ground truth step.As such, in this segment, the fabric exhibits a non-mean zero distribution of noise.This complex distribution of noise explains why UKR does not outperform TLS as previously seen during the simulation ( §V-A).However despite this complex noise breaking the zero-mean noise assumption made by UKR, it still performs equally as well as the supervised learning techniques, without requiring explicitly defined noiseless body motions.

VI. DISCUSSION
In this paper, the application of unsupervised non-parametric learning in estimating human motion through noise corrupted fabric-embedded sensors has been investigated.A major issue in the implementation of artefact elimination techniques in fabric-embedded sensors, is the ability to deal with the wide range of human motion and wearer/fabric interactions that occur during everyday usage.Using explicit calibration models (e.g., motion capture) to create motion artefact elimination models is unsuitable due to the large number of calibrations that are required to capture all motions.
To address this, it has been proposed to exploit unsupervised statistical learning techniques to not only deal with the effects of stochastic perturbations in measurements, but allow for the automatic recalibration of artefact elimination models.In this, an approach to noiseless measurement reconstruction has been presented based on the use of unsupervised kernel regression for learning lower-dimensional, latent space representations of the noisy motion data.Evaluation of this approach in simulation has shown its ability to outperform supervised errors-in-variables models when explicitly accounting for noise in the independent variables.Experiments in learning human gait cycles has shown that it performs equally as well as the supervised learning techniques, without the need to explicitly perform a manual calibration stage.
In principle, latent space representations can be used to overcome the problems of motion artefacts for any motion.However, the distribution of noise can play a role the accurate modelling and prediction of motions (see §V-B).As such, future work will investigate how information about the distribution of motion artefacts into the learning process (e.g., the heteroscedastic nature), can be used to better account for artefact generation.In addition unsupervised learning techniques that allow for prior knowledge about motions to be incorporated into manifold learning [35], will be investigated to enable not only noiseless measurement reconstruction, but the noiseless prediction of other unobserved points on the body (e.g., predicting end-effector information from sensors mounted on the upper arm).This will allow for greater applicability to systems that desire autonomous motion estimation e.g., wearable exoskeleton devices or intention-prediction in prosthetics.

2 AFig. 3 :
Fig.3: A) Illustration of complex data in an observed high (H) dimensional space.B) Finding existing structure within a lower (L) dimensional (latent) representation of the data, means that the structure of the manifold becomes clear.C) Reconstructing latent variables back to the observed data space removes components of the signal that have limited manifold contribution (i.e., noise).

Fig. 4 :
Fig. 4: (A) Ground truth X and noisy input data W, (B) predictions using TLS estimator and manifold estimation with UKR, (C) mean squared error (MSE) in predictions at different noise levels for both axes (mean ± s.d.over 20 trials).

Fig. 5 :
Fig. 5: (A) Data collection equipment.Shown are Xsens trackers attached rigidly to the body under clothing (green dashed circles) and fabric-embedded gyroscopes (red solid circles).B) (Noisy) measurements W compared to mean walking step X. (C) Predictions using TLS and UKR modles.(D) Mean of noiseless (X) and noisy data (W).(E) Knee angle during gait cycle, as measured from both systems, and predictions made with UKR.

TABLE I :
Mean angle error (degrees • ) in model prediction (mean ± standard deviation).Results for Participant 1 (P1) are over 30 steps, results for all are over four participants.