As a data-driven endurance athlete, I use several devices to track my performance and progress. Like many others, I occasionally end up in troubleshooting communities (especially those dedicated to Garmin products) trying to understand why a metric looks off or how to recalibrate a sensor.
It’s really fascinating to read the discussions. You quickly see how deeply people are driven by their data, planning their sleep, training intensity, or even rest days based on what their device reports. And as an RWD scientist, I can’t help but pause and wonder: are we really generating real-world data?
The observer effect, revisited
We tend to think of wearables as passive measurement tools, quietly capturing “what happens in real life.” But they are anything but passive. Once people start tracking their steps, heart rate variability, or recovery scores, they rarely remain neutral observers. Many adjust their habits to improve their numbers, going to bed earlier, taking a walk to “close the ring,” or pushing a little harder to hit a training target.
In other words, the act of measurement changes the phenomenon being measured, a behavioral echo of the observer effect. From a scientific perspective, this means wearable-derived data may no longer reflect natural behavior, but rather behavior under feedback influence.
The self-optimization loop
Wearables don’t just collect data; they feed it back. Metrics are turned into insights, insights into nudges, and nudges into action. This creates a self-optimization loop:
- The device measures a behavior (e.g., sleep duration).
- The user interprets the feedback (“My sleep score was low”).
- The user changes their next behavior to improve the metric.
- The next data point now reflects not spontaneous behavior, but a reaction to feedback.
The more sophisticated and gamified these systems become, the more this feedback loop deepens. What we end up capturing is not simply how people live, but how they behave when they are being measured.
Real-world or real-world experimental data?
If that’s the case, perhaps wearable-derived data shouldn’t be treated as purely observational. Instead, they might represent a kind of continuous natural experiment, a world where individuals are both participants and investigators in their own behavioral interventions.
That doesn’t make the data less valuable. But it changes what they mean. Rather than calling them “real-world data,” maybe we should start thinking of them as real-world experimental data — collected in natural settings, but under the continuous influence of digital feedback mechanisms.
Methodological implications
This feedback-driven behavior introduces several analytical and interpretive challenges:
- Measurement-induced bias: the data collection process itself alters the behavior being measured.
- Engagement bias: those who are most engaged with their devices — and their metrics — generate the most complete data, potentially skewing representativeness.
- Temporal reactivity: behavior may change between the initial (curious, motivated) phase and long-term, habituated use.
- Selective attrition: people who dislike or mistrust what their data reveal may disengage, leaving only the self-optimizers in the dataset.
To deal with this, researchers might:
- Differentiate between pre-feedback and post-feedback phases in their analyses.
- Model user engagement or feedback exposure as covariates.
- Conduct sensitivity analyses among stable, long-term users.
- Quantify feedback intensity (e.g., frequency of alerts, coaching tips, gamification elements) as potential modifiers of behavior.
The evolving definition of “real-world”
This argumentation goes beyond methodology, it’s conceptual. What counts as “real-world” behavior in a world where people continuously interact with technologies that shape their choices?
Maybe the notion of naturalistic behavior itself needs to evolve. For many, checking a readiness score before deciding whether to train is as routine as checking the weather before heading out. Digitally mediated decision-making has become part of daily life.
So perhaps this is the real world now, not a deviation from it.
The ethical and epistemological frontier
This also opens deeper questions. If algorithms shape how people act to optimize metrics, then the data we collect are, in part, reflections of those algorithms’ design choices. In studying “real-world behavior,” we are increasingly studying the interaction between human psychology and digital systems, a hybrid organism of user and algorithm.
As we build health evidence on this data, we need to be transparent about that interplay. Governance frameworks and data interpretation guidelines should consider the influence of digital feedback, much as we already do for confounding and bias.
A new class of evidence
Perhaps it’s time to expand our evidence taxonomy. Alongside randomized controlled trials and traditional real-world data, we could acknowledge a third category: augmented RWD — data that emerge from continuous human–machine feedback loops in naturalistic settings.
These are contextualized data — valuable, but shaped by the systems that produce them. Understanding and documenting that context is part of the scientific responsibility.
Conclusion
Wearables have expanded our ability to observe health beyond clinics and labs. However, they also blur the line between observation and intervention. The data they produce are real, but no longer raw.
As both a data-driven athlete and a real-world data scientist, I find this paradox fascinating. We are no longer just observing the world as it is, we’re observing the world as it reacts to being measured.
That realization doesn’t diminish the value of wearable-derived data. It simply invites us to ask more precise questions, not just what people do, but how data itself shapes what they do.
