The intuition of this layer is part of what makes it so interesting to build. People already have strong priors about exercise, sleep, and substance use. The model's job is not to confirm what everyone already suspects, but to find where the signal actually exists and how strong it is compared to everything else in the system.

Physical Activity

NHANES grouped physical activity into two primary types:

• Moderate activity: causes light sweating or a moderate increase in breathing and heart rate. Brisk walking, casual cycling, recreational swimming, and light yoga.

• Vigorous activity: causes heavy sweating or a large increase in breathing and heart rate. Running, HIIT, competitive sports, and heavy resistance training.

Sedentary time was also collected independently, capturing how much of the day someone spends sitting regardless of how much they exercise. The visual below shows how both activity types associate with outcomes across all five dimensions.

More movement, better outcomes. The relationship holds across all five dimensions and the gap between the most and least active groups is one of the more consistent findings across the entire dataset. The person getting regular activity in is pulling away from the person who is not, and the data shows it across energy, mood, cognition, and beyond.

What is more interesting is what happens when you separate the two activity types. Moderate activity held its own against vigorous across mental health outcomes and in some cases edged ahead. For mood and energy in particular, a long walk appears to do at least as much as a hard run or lift.

Sleep

NHANES captured sleep data across several dimensions: average hours, weekday and weekend hours separately, bedtime and wake time for both, and a derived measure of social jetlag. That last variable reflects the shift people make between their weekday and weekend schedules, the kind of internal clock disruption that tends to fly under the radar but has a real effect over time.

Sleep duration shows one of the cleaner patterns. Higher mood scorers cluster tightly around seven hours while lower mood scorers spread further into both tails, undersleeping and oversleeping at higher rates. The middle tertile sits in between, but the separation at the extremes is where the real story lives.

What is more surprising than hours alone is that timing appears to be just as impactful. Weekday bedtime and weekend wake time both showed meaningful associations with anxiety and mood outcomes, independent of total duration. Someone sleeping seven hours on a shifted schedule looks different in the data than someone sleeping seven hours on a consistent one.

Duration gets most of the attention. But what the data suggests is that when and how consistently someone sleeps may matter just as much.

Substance Use

Substance use was measured through alcohol, tobacco, and caffeine-related variables. Alcohol produced the clearest signal of all three. It was tracked across quantity, frequency, binge behavior, and a chronic use flag derived from sustained heavy consumption patterns. The visual below compares non-drinkers against chronic users across the four mental health outcomes the FPI is built to improve.

Across all four outcomes, chronic users score consistently and meaningfully below non-drinkers. If you find yourself running low on energy, struggling to focus or concentrate, or just feeling a little off mentally, it is worth asking whether drinking patterns are part of the picture.

Something worth noting is that cognition and mood show slightly negative scores for non-drinkers too, sitting just below the population average despite being the “healthy” group. That feels counterintuitive but reflects a well-documented pattern called the sick quitter effect, where people who abstain entirely are more likely to have stopped due to illness or medications than to have simply never had the habit.

Examples like this are precisely why the context layer exists. Before any behavioral signal gets evaluated, the model first accounts for age, medical history, and the broader circumstances someone is operating within. A non-drinker managing multiple chronic conditions is not the same baseline as a non-drinker in good health, and without that separation the data will be misleading.

From a Data Perspective

This post closes out Phase 2. Every domain across context, drivers, and outcomes has now been preprocessed and organized into the model-ready dataset. The real question preprocessing is meant to answer is not what data is available, but what data is actually worth using.

One step worth noting is the application of survey weights. NHANES is a complex survey designed to represent the broader U.S. population rather than a simple random sample, and the final dataset carries four survey design variables as pipeline infrastructure. Without them the model would be drawing conclusions from a sample rather than the population it is meant to represent.

From here the series moves into modeling and production across the next several updates:

• Update #9: Pruning, where the variable set gets trimmed to what the data can actually support.

• Update #10: Stratification, where comparison groups get formally defined before any behavioral signal is evaluated.

• Update #11: App design and layout, where the model output starts becoming something a person can actually use.

The preprocessing phase has been slower than a typical series update cadence, and intentionally so for two reasons. The first is that a system designed to operate at this level of complexity requires genuine depth in each domain before any of it gets modeled. It is better to understand each domain before the model runs than to untangle bad assumptions after it does.

The second is that documenting each domain publicly while actively building it forced a level of clarity that pure development work rarely demands. When you have to explain why a variable belongs in the model or why one was excluded, you either know the answer or you find out quickly that you don’t. That discipline carried into the modeling work and made the decisions that followed cleaner and more defensible.

The pace picks up from here and the next few updates will reflect it. Grateful as always for the support along the way.

That is what the context layer is for.

Context refers to the parts of a person’s profile that are fixed, slow-moving, or not treated as direct behavioral levers, but still meaningfully shape who they should be compared with. It is the baseline foundation that comes before behavior.

For this stage of the FPI, the context layer is organized around three core areas that are supported by research literature and available within the NHANES scope: basic information, medical history, and socioeconomic status. Together, these categories help define the comparison group before the system moves into the drivers people can actually change.

Basic Information

The first group is the most straightforward. It is mainly split between demographics such as age and gender, and body context parameters such as height and weight. These variables are simple to interpret, easy for a user to answer, and explain meaningful differences in how functional outcomes show up across people.

Age is the clearest example. It does not determine someone’s functional health, but it changes how the rest of the profile should be read. A response that looks unusual in one age group may be much more common in another, and a fair comparison requires accounting for that before interpreting deeper health or behavioral patterns.

When you look across an entire population, small shifts like these can still reflect real baseline separation between groups. That separation also tends to compound once behavioral drivers enter the picture, since age shapes how the body responds to the same inputs differently.

Weight is a more nuanced parameter. It is not the target outcome of the FPI, and treating it that way would pull the system away from functional health. At the same time, weight still carries contextual information that may matter for how people experience energy, physical demand, and overall health. It is one of those variables that can be easy to overfocus on in health products, but also wrong to ignore entirely.

These kinds of basic inputs are low-effort and high-value signals. They are not the most complex part of the system, but they help set up cleaner stratification later on by making sure people are compared against groups that better reflect their starting point.

Medical History

The second group moves into territory that requires a bit more care. While basic information helps define someone’s general profile, medical history adds a stronger signal of what their body may already be managing. Someone with a cardiovascular background and someone dealing primarily with gut health issues have different nutritional needs, different risk profiles, and different bodies to design guidance around.

NHANES captures medical history in two ways. The survey collects overall prescription medication count alongside a detailed inventory of specific diagnosed conditions. Grouping participants by total medication and condition burden creates a reliable proxy for overall medical load and is essential to the first layer of stratification. From there, the condition-level data is preserved for future analysis to allow the model to go further, placing individuals within more specific clusters that better reflect the unique biological context shaping their outcomes.

The visual below takes a broad look at how medical conditions are distributed across the U.S. population.

More than half of American adults live with at least one chronic condition, and a meaningful share are managing several simultaneously. That spread is what makes this dataset so well suited for the kind of layered analysis the FPI is built around. There is enough variation across the medical burden spectrum to build reliable comparison groups at multiple levels, not just at the extremes.

At a broader level, this section ties directly into the core thesis of the FPI and much of what Digestive Data has been building toward. People with different chronic conditions and medical backgrounds should be receiving different behavioral guidance to navigate their health.

Socioeconomic

The third area is the one most commonly left out of health tools entirely. Socioeconomic context does not fit neatly into a behavioral framework, so it often gets treated as outside the scope of the model. In reality, it impacts someone’s access to quality food, time available in a day, the realistic ability to prioritize sleep or exercise. These are not gaps in behavioral discipline. These are real-life constraints.

This layer includes variables most people would immediately recognize: insurance coverage, hours worked, income level, and food security status. The last one most directly pertains to the constraints mentioned above and tends to capture what the others collectively produce, which is why it is the focus of the visual below.

For every outcome, the pattern is consistent. Food secure participants score higher on overall health, energy, mood, focus, and stress management, with several gaps exceeding a full point. These are not marginal differences at a population-level. They are a result of the circumstances people are operating within daily, and a model that ignores them will consistently misread the people most shaped by them.

Socioeconomic status is included in the context layer for the same reason medical history is. Not because the FPI is designed to change it, but because pretending it is not there produces a less honest model. The goal has always been to meet people where they actually are, and that requires knowing something about the conditions they are operating within before evaluating anything else.

Biomarkers as Optional Enrichment

A previous update covered the biological backbone of the FPI, where lab markers were grouped into physiological axes like inflammatory, metabolic, hepatic, kidney, lipid, and hematologic domains. Those markers still matter, but they serve a slightly different role than the context variables discussed here.

In the app, biomarkers will function as context enrichment rather than the starting point. Lab panels can improve the quality of someone’s profile when available, but they are also expensive, unevenly accessible, and not something every user can be expected to provide. The first version of the FPI should still be useful without requiring bloodwork.

The FPI is being built around functional outcomes first: what people feel, experience, and want to improve in daily life. Biomarkers can strengthen the profile and add biological depth over time, but they should support the functional model rather than replace it.

From a Data Perspective

Context variables are an essential layer of the FPI and are fundamentally handled differently than the rest of the model. They serve as confounders, effect modifiers, and stratifiers, which means their job is to define who someone is being compared against before any behavioral signal is evaluated.

Because of that role, most of these variables are passed through with minimal transformation. Raw NHANES coding is preserved where possible, sentinel values are cleaned to NaN, and sparsity decisions are made on a case by case basis depending on whether a variable can meaningfully contribute to a comparison group. Some low count variables were dropped. Others, like specific liver condition subtypes, were kept at this stage despite sparse coverage because the biological context they carry is too relevant to exclude.

The next post wraps up preprocessing entirely. From there the work shifts into actual model building, where many of these variables will be pruned down to what the data can reliably support. The process has been thorough by design, but the model that comes out the other side will be leaner than what went in. As always, I appreciate your patience and support with this build.

Chronic disease often gets discussed in clinical terms. But what most people with a persistent condition actually want is simpler and harder to measure. To feel better. To have more energy. To think more clearly and carry less anxiety. The core foundation of the FPI is that many patients experience similar functional challenges across a wide array of unique chronic conditions. Different diseases, often similar daily burden. That shared functional cost is where a population-based framework can find personalized behavioral direction even when the underlying condition differs.

If you have been reading these updates, you may have noticed the name change from DSI to FPI. While my health story was specific to digestive health, this tool was always meant for anyone managing chronic disease. Since NHANES is far richer in nutrition and lifestyle data than GI-specific measures, the system has more depth at the behavioral level than anywhere else. Functional performance is what this project is actually measuring, and the name now reflects that.

The Raw Candidate Pool

Before any filtering, the preprocessing script assembled every NHANES variable that could plausibly capture how poor health is experienced in daily life. These variables span depression screeners, functional limitation surveys, self-rated general health, body composition, a physical balance exam, and respiratory modules. There were dozens of candidates, and many of them were engineered into composites before any selection decisions were made.

This step was primarily about preserving all possible signal before condensing and making cuts. Anxiety severity was built by combining how often someone experiences it with how intense it is when it occurs. Cognitive limitation was blended from two different survey instruments. Fall burden was folded from three separate measures.

The objective was to build everything correctly before deciding what to use, leaving 33 possible model outputs after initial preprocessing. While only a handful will actually end up being used after filtering, variables must be handled carefully as those collapsed carelessly at this stage cannot be recovered.

Three Core Decisions in the Filtering Process

Ultimately, the end goal for this product is an app where someone can select from a handful of outcomes and get personalized guidance based off who they are. With this in mind, the outcomes need to be narrowed down to those that are both representative of real-world symptoms and have meaningful variance in responses.

The filtering process involved many small decisions and a few larger ones. These three shaped the outcome layer most.

1. Body composition was reclassified as context. Weight, BMI, and waist circumference matter to daily function, but they sit at a different level of the system than the outcomes the FPI targets. They reflect years of accumulated behavior, genetics, and medical history as much as anything happening right now. Using them as outcomes would let the model predict body frame rather than behavioral change, which strays from the core premise of the FPI.

2. Physical limitation was removed as a primary outcome. Over 80% of the 18-64 population reported no physical limitation, producing a floor effect with no usable variance. A variable where nearly everyone scores at zero has nothing to model. The chronic disease experience absolutely includes physical function, but forcing signal out of a floor effect will produce useless predictions.

3. Certain terms were collapsed into composites. One example is the PHQ-9 appetite item which asks about "poor appetite or overeating". While this question is clinically relevant to depression as it directly covers dietary disruption, it cannot be modeled on its own. This is because the same score can mean two opposite behaviors, with no coherent direction for a recommendation to point. As a result, the information is preserved through a depression composite rather than abandoned entirely.

The Five Core Outcomes

After filtering, five outcomes remained. These fundamentally capture different aspects of what people experience, have varying interactions with daily behaviors, and have a rich variance in response necessary for modeling.

1. Overall Health. Drawn from a single NHANES question asking participants to rate their general health on a five-point scale. While the most general, it is one of the strongest predictors of future hospitalization in the research literature and has near-full population coverage.

2. Energy Levels. Comes from one PHQ-9 item capturing how often someone has felt tired or had little energy over the past two weeks. Scored and inverted so higher reflects more energy. Fatigue is also one of the most universally reported symptoms across chronic conditions.

3. Stress & Anxiety. Built from two functional limitation survey items capturing how often someone experiences anxiety and how severe it is. The two items are z-scored and averaged into a single composite. This is kept separate from mental resilience because the two respond to meaningfully different behavioral inputs.

4. Focus & Cognition. Blends a functional limitation item on difficulty concentrating with a PHQ-9 cognition item, averaged when both are available and falling back to one when only one is present.

5. Mental Resilience. Uses the full PHQ-9 depression score sum across nine symptom dimensions, ranging from zero to twenty-seven. Scored and inverted so that higher reflects greater resilience. The total is more sensitive to gradual behavioral change than any individual item.

The distributions above reflect each outcome across the full 18-64 working-age population. Mental Resilience in particular is visibly right-skewed, which is expected in a generally healthy working-age population. Non-normality across these outcomes was handled carefully, and the gradient boosting models used in the FPI are robust to it by design.

The larger takeaway is what these five outcomes represent: the pillars that every behavioral recommendation in the system will be built on.

Do These Five Outcomes Overlap?

Naturally, five outcomes measuring different dimensions of someone’s health are going to share some common ground. The matrix below shows exactly how much.

Pairwise correlations across the working-age population range from weak to relatively strong, spanning 0.22 to 0.66 depending on the pairing. This reflects the biological reality that a person struggling with their overall health is more likely to also struggle with energy, depression, and cognition.

The more important point is behavioral. Each of these outcomes responds differently to the same drivers. Sleep timing associates more strongly with anxiety than other outcomes. Vitamins and supplements affect energy more heavily than cognition. Five separate models will surface those distinctions. One composite score would average them away, producing recommendations that are broader and less useful to the people relying on them.

From a Data Perspective

Defining these five outcomes required balancing two things that do not always align: what people actually experience living with chronic disease every day, and what a large population dataset can reliably measure and model. The goal was never to capture every dimension of poor health, but to identify the outcomes that are both meaningful to the people using this system and supported by enough signal in the data to generate trustworthy, personalized recommendations.

The next two posts close out Phase 2. One covers the remaining behavioral driver domains: sleep, physical activity, and substance use. The other covers the context layer that will anchor the comparison groups: demographics, medical history, and socioeconomic status.

The end of preprocessing is close. Thank you for your patience with this build.

Nutrition in NHANES goes considerably further than most datasets allow. Over 300 raw variables across dietary intake, supplement behavior, and individual food items were collected to provide a complete picture of someone's nutritional profile. The goal was not to keep them in their original form, but to engineer a more deliberate set of signals that reflect how food and eating actually show up in daily life.

This update covers the seven subcategories within the nutritional layer, the decisions made to build each one, and what they capture together.

Nutrient Intake

The most fundamental data within this layer comes from dietary intake. Across two days of recall, participants were tracked across roughly 70 specific variables capturing absolute consumption. These covered everything from macronutrients and fiber to individual vitamins, minerals, and fatty acid subtypes.

To make this information useful, intake was averaged across both recall days and standardized for modeling. That process also produced a measure of dietary consistency across days, which serves as a confidence signal in the model. Participants whose intake was more stable across both recall days have a more reliable relationship between what they reported eating and the outcomes FPI is designed to predict.

Below is a look at how several of those nutrient distributions actually sit across the population.

After standardization, most variables distributed close to normal. The one notable pattern was directional: macronutrients tended toward a slight right skew while vitamins and minerals often leaned left. For macros, the average is inflated by a subset eating significantly more than most, making typical intake look higher than it is. On the contrary, most people have enough vitamins and minerals in their diet but those who fall short tend to fall well short.

Theobromine is a good example of how this can break down for less common nutrients. Its distribution is visibly bimodal, with two distinct peaks driven almost entirely by whether someone consumed chocolate or cocoa on a given day. For nutrients this specific, the shape of the population distribution is not about gradual dietary variation. It is about whether one or two particular foods showed up at all.

Density & Dietary Ratios

While raw intake captures what entered the system, it doesn’t account for whether that intake was appropriate relative to how much someone was eating overall. Calculating density measurements was essential for making meaningful comparisons across people with very different caloric baselines.

To address this, each nutrient was normalized per 1,000 kilocalories consumed. A person eating 3,000 calories a day and a person eating 1,600 may report similar raw totals, but their nutritional efficiency looks very different once intake is scaled to consumption. Each density variable was derived independently and added alongside its raw counterpart, giving the system two distinct lenses on the same nutrient.

Beyond individual density variables, a set of ratios between key nutrients was created to capture broader dietary patterns rather than any single measurement in isolation. Combinations like protein-to-fiber and carb-to-sugar reveal things neither variable shows alone. Someone could have perfectly adequate protein intake, but if their fiber is consistently low relative to it, that imbalance may matter more to their outcomes than any single nutrient in isolation.

Together, density variables and ratios move the nutritional layer beyond simple accounting. They capture not just what someone consumed, but how balanced and efficient that consumption actually was.

Food Groups

The first two sections focused on sheer and relative intake. This one looks at the specific foods behind those numbers.

Five groups were selected based on their established relevance to chronic disease and systemic health. Each food item reported across both recall days was mapped to a group using USDA food codes, aggregated into daily gram totals, and averaged across days to produce a consistent measure for each participant.

• Fruits and vegetables: apples, berries, citrus, broccoli, spinach, carrots, tomatoes, potatoes, and fruit juices

• Legumes and beans: beans, peas, lentils, soy products, and meat alternatives

• Whole grains: oatmeal and low-sugar ready-to-eat cereals

• Red and processed meat: beef, pork, lamb, organ meats, bacon, sausages, and cold cuts

• Ultra-processed foods: chips, cookies, pastries, candy, soft drinks, energy drinks, pizza, and fast food sandwiches

What stands out immediately is how unevenly these groups are represented across what we eat.

Ultra-processed foods appeared in 92% of diets, a stark reflection of how deeply convenience has shaped the way most people eat. While fruits and vegetables held strong, other health-supportive categories like whole grains and legumes rarely made it to the dinner table. The contrast across groups is exactly the kind of variability that will allow the model to find meaningful differences between profiles.

These five categories are among the most actionable variables in the system. Unlike nutrients which require interpretation, food groups map directly to decisions people make every day. The aim is to understand where someone's food patterns diverge from healthier profiles and use that to surface the dietary adjustments most likely to matter for people like them.

Meal Time & Sourcing

Most nutritional datasets capture what people eat. NHANES also captured when they ate, how long they were eating across the day, and where their food came from. That behavioral layer adds a dimension raw nutrient data cannot provide on its own.

Timing variables included the time of first and last eating occasion, total eating window, number of eating occasions, and meal occasion proportions across breakfast, lunch, dinner, and snack. Research has increasingly shown that when someone eats matters to metabolic health and daily function independently of what they consume, making these variables a necessary addition to the nutritional picture. The chart below captures what that looks like at the population scale.

The distance between first and last meal is particularly informative. It can reflect anything from intermittent fasting to late-night snacking habits that slow down overnight recovery. Identifying where someone falls within that range changes how their nutritional profile should be read and what recommendations are actually relevant to them.

Each food item was also tagged by where it came from, spanning grocery stores, fast food, restaurants, and food assistance programs. That sourcing signal, paired with timing, shifts this sublayer from a nutritional snapshot into something closer to a full picture of how someone actually eats day to day.

Supplement Use

To round out the nutritional layer, supplement use was tracked separately from dietary intake. Since the body does not absorb nutrients the same way from a pill as it does from food, that distinction is worth preserving for future modeling.

Prevalence varied widely across supplements. Vitamin D was the most commonly reported at 45%, while several others like lutein, phosphorus, and potassium sat below 15%. That sparsity created a modeling challenge. For nutrients used by fewer than ten percent of the population, continuous variables were encoded as binary flags instead to capture use versus non-use rather than amount.

More than half of participants reported taking at least one supplement in the past 30 days, and among those who did, the average user was taking nearly four. That pattern suggests a population that is actively, and in some cases aggressively, trying to manage their nutritional intake outside of diet alone.

Capturing supplements alongside dietary intake means the model has access to total nutritional exposure rather than just what showed up on someone’s plate.

From a Data Perspective

The nutritional layer is the most variable-dense sublayer in the FPI, and the one that required the most deliberate engineering. After preprocessing, it produced 183 variables, nearly half of all possible features in the model. That density reflects how much signal NHANES captured, and is a large part of why this sublayer warranted its own dedicated post.

Some of these variable decisions came from well established research and common nutritional practices, such as nutrient density measurements and food group classification. Others capture things that are often overlooked in standard nutritional analysis, like meal timing, dietary consistency, and food sourcing. Together they aim to create a grounded foundation while leaving room for more nuanced recommendations as the system develops.

The next update rounds out the driver layer by covering the remaining three domains: substance use, sleep, and physical activity. Within the next few posts, the foundation will be complete and the system will move from being built to being run. As always, I appreciate the patience and support as the FPI continues to take shape.

The FPI is built on the premise that the biological picture and the lived experience of chronic disease need to be understood together. While symptoms and lab results may not always align, biomarkers provide an essential biological foundation for understanding how well the system is being tolerated at a structural level. They reflect physiological strain that may not yet rise to visible disease, but still shapes daily function over time.

This update focuses on one part of that picture: what biomarkers reveal when organized as a coordinated system rather than read in isolation.

From Raw Labs to Structured Axes

NHANES collected 93 biomarker variables for each participant, spanning inflammatory markers, metabolic panels, blood counts, renal indicators, liver enzymes, and lipid measures. While extensive, raw laboratory data requires careful preparation before it can produce anything meaningful.

The cleaning approach followed three core steps:

• Remove structural duplication. Highly correlated variables representing the same physiological signal in different units or formats were identified using pairwise correlations above 0.995. One representative marker was retained to remove redundancy while preserving each signal.

• Consolidate repeated measures. Blood pressure readings for systolic, diastolic, and pulse were collapsed into mean values to avoid future model overweighting.

• Normalize and standardize distributions. Skewed metabolic markers were log-transformed where necessary, and all variables were standardized to allow comparison across systems operating on different scales.

By the end of this process, 60 usable biomarkers remained. After removing duplicates and standardizing the remaining variables, the next step was feature engineering.

Rather than grouping markers by the laboratory panels in which they were collected, they were reorganized by the biological systems they most directly influence. ALT and AST, for example, were grouped within a hepatic axis because they move together as indicators of liver strain. Albumin and globulin were placed within a malabsorption context due to their link to protein balance and inflammatory stress.

This shift moves the focus from how tests are administered to what systems they reflect. Individual biomarker relationships remain intact, but they now sit within coordinated physiological dimensions instead of isolated lab panels. All 60 variables were grouped into eight system-level axes, shown below.

Across the eight dimensions, distribution patterns are largely stable, with only modest skew in most cases. The metabolic axis remains more right-skewed, a feature that was further corrected and will be incorporated into downstream modeling decisions.

Interaxis Correlation

With the eight axes defined, the next step was to determine whether they function as distinct physiological dimensions. If the structure is valid, the axes should not simply mirror one another. Some overlap is natural in biology, but persistently high correlations would indicate redundancy.

To evaluate this, pairwise correlations were calculated across all eight axes at the population level.

Across the matrix, cross-axis correlations fall between 0 and 0.30, with most near zero. This supports the interpretation that the axes represent largely independent physiological dimensions and confirms successful separation of biological domains.

Because these calculations are based on large population-level data, effect sizes are expected to be modest. Heterogeneity dampens correlations. As a result, a low-to-moderate coefficient in this context can still indicate meaningful biological linkage in more narrowly defined subgroups.

Taken together, the structure holds. The axes remain largely independent, with only mild associations between certain domains. Those modest links may become more informative under deeper or subgroup-specific analysis.

Spotlight: Malabsorption & Hematologic Burden

The strongest cross-domain signal appears between the Malabsorption and Hematologic axes at r = 0.30.

On its surface, a correlation of 0.30 may seem modest. But across a broad, unadjusted population, a coefficient of this size points to moderately coordinated physiology. In this case, the link makes biological sense. The intestine governs absorption of iron, B12, and protein substrates required for red blood cell production. When nutrient absorption shifts, hematologic markers tend to follow. The relationship is not deterministic, but it reflects a coherent pathway connecting digestive function to systemic blood dynamics.

Practically, this matters because inefficient absorption does not stay confined to the gut. It can surface as downstream vulnerability in blood-based markers before overt deficiency is diagnosed. Identifying these bridges early provides a clearer sense of where physiological stress may be accumulating and where attention may be warranted next.

This pairing illustrates the broader aim of the FPI framework. Instead of evaluating systems in isolation, it looks for coordinated strain across domains. Further analysis will examine which specific markers within each axis are most responsible for these alignments and under what contexts they strengthen.

Inside the Axes

Having verified independence across axes, attention now shifts to relationships within each axis.

Rather than reviewing all eight axes in detail, this section focuses on two of the most relevant domains as a spotlight: the Hepatic axis and the Malabsorption axis. The goal is not to be exhaustive, but to examine whether the internal structure within these domains reflects real biological relationships.

Within the Hepatic axis, ALT and AST are strongly correlated at 0.76, which is expected for markers of hepatocellular injury. GGT also shows moderate alignment with these enzymes, linking liver cell stress with biliary and metabolic strain. Together, these relationships validate established and well-researched liver physiology.

The Malabsorption axis shows similarly coherent structure. Albumin correlates with calcium (0.51), consistent with the role of albumin in binding circulating calcium. Total protein and globulin are also strongly linked (0.66), reflecting their shared contribution to overall protein balance. These internal relationships are clinically grounded and reinforce that the axis captures meaningful information.

The key is structural balance. The system separates domains of burden while preserving the internal complexity that makes biological interpretation possible. Structure is introduced without flattening the nuance that allows real signals to emerge.

From a Data Perspective

Working with biomarkers at this scale requires restraint as much as analytical judgment. It is easy to bake in assumptions, overinterpret correlations, or collapse complexity into a single composite score. The goal here is different. The objective is to introduce structure without distorting physiology.

The biomarker layer is the first context subgroup completed within the FPI. It provides the structural backbone necessary to map biology onto functional outcomes in later phases. The growing system map below visualizes how these domains fit within the broader framework across context, drivers, and outcomes, along with key subgroups and representative variables.

The next update will move into diet structure within the driver layer, capturing variation across protein intake, fiber density, sodium exposure, eating patterns, and food sourcing behaviors. With the biological backbone established, the focus now shifts toward building out remaining subgroups and analyzing the relationships that connect them.

Most people assume that generating data-driven insights starts with exploratory analysis or data cleaning. But the reality of a system as complex as chronic disease is that the data arrives complicated, fragmented, and far from perfectly packaged.

In the previous FPI update, I laid out the three core layers that define the framework behind the Functional Performance Index: context, drivers, and outcomes. That framework describes how daily function should be interpreted, but the variables do not magically place themselves into clean categories.

The real process of understanding the data begins earlier. It starts with deciding how each signal should be classified and where clear boundaries need to exist.

This post focuses on that preprocessing step. Decoding raw variable names, adding metadata that governs how each variable should behave inside the system, and organizing related parameters. Taken together, these steps allow daily function to be interpreted as a system rather than a collection of isolated measures.

Decoding Raw Variable Names

Large health datasets rarely arrive in plain language. In NHANES, most variables are identified by short codes and abbreviations created for data collection rather than interpretation. On their own, they offer little clarity about what a variable captures or how it should be understood.

Preprocessing began by reviewing the documentation for each data file and writing out clear descriptions for every variable. That process quickly revealed a pattern. Many variables look nearly identical at first glance, particularly in their wording, despite being designed to capture different aspects of health.

A frequent example is the difference between questions framed as “have you ever had” and those that ask “do you still have.” Although the wording appears subtle, the distinction is meaningful. The latter identifies individuals who are currently living with an active condition, which is critical for separating historical diagnosis from present burden.

Thyroid related variables illustrate this clearly. Both reference the same condition, but one reflects a past diagnosis while the other captures current burden. This separation creates a natural comparison between individuals with a history of disease and those who no longer experience ongoing impact, making it possible to study factors associated with improvement.

After understanding these distinctions, the next step is making them explicit through allowed values and metadata.

Defining Allowed Values

Once variables are decoded, classification does not stop at assigning them to a layer. Each parameter also needs rules that define how it should behave inside the system.

This is where allowed values come in. Rather than leaving interpretation open ended, each variable is paired with a small set of metadata fields that constrain how it can be used. These rules ensure consistency across the system and prevent similar looking signals from being treated the same by default.

The table above shows the allowed values used to define how variables behave inside the FPI. Each is assigned values across several dimensions:

• Layer. Indicates whether the variable belongs to context, drivers, or outcomes. This defines the role the signal plays within the system.

• Type. Further specifies what kind of signal it is within its layer. For example, context variables are separated into categories like demographics or medical history, drivers into domains like diet or activity, and outcomes into energy, mood, stress, cognition, physical capacity, or overall health.

• Actionability. Describes whether the signal can be directly changed, indirectly influenced, or should be treated as non actionable. This prevents fixed characteristics from being treated as levers for change.

• Temporal behavior. Defines how the signal changes over time. Variables may be static, slow moving, fast moving, or episodic, which determines the time horizon over which changes should be interpreted.

• Use flag. Specifies whether a variable should be included in modeling, included conditionally, or excluded entirely. This adds an explicit gate for downstream use.

These allowed values are not descriptive labels. They act as constraints that tell the system how seriously to take a signal, how often it can change, and when it should be used. With these rules in place, variables behave consistently across analyses.

The next step is applying this structure to real variables and showing how similar signals are intentionally treated differently in practice.

Applying Metadata Rules

Once allowed values are defined, they can be applied to real variables in a way that preserves meaning. The screenshot below shows how similar variables are classified using allowed values, ensuring each contributes to the system as intended.

Context variables capture historical and structural factors such as prior doctor visits, previously diagnosed conditions, and socioeconomic stress. While difficult to change, they are essential for defining baseline comparison profiles and enabling meaningful personalization.

Driver variables represent signals that can change over time, including nutritional intake, active medications, dietary structure, sleep patterns, and activity levels. These function as levers rather than background conditions and form the basis for recommendations within similar baseline profiles.

Outcome variables reflect where poor daily function is most apparent. Some capture direct symptom burden while others represent broader functional impact across energy, mood, cognition, stress, and physical capacity. Together, these outcomes link behavioral inputs to the dimensions of daily life that matter most.

This structure supports consistent interpretation and preserves meaningful differences across the system, creating a necessary foundation for delivering the kind of personalized recommendations the FPI aims to provide.

From a Data Perspective

This phase of the FPI closely mirrors what I learned working with large-scale data at PNC Bank. When data scales, it never arrives neatly organized. Variables overlap, concepts are represented multiple ways, and similar signals appear across different sources with slightly different definitions.

In those environments, the challenge is not running models. It is deciding how data should be understood. Which variables describe the same idea. Which ones capture different states. And which distinctions matter enough to preserve.

People often say that data scientists spend 60 to 80 percent of their time cleaning and preparing data. While not always the most exciting step, taking the time to define structure before anything is modeled is what prevents small assumptions from compounding into systematic errors.

As the FPI continues to take shape, the breadth of the data suggests a clear next step. Rather than expanding across every subsection at once, the focus will likely shift toward building depth within individual domains and refining how signals interact within those spaces. That process will guide how the system moves from structure into insight.

This post focuses on the developing framework underneath that idea. Not results yet, and not recommendations, but the logic that connects context, drivers, and outcomes into a single functional signal.

Chronic disease is often discussed as a collection of isolated symptoms or habits. The FPI approaches it as a system. This update breaks down how that system is organized and why each layer matters.

Why Chronic Disease Needs Layers

Chronic disease is frequently framed as one problem, even though it is shaped by many interacting factors. Symptoms, behaviors, lab values, and personal circumstances all get mixed together, making it difficult to tell what is actually driving poor daily function versus what is simply being experienced.

The Functional Performance Index separates health into three layers to keep those signals distinct:

• Context defines the baseline conditions a person brings with them, such as demographics, medical history, and medications. These factors shape interpretation but are not targets for change.

• Drivers represent modifiable behaviors and physiological processes, including diet, activity, sleep, and metabolic markers. These are the levers that influence daily function.

• Outcomes capture how health is experienced day to day, reflecting impact across energy, mood, stress, cognition, physical capacity, and overall health.

Each layer answers a different question. Context explains who someone is. Drivers explain what may be influencing change. Outcomes reflect what that change feels like.

Keeping these layers separate allows patterns to be interpreted more clearly and compared more meaningfully across similar profiles. Together, they span over 1,000 parameters across context, drivers, and outcomes. Defining these layers is the first step toward understanding complex interactions without oversimplifying them.

Context: Who You Are Before Anything Changes

Context defines the baseline conditions a person brings with them. It describes the constraints and background factors that shape how health is expressed and interpreted across chronic disease and daily function.

This includes factors like demographics, body composition, medical history, medications, and access-related variables such as income or food security. These elements influence outcomes, but they are not levers to pull.

Two people can follow the same diet, have similar symptoms, or show comparable lab values and still experience very different outcomes because their context is different. Age, prior disease, prescriptions, and baseline physiology all change how signals should be read.

In the FPI, context is used to create appropriate comparison groups. Instead of asking how someone compares to the population at large, the system asks how they compare to others with similar baseline profiles.

Context does not explain what to change. It explains how to interpret what is already there.

Once that baseline is established, the next layer focuses on what may actually be pushing the system toward or away from poor daily function.

Drivers: What Actually Moves the System

Drivers are the factors that actively push daily function toward better or worse outcomes. Unlike context, these are not fixed. They represent behaviors and physiological processes that can change over time and meaningfully influence how someone feels and performs.

In the FPI, drivers fall into two broad categories: behavioral and metabolic.

Behavioral drivers capture patterns like diet composition, eating regularity, physical activity, sleep tendencies, and substance use. These inputs shape energy, recovery capacity, and day-to-day variability. Small changes here often accumulate into large effects over time.

Metabolic drivers reflect how the body is responding internally. This includes markers such as blood pressure, cholesterol profiles, glucose control, inflammation, iron status, and kidney-related measures. These signals help explain whether behaviors are being tolerated, compensated for, or contributing to downstream strain.

In the FPI, drivers are treated as levers. By comparing individuals to similar baseline profiles, the system can identify which drivers are most strongly associated with improved or worsened outcomes in that specific context.

The final layer reflects how context and drivers combine to shape how someone feels and functions every day.

Outcomes: What Poor Daily Function Feels Like

Outcomes capture how health is experienced in daily life. They reflect the downstream impact of context and drivers rather than underlying mechanisms.

Within the FPI, outcomes are interpreted as signals. They capture what the system feels like once multiple influences interact over time. The six core outcomes are:

• Energy and Vitality. Captures how much fuel someone has to get through the day, and how consistently that energy holds.

• Mood and Mental Wellbeing. Reflects emotional stability and the psychological burden of managing chronic health day to day.

• Stress and Anxiety. Tracks the cognitive and physiological stress load shaped by behavioral patterns and health status.

• Focus and Cognition. Captures concentration, mental clarity, and the ability to think and perform under normal daily demands.

• Physical Capacity. Reflects physical limitation, mobility, and the ability to function without pain or fatigue as a barrier.

• Overall Health. A composite signal of how well someone feels across all dimensions relative to similar profiles.

Together these six outcomes reflect how well or poorly the body is functioning across the dimensions that matter most for daily life and long-term performance. Outcomes are compared across similar profiles to identify where function is most impaired and which drivers are most likely to shift things in a better direction.

Outcomes do not explain poor daily function. They show where it is most apparent and where change is likely to matter most.

Putting the Layers Together

The Functional Performance Index is built on a simple premise: daily function cannot be captured by a single metric.

Context establishes the baseline a person starts from. Drivers introduce change within that baseline. Outcomes reflect how those forces are experienced in daily life. Each layer serves a different purpose and plays a unique role in personalizing behavioral guidance.

Keeping these layers separate prevents common misinterpretations. Symptoms are not assumed to explain themselves. Behaviors are not evaluated without reference to baseline physiology. Comparisons are drawn from similar profiles rather than the population at large.

The goal is not to reduce chronic disease to simplicity, but to structure its complexity in a way that preserves meaning. Better daily function emerges from understanding how context, drivers, and outcomes interact over time.

This layered approach allows the FPI to translate population-level data into insights that remain interpretable at the individual level.

From a Data Perspective

The FPI is being developed using data drawn from 37 distinct datasets, together tracking over 1,000 parameters across context, drivers, and outcomes.

This scale makes it possible to move beyond broad averages and look at chronic disease and daily function in more specific situations. Instead of asking what tends to matter in general, the system can ask what has tended to matter for people with similar starting points, constraints, and health backgrounds.

The output is not a medical diagnosis, but a clearer sense of where attention is most likely to be useful. In practice, this means someone can better understand how their functional outcomes compare to similar profiles, and which drivers are most commonly associated with improvement in those contexts.

While this approach remains flexible, defining the framework upfront is a critical step in any data science or product development process. It sets realistic expectations for what the system can support and creates a foundation for thoughtful iteration rather than reactive adjustment.

The aim is to build a tool that helps people make sense of their daily function with more context and less guesswork. One that organizes complexity in a way that feels practical, interpretable, and honest about what the data can support as the system continues to evolve.

January may not transform everything overnight, but it does offer a moment to step back and take stock. Health, systems, and long-term progress unfold gradually, and the start of a new year can help clarify where that progress is heading.

This post is about that direction. What Digestive Data is becoming, what it is building toward, and how this project will evolve in the year to come.

The Vision of Digestive Data

Digestive Data began as a mix of research based writing and small, focused data projects aimed at answering specific questions about digestive health.

After completing more than ten of these projects alongside the research posts, one challenge became increasingly clear: digestive health data is deeply fragmented, and that fragmentation extends across chronic disease broadly.

Symptoms are tracked separately from behaviors. Outcomes are disconnected from the factors that influence them. Even well designed studies often isolate individual variables, while real digestive health is shaped by many interacting forces over time.

The complexity is not the problem. The lack of integration is.

That problem, which I never grew up even being aware of, became a focal point of my life at eighteen when I lost my colon. Over the following fourteen months, I underwent four surgeries to create and reconnect my J pouch, allowing me to live without an ostomy.

Throughout recovery, I was given a large amount of advice by incredibly skilled doctors and surgeons. Much of it was thoughtful and well intentioned, but I began to notice that it was often contradictory or rooted in general solutions applied to a very specific situation. This is not a failure of care. It reflects how difficult digestive health and chronic illness is to understand on a truly personal level.

Almost five years later, I started Digestive Data with the goal of providing data backed clarity to an issue that is incredibly common, often overgeneralized, and rarely personalized.

Now, that goal is even clearer: build the kind of tool my eighteen year old self wishes I had. One that provides personalized, data-driven guidance to people managing chronic disease without asking them to sacrifice their quality of life.

That vision calls for a different framework than what currently exists.

Introducing the Functional Performance Index

The Functional Performance Index, or FPI, is a data-driven framework built to identify which behavioral inputs most drive how someone feels and performs every day. It uses population-level data from NHANES, the National Health and Nutrition Examination Survey, to rank the specific changes most likely to move the needle for a given individual.

The Data Foundation

NHANES includes data on over 11,000 participants and tracks thousands of parameters across nutrition, biomarkers, lifestyle behaviors, environmental factors, and health outcomes. Its breadth makes it one of the most powerful foundations available for understanding how daily behaviors connect to functional health outcomes at a population level.

Personalization comes not only from tracking one individual over time, but from situating that individual within a large set of similar profiles and patterns.

What FPI Is (and What It Is Not)

The Functional Performance Index is designed to translate complex behavioral and health data into a clear, customized picture of which habits are most driving how someone feels and functions. More specifically, it is built to be:

• A behavioral signal grounded in population-level patterns

• A personalized ranking of the inputs most driving individual outcomes

• A practical guide to which changes will move the needle most

The output will be a ranked set of behavioral recommendations accompanied by individualized guidance. By situating an individual within a large set of similar profiles in the data, the FPI can identify which adjustments are most likely to improve daily function and overall wellbeing.

The FPI is not medical advice, not a diagnosis, and not a replacement for clinicians. The primary focus is to support better understanding and decision-making, giving people the kind of personalized behavioral clarity that conventional care rarely provides.

What to Expect

Research-based posts will continue on a regular cadence, focused on emerging findings in chronic disease, digestive health, and the factors that shape daily function.

The standalone data projects have been replaced by FPI updates, shared every two weeks. They will document progress, patterns, and insights as the Functional Performance Index is developed and refined, covering both the methodology and the findings as they emerge.

These posts are guided by the research but built for anyone who wants to understand not just what the data shows, but how a tool like this gets built from the ground up.

From a Data Perspective

Working with real-world health data is inherently iterative.

As the FPI develops, specific approaches may change based on what proves most informative, what holds real predictive value, and what people actually find useful in understanding their daily function. The commitment is not to a fixed method, but to building something that works and being transparent about how it gets there.

That balance between technical rigor and real-world relevance will continue to evolve as the FPI takes shape.

Looking Ahead

I’m grateful for where Digestive Data is today and genuinely excited about where it’s heading.

The vision for 2026 is to build a tool that treats chronic disease management as personal, behavioral, and worth understanding deeply. One that respects complexity without becoming inaccessible, and that has the potential to give people something the current system rarely offers: a clear, personalized picture of which changes will most improve how they feel and function every day.

This is the start of that next phase, and I truly appreciate your support as Digestive Data continues to evolve.