Long-Context Open-Ended Sleep Report Generation: Challenges and Training Solutions

1. Introduction

Imagine a Monday morning. When his alarm rings, John’s first thought is, “I slept enough last night. Why am I still so tired?”

He opens his Speediance sleep history. During the past month, work kept him to five or six hours of sleep on several nights, while he slept for nine hours on some weekends. His total sleep duration sometimes looked normal, but repeated awakenings interrupted the night. Travel, evening workouts, caffeine, and social events also appeared across the month. The data records every night clearly, yet dozens of metrics do not tell him which pattern matters most or which one thing he should change tonight.

This is the hardest and most useful part of automatic sleep report generation. A model must find important patterns in long, multi-dimensional data, use the person’s context to explain possible causes, and turn a complex judgment into safe and practical advice. We study long-context open-ended sleep report generation so that a model can read the numbers accurately, connect the evidence, and state the next actions clearly.

2. Problem definition

The task is to generate a personalized, open-ended sleep report from structured sleep data. “Open-ended” means that the model does not select an answer from fixed choices. It must organize the evidence, explanations, and recommendations itself.

2.1 Input

Data category	Main content	Examples
Daily user data	Demographics, nightly sleep logs, and differences between workday and rest-day schedules	Bedtime and wake time, sleep-onset latency, nighttime awakenings, wake after sleep onset, and nap records
Aggregated statistics	Summary features calculated from multiple days	Average sleep duration, nights below 7 hours, weekend wake-time shift, sleep-midpoint shift, sleep efficiency, awakenings, and naps
Optional user context	Life information that may help explain sleep changes	Alcohol and caffeine intake, daily routines, training schedules, travel, and important events

WASO means wake after sleep onset. It is the total time spent awake between first falling asleep and the final awakening.

2.2 Output

Report field	Goal	Quality requirement
Insight	Identify the most important sleep patterns	Cite accurate data and separate isolated events from sustained patterns
Etiology	Explain possible causes	Combine several metrics and use appropriately qualified causal language
Recommendation	Propose the next actions	Keep them safe, specific, directly grounded in the data, and focused on the top 1–2 priorities

3. Task challenges

3.1 Precise extraction from long-context tables

The task looks like a conversion from data to a report, but it requires several layers of ability. An input can contain daily logs spanning days or months, along with many statistics derived from those logs. The model must locate and extract the right facts from this long context.

Consider counting. A prompt may ask, “How many of the past N nights had less than 7 hours of sleep?” The model only needs to inspect each night and add one to a count, yet a frontier closed-source model, GPT-5.5, reached about 87% accuracy on this task in our tests. Common errors included skipping dates, mishandling a boundary such as exactly 7 hours, and ignoring records near the beginning of a long input.

Code can calculate these counts and insert them into the prompt, but this approach has three basic limits:

1. It cannot cover every extraction need. Precise extraction also includes extrema in continuous values, trend changes, and conditions that combine several fields. We cannot write a rule for every possible request.
2. It generalizes poorly. Real applications can change the data schema, field definitions, and criteria for an abnormal value. Hard-coded rules do not adapt well to these changes.
3. It breaks the reasoning chain. Counting is rarely the final goal. The model may also need to decide whether several short nights form a stable pattern and connect that pattern to possible causes and actions.

If rules handle all key feature extraction, the model cannot build those intermediate representations in its own reasoning. This limits the depth and quality of the later analysis.

3.2 Open-ended reasoning and precise attribution

Finding the important sleep problems in raw data is more than a lookup task. The model must combine evidence from several dimensions instead of relying on one metric.

A shallow model often uses a one-step causal link. It may see low sleep efficiency and immediately recommend an earlier bedtime. It may see short sleep duration and simply recommend more sleep. These suggestions often fail in real settings.

Sleep efficiency shows why:

$$\text{Sleep efficiency}=\frac{\text{total sleep time}}{\text{total time in bed}}$$

Case	Data pattern	More likely central issue
A	4.5 hours of sleep with a normal time in bed	Insufficient sleep duration
B	7.5 hours of sleep during 10 hours in bed	Possible fragmentation, repeated awakenings, or a high share of light sleep

If a model sees only low efficiency and recommends more sleep, the advice will not help in Case B. It may even encourage the user to stay in bed longer and make fragmentation worse.

A high-quality report needs three abilities:

1. Cross-check several metrics. Combine duration, efficiency, awakening frequency, WASO, and related measures to identify the most plausible source of low efficiency.
2. Build a qualified causal chain. Trace an observed pattern to a possible intermediate mechanism, such as fragmentation, limited sleep opportunity, or difficulty falling asleep, then propose a matching action. The report should describe evidence for a possible cause and should not turn correlation into a confirmed medical diagnosis.
3. Make a personal tradeoff. Prioritize the one or two actions that matter most instead of listing every possible recommendation.

4. Training challenges

4.1 Producing high-quality, knowledge-grounded SFT teacher data

SFT means supervised fine-tuning. It teaches the target model to map an input to a high-quality example answer. Directly using raw answers from a closed-source teacher model creates two problems.

First, the teacher may apply domain knowledge inconsistently. It may vary in how it defines insufficient sleep or in which factors it says can affect a metric.

Second, reasoning depth is unstable. Some teacher examples in our experiments had strong reasoning paths, but that quality did not appear reliably. Uncertain concept boundaries also make the reasoning hard to audit. For example, a rigorous medical rule may use 7 hours as a threshold for insufficient sleep, while teacher models may classify a value such as 7 hours and 10 minutes differently. If the concept judgment changes from sample to sample, the later reasoning cannot stay rigorous.

4.2 Designing useful rewards for open-ended reports

Our training pipeline starts with SFT, continues with preference-based weighted DPOP, and ends with GRPO reinforcement learning.

Mathematics, code, and multi-step agent tasks often have clear intermediate steps that an evaluator can verify. An open-ended report has no single required writing order, and different parts of one report can have different quality. If the whole report receives one reward, every token in the insight, etiology, and recommendation fields shares the same signal. This global reward performed poorly in our early experiments.

The central question is how to send feedback to the number, span, sentence, or section that is wrong without damaging content that is already correct.

5. Solution overview

Our approach links knowledge construction, teacher-data generation, and student-model training.

1. Extract a reasoning graph from sleep references. The graph connects observations, patterns, hypotheses, safety limits, and recommendations through paths that can be checked.
2. Activate the concept nodes that match the user’s metrics and place the selected reasoning paths in the teacher prompt. The teacher then follows a draft–critique–revision loop to produce training examples.
3. Train the student model in stages. SFT learns from strong examples, weighted DPOP learns a preference for better answers, and fine-grained GRPO directly optimizes reports sampled from the current model.

6. Generating SFT teacher examples with a reasoning graph

We first extract concepts and relations from sleep-domain references. A path can start with repeated nighttime awakenings, connect them to a pattern of fragmented sleep, move to several data-constrained causal hypotheses, and end with matching recommendations. A safety node prevents the model from presenting limited evidence as a confirmed diagnosis.

When the user's metrics meet predefined conditions, the system activates the initial concepts and then the related downstream nodes. The teacher receives reasoning paths that apply to this user, rather than a general list of sleep facts. It then checks the numbers, reasoning, and advice through several rounds of self-reflection and revises the draft. This process makes the teacher examples more stable and the main judgments easier to audit.

7. Localizing preference feedback with weighted DPOP

SFT learns the conditional distribution of teacher answers, but the trained model can still write the wrong number or copy the teacher’s broad style without preserving precise details. We therefore introduce weighted DPOP, a variant of direct preference optimization (DPO) with a positive-example constraint. Its main rule is simple: text that the teacher actually changed should receive a stronger learning weight, while unrelated tokens should not receive the same feedback.

A report is usually long, and its three fields serve different purposes. insight describes patterns, etiology explains possible causes, and recommendation proposes actions. One field can also discuss several topics, such as sleep timing, duration, and a medical threshold. One global reward dilutes the local signal and can penalize sentences that are already correct.

7.1 Preference pairs and multi-level weights

We select an error type before constructing each chosen–rejected pair. The types include numeric perturbation, full teacher revision, injected reasoning errors, and format or safety errors. A rule uses the error type to select a mask. The system then assigns weights at the token, span, sentence, and section levels.

For preference example $i$, we define:

$$z_i=(x_i,y_i^w,y_i^r,\tau_i,\mathcal{S}_i,a_i)$$

Here, $x_i$ is the input prompt; $y_i^w$ and $y_i^r$ are the chosen and rejected answers; $\tau_i$ is the pair type; $\mathcal{S}_i$ is the set of spans changed by the teacher; and $a_i$ is the sample-level weight.

The weight of token $j$ is:

$$\tilde{m}_{ij}=\max_{s\in\mathcal{S}_i}\left\{ m_{\text{base}}, m_{\text{section}}\mathbf{1}\{j\in\text{section}(s)\}, m_{\text{sent}}\mathbf{1}\{j\in\text{sentence}(s)\}, b(c(s))\mathbf{1}\{j\in\text{span}(s)\} \right\}$$

A token can belong to several levels at the same time. The system uses the highest applicable weight.

Level	Scope	Weight	Purpose
Background	Every response token	0.05	Keep a minimal global learning signal
Same section	The same insight, etiology, or recommendation field as the changed span	0.25	Focus on the same semantic field
Same sentence	The sentence that contains the changed span	0.60	Strengthen the local context
Changed span	Text directly marked by the teacher	Low 0.70; medium 0.85; high 1.00	Concentrate learning on the specific error

The sample-level weight $a_i$ uses the highest error severity in the pair:

Severity	$a_i$	Typical issue
High	1.20	Wrong numeric fact; medical or safety overclaim; wrong main causal explanation; clearly wrong recommendation direction
Medium	1.00	Missing key metric; vague advice; missing time bound; important but non-critical reasoning omission
Low	0.80	Wording, structure, or minor completeness issue

For example, a rejected answer says “18 nights below 7 hours” and “this proves that insomnia drives the pattern.” The chosen answer says “13 nights” and “this could reflect limited sleep opportunity and schedule drift.” Both changed spans have high severity, so $a_i=1.2$.

The numeric tokens “13” and “18” each receive 1.00. Other tokens in the same sentence receive 0.60. Unrelated content in the same field receives 0.25, and sentences in an unchanged field receive only 0.05. A numeric_perturbation pair uses an even narrower mask: only the number that changed is upweighted, while other words in the sentence are not penalized.

7.2 Objective and positive-example constraint

The weighted DPOP objective is:

$$\ell_i^{\text{DPOP}} =a_i\left[ -\log\sigma\left(\beta(\Delta_{\pi,i}-\Delta_{\text{ref},i})\right) +\lambda\left[\ell_{\text{ref}}(y_i^w\mid x_i)-\ell_{\pi}(y_i^w\mid x_i)\right]_+ \right]$$

$\Delta_{\pi,i}$ and $\Delta_{\text{ref},i}$ are the policy and reference models’ weighted log-probability differences between the chosen and rejected answers. The first term raises the policy model’s preference for the better answer and lowers its preference for the worse answer.

The token weights enter through the weighted log probability:

$$\log\pi(y)=\sum_t\tilde{m}_t\log\pi(y_t\mid\cdot)$$

The second term is the positive-example constraint, where $[u]_+=\max(u,0)$. We observed a degenerate pattern in which the log probabilities of both the chosen and rejected answers decreased, while the rejected answer decreased more. The preference margin improved, but the model also became less confident in the correct answer. This constraint becomes active when the policy model is less confident in the chosen answer than the reference model, which blocks that negative direction.

7.3 DPOP-stage results

Training stage	Insight	Etiology	Recommendation	Mean
SFT starting point	4.5717	4.4717	4.6733	4.5722
DPOP, 100 steps	4.6017	4.5800	4.7083	4.6300

8. Optimizing current-model outputs with fine-grained GRPO

DPOP is an off-policy method, so it does not directly optimize outputs just sampled from the current policy. It also depends on preference data that was already constructed. DPOP improved the report, but a small gap from GPT remained across the scoring criteria. We therefore continued from the DPOP checkpoint with on-policy training. The current model generates a report, and a teacher-critic model returns local revisions and rewards.

Our early GRPO experiments gave one score to each whole report. This approach performed poorly on long, open-ended reports, and the loss was unstable. We replaced it with a local design similar to weighted DPOP. The evaluator marks good spans, bad spans, and missing obligations, then divides each span’s total reward across its tokens.

GRPO means group relative policy optimization. Here it uses several outputs sampled from the current policy and learns from their localized reward signals.

8.1 Missing-obligation penalty

missing_obligations marks required content that the report omitted. Ideally, the system would assign a reward at the location where the content should appear. Missing content has no tokens, so there is no direct span to target. We use a compromise and distribute the penalty near the response boundary and final sentence.

$$r_t^{\text{miss}} =-\sum_m\alpha_m v(c_m) \left[q_t^{D(m)}+\gamma q_t^{L(m)}\right]$$

Here, $r_t^{\text{miss}}$ is the penalty assigned to token $t$ for an omitted requirement. The coefficient $\alpha_m$ is the base penalty strength and is currently 0.85. The teacher-critic model assigns the omission severity $c_m$, and $v(c_m)$ maps it to a number.

Severity$c_m$	$v(c_m)$	Relative penalty strength
Low	0.35	Weakest
Medium	0.70	Medium
High	1.00	Strong
Critical	1.20	Strongest

$q_t^{D(m)}$ is token $t$’s share of the penalty from the response-boundary region. It is 0 outside the region, and all shares inside the region sum to 1. The term $q_t^{L(m)}$ applies the same idea to the final-sentence region, and $\gamma$ controls its relative strength.

8.2 Token-level training objective

With token-level rewards, the final objective is:

$$\mathcal{L} =-\frac{\sum_{k,t}r_{k,t}\log\pi_\theta(y_{k,t}\mid\cdot)} {\sum_{k,t}|r_{k,t}|+\epsilon}$$

A positive reward raises the generation probability of the related tokens, while a negative reward lowers it. The denominator normalizes by the absolute reward magnitude, which makes reward scales easier to compare across samples.

9. Final results

We compare the SFT checkpoint, the model trained for 100 DPOP steps from that checkpoint, the fine-grained GRPO model, and GPT-5.5 base. The original run name pre-CGD ckpt200 is the SFT starting point, DPOP100 is the 100-step DPOP model, and No-anchor GRPO ckpt65 is the fine-grained GRPO model. Other prefixes in the run names came from experiment settings and do not describe separate methods.

The evaluation uses two input conditions:

• Without injected counting facts: The model reads the raw records and performs the calculation itself.
• With injected counting facts: The prompt also supplies manually calculated facts such as “X of XX nights had less than 7 hours of sleep.” In another test setting without chain-of-thought prompting or tools, GPT-5.5 reached about 93% counting accuracy, so we evaluated this condition separately.

9.1 Without injected counting facts

Model stage	Insight	Etiology	Recommendation	Mean
SFT (pre-CGD ckpt200)	4.5717	4.4717	4.6733	4.5722
Weighted DPOP (DPOP100)	4.6017	4.5800	4.7083	4.6300
Fine-grained GRPO (No-anchor GRPO ckpt65)	4.6267	4.6400	4.7067	4.6578
GPT-5.5 base	—	—	—	—

9.2 With injected counting facts

Model stage	Insight	Etiology	Recommendation	Mean
SFT (pre-CGD ckpt200)	4.6533	4.5950	4.7133	4.6539
Weighted DPOP (DPOP100)	4.6883	4.7167	4.7417	4.7156
Fine-grained GRPO (No-anchor GRPO ckpt65)	4.7733	4.7400	4.7950	4.7694
GPT-5.5 base	4.7367	4.6967	4.8017	4.7450

GPT-5.5 scored all values in these tables. The three report fields and the mean generally improve from SFT to weighted DPOP and then to fine-grained GRPO. With injected counting facts, fine-grained GRPO reaches a mean of 4.7694, above the GPT-5.5 base score of 4.7450. GPT-5.5 base remains slightly higher on the recommendation score.

We also ran cross-model scoring with GPT-5.4 and observed a larger advantage. An internal API issue prevented us from completing the full autoeval_model=gpt5.4, gen_model=5.5 base setting. The incomplete setting cannot support a full conclusion, so we treat the trend only as an additional observation.

9.3 Detailed scores

The next two figures show AutoEval scores for different final sources at the report-field and scoring-principle levels. AutoEval means that an evaluator model assigns a score from 1 to 5 by following a predefined rubric.

10. Outlook

The goal of this research still comes back to an ordinary night. After reading the report, a user should understand why they have recently awakened early, why weekend catch-up sleep did not remove their fatigue, and which one action matters most tonight.

Future sleep reports must cover longer periods and connect more signals, including training load, stress, caffeine, travel, and major life events. We want the report to grow from a daily scorecard into a sleep assistant that follows personal changes, updates its judgment when new evidence appears, and respects safety limits. The value is not only in writing text that sounds like an expert analysis. It is also in helping a user understand the data and turn long-term health management into a clear next action.