diff --git a/docs/dev/adrs/suggestions/documentation-ci-build.md b/docs/dev/adrs/accepted/documentation-ci-build.md similarity index 67% rename from docs/dev/adrs/suggestions/documentation-ci-build.md rename to docs/dev/adrs/accepted/documentation-ci-build.md index f3a858832..e7e0e8885 100644 --- a/docs/dev/adrs/suggestions/documentation-ci-build.md +++ b/docs/dev/adrs/accepted/documentation-ci-build.md @@ -1,6 +1,6 @@ # ADR: Documentation CI and Build Verification -**Status:** Proposed +**Status:** Accepted **Date:** 2026-05-31 ## Group @@ -77,6 +77,20 @@ Use `codespell` first for low-noise spelling checks. Consider `Vale` after the project has a small EasyDiffraction style vocabulary and an allowlist for crystallographic terms, package names, and CIF tags. +## Implementation Status + +Most of this ADR is already in place; it is accepted to record the +chosen direction and to track the remaining gaps. + +| # | Decision | Status | +| --- | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 | MkDocs `--strict` build | **Done** — `docs-build` pixi task runs `mkdocs build --strict`; wired into the `lint-format.yml` "docs strict build" gate and the `docs.yml` deploy workflow. | +| 2 | `mkdocstrings` for API pages | **Done** — `mkdocstrings` + `mkdocstrings-python` configured in `docs/mkdocs.yml` (handler `paths: ['src']`); the `api-reference/*.md` pages use `:::` directives. | +| 3 | Snippet smoke tests | **Not done** — no task imports or executes the user-facing snippets in `quick-reference/`, `user-guide/first-steps.md`, or `user-guide/analysis-workflow/*.md`. Highest-value remaining gap. | +| 4 | Tutorial freshness check | **Partial** — `notebook-prepare` plus `notebook-tests`/`notebook-exec-ci` exist, but no no-write task asserts that `notebook-prepare` leaves the committed `.ipynb` unchanged. | +| 5 | `lychee` link checking | **Done for local/relative links** — `link-check` pixi task (config in `lychee.toml`) is wired into `lint-format.yml`. External-URL checking is deferred (see issue 114). | +| 6 | `codespell`, then `Vale` | **codespell done** — `spell-check` pixi task wired into `lint-format.yml`. `Vale` deferred. | + ## Options Considered ### MkDocs strict build @@ -179,11 +193,14 @@ Cons: ## Deferred Work -- Decide whether link checking runs on every pull request, nightly, or - both. -- Decide whether snippet smoke tests extract fenced code blocks +- Add snippet smoke tests for user-facing examples (decision 3). Tracked + by the `documentation-snippet-tests` implementation plan. Open + question carried into that plan: extract fenced code blocks automatically or rely on explicitly named snippets. -- Decide whether docs CI should build only source Markdown or also build - rendered notebooks. -- Add the chosen checks to `pixi.toml`, CI configuration, and developer - documentation after this ADR is accepted. +- Add a no-write `notebook-prepare-check` task that fails CI when the + committed notebooks are out of date with their `.py` sources (decision + 4). +- Enable external-URL link checking in the docs gate (decision 5), + scheduled or cached to avoid flakiness. Tracked by issue 114. +- Adopt `Vale` prose linting once an EasyDiffraction style vocabulary + and crystallographic-term allowlist exist (decision 6). diff --git a/docs/dev/adrs/accepted/plotting-docs-performance.md b/docs/dev/adrs/accepted/plotting-docs-performance.md index 3d68101a4..d6a9109fd 100644 --- a/docs/dev/adrs/accepted/plotting-docs-performance.md +++ b/docs/dev/adrs/accepted/plotting-docs-performance.md @@ -450,7 +450,7 @@ Settled in discussion on 2026-06-02: payload + faster draw) — a separate, data-side optimization. - A docs CI budget check (page weight / figure count) to catch regressions, aligning with - [`documentation-ci-build.md`](../suggestions/documentation-ci-build.md). + [`documentation-ci-build.md`](documentation-ci-build.md). - Hoist a single importmap into the **report** template `
` for standalone reports that render multiple Three.js scenes (the same per-scene-importmap bug as docs, but governed by diff --git a/docs/dev/adrs/accepted/test-suite-and-validation.md b/docs/dev/adrs/accepted/test-suite-and-validation.md index b35aea302..6d0744057 100644 --- a/docs/dev/adrs/accepted/test-suite-and-validation.md +++ b/docs/dev/adrs/accepted/test-suite-and-validation.md @@ -66,7 +66,7 @@ accumulated: slow and therefore runs on pull requests only. There is no fast, every-push check that the site builds strictly, links resolve, and prose is clean. This overlaps the unimplemented - [Documentation CI and Build Verification](../suggestions/documentation-ci-build.md) + [Documentation CI and Build Verification](documentation-ci-build.md) suggestion. This ADR amends [Test Strategy](../accepted/test-strategy.md): the @@ -338,7 +338,7 @@ prompt drift feedback. It is deliberately **separate from `docs.yml`**, which executes all tutorials and then builds and deploys — slow, and therefore pull-request-only. The detailed catalogue of documentation checks is owned by -[Documentation CI and Build Verification](../suggestions/documentation-ci-build.md), +[Documentation CI and Build Verification](documentation-ci-build.md), which this ADR coordinates with: that ADR defines _what_ the checks are; this ADR's decision is that the cheap, deterministic subset runs as part of the every-push test workflow. Promoting that ADR is part of this @@ -443,15 +443,15 @@ drafting conversation, per the dependency-approval rule): - `pytest-benchmark` — performance-regression benchmarks (§7). Coordinated with -[Documentation CI and Build Verification](../suggestions/documentation-ci-build.md), +[Documentation CI and Build Verification](documentation-ci-build.md), which carries the documentation-check tools (`codespell`, a link checker such as `lychee`, and later `Vale`) used by §9. ## Related ADRs - [Test Strategy](../accepted/test-strategy.md) — amended by this ADR. -- [Documentation CI and Build Verification](../suggestions/documentation-ci-build.md) - — coordinated with §9. +- [Documentation CI and Build Verification](documentation-ci-build.md) — + coordinated with §9. - [Lint Complexity Thresholds](../accepted/lint-complexity-thresholds.md) — sibling Quality guardrail. - [Notebook Generation Source of Truth](../accepted/notebook-generation.md) diff --git a/docs/dev/adrs/index.md b/docs/dev/adrs/index.md index 17e2d85a1..b4e98e0d9 100644 --- a/docs/dev/adrs/index.md +++ b/docs/dev/adrs/index.md @@ -33,7 +33,7 @@ folders. | Documentation | Accepted | Help Method Discoverability | Requires primary public objects and facades to expose consistent `help()` output. | [`help-discoverability.md`](accepted/help-discoverability.md) | | Documentation | Accepted | Notebook Generation Source of Truth | Treats tutorial `.py` files as editable sources and notebooks as generated artifacts. | [`notebook-generation.md`](accepted/notebook-generation.md) | | Documentation | Accepted | Plotting & Docs Performance for Interactive Figures | Self-hosts a lazy, shared figure runtime so docs pages load fast and progressively while staying interactive. | [`plotting-docs-performance.md`](accepted/plotting-docs-performance.md) | -| Documentation | Suggestion | Documentation CI and Build Verification | Proposes strict MkDocs builds, API-derived docs, snippet smoke tests, link checks, and prose/spelling checks. | [`documentation-ci-build.md`](suggestions/documentation-ci-build.md) | +| Documentation | Accepted | Documentation CI and Build Verification | Strict MkDocs builds, API-derived docs, snippet smoke tests, link checks, and prose/spelling checks. | [`documentation-ci-build.md`](accepted/documentation-ci-build.md) | | Experiment model | Accepted | Immutable Experiment Type | Makes experiment type axes creation-time state rather than mutable runtime state. | [`immutable-experiment-type.md`](accepted/immutable-experiment-type.md) | | Experiment model | Accepted | Automatic Line-Segment Background Estimation | Detects line-segment background control points from the measured pattern, peak-insensitive and editable. | [`background-auto-estimate.md`](accepted/background-auto-estimate.md) | | Experiment model | Accepted | Calculation Without Measured Data | Adds a writable `data_range` category so a structure-only experiment is calculable and plottable without loaded data. | [`calculation-without-measured-data.md`](accepted/calculation-without-measured-data.md) | diff --git a/docs/dev/adrs/suggestions/model-sample-absorption.md b/docs/dev/adrs/suggestions/model-sample-absorption.md new file mode 100644 index 000000000..838e71986 --- /dev/null +++ b/docs/dev/adrs/suggestions/model-sample-absorption.md @@ -0,0 +1,377 @@ +# ADR: Model Sample Absorption (Debye–Scherrer, μR) + +## Status + +Proposed. + +## Date + +2026-06-12 + +## Group + +Experiment model. + +## Context + +This ADR follows the conventions in +[`AGENTS.md`](../../../../AGENTS.md). + +The calculators (`cryspy`, `crysfml`) currently apply **no** +sample-absorption correction. For a cylindrical sample in Debye–Scherrer +geometry the transmission through the sample is an angle-dependent +factor that attenuates low-angle peaks more than high-angle peaks; +omitting it leaves an angle-dependent intensity residual that scales +with the sample's μR (linear absorption coefficient × radius). + +This is not hypothetical. The verification reference +`pd-neut-cwl_tch-fcj_lab6` was refined in FullProf with **μR = 0.7**; +the unmodelled correction is the _entire_ intensity residual on the +companion `pd-neut-cwl_tch-fcj_abs_lab6` page (≈5 % profile difference), +while the μR = 0 page passes to corr 0.9999. See +[issue #119](../../issues/open.md). + +### What the three reference sources provide + +**FullProf** splits absorption into a refineable **magnitude** and a +**correction type**, on two different axes: + +- **CW (and symmetric θ–2θ flat plate):** `muR` on the `.pcr` Lambda + line — a single value (μ·R). In the LaB₆ reference it is **fixed** + (the Lambda line carries no refinement codeword), confirming that + absorption is typically entered as a known constant, not refined. A + `2nd-muR` field on the same line models a second coaxial cylinder (the + sample container / capillary wall); `Cthm` and `Rpolarz` on that line + are **polarization**, not absorption, and are out of scope here. +- **TOF:** `Iabscor` selects the correction _form_ — `1` flat plate ⟂ + incident beam, `2` cylindrical, `3` exponential `A = exp(−ABS·λᶜ)`. + TOF absorption is wavelength-dependent, not a pure function of 2θ. + +**CrysFML08** (`Src/CFML_Powder/Pow_Lorentz_Absorption.f90`) already +implements the CW formulas in Fortran: + +- `Lorentz_abs_CW(sinth, costh, postt, tmv, …, ilor, cabs, …)` with + `tmv = μR`, `ilor` = geometry (`DBS`, `BB`, `TBG`, `TFX`, `FILMS`…) + and `cabs ∈ {HEWAT, LOBANOV}`. +- `Powder_Lorentz_IntegInt_CW(…, muR, …)` — the bare Hewat form. + +**However**, these routines are **not wrapped** in CrysFML's +`PythonAPI/`, and the high-level entry our backend actually calls +(`cw_powder_pattern_from_dict`) computes a plain Lorentz factor +`0.5/(sin²θ·cosθ)` with no absorption term. So the issue's claim that +absorption is "reachable via `Lorentz_abs_CW` through pycrysfml" is +**not true today** — it would require upstream wrapping or an upstream +call-site change we do not control. + +**cryspy** has **no absorption code at all** (only Debye–Waller and +sphere _extinction_, which are different physics). CW intensity is +assembled in `procedure_rhochi/rhochi_pd.py` as +`0.5 · scale · Lorentz(θ) · |F|² · mult`, with no slot for an A(θ) +factor. + +### Consequence of the source survey + +Neither backend can apply the correction internally without changes we +do not own. The only way to get **identical** results across both +calculators is to compute A(θ) **ourselves in EasyDiffraction** and +apply it to the calculated pattern. This makes absorption a +**calculator-independent** correction — unlike `extinction`, which is +threaded into cryspy's own dict and is therefore `cryspy`-only. + +### CIF dictionary support (`tmp/iucr-dicts`) + +There is **no** standard data name for μR or for the Hewat coefficient. +The standard items cover the _physical provenance_ only: + +| Quantity | Standard CIF item | Units | +| ------------------------- | ------------------------------------------------------------- | ----- | +| Linear absorption μ | `_exptl_absorpt.coefficient_mu`, `_pd_char.atten_coef_mu_*` | mm⁻¹ | +| Sample radius / thickness | `_pd_spec.size_axial/_equat/_thick` | mm | +| Sample shape | `_pd_spec.shape` ∈ {`cylinder`, `flat_sheet`, `irregular`} | code | +| Correction type | `_exptl_absorpt.correction_type` (incl. `cylinder`, `sphere`) | code | +| Beam path | `_pd_spec.mount_mode` ∈ {`reflection`, `transmission`} | code | + +So the refineable μR itself needs a **project-namespaced** +(`_easydiffraction_absorption.*`) tag, with the standard items available +later as optional provenance (see Deferred Work). + +### Evidence from the FullProf example suite + +A survey of all 68 `.pcr` files shipped with FullProf (`Examples/`) +shows that **every** absorption example — CW and TOF — is **cylindrical +(Debye–Scherrer)**; not one uses flat-plate or exponential absorption, +and the container term is never used: + +- **CW (cylindrical `muR`):** `dy*`, `DyMnGe*`, `cuf1k`, `hocu`, + `si3n4r`, `sin_3t2` — μR ∈ {0.068, 0.15, 0.40, 1.28}, all **fixed** + (no refinement codeword on the Lambda line). Note μR reaches **1.28**, + slightly past Hewat's nominal ≈1.0 validity — the practical motivation + for the Lobanov form later. +- **TOF (`Iabscor`):** `Ceo2_PEARL`, `nac-osiris(n)`, `hrpd`, `arg_si`, + `cecual`, `cecoal`, `lamn_pol` — **all `Iabscor = 2` (cylindrical)**, + with `ABSCOR1` non-zero and **refined** (non-zero codewords). +- **Never observed:** `Iabscor = 1` (flat plate), `Iabscor = 3` + (exponential), or a non-zero `2nd-muR` (container) in any file. + +Two design consequences: + +1. **Ship a single cylindrical type now.** The cylinder is the only + geometry the reference toolchain actually exercises, so Phase 1 + builds `none` + `cylinder-hewat` only; everything else becomes a + documented future extension (§Deferred Work) that plugs into the same + switchable category without rework. +2. **Refineable, default-fixed.** CW practice fixes `muR`; TOF practice + refines it. So `mu_r` is a normal refineable `Parameter` but ships + `free = False` (matching the CW reference and the Biso degeneracy), + not a constant. + +## Decision + +### 1. Add a switchable `absorption` category on the experiment + +Introduce `experiment.absorption`, mirroring `experiment.extinction`: a +`SwitchableCategoryBase` whose concrete classes are registered with an +`AbsorptionFactory`, gated by `Compatibility` and `CalculatorSupport`. +It follows +[`switchable-category-owned-selectors.md`](../accepted/switchable-category-owned-selectors.md): + +```python +experiment.absorption.type # writable selector (str) +experiment.absorption.show_supported() # supported types, active starred +``` + +The owner exposes only `experiment.absorption` and a private +`_swap_absorption` hook (Family A: the hook **replaces the category +instance** when `type` changes), exactly as `extinction` does today. + +### 2. Application point — a pointwise envelope on the calculated pattern + +Because A(θ) is a **slowly-varying smooth envelope** of sin²θ (its +fractional change across one peak FWHM is ≪ 1 for any realistic μR), +applying it per-reflection vs. pointwise to the summed pattern differs +only at second order in (FWHM · dA/dθ) — negligible. We therefore apply +it once, after pattern assembly, in **both** backends: + +``` +y_corrected(2θ_i) = A(θ_i) · y_calc(2θ_i) +``` + +This is a single shared helper +(`analysis/calculators/absorption.py::factor(two_theta, params)`) called +from the post-calculation step of both `cryspy.py` and `crysfml.py`. The +two backends thus stay bit-for-bit consistent on the absorption term, +and the helper is unit-testable in isolation against FullProf output +(validated to 4 decimals in the issue). + +### 3. Supported types — the taxonomy + +The `type` selector lists factory tags gated by `Compatibility` (sample +form, beam mode, scattering type, radiation) and `CalculatorSupport`. +**Phase 1 builds only the first two rows;** the rest are the planned +extension surface, designed here so they later plug into the same +category by registering a class (see Deferred Work). + +| Tag | Beam mode | Sample form | Radiation | Calculators | Parameters | Status | +| ------------------ | --------- | ----------- | ------------- | --------------- | --------------- | ----------- | +| `none` | any | any | neutron, xray | cryspy, crysfml | — | **Phase 1** | +| `cylinder-hewat` | CWL | powder | neutron, xray | cryspy, crysfml | `mu_r` | **Phase 1** | +| `cylinder-lobanov` | CWL | powder | neutron, xray | cryspy, crysfml | `mu_r` | future | +| `tof-cylinder` | TOF | powder | neutron | cryspy, crysfml | `mu_r` (λ-dep.) | future | +| `flat-plate` | CWL | powder | neutron, xray | cryspy, crysfml | `mu_t` | future | +| `tof-exponential` | TOF | powder | neutron | cryspy, crysfml | `coeff`, `exp` | future | + +Notes: + +- `none` is the **default** (A ≡ 1). The category always exists for + powder so the user can discover and switch it on via + `show_supported()`; opting out is `type = 'none'`, not deleting the + category. (`scattering_type = 'total'` / pdffit is out of scope.) +- **Why a single built type is correct now:** the FullProf example suite + uses only cylindrical absorption (§"Evidence from the FullProf example + suite"), and our sole verification reference is CW cylindrical (LaB₆, + μR = 0.7). Building `cylinder-hewat` alone closes the known gap with a + tested oracle; the others would ship untested physics. This also + respects [`AGENTS.md`](../../../../AGENTS.md) §Architecture ("don't + introduce abstractions before a concrete second use case") — but the + switchable-category contract is still required even for a single + implementation (as `extinction` is today with only `becker-coppens`), + which is what keeps the extension surface free. +- Radiation is **not** a discriminator for the cylindrical geometric + envelope — the Hewat/Lobanov constants depend on geometry, not on + neutron vs X-ray. X-ray simply tends to larger μ; the same formula + applies. Both are supported. +- The future rows are designed (tags, parameters, CIF) but **not + built**. TOF in particular is λ-dependent and the slow-envelope + argument does not carry over unchanged (each detector bin mixes + wavelengths), so it needs its own application path; it is listed so + the taxonomy and CIF tags are settled once. + +### 4. The μR parameter + +The single refineable/settable quantity is **`mu_r`** (μ·R), matching +FullProf's `muR` and CrysFML's `tmv`. It is a `Parameter` +(`RangeValidator(ge=0.0)`, default `0.0`), **free=False by default** +(absorption is normally fixed, per the LaB₆ reference). Storing μ and R +separately is rejected (see Alternatives); they can be added later as +read-only provenance (Deferred Work). + +`flat-plate` uses `mu_t` (μ·thickness); the TOF exponential form uses +`coeff` and `exp` (`A = exp(−coeff·λ^exp)`). + +### 5. Equations + +**Hewat** (cylinder, validated to 4 decimals vs FullProf; fit range μR ≲ +1.5): + +``` +A(θ) = exp( −(1.7133 − 0.0368·sin²θ)·μR + (0.0927 + 0.375·sin²θ)·μR² ) +``` + +**Lobanov–Alte da Veiga** (cylinder, extends to μR ≈ 10 via a branch at +μR = 3; `s ≡ sinθ`): + +``` +μR ≤ 3: + k1 = (25.99978 − 0.01911·s^0.25)·exp(−0.024514·s) + 0.109561·√s − 26.0456 + k2 = −0.02489 − 0.39499·s + 1.219077·s^1.5 − 1.31268·s² + 0.871081·s^2.5 − 0.2327·s³ + k3 = 0.003045 + 0.018167·s − 0.03305·s² + A = exp( −((k3·μR + k2)·μR + k1)·μR ) (normalised; k0 = 1.697653) + +μR > 3: + A = k7 + (k4 − k7) / (1 + k5·(μR − 3))^k6 (k4…k7 polynomials in s) +``` + +(Lobanov constants transcribed from CrysFML08 +`Pow_Lorentz_Absorption.f90`; the implementation will copy them verbatim +and unit-test against that source.) + +**Flat plate, symmetric θ–2θ** (μt = μ·thickness): + +``` +A(θ) = exp( −2·μt / sinθ ) # transmission, symmetric reflection +``` + +**TOF exponential** (deferred; per FullProf `Iabscor = 3`): + +``` +A(λ) = exp( −coeff · λ^exp ) +``` + +### 6. User-facing API + +```python +# Discover what is available for this experiment (Phase 1) +experiment.absorption.show_supported() +# -> none (*), cylinder-hewat + +# Turn on the cylindrical Debye–Scherrer correction +experiment.absorption.type = 'cylinder-hewat' +experiment.absorption.mu_r = 0.7 # fixed value from the beamline + +# (Optional) refine it — off by default because it is near-degenerate +# with Biso and scale +experiment.absorption.mu_r.free = True +``` + +`experiment.absorption.type = 'none'` restores A ≡ 1. + +### 7. CIF mapping + +Project-namespaced block, one identity tag plus the magnitude: + +``` +_absorption.type cylinder-hewat +_absorption.mu_r 0.7 +``` + +IUCr-aligned export (per +[`iucr-cif-tag-alignment.md`](../accepted/iucr-cif-tag-alignment.md)) +uses `_easydiffraction_absorption.type` / `.mu_r`, and may additionally +emit the standard provenance `_exptl_absorpt.correction_type cylinder`. +`flat-plate` writes `_absorption.mu_t`; TOF forms write their own +fields. The `_absorption.type` tag is the single source of truth for the +active type (no owner-level selector tag), per +[`switchable-category-owned-selectors.md`](../accepted/switchable-category-owned-selectors.md). + +### 8. Cost + +A(θ) is one vectorised `exp` over the 2θ grid per pattern evaluation — +microseconds, utterly dominated by the diffraction calculation itself. +Refining `mu_r` adds one parameter; each fit iteration recomputes the +envelope at negligible cost. There is no per-reflection loop and no +backend round-trip. + +## Consequences + +- **Closes the LaB₆ absorption residual.** The + `pd-neut-cwl_tch-fcj_abs_lab6` verification page becomes the + acceptance test: with `cylinder-hewat`, `mu_r = 0.7` it should reach + the same corr as the μR = 0 page. +- **Calculator-consistent by construction.** Both backends call the same + helper, so the absorption term can never drift between `cryspy` and + `crysfml`. New backends inherit it for free. +- **No upstream dependency.** We do not wait on cryspy or CrysFML Python + wrappers; nothing in `pyproject.toml` changes. +- **New switchable category to wire.** Owner attribute, `_swap_*` hook, + factory, `__init__.py` registration, enums, CIF round-trip, and the + `none` default — the full + [`switchable-category-owned-selectors.md`](../accepted/switchable-category-owned-selectors.md) + surface. Mitigated by mirroring `extinction` closely. +- **Degeneracy is documented, not hidden.** `mu_r.free = False` by + default; help text warns that refining μR together with Biso/scale + correlates strongly. Explicit modelling is preferred precisely so Biso + is not biased by soaking up absorption. +- **Pointwise envelope is an approximation** (second order in + FWHM·dA/dθ). Validated to 4 decimals against FullProf; acceptable and + documented. + +## Alternatives Considered + +1. **Per-reflection multiply inside each backend.** More "correct" in + principle, but cryspy has no slot (would need to patch its intensity + loop) and crysfml's routine is in unwrapped Fortran. Two divergent + code paths, an upstream dependency, and no measurable accuracy gain + over the envelope. Rejected. +2. **Flat instrument parameter `instrument.mu_r`** (as the issue + sketches). Simpler, but it cannot carry a correction-type selector + (Hewat vs Lobanov vs flat-plate vs TOF), breaks the + switchable-category uniformity the project standardised on, and has + no natural home for TOF forms. Rejected in favour of the switchable + category. +3. **Store μ and R separately, compute μR.** Matches the CIF items, but + the two inputs are perfectly correlated for the correction and only + their product matters; FullProf and CrysFML both parametrise by the + product. Storing them separately invites a confusing two-knob UI for + one degree of freedom. Deferred to optional provenance. +4. **Let Biso absorb it.** The status quo. Biases the thermal parameters + and fails the `_abs_` verification page. Rejected — this ADR exists + to avoid exactly that. + +## Deferred Work + +All of these are designed into the taxonomy and CIF tags above but **not +built in Phase 1** — each is a new class registered on the same +`AbsorptionFactory`, gated by `Compatibility`/`CalculatorSupport`, with +no change to the category, the swap hook, or the application helper. +None appears in the FullProf example suite (only the cylinder does), so +none is urgent; each should land **with a verification dataset**, not on +spec alone. + +- **`cylinder-lobanov`** — extends valid μR to ≈10 (branch at μR = 3). + Real CW neutron examples reach μR = 1.28, past Hewat's ≈1.0 validity, + so this is a genuine eventual want, not hypothetical. Add when a + high-μR reference exists; Hewat alone meets the current LaB₆ case. +- **TOF absorption** (`tof-cylinder`, `tof-exponential`): the + λ-dependent forms (FullProf `Iabscor = 2 / 3`). The pointwise-2θ + envelope does not transfer directly — needs its own application path. + FullProf TOF examples all use `Iabscor = 2` (cylindrical) and refine + it, so `tof-cylinder` is the natural next target. +- **`flat-plate`** (CW symmetric θ–2θ, `mu_t`) and other geometries + (`Iabscor = 1`): present in the file formats but used by **zero** + FullProf examples — lowest priority. +- **Optional (μ, R) provenance** mapped to + `_exptl_absorpt.coefficient_mu` and `_pd_spec.size_*`, read-only, with + `mu_r` remaining the single refineable knob. +- **Container / `2nd-muR`** (sample-in-holder coaxial cylinder): never + used in any FullProf example; revisit only if a case needs it. +- **Single-crystal absorption** (different formalism entirely). diff --git a/docs/dev/issues/open.md b/docs/dev/issues/open.md index 8477f9203..76634fe41 100644 --- a/docs/dev/issues/open.md +++ b/docs/dev/issues/open.md @@ -31,19 +31,30 @@ A(θ) = exp( -(1.7133 − 0.0368·sin²θ)·μR + (0.0927 + 0.375·sin²θ)·μR A Lobanov–Alte-da-Veiga form covers `μR > 3`. -**Implementation sketch:** - -- Add a `μR` instrument parameter for CWL powder (Debye–Scherrer). -- `crysfml`: CrysFML08 already implements this — reachable via - `Lorentz_abs_CW(..., ilor='DBS', cabs='HEWAT', tmv=μR)` through - pycrysfml. -- `cryspy`: multiply each reflection's intensity by `A(θ_hkl)`, - analogous to the existing Lorentz factor (one extra term). +**Design:** captured in +[`adrs/suggestions/model-sample-absorption.md`](../adrs/suggestions/model-sample-absorption.md) +— a switchable `experiment.absorption` category (mirroring `extinction`) +with a calculator-independent A(θ) envelope. + +**What the backends actually provide (corrected):** + +- `cryspy`: **no** absorption code at all (only Debye–Waller and sphere + _extinction_); its CW intensity loop has no slot to multiply A(θ). +- `crysfml`: CrysFML08 implements `Lorentz_abs_CW` in Fortran, but it is + **not** wrapped in `PythonAPI/`, and the high-level + `cw_powder_pattern_from_dict` path we call applies a plain Lorentz + factor with no absorption. So it is **not** reachable through + pycrysfml today without upstream changes. + +**Implication:** neither backend can apply the correction internally +without changes we do not own. The chosen approach computes A(θ) in +EasyDiffraction and applies it as a pointwise envelope on the calculated +pattern, identically for both calculators (see the ADR). **Note:** absorption is nearly degenerate with Biso + scale (its angle term is linear in `sin²θ`, like the Debye–Waller), so refining Biso can partly absorb it — but that biases Biso, so an explicit correction is -preferable. +preferable. In FullProf `μR` is normally **fixed**, not refined. **References:** @@ -52,10 +63,16 @@ preferable. - CrysFML08: [`Src/CFML_Powder/Pow_Lorentz_Absorption.f90`](https://code.ill.fr/scientific-software/CrysFML2008/-/blob/master/Src/CFML_Powder/Pow_Lorentz_Absorption.f90), `Lorentz_abs_CW`. -- FullProf `μR`: `.pcr` Lambda line field 7; `iabscor = 2` selects - HEWAT. +- FullProf splits absorption into a refineable **magnitude** and a + **type**: CW uses `μR` on the `.pcr` Lambda line (fixed there — no + refinement codeword), with the cylindrical Hewat form implied; TOF + uses `Iabscor` (`1` flat plate, `2` cylinder, `3` exponential + `exp(−ABS·λᶜ)`). `Cthm`/`Rpolarz`/`2nd-muR` on the Lambda line are + polarization and container terms, not the primary absorption knob. -**Depends on:** adding a `μR` instrument parameter. +**Depends on:** the switchable `experiment.absorption` category in the +ADR above (supersedes the earlier "add a `μR` instrument parameter" +sketch). --- @@ -2041,7 +2058,7 @@ The fast docs gate (`docs-build` + `link-check` + `spell-check`) catches broken nav/internal links and typos on every push, but does not yet check external URLs. Add a `lychee` link checker (with an allowlist for rate-limited/unstable domains), coordinated with the -[Documentation CI and Build Verification](../adrs/suggestions/documentation-ci-build.md) +[Documentation CI and Build Verification](../adrs/accepted/documentation-ci-build.md) ADR. Run it nightly or on pull requests to avoid flakiness from external sites. Also covers link-checking of URLs that appear only inside executed notebook output cells (a feature that does not exist yet). diff --git a/docs/dev/issues/recommended-priorities.md b/docs/dev/issues/recommended-priorities.md index a70b716de..9c8457024 100644 --- a/docs/dev/issues/recommended-priorities.md +++ b/docs/dev/issues/recommended-priorities.md @@ -83,7 +83,7 @@ session: - **[`cif-numeric-precision.md`](../adrs/suggestions/cif-numeric-precision.md)** — s.u.-aware CIF serialization (file size + meaningful precision). - **[`fit-output-files-and-data-exports.md`](../adrs/suggestions/fit-output-files-and-data-exports.md)**. -- **[`documentation-ci-build.md`](../adrs/suggestions/documentation-ci-build.md)**. +- **[`documentation-ci-build.md`](../adrs/accepted/documentation-ci-build.md)**. - **[`in-house-calculation-engine.md`](../adrs/suggestions/in-house-calculation-engine.md)** — drafted 2026-06-10 (in review): own the core (neutron powder Rietveld) in-repo, keep cryspy/crysfml/pdffit for the frontier. diff --git a/docs/dev/plans/documentation-snippet-tests.md b/docs/dev/plans/documentation-snippet-tests.md new file mode 100644 index 000000000..63d80ec77 --- /dev/null +++ b/docs/dev/plans/documentation-snippet-tests.md @@ -0,0 +1,210 @@ +# Plan: Documentation snippet smoke tests + +This plan follows [`AGENTS.md`](../../../AGENTS.md). No deliberate +exceptions to those instructions are required. + +## ADR + +Implements decision 3 ("Add snippet smoke tests for user-facing +examples") of the accepted ADR +[`documentation-ci-build.md`](../adrs/accepted/documentation-ci-build.md). +No new ADR is required. The ADR's "Implementation Status" table marks +this decision as the highest-value remaining gap, and its "Deferred +Work" section points at this plan. + +## Motivation + +User-facing code snippets drift from the public API. A recent pass found +`from easydiffraction import Structure / Experiment` (neither exported) +and `download_from_repository(...)` (does not exist) live in +`user-guide/first-steps.md`. The strict MkDocs build and link checker do +not execute Python, so this class of breakage reaches readers. A small, +fast, backend-free smoke test that exercises the documented public API +shape would catch it before merge. + +## Branch and PR + +- Branch: `documentation-snippet-tests` (flat slug, off `develop`). + Created and checked out by `/draft-impl-1`. +- PR targets `develop`, not `master`. Do not push unless asked. + +## Decisions + +- **Explicit markers, not blanket extraction.** Only fenced + ` ```python ` blocks explicitly opted in are executed. Many documented + snippets are intentionally non-self-contained (they reference a + `project` built in an earlier block, download data, or run `fit()` + against a real backend); auto-running every block would force heavy + fixtures and network/backends, which the ADR rules out. The opt-in + marker is an HTML comment on the line immediately before the fence: + ``. This keeps the test set curated and the + intent visible in the source Markdown. +- **API shape only, no computation.** Marked snippets construct small + in-memory objects and assert public names exist (`Project()`, + `project.structures.create(...)`, `experiment.peak.type = ...`, + `project.analysis.minimizer.show_supported()`, + `project.display.parameters.all()`). They must not download data, run + `fit()`, or select a real calculator/sampler backend. +- **No network, no real backends, no notebooks.** The runner sets a + guard (monkeypatched `download_data`/`download_tutorial` that raise, + and a check that no marked snippet imports a calculator backend). + Snippets run in a unique temp working directory. +- **Test tier: `tests/functional/`.** These are fast, in-process, + backend-free checks of the public API as documented — the same tier as + the existing functional suite (`pixi run functional-tests`, no + `-n auto`, no backends). They are not unit tests (they do not mirror a + single `src/` module) and not integration tests (no network/backends), + so `tests/unit/` structure mirroring (`test-structure-check`) is + unaffected. +- **One always-on shape check, independent of markers.** In addition to + marked-snippet execution, a parametrised test scans the doc set for + `from easydiffraction import