# Deep Dive #8 — Open, Verified Generative Downscaling: a public counterpart to CorrDiff *NSF NCAR / RAL · strategy working document · draft July 2026* **One line:** Coarse forecasts and climate projections are useless for local decisions until they're downscaled to the neighborhood; the best generative-downscaling tooling (NVIDIA's CorrDiff) is vendor-owned and unverified. Build the open, physically-grounded, *independently-verified* alternative — with uncertainty as a first-class output, not an afterthought. --- ## Why this could be your project Downscaling is where forecasts become *visual and local* — the exact intersection of your strengths. The deliverable is a **toolkit plus a verification harness plus visualization of high-resolution fields**, which is software/DevX/viz work wrapped around an ML core. It also sits on top of RAL's existing downscaling practice (WRF/RTFDDA for wind & solar energy, fire, air quality), so you'd be modernizing a capability the lab already sells, not inventing a new mission. The heaviest lift — training generative models — is the part you'd partner with ML scientists on (e.g., the MILES-CREDIT team); the toolkit, evaluation, and viz are yours. *Caveat up front:* this is one of the more research-heavy ideas (feasibility scored 3/5). The way to de-risk it is to lead with the **toolkit + verification** framing rather than "train the world's best downscaler." --- ## The problem, precisely Global AI and physics models produce coarse output (~25 km); operational NWP like HRRR is finer but still misses street- and ridge-scale structure. Every high-value local decision — where the fire wind accelerates through a canyon, which substation floods, whether a solar farm is shaded, what a neighborhood's PM2.5 will be — needs **downscaling**. The field is moving fast but has clear gaps: - **The leading tool is proprietary.** NVIDIA's **CorrDiff** (a regression+diffusion generative model, ~25 km → ~2–3 km, reportedly ~1000× faster and ~3000× more energy-efficient than numerical downscaling) is the state of the art, shipped inside NVIDIA's Earth-2 / PhysicsNeMo stack. It's impressive and increasingly open-weight — but it's a vendor's product, tuned to a vendor's agenda, and **not independently verified**. - **Open options lag and are fragmented.** DL4DS (2022) was "the first open-source deep-learning downscaling library"; pyESD (2023) covers empirical-statistical downscaling. Neither is a maintained, production-grade, verified toolkit. - **Generalization is the unsolved scientific risk.** A November 2025 critical review flags that ML downscaling still struggles to extrapolate to unseen conditions — can a "Perfect Prognosis" model trained on today's climate be trusted for a warmer future or an unprecedented event? An AIES 2025 intercomparison over Spain asked exactly this and found the answer is not obviously yes. - **Uncertainty and physical consistency are usually missing.** Generative downscaling can hallucinate plausible-but-wrong fine structure. For public-safety and infrastructure use, "looks realistic" is not "is trustworthy." Net: there's a **rigor-and-trust gap** around a technique that's already being deployed. That gap is a public-institution opportunity. --- ## Why a public institution wins here Downscaling for consequential decisions needs exactly what a vendor can't self-supply: **independent verification, honest uncertainty, physical consistency, and open reproducibility.** NCAR/RAL can also bring things NVIDIA can't: - **Physics-grounded training targets.** RAL's **FastEddy** (GPU large-eddy simulation, resolving down to ~5–10 m, with nested mesoscale→LES capability) can generate high-resolution ground truth for training and validating downscalers — a genuine scientific asset a chip vendor doesn't have. - **Verification heritage** — this couples tightly to Deep Dive #1: the downscaling toolkit's differentiator *is* its verification harness. The two ideas reinforce each other. - **Domain coupling** — downscaled winds feed fire spread (#5), downscaled fields feed air-quality plumes (plume-viz), downscaled irradiance feeds energy forecasts. RAL owns those downstream applications. --- ## What exactly to build **A. An open downscaling toolkit.** A clean, documented library that wraps multiple downscaling methods behind one interface: statistical (fast baselines), regression, and generative/diffusion (the CorrDiff-class approach), so users can trade speed vs. fidelity. Containerized, cloud- and HPC-runnable, xarray/Zarr-native. **B. Uncertainty as a first-class output.** Every downscaled field ships with calibrated uncertainty (ensemble or diffusion-sample spread), and the API makes it hard to *ignore* the uncertainty. This is the trust differentiator. **C. A downscaling verification harness.** Borrow the MET-AI machinery (#1): evaluate downscalers against held-out high-res truth (HRRR, observations, FastEddy LES), test **out-of-distribution generalization explicitly** (train on one regime, test on another), check for physical consistency (conservation, spectra), and publish a downscaling leaderboard. **D. Reference models + open training data.** One or two well-documented reference generative models (e.g., a coarse-NWP → convection-permitting downscaler for a US region), released open with their training data — a public good and a citation magnet. **E. Visualization of the payoff.** Interactive comparison of coarse vs. downscaled fields with the uncertainty visible — your viz lane, and the thing that makes the value obvious to a non-expert. ### Phased build - **MVP:** toolkit skeleton + statistical & regression baselines + the verification harness on one region, benchmarking against CorrDiff-class output where available. Immediately useful and low-research-risk. - **v1:** a trained reference diffusion downscaler with calibrated uncertainty; explicit OOD generalization tests; open data release. - **v2:** FastEddy-grounded urban/complex-terrain downscaling; coupling hooks into fire (#5), plume-viz, and energy applications. --- ## Technical risks & unknowns - **Generative models hallucinate fine structure** — the central scientific risk; the verification harness and uncertainty outputs are the mitigation, and are themselves the fundable product. - **Training compute and data curation** are significant — partner with the MILES-CREDIT/ML team and lean on Derecho/Casper GPUs. - **Generalization to future climate / unprecedented events** may not be solvable in general; be honest and scope claims to verified regimes. - **Not first to market** — CorrDiff exists. Compete on *trust, openness, verification, and domain coupling*, not on being first or fastest. ## What success looks like A published, open downscaling benchmark that people cite; a reference model used by RAL's own energy/fire/AQ applications; and downscaled products that carry uncertainty a decision-maker can actually act on. --- # Positioning & funding — #8 ## The narrative "Generative downscaling is already being used for local decisions — but the best tool is a vendor's black box with no independent verification and no honest uncertainty. For anything where being wrong at the neighborhood scale costs money or lives, that's not good enough. NCAR/RAL can build the open, verified, physics-grounded alternative — and we have the LES ground truth (FastEddy) and the verification heritage that a chip company doesn't." ## Target sponsors & why each buys | Sponsor | The pitch to them | Vehicle | |---|---|---| | **DOE ASCR / SciDAC** | Generative surrogates + UQ for Earth-system prediction; leadership-compute-scale ML | DOE FOAs (recent $13.5M AI/ML-for-climate), SciDAC | | **NSF CSSI** | An open downscaling toolkit + benchmark as national software infrastructure | CSSI | | **Commercial (energy, insurance)** | Local irradiance/wind/precip with uncertainty = tradable, underwritable | Contracts; RAL's energy-forecasting client base (Xcel, Kuwait, Korea) | | **NOAA** | Verified downscaling supports impact-based decision support | EPIC / weather program | | **Philanthropy (climate)** | Open, trustworthy local climate projections for adaptation | Schmidt Sciences software institutes; Google.org climate track | ## The differentiated value prop - **vs. NVIDIA CorrDiff:** open, independently verified, uncertainty-quantified, physics-grounded via LES — the trust layer they structurally can't provide. - **To funders:** infrastructure that de-risks *everyone's* downscaling, not one product. ## The ask (illustrative) **2–3 FTE for ~18–24 months**, ideally as a joint software+ML effort, to ship the toolkit + verification harness (fundable on its own) and one open reference model. Sequence it *after* or *alongside* #1 so the verification machinery is shared. ## Risks to the funding case - "NVIDIA already did this" → reframe: they built *a* downscaler; we're building the *open standard for trusting* downscalers. - Research risk → lead with the toolkit/verification (engineering) deliverables that have value even if the generative model underperforms.