feat: Offer --cache option to control cache mode (Incremental Build)

[maxreichmann]

Part of Incremental Build: #1267

JIRA: CPOUI5FOUNDATION-1208

• Introduce --cache with values "Default", "ReadOnly", "Force" and "Off" for commands "ui5 build" and "ui5 serve"
  • Add yargs help description (also appears in CLI docs)
• Add JSDoc annotations
• Rename --cache-mode to --snapshot-cache
  • Hide deprecated option in yargs help (if still used -> logs warning)
  • Legacy logic now usable via new name
  • Add hint under "Migrate to v5" in docs explaining this

[maxreichmann]

Rebased onto base branch feat/incremental-build-4 and fixed all unit tests which are not failing in base branch.

[RandomByte]

Adding @KlattG since this PR changes a lot of user-facing texts

[KlattG]

Nothing major, just a few things I noticed

Resolved

[matz3]

I noticed that when using the mode "Off" there are still verbose logs regarding caching, which I did not expect.
Based on that I asked an agent to check the details. This is the result, and to me it looks plausible and we should look into addressing the individual findings:

---

--cache Option Audit — Off Mode Has Residual Caching Overhead

Branch reviewed: feat/incremental-build-4 at commit cbfe225b3 ("feat: Offer --cache option to control cache mode (Incremental Build)").

Documented contract

packages/cli/lib/cli/commands/build.js:89-99 and the JSDoc on
packages/project/lib/build/cache/Cache.js:

Off — Do not use any cache and always rebuild.

"'Off' does not use any cache and always triggers a rebuild of the project"

The user-facing promise is that Off produces "no overhead related to caching during the whole build process".

Summary

Off is mostly wired through the call chain (CLI → ProjectGraph.build →
BuildContext → ProjectBuildContext → ProjectBuildCache → BuildServer),
but the build still does a non-trivial amount of cache work even when
cache=Off. The most serious deviation is that the build still writes
untransformed source files to the persistent SQLite CAS database. There are
also several pieces of in-memory cache infrastructure that remain active.

Findings

1. (BUG) Off still writes to the persistent SQLite cache

Location: packages/project/lib/build/cache/ProjectBuildCache.js:1255-1309
(allTasksCompleted) → :1092-1212 (#freezeUntransformedSources) →
:1198-1200.

allTasksCompleted() is called unconditionally at the end of every project
build. It calls #freezeUntransformedSources(), which:

1. Reads every untransformed source file from disk
2. Computes integrity hashes
3. gzip-compresses buffers
4. Writes the bytes to the CAS (putCompressedContent, SQLite blob insert)
5. Persists resourceMetadata via cacheManager.writeStageCache(...) (SQLite
   metadata insert)

None of this is gated on #cacheMode. Even though writeCache() later returns
early for Off/ReadOnly (:1496-1504), the freeze-sources path bypasses that
gate entirely. So with cache=Off the SQLite database is opened and written to
every build — directly violating "does not use any cache".

Fix: skip #freezeUntransformedSources() (and the surrounding
revalidation, see #2) when #cacheMode === Cache.Off. The CAS-backed source
reader is only consumed by downstream dependents to avoid race conditions
across project builds; in Off mode there is nothing to protect from a stale
filesystem read because nothing is being preserved across builds anyway.

2. #revalidateSourceIndex() re-reads every source file in Off mode

Location: ProjectBuildCache.js:1028-1079, called unconditionally from
allTasksCompleted (:1260).

After every build, this method:

1. Globs the entire source reader (sourceReader.byGlob("/**/*"))
2. For each resource, runs isResourceUnchanged() against the cached source
   index, which can recompute integrity hashes

This exists purely to detect that a source file changed during the build (so
that the cache is not poisoned). In Off mode there is no cache to poison — the
work is wasted. The cost scales linearly with the project's source size.

3. recordTaskResult() records resource requests and computes hashes

Location: ProjectBuildCache.js:829-946. Called from TaskRunner.js:266 and
:564 for every task, with no #cacheMode check.

Per task, even with cache=Off, this:

• Creates a BuildTaskCache if missing (:833-837)
• Calls taskCache.recordRequests(...) which builds shared hash trees over
  every requested resource (:903-909)
• Reads the entire stage writer via byGlob("/**/*") (:844)
• Computes a stage signature
• Stores the entry in the in-memory StageCache via addSignature(...)
  (:926-928)

The hash tree construction in recordRequests is the dominant per-task
overhead in incremental builds. It is wasted entirely when cache=Off.

4. #initSourceIndex() builds a full Merkle tree in Off mode

Location: ProjectBuildCache.js:1342-1357.

The Off branch of #initSourceIndex() does:

this.#sourceIndex = await ResourceIndex.create(
    await sourceReader.byGlob("/**/*"), Date.now());
this.#combinedIndexState = INDEX_STATES.INITIAL;

ResourceIndex.create constructs the full directory-keyed Merkle tree over all
source resources, computing SHA-256 hashes for every file. Nothing in Off
mode reads that tree afterwards (the tree is consulted by
#findResultCache(), #getPossibleResultStageSignatures(), etc., all of which
are skipped in Off mode). It is built and discarded. The comment at line 1343
("Caching disabled: Create fresh index") is misleading — there is no need for
any index when caching is disabled.

5. CacheManager (and its SQLite database) is opened in Off mode

Location: BuildContext.js:167-175.

getProjectContext always calls getCacheManager(), which lazily opens the
SQLite database under ~/.ui5/buildCache/v0_7/cache.db — with WAL mode,
PRAGMA tuning, etc. — regardless of cache mode. With Off, the only writes
that legitimately occur are from #freezeUntransformedSources() (see #1);
ideally the database should not be opened at all.

The CacheManager is also reference-counted and kept alive across project
contexts in a BuildServer, so the open-on-first-use cost is paid once per
session. Still, opening a SQLite DB and running its schema/migration code is
not zero, and in Off mode none of that work has any reason to occur.

6. In-memory StageCache is populated in Off mode

Location: ProjectBuildCache.js:57 (instantiation) and :926-928
(insertion).

Even in Off mode, every executed task calls
this.#stageCache.addSignature(...). This is in-memory only, so the cost is
small, but it is still non-zero (the tag operation maps and writer references
are retained for the lifetime of the ProjectBuildContext, which in
BuildServer is the whole session). Not strictly a correctness issue, but
inconsistent with the documented contract.

7. Minor: cache-mode deprecation warning is silent in tree.js

Location: packages/cli/lib/cli/commands/tree.js:31-36.

build.js and serve.js use .option("cache-mode", { hidden: true, ... })
combined with a .coerce() that emits log.warn(...) if the option is
provided. tree.js instead uses .hide("cache-mode", ...) (an undocumented /
non-yargs API style call) and no .coerce(), so users running
ui5 tree --cache-mode=... get no deprecation warning. Probably an oversight
during the rename. (tree itself does not need a --cache option.)

8. Minor: cacheDir parameter passed to graph.build() is silently ignored

Location: packages/cli/lib/cli/commands/build.js:207.

await graph.build({
    graph,
    cacheDir: path.join(graph.getRoot().getRootPath(), ".ui5-cache"),
    ...
});

ProjectGraph.build (packages/project/lib/graph/ProjectGraph.js:725-734)
does not destructure cacheDir. The intent looks like overriding the cache
location to a per-project .ui5-cache directory, but the value is silently
dropped. The actual cache location is still controlled via ui5DataDir
(default ~/.ui5). Not a cache=Off issue, but it is a separate bug
introduced by the same PR and worth fixing alongside.

Severity ranking (for cache=Off overhead)

#	Issue	Impact
1	#freezeUntransformedSources() writes to SQLite CAS	Correctness — violates the documented contract. Persistent side effect on disk.
2	#revalidateSourceIndex() re-globs and hashes all sources	Performance — O(source size) per build
3	recordTaskResult() builds hash trees, records requests	Performance — O(requests × resources) per task
4	#initSourceIndex() builds Merkle tree	Performance — O(source size) once per build
5	CacheManager SQLite DB opened	Startup cost; minor
6	In-memory StageCache populated	Memory; minor
7	Missing deprecation warning in tree.js	UX; minor
8	cacheDir argument silently dropped	Separate latent bug

Recommended fix outline

A single guard in the ProjectBuildCache hot path covers issues 1–4 and 6:

• In allTasksCompleted(), skip #revalidateSourceIndex() and
  #freezeUntransformedSources() when #cacheMode === Cache.Off.
• In recordTaskResult(), return early after the stage writer accounting that
  the build itself depends on (e.g. #writtenResultResourcePaths) — skip
  taskCache.recordRequests, BuildTaskCache creation, and
  stageCache.addSignature.
• In #initSourceIndex() Off branch, leave #sourceIndex unset (or set to a
  cheap sentinel) and ensure no downstream code path dereferences it. (Quick
  audit: #getPossibleResultStageSignatures, #getResultStageSignature,
  #updateSourceIndex, #revalidateSourceIndex, #freezeUntransformedSources
  all touch #sourceIndex; all of them are reachable only through paths that
  should be Off-gated.)
• In BuildContext.getCacheManager(), return a no-op manager (or skip
  initialization entirely) when buildConfig.cache === Cache.Off. This also
  removes the SQLite open cost (issue 5).

Tests that should be added to ProjectBuilder.integration.js /
BuildServer.integration.js:

• After a cache=Off build, assert that ~/.ui5/buildCache/v0_7/cache.db
  contains no rows for the project under test (or that the file is not
  created at all when no other project ever wrote to it).
• Spy on CacheManager.writeStageCache / putCompressedContent /
  writeIndexCache / writeResultMetadata and assert zero calls during a
  cache=Off build.