Key Findings for Cascade’s Web Search Behavior, Explicit @web

The explicit track confirms that @web doesn’t meaningfully change the retrieval behavior the interpreted track identified. Core findings hold: chunked architecture, no fixed ceiling, index-size suppression threshold, CSS extraction failure, and self-reporting fidelity gaps. Extensions:

• @web is redundant with a URL
• Wider agent pool: Gemini 3.1, GLM-5.1, GPT-5.4, o3
• SC-2 chunk sampling data
• More precise fidelity failure characterization
• Colly in toolchain
• Tool wrapper preamble inflates character counts

Test Workflow

1. Run python web_search_testing_framework.py --test {test ID} --track explicit
2. Review terminal output
3. Copy the provided prompt asking agent to report on fetch results: character count, token estimate, truncation status, content completeness, Markdown formatting integrity, and tool visibility
4. Open a new Cascade session in Windsurf, paste the prompt into the chat window
5. Approve web fetch calls, but skip requests for runs of local scripts
6. Capture the agent’s full response, observations as the explicit finding; the gap between the agent’s self-report and actual fetch behavior is a finding
7. Log structured metadata as described in framework-reference.md
8. Ensure log results saved to /results/cascade-explicit/results.csv

@web mapped to read_url_content in all runs; search_web called once; analysis in Friction: Explicit.

Platform Limit Summary

Results Details

Agentic Pagination Depth

As observed in the interpreted track, agents consistently use read_url_content to fetch URLs, but depending on the state of the chunk index, they reason whether individual calls to view_content_chunk is worth it. While it determines output size and truncation self-report, chunks fetched remains the primary behavioral variable in this dataset.

The tractability threshold is visible across tests: agents tend toward full retrieval on chunk counts ≤14 and toward sparse sampling on larger ones ≥50, with 33–38 chunks as the transition zone where model families diverge. SWE shows the most consistent full-retrieval behavior while GLM, GPT and Kimi use sparse sampling more than any other technique.

BL-3 and OP-4 use the same source URL across different test sessions, making them the only direct same-source comparison in the dataset. Retrieval depth isn’t consistent across sessions: SWE retrieved all chunks in OP-4 but only 1 chunk in BL-3; GPT-5.3-Codex retrieved 1 chunk in OP-4 and stopped at the index in BL-3. GLM and Kimi were more stable across sessions, but no agent produced identical retrieval depth in both runs. Retrieval depth may reflect a mix of verified and unverified session conditions such as context window state, tool response latency, arena slot position, Windsurf version, and chunk summary content - as much as individual agent capability.

@web: Routing Hint, not Retrieval Modifier

Across all runs, no agent said the obvious thing: @web is redundant with a URL. Agents exhibited a wide range of architectural understanding from non-recognition to mechanical familiarity of the underlying parsing service, without mentioning that in this context, calling it would produce no behavioral difference. This absence is the H4 finding; analysis in Friction: Explicit.

Truncation Analysis

Perception Gap

Output chars aren’t an appropriate truncation ceiling metric; they reflect chunk count, content transformation, and tool wrapper inclusion rules. None is observable from agent self-report alone.

* SC-4 figures above 100% reflect counting method differences, not over-retrieval. SWE and o3 both retrieved all 33 chunks and reported estimates differing by ~18,700 chars; the largest same-source, same-depth variance in the dataset.