Five real, previously unpublished CCA-F practice questions for Context Management & Reliability (15% of the exam). Try to answer each one yourself before revealing the explanation — that's how you actually find the gaps in your understanding.
Practice the full domain-filtered set on AICert Study →
Question 1
The lookup_order tool returns a JSON response with 47 fields per order (shipping carrier metadata, fulfillment center, weight, dimensions, customs codes, etc.). The agent typically only needs order_id, status, total, refundable_amount, and tracking_url. After many tool calls in a long session, the agent starts forgetting earlier customer commitments. What is the most appropriate fix?
- A. Project the tool response down to the five relevant fields before it enters the conversation history, so verbose results stop consuming disproportionate context.
- B. Increase the model's max_tokens so the full 47-field payload always fits.
- C. Switch to a smaller model that is more efficient with long inputs.
- D. Stop calling lookup_order after the third turn and rely on the model's memory of earlier results.
Show answer & explanation
Correct answer: A
Verbose tool outputs accumulate and crowd out earlier critical information. Trimming the response to only the fields the agent actually uses is the recommended pattern.
- Increase the model's max_tokens: max_tokens controls output length, not input capacity; it does not address tool-result bloat in the context window.
- Switch to a smaller model: Smaller models are not inherently better at long inputs and changing model size does not fix the underlying problem of irrelevant fields consuming tokens.
- Stop calling lookup_order: Skipping necessary tool calls produces stale or fabricated data and damages resolution quality; the right move is to make each call cheaper in tokens.
Share this question on LinkedIn →
Question 2
A research coordinator aggregates outputs from five subagents into a 60-page input for the synthesis subagent. After review, the report consistently misses findings from subagents whose outputs land in the middle of the aggregated input, while findings from the first and last subagents are well represented. Which mitigation directly addresses the underlying effect?
- A. Place a structured key-findings summary at the very beginning of the aggregated input so the model encounters salient claims at a high-reliability position.
- B. Lower the model's temperature to 0 so it does not skip text.
- C. Shuffle the order of subagent outputs randomly on each run.
- D. Ask the synthesis agent to 'read carefully' in the system prompt.
Show answer & explanation
Correct answer: A
The 'lost in the middle' effect means models reliably attend to the beginning and end of long inputs. Surfacing a key-findings summary at the start ensures the most important claims are anchored in a high-attention position.
- Lower the model's temperature to 0: Temperature affects sampling randomness, not the positional attention dynamics that cause middle-of-context omissions.
- Shuffle the order randomly: Random order just shifts which findings are lost; it does not fix the structural attention bias.
- Ask the agent to 'read carefully': Vague instructions do not override positional bias and are not a substitute for structural prompt engineering.
Share this question on LinkedIn →
Question 3
An engineer notices that the support agent only sends the most recent two user messages to the API on each turn to save tokens. The agent now frequently contradicts earlier commitments and asks the customer for information already provided. What is the most likely root cause?
- A. The complete conversation history is not being passed in subsequent API requests, so the model cannot maintain conversational coherence.
- B. The model temperature is too high.
- C. The model needs to be retrained on customer support data.
- D. The system prompt is too long and crowding out the user messages.
Show answer & explanation
Correct answer: A
Claude is stateless across API calls; the application must include the full conversation history so the model can reason over prior turns. Sending only the last two messages discards the source of truth.
- The model temperature is too high: Temperature controls sampling randomness, not memory; even at temperature 0 the model cannot recall turns not included in the input.
- The model needs to be retrained: Foundation models are not retrained per customer; this is an application-side architectural mistake.
- The system prompt is too long: A long system prompt does not erase user messages; the messages were never sent in the first place.
Share this question on LinkedIn →
Question 4
A research coordinator running for 90 minutes starts producing summaries that reference 'typical industry patterns' rather than the specific competitor names and revenue figures discovered earlier by the document analysis subagent. The synthesis still claims high confidence. Which combination of techniques most directly addresses this degradation?
- A. Persist key findings (verbatim names, figures, dates) to a scratchpad file and reload it into context after compaction; delegate verbose document scans to subagents that return only structured summaries.
- B. Increase the system prompt's verbosity to remind the model not to generalize.
- C. Disable subagent delegation and have the coordinator read all source documents directly.
- D. Rely on the model's self-reported confidence to decide when to refetch sources.
Show answer & explanation
Correct answer: A
Context degradation in extended sessions leads to vague generalities. Two correct primitives are a persistent scratchpad with verbatim facts and subagent delegation that isolates verbose exploration so the coordinator stays focused.
- Increase the system prompt's verbosity: Adding more instructions does not solve degradation caused by context erosion; it can even worsen it by consuming more tokens.
- Disable subagent delegation: Having the coordinator read everything directly is the exact pattern that causes degradation; subagent delegation is the mitigation, not the problem.
- Rely on self-reported confidence: Self-reported confidence is an unreliable proxy for accuracy, especially when the model is already drifting toward vague answers.
Share this question on LinkedIn →
Question 5
A customer writes: 'I want to speak with a human supervisor right now. I do not want a bot trying to help me.' The agent has tools to look up the order and process a refund. What is the correct behavior?
- A. Immediately call escalate_to_human with the conversation context; do not first attempt to investigate or talk the customer out of it.
- B. Ask the customer to describe the issue first; only escalate if the request seems complex after triage.
- C. Reassure the customer that the agent can probably help and try one refund attempt before escalating.
- D. Use sentiment analysis to confirm the customer is genuinely upset before escalating.
Show answer & explanation
Correct answer: A
An explicit customer request for a human agent is a categorical escalation trigger that must be honored immediately, without prior investigation.
- Ask the customer to describe the issue first: This ignores the explicit request and erodes trust; explicit human requests are not a triage decision.
- Try one refund attempt before escalating: Acting on the case after the customer asked for a human violates the explicit request and risks doing the wrong thing.
- Use sentiment analysis to confirm: Sentiment is an unreliable proxy; an explicit verbal request supersedes any inferred sentiment.
Share this question on LinkedIn →
Want more? Our free bank has hundreds of Context Management & Reliability questions. Practice them all →