When a short question to AI gets a long, off-target design explanation

This is a translated article. Read the original in Korean ↗

We were building a synchronized web app (formation) for a dance studio: one leader plays a video, and everyone else connects on their own phone to follow along at the same playback position. Video and music synchronization were all wired up, and someone had just said, "Now test it yourself with two windows." In the very next turn, one short question — "How do I add that?" — split into two completely different interpretations.

Conclusion

Q. What if you ask an AI coding assistant a short question and get a long explanation about something else entirely? A. Look at how the answer ended, not at the question. If the answer ended with another question, such as "Should we do this or that?", don't assume a fact that may already have been in that answer (for example, "It's not implemented yet") came across as confirmation. Open the screen and check for yourself.

When AI receives a short, ambiguous question, it fills in the gaps using the immediately preceding conversation. If that context was about design, such as a "remaining work" list, the next short question is easily pulled into that context and read as "How should we implement this?" Meanwhile, the person asking may have been looking at the screen in front of them and asking a usage question: "Does this work right now or not?"

Situation

The app was being built on Cloudflare Workers + Durable Objects, with the leader sending playback position and state for viewers to follow, and tested locally using wrangler dev (http://localhost:8787) . The scope had already been divided early on — the design document listed video selection by the leader as a required Phase 1 feature, while attaching a formation.json file containing dancers' positions and movement paths was separated out as an "optional feature to add later (add-on)" .

That division remained in place even after the leader and viewer pages (Phase 2b) were implemented. While putting together a "Progress so far" table and a "What's left" list, the assistant still listed "formation attachment feature (optional add-on)" as unimplemented, and explicitly asked for testing: "First, try running it in two desktop windows and let me know what you run into" .

Symptom

A short question came in the very next turn: "How am I supposed to add formation?"

The assistant interpreted this as "How should the formation feature be implemented?" It gave a long explanation, complete with a table, of an architecture routed through the cloud (leader selects → sends over WebSocket → room server stores and broadcasts → viewers receive automatically), then presented a "Things to add" list (such as adding a movement-path file input to the leader's room creation form). It ended by asking, "Should I implement this as Phase 3 now? Or should we do the basic two-window desktop test first and add it afterward?"

A follow-up came immediately: "Uh... that's not what I meant. Right now, when I open it as the leader, it asks me to select a video file, but there's no step for selecting formation. Doesn't there need to be a way to select it before I can select it? Is it not there yet?" This is a firsthand observation written after actually opening the leader screen ("Right now, when I open it as the leader, it asks me...").

Cause

"How am I supposed to add formation?" could be read in two ways — (A) "How should we implement this feature in the system?" (design), or (B) "What do I do on the current screen to add it?" (usage). The assistant was pulled toward (A) by the context of the "remaining work" list it had written in the preceding turn .

What made this interpretation even further off the mark was that, in that same turn 68, the assistant itself had requested testing: "First, try running it in two desktop windows and let me know what you run into." If the person was opening the screen as requested, "How am I supposed to add it?" was more naturally read as (B): a question about operating the current screen.

The answer actually already contained a sentence that answered the question: "There's exactly one thing missing — the part where the leader 'sends' formation. Right now, the leader only receives it and doesn't send it" . That also answers the question in turn 71 ("Is it not there yet?"). But the sentence was buried among the architecture explanation and tables, and instead of reaffirming that fact, the message ended with a new question: "Should we implement it or test first?" The explicit confirmation the person would have wanted — "That step doesn't exist yet" — never occupied the conclusion of the answer. As a result, instead of getting an answer, the person had to open the screen and check again themselves.

Fix

Immediately after the follow-up question (turn 71), the assistant changed its interpretation. "It's not there yet — you're right. I deliberately left it out of Phase 2b, so the leader's room creation screen currently only has a video input, with no step for selecting movement paths." Instead of ending with a question, it first answered with a confirmation, then started implementing it .

It then completed the feature by adding a movement-path file (.json) input to the leader's room creation form and wiring up parsing and transmission (publishFormation) with the server-side hasFormation flag. In its final report, the assistant explicitly acknowledged that the point raised in turn 71 was correct: "I've finished adding the movement-path (formation) selection step. You were right — it had been deliberately left out of Phase 2b, and now I've added it" .

The key is that the same fact — it wasn't there yet — appeared in both turn 70 and turn 72. The difference was in presentation, not content: turn 70 buried the fact in the middle of a long explanation and ended with a question, while turn 72 presented it as the first sentence of the answer and as confirmation.

Reporter's Notes — same answer, different question

Park Doeun

You know that dance studio app where the leader plays a video and everyone watches the same point on their phones? While adding movement paths (formation), one question led to completely different interpretations.

Jung Bada

What different interpretations? If someone asks, "How do I add formation?", of course you should explain how to implement it. What's wrong with that?

Jung Haneul

But the turn right before that question had a "What's left" list. Formation had just been listed there as an optional item.

Park Dohyun

Just because a question follows a list doesn't mean it's asking, "How do we implement that list?" What the person was looking at in that moment matters more.

Park Doeun

Right, and in that preceding turn, the assistant had even asked them to test it themselves with two windows. So they could already have been looking at the screen.

Jung Bada

Still, didn't the answer say, "The leader doesn't send formation yet"? Doesn't that mean the correct answer was already in there?

Jung Haneul

It was in there, but it was buried in the middle of an architecture table and design explanation, and the ending was another question: "Should we implement it now or test first?" It didn't end with confirmation.

Park Doeun

Why is ending with a question a problem?

Jung Haneul

When an answer ends with a question, the facts inside it don't come across to the other person as something that's been confirmed. So the person ended up having to look at the screen themselves and ask again.

Park Dohyun

What did they ask?

Park Doeun

"When I open it as the leader right now, I only see video selection, with no step for choosing formation. Is that not there yet?" They asked after opening the screen themselves. It wasn't a guess.

Jung Bada

Oh... then my initial judgment that "of course you should explain how to implement it" was wrong. They weren't asking about the design. They were asking, "Is that button on this screen right now or not?"

Jung Haneul

Exactly. And in the very next answer, the assistant started by confirming, "It's not there yet, you're right," and only then began implementing it.

Park Dohyun

So the lesson isn't that the design explanation itself was wrong. The problem was the order — failing to give confirmation first.

Jung Haneul

That's it. The assistant should have kept in mind that the preceding context could pull it toward a different interpretation, and given a confirmation like "It's not there yet" before ending the answer with a question.

Park Doeun

Then if it were me, when I got an ambiguous question, I'd first confirm the current state in one line, then go into the design explanation.

Jung Bada

The same goes for the person asking, then. If the answer comes back as another question, they should look at the screen and verify it themselves instead of trusting what was tucked inside the answer.

Park Dohyun

One short question can split into two interpretations, design and usage, and whether the answer ends with confirmation or another question is the signal — that's the whole story here.