GPT-5.4 Thinking Delivers Strong Yet Misaligned Answers

I spent days testing GPT-5.4 Thinking and walked away impressed, then uneasy. The model’s outputs were polished, nuanced, and often insightful. They were also, too often, answers to adjacent questions rather than the one I actually asked. For anyone considering it for high-stakes or client-facing work, that gap between brilliance and brief adherence matters.

OpenAI pitches GPT-5.4 Thinking as a reasoning-forward upgrade capable of professional-grade tasks. In controlled evaluations, the company touts strong wins — including reports of an 83% advantage on certain pro-level benchmarks — and, in my tests, the model frequently wrote like a sharp analyst. But the core pattern I observed was answer-quality high, instruction-following inconsistent.

Test Setup and Scope for Evaluating GPT-5.4 Thinking

I used a paid ChatGPT plan and mixed practical, creative, and analytical prompts. Unlike “quick Q&A” trials, this model rewarded deeper, multi-step tasks: structured briefs, explicit constraints, and multi-turn follow-ups. That setup is closer to how professionals actually work — but it also made deviations more conspicuous when the model veered off-brief.

The goal wasn’t to break the model or stump it with trick prompts. I asked for outputs a consultant, researcher, or designer might need: concept design work, an itinerary tuned to real-world constraints, a policy analysis with a defensible stance, and a pedagogical explanation framed by a specific learning theory.

Design Reasoning Versus Image Execution in Testing

First, a visual challenge: a flying “helicarrier” concept with explicit propulsion orientation and deck operations. The reasoning step was excellent. GPT-5.4 Thinking critiqued my initial idea, explaining why four downward-facing turboprops are theatrically appealing yet aerodynamically weak for lift, and it foregrounded realistic constraints like weight-to-power ratios and deck safety zones.

Then came the image request: render the most probable design based on its analysis. Instead of translating its own conclusions into a coherent illustration, the model produced a generic image with propulsion details that contradicted its write-up. A subsequent “engineering rendering” came back with mislabeled or gibberish annotations. The thinking was solid; the visualization pipeline ignored it.

That split reflects a broader multimodal reality: many systems keep language reasoning and image generation loosely coupled. Unless the model can reliably bind design constraints to the renderer, you’ll get pretty pictures that defy the spec. For workflows needing traceable, engineering-grade visuals, this is a red flag.

Trip Planning Shows the Catch in Real Itineraries

I asked for a one-week Boston itinerary centered on technology and history, in March, with both premium and student-budget variants. The first draft was list-heavy and grouped days by theme rather than neighborhood — technically correct, logistically clumsy. With a few nudges, it reorganized by location, added foul-weather swaps, and produced credible day-by-day cost estimates.

Useful? Yes. Hands-off? Not yet. Formatting began as a wall of text; I had to request clearer structure. The content was strong enough for a traveler to refine, but a professional planner would still need to massage it into client-ready form. If GPT-5.4 Thinking “does pro tasks,” it currently does them like a capable associate who benefits from editorial supervision.

Longform Thinking Is Powerful But Prone To Drift

On analytical writing, the model shined. Asked whether social media has improved or worsened communication, and to take and defend a position, it delivered a tightly argued essay that weighed civic mobilization and knowledge access against polarization, incentive misalignment, and attention economics. The stance was clear and evidence-aware.

Yet when I requested an explanation of GPT-5.4 using educational constructivism — the “learn by doing” framework — it largely skipped the “doing.” Instead of activity-led scaffolds or hands-on mini-experiments, it defaulted to feature descriptions and high-level metaphors. The answer read well; it wasn’t the answer to the question.

Why Instruction Following Still Lags in Practice

Instruction fidelity is a known challenge across frontier models. Stanford’s AI Index has repeatedly noted gaps in reliability and adherence under varied prompts, and research from Anthropic on “helpful, honest, harmless” behavior shows trade-offs between initiative and obedience. NIST’s AI Risk Management Framework, meanwhile, stresses faithfulness to user intent as a core reliability axis.

GPT-5.4 Thinking clearly advances reasoning quality, but my tests suggest a recurring pattern: when the model can deliver a “great answer,” it sometimes prioritizes that over the exact brief. For regulated or precision-heavy work — engineering visuals, legal memos, medical summaries — being 95% brilliant yet 5% off-brief can be costlier than a simpler tool that follows directions to the letter.

A few guardrails helped: asking the model to restate the objective before answering, providing acceptance criteria, and instructing it to flag uncertainties. These mitigations improved adherence, but they also added process overhead — the opposite of “drop-in professional replacement.”

The Bottom Line on GPT-5.4 Thinking’s Trade-Offs Today

GPT-5.4 Thinking feels like a brilliant grad student: insightful, fast, and thorough — with a tendency to go off on the answer it wants to give. OpenAI’s performance claims, including eye-catching win rates, capture how far reasoning has come. They don’t erase the day-to-day cost of keeping the model on-brief.

As a co-pilot, it’s terrific. As an autonomous specialist, not yet. If OpenAI can tighten instruction fidelity — ensuring the model answers the question asked and grounds multimodal outputs in its own analysis — GPT-5.4 Thinking will be more than impressive. It will be trustworthy. Until then, expect strong work product that still needs a steady editor’s hand.