In most email clients threading is just an optional feature that can be easily disabled. Gmail's web interface has been inspired by Google Groups, so it's obvious that threading was an important feature.
Paul Buchheit, the former Google engineer that developed Gmail, says that Google tries to increase the adoption of Google Apps by making Gmail more enterprise-friendly.
It's my opinion that when designing products, especially new products, it's better to have some people love it than have everyone tolerate it. This generally means aiming for simplicity and philosophical consistency. If you're aiming for "everyone tolerates it", then the approach switches more towards creating a "giant pile of features". I suspect that this change is driven by their desire for greater enterprise adoption (Google apps), where the "more features and checkboxes" approach very often wins out, even if it's ultimately a worse product.
Google says that the conversation view is not for everyone. "Threading enthusiasts say they spend less mental energy drawing connections between related messages and that their inboxes are much less cluttered. On the other hand, email traditionalists like many former Outlook users think conversation view just complicates something that has worked for years."
Conversation view can be disabled from the settings page by checking "conversation view off" in the General tab. This option may not be available right away, but Google promises that it "will be rolling out over the next few days". Google Apps users will only see this option if the administrator has enabled "pre-release features".
Why would you disable Gmail's conversation view?