We've lost a few nice jobs here at Small Box recently. For the most part we have lost them because we have been unwilling to quote any further than we can "see". We know from experience that large Web projects require an investment in planning before any other work can be done. So we have been responding to RFPs and other inquiries with smaller planning proposals. Something that will create clarity and definition for the additional work to come.
We believe project planning needs to be done regardless so let's start there and see what we turn up. We may uncover some great open source solutions that reduce the cost and timeline. Maybe some of the "required" functionality isn't really needed once we have completely researched and mapped out the project.
The Web is nearly infinite and a Website should be a living thing, not a "finished" thing. So we need to have a flexible plan that can accommodate the ever changing nature of the Web. Planning may be educated guessing (see 37 Signals) but it is a needed map to take ideas further down the road.
Planning isn't cheap but neither is redoing a poorly planned project. The right foundation will build a Website that lasts for years to come. We have also seen many projects come to us that were rushed into production and subsequently missed the mark and had to be redone. Over time this ends up costing much more and negatively impacts the business or organization along the way. Those costs are hard to estimate but a bad Website can destroy a business. If a company's Website can't be found in search engines, is impossible to use and hard to update then it is probably doing more harm than good.
Not every project requires a separate planning agreement. Some projects we can "see" all the way to the end since we have done very similar ones in the past and the client clearly understands their needs and goals. But we find when taking on a large, custom Website or Web application we must insist on planning before we build.