Apple formally responds with a FAQ, and a software update will be made available later to do 3 things:

1. reduces the size of the crowd-sourced Wi-Fi hotspot and cell tower database cached on the iPhone,
2. ceases backing up this cache, and
3. deletes this cache entirely when Location Services is turned off.

After personally looking into the files and testing it this weekend, most of what Apple says seems to be plausible, and heres why I believe so:

I think it was an oversight that the cache wasn't being purged, possibly due to last minute iOS 4 changes. Apple has been rushing things lately on the iOS side, and a stressed out engineer could have missed a change he was tasked to make. iOS 3 contained a different cache file called h-cells.plist. iOS 4 (rather 3.2, the iPad only release) changed it to the consolidated.db, and added WiFi. These changes occurred as Apple began using their own WiFi service instead of Skyhook for WiFi and Google for cell tower info. I'm not 100% sure if the previous h-cells.plist was being purged on a regular basis, but from what I've seen, it wasn't as extensive as consolidated.db.

The file being backed up is likely a proper bug/mistake. H-cells.plist wasn't ever backed up, and was stored in /Library/caches/locationd for the root user. The new consolidated.db file is also stored in the same folder path. So where does the bug come in? iOS 4 added a persistent settings file to allow location to be enabled or disabled on an app by app basis. This is controlled by the initial user prompt when using an app, or via a control panel of per app toggles in the system Settings app. iOS 3 and prior would prompt multiple times to use location and lacked a central settings panel. This setting file also gets stored in /Library/caches/locationd. I'd bet that the person who implemented it added the full /Library/caches/locationd folder to the backup include list, instead of his one file. The proper way would have been to put the settings file in the normal place, /Library/Preferences. Definite code/implementation review failure here.

Number three is tied in with Apple claiming it's a bug that turning off Location Services doesn't always disable the cache. Deleting the file will ensure this isn't an issue. In my private testing, turning off Location Services did stop updating the file, but others are reporting it didn't stop for them.