Originally Posted by
gglave
How? The amount of data to cache hourly would be geographically staggering.
Let's say I want to fly out of PDX, and the dream map options are $250, $50, $750, $1000 and $1500. Every hour the system needs to calculate every destination from PDX, from San Diego to Reykjavik to Salem to Naples (both of them) and cache the fares, then do the same thing for every other airport code.
It's daunting - I work in the software field and I'm 100% confident this is why Travelocity dropped the feature.
Parallel Processing?
If you break it down to setting each core to calculate one destination and sub-destinations from there, you can reduce the processing time by an factor of the number of cores working on the problem.
Spread that work over a number of X CPUs with Y cores and assuming each core can run query as fast as it would take for a single system to run the process, you have an X*Y speed increase per City.
If you're willing to trade absolute accuracy for speed, you could take each routing as you calculate it and compare it to the inventory for that segment. Then based on the inventory, it should give you an idea as to whether or not you need to update the cached data the next time. Y9 on all legs probably means you're not going to need to check again in the next hour. If someone picks that destination, then check it again.
But I've avoided doing anything serious with software in a while. I just know that threaded, multi-cpu systems are where you want to start if you want complex calculations done fast.