You need a "small" size carrier. Those fit within the 8" limit, give or take. At 11", what you're looking at is a 'medium' or even 'large' size, and those are too big for Frontier.
The '
Sherpa Original Deluxe - Small' gold standard for cat carriers measures 8.5" in height. This is the size that's guaranteed on board.
That said, soft-sided carriers are 'squishable' and you should easily be able to fit even an 11" carrier under the seat in front of you. If you decide to go with a larger bag, your cat may be denied boarding if you enounter especially zealous staff at check-in, but the vast majority of travelers get away with bringing a larger carrier as long as it is soft-sided and can be squished to fit.
Tip for Frontier: By all means, buy stretch seating with a pet carrier. Frontiers' regular seats (bulkhead and exit rows are not available with a carrier on Frontier) are very cramped as is with only 28" of pitch and you will be miserable with a pet carrier on a 5+-hour transcontinental flight.