I just talked to Cap One rep. I want to update a significant information for y'all.

Originally, I was planning to cancel the card as I heard from all the DPs saying that Cap One doesn't do prorated refund and you have to cancel within 30 days to get your AF back.

This morning I am here with the great news! Now Capital One will prorate annual fee if you perform a product change. In my case, I double checked with the rep that upgrading my Venture X to Venture One will result in prorating my $ 395 AF that hits on November 20th. Per her source, this prorate thing on product change was recently implemented by Capital One so that's why previously when people ask to get annual fee back, they can only cancel to get AF refunded within 39 days after annual fee hits.

Timeline
Nov 16th, used up my second year $300 Cap One Travel Credits
Nov 20th, $395 AF hit
Dec 19th, 10,000 Anniversary Venture Miles posted
Dec 19th, Called to cancel, but did product change to VentureOne instead and was promised on prorated refund (very likely I am probobly still got charged for $395/12 = $33 for a month worth of AF)