The Delta website shows the middle seats on the upper deck (D-G), but does not show the available window ones.
Looking at the Air France website, the window seats seem to cost extra "Upper deck duo seat" $43 per person.
My guess is that Air France is blocking these seats from Delta to get revenue from them (so Delta legitimately can't see them?). And because you have a Delta ticket, Air France has no way to bill the $43
Maybe someone else has experience with this, but I was in a similar predicament a few years ago (IAD->CDG flight too!), and the remaining seats on the upper-deck showed up 48-hours or so before check-in, and I was able to select them for free from the Delta seat selector... Of course, they have to be available for this to happen... So, there is always that issue...