If the boarding pass has your Delta number on it, then even if you apply for a retro-credit I presume that part of that process is verifying if the miles have already been booked to another SkyTeam account, so it would fail.

At the very least you will have to ring SkyMiles and explain the situation and find out how to attempt to resolve this (I fear it may be VERY difficult to resolve). You will no doubt be told by SkyMiles that you will have to ring Flying Blue as they will be powerless to take the miles from your FB account, and certainly won't credit you the miles a second time.

However it might pan out, you will definitely have to contact both programs, preferably by phone and asking to speak to supervisors/managers if possible.

It would be worthwhile trying to figure out in advance how your Flying Blue number got associated with the booking - perhaps you did enter it yourself at some stage either during the booking process, afterwards when checking your bookings, or at check-in? You said you had to call Air France to give them your SkyMiles number - does that mean you had previously given them your Flying Blue number? There can surely be no way that either Delta or AF/KL magically ported in your Flying Blue number into the booking (how would they know it?), so unless you can explain what went wrong, they may be unsympathetic to your problem. But then again, if your boarding pass shows your SkyMiles number, and you are adamant that your FB number or card was never produced/entered/quoted/offered/asked for/requested at any stage of booking, checking-in or flying these flights, then it truly is amazing that that's where the mileage ended up.

EDITED TO SAY: I just see now that you replied that your KL number was on your boarding pass. I'm fairly sure that that means it's now too late to try and change it. You should have spotted this at checkin and had it changed then. It may be extremely difficult to undo this now.