OP - The sole question is the time of your actual arrival at DUS versus the time you were originally scheduled to arrive. I take that to be 3:20 AM vs 10:45 PM or a delay of 4:35. Thus, the threshold for EC 261/2004 compensation is met unless the departure delay was due to "extraordinary circumstances".
The rest of the story is irrelevant and simply complicates and delays any claim. I am not sure what having AB staff lolling around at a bus does when you apparently found the bus, boarded it and the driver took you to DUS. I actually credit AB for rounding up 4 buses at that time of night and packing you off on your way with due speed.
The one missing fact from your post is the reason for the two delays. Those ultimately caused a sufficient delay that your aircraft could not land at DUS and required the diversion. Do you know what that is?
If you don't, I would still file the EC 261/2004 claim. If you do, please let us know and you may get some better guidance here.