Three or four days is hardly anything in India. Getting from one place to another takes quite a bit of time. With only 3-4 days I would stick with a single destination. Pretty much every place in India is a good place to get local culture, and the neat thing is, it's different every place you go.

For colonial architecture, Kolkata is actually at the top of my list. Delhi has plenty also, but it's much more spread out, and the intervening spaces are largely filled in with 60s/70s-era hideousness. That said, from a historical perspective Delhi probably has the largest concentration of great sightseeing (although you have to do a lot of driving around town to see it all).

Chennai is also an interesting destination, and culturally the south of India is very distinct vs. the north (temples, food, attire, etc.). There is beach potential nearby, although to be honest I would never go to India expecting to find a nice beach. They are all a lot worse/dirtier/more crowded than the beach destinations in SE Asia.