You're supposed to... but the police don't normally stop pedestrians* in the street randomly to check their ID, which is what I understand was happening in parts of Hamburg a couple of weeks ago.

(*Cars sometimes get stopped for random checks, especially late on Friday/Saturday nights... but although they will ask to see your ID, driving licence and car registration document the real purpose of these late-night Verkehrskontrollen is to look for people who may be over the limit. Happened to me last year... they just checked my documents and sent me on my way, no breathalysing)