The airport is very close to the city center but security lines for departing flights can often be long and slow (unless there has been a recent improvement). So it would depend on your personal risk tolerance.

Although Lisbon is a lovely city, personally I wouldn't bother; how much can you do / see in an hour or so? Unless you had something specific in mind.