Here are a copy of links to possible dangers of lithium batteries. The first one is about the FAA warning. The second one is an older article (2004) talking about the dangers of the batteries. I think part of the problem is that they burn at a very high temperature.

I think having a few here and there isn't much of a danger but if you have a bunch of them then it is more of a fire hazard. Loose batteries are more likely to make contact with metal items or other batteries which could be dangerous.

http://www.cellplaza.org/blog/1871/f...y-fire-safety/
http://findarticles.com/p/articles/m...8/ai_n6280925/