It has been noted in various sources as either the second, third, fourth or fifth house built in town. What is not in dispute: In its early years, Caldwell House was considered the finest residence in ...