In the 1880s and 1890s it was occupied by a military academy, for whose accommodation the extensions on the right (since demolished) were added.

In 1902 the house became a public library, and the grounds a public park.

Built in 1772 as a country house, the Manor House was owned from 1796 by the banking family of Baring.

After 1850, when the area was becoming built up, they let the house rather than live there.