Main constructions at the academy include the Lingxing Gate, Banchi, Lisheng Gate, Lisheng Hall, Zhuzi Temple, Bailudong and the Imperial Writing Pavilion. The Zhuzi Temple was built to commemorate Zhu Xi. In the back of the academy is a cave with a stone white deer inside — the academy's namesake, as bailudong roughly translates to "white deer cave".

The academy, circled by carved stone walls, boasts well-designed architecture, graceful willows and a number of inscriptions by scholars and calligraphers throughout Chinese history.

In recent years, Bailudong Academy has become an important base of Chinese traditional culture study in China, according to Li Ningning, a professor at Jiujiang University.