Story, Inspiration & Ideation
Hearing the Forest is a digital immersive experience inspired by Chinese myths, particularly the story of the Moon, the Jade Rabbit, and the trees that grow there. In this experience, players journey into a silent forest on the edge of the Moon, guided by spirits and the mythical Jade Rabbit. As players interact with these spirits, they learn stories of the forest and receive unique seeds, each capable of growing into a tree with its own shape and color. As the trees grow, music notes emerge, syncing with their growth to create a serene, harmonious environment.
The project is inspired by the Lantern Festival, a celebration marking the end of the Chinese Spring Festival. While the Lantern Festival is full of energy and light, my vision is to bring a quiet, dream-like introspective version of the festival to life. Here, the peaceful Moon casts its soft glow over the world, providing a tranquil atmosphere where players can connect with the ancient myths of China. By planting trees in this serene forest, players breathe new life into the world, much like the way the Moon's light brings calm to the night sky.
Production
Procedural Generation for Terrain
To create a natural landscape while controlling workload, I used procedural generation for terrain generation. The terrain mesh was generated with a noise function from a predefined seed, so that the peaks and valleys look natural. The same function was also applied in generating the terrain material, whose color changes with altitude.
Tree Generation: Connection to Assistants API
Trees are one of my favorite imageries because it represents vitality and eternity. The branching patterns of trees are beautiful, and what's magical to me is that they can be simulated by algorithm. In the scene, I applied the L-systems algorithm for tree generation. I also trained an AI agent to generate L-system rules and send it to my scene via API, so that every seed contains a unique L-system rule capable of generating a unique tree pattern.
There are two modes of seed generation: when the Assistant AI API is connected, seeds are generated real-time by Assistant AI; otherwise, seeds are randomly selected from a preset selection of seeds.
Lightings and Post-Processing
To create a dream-like tranquil atmosphere, the scene is lit by lanterns with point light and other objects with emmisive materials. Post-processing modules like bloom, depth of field, vignette, and motion blurr also help enhancing the immersive gaming experience. Additional blue color fiter was included in underwater post-processing to simulate underwater perspective.
3D Audio and Real-time Audio
During the experience, each growing tree is accompanied by a unique chime audio. This is realized by the communication between a Max MSP patch and Unity via OSC. The L-system rule is sent to the Max patch, each character in the rule mapping to a musical rules like ascending a key, adding a new channel, etc. This way, an audio is generated real-time as a tree grows.
There are also 3D audios in the scene to enhance the sense of space and immersion, such as the water audio moving with the glowing Fish spirit.
Playtesting Documentation
