Pareiduo

Research has been conducted on object detection from drawings or sketches, which represent more abstract representations than photos. While paintings and drawings have their own variety, the modalities surrounding them are far from exhausted. This project experiments with sand drawings and tests ChatGPT 4.0's ability to detect objects from sand drawings. The experiment involves a game process with humans interacting with a physical device. The device, which integrates a speaker, camera, and other technical elements, ensures a natural interaction while playing with sand and listening.

Pareiduo is a collaborative game of hidden vision and shared imagination. Each player, who plays the role of light or shadow respectively, will receive a secret object by leaning close to the device—one shapes the dark sand, the other sculpts the glowing voids between. Through touch and play, they mold a shared surface, guided by their own interpretations. Meanwhile, a language model observes the same image as it forms and speaks aloud what it perceives.

The goal is to create a single drawing that reveals both hidden objects to the AI. But beyond winning, Pareiduo is about the creative spark between abstract forms and everyday objects—where human interpretation and machine perception meet in an open-ended dialogue.