“Minimoji” is a Memoji-inspired iOS concept app that lets the user see themselves as a 3D minifigure. The app recognizes head movements and mimics the user’s facial expressions – like looking into a magic LEGO mirror.
This simple prototype explores how emerging technologies, such as machine learning and computer vision, can be used to power a digital LEGO experience, all while safeguarding the user’s privacy.
Built using Apple’s Swift, ARKit, CoreML, and the TrueDepth camera system, this basic proof-of-concept utilizes machine learning to train a model which recognizes head movement and facial expressions, such as smiling and grinning. It then displays a 3D character and mimics the user’s movements and facial expressions.
With additional development, the app could for example display even more detailed expressions in an even more fluid manner, or be incorporated in an augmented reality experience.