Christmas Face Filters
An interactive musical piece.Overview
This project is an interactive, festive application designed to spread holiday cheer by allowing users to apply fun Christmas-themed filters to their faces using a webcam. Built with p5.js and machine learning models (HandPose and PoseNet), this project offers an intuitive, hands-free experience where users can select filters by simply pointing in the air.
This project is an interactive, festive application designed to spread holiday cheer by allowing users to apply fun Christmas-themed filters to their faces using a webcam. Built with p5.js and machine learning models (HandPose and PoseNet), this project offers an intuitive, hands-free experience where users can select filters by simply pointing in the air.
Tools
ml5
p5.js
Procreate
Timeline
Dec 2023
ml5
p5.js
Procreate
Timeline
Dec 2023
Final Outcome
︎Go to p5.jsTarget Audience
The target audience is people of all ages, sex, and race (even Santa can try it!). Anyone who wants to join in the Christmas spirit is welcome to try. An innovative and interactive feature of allowing the user to pick a filter by pointing in the air will potentially attract more teenagers and young adults.
Logics
Using two machine learning models - HandPose and PoseNet - it will detect key facial parts (eyes, nose, and mouth) and apply selfie filters to the user.HandPose will be used for the user to select a filter by
moving their hand in the air instead of clicking a button or touching the screen like other selfie apps.
The position of the index finger will determine the filter: each of the five boxes represents a certain filter, and the user can point to one of
these boxes to change filters. If the index finger isn’t on any of the boxes, no filter will be applied, and the canvas will only show the normal self.
PoseNet will be used to find the user’s nose. Then, it will attach features of the filters in relation to the coordinates of the nose. For instance, it will add Santa’s white mustache right below where the nose is.
It will offer five face filters – Santa, snowman, Rudolf, elf, and polar bear. It is geared towards offering a fun, interactive experience to the user, in which they can become a Christmasy creature.
Process
Before I began coding, I collected free-licensed visual assets from Freepik. I then made a few adjustments using Adobe Illustrator such as re-scaling and simplifying the images.
Then, I started coding by uploading two machine learning models.
Next, I assigned variables to the coordinates of the nose and the index finger so that these values determines the filter type and where to apply the face filter.
Then, I created five if statements for the five boxes on the canvas. If the x and y coordinates are within a certain range, it will output an image accordingly. All the values were calculated in relation to the width and height of the box.
Challenge & Solution
My concern was that the program might lag as the code becomes heavy with two machine learning models and multiple large-sized images. Only the HandPose model was being loaded because the PoseNet model takes longer time to load. Therefore, I added ‘setTimeout’ to earn time to load the PoseNet model before the HandPose is loaded, like below.
User Testing and Development
During the process, I did user testing with my peers. One commentedthat the experience would be more entertaining if there were more
decorations around the canvas, so I added a small Christmas ornament at the top of the canvas.
Another mentioned that it is quite difficult to aim at the box accurately so I added one-sentence instruction at the bottom of the canvas to make the user experience easier. Generally, they liked how my project is interactive and offers various types of cute face filters.