Fan Xiang

← Teaching / Student work

A participant using FaceType in front of a projection of expressive writing
Figure 5. Concept image of FaceType.

Student work · 2021–22

FaceType

Expression as a Mode of Writing

FaceType is an interactive installation that transforms the emotion, rhythm, and emphasis of a person’s speech into a dynamic writing tool. Inspired by Chinese calligraphy, the project explores how computer-generated type might express changes in a speaker’s emotions in the way handwritten calligraphy does.

Technically, a camera captures facial expression while a microphone captures vocal pauses, speech rate, volume, and syllable timing. These signals alter letterforms, brush texture, spacing, width, size, and writing speed in real time.

Team
Kevin Maher · Fan Xiang · Liang Zhi
Recognition
Tsinghua University Outstanding Graduate Thesis · Outstanding Graduation Project · 2021
Selected
IEEE VISAP · 2021
ACM SIGGRAPH Art Papers · 2022
View VISAP catalogue ↗Read the ACM paper ↗

How it works

Making the qualities of a voice visible

At the original 2021 Tsinghua University Postgraduate Exhibition, recent recordings appeared on an LCD while older phrases accumulated across a projected canvas. Nearly half the audience tried the work; many adjusted their expression after seeing the typography respond.

Observing participants led to a second iteration. The system began sampling multiple emotions within each word, voice volume became a clearer control for character size, and video feedback was added so participants could see how facial expression entered the writing.

Diagram mapping emotion to calligraphic stroke
Figure 2. Mapping emotion to calligraphic stroke.
Diagram of relationships between emotion and brush attributes
Figure 3. Relationships between emotion and brush attributes.
Different expressions of the word OK generated by FaceType
Figure 4. Different expressions of the word “OK” generated by FaceType.
FaceType installation at the 2021 Tsinghua University Postgraduate Exhibition
Figure 6. FaceType at the 2021 Tsinghua University Postgraduate Exhibition.
Participants and the expressive writing generated from their speech
Figure 7. Participants and the expressive writing generated from their speech.
FaceType output produced by several participants over one hour
Figure 8. FaceType output produced by several participants over one hour.
Summary of the original FaceType interaction and its redesign
Figure 9. The original interaction and the changes made after observing participants.
Iterated FaceType interface
Figure 10. The iterated FaceType interface.

Images reproduced from the authors’ ACM article. © Kevin Maher, 2022.