Generating Handwriting via Decoupled Style Descriptors

European Conference on Computer Vision (ECCV) 2020

Atsunobu Kotani, Stefanie Tellex, James Tompkin

Handwriting synthesised in a target style, with reference samples and generated output — We synthesise handwriting (bottom) in a target style (top) via learned spaces of style and content, and can exploit any available reference samples (middle) to improve output quality.

Abstract

Representing a space of handwriting stroke styles includes the challenge of representing both the style of each character and the overall style of the human writer. Existing VRNN approaches to representing handwriting often do not distinguish between these different style components, which can reduce model capability. Instead, we introduce the Decoupled Style Descriptor (DSD) model for handwriting, which factors both character- and writer-level styles and allows our model to represent an overall greater space of styles. This approach also increases flexibility: given a few examples, we can generate handwriting in new writer styles, and also now generate handwriting of new characters across writer styles. In experiments, our generated results were preferred over a state of the art baseline method 88% of the time, and in a writer identification task on 20 held-out writers, our DSDs achieved 89.38% accuracy from a single sample word. Overall, DSDs allows us to improve both the quality and flexibility over existing handwriting stroke generation approaches.

Paper

Supplemental document first page — Supplemental
PDF 5.6 MB | MP4 5.4 MB

Video

Download video (MP4, 78 MB)

Supplemental video: a walkthrough of how we decouple character- and writer-level styles to synthesise new handwriting.
Download supplemental video (MP4, 5.4 MB)

Code + Data

We release our model code together with the BRUSH handwriting dataset at our GitHub repository. BRUSH collects 27,649 online handwriting samples from 170 writers. Each sample is recorded as online stroke data—a sequence of (x, y) pen coordinates with an end-of-stroke flag—alongside its target sentence and a per-point character label, rather than as a rendered image. We release BRUSH for non-commercial research use.

Slides

Download slides (PDF, 4.6 MB) | Download slides (PPTX, 5.8 MB)

Citation

@inbook{Kotani_2020,
    author    = {Kotani, Atsunobu and Tellex, Stefanie and Tompkin, James},
    title     = {Generating Handwriting via Decoupled Style Descriptors},
    booktitle = {Computer Vision -- ECCV 2020},
    publisher = {Springer International Publishing},
    year      = {2020},
    pages     = {764--780},
    isbn      = {9783030586102},
    doi       = {10.1007/978-3-030-58610-2_45},
}

Acknowledgements

This work was supported by the Sloan Foundation and the National Science Foundation under award number IIS-1652561. We thank Kwang In Kim for fruitful discussions and for being our matrix authority. We thank Naveen Srinivasan and Purvi Goel for the ECCV deadline snack delivery service. Finally, we thank all anonymous writers who contributed to our dataset.