SVGDreamer: Text Guided SVG Generation with Diffusion Model

Haitao Zhou1
Chuang Wang1
1Beihang University   2The University of Hong Kong
Paper status - Accepted by CVPR 2024 🎉🎉🎉

SVGDreamer teaser
Given a text prompt, SVGDreamer can generate a variety of vector graphics. (Various colored suffixes indicate different styles.)
SVGDreamer teaser
Our proposed tool, SVGDreamer, excels at generating editable vector graphics. As such, it can be used to create vector graphic assets.
TL;DR: SVGDreamer is a text-to-SVG synthesis method that improves editability, visual quality, and diversity. It uses Semantic-driven Image Vectorization (SIVE) for element control and Vectorized Particle-based Score Distillation (VPSD) to address shape, color, and convergence challenges.

Abstract

Recently, text-guided scalable vector graphics (text-to-SVGs) synthesis has shown promise in domains such as iconography and sketch. However, existing text-to-SVG generation methods lack editability and struggle with visual quality and result diversity. To address these limitations, we propose a novel text-guided vector graphics synthesis method called SVGDreamer. SVGDreamer incorporates a semantic-driven image vectorization (SIVE) process that enables the decomposition of synthesis into foreground objects and background, thereby enhancing editability. Specifically, the SIVE process introduce attention-based primitive control and an attention-mask loss function for effective control and manipulation of individual elements. Additionally, we propose a Vectorized Particle-based Score Distillation (VPSD) approach to tackle the challenges of shape over-smoothing, color over-saturation, limited diversity in results, and slow convergence in existing text-to-SVG generation methods. VPSD models SVGs as distributions of control points and colors to counteract over-smoothing and over-saturation. Furthermore, VPSD leverages a reward model to reweight vector particles, which improves aesthetic appeal and accelerates convergence. Extensive experiments have been conducted to validate the effectiveness of SVGDreamer, demonstrating its superiority over baseline methods in terms of editability, visual quality, and diversity.


Methodology


our method consists of two parts: semantic-driven image vectorization(SIVE) and SVG synthesis through VPSD optimization.

SVGDreamer method
Overview of our proposed SVGDreamer. The method consists of two parts: semantic-driven image vectorization (SIVE) and SVG synthesis through VPSD optimization. The result obtained from SIVE can be used as input for VPSD, which helps in establishing a dependable initialization point.

Application | Vector Element Asset


Our proposed tool, SVGDreamer, is capable of generating vector graphics with exceptional editability. Additionally, we present further examples below. These generated SVGs can be decomposed into background and foreground elements, which can then be recombined to create new SVGs.

experiments
Examples showcasing the editability of the results generated by our SVGDreamer.

Application -- Poster Design


SVGDreamer can be utilized to create vector graphic assets for poster design. As shown in below, all graphic elements in the three poster examples are generated by our SVGDreamer. Designers can easily recombine these elements with glyph to create unique posters (in SVG format).

experiments
Comparison of synthetic posters generated by different methods. The input text prompts and glyphs to be added to the posters are displayed on the left side.

Additional Results


experiments
Additional qualitative results of SVGDreamer.

Citation

@article{svgdreamer_xing_2023,
  title={SVGDreamer: Text Guided SVG Generation with Diffusion Model},
  author = {Xing, Ximing and Zhou, Haitao and Wang, Chuang and Zhang, Jing and Xu, Dong and Yu, Qian},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2024},
  pages = {4546-4555}}


Acknowledgements

We thank Ximing Xing for providing us with the source code of the web page to help us build the project home page.