Towards consistent sketch-guided local 3D shape editing

Toyohashi University of Technology, University of Fukui
Pattern Recognition 2026
Second research result visualization

Sketch-guided shape editing comparison against prior 3D shape modeling methods. The upper row displays the initial shapes generated via each method using the initial sketch, while the bottom generates the edited shapes using the edited sketch.

Abstract

Modeling complex 3D shapes from 2D sketch images has gained significant attention due to recent advancements in deep generative models. While many efforts have focused on producing geometries that faithfully reflect sketches, there has been limited exploration of sketch-guided local editing. Existing shape generation techniques often modify not only the intended edited areas but also the unedited parts, compromising geometric coherence and fairness among different parts of the shape. This paper introduces a shape editing network (SEN) under the diffusion-based generative framework to specifically enhance consistency in sketch-guided local shape editing. Our core insight is capturing partial geometric structure correlations to achieve consistent editing results. To accomplish this, the SEN is designed to conditionally learn local edits of a partial shape while considering the geometries of the unedited parts and the sketch image. We also devise an attention fusion module that progressively blends and aligns cross-modal conditioning streams between the latent variables of the sketch and point cloud representations. Additionally, by incorporating a guided sampling strategy for diffusion modeling, our SEN allows for more flexible control, even with complex or limited conditioning patterns. Our method significantly improves the geometric coherence of the synthesized parts while preserving the shapes of unedited areas. We demonstrate the advantages of our approach qualitatively and quantitatively across various shape categories, outperforming state-of-the-art methodologies.

BibTeX

@article{aizawa2026towards,
  title={Towards Consistent Sketch-Guided Local 3D Shape Editing},
  author={Aizawa, Tomohiro and Gu, Chunzhi and Kuriyama, Shigeru},
  journal={Pattern Recognition},
  pages={113095},
  year={2026},
  publisher={Elsevier}
}