Abstract
Modeling complex 3D shapes from 2D sketch images has gained significant attention due to recent advancements in deep generative models. While many efforts have focused on producing geometries that faithfully reflect sketches, there has been limited exploration of sketch-guided local editing. Existing shape generation techniques often modify not only the intended edited areas but also the unedited parts, compromising geometric coherence and fairness among different parts of the shape. This paper introduces a shape editing network (SEN) under the diffusion-based generative framework to specifically enhance consistency in sketch-guided local shape editing. Our core insight is capturing partial geometric structure correlations to achieve consistent editing results. To accomplish this, the SEN is designed to conditionally learn local edits of a partial shape while considering the geometries of the unedited parts and the sketch image. We also devise an attention fusion module that progressively blends and aligns cross-modal conditioning streams between the latent variables of the sketch and point cloud representations. Additionally, by incorporating a guided sampling strategy for diffusion modeling, our SEN allows for more flexible control, even with complex or limited conditioning patterns. Our method significantly improves the geometric coherence of the synthesized parts while preserving the shapes of unedited areas. We demonstrate the advantages of our approach qualitatively and quantitatively across various shape categories, outperforming state-of-the-art methodologies.
Overview of Shape Edting Network (SEN). Given the input partial shape (a) and the image (c) sketching the target shape, SEN iteratively regresses the noise residual (d) of (b) to eventually produce the editing result (e), which can then be composed to (a) to constitute the full 3D shape (f). Particularly, the Attention Fusion Model, which comprises a series of Attention Fusion Units, progressively merges cross-modal embeddings to understand the two-fold conditioning streams.
Qualitative results across different shape categories.
Qualitative results against prior shape modeling methods
Fourth image description.
BibTeX
@article{aizawa2026towards,
title={Towards Consistent Sketch-Guided Local 3D Shape Editing},
author={Aizawa, Tomohiro and Gu, Chunzhi and Kuriyama, Shigeru},
journal={Pattern Recognition},
pages={113095},
year={2026},
publisher={Elsevier}
}