Sketch-based Interfaces and Modeling

Sketch-based Interfaces and Modeling

Foreword

Preface

Contents

Introduction

Sketch-based Interfaces

Sketching Issues and Research Topics in HCI

Recognition

Modes

Sketch-based Applications

Creation and Modification of 3D Models

Suggestive Systems

Constructive Systems

Augmentation

Deformation

Modeling Applications

References

Sketch-based Interfaces

Multi-domain Hierarchical Free-Sketch Recognition Using Graphical Models

Introduction

The Challenges of Free-Sketch Recognition

Knowledge Representation

Hierarchical Shape Descriptions

Handling Noise in the Drawing

Signal-Level Noise: Objective vs. Subjective Measures

Description-Level Variation: Optional Components and Constraints

Strengths and Limitations

Recognition Overview

Hypothesis Evaluation

Dynamically Constructed Graphical Models

Shape Fragments: Evaluating a Single Hypothesis

Network Structure

Conditional Probability Distributions

Observing Evidence from Stroke Data

Recognizing a Complete Sketch

Linking Shape Fragments

Missing Nodes

Implementation and Bayesian Inference

Hypothesis Generation

Selecting an Interpretation

Application and Results

Remaining Challenges and Extensions

Using Single-Stroke Classification to Improve Grouping

Conclusion

References

Minimizing Modes for Smart Selection in Sketching/Drawing Interfaces

Introduction

The Cost of Modes

The Temporal Cost of Modes

Mode Errors: The Mode Problem

Overloaded Loop Selection: UI Design to Infer Selection Mode

The Inferred Mode Protocol for Stylus Drawing and Selection with a Pen

The Mode Problem in Electronic Whiteboard Programs

Analytical Tool: The Interaction Flow Diagram

Interaction Flow Analysis of Mode-Based Selection and Drawing

Inferred Mode Protocol: Inferring Draw/Select Mode

Sloppy Selection: Inferring Intended Content of an Ambiguous Selection

Cycle Tap Selection: Exploiting Structure Recognition

Conclusion

References

Mathematical Sketching: An Approach to Making Dynamic Illustrations

Introduction

Mathematical Sketching

Philosophical Considerations

Generalizing Mathematical Sketching as a Paradigm

Observations on Mathematical Sketching

The MathPad2 Prototype

MathPad2 Architecture

The Gestural UI

Writing, Recognizing, and Correcting Mathematics

Making Drawings

Nailing Diagram Components

Grouping Diagram Components

Associations

Implicit Associations

Explicit Associations

Supporting Mathematical Toolset

Graphing Equations

Solving Equations

Evaluating Expressions

Mathematical Expression Recognition

Mathematical Symbol Recognition

Mathematical Expression Parsing

Preparing Mathematical Sketches

Association Inferencing

Drawing Dimension Analysis

Drawing Rectification

Angle Rectification

Location Rectification

Size Rectification

Translating and Visualizing Mathematical Sketches

Moving Forward with Mathematical Sketching

The Computational and Symbolic Toolset

Functions and Macros

Moving to 3D

Interactivity

Generating Mathematics from Drawings

Adding Specific Underlying Mathematical Engines

Alternate Forms of Dynamic Illustration

Evaluation

Conclusion

References

Pen-based Interfaces for Engineering and Education

Introduction

Sketch Parsing

Mark-Group-Recognize

Enumerate-Recognize-Prune

Parsing Step 1: Enumerating Candidate Symbols

Ink Density Locator:

Segment Difference Locator:

Parsing Step 2: Pruning Using Domain Knowledge

Automated Error Correction

Recognition

Graph-based Recognizer

Representation

Measuring Similarity

Primitive Count Error:

Primitive Type Error:

Relative Length Error:

Number of Intersections Error:

Intersection Angle Error:

Intersection Location Error:

Graph Matching

Stochastic Matching

Greedy Matching

Hybrid Matching

Training

Feature-based Recognizer

Image-based Recognizer

Representation

Template Matching with Multiple Classifiers

Hausdorff Distance:

Modified Hausdorff Distance:

Tanimoto Coefficient:

Yule Coefficient:

Combining Classifiers:

Handling Rotations

Educational Applications

Conclusion

References

Flexible Parts-based Sketch Recognition

Introduction

Algorithm Design Issues

Image-based vs. Stroke-based

Use of Timing Information

Top-Down vs. Bottom-Up

Object Template Representation

Degree of Supported Variation

Search Algorithm

Recognition as Search

Related Work

Hierarchy-of-Parts Models

Recognition Algorithm

Template Construction

Template Matching

Curve Matching

Template Hierarchy

Application to 3D Model Construction

Results and Discussion

Failure Modes

Comparison with Template Editing

Scalability

Constellation of Parts Models

Learning the Model

Labeling Likelihood

Recognition Algorithm

Results and Discussion

Conclusions

References

Sketch-based Retrieval of Vector Drawings

Introduction

Feature Extraction from Sketches and Drawings

Topology

Topological Relationships

Topology Graph

From Topology Graphs to Descriptors

Multilevel Description

Spatial Proximity

Geometry

Application Examples for 2D Drawings

Toward 3D Modeling Using Implicit Retrieval

Modeling 3D Technical Objects

Overview of the Modeling Tool

3D Object Description

Query Formulation and Execution

Modeling Lego Scenes

Part Library

Retrieval Mechanism

Results Presentation

Conclusions

References

Sketch-based Modeling

A Sketching Interface for Freeform 3D Modeling

Introduction

Related Work

User Interface

Modeling Operations

Overview

Creating a New Object

Painting and Erasing on the Surface

Extrusion

Cutting

Smoothing

Transformation

Algorithm

Creating a New Object

Painting on the Surface

Extrusion

Cutting

Smoothing

Implementation

User Experience

Conclusions

References

The Creation and Modification of 3D Models Using Sketches and Curves

Introduction

FiberMesh: Designing Freeform Surfaces with 3D Curves

User Interface

Sketching Tool

Deformation Tool

Rubbing Tool

Erasing Tool and Type Change Tool

Algorithm

Curve Deformation

Surface Optimization

Meshing Implementations

Results

Discussion

SilSketch: Automated Sketch-based Editing of Surface Meshes

Interface

Image-Space Silhouettes

Silhouette Detection

Silhouette Extraction

Handle Estimation

Finding Handle/Target Correspondences

ROI Estimation

Results

Discussion

Conclusion

References

Sketch-based Modeling and Assembling with Few Strokes

Introduction

Related Work

Stroke Capture

Creation Phase

Rotational Blending Surface

Cross-Sectional Blending Surfaces

Editing Phase

Orthogonal Deformation Stroke

Cross-Sectional Oversketch

Transformation Stroke

Stroke Interpretation

Translation

Active Model

Rotation

Scaling

Results and Discussion

Gestural Modeling

Transformation Stroke

Conclusions

References

ShapeShop: Free-Form 3D Design with Implicit Solid Modeling

Introduction

The ShapeShop Interface

Pencil-based Interaction

Sketching Assistance

Sketch-based Modeling Operations

Blobby Inflation

Sweep Surfaces

Cutting

Blending

Surface Drawing

Selection and Transformation

Technical Details

Hierarchical Implicit Volume Modeling

BlobTree Visualization

Sketchable Implicit Sweep Primitives

Hierarchical Spatial Caching

The ShapeShop System

Discussion

References

Inferring 3D Free-Form Shapes from Complex Contour Drawings

Introduction

Overview and Background

Related Work

Shape from Drawings.

Contour Completion.

Sketching Interfaces.

Notation and Problem Formulation

Figural Completion for Smooth Surfaces

Preprocessing an Input Stroke and Guessing T-points and Cusps

Pairwise Completion

Computing the Completion for a Tee/Cusp Pairing

Greedy Search for the Best Configuration

Checking the Consistency of the Pairing

Gluing Segments and Assigning Huffman's Labels

Results and Limitations of the Figural Completion Algorithm

From Drawing to Williams' Abstract Topological Manifold

Triangulating the Panels; The Issue of Two Distinct Points Having the Same 2D Location

Paneling Construction

Creating Multiple Copies of Each Panel

Establishing Correspondences Between Edges of the Panels

Ordering the Clusters.

Extra Vertices.

Constructing a Topological Embedding

Embedding Vertices.

Embedding edges.

Embedding faces.

Smoothing the Embedding Using the Mass-Spring System

Editing Gestures

Limitations and Conclusions

References

The Creation and Modification of 3D Models Using Sketches and Curves

Introduction

Sketch-based Creation and Modification of 3D Shapes

Constructing the 3D Wireframe

Modifying the Wireframe

Curve Modification in the Image Plane

Unprojection to 3D

Surface Creation and Modification

Initial Surface Creation

Surface Modification

Surface Modification Using Pressure Force

Surface Modification Using V-Spring Method

Examples

Creating 3D Shape Templates from Sketches for Automotive Styling Design

Overview

Template Alignment

Template Deformation Based on Fiducial Points

Edge Representation and Manipulation

Surface Representation and Manipulation

Examples

Conclusions

References

Dressing and Hair-Styling Virtual Characters from a Sketch

Introduction

Sketching in Distance Fields: Application to Garment Design

Expressing Prior Knowledge

The Sketch-based Interface

Typical Garment Design Session

Contour Mode

Front/Back Modes

Vertical Symmetry.

Gestural Interface Components.

Construction of the Garment Surface in 3D

Distance Field

Converting the 2D Contours into 3D

Surface Generation from 3D Contours

Mimicking Cloth Tension.

Drawing Folds

Folding Mode.

Incorporating Geometric Properties: Sketch-based Modeling of Developable Surfaces

Expressing Prior Knowledge

Sketching Seams and Darts

Creating a Developable Surface via Approximation

Creating a Developable Surface Directly from the 3D Boundary Lines

Automatic Generation of Folds

Physically-based Simulation of Folds

Procedural Generation of Folds

Sketch-based Interface for a Physically-based System: Hairstyle Design from a Sketch

Expressing Prior Knowledge

The Sketch-based Interface

Hairline

Example Strands

Volume and Cut

Shaping the Hair in 3D

Determining Helical Parameters

Generalizing to a Full Head of Hair

Setting the Volume, Adjusting the Cut

Discussion and Concluding Remarks

Mapping to a Procedural Model

Simplifying Assumptions

Non-intrusive Sketching Interface

Sketching vs. Annotation

References

Index