Domain-Specific Languages

When carefully selected and used, Domain-Specific Languages (DSLs) may simplify complex code, promote effective communication with customers, improve productivity, and unclog development bottlenecks. In Domain-Specific Languages, noted software development expert Martin Fowler first provides the information software professionals need to decide if and when to utilize DSLs. Then, where DSLs prove suitable, Fowler presents effective techniques for building them, and guides software engineers in choosing the right approaches for their applications.   This book’s techniques may be utilized with most modern object-oriented languages; the author provides numerous examples in Java and C#, as well as selected examples in Ruby. Wherever possible, chapters are organized to be self-standing, and most reference topics are presented in a familiar patterns format.   Armed with this wide-ranging book, developers will have the knowledge they need to make important decisions about DSLs—and, where appropriate, gain the significant technical and business benefits they offer.   The topics covered include: How DSLs compare to frameworks and libraries, and when those alternatives are sufficient Using parsers and parser generators, and parsing external DSLs Understanding, comparing, and choosing DSL language constructs Determining whether to use code generation, and comparing code generation strategies Previewing new language workbench tools for creating DSLs

Table of Contents:
Preface xix Part I: Narratives 1 Chapter 1: An Introductory Example 3 Gothic Security 3 The State Machine Model 5 Programming Miss Grant’s Controller 9 Languages and Semantic Model 16 Using Code Generation 19 Using Language Workbenches 22 Visualization 24 Chapter 2: Using Domain-Specific Languages 27 Defining Domain-Specific Languages 27 Why Use a DSL? 33 Problems with DSLs 36 Wider Language Processing 39 DSL Lifecycle 40 What Makes a Good DSL Design? 42 Chapter 3: Implementing DSLs 43 Architecture of DSL Processing 43 The Workings of a Parser 47 Grammars, Syntax, and Semantics 49 Parsing Data 50 Macros 52 Chapter 4: Implementing an Internal DSL 67 Fluent and Command-Query APIs 68 The Need for a Parsing Layer 71 Using Functions 72 Literal Collections 77 Using Grammars to Choose Internal Elements 79 Closures 80 Parse Tree Manipulation 82 Annotation 84 Literal Extension 85 Reducing the Syntactic Noise 85 Dynamic Reception 86 Providing Some Type Checking 87 Chapter 5: Implementing an External DSL 89 Syntactic Analysis Strategy 89 Output Production Strategy 92 Parsing Concepts 94 Mixing-in Another Language 100 XML DSLs 101 Chapter 6: Choosing between Internal and External DSLs 105 Learning Curve 105 Cost of Building 106 Programmer Familiarity 107 Communication with Domain Experts 108 Mixing In the Host Language 108 Strong Expressiveness Boundary 109 Runtime Configuration 110 Sliding into Generality 110 Composing DSLs 111 Summing Up 111 Chapter 7: Alternative Computational Models 113 A Few Alternative Models 116 Chapter 8: Code Generation 121 Choosing What to Generate 122 How to Generate 124 Mixing Generated and Handwritten Code 126 Generating Readable Code 127 Preparse Code Generation 128 Further Reading 128 Chapter 9: Language Workbenches 129 Elements of Language Workbenches 130 Schema Definition Languages and Meta-Models 131 Source and Projectional Editing 136 Illustrative Programming 138 Tools Tour 140 Language Workbenches and CASE tools 141 Should You Use a Language Workbench? 142 Part II: Common Topics 145 Chapter 10: A Zoo of DSLs 147 Graphviz 147 JMock 149 CSS 150 Hibernate Query Language (HQL) 151 XAML 152 FIT 155 Make et al. 156 Chapter 11: Semantic Model 159 How It Works 159 When to Use It 162 The Introductory Example (Java) 163 Chapter 12: Symbol Table 165 How It Works 166 When to Use It 168 Further Reading 168 Dependency Network in an External DSL (Java and ANTLR) 168 Using Symbolic Keys in an Internal DSL (Ruby) 170 Using Enums for Statically Typed Symbols (Java) 172 Chapter 13: Context Variable 175 How It Works 175 When to Use It 176 Reading an INI File (C#) 176 Chapter 14: Construction Builder 179 How It Works 179 When to Use It 180 Building Simple Flight Data (C#) 180 Chapter 15: Macro 183 How It Works 184 When to Use It 192 Chapter 16: Notification 193 How It Works 194 When to Use It 194 A Very Simple Notification (C#) 194 Parsing Notification (Java) 195 Part III: External DSL Topics 199 Chapter 17: Delimiter-Directed Translation 201 How It Works 201 When to Use It 204 Frequent Customer Points (C#) 205 Parsing Nonautonomous Statements with Miss Grant’s Controller (Java) 211 Chapter 18: Syntax-Directed Translation 219 How It Works 220 When to Use It 227 Further Reading 227 Chapter 19: BNF 229 How It Works 229 When to Use It 238 Chapter 20: Regex Table Lexer (by Rebecca Parsons) 239 How It Works 240 When to Use It 241 Lexing Miss Grant’s Controller (Java) 241 Chapter 21: Recursive Descent Parser (by Rebecca Parsons) 245 How It Works 246 When to Use It 249 Further Reading 249 Recursive Descent and Miss Grant’s Controller (Java) 250 Chapter 22: Parser Combinator (by Rebecca Parsons) 255 How It Works 256 When to Use It 261 Parser Combinators and Miss Grant’s Controller (Java) 261 Chapter 23: Parser Generator 269 How It Works 269 When to Use It 272 Hello World (Java and ANTLR) 272 Chapter 24: Tree Construction 281 How It Works 281 When to Use It 284 Using ANTLR’s Tree Construction Syntax (Java and ANTLR) 284 Tree Construction Using Code Actions (Java and ANTLR) 292 Chapter 25: Embedded Translation 299 How It Works 299 When to Use It 300 Miss Grant’s Controller (Java and ANTLR) 300 Chapter 26: Embedded Interpretation 305 How It Works 305 When to Use It 306 A Calculator (ANTLR and Java) 306 Chapter 27: Foreign Code 309 How It Works 309 When to Use It 311 Embedding Dynamic Code (ANTLR, Java, and Javascript) 311 Chapter 28: Alternative Tokenization 319 How It Works 319 When to Use It 326 Chapter 29: Nested Operator Expression 327 How It Works 327 When to Use It 331 Chapter 30: Newline Separators 333 How It Works 333 When to Use It 335 Chapter 31: External DSL Miscellany 337 Syntactic Indentation 337 Modular Grammars 339 Part IV: Internal DSL Topics 341 Chapter 32: Expression Builder 343 How It Works 344 When to Use It 344 A Fluent Calendar with and without a Builder (Java) 345 Using Multiple Builders for the Calendar (Java) 348 Chapter 33: Function Sequence 351 How It Works 351 When to Use It 352 Simple Computer Configuration (Java) 352 Chapter 34: Nested Function 357 How It Works 357 When to Use It 359 The Simple Computer Configuration Example (Java) 360 Handling Multiple Different Arguments with Tokens (C#) 361 Using Subtype Tokens for IDE Support (Java) 363 Using Object Initializers (C#) 365 Recurring Events (C#) 366 Chapter 35: Method Chaining 373 How It Works 373 When to Use It 377 The Simple Computer Configuration Example (Java) 378 Chaining with Properties (C#) 381 Progressive Interfaces (C#) 382 Chapter 36: Object Scoping 385 How It Works 386 When to Use It 386 Security Codes (C#) 387 Using Instance Evaluation (Ruby) 392 Using an Instance Initializer (Java) 394 Chapter 37: Closure 397 How It Works 397 When to Use It 402 Chapter 38: Nested Closure 403 How It Works 403 When to Use It 405 Wrapping a Function Sequence in a Nested Closure (Ruby) 405 Simple C# Example (C#) 408 Using Method Chaining (Ruby) 409 Function Sequence with Explicit Closure Arguments (Ruby 411 Using Instance Evaluation (Ruby) 412 Chapter 39: Literal List 417 How It Works 417 When to Use It 417 Chapter 40: Literal Map 419 How It Works 419 When to Use It 420 The Computer Configuration Using Lists and Maps (Ruby) 420 Evolving to Greenspun Form (Ruby) 422 Chapter 41: Dynamic Reception 427 How It Works 428 When to Use It 429 Promotion Points Using Parsed Method Names (Ruby) 430 Promotion Points Using Chaining (Ruby) 434 Removing Quoting in the Secret Panel Controller (JRuby) 438 Chapter 42: Annotation 445 How It Works 446 When to Use It 449 Custom Syntax with Runtime Processing (Java) 449 Using a Class Method (Ruby) 451 Dynamic Code Generation (Ruby) 452 Chapter 43: Parse Tree Manipulation 455 How It Works 455 When to Use It 456 Generating IMAP Queries from C# Conditions (C#) 457 Chapter 44: Class Symbol Table 467 How It Works 468 When to Use It 469 Statically Typed Class Symbol Table (Java) 469 Chapter 45: Textual Polishing 477 How It Works 477 When to Use It 478 Polished Discount Rules (Ruby) 478 Chapter 46: Literal Extension 481 How It Works 481 When to Use It 482 Recipe Ingredients (C#) 483 Part V: Alternative Computational Models 485 Chapter 47: Adaptive Model 487 How It Works 488 When to Use It 492 Chapter 48: Decision Table 495 How It Works 495 When to Use It 497 Calculating the Fee for an Order (C#) 497 Chapter 49: Dependency Network 505 How It Works 506 When to Use It 508 Analyzing Potions (C#) 508 Chapter 50: Production Rule System 513 How It Works 514 When to Use It 517 Validations for club membership (C#) 517 Eligibility Rules: extending the club membership (C#) 521 Chapter 51: State Machine 527 How It Works 527 When to Use It 529 Secret Panel Controller (Java) 530 Part VI: Code Generation 531 Chapter 52: Transformer Generation 533 How It Works 533 When to Use It 535 Secret Panel Controller (Java generating C) 535 Chapter 53: Templated Generation 539 How It Works 539 When to Use It 541 Generating the Secret Panel State Machine with Nested Conditionals (Velocity and Java generating C) 541 Chapter 54: Embedment Helper 547 How It Works 548 When to Use It 549 Secret Panel States (Java and ANTLR) 549 Should a Helper Generate HTML? (Java and Velocity) 552 Chapter 55: Model-Aware Generation 555 How It Works 556 When to Use It 556 Secret Panel State Machine (C) 557 Loading the State Machine Dynamically (C) 564 Chapter 56: Model Ignorant Generation 567 How It Works 567 When to Use It 568 Secret Panel State Machine as Nested Conditionals (C) 568 Chapter 57: Generation Gap 571 How It Works 571 When to Use It 573 Generating Classes from a Data Schema (Java and a Little Ruby) 573 Bibliography 579 Index 581