The busy Java developer’s guide to Scala: Building a calculator, Part 2

Domain-specific languages (DSLs) have become a hot topic; much of the buzz around functional languages is their ability to build such languages. In this latest installment, Ted Neward tackles the problem of transforming textual input into the AST for interpretation by continuing with a simple calculator DSL that demonstrates the power of functional languages for building “external” DSLs. To parse textual input and transform it into the tree structure used by the interpreter in the last article, Ted introduces parser combinators, a standard Scala library designed solely for the task. (In the previous article, we built a calculator interpreter and AST.)

Jim McBeath: Scala Parser Combinators

Scala Parser Combinators
One of the reasons I chose to do my StringArt applet as my first Scala applet was because Scala includes a nifty capability called parser combinators.

A parser is a function that converts a stream of input tokens into a data structure that can be more easily digested by the application. A combinator is a way of combing two functions to produce a composed function. A parser combinator is a way of combining two parsers to produce another parser.

The Parser classes in Scala include methods to combine Parsers in various ways and to add functions to be executed when a parse succeeds, to build the resulting datastructure.

Parser combinators simplify the creation of some parsers, including the simple expression parsing I wanted. When using Scala, the specification of the syntax to be parsed, which might normally be done in a separate file which is read by a separate parser (as when using a parser generator such as yacc), can be done directly in a Scala source file.

Scala’s syntax has two features that make it possible to write a syntax specification directly in Scala that still looks a lot like a BNF description.

1. (Almost) any operator string can be used as a method name.
2. A method that takes a single argument can be written without parentheses.

The parser combinator class is designed to be able to generate an abstract syntax tree (AST) representing the parse, which can then be easily manipulated by the application. In the following example, we create a parser that builds an AST which we then evaluate to produce a value.

The Magic Behind Parser Combinators - Code Commit

The Magic Behind Parser Combinators

If you’re like me, one of the first things that attracted you to Scala was its parser combinators. Well, maybe that wasn’t the first thing for me, but it was pretty far up there. Parser combinators make it almost too easy to create a parser for a complex language without ever leaving the comfortable play-pen afforded by Scala. Incidentally, if you aren’t familiar with the fundamentals of text parsing, context-free grammars and/or parser generators, then you might want to do some reading before you continue with this article.
Intro to Parser Combinators

In most languages, the process of creating a text parser is usually an arduous and clumsy affair involving a parser generator (such as ANTLR, JavaCC, Beaver or ScalaBison) and (usually) a lexer generator such as JFlex. These tools do a very good job of generating sources for efficient and powerful parsers, but they aren’t exactly the easiest tools to use. They generally have a very steep learning curve, and due to their unique status as compiler-compilers, an unintuitive architecture. Additionally, these tools can be somewhat rigid, making it very difficult to implement unique or experimental features. For this reason alone, many real world compilers and interpreters (such as javac, ruby, jruby and scalac) actually use hand-written parsers. These are usually easier to tweak, but they can be very difficult to develop and test. Additionally, hand-written parsers have a tendency toward poor performance (think: the Scala compiler).

When creating a compiler in Scala, it is perfectly acceptable to make use of these conventional Java-generating tools like ANTLR or Beaver, but we do have other options. Parser combinators are a domain-specific language baked into the standard library. Using this internal DSL, we can create an instance of a parser for a given grammar using Scala methods and fields. What’s more, we can do this in a very declarative fashion. Thanks to the magic of DSLs, our sources will actually look like a plain-Jane context-free grammar for our language. This means that we get most of the benefits of a hand-written parser without losing the maintainability afforded by parser generators like bison. For example, here is a very simple grammar for a simplified Scala-like language, expressed in terms of parser combinators:

Ruminations of a Programmer: External DSLs made easy with Scala Parser Combinators

External DSLs are hard since implementing them involves reinventing most of the mechanisms found in a general purpose language. Designing internal DSLs are equally hard, more so in a statically typed language. Dynamically typed languages like Ruby offer strong meta-programming facilities, which help in implementing internal DSLs. But metaprogramming in Ruby is still considered elitist by many, and is not an art mastered by programmers at large.

Parser combinators offer a unique value here. They allow programmers to write executable grammars, in the sense that designing and implementing a DSL is almost equivalent to writing the EBNF notations in the syntax of the native language. So what really are parser combinators and what kind of language support do we need to implement parser combinator libraries ? Here is how Gilad Bracha describes them ..

The basic idea is to view the operators of BNF (or regular expressions for that matter) as methods that operate on objects representing productions of a grammar. Each such object is a parser that accepts the language specified by a particular production. The results of the method invocations are also such parsers. The operations are called combinators for rather esoteric technical reasons (and to intimidate unwashed illiterates wherever they might lurk).

Combinators have their theoretical underpinnings in functional programming. A parser combinator is a higher order function that accepts a parser and applies transformation functions to generate more complex parsers. Hence parser combinators are easily implemented in languages that have strong support for functional programming. In Scala, parsers are implemented as monads - hence defining combinators for parsers are just monadic transformations implementing sequencing, alternation or any other composition operations.

This post is not an introductory material on parser combinators or their implementations in Scala. Here I would like to narrate my experience in designing an external DSL for a financial application using the parser combinator library of Scala. In one of my earlier posts, I had talked about monads as abstractions for containers and computations. Parser combinator implementation in Scala is a great example of the power of monads in evolving a DSL.

In developing a financial application involving securities trade and settlement processing, we’ve been using XML as the DSL for getting buy/sell orders from client, trade/execution information from the exchange and settlement information from clearing agencies. Needless to say, XML processing is one of the key functions that pervade our codebase. In one of my very early posts, I had ranted about executable XMLs (aka Lisp) and had considered using Scheme as the DSL for securities trade processing operations. SISC offers a fully compliant Scheme implementation on top of the JVM, still all ideas fell through the cracks of enterprise decision making deliberations. After a fairly long time, I have found out another alternative - DSLs designed using Scala parser combinators ..

Home - scalamodules - GitHub

Home

ScalaModules aims at Scala-based OSGi development. The mission of ScalaModules is to employ the power of the Scala programming language to ease OSGi development.

Of course using Scala for OSGi will itself be beneficial, because of the great simplifications Scala brings compared to Java. But ScalaModules will also make use of the additional possibilities offered by Scala, mainly the chance to create a Domain Specific Language.

Therefore with ScalaModules your code will be more intuitive and concise as well as less verbose and less involved compared to Java-based OSGi development. Check it out!

Heiko Seeberger

Heiko Seeberger

Welcome to my blog!

Brizzled: SBT: A Scala-based Simple Build Tool

Awhile ago, I embarked on an effort to build yet another build tool, this one in Scala. I shelved that effort, and I’ve switched to SBT, instead.

I’ve had it with Ant, for all the reasons outlined in my previous build tool discussion, and then some.

I looked at Maven and Ivy, but, really, I am so over XML configuration files. (Again, see previous rant.)

At one suggestion, I began to look at Gradle, a Groovy build tool with Maven, Ivy and Ant integration. Now this was looking promising: A simple code-oriented configuration (no XML! Yay!), with complete access to Ant tasks and full external dependency management.

Before diving feet-first into Gradle, though, I decided to take a closer look at SBT, a tool Daniel Spiewak recommended. SBT bills itself as “a simple build tool for Scala projects that aims to do the basics well.” SBT uses Scala, much the same way that Gradle uses Groovy; this is appealing to me, since I’m doing most of my “elective programming” in Scala these days. After digging a bit, I found that SBT has complete support for external dependency management.

Configgy: Scala configuration and logging

Configgy is a library for handling config files and logging for a scala daemon. The idea is that it should be simple and straightforward, allowing you to plug it in and get started quickly, writing small useful daemons without entering the shadowy world of java frameworks.

Features for configuration include:

* very simple config file format
* values can be bools, ints, strings, or lists of strings
* sets of key/value pairs can be grouped into blocks and nested inside each other
* some blocks can inherit default values from other blocks
* values can depend on previously-set values, or environment variables
* files can be “imported” into each other
* scala API for subscribing to a config node and receiving notification of changes
* JMX hooks for viewing/changing config in realtime

Features for logging include:

* simple API for writing log messages
* configured via the config file
* log to multiple files, console, or syslog servers
* automatic logfile rolling

Configgy: Scala configuration and logging

Configgy

Configgy is a library for handling config files and logging for a scala daemon. The idea is that it should be simple and straightforward, allowing you to plug it in and get started quickly, writing small useful daemons without entering the shadowy world of java frameworks.

Features for configuration include:

* very simple config file format
* values can be bools, ints, strings, or lists of strings
* sets of key/value pairs can be grouped into blocks and nested inside each other
* some blocks can inherit default values from other blocks
* values can depend on previously-set values, or environment variables
* files can be “imported” into each other
* scala API for subscribing to a config node and receiving notification of changes
* JMX hooks for viewing/changing config in realtime

Features for logging include:

* simple API for writing log messages
* configured via the config file
* log to multiple files, console, or syslog servers
* automatic logfile rolling