What I should have said

Defining an Operator Invoke Extension Function on Kotlin Function Types

2022-08-18T00:00:00-05:00

I’ve been producing a lot of YouTube Videos lately. They’re about refactoring and TDD and all that good stuff - you really should subscribe.

In one episode I have a class that takes a function type property in its constructor. The property defines a strategy for how to update items.

Here I’m going to go off on the tangent that Jekyll’s home-page generation fails if you have a code block that is split on the front page. So this text is just to move this code block down the page:

class Stock(
    private val stockFile: File,
    private val zoneId: ZoneId,
    private val updater: (items: List<Item>, days: Int, on: LocalDate) -> List<Item>
) {
    fun stockList(now: Instant): Result4k<StockList, StockListLoadingError> =
        stockFile.loadItems().flatMap { stockList: StockList ->
            val daysOutOfDate = stockList.lastModified.daysTo(now, zoneId).toInt()
            when {
                daysOutOfDate > 0 -> update(stockList, updater, now, daysOutOfDate, zoneId).savedTo(stockFile, now)
                else -> Success(stockList)
            }
        }
}

private fun update(
    stockList: StockList,
    updater: (items: List<Item>, days: Int, on: LocalDate) -> List<Item>,
    now: Instant,
    daysOutOfDate: Int,
    zoneId1: ZoneId
): StockList = stockList.copy(
    lastModified = now,
    items = updater(stockList.items, daysOutOfDate, LocalDate.ofInstant(now, zoneId1))
)

So a Stock is an object that can load a StockList from a file, updating the stock Items using a strategy defined by the updater function. It’s all a bit of a tangle, yes, but this is the middle of a refactor.

The nastiness of that update function is because updater has the wrong signature as far this code is concerned. We need it to work on a StockList, but it is defined on List<Item> because that’s a simpler interface for our clients.

So here’s a ‘cute’ trick. Take the updater parameter to update and make it the receiver.

...
                daysOutOfDate > 0 -> updater.update(stockList, now, daysOutOfDate, zoneId).savedTo
...
private fun ((items: List<Item>, days: Int, on: LocalDate) -> List<Item>).update(
    stockList: StockList,
    now: Instant,
    daysOutOfDate: Int,
    zoneId1: ZoneId
): StockList = stockList.copy(
    lastModified = now,
    items = this(stockList.items, daysOutOfDate, LocalDate.ofInstant(now, zoneId1))
)

Now convert it to an operator fun invoke extension

...
                daysOutOfDate > 0 -> updater.invoke(stockList, now, daysOutOfDate, zoneId).savedTo(stockFile, now)
...

private operator fun ((items: List<Item>, days: Int, on: LocalDate) -> List<Item>).invoke(
    stockList: StockList,
    now: Instant,
    daysOutOfDate: Int,
    zoneId1: ZoneId
): StockList = stockList.copy(
    lastModified = now,
    items = this(stockList.items, daysOutOfDate, LocalDate.ofInstant(now, zoneId1))
)

and finally remove the vestigial invoke:

class Stock(
    private val stockFile: File,
    private val zoneId: ZoneId,
    private val updater: (items: List<Item>, days: Int, on: LocalDate) -> List<Item>
) {
    fun stockList(now: Instant): Result4k<StockList, StockListLoadingError> =
        stockFile.loadItems().flatMap { stockList: StockList ->
            val daysOutOfDate = stockList.lastModified.daysTo(now, zoneId).toInt()
            when {
                daysOutOfDate > 0 -> updater(stockList, now, daysOutOfDate, zoneId).savedTo(stockFile, now)
                else -> Success(stockList)
            }
        }
}

private operator fun ((items: List<Item>, days: Int, on: LocalDate) -> List<Item>).invoke(
    stockList: StockList,
    now: Instant,
    daysOutOfDate: Int,
    zoneId1: ZoneId
): StockList = stockList.copy(
    lastModified = now,
    items = this(stockList.items, daysOutOfDate, LocalDate.ofInstant(now, zoneId1))
)

So what we have is an operator invoke function defined as an extension on a function type. This way we have ‘seamlessly’ (well, there is a seam, but it is invisible) converted our updater from the function we have to the function we want.

To be honest, I’m pretty sure that the benefit is not worth the cleverness here, but I present the technique on the off-chance that it digs you out of a hole somewhere. If it does, please let me know on Twitter.

Writing Software to Write About Writing Software - Part 1

2021-10-05T00:00:00-05:00

O’Reilly recently published the Kotlin book I wrote with Nat Pryce: Java to Kotlin: A Refactoring Guidebook. I thought it might be interesting to write about the software we wrote to support our writing.

O’Reilly’s publishing system (Atlas) will accept Word, DocBook or AsciiDoc manuscripts. My experience trying to manage simultaneous work in large Word documents has not been great, although perhaps Office 365 is better - I’ve been generally impressed when I have used it. Being developers though, we naturally gravitated to plain-text formats, and our editors steered us towards AsciiDoc rather than the XML-based DocBook. AsciiDoc is a lot like Markdown in that the formatting works reasonably in plain text and can then be rendered into more sophisticated formats like HTML and PDF. Unlike Markdown, it only really has one dialect, but the tooling has migrated from Python to Ruby which has led to some versioning issues. We also discovered quite late in the day that Atlas is based on an old and relatively buggy version of AsciiDoctor (the toolchain), but by that time we were committed.

The Atlas publishing system hosts a Git repository, so you can push book source in the supported formats and then click a button in the UI to render a PDF, HTML or EPUB book, which is available to download after a minute or so. For reasons that we couldn’t fathom, it doesn’t just automatically build every commit. At first Nat built an AsciiDoctor Docker image so that we could render locally, but once we discovered that Atlas was both customised and out of date we settled on using it rather than a local build to be sure that we remained compatible with its idiosyncrasies.

Our book has many, many, many code examples. Obviously we want to make sure that these compile and work as expected, so just typing them into the manuscript wasn’t an option. In my previous attempt at writing a book I had chosen to embed the book text into comments in Kotlin source, so I could manage both text and source in IntelliJ. So I had Kotlin source like:

import org.junit.*

/*-
## Title

This is book text - anything in /*- … -*/ blocks is extracted to Markdown.

Code in //` blocks is also extracted.
-*/
class Test {

//`
    @Test fun `2 plus 2 is 4`() {
        assertEquals(4, add(2, 2))
    }
//`
}
/*-
More text here.
 */

This would render (to Markdown) as:

## Title

This is book text - anything in /*- … -*/ blocks is extracted to Markdown.

Code in //` blocks is also extracted.

```kotlin
@Test fun `2 plus 2 is 4`() {
    assertEquals(4, add(2, 2))
}
.```

More text here.

And no, there shouldn’t be a dot before the kotlin code block closure, but this blog is itself written in Markdown, which appears, at least in its Jekyll incarnation, to be unable to render itself. Apart from that sort of issue, this scheme worked OK, but it was a challenge to manage multiple versions of the same code. I bodged it by using hidden object namespaces and occasionally changing packages, but it was clear that the approach wasn’t ideal for a book which would be showing the evolution of code through multiple refactorings.

For our book then, Nat and I decided to separate the manuscript text and example code, pulling the text from the code into the AsciiDoc source with markers. So in our AsciiDoc source we might write:

A little compiler magic allows Kotlin code to accept a `java.util.List` as a `kotlin.collections.List`:

// begin-insert: src/test/java/collections/ListInteropTest.kt
// end-insert

In the best traditions of developer documentation we then wrote code to fetch the contents of ListInteropTest.kt and expand it into the AsciiDoc.

Ordinarily we would take the AsciiDoc source with these markers and write a new file with the inserted code. This new file would then be sent to AsciiDoctor to render. In practice this doesn’t work well, because we’d like the book source that we’re editing to be as close to the rendered output as reasonable (this is the point of ASCII-based markup after all), and in particular to be able to see the code that we’re writing about there in the editor where we are writing about it.

So we made the expansion software overwrite the AsciiDoc source document with its expanded version. By making the markers AsciiDoc comments we can leave them in the file when we expand them. They aren’t rendered in the final output, but remain in the book source so that we can navigate to the included file, and replace them with the new version when the code is changed.

The result is that after running the expansion code our AsciiDoc would be:

A little compiler magic allows Kotlin code to accept a `java.util.List` as a `kotlin.collections.List`:

// begin-insert: src/test/java/collections/ListInteropTest.kt
[source,kotlin]
----
val aList: List<String> = SomeJavaCode.mutableListOfStrings("0", "1")
aList.removeAt(1) // doesn't compile
----
// end-insert

This is a trick that has worked well for me in the past. Nat was resistant to the idea at first, and neither of us really liked having a tool that overwrites source files, but in this case it is expedient because it means that the code examples are expanded in the Git version, and we didn’t have to work out how to run some sort of pre-build step on Atlas. A handy side-effect is that if we modified the rendering code we could run it and see any behaviour changes as Git diffs to the AsciiDoc source rather than having to compare rendered output - the book source acts as a kind of approval test of its expansion code.

The observant reader might be wondering how ListInteropTest.kt compiles if aList.removeAt(1) doesn’t. Well I’ve been hiding a detail - the book source actually has:

// begin-insert: src/test/java/collections/ListInteropTest.kt#immutable
// end-insert

Here immutable is a marker that controls what from the code source is expanded into the book source. With no marker the whole file is inserted (minus import statements and leading and trailing whitespace). With a marker, well, here is the relevant section of the source file:

    @Test
    fun testKotlinCanAcceptJavaListAsImmutableList() {
        /// begin: immutable
        val aList: List<String> = SomeJavaCode.mutableListOfStrings("0", "1")
        /// end: immutable
        /*
        /// begin: immutable
        aList.removeAt(1) // doesn't compile
        /// end: immutable
        */

        assertEquals(listOf("0", "1"), aList)
    }

You can see that the code is a test that checks that the behaviour we are documenting is as we say it is. We have only rendered lines between /// begin and /// end markers with the same tag as the fragment. This gives the desired behaviour in this case, which is that the compiler doesn’t see the commented out code when we run our tests on the Kotlin source, and the book reader doesn’t see the comments that hide it from the compiler:

A little compiler magic allows Kotlin code to accept a `java.util.List` as a `kotlin.collections.List`:

// begin-insert: src/test/java/collections/ListInteropTest.kt
[source,kotlin]
----
val aList: List<String> = SomeJavaCode.mutableListOfStrings("0", "1")
aList.removeAt(1) // doesn't compile
----
// end-insert

We ended up with a set of ///-prefixed source markers to control what was rendered. /// begin and /// end we’ve seen, /// mute and /// resume acted in the opposite way and also allowed us to specify some text to show in place of the elided code. So our source Java might be:

/// begin: excerpt
public class Person {
    private final String firstName;
    private final String lastName;

    public Person(
        String firstName,
        String lastName
    ) {
        this.firstName = firstName;
        this.lastName = lastName;
    }

    /// mute: excerpt [... getters, equals, hashCode]
    public String getFirstName() { return firstName; }

    public String getLastName() { return lastName; }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        Person person = (Person) o;
        return firstName.equals(person.firstName) && lastName.equals(person.lastName);
    }

    @Override
    public int hashCode() {
        return Objects.hash(firstName, lastName, townOrCity, postalCode, countryCode);
    }
    /// resume: excerpt
}
/// end: excerpt

but the AsciiDoc would expand thus:

// begin-insert: src/main/java/builders/Person.java#excerpt
[source,java]
----
public class Person {
    private final String firstName;
    private final String lastName;

    public Person(
        String firstName,
        String lastName,
    ) {
        this.firstName = firstName;
        this.lastName = lastName;
    }

    ... getters, equals, hashCode
}
----
// end-insert

We flirted with other directives, but with a little creativity these were sufficient to cover all our needs for showing and hiding code. The snipping of code given a source file and marker name, and the expansion of the snipped code into the book source were performed by Kotlin invoked by a Gradle build. Inevitably this involved increasingly complex regular expressions and little territorial wars between Nat and me, but overall it worked well for us. You can see our code on GitHub, but the whole build system is probably an area where you’re best writing your own version so that you can customise it to make sense to you.

The tooling as described so far worked well for relatively static code examples, but our book is about refactoring, and in particular long-running sequences of refactorings to migrate from Java to idiomatic Kotlin. We needed to find a way to render multiple versions of the same code, and preferably to show the differences between code at different versions.

We toyed with keeping different versions of the code in different source roots, or different packages, but this turns out to be unsupportable for all the reasons you can probably predict. Instead, we decided to keep the history in Git, and extend the expansion code to fetch a particular version.

The simplest thing that could possibly work is just to include the Git SHA as well as the file path and marker in the AsciiDoc comments:

// begin-insert: 67b499c6:src/main/java/travelator/itinerary/Route.java#route
// end-insert

Now fetch src/main/java/travelator/itinerary/Route.java from the commit with SHA 67b499c6, apply the snippeting with the route marker, and expand the results inline in the AsciiDoc.

This turns out, though, to be too simple to actually work. The problem is that writing is an iterative process, and that applies both to the book text and the example code. If we were just documenting the history of a codebase it would be fine, but in practice we found ourselves having to go back to commits and edit them. Sometimes this would be to add snippeting markers or reformatting to fit the book pages, but often we would rework a refactoring from scratch. In any case, editing a commit changes its SHA, and if you rebase later commits, all their SHAs too, so that we would be constantly having to update many markers.

We needed something more stable than SHAs, and settled on commit comments. We prefixed the comments for code versions that we wanted to reference with, for example encapsulated-collections.0 and then said:

// begin-insert: encapsulated-collections.0:src/main/java/travelator/itinerary/Route.java#route
// end-insert

Now the expansion code just has to find a commit with a comment prefixed with encapsulated-collections.0. I say just, but in practice searching a repo for a commit is quite expensive, so in the end Nat wrote a script to apply tags with all the versions to these commits:

#!/bin/bash
set -eou pipefail

basedir="$(realpath $(dirname "$0"))"
current="$basedir/build/current-tagged"
last="$basedir/build/last-tagged"

cd ../refactoring-to-kotlin-code

function active_branches() {
  git show-ref --heads | grep -v 'refs/heads/attic/' | grep -v 'refs/heads/dummy-master'
}

active_branches | cut -d ' ' -f 1 | shasum > "$current"

if ! diff -q -N "$current" "$last" > /dev/null
then
    tagfmt='*.[0-9]*'

    # delete tags
    git tag -l "$tagfmt" | xargs git tag -d > /dev/null
    
    # Recreate tags
    git log --topo-order --format="%H %S %s" $(active_branches | cut -d ' ' -f 2) | awk '
        $3 ~ /([a-z0-9_\-]+)([.][0-9]+)+/ && $3 != tagname {
            commit = $1
            branch = $2
            tagname = $3

            if (tagname in commits && commits[tagname] != commit) {
                print "ERROR: " tagname " refers to different commits in branches " branch " and " branches[tagname] > "/dev/stderr"
                exit 1
            }

            branches[tagname] = branch
            commits[tagname] = commit

            print tagname, commit
        }
    ' | xargs -n 2 git tag -f
    mv "$current" "$last"
fi

No, I’ve no idea how it works either, but it does, and when it doesn’t, it’s Nat’s problem. My problem was fetching the version of a file with a particular tag (encapsulated-collections.0 in this case), which is just git show encapsulated-collections.0:src/main/java/travelator/itinerary/Route.java in the right directory. The expanding code runs this when it sees the tag prefix and then proceeds as before. If you’re interested the latest version of the expanding code is available on GitHub.

At this point we have a toolchain that allows us to fetch code at a particular version from a Git history and expand it, with some modifications, into the book source. We settled on a strategy of having each chapter in its own .asciidoc file: 06-java-to-kotlin-collections.asciidoc, 07-actions-to-calculations.asciidoc etc. This meant that we could each work on different chapters without merge issues. Naming the chapters with their number meant that we were constantly renaming them as we reorganised the book, but Git seems to cope well with renaming these days. All the chapters were pulled into a single text with a global AsciiDoc file that used include directives:

:xrefstyle: full
:sectnums:
:sectnumlevels: 2

[role="pagenumrestart"]

include::chapters/01-intro.asciidoc[]

// Foundation
include::chapters/02-java-to-kotlin-projects.asciidoc[]
include::chapters/03-java-to-kotlin-classes.asciidoc[]
include::chapters/04-optional-to-nullable.asciidoc[]
include::chapters/05-beans-to-values.asciidoc[]
include::chapters/06-java-to-kotlin-collections.asciidoc[]
include::chapters/07-actions-to-calculations.asciidoc[]

// Function structure
include::chapters/08-static-methods-to-top-level-functions.asciidoc[]
include::chapters/09-multi-to-single-expression-functions.asciidoc[]
include::chapters/10-functions-to-extension-functions.asciidoc[]
include::chapters/11-methods-to-properties.asciidoc[]
include::chapters/12-functions-to-operators.asciidoc[]

// Pipelines
include::chapters/13-streams-to-sequences.asciidoc[]
include::chapters/14-accumulating-objects-to-transformations.asciidoc[]

// Types
include::chapters/15-encapsulated-collections-to-typealiases.asciidoc[]
include::chapters/16-interfaces-to-functions.asciidoc[]
include::chapters/17-mocks-to-maps.asciidoc[]
include::chapters/18-open-to-sealed-classes.asciidoc[]

// Layers
include::chapters/19-throwing-to-returning.asciidoc[]
include::chapters/20-performing-io-to-passing-data.asciidoc[]
include::chapters/21-exceptions-to-values.asciidoc[]
include::chapters/22-classes-to-functions.asciidoc[]

// Conclusion
include::chapters/23-continuing-the-journey.asciidoc[]

include::bibliography.asciidoc[]

Actually, Atlas uses another file, atlas.json to specify what it will render and how. In our case this includes other files that make up the whole book but which O’Reilly manage:

{
  "files": [
    "cover.html",
    "praise.html",
    "titlepage.html",
    "copyright.html",
    "toc.html",
    "text/src/preface.asciidoc",
    "text/src/content.asciidoc",
    "ix.html",
    "author_bio.html",
    "colo.html"
  ]
  //...
}

We kept versioned example code in a separate repo to the book text and toolchain - that way we could manage the evolution of the text and examples separately. In the code repo we kept each running example in a separate branch, so that we could modify one example without having to update all the others. For example, there is one example that starts in Chapter 10: Functions to Extension Functions, is further refactored in Chapter 20: Performing I/O to Passing Data, and again in Chapter 21: Exceptions to Values. This code is on a branch named customer-report, and has commits with labels like:

    exceptions-to-values.7 : flatMap errors
    exceptions-to-values.6 : Collect reasons for failure
    ...
    io-to-data.2 : Show calculation in check
    io-to-data.1 : convert tests to Kotlin
    ...
    extensions.4 : Fix compile
    extensions.3 : Convert HighValueCustomersReport to Kotlin (doesn't compile)

Running Nat’s script from earlier will give these commits tags:

    exceptions-to-values.7
    exceptions-to-values.6
    ...
    io-to-data.2
    io-to-data.1
    ...
    extensions.4
    extensions.3

Looking in, for example, 20-performing-io-to-passing-data.asciidoc we find:

// begin-insert: io-to-data.2:src/test/java/travelator/marketing/HighValueCustomersReportTests.kt#check
[source,kotlin]
----
private fun check(
    inputLines: List<String>,
    expectedLines: List<String>
) {
    val output = StringWriter()
    val reader = StringReader(inputLines.joinToString("\n"))
    generate(reader, output)
    val outputLines = output.toString().lines()

    assertEquals(expectedLines, outputLines)
}
----

As you can imagine, we got pretty good at interactively rebasing the code on these branches, but still it was easy to break code, so that something that had worked no longer did. In the next installment I’ll look at how we made sure that all the code that we published in the book worked as expected.

Extracting an Extension Function from a Call Chain

2020-12-01T00:00:00-06:00

Kotlin extension functions let us express our algorithms as functional pipelines. We can see these as chains of calls, where the output of a stage is fed into the next. Here’s a chain that reads lines representing CustomerData from a reader:

val valuableCustomers = reader
    .buffered()
    .lineSequence()
    .drop(1) // header
    .map(String::toCustomerData)
    .filter { it.score >= 10 }
    .sortedBy(CustomerData::score)
    .toList()

Chains like this are nice and easy to read (at least for English speakers), as we start at the top left and work our way to the bottom right. This one is long enough to be intimidating though, and mixes concerns at different levels. It would be nice to break it into named sections.

We can do that by extracting extension methods that represent parts of the call chain. In this case we’d like to extract the drop(1) to .filter lines into a function named valuableCustomers.

IntelliJ doesn’t (yet) have a single refactoring that will do that. We can combine individual refactorings to get there safely though.

First we select from the start of the expression to end of the chain that we want. In this case, from reader to the end of the filter line.

                        reader
    .buffered()
    .lineSequence()
    .drop(1) // header
    .map(String::toCustomerData)
    .filter { it.score >= 10 }

“Extract Function”, calling it valuableCustomers:

val valuableCustomers = valuableCustomers(reader)
    .sortedBy(CustomerData::score)
    .toList()

private fun valuableCustomers(reader: Reader) = reader
    .buffered()
    .lineSequence()
    .drop(1) // header
    .map(String::toCustomerData)
    .filter { it.score >= 10 }

Now we select from the start of the extracted function to the beginning of chain that we want. In this case, from reader to the end of lineSequence():

                                                reader
    .buffered()
    .lineSequence()

“Introduce Parameter”. As there will be no uses of the old parameter once the new one is there, it shows that reader will be deleted. Accept the refactor.

That might work, but there is a currently a bug in IntelliJ that means that in some cases it doesn’t. It doesn’t work here, leaving us with both parameters and losing the buffered().lineSequence() code altogether:

val valuableCustomers = valuableCustomers(reader)
    .sortedBy(CustomerData::score)
    .toList()

private fun valuableCustomers(reader: Reader, sequence: Sequence<String>) = sequence
    .drop(1) // header
    .map(String::toCustomerData)
    .filter { it.score >= 10 }

If this happens, we “Undo” (keeping the reader to lineSequence() selection) and “Introduce Local Variable”:

private fun valuableCustomers(reader: Reader): Sequence<CustomerData> {
    val lineSequence = reader
        .buffered()
        .lineSequence()
    return lineSequence
        .drop(1) // header
        .map(String::toCustomerData)
        .filter { it.score >= 10 }
}

Now we select the whole new statement, from val to lineSequence() and “Introduce Parameter”. This is the point that directly introducing the parameter should have got to:

val valuableCustomers = valuableCustomers(
    reader
        .buffered()
        .lineSequence()
)
    .sortedBy(CustomerData::score)
    .toList()

private fun valuableCustomers(lineSequence: Sequence<String>): Sequence<CustomerData> {
    return lineSequence
        .drop(1) // header
        .map(String::toCustomerData)
        .filter { it.score >= 10 }
}

Now Alt-Enter on the lineSequence parameter and “Convert parameter to receiver”:

val valuableCustomers = reader
    .buffered()
    .lineSequence().valuableCustomers()
    .sortedBy(CustomerData::score)
    .toList()

private fun Sequence<String>.valuableCustomers(): Sequence<CustomerData> {
    return drop(1) // header
        .map(String::toCustomerData)
        .filter { it.score >= 10 }
}

Alt-Enter on return and “Convert to expression body”, and then tidying up the formatting gives us:

val valuableCustomers = reader
    .buffered()
    .lineSequence()
    .valuableCustomers()
    .sortedBy(CustomerData::score)
    .toList()

private fun Sequence<String>.valuableCustomers(): Sequence<CustomerData> = 
    drop(1) // header
    .map(String::toCustomerData)
    .filter { it.score >= 10 }

and we’re done, at least until JetBrains give us an “Extract extension function” refactor to do it automagically.

Well, almost done.

Refactoring never sleeps, so we do it again and again and again to give us:

val valuableCustomers = reader
    .asLineSequence()
    .valuableCustomers()
    .toListSortedBy(CustomerData::score)

private fun Sequence<String>.valuableCustomers(): Sequence<CustomerData> =
    withoutHeader()
        .map(String::toCustomerData)
        .filter { it.score >= 10 }

fun Reader.asLineSequence() = buffered().lineSequence()

fun <T, R : Comparable<R>> Sequence<T>.toListSortedBy(selector: (T) -> R?) =
    sortedBy(selector)
        .toList()

fun Sequence<String>.withoutHeader() = drop(1)

Helpfully the three new extracted functions look like they are more generally applicable, so we’ve made them public and can extract them into our library of handy extensions.

This refactoring is taken from an example in Java to Kotlin, A Refactoring Guidebook, by Nat Pryce and me, due to be published in 2021 by O’Reilly. If you like it, you can read more work in progress on O’Reilly Online Learning.

PS - Vesa Marttila points out that we are using a Reader here without closing it. The Sequence returned by Reader.asLineSequence() makes no attempt to close the BufferedReader, even if it is read to the end. I usually work on the principle that whoever creates a resource which might leak should dispose of it, but that isn’t the BufferedReader in this case, as that doesn’t itself hold any resources. At some level, this code should probably be surrounded in a block like

FileReader(filename).use { reader ->
    ...
}

Inline Tiny Types With Validation

2020-11-22T00:00:00-06:00

It’s been a long time since my last post. This is largely because my writing efforts have been directed towards a book! Nat Pryce (co-author of the excellent GOOS) and I are hard at work on Java to Kotlin, A Refactoring Guidebook, due to be published in 2021 by O’Reilly. You can read our work in progress on O’Reilly Online Learning.

Nat and I are both fans of tiny-types. The basic idea here is to have a separate type for individual domain concepts rather than just using (usually) Strings. So we might define GivenName and FamilyName types so that a Customer can be

data class Customer(
    val id: CustomerId,
    val givenName: GivenName,
    val familyName: FamilyName
)

Compared to raw strings, we can’t mix up the order of our given and family names when we create a customer, nor can we accidentally pass an OrderId where we need a CustomerId.

Here’s a typical tiny type in Kotlin:

data class GivenName(val value: String)

As you can imagine, tiny types are a bit of a pain in Java, where we have to define equals, hashCode and toString by hand, but Kotlin helps them make economic sense. For bonus points, we can even have our types behave like a CharSequence (the closest interface to String.

data class FamilyName(val value: String): CharSequence by value

This lets us write, for example

familyName.isNotBlank()

because isNotBlank is helpfully defined on CharSequence not String.

On the subject of non-blank-ness, it would be really helpful if we were prevented from creating a blank CustomerId. That would go a long way to avoiding the errors we get when functions are partial - only defined on some of their inputs (see Failure is not an Option, Part 7 - Avoiding Failure).

We can prevent the creation of an invalid CustomerId by checking the value in an init block.

data class CustomerId(val value: String): CharSequence by value {
    init {
        require(value.isNotBlank())
    }
}

Now we may get an error when we try to create a CustomerId, but if we have an instance, we can rely on the fact that it isn’t blank.

Given how easy they are to write in Kotlin, about the only drawback with tiny types is we actually end up with 2 objects to represent one value. That’s the initial String, and the data class wrapper around the string. The extra object probably consumes an extra 16 bytes these days, which is quite an overhead if we have a lot of wrapped strings. Potentially worse is the issue of cache coherency, because the initial string was probably created in the bowels of some JDBC code and may end up a long way in memory from its wrapper. So calling a method on a tiny type reference may involve fetching it into cache, and then the value that it is wrapping.

Kotlin Inline Classes, whilst still experimental, are aimed at removing this overhead.

inline class GivenName(val value: String)

In most circumstances the compiler arranges for instances of inline classes to be passed around as just the single contained value, so that we don’t have the overhead of creating an extra wrapper instance. This should be perfect for tiny types.

There’s one major problem though, we can’t easily validate inline classes.

inline class CustomerId(val value: String) {
    init { // Error: Inline class cannot have an initializer block
        require(value.isNotBlank())
    }
}

init blocks are disallowed because Java code is permitted to call Kotlin methods that take inline classes, passing the base type (String above). In this case the init block won’t have been called, so users of inline classes can’t know that validation will have been performed. So init blocks are disallowed.

For the same reason the primary constructor must be public. In fact, every route to validating an inline class is deliberately blocked because it might not have been taken when called from Java.

If you have a pure-Kotlin codebase, or just only ever call down into Java rather than back into Kotlin, this is a shame. It would be really nice to use inline classes to implement validated tiny types. Can we find a way to make this work?

What we need to do is either make sure that calling CustomerId("") throws an exception, or prevent callers from calling the CustomerId(val value: String) at all, instead making them go through some factory which can perform pre-validation.

We can’t throw an exception from the constructor because we can’t have an init block, but there is a sneaky way to achieve the second, at least for pure Kotlin code - make the value another inline class.

inline class CustomerId(private val __value: __DontMakeMe) {
    val value get(): String = __value.value
}

inline class __DontMakeMe(val value: String)

Now, in Kotlin, we can’t call the constructor with a String:

val id = CustomerId("42") // Type mismatch: inferred type is String but __DontMakeMe was expected

We can still call it, but there’s enough warning that we are subverting things:

val id = CustomerId(__DontMakeMe("42"))

Now we can create a factory function to validate a String and create a CustomerId with it.

fun customerIdOf(s: String): CustomerId? =
    when {
        s.isNotBlank() -> CustomerId(__DontMakeMe(s))
        else -> null
    }

Looking at the bytecode, we end up calling a static ‘constructor’ function for both CustomerId and __DontMakeMe here, but both just check that their parameter is not null.

I’ve chosen to return a nullable CustomerId? from customerIdOf so that callers know the call might fail, rather than having to read the code or documentation to see that it might throw an exception.

Inline classes can implement interfaces, although not through delegation at present.

inline class FamilyName(private val __value: __DontMakeMe): CharSequence {
    val value get(): String = __value.value

    override val length get() = value.length

    override fun get(index: Int) = value.get(index)

    override fun subSequence(startIndex: Int, endIndex: Int) =
        value.subSequence(startIndex, endIndex)
}

fun familyNameOf(s: String): FamilyName? =
    when {
        s.isNotBlank() -> FamilyName(__DontMakeMe(s))
        else -> null
    }

We can also define toString in order to mask sensitive information

inline class CardNumber(private val __value: __DontMakeMe) {
    val value get(): String = __value.value

    override fun toString() = "************" + value.substring(12)
}

fun cardNumberOf(s: String): CardNumber? =
    when {
        s.length == 16 && s.all(Char::isDigit) -> CardNumber(__DontMakeMe(s))
        else -> null
    }

As I’ve said, this is a subversion of the inline class initialisation rules and won’t be safe if you accept calls in from Java. I’m also not sure how marshalling technologies like Jackson or kotlinx.serialisation will interact with the technique. For pure Kotlin codebases though I think it’s worth a try, as tiny types can significantly improve the readability and safety of our code.

Come back soon, because I’ll present a tiny tiny types framework using phantom types to reduce the amount of boilerplate code required.

Postscript - there is some discussion of other ways to achieve validation on the reddit discussion.

Fun With Maps Part 2

2019-07-02T00:00:00-05:00

Recap

In the previous episode, we looked at parsing a log file. We initially used data classes to represent the different types of events that we saw, but found that they could be clumsy when representing type hierarchies, and we had to fall back on reflection in order to select properties specified by strings.

In contrast using maps to represent the data was (in this instance) less awkward, and they made it easy to select properties by name. Whilst I at least missed the reassurance that properties would be present and a given type that strong typing gave us, introducing a phantom type and extension properties allowed me to regain some of that confidence in a gradual way. I called the technique Phantom-typed PropertySets, but I’m hoping that someone else has given the technique a better name.

So we have a way to alleviate some of the loose-typing problems of maps. What can we do to make data classes more convenient? In particular to allow us access to properties in a dynamic way like we can with maps?

Accessing properties with reflection

We saw that we can use reflection access a the value of a Kotlin property by name. In short

@Suppress("UNCHECKED_CAST")
private fun Any.propertyValue(name: String): Any? =
    (this::class.memberProperties as Collection<KProperty1<Any, Any?>>).find {
        it.name == name
    }?.get(this)

That nasty cast is a result of the out variance of KClass - the less said about it the better. Let’s see it working.

data class Person(val firstName: String, val lastName: String)

class PropertyValueTests() {

    val person = Person("Fred", "Flintstone")

    @Test fun tests() {
        assertEquals("Fred", person.propertyValue("firstName"))
        assertNull(person.propertyValue("dob"))
    }
}

Compared to Java reflection, that’s pretty convenient already. Could we do better? Could we just treat our data object as a map?

We would be on the right lines if we can get the following to work.

class ObjectToMapTests() {

    val person = Person("Fred", "Flintstone")
    val map = person.asPropertyMap()

    @Test fun tests() {
        assertEquals("Fred", map["firstName"])
        assertNull(map["dob"])

        assertEquals(2, map.size)
        assertEquals(setOf("firstName", "lastName"), map.keys)
        assertEquals(listOf("Fred", "Flintstone"), map.values)

        // etc
    }
}

SpaceBlanket

If you fancy a challenge, go ahead and write your own implementation of asPropertyMap. It’s an interesting task, and you may well make different trade-offs than I did.

This was what I came up with.

fun Any.asPropertyMap(): Map<String, Any?> = SpaceBlanket(this)

internal class SpaceBlanket(private val thing: Any) : Map<String, Any?> {

    override val entries: Set<Map.Entry<String, Any?>>
        get() = keys.map { AbstractMap.SimpleImmutableEntry(it, get(it)) }.toSet()

    override val keys by lazy { properties.filter { it.visibility == KVisibility.PUBLIC }.map { it.name }.toSet() }

    override val size: Int get() = keys.size

    override val values: Collection<Any?>
        get() = keys.map { this[it] }

    override fun containsKey(key: String) = keys.contains(key)

    override fun containsValue(value: Any?) = values.contains(value)

    override fun get(key: String): Any? = properties.firstOrNull { it.name == key }?.get(thing)

    override fun isEmpty() = size == 0

    override fun equals(other: Any?) = when (other) {
        is SpaceBlanket -> this.entries == other.entries
        is Map<*, *> -> other == this
        else -> false
    }

    // copied from AbstractMap
    override fun hashCode(): Int {
        var h = 0
        val i = entries.iterator()
        while (i.hasNext())
            h += i.next().hashCode()
        return h
    }

    override fun toString() = HashMap(this).toString()

    @Suppress("UNCHECKED_CAST")
    private val properties by lazy {
        thing::class.memberProperties as Collection<KProperty1<Any, Any?>> // nasty cast due to out variance of thing::class
    }
}

I’ve published this on GitHub as a proof of concept. There are lots more features you could reasonably want, in particular recursively yielding maps for all non-primitive properties, but it is still pretty handy. We can go back to the filterFile example from Part 1 and write

fun filterFile(file: File, propertyName: String, propertyValue: String) = file.useLines { lines ->
    lines.map { it.toLogEntry() }
        .filter { it.asPropertyMap()[propertyName].toString() == propertyValue }
        .forEach(::println)
}

or to dump all the properties of a Kotlin object with their values

fun Any.dumped() = println(
    this
        .asPropertyMap()
        .entries
        .joinToString(",") { "${it.key} = ${it.value}" }
)

Or check a subset of the properties of an object in one go

fun check(entry: IPEntry) {
    val expected = mapOf("index" to "index #0", "interfaceName" to "VLINK1")
    val relevantProperties: Map<String, Any?> = entry
        .asPropertyMap()
        .entries
        .filter { it.key in expected.keys }
        .map { it.key to it.value }
        .toMap()
    assertEquals(expected, relevantProperties)
}

Converting to a map isn’t the only way to solve these sorts of problems, but the chances are that you can work out how to solve them with a map, so it may well be a good start.

Conclusion

Data classes are a really nice feature of Kotlin, and should be your default choice for representing data, but they have limitations.

If you have unknown data, a heavily subtyped data model, or need dynamic access to properties, maps may be a better choice. You can bring some gradual typing to your maps using Phantom-typed PropertySets.

If you choose data classes and find that you occasionally need map-like access, SpaceBlanket or something like it can retrofit a map interface to standard Kotlin objects.

Thanks to Nat Pryce, Rob Fletcher and Robert Stoll for providing feedback on this article.

Fun With Maps Part 1

2019-06-30T00:00:00-05:00

I’ve been listening to an (and another) excellent podcast by Clojure programmers recently and learned that nobody builds classes in Clojure; and certainly not to represent data. Instead they populate maps for all the uses where Kotlin developers would reach for a data class.

I’ve got very used to data classes and like them a lot, but there are certainly situations in which they don’t shine. Today I’m going to look using maps instead.

The Brief

Let’s say that we want to parse a log file into semantic representations of different sorts of errors. Here’s one I borrowed from IBM

03/22 08:51:01 INFO   :.main: *************** RSVP Agent started ***************
03/22 08:51:01 INFO   :...locate_configFile: Specified configuration file: /u/user10/rsvpd1.conf
03/22 08:51:01 INFO   :.main: Using log level 511
03/22 08:51:01 INFO   :..settcpimage: Get TCP images rc - EDC8112I Operation not supported on socket.
03/22 08:51:01 INFO   :..settcpimage: Associate with TCP/IP image name = TCPCS
03/22 08:51:02 INFO   :..reg_process: registering process with the system
03/22 08:51:02 INFO   :..reg_process: attempt OS/390 registration
03/22 08:51:02 INFO   :..reg_process: return from registration rc=0
03/22 08:51:06 TRACE  :...read_physical_netif: Home list entries¢ returned = 7
03/22 08:51:06 INFO   :...read_physical_netif: index #0, interface VLINK1 has address 129.1.1.1, ifidx 0
03/22 08:51:06 INFO   :...read_physical_netif: index #1, interface TR1 has address 9.37.65.139, ifidx 1
03/22 08:51:06 INFO   :...read_physical_netif: index #2, interface LINK11 has address 9.67.100.1, ifidx 2
03/22 08:51:06 INFO   :...read_physical_netif: index #6, interface LOOPBACK has address 127.0.0.1, ifidx 0
03/22 08:51:06 INFO   :....mailslot_create: creating mailslot for timer
03/22 08:51:06 INFO   :...mailbox_register: mailbox allocated for timer
03/22 08:51:06 INFO   :.....mailslot_create: creating mailslot for RSVP
03/22 08:51:06 INFO   :....mailbox_register: mailbox allocated for rsvp
03/22 08:51:06 INFO   :.....mailslot_create: creating mailslot for RSVP via UDP

Data Classes

If we can get past wondering why some developer decided that it was worth outputting the day and month but not the year of each entry, we would probably decide to represent each entry with a data class

data class LogEntry(val timestamp: Instant, val level: String, val message: String)

fun String.toLogEntry() = LogEntry(
    substring(0, 15).toInstantSomehow(),
    substring(16, 23).trim(),
    substring(24).trim()
)

fun showRecentAddresses(file: File) = file.useLines { lines ->
    lines.map { it.toLogEntry() }
        .filter { it.timestamp.isAfter(lastTuesday) }
        .filter { it.message.contains("interface .*? has address".toRegex()) }
        .map { it.message.split(",")[1] }
        .forEach(::println)
}

That’s fine for a one-off analysis, but no good deed goes unpunished. Soon we’re asked to write a suite of analysis software so that the rest of the team can get on with not writing the year into timestamps. We end up with data classes for the different types of log lines that we know how to parse, with an interface for the common data in every line.

interface LogEntry {
    val timestamp: Instant
    val level: String
    val loggerName: String
    val message: String
}

data class BasicEntry(
    override val timestamp: Instant,
    override val level: String,
    override val loggerName: String,
    override val message: String
) : LogEntry

private val lineParser =
    """^(?<timestamp>.{14}) (?<level>.*?) *?:\.+(?<name>.*?): (?<message>.*)$""".toRegex()

fun String.toBasicEntry() = lineParser.find(this)?.groups?.let { groups ->
    BasicEntry(
        timestamp = groups["timestamp"]?.value?.toInstantSomehow()
            ?: error("no timestamp in line $this"),
        level = groups["level"]?.value
            ?: error("no level in line $this"),
        loggerName = groups["name"]?.value
            ?: error("no logger name in line $this"),
        message = groups["message"]?.value
            ?: error("no messagein line $this")
    )
} ?: error("Could not parse as BasicEntry $this")

data class IPEntry(
    override val timestamp: Instant,
    override val level: String,
    override val loggerName: String,
    override val message: String,
    val index: Int,
    val interfaceName: String,
    val address: String
) : LogEntry

fun BasicEntry.toIPEntry(): IPEntry? =
    if (loggerName == "read_physical_netif")
        IPEntry(timestamp, level, loggerName, message, SOMECODE(), SOMECODE(), SOMECODE())
    else
        null

data class RegProcessEntry(
    override val timestamp: Instant,
    override val level: String,
    override val loggerName: String,
    override val message: String
) : LogEntry

fun BasicEntry.toRegProcessEntry(): RegProcessEntry? =
    if (loggerName == "reg_process") SOMECODE() else null

data class MailslotCreateEntry(
    override val timestamp: Instant,
    override val level: String,
    override val loggerName: String,
    override val message: String,
    val createdFor: String,
    val createdVia: String?
) : LogEntry

fun BasicEntry.toMailSlotCreateEntry(): MailslotCreateEntry? =
    if (loggerName == "mailslot_create") SOMECODE() else null

val converters = listOf<(BasicEntry) -> LogEntry?>(
    BasicEntry::toIPEntry,
    BasicEntry::toRegProcessEntry,
    BasicEntry::toMailSlotCreateEntry
    //...
)

fun String.toLogEntry(): LogEntry {
    val rawEntry = this.toBasicEntry()
    return converters.asSequence()
        .mapNotNull { it(rawEntry) }
        .firstOrNull()
        ?: rawEntry
}

fun showRecentAddresses(file: File) = file.useLines { lines ->
    lines.map { it.toLogEntry() }
        .filterIsInstance<IPEntry>()
        .filter { it.timestamp.isAfter(lastTuesday) }
        .map { "${it.interfaceName} : ${it.address}" }
        .forEach(::println)
}

// etc

This isn’t the only formulation of this data with data classes, and I’m not sure that it’s the best, but it is the sort of thing that we end up with in these situations. And it is, well, horrible, largely because you can’t inherit from a data classes. We could have made BasicEntry the only property of LogEntry to remove some duplication, or make BasicEntry the envelope containing a typed contents of the parsed payloads, but it’s all rather icky because of the inheritance limitation of data classes.

There is another problem with data classes - we don’t have (easy) runtime access to their properties. Which is a problem because now the ops people would like us to use our technology to filter lines that contain a given property value. They are happy to look in our source for the names of the properties, but don’t want to be recompiling Kotlin every time they want a change, so we need to take the property name and value on the command-line.

java -jar filter-log.jar file.log interfaceName=VLINK1

So we set to and write

fun filterFile(file: File, propertyName: String, propertyValue: String) =
    file.useLines { lines ->
        lines.map { it.toLogEntry() }
            .filter { it.hasProperty(propertyName, propertyValue) }
            .forEach(::println)
    }

How do we write LogEntry.hasProperty without lots of tables or trying to keep code up to date? Reflection is the answer, but it has just taken me 30 minutes to get the following to compile

@Suppress("UNCHECKED_CAST")
private fun Any.hasProperty(name: String, value: String): Boolean =
    (this::class as KClass<Any>).memberProperties.any {
        it.name == name && it.get(this).toString() == value
    }

and even now it does it is relatively slow at runtime. Once we start mixing runtime selection with data classes we find all sorts of things that we have to use reflection to solve - we can’t even ask a data class for the number of its properties without reflection (or a compiler plugin). Serialising to and from JSON? We have clever libraries that hide it but it’s all refection underneath.

Now compared to Java, data classes are a real boon. In Kotlin they should certainly be your default choice for representing data as objects. But even when I offered data classes as a solution to a Clojure developer, he said that he would generally stick with maps. Why is this?

Maps

Let’s have a look at solving the same problem with maps.

First the simple case

fun String.toLogMap(): Map<String, Any?> = mapOf(
    "timestamp" to substring(0, 15).toInstantSomehow(),
    "level" to substring(16, 23).trim(),
    "rawMessage" to substring(24).trim()
)

fun showRecentAddresses(file: File) = file.useLines { lines ->
    lines.map { it.toLogMap() }
        .filter { (it["timestamp"] as Instant).isAfter(lastTuesday) }
        .filter { (it["rawMessage"] as String).contains("interface .*? has address".toRegex()) }
        .map { (it["rawMessage"] as String).split(",")[1] }
        .forEach(::println)
}

Hmmm, those casts are a bit ugly. They wouldn’t be there in Clojure of course, and we can do better in Kotlin as we’ve given up type safety anyway

@Suppress("EXTENSION_SHADOWED_BY_MEMBER")
inline fun <reified R> Map<String, Any?>.get(key: String) = get(key) as R

fun showRecentAddresses(file: File) = file.useLines { lines ->
    lines.map { it.toLogMap() }
        .filter { it.get<Instant>("timestamp").isAfter(lastTuesday) }
        .filter { it.get<String>("rawMessage").contains("interface .*? has address".toRegex()) }
        .map { it.get<String>("rawMessage").split(",")[1] }
        .forEach(::println)
}

This is a bit worse than the data class version, why would we give up the type safety that we had? Well the equivalent of the more developed example is quite a bit simpler because we don’t have to try to bend data classes to our will.

typealias LogEntryProperties = Map<String, Any?>

val lineParser = """^(?<timestamp>.{14}) (?<level>.*?) *?:\.+(?<name>.*?): (?<message>.*)$""".toRegex()

fun String.extractBasicProperties(): LogEntryProperties =
    lineParser.find(this)?.groups?.let { groups ->
        mapOf(
            "timestamp" to (groups["timestamp"]?.value?.toInstantSomehow() ?: error("no timestamp in line $this")),
            "level" to (groups["level"]?.value ?: error("no level in line $this")),
            "loggerName" to (groups["name"]?.value ?: error("no logger name in line $this")),
            "message" to (groups["message"]?.value ?: error("no messagein line $this"))
        )
    } ?: error("Could not extract properties from $this")


fun LogEntryProperties.withIPProperties(): LogEntryProperties =
    if (get<String>("loggerName") == "read_physical_netif")
        this + mapOf(
            "interface/index" to SOMECODE(),
            "interface/name" to SOMECODE(),
            "interface/address" to SOMECODE()
        )
    else
        this

fun LogEntryProperties.withRegProcessProperties(): LogEntryProperties = SOMECODE()

fun LogEntryProperties.withMailSlotCreateProperties(): LogEntryProperties = SOMECODE()

val extractors = listOf(
    LogEntryProperties::withIPProperties,
    LogEntryProperties::withRegProcessProperties,
    LogEntryProperties::withMailSlotCreateProperties
)

fun String.toLogEntryProperties(): LogEntryProperties =
    extractors.fold(this.extractBasicProperties()) { acc, f -> f(acc) }

fun showRecentAddresses(file: File) = file.useLines { lines ->
    lines.map { it.toLogEntryProperties() }
        .filter { it.containsKey("interface/index") }
        .filter { it.get<Instant>("timestamp").isAfter(lastTuesday) }
        .map { "${it["interface/name"]} : ${it["interface/address"]}" }
        .forEach(::println)
}

(Notice that we have added a namespace prefix to the property names - this isn’t strictly necessary but helps to avoid collisions in the event that more than one extractor matches a line.)

Because data is just stored in a map we don’t have to define data classes to match its shape, but we have lost the type safety that the properties of the data class were giving us, both with respect to their names and types. The data class implementation will almost certainly be faster, because reading a property is a single method call and field access, and more space efficient, because we won’t be creating sparse map structures. In short, data classes should probably be your default representation, even though they can be clumsy.

But what about that additional requirement of filtering by properties on the command line? That’s where the maps shine

fun filterFile(file: File, propertyName: String, propertyValue: String) = file.useLines { lines ->
    lines.map { it.toLogEntryProperties() }
        .filter { it[propertyName].toString() == propertyValue } // this is now just a map lookup - no reflection
        .forEach(::println)
}

Not having to deal with reflection is really nice here, the map really is first class data, whereas the data class makes you jump through reflective hoops.

So, data classes are good for some things, maps are good for others. Can we meet in the middle?

We can, but you have to decide whether your fundamental representation is going to be data classes, and you just wish that they behaved more like maps sometimes, or you are going to start from maps, and wouldn’t it be nice if they could be more type safe? As the maps version is still fresh, let’s look at making maps more type safe.

PropertySet

I have a technique for making maps more type-safe that I haven’t tried in anger, but I think has promise. I’m tentatively calling it the Phantom-typed PropertySet.

The problem it (partially) solves is that of passing information about what properties a map contains at compile time. Without this we run into the partial function problem described in Avoiding Failure - functions that take a Map<String, Any?> can fail at runtime because the actual map passed does not have the required keys (or they are the wrong type).

My plan is to parameterise Map<String, Any?> with another type that prevents them from all looking alike. Let’s call that type PropertySet.

interface PropertySet<out T> : Map<String, Any?>

We can implement PropertySets by delegating to another map

class MapPropertySet<out T>(private val delegate: Map<String, Any?>)
    : PropertySet<T>,
      Map<String, Any?> by delegate {
    constructor(vararg properties: Pair<String, Any>) : this(mapOf(*properties))
}

fun <T> propertySetOf(vararg properties: Pair<String, Any>): PropertySet<T> =
    MapPropertySet(*properties)

and provide a function to combine them like maps

operator fun <T> PropertySet<T>.plus(other: PropertySet<T>): PropertySet<T> =
    MapPropertySet(
        (this.entries + other.entries).map { entry -> entry.key to entry.value }
            .toMap())

Now let’s look at our log parsing example. A PropertySet needs a type, but the type doesn’t have to do anything. An empty interface will do fine.

interface Base

We can parse the basic log information into a PropertySet<Base> just like we did for plain maps

fun String.extractBasicProperties(): PropertySet<Base> =
    lineParser.find(this)?.groups?.let { groups ->
        propertySetOf<Base>(
            "timestamp" to (groups["timestamp"]?.value?.toInstantSomehow()
                ?: error("no timestamp in line $this")),
            "level" to (groups["level"]?.value
                ?: error("no level in line $this")),
            "loggerName" to (groups["name"]?.value
                ?: error("no logger name in line $this")),
            "message" to (groups["message"]?.value
                ?: error("no messagein line $this"))
        )
    } ?: error("Could not extract properties from $this")

Now let’s say that we need semantic access to the timestamp. We can provide that with an extension property on PropertySet<Base>.

val PropertySet<Base>.timeStamp get() = get<Instant>("timestamp")

That property allows access to, and knowledge of the type of, timestamp only when we have the right type of PropertySet.

val aPropertySet: PropertySet<Base> = SOMECODE()
val timestamp: Instant = aPropertySet.timeStamp

val aDifferentPropertySet: PropertySet<Any> = SOMECODE()
// doesn't compile
// val nogo = aDifferentPropertySet.timeStamp

and we still have access to other properties with the unsafe-but-usefully-dynamic map interface

val level = aPropertySet.get<String>("level")

Now we can do the same for our other ‘types’

interface IP : Base

val PropertySet<Base>.loggerName get() = get<String>("loggerName")

fun PropertySet<Base>.withIPProperties(): PropertySet<Base> =
    if (loggerName == "read_physical_netif")
        this + propertySetOf(
            "interface/index" to SOMECODE(),
            "interface/name" to SOMECODE(),
            "interface/address" to SOMECODE()
        )
    else
        this

val PropertySet<IP>.name get() = get<String>("interface/name")
val PropertySet<IP>.address get() = get<String>("interface/address")

interface RegProcess

fun PropertySet<Base>.withRegProcessProperties(): PropertySet<Base> = SOMECODE()

interface MailSlotCreate

fun PropertySet<Base>.withMailSlotCreateProperties(): PropertySet<Base> = SOMECODE()

Parsing is the same as for the maps example

val extractors = listOf(
    PropertySet<Base>::withIPProperties,
    PropertySet<Base>::withRegProcessProperties,
    PropertySet<Base>::withMailSlotCreateProperties
)

fun String.toLogEntryProperties(): PropertySet<Base> =
    extractors.fold(this.extractBasicProperties()) { acc, f -> f(acc) }

Now the only thing missing to build showRecentAddresses is the downcast from PropertySet<Base> to PropertySet<IP>. In order to do this we need to know that the map actually does have the properties that the PropertySet<IP> extension properties imply. We can get away here with just checking that one of them is there.

@Suppress("UNCHECKED_CAST")
fun PropertySet<Base>.asSetOfIP(): PropertySet<IP>? =
    if (containsKey("interface/index")) this as PropertySet<IP> else null

and now mapNotNull will conveniently cast and filter

fun showRecentAddresses(file: File) = file.useLines { lines ->
    lines.map { it.toLogEntryProperties() }
        .mapNotNull { it.asSetOfIP() }
        .filter { it.timeStamp.isAfter(lastTuesday) } // it is PropertySet<IP> now
        .map { "${it.name} : ${it.address}" }
        .forEach(::println)
}

and because we really are just dealing with maps, our previous filterFile continues to work fine.

fun filterFile(file: File, propertyName: String, propertyValue: String) = file.useLines { lines ->
    lines.map { it.toLogEntryProperties() }
        .filter { it[propertyName].toString() == propertyValue }
        .forEach(::println)
}

This technique of taking a map and tagging it with a type-erased but reconstructable marker, and synthesising properties based on the marker type, is a nice way to add gradual typing to a map-based data model. What if we want to go the other way round - represent a statically-typed model as a map? We’ll look at that in the next episode.

Thanks to Nat Pryce, Rob Fletcher and Robert Stoll for providing feedback on this article.

Refactoring to Kotlin

2019-06-20T00:00:00-05:00

Nat Pryce and I have been asked to run our popular Refactoring to Kotlin Workshop at KotlinConf 2019 in Copenhagen.

If you are attending the conference but relatively new to the language it is an excellent way to get up to speed on the language and its idioms.

More Details

Failure is not an Option - Functional Error Handling in Kotlin. Part 7 - Avoiding Failure

2019-06-19T00:00:00-05:00

This is Part 7 in a series looking at functional error handling in Kotlin. The parts are

Today I’d like to talk about avoiding error handling.

If I was cleverer than I am, this would have been the first part of this series. But I was seduced by the techniques of error handling and, as is often the case, not listening to what pain was telling me about my situation. It has taken 12 months of reading and thinking to realise what some people (hi Barry) were telling me a year ago. In my defence, I have not seen the advice in this post spelled out so directly, although I may just not have understood it when I saw it. My hope is that even if this is just another blog post saying the same thing as many others, it will increase the chances that people come to this information when they are ready to receive it.

As we’ve seen from the other posts in this series, handling errors is tedious and itself error-prone - and we could all do with less tedium and fewer errors. Exceptions are less tedious in code than other techniques, but as they allow us to sweep possible problems under the carpet, we either get less reliable code, or the tedium of lifting a lot of carpets. Result types are tedious in code, but potentially more reliable. Checked exceptions were a valiant effort to get the best of both worlds, but in the end foundered on the rock of first-class functions.

Errors are particularly pernicious in code because they are transitive. If a function relies on code that can fail then it either has to have a strategy for succeeding despite the failure of its compatriot, or it is itself subject to failure. Pretty soon any moderately complex functionality has a combinatorial explosion of ways that it can fail. We then end up adding code to reason with the possible errors just so that a function can fail with a error that makes sense to its callers. And that error handling code itself is subject to errors!

If we rely on exceptions for our error handling then the net effect is that most complex code paths can effectively throw any exception, and without checked exceptions we have no easy way to work out which. And even if we do work it out, then just swapping out one function for another can invalidate the analysis. Exactly the same thing is true for a Result type but with more complicated return types and control flow. (Actually there is another approach using sealed classes to represent errors within a bounded context. We’ll cover that in another post - it can reduce the types of errors that have to be considered, but doesn’t change their transitive nature.)

So what do we do? Mostly we suck it up. We catch exceptions at the top level and log and return an error, and/or we accept that every non-trivial function will return Either<R, Exception> and handle that at the top level in the same way. At some boundaries we try to deal with errors we know might happen in a sensible way, and then we run our systems and add code to deal with the other problems that we actually see.

I don’t think that this can ever change completely, but it would be good if it applied to less of our codebase. Our aim should be to reduce the number of functions that can fail. If in particular we can do that for lower-level functions, then the higher level functions that call them will also become less subject to failure, which in time should significantly reduce the proportion of functions that have to consider error cases at all. The boundary of code that is always subject to failure will rise up through the layers. This seems like a far better outcome than my previous experience, which was that learning better error handling techniques just allowed me to tolerate the pain all over, instead of encouraging me to expand the pain-free zones.

With the aim of reducing with number of our functions that can fail in mind, what are the reasons that a function can fail?

Type of Errors

Here are some basic categories of errors that can occur in a program.

General Environmental Failures

These are things that can just happen, but shouldn’t if things are not very badly broken. We might run out of memory, or find that a referenced method is not present because required libraries are not linked. All bets are off in these circumstances - they are represented by throwing a subclass of Error and we should probably just let the program die.

External Failures

Generally represented on the JVM by IOException - these represent a failure external to our program. Maybe a function tried to read from a file that no longer exists, or write to a network socket that has died.

In practice the prevalence of external errors varies widely even when they can occur in principle. Both reading from an open file and writing to a socket could fail, but files on our local filesystem are rarely deleted while we are reading them. In contrast we should assume that any network call will fail at some point. If a function is likely to fail in this way, we need to plan for it. If we need to plan for it, it would be nice if we had been told that it was likely to happen, either by declaring a thrown exception or returning a result type.

Also in practice, functions that can fail with IOException are generally ‘impure’ in a functional programming sense - their result doesn’t just depend on the value of their arguments. I’ll have to talk about this topic another time, but one way to deal with this impurity is to push such functions to the edges of our systems. If we manage to push it to the top edge, then most of our system layers will not have to deal with any such error, but this is a hard trick to pull off.

For now then let’s just assume that many, but by no means all, of our functions will be subject to failure with an IOException, either because they read from external systems, or invoke functions that are themselves subject to failure with an IOException.

Partial Functions

The other common reason that a function may not be able to return normally (where abnormally includes returning an Error as part of a Result type) is that it is only able to give an output for a restricted range of its inputs - a partial function. A good example from previous episodes was

fun parseInt(s: String): Int = SOMECODE()

This is only able to return an Int for a special subset of all strings that might be passed in. An even more common example is

class Array<T> {
    operator fun get(index: Int): T = SOMECODE()
}

Array indexing can only succeed if the index is contained in the array, which (in most but not all languages) cannot be known at compile time.

If a partial function is passed an input for which it is undefined it is polite for it to let us know - generally by throwing an exception, returning null, or returning an error code.

It is partial functions that I’d like to think about today, as in most systems they represent the majority of the errors that can occur.

Healing Partial Functions

As this is a Kotlin post, it’s worth noting every function that takes a reference and cannot cope with it being null is a partial function. Kotlin allows us to make those functions total by specifying that they cannot be invoked with a null argument.

Can we heal partial functions by changing the type of their parameters in other cases?

In Part 5 I looked at various implementations of

fun addAsInts(s1: String, s2: String, s3: String): Either<Exception, Int> = resultOf {
    Integer.parseInt(s1) + Integer.parseInt(s2) + Integer.parseInt(s3)
}

Adding integers represented as strings is a partial function because it is only defined where each of its inputs can be interpreted as an integer. Here it is made a total function by expanding its result type so that we can return a failure result in these cases - but now all callers have to deal with this result rather than a plain old Int. As Part 5 showed, that can lead to some pretty cryptic code, especially when we come to combining results from several partial functions.

Now the example was chosen to illustrate how to combine results, but we can also solve the partial function problem by simply changing the signature.

fun add(i1: Int, i2: Int, i3: Int): Int = i1 + i2 + i3

Now the function is total again, and we don’t have to pollute our calling code with error handling.

Where the data is naturally a string in the first place (typed by the user perhaps) then we are going to have to convert to Int somewhere. We could do that just before we need to interpret it, but that makes every function above it in the call stack either partial or subject to the errors in the conversion. So in this case we would almost certainly convert to Int as soon as possible, especially as the earlier we convert the more context we have available to report errors. The alternative is trying to recreate this context from an exception or result code propagated from several layers down.

In other cases though it’s easy to underestimate how much simpler this makes things and pass down strings that represent URI’s, or Files or DateTimes and hence make the functions that interpret them partial.

With a bit of work many otherwise partial functions can be made total. Integer division, for example, can fail.

// throws ArithmeticException if count is 0
fun average(sum: Int, count: Int) = sum / count

We can fix this average by introducing a type to represent a NonZeroInt and using it as a parameter. This makes the function total - it is now defined for all possible (runtime) values of its parameters.

data class NonZeroInt(val value: Int) {
    init {
        require(value != 0)
    }
}

fun average(sum: Int, count: NonZeroInt) = sum / count

// Can't fail because divisor.value can never be 0
operator fun Int.div(divisor: NonZeroInt) = this / divisor.value

Similarly, many List operations can fail on an empty list, but be made total on a non-empty list.

class NonEmptyList<T>(private val wrapped: List<T>) :
    List<T> by wrapped {
    init {
        require(wrapped.isNotEmpty())
        wrapped.first()
    }
}

Failure is not an Option - Functional Error Handling in Kotlin. Part 6 - What Should You Do While Waiting for the Standard Result Type

2019-06-16T00:00:00-05:00

This is Part 6 in a series looking at functional error handling in Kotlin. The parts are

It has been over a year since I wrote Part 5, and quite a lot has happened in that time. Arrow’s Try has matured but, as I read it, is now being deprecated as their approved way of representing errors. The Kotlin Standard Library has gained a Result type that we aren’t allowed to return, and Result4k became useable by dint of renaming its types Success and Failure rather than Ok and Err.

So if you want to pick a functional error handling type for your project, not much has become clearer in 12 months!

I can however report one experimental result. I migrated a multi-person-month codebase from a result type based on Arrow Either to a hand-rolled algebraic data type

sealed class Result<out R>

data class Success<out R>(val value: R) : Result<R>()
data class Failure(val exception: Exception) : Result<Nothing>()

The refactoring took less than a day and introduced no errors (that I found ;-). This is good news, because it is an existence proof that, at least for my style of programming, you could pick one result type for your codebase and change your mind with little pain.

This of course only applies if you can change the clients of your API. Library writers should probably still just throw exceptions unless they have a compelling reason not to, or else define their own minimal result type.

Based on talking to Andrey Breslav at KotlinConf I suspect that Jetbrains are just being ultra-cautious, and that restrictions on the use of the standard Result will be removed as they gain confidence in it. It’s worth noting though that the implementation is an inline class, not an algebraic data type. What this means in practice is that you cannot switch on the type. Instead of writing

val thang = when (result) {
    is Success -> "win"
    is Failure -> "fail"
}

you have to write

val thang = when {
    result.isSuccess -> "win"
    result.isFailure -> "fail"
    else -> error("Could contracts sort this out?")
}

which is obviously yuk, so if you want to have an easy migration to the standard type when it is allowed as a return type, I would learn to love fold

val thang = result.fold(
    onSuccess -> { "win" },
    onFailure -> { "fail" }
)

which you can use with both an ADT and the standard type, although note that the latter has the onSuccess parameter first in contrast to most Either-based results, so to future-proof as well as for clarity I would make sure to name both arguments.

Next Time

I hadn’t actually meant for this to be the subject of Part 6, but in the end I thought that a review was appropriate given the amount of time that has passed. This has been a bit off the cuff though, so I’d love to know if you agree or have a different perspective.

I have been thinking a lot about the broader topic of error handling recently, as Nat Pryce and I have proposed a session based on our experiences in this area for KotlinConf 2019. With luck the next episode can get around to what I had planned to write about - avoiding errors.

Kotlin Hacks - .printed()

2019-06-06T00:00:00-05:00

Say you have a function

fun sum(x: Int, y: Int) = x + y

and for some reason you suspect that it is going wrong. You’d like to print the return value to the console. So you extract the expression to a variable, print it, and return it.

fun sum(x: Int, y: Int): Int {
    val result =  x + y
    println(result)
    return result
}

Hmmm, for a temporary change, that escalated pretty quickly. We could do better with also, which translates a receiver into a lambda parameter

fun sum(x: Int, y: Int) = (x + y).also { println(it) }

This at least doesn’t change the shape of our code and so is easily reversible.

We can do better though by extracting the also invocation into a function - printed

fun <T> T.printed(): T = this.also { println(it) }

fun sum(x: Int, y: Int) = (x + y).printed()

If you want some choices about where and how the value is printed then default arguments ride to the rescue

fun <T> T.printed(format: String = "%s\n", stream: PrintStream = System.out): T =
    this.also { stream.format(format, this) }

which can be used as previously with no arguments, or, for example

fun sum(x: Int, y: Int) = (x + y).printed("sum is %s\n", System.err)

Use of printed isn’t restricted to return values

fun sum(x: Int, y: Int) = x.printed() + y.printed()

and this use in the middle of expressions can be really useful to avoid having to tear apart chains of calls

fun foo(x: Int, y: Int) = bar(x).baz(y) 

fun foo(x: Int, y: Int) = bar(x).printed().baz(y)

We can pull a similar stunt for printing collections on multiple lines

fun <T: Iterable<*>> T.printedAsList(format: String = "%s\n", stream: PrintStream = System.out): T =
    this.also { it.map { item -> item.printed(format, stream) } }

and even, although this is quite fragile, have overloads for InputStreams and Readers

fun InputStream.printed(stream: PrintStream = System.out, charset: Charset = Charsets.UTF_8) =
    bufferedReader(charset).printed(stream)

fun Reader.printed(stream: PrintStream = System.out, charset: Charset = Charsets.UTF_8) =
    readText().printed(stream = stream).byteInputStream(charset)

I hope that you find these useful - I keep them in a file called debugging.kt and just copy it into every project. If you have any improvements please let me know in the comments below.

Press to Test - Test Driven Development in Android Part 7

2019-06-03T00:00:00-05:00

This is the final part in a series documenting my experiences learning Android development in Kotlin. The code is available to follow along on GitHub.

In Part 1 I got a simple UI toy up and running, with tests running via Espresso in an emulator.

In Part 2 I used Roboletric to get (a lightly refactored version of) the same tests running in a local JVM rather than the emulator.

Part 3 was spent working out how to wait for a condition (the snack bar disappearing) on both the emulator and Robolectric tests.

In Part 4 and Part 5 I added unit tests that can run without the ten-second Robolectric tax by introducing something that I think is a ViewModel, but doesn’t look anything like Google’s examples.

Part 6 was spent using Data Binding to wire up the ViewModel in a way that I think is probably more like the Android Architecture Components have in mind.

This final episode tries to draw some conclusions from the journey to date.

But First, Some Swing

Between 1998 and 2010 most of my development work was in Swing, a now deprecated though still working cross-platform GUI toolkit.

I thought that it would be instructive to reimplement PressToTest, test-first, in Swing. I won’t take you through the process, but here is the result.

class PressToTestTests {

    var boomCount = 0
    val button = JButton().apply {
        PressToTestModel(this, "Press to Test", "Release to Detonate") {
            boomCount++
        }
    }

    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", button.text)

        button.dispatchEvent(mouseEvent(MOUSE_PRESSED))
        assertEquals("Release to Detonate", button.text)

        button.dispatchEvent(mouseEvent(MOUSE_RELEASED))
        assertEquals("Press to Test", button.text)
    }

    @Test
    fun `clicking button sets off the explosion`() {
        assertEquals(0, boomCount)

        button.doClick()
        assertEquals(1, boomCount)

        button.doClick()
        assertEquals(2, boomCount)
    }

    private fun mouseEvent(eventType: Int) = MouseEvent(button, eventType, currentTimeMillis(), 0, 0, 0, 0, false)
}

class PressToTestModel(
    private val button: JButton,
    private val defaultMessage: String,
    private val pressedMessage: String,
    private val action: () -> Unit
) {

    init {
        button.text = defaultMessage
        button.addActionListener {  action() }
        button.addMouseListener(object : MouseAdapter() {
            override fun mousePressed(e: MouseEvent) {
                button.text = pressedMessage
            }
            override fun mouseReleased(e: MouseEvent) {
                button.text = defaultMessage
            }
        })
    }
}

fun main() {
    val button = JButton().apply {
        PressToTestModel(this, "Press to Test", "Release to Detonate") {
            JOptionPane.showMessageDialog(this, "Boom")
        }
    }

    JFrame("Press to Test").apply {
        contentPane.add(button)
        pack()
        setLocationRelativeTo(null)
        defaultCloseOperation = JFrame.EXIT_ON_CLOSE
        isVisible = true
    }
}

That’s it. No XML, no 20,000 line Java source files generated from that XML, no Java source files generated to implement the data binding from that XML, no Gradle plugins to make sure that all that generation is done, and no IDE plugins to make sure that when I drill into the generated code I end up looking at the XML that it came from.

If I wanted a more acceptance-level test I could invoke main from a JUnit test, find the button by ID or text and manipulate it using a Robot.

Now this is a disingenuous comparison. Swing had the luxury (and curse) of rendering the UI itself, so that almost everything is pure Java. Android also has a more complicated app lifecycle model, so that managing consistency of state between a long-running application and a short-running activity is (necessarily?) complicated. And we shouldn’t forget that Swing assumes that it is running on a desktop-class OS, where Android is mobile - although the cheapest current Android phones are a class above the desktop that Swing was targeting in 1998.

Could Do Better

What I have discovered in this journey is that, even with its constraints, Android doesn’t really seem to try hard enough to make unit testing its UIs easy.

A GUI decoupling pattern like MVVM should allow us to test our view logic without having real UI components, by working in terms of interfaces and data-binding. Our Android ViewModel is able to implement the OnTouchListener and OnClickListener interfaces to be notified of changes, and bind its buttonText LiveModel to the button’s text, all without being aware of Button. In this way we can test the ViewModel without having to create an actual Button, which we can only do on an ‘actual’ device. All the pieces to implement this pattern are in place.

As I found though, if you then try to test this setup, you find that you cannot create the events that are propagated through the interfaces - in particular MotionEvent - in your tests. Introducing the Robolectric fake Android API can solve this problem, but only at the expense of horrible test startup times.

There is another problem with Robolectric, which is that in order to use it, you have to run your tests with the RobolectricTestRunner (maybe not directly, the AndroidJUnit4 test runner delegates to Robolectric for unit tests). Each test can only have one runner, which prevents the use of other interesting testing techniques. The RobolectricTestRunner is also intimately tied to JUnit 4, and Google are in no hurry to fix that. To be fair, they are caught a bit between a rock and a hard place - they could fix the single test runner issue by reimplementing Robolectric support as a JUnit Rule, but these aren’t well supported by JUnit 5. Sigh.

Given the difficulties that I found using Robolectric (slow startup, having to poke it to advance time, having to kick it to acknowledge the activity lifecycle, cannot integrate with other testrunners) I’m really not sure that it is worth the effort over a combination of the more completely decoupled unit tests that I ended up writing and running code on the simulator. Except that the simulator would be a challenge to run as part of a server-hosted continuous integration setup, which is probably the reason that people persist with Robolectric.

On the plus side, one of the reasons that Robolectric seems a marginal gain is that running acceptances tests against a real app running in a simulator proves to be so simple and reliable. So kudos to Google for Espresso and the tooling around it.

Wrap Up

I think that this toy example has run its course. Thanks to all those who provided feedback.

What about you? Do you agree with my conclusions, or do you have a different experience? Maybe there are tricks that I have missed? If so, please do let me know, either in the comments below, via Twitter or email. Thank you.

Press to Test - Test Driven Development in Android Part 6

2019-05-30T00:00:00-05:00

This is Part 6 in a series documenting my experiences learning Android development in Kotlin. The code is available to follow along on GitHub.

In Part 1 I got a simple UI toy up and running, with tests running via Espresso in an emulator.

In Part 2 I used Roboletric to get (a lightly refactored version of) the same tests running in a local JVM rather than the emulator.

Part 3 was spent working out how to wait for a condition (the snack bar disappearing) on both the emulator and Robolectric tests.

In those last two episodes I spent my time avoiding the Android Architecture Components, which include a ViewModel and LiveData and other things that are designed to help in this sort of situation. This was because every time I read the documentation I wanted to give up programming and live on a remote island with only 1950’s technology - maybe the Isle of Wight.

The time has now come to bite the bullet, because at some point I have to find some work, and I don’t suppose most potential gigs will be hand-rolling their own ViewModels. So today I shall mostly be wearing Architecture Components.

The Warmup

I start by working my way through the Data Binding Codelab. It all kind of makes sense, although for my taste too many important details are buried in XML files, and in code embedded in XML attributes in particular. One thing that I do become aware of is that lifecyle is important - as activities can be restarted we want continuity, but not at the expense of memory leaks.

OK, I can’t put it off any longer.

Migration

Migration of PressToTest starts in Gradle

android {
    ...
    dataBinding {
        enabled true
    }
}

and then I need to add a data element to activity_main.xml. Unfortunately I seem to need to add it into a root element that doesn’t exist in my (Android Studio generated) layout file. Ho hum. I add the missing root element and the data element and this version still builds and runs.

Now, to cut a long story short, there is an unedifying hour of faff when I discover that my project (created only a week ago using Android Studio’s new project command) is using APIs in com.android.support and the data binding examples are all using APIs from androidx. I try to fix Gradle library references by hand and end up in all sorts of trouble trying to find compatible versions of things. I go to bed with a messed up build.

With early morning clarity I think to Google for “androidx migration”. The results point me to an Android Studio command - Migrate to AndroidX - and then warn about all the ways that it will fail to go a good job. I’ve nothing to loose, so I rollback, invoke the command, and everything builds and runs just as before!

[Edit - looks like I spoke too soon. It did run, but later on the tests failed to build and had to be fixed.]

Making the button label in ViewModel a LiveData seems like the next logical step.

class ViewModel(
    private val defaultText: String,
    private val pressedText: String,
    private val onButtonTextChanged: (String) -> Unit,
    private var goBoom: () -> Unit
) {
//    private var buttonText: String by Delegates.observable(defaultText) { _, _, newValue ->
//        onButtonTextChanged(newValue)
//    }

    var buttonText = MutableLiveData<String>(defaultText)

Now I find that MutableLiveData doesn’t have a constructor taking the default value; except it does in the code lab that I’ve just completed. I sigh, look for the differences between the codelab build and my own and add implementation "androidx.lifecycle:lifecycle-extensions:2.1.0-alpha04" which seems to do the trick.

Apart from that, the binding of the button text to a LiveData is remarkably painless. Add a data section to the layout and reference our ViewModel

    <data>
        <variable
            name="viewmodel"
            type="com.oneeyedmen.presstotest.ViewModel"/>
    </data>

Tell the layout that the button’s text should use viewmodel.buttonText

    <Button
            android:text="@{viewmodel.buttonText}"

Have the ViewModel update the MutableLiveData

class ViewModel(
    // ...
) {

    var buttonText = MutableLiveData<String>(defaultText)

    @VisibleForTesting
    internal fun onTouchAction(actionCode:Int) {
        when (actionCode) {
            MotionEvent.ACTION_DOWN -> buttonText.value = pressedText
            MotionEvent.ACTION_UP -> buttonText.value = defaultText
        }
    }
}

and use DataBindingUtil.setContentView rather then AppCompatActivity.setContentView to wire up the databinding in MainActivity

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        DataBindingUtil.setContentView<ActivityMainBinding>(this, R.layout.activity_main).let {
            it.lifecycleOwner = this
            it.viewmodel = ViewModel(
                button = button,
                defaultText = getString(R.string.default_button_label),
                pressedText = getString(R.string.pressed_button_label),
                goBoom = this::boom
            )
        }
    }

    private fun boom() {
        Snackbar.make(button, getString(R.string.explosion), Snackbar.LENGTH_SHORT)
            .setAction("Action", null).show()
    }
}

Awesomely this runs, and works, and passes the InstrumentedAcceptanceTests. Less awesomely, Android Studio lets me do all those things without warning me that my unit tests no longer compile. No matter, the test can be simpler now because it can read directly from the buttonText LiveModel safe in the knowledge that data binding has its back.

class PressToTestTests {
    // ...

    val viewModel = ViewModel(
        defaultText = "Press to Test",
        pressedText = "Release to Detonate",
        goBoom = { boomCount++ }
    )

    private val buttonText get() = viewModel.buttonText.value

    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_DOWN)
        assertEquals("Release to Detonate", buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_UP)
        assertEquals("Press to Test", buttonText)
    }
}

Run that and - oh no not again java.lang.RuntimeException: Method getMainLooper in android.os.Looper not mocked. See http://g.co/androidstudio/not-mocked for details. The top hit on Google provides the answer - add an InstantTaskExecutorRule, as had been previously predicted on Reddit.

Now I run the last remaining test - the Robolectric InternalAcceptanceTest, which fails - apparently Espresso says that the button’s label hasn’t changed after the touch event. I try adding the InstantExecutorRule - no dice. I try waiting for the label change - no dice. I break out the debugger and find that when the ‘LiveData’ changes it sees its observer as inactive, so does nothing.

I return to Google, ending up searching for ‘“livedata” “robolectric” “inactive”’, and the top hit of LiveData seems to not notify observers looks promising. It’s an unfixed bug in Robolectric, with a workaround of activityController.get().getLifecycle().handleLifecycleEvent(Lifecycle.Event.ON_START).

Unfortunately there’s no clue as to where to put this code, but I figure that before every test would be a good bet, so I add an @Before to the InternalAcceptanceTests

class InternalAcceptanceTests : AcceptanceTests(robolectricWaiter) {
    @Before fun hack() {
        activityController.get().getLifecycle().handleLifecycleEvent(Lifecycle.Event.ON_START)
    }
}

which of course doesn’t compile, and I have no idea what the activityController is. I find that handleLifecycleEvent is a method on LifecyleRegistry, and LifeCycleRegistry is a subtype of LifeCycle, so in the end just guess and try

class InternalAcceptanceTests : AcceptanceTests(robolectricWaiter) {
    @Before fun hack() {
        (activityRule.activity.lifecycle as LifecycleRegistry).handleLifecycleEvent(Lifecycle.Event.ON_START)
    }
}

which works, at least for this test! Quick, check it in.

Events Dear Boy

Now we have used Data Binding to observe ViewModel.buttonText and update the, erm, button text, but we still have manual wiring of ViewModel to listen to the Button’s events.

Now it looks like I can associate the button’s OnTouchListener with the ViewModel if I revert to exporting a listener property, viz

class ViewModel(
    //...
) {
    val onTouchListener = OnTouchListener { _, event ->
        onTouchAction(event.action)
        false
    }
}

and do the binding in the XML

    <Button
        android:id="@+id/button"
        android:text="@{viewmodel.buttonText}"
        app:onTouchListener="@{viewmodel.onTouchListener}"
        ...
    />

That passes the tests, even when I remove the code to add the OnTouchListener to the Button in the ViewModel constructor. So I do the same with the onClickListener so that in the end the ViewModel secondary constructor disappears and we are left with

class PressToTestTests {

    @get:Rule
    val rule = InstantTaskExecutorRule()

    private var boomCount = 0

    val viewModel = ViewModel(
        defaultText = "Press to Test",
        pressedText = "Release to Detonate",
        goBoom = { boomCount++ }
    )

    private val buttonText get() = viewModel.buttonText.value

    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_DOWN)
        assertEquals("Release to Detonate", buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_UP)
        assertEquals("Press to Test", buttonText)
    }

    @Test
    fun `clicking button sets off the explosion`() {
        assertEquals(0, boomCount)

        viewModel.onClick()
        assertEquals(1, boomCount)

        viewModel.onClick()
        assertEquals(2, boomCount)
    }
}

class ViewModel(
    private val defaultText: String,
    private val pressedText: String,
    private var goBoom: () -> Unit
) {
    var buttonText = MutableLiveData<String>(defaultText)

    val onTouchListener = OnTouchListener { _, event ->
        onTouchAction(event.action)
        false
    }

    val onClickListener = OnClickListener { onClick() }

    @VisibleForTesting
    internal fun onTouchAction(actionCode: Int) {
        when (actionCode) {
            MotionEvent.ACTION_DOWN -> buttonText.value = pressedText
            MotionEvent.ACTION_UP -> buttonText.value = defaultText
        }
    }

    @VisibleForTesting
    internal fun onClick() {
        goBoom()
    }
}

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        DataBindingUtil.setContentView<ActivityMainBinding>(this, R.layout.activity_main).let {
            it.lifecycleOwner = this
            it.viewmodel = ViewModel(
                defaultText = getString(R.string.default_button_label),
                pressedText = getString(R.string.pressed_button_label),
                goBoom = this::boom
            )
        }
    }

    private fun boom() {
        Snackbar.make(button, getString(R.string.explosion), Snackbar.LENGTH_SHORT)
            .setAction("Action", null).show()
    }
}

<?xml version="1.0" encoding="utf-8"?>
<layout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools">

    <data>
        <variable
            name="viewmodel"
            type="com.oneeyedmen.presstotest.ViewModel"/>
    </data>
    <androidx.constraintlayout.widget.ConstraintLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        tools:context="com.oneeyedmen.presstotest.MainActivity">

        <Button
            android:id="@+id/button"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:text="@{viewmodel.buttonText}"
            android:onClickListener="@{viewmodel.onClickListener}"
            app:onTouchListener="@{viewmodel.onTouchListener}"
            app:layout_constraintBottom_toBottomOf="parent"
            app:layout_constraintEnd_toEndOf="parent"
            app:layout_constraintStart_toStartOf="parent"
            app:layout_constraintTop_toTopOf="parent" />

    </androidx.constraintlayout.widget.ConstraintLayout>
</layout>

Punt

That just leaves the goBoom action not translated to the new newness. Googling for “livedata snackbar” gives a lot of hits about SingleLiveEvent, but this isn’t part of the standard API and there seem to be competing implementations. I’m reasonably convinced that I don’t have to care about lifecycles with the current goBoom: () -> Unit implementation, and it has been a tiring day. so I punt and leave the snackbar invocation as it is. Philosophically I think that there is a difference between the databinding used for the label and the boom - the former is just data synchronisation whereas the latter is an effect (both in the audiovisual and functional programming sense) - so I’m happy to leave them looking different.

Wrap Up

Mid way through this post I would have bet that I would be rejecting the data binding approach now and rolling back the code. Actually, when I compare the two solutions I think that the data-binding wins out. The details hidden in the XML aren’t too bad and ViewModel is simplified by the removal of that ugly constructor. It’s a shame that it requires code-generation behind the scenes to work though, and that I had to bodge the Robolectric Espresso test to get it working.

I think that I have one more installment in me. I plan to return to my happy place and reimplement this example to give some perspective on the Android solution.

Press to Test - Test Driven Development in Android Part 5

2019-05-29T00:00:00-05:00

This is Part 5 in a series documenting my experiences learning Android development in Kotlin. The code is available to follow along on GitHub.

In Part 1 I got a simple UI toy up and running, with tests running via Espresso in an emulator.

In Part 2 I used Roboletric to get (a lightly refactored version of) the same tests running in a local JVM rather than the emulator.

Part 3 was spent working out how to wait for a condition (the snack bar disappearing) on both the emulator and Robolectric tests.

In Part 4 I spent a whole day trying to write a single unit test that was sufficiently decoupled from the Android runtime to run without Robolectric.

This time I’ve learned not to try to predict what will happen! But my aim is to add unit tests for the ‘Boom’ message and to try to simplify the tests, perhaps by making use of supplied support classes.

The Code So Far

Here’s the code to date - if you haven’t already you could read Part 4 to see how it got that way.

class PressToTestTests {

    var buttonText = "DEFAULT"
    val viewModel = ViewModel { buttonText = it }

    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_DOWN)
        assertEquals("Release to Detonate", buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_UP)
        assertEquals("Press to Test", buttonText)
    }
}

class ViewModel(
    private val onButtonTextChanged: (String) -> Unit
) {

    private val defaultText = "Press to Test"
    private val pressedText = "Release to Detonate"

    var buttonText: String by Delegates.observable(defaultText) { _, _, newValue ->
        onButtonTextChanged(newValue)
    }

    init {
        // required to sync the view on creation
        buttonText = defaultText
    }

    fun onTouchAction(actionCode:Int) {
        when (actionCode) {
            MotionEvent.ACTION_DOWN -> buttonText = pressedText
            MotionEvent.ACTION_UP -> buttonText = defaultText
        }
    }

    val onTouchListener = View.OnTouchListener { v, event ->
        onTouchAction(event.actionMasked)
        false
    }
}

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        val viewModel = ViewModel(onButtonTextChanged = button::setText)
        button.setOnTouchListener(viewModel.onTouchListener)

        button.setOnClickListener { view ->
            Snackbar.make(view, getString(R.string.explosion), Snackbar.LENGTH_SHORT)
                .setAction("Action", null).show()
        }
    }
}

Data Binding or More Tests?

After completing this I spend a couple of hours overall reading up on Data Binding. It looks like I’ll have to modify quite a bit of the app to use it, and probably work my way through a codelab in order to understand what the documentation is talking about. I decide to press on without it for now, if only because working my way through documentation doesn’t provide blog content.

So, One More Test

The unit test that we are missing is that a snackbar should show “Boom” when the button is clicked. Given all the fun we had trying to test with a Button I work on the principle that we should avoid any direct reference to snackbars in the ViewModel and its tests. So I pull the same stunt as before - hide the code-to-make-the-boom behind a function passed into the ViewModel’s constructor.

class PressToTestTests {

    var buttonText = "DEFAULT"
    var boomCount = 0
    val viewModel = ViewModel(
        onButtonTextChanged = { buttonText = it },
        boom = { boomCount++ }
    )

    @Test
    fun `clicking button sets off the explosion`() {
        assertEquals(0, boomCount)

        viewModel.onClick()
        assertEquals(1, boomCount)
    }
}

expose a couple of methods on the ViewModel, one we can call from our tests as it doesn’t need an event (which we can’t create in unit tests), and one that gives access to an event listener

class ViewModel(
    private val onButtonTextChanged: (String) -> Unit,
    private var goBoom: () -> Unit
) {

    fun onClick() {
        goBoom()
    }

    val onClickListener = View.OnClickListener { onClick() }
}

and then install into our activity

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        val viewModel = ViewModel(
            onButtonTextChanged = button::setText,
            goBoom = this::boom)
        button.setOnTouchListener(viewModel.onTouchListener)
        button.setOnClickListener(viewModel.onClickListener)
    }

    private fun boom() {
        Snackbar.make(button, getString(R.string.explosion), Snackbar.LENGTH_SHORT)
            .setAction("Action", null).show()
    }
}

Now that I know not to drive down closed roads, this is a quick and easy, runs as a unit test in milliseconds, and passes our acceptance tests. Woohoo!

Review

I have a bit of a tidy up, moving installation of ViewModel with Button to a ViewModel constructor and then review the code.

class PressToTestTests {

    var buttonText = "DEFAULT"
    var boomCount = 0

    val viewModel = ViewModel(
        defaultText = "Press to Test",
        pressedText = "Release to Detonate",
        onButtonTextChanged = { buttonText = it },
        goBoom = { boomCount++ }
    )

    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_DOWN)
        assertEquals("Release to Detonate", buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_UP)
        assertEquals("Press to Test", buttonText)
    }

    @Test
    fun `clicking button sets off the explosion`() {
        assertEquals(0, boomCount)

        viewModel.onClick()
        assertEquals(1, boomCount)

        viewModel.onClick()
        assertEquals(2, boomCount)
    }
}

class ViewModel(
    private val defaultText: String,
    private val pressedText: String,
    private val onButtonTextChanged: (String) -> Unit,
    private var goBoom: () -> Unit
) {
    private var buttonText: String by Delegates.observable(defaultText) { _, _, newValue ->
        onButtonTextChanged(newValue)
    }

    constructor(
        button: Button,
        defaultText: String,
        pressedText: String,
        goBoom: () -> Unit

    ) : this(
        defaultText = defaultText,
        pressedText = pressedText,
        onButtonTextChanged = button::setText,
        goBoom = goBoom
    ) {
        button.setOnTouchListener { _, event ->
            onTouchAction(event.actionMasked)
            false
        }
        button.setOnClickListener {
            onClick()
        }
    }

    init {
        // sync the view on creation
        buttonText = defaultText
    }

    @VisibleForTesting
    internal fun onTouchAction(actionCode:Int) {
        when (actionCode) {
            MotionEvent.ACTION_DOWN -> buttonText = pressedText
            MotionEvent.ACTION_UP -> buttonText = defaultText
        }
    }

    @VisibleForTesting
    internal fun onClick() {
        goBoom()
    }
}

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        ViewModel(
            button = button,
            defaultText = getString(R.string.default_button_label),
            pressedText = getString(R.string.pressed_button_label),
            goBoom = this::boom
        )
    }

    private fun boom() {
        Snackbar.make(button, getString(R.string.explosion), Snackbar.LENGTH_SHORT)
            .setAction("Action", null).show()
    }
}

Actually that isn’t bad. The test is nice and simple and doesn’t rely on any awkward Android classes, MainActivity is clean, but the relationship between the ViewModel, the button and the snackbar is nicely explicit. The nastiness is hidden away in in a secondary constructor of ViewModel, and that looks worse than it is because of Kotlin’s awkward constructor delegation layout and my desire to fit the code on mobile screens.

Philosophically I like that ViewModel is now self-contained and asks only to be created with strings, a button and the goBoom effect. Overall, given the constraints on unit testing, this may be the least-worst solution. Which is least-bad-enough for me.

Wrap Up

This hasn’t been a full day’s work, but I’m going to stop this post here, as we now have complete acceptance and unit test coverage of the app, albeit hand-crafted. I’d still like to try out that Data Binding though, so I’ll spend the rest of the day working through that codelab so that I can refactor my cobbling tomorrow.

Press to Test - Test Driven Development in Android Part 4

2019-05-26T00:00:00-05:00

This is Part 4 in a series documenting my experiences learning Android development in Kotlin. The code is available to follow along on GitHub.

In Part 1 I got a simple UI toy up and running, with tests running via Espresso in an emulator.

In Part 2 I used Roboletric to get (a lightly refactored version of) the same tests running in a local JVM rather than the emulator.

Part 3 was spent working out how to wait for a condition (the snack bar disappearing) on both the emulator and Robolectric tests.

This time we’re going to to capitalise on my new-found Android expertise to test-drive the development of the functionality from scratch. Let’s remind ourselves what it is that we want to verify.

The Requirement

The app should show a single button with the label Press to Test. When you press, but don’t release, the button, its label should change to Release to Detonate. Releasing the button should result in some audiovisual extravaganza and reset the button for the next victim.

Unaccountably, a week after first publishing this idea, there is still no app in the Play Store for this.

Test First

This is test-first, so the first thing to do is to create a test and check that we can run it.

class PressToTestTests {
    @Tests
    fun test() {
        fail()
    }
}

Check ✔︎. Or rather ✘, as the test obligingly fails.

Now we have to decide which of our two features - the audiovisual extravaganza when you release the button, or the changing button text - to (test and then) implement first. Let’s start with the changing button text, as it comes first chronologically.

Our test wants to look like this

class PressToTestTests {
    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", buttonText)

        touchButton()
        assertEquals("Release to Detonate", buttonText)

        untouchButton()
        assertEquals("Press to Test", buttonText)
    }
}

where the buttonText property, touchButton and untouchButton are as yet undefined.

Let’s try the simplest thing that could possibly work.

class PressToTestTests {
    val button = Button(null)
    val buttonText: String get() = button.text.toString()

    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", buttonText)
    }
}

Not So Fast

Frankly I expected this to fail when I created the button, but instead I get “java.lang.RuntimeException: Method getText in android.widget.TextView not mocked. See http://g.co/androidstudio/not-mocked for details.” when I access button.text. I like an error message that I can click, even if is it to a page that “is obsolete and not maintained.” Anyway the gist is that there is only a stub for Button in the android.jar file that we develop against locally, and all its methods throw.

That confuses me for a while, as the test next to it in the source tree (InternalAcceptanceTests) is working with a button, albeit via Robolectric. Let’s try a variant of that test as a unit test.

class PressToTestTests {
    
    @get:Rule
    val activityRule = ActivityTestRule(MainActivity::class.java)
    
    val buttonText: String get() = activityRule.activity.button.text.toString()
    
    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", buttonText)
    }
}

This throws java.lang.IllegalStateException: No instrumentation registered! Must run under a registering instrumentation in the activityRule constructor. I ponder, and then realise that InternalAcceptanceTests are running with a special runner - @RunWith(AndroidJUnit4::class). I add that to my current tests

@RunWith(AndroidJUnit4::class)
class PressToTestTests {
    ...
}

and this now works, which is to say - I can check the button’s initial text without any exceptions. Success - except that in place of the exceptions there is a ten second wait while my test starts up - the Robolectric tax.

Now while I only pay that tax once per test run, it’s hardly conducive to a rapid TDD cycle. In fact, ironically, if the emulator is already running I can run my two acceptance tests on it, including installation, in about 5 seconds; whereas the ‘local’ Robolectric tests take at least 10 seconds. As an old Swing developer, used to just firing up my UI in my test VM and throwing events at it, this makes me a bit sad. It looks like if I want fast feedback from my unit tests I’m going to have to avoid real Android views.

Decoupling

So it’s off to Google again. In some of my previous reading I’ve come across the MVVM (Model-View-ViewModel) pattern, and that seems to promise decoupling of my view logic from actual Android view components. But oh my goodness it’s difficult to find a simple example unencumbered by combinations of Architecture Components and Dagger and RxJava and Data Binding and [Click to Add Favourite Framework].

While I thrash around the Internet trying to work out what a simple solution would look like using none or more of these parts, I can at least sketch out what a viewmodel might look like. It is supposed to sit between the UI and model (the application state), and translate between them - I guess Model-ViewModel-View was a less snappy title. Anyway, as we have no application state, only UI state, this should be quite simple. I recast the tests like this.

class PressToTestTests {

    val viewModel = ViewModel()

    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", viewModel.buttonText)

        viewModel.onButtonTouched()
        assertEquals("Release to Detonate", viewModel.buttonText)

        viewModel.onButtonUntouched()
        assertEquals("Press to Test", viewModel.buttonText)
    }
}

which lets me write a very simple implementation

class ViewModel {

    private val defaultText = "Press to Test"
    private val pressedText = "Release to Detonate"

    var buttonText = defaultText

    fun onButtonTouched() {
        buttonText = pressedText
    }

    fun onButtonUntouched() {
        buttonText = defaultText
    }
}

which passes the tests! At least now I have something I can check in to appear to be making progress.

Having checked it in though, it seems that ViewModel may not be trying hard enough. We know that the UI does not have separate events for onButtonTouched and onButtonUntouched. Instead I choose to let ViewModel expose an OnTouchListener which will take the logic that is currently sitting in MainActivity.kt so that we can test that.

class ViewModel {

    private val defaultText = "Press to Test"
    private val pressedText = "Release to Detonate"

    var buttonText = defaultText

    val onTouchListener = View.OnTouchListener { v, event ->
        when (event.actionMasked) {
            MotionEvent.ACTION_DOWN -> buttonText = pressedText
            MotionEvent.ACTION_UP -> buttonText = defaultText
        }
        false
    }
}

Now the tests just simulate the events that will come from a button

    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", viewModel.buttonText)

        viewModel.onTouchListener.onTouch(null, motionEventWithAction(MotionEvent.ACTION_DOWN))
        assertEquals("Release to Detonate", viewModel.buttonText)

        viewModel.onTouchListener.onTouch(null, motionEventWithAction(MotionEvent.ACTION_UP))
        assertEquals("Press to Test", viewModel.buttonText)
    }

    private fun motionEventWithAction(action: Int) = MotionEvent.obtain(0, 0, action, 0.0F, 0.0F, 0)

except that, sigh, java.lang.RuntimeException: Method obtain in android.view.MotionEvent not mocked. See http://g.co/androidstudio/not-mocked for details. I’ve been here before and I know that I can solve this problem with Robolectric, but Robolectric is the problem that I’m trying to solve.

I can’t be the first person to have this problem, so back to Google. I’m definitely not the first person to have this problem! But both cures (Robolectric or PowerMock) are worse than the disease. The problem is that MotionEvent is a final class, and evidently intimately intertwingled with the native workings of Android. This is what happens when frameworks are not created hand-in-hand with testing, and is a real shame. This one problem must be wasting hours of developer time and certainly seems to be forcing Google to develop over-complicated workarounds.

But I need to stop moaning and make some progress - it’s especially hard to justify this testing wheelspin when the app is already working and already has two other sets of tests to prove it! So I move the problem by adding a method onTouchAction which I can test, and leaving onTouchListener untested for now. When everything is wired up it will be tested by our acceptance tests in any case.

class ViewModel {

    private val defaultText = "Press to Test"
    private val pressedText = "Release to Detonate"

    var buttonText = defaultText

    fun onTouchAction(actionCode:Int) {
        when (actionCode) {
            MotionEvent.ACTION_DOWN -> buttonText = pressedText
            MotionEvent.ACTION_UP -> buttonText = defaultText
        }
    }

    val onTouchListener = View.OnTouchListener { v, event ->
        onTouchAction(event.actionMasked)
        false
    }
}

so now the tests just say

    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", viewModel.buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_DOWN)
        assertEquals("Release to Detonate", viewModel.buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_UP)
        assertEquals("Press to Test", viewModel.buttonText)
    }

and I can wire in my view model (at least for this interaction)

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        val viewModel = ViewModel()

        button.setOnTouchListener(viewModel.onTouchListener)

        button.setOnClickListener { view ->
            Snackbar.make(view, getString(R.string.explosion), Snackbar.LENGTH_SHORT)
                .setAction("Action", null).show()
        }
    }
}

But I Digress

You know, I’ve been faffing around for so long now that I actually ran the acceptance tests expecting them to pass. And when they didn’t, I actually ran the app and played with it, thinking that I must have broken the tests rather than that my cunning plan didn’t work. This one of the under-appreciated problems with development - when ‘easy’ things aren’t we still feel the need to make progress, and that need expresses itself in blind optimism, which then results in bugs.

Finally the evidence of my own clicking and eyes helps me realise that there is no linkage between our ViewModel and the button text. My previous Googling suggests that there is some data binding library that will help out, but for now I’m going to just hack it in manually.

More Noncoupling

What we want is to make changes to ViewModel.buttonText change the text of a button, without having the ViewModel know about the Button type, because we can’t create a Button in our unit tests without booting up Robolectric. If Button implemented an interface we could use that, but it doesn’t because, well it just doesn’t. The simplest way to implement this decoupling in Kotlin is just to use a function type to hide the actual code being run. So I add a function property to ViewModels constructor

class ViewModel(
    private val onButtonTextChanged: (String) -> Unit
) {

and arrange for its implementation to set the text on a button

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        val viewModel = ViewModel(onButtonTextChanged = button::setText)
        button.setOnTouchListener(viewModel.onTouchListener)

        button.setOnClickListener { view ->
            Snackbar.make(view, getString(R.string.explosion), Snackbar.LENGTH_SHORT)
                .setAction("Action", null).show()
        }
    }
}

Now the tests can work in terms of the last set text

class PressToTestTests {

    var buttonText = "DEFAULT"
    val viewModel = ViewModel { buttonText = it }

    @Test
    fun `button message changes on pressing`() {
        assertEquals("Press to Test", buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_DOWN)
        assertEquals("Release to Detonate", buttonText)

        viewModel.onTouchAction(MotionEvent.ACTION_UP)
        assertEquals("Press to Test", buttonText)
    }
}

and I can implement functionality in ViewModel with an observable

class ViewModel(
    private val onButtonTextChanged: (String) -> Unit
) {

    private val defaultText = "Press to Test"
    private val pressedText = "Release to Detonate"

    var buttonText: String by Delegates.observable(defaultText) { _, _, newValue ->
        onButtonTextChanged(newValue)
    }

    init {
        // required to sync the view on creation
        buttonText = defaultText
    }

    fun onTouchAction(actionCode:Int) {
        when (actionCode) {
            MotionEvent.ACTION_DOWN -> buttonText = pressedText
            MotionEvent.ACTION_UP -> buttonText = defaultText
        }
    }

    val onTouchListener = View.OnTouchListener { v, event ->
        onTouchAction(event.actionMasked)
        false
    }
}

Wrap Up

Finally (7 hours including research and typing this article) I have a true unit test of the first interaction, and the emulator and Robolectric versions of the acceptance tests still pass. I don’t know if what I have counts as MVVM or not, perhaps only MVM, or maybe it’s just a mongrel that will become more refined as I learn more about the official support for testing. At least I now know some of the constraints on Android TDD and may appreciate why some things are as they are.

Looking back at the code I see a hidden advantage with the ViewModel - it is now a single point of truth about the default button text, which was previously duplicated between activity_main.xml and MainActivity.kt. Generally though things have gotten a bunch more complicated in the name of testability. I’m trusting that in the next episode we will be able to remove some code as I learn more about Android UI patterns.

Press to Test - Test Driven Development in Android Part 3

2019-05-25T00:00:00-05:00

This is Part 3 in a series documenting my experiences learning Android development in Kotlin. The code is available to follow along on GitHub.

In Part 1 I got a simple UI toy up and running, with tests running via Espresso in an emulator.

In Part 2 I used Roboletric to get (a lightly refactored version of) the same tests running in a local JVM rather than the emulator.

In this episode I was going to look at writing unit tests for the interaction. But there was some feedback on the Kotlin Test Slack channel for Part 1 that said “It’s unfortunate that he kind of blurred the scope of testing, waiting for 3 seconds for the snackbar to disappear.” This criticism is well founded - these randomish waits for things to happen lead to brittle tests where we we can’t trust that failures are actually problems. In Part 2 I compounded the problem in the Robolectric tests, although to be fair I did it in the name of consistency.

To recap - the issue is that the UI displays a message in a Snackbar, which automatically hides. Instead of waiting for the view to disappear, the tests just sleep for 3 seconds and then check that it has gone. The problem is compounded because on the emulator this happens automatically, whereas in Robolectric we have to idle the shadow looper (whatever that means) to induce it.

In reality, I think that in the early stages of real project I would be happy enough to leave the wait in the tests and see if there were actually problems. But this whole project is an academic exercise, so I suppose that we should see if we can address this issue, for academic interest.

ConditionWatcher

In Part 2 I wrote “I’m pretty sure that if I understood what this article is saying I could find a common solution that does not have a fixed wait time,” so I went off and read that article. There was a lot of information about how Roboelectric can be told that things are ready for its next interaction, and then code for a ConditionWatcher that basically tests a condition periodically until either it is met or a timeout occurs

So I pull the ConditionWatcher jar into the project and try it out.

    @Test
    fun clicking_button_shows_temporary_BOOM_message() {

        onView(snackBarMatcher).check(doesNotExist())

        onView(buttonMatcher).perform(click())
        onView(snackBarMatcher).check(isDisplayed())

        ConditionWatcher.waitForCondition(object: Instruction() {
            override fun getDescription() = "Snackbar has gone"

            override fun checkCondition(): Boolean = try {
                onView(snackBarMatcher).check(doesNotExist())
                true
            } catch (x: AssertionFailedError) {
                false
            }
        })
        onView(snackBarMatcher).check(doesNotExist())
    }

You can see there a nasty interface mismatch between our way of asserting the state of our GUI through Espresso’s onView(matcher).check(viewAssertion), which throw AssertionFailedError if they are not satisfied, and ConditionWatcher’s condition, which returns false if it is not satisfied. But it’s not a fatal problem, just an irritation, and this code does pass when we run it in the emulator.

When we run it in a local JVM under Robolectric though it doesn’t pass, because as we previously found out, in order to make the Snackbar hide we have to poke the ShadowLooper. What we need is something like ConditionWatcher but which does this poking as well as waiting. While we’re at it it would be nice to be able to set different timeouts and poll intervals per call, and to wait for a condition without creating an Instruction object.

So I spend a happy 90 minutes or so writing my own Waiter class.

Waiter

data class Waiter(
    val defaultDescription: String = "something",
    val defaultTimeoutMillis: Long = 60000,
    val defaultPollMillis: Long = 250,
    val twiddler: () -> Unit = {}
) {
    fun waitFor(
        description: String? = null,
        timeoutMillis: Long = -1,
        pollMillis: Long = -1,
        condition: () -> Boolean
    ) {
        val resolvedDescription = description ?: defaultDescription
        val resolvedTimeoutMillis = if (timeoutMillis >= 0 ) timeoutMillis else defaultTimeoutMillis
        val resolvedPollMillis = if (pollMillis >= 0 ) pollMillis else defaultPollMillis

        val endT = System.currentTimeMillis() + resolvedTimeoutMillis
        while (!condition()) {
            if (System.currentTimeMillis() > endT)
                throw TimeoutException("Timeout waiting for $resolvedDescription after more than $timeoutMillis ms")
            Thread.sleep(resolvedPollMillis)
            twiddler()
        }
    }

    inline fun <reified T: Throwable> waitForNo(
        description: String? = null,
        timeoutMillis: Long = -1,
        pollMillis: Long = -1,
        crossinline block: () -> Unit
    ) {
        waitFor(description, timeoutMillis, pollMillis) {
            try {
                block()
                true
            } catch (t: Throwable) {
                if (t is T) false else throw t
            }
        }
    }
}

and yes, I did write it test first.

To be honest, this is probably overkill, but then it is an academic exercise, so I might be forgiven for stretching my coding legs a little. waitForNo is a bit suss, but is there to integrate with Espresso thus

@RunWith(AndroidJUnit4::class)
abstract class AcceptanceTests(private val waiter: Waiter) {

    @get:Rule
    val activityRule = ActivityTestRule(MainActivity::class.java)

    val button = onView(withId(R.id.button))
    val snackBar get() = onView(
        allOf(
            withId(android.support.design.R.id.snackbar_text),
            withText("BOOM!")
        )
    )

    @Test
    fun button_message_changes_on_pressing() {
        button.check(matchesIsDisplayed(withText("PRESS TO TEST")))

        button.perform(Finger.pressAndHold())
        button.check(matchesIsDisplayed(withText("RELEASE TO DETONATE")))

        button.perform(Finger.release())
        button.check(matchesIsDisplayed(withText("PRESS TO TEST")))
    }

    @Test
    fun clicking_button_shows_temporary_BOOM_message() {
        snackBar.check(doesNotExist())

        button.perform(click())
        snackBar.check(matchesIsDisplayed())

        waiter.waitForNo<AssertionFailedError>("Snackbar gone") {
            snackBar.check(doesNotExist())
        }
    }
}

private fun matchesIsDisplayed(matcher: Matcher<View> = Matchers.any(View::class.java)) = matches(
    allOf(
        ViewMatchers.isDisplayed(),
        matcher
    )
)

In androidTest - we don’t have to pander to Robolectric

class InstrumentedAcceptanceTests : AcceptanceTests(Waiter())

Whereas in test we need a waiter that pokes Robolectric every time around its loop.

class InternalAcceptanceTests : AcceptanceTests(robolectricWaiter)

private val robolectricWaiter = Waiter(twiddler = { ShadowLooper.runMainLooperToNextTask() })

There we have it, we can run the same tests on the emulator and locally, and they will both wait for the snackbar to disappear of its own accord, or fail with a TimeoutException after a minute.

Wrap Up

Engineering is the balance between getting things right and getting things done. The code is now a more ‘right’ in that it doesn’t just sleep, but I wonder if runMainLooperToNextTask will result in the Robolectric scheduler getting out of sync with the time that we have waited for the condition to hold. Without an actual problem though - sufficient unto the day is the evil thereof.

Next time I’ll get back to trying to write some proper unit tests.

Press to Test - Test Driven Development in Android Part 2

2019-05-24T00:00:00-05:00

This is Part 2 in a series documenting my experiences learning Android development in Kotlin.

In Part 1 I got a simple UI toy up and running, with tests running via Espresso in an emulator (and (I assume but haven’t verified) on an actual device).

I left that episode impressed with how easy it was to run functional test code, but concerned that running all my UI tests this way would be very slow for a real project. Let’s see if we can run the same sort of tests in a local JVM rather than on an Android device.

Just Too Easy

In my Googling around for how to test Android apps Roboletric keeps coming up, and I learn that it is a set of fake implementations of Android APIs to let you run Android code in a JVM. Running my Android code in a JVM is exactly what I want to do, so I search for “espresso robolectric testing” and find articles with reassuring titles like Write Once, Run Everywhere Tests on Android. They say that I can just run the Espresso tests as local JVM tests!

It seems that tests in the androidTest source root will run on an external device / emulator when the test libraries are referenced with androidTestImplementation, and the same code in the test source root will run in a local JVM with testImplementation libraries (including Robolectric). Oh, and a magic Gradle incantation, viz

android {
    testOptions.unitTests.includeAndroidResources = true
}

This seems too good to be true, but I go ahead and add the testImplementation declarations to my Gradle build file, giving me

dependencies {
    // ...
    
    testImplementation 'junit:junit:4.12'

    // Core library
    testImplementation 'androidx.test:core:1.1.0'
    androidTestImplementation 'androidx.test:core:1.1.0'

    // AndroidJUnitRunner and JUnit Rules
    testImplementation 'androidx.test:runner:1.1.1'
    androidTestImplementation 'androidx.test:runner:1.1.1'
    testImplementation 'androidx.test.ext:junit:1.1.0'
    androidTestImplementation 'androidx.test.ext:junit:1.1.0'
    testImplementation 'androidx.test:rules:1.1.1'
    androidTestImplementation 'androidx.test:rules:1.1.1'

    // Espresso dependencies
    testImplementation 'androidx.test.espresso:espresso-core:3.1.1'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.1.1'

    testImplementation 'org.robolectric:robolectric:4.0'
}

By the way, I should say that this project is available on GitHub if you want to see the full contents of files.

Now I just copy my ExampleInstrumentedTest into src/test/java (along with its helper Finger object).

It’s here that things get a little weird. Android Studio is convinced that there are now two clashing ExampleInstrumentedTests, showing compilation errors in each, whereas Gradle is happy. As Android Studio just delegates to Gradle to run the tests these days, both versions of the tests can be run. When I do, I find that the emulator version (in androidTest) continues to run and pass, whereas the JVM version (in test) fails one of the two tests.

Hmmm

In these situations my OCD doesn’t know which way to turn - I now have a broken test and a broken IDE (or at least IDE configuration). I decide that the latter is more fundamental - I need to trust that wiggley error lines are real when I see them.

So I rename the src/androidTest/java/com/oneeyedmen/presstotest/ExampleInstrumentedTest.kt to ExternalInstrumentedTest, only to find that Android Studio renames src/test/java/com/oneeyedmen/presstotest/ExampleInstrumentedTest.kt to externalInstrumentedTest.kt. Yes, with a lower case e. The Android Studio bug tracker is software only its mother could love, so I don’t know whether that’s a known issue, but I resort to Finder and BBEdit to fix the names and contents the old way and then find that the Finger support object clashes even when I make it private in the test files.

I eventually end up with lots of code duplicated under different names, but at least all the squigglies have gone, so that I can relax into working out why the Robolectric test is failing.

That Failing Test

The test that is failing is

    @Test
    fun clicking_button_shows_temporary_BOOM_message() {

        onView(snackBarMatcher).check(doesNotExist())

        onView(buttonMatcher).perform(click())
        onView(snackBarMatcher).check(isDisplayed())

        Thread.sleep(3000)
        onView(snackBarMatcher).check(doesNotExist())
    }

on the line after the Thread.sleep, with the message View is present in the hierarchy.

Long and bitter experience with UI toolkits has taught me that they often need cycles to properly update their state. To be honest I was surprised that just sleeping was enough to make this test pass on the emulator, so I’m more disappointed than shocked that it doesn’t work in Robolectric. Just in case I try a 6 second sleep, but that doesn’t help, so it’s off to Google again.

That search results are frustrating. It seems that the snackbar wasn’t supported under Robolectric for quite a while, so that many of the results are people devising solutions to that problem. I think that snackbar is now supported, in as much as the first two assertions pass, and suspect that the support just doesn’t extend to the auto-hide behaviour. Stepping through code in the debugger doesn’t seem to help one way or the other - the hiding responsibility seems split between a SnackbarManager and a Looper.

After a couple of hours I’m about to give up when I try a last search for “robolectric looper” and come across a Stack Overflow Answer that at least points the way towards

    @Test
    fun clicking_button_shows_temporary_BOOM_message() {

        onView(snackBarMatcher).check(doesNotExist())

        onView(buttonMatcher).perform(click())
        onView(snackBarMatcher).check(isDisplayed())

        ShadowLooper.idleMainLooper(3, TimeUnit.SECONDS)
        onView(snackBarMatcher).check(doesNotExist())
    }

Success! In fact it looks like I could use ShadowLooper.runMainLooperToNextTask() without relying on a fixed time, but that would then cause internal and external test logic to diverge, which is undesirable. I’m pretty sure that if I understood what this article is saying I could find a common solution that does not have a fixed wait time - but I’m now worn out by the stress of having made so little progress for so long, so that’s a subject for another post.

That Duplication

Now I’m left with two structurally identical tests in two different source trees, where they can’t share the same code but can’t have the same names without confusing Android Studio. I can’t make the androidTest source depend on the test source, because that would lead to Robolectric being mixed in to our external tests. A little more Googling comes up with this Stack Overflow Answer, and the following changes.

In build.gradle

    sourceSets {
        String sharedTestDir = 'src/sharedTest/java'
        test {
            java.srcDir sharedTestDir
        }
        androidTest {
            java.srcDir sharedTestDir
        }
    }

In sharedTest

@RunWith(AndroidJUnit4::class)
abstract class AcceptanceTests {

    @get:Rule
    val activityRule = ActivityTestRule(MainActivity::class.java)

    @Test
    fun button_message_changes_on_pressing() {
        val button = onView(buttonMatcher)

        button.check(isDisplayed(withText("PRESS TO TEST")))

        button.perform(Finger.pressAndHold())
        button.check(isDisplayed(withText("RELEASE TO DETONATE")))

        button.perform(Finger.release())
        button.check(isDisplayed(withText("PRESS TO TEST")))
    }

    @Test
    fun clicking_button_shows_temporary_BOOM_message() {

        onView(snackBarMatcher).check(doesNotExist())

        onView(buttonMatcher).perform(click())
        onView(snackBarMatcher).check(isDisplayed())

        sleep(3000)
        onView(snackBarMatcher).check(doesNotExist())
    }

    protected abstract fun sleep(millis: Long)
}

In test

class InternalAcceptanceTests : AcceptanceTests() {
    override fun sleep(millis: Long) = ShadowLooper.idleMainLooper(millis, TimeUnit.MILLISECONDS)
}

In androidTest

class InstrumentedAcceptanceTests : AcceptanceTests() {
    override fun sleep(millis: Long) = Thread.sleep(millis)
}

Wrap Up

Today has been a lot more tiring than yesterday - lots of wallowing without any real idea if I can solve my problems. In the end though the result has been worth it - having the same tests running either in a local JVM or an emulator is a major productivity aid in real projects. The local JVM provides fast feedback when things are broken, while the instrumentation tests give confidence that app is actually in a state to ship.

Next time, I think I’ll probably look at true unit testing of our UI, breaking the dependency on Android.

Press to Test - Test Driven Development in Android Part 1

2019-05-23T00:00:00-05:00

This is Part 1 in a series documenting my experiences learning Android development in Kotlin.

I’m going to start with a simple GUI toy that I have implemented in several GUI toolkits over the years - Press to Test.

Creating the Project

This is my first Android project, so I have to install Android Studio and its tools, which takes some time. Once I can finally open the app I start by picking “Empty Activity” from the Android Studio “New Project” wizard, selecting a package for the source and leaving everything else as default. As a old XP developer I’m pleased to see that there are already two tests, an example unit test

class ExampleUnitTest {
    @Test
    fun addition_isCorrect() {
        assertEquals(4, 2 + 2)
    }
}

and an example instrumented test, designed to run on a device or emulator

@RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {
    @Test
    fun useAppContext() {
        // Context of the app under test.
        val appContext = InstrumentationRegistry.getTargetContext()
        assertEquals("com.oneeyedmen.presstotest", appContext.packageName)
    }
}

Oh, and there is an actual application too

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
    }
}

Running this prompts me to create an emulator and then boots it up and runs the app, which due to some magic in activity_main.xml draws “Hello World” in the middle of the simulated phone’s screen.

The unit test runs and passes as expected, and impressively ExampleInstrumentedTest also runs in Android Studio, installs the app on the emulator, and passes. As I was taught never to trust a test I hadn’t seen fail, I try breaking the test by changing the constant in assertEquals and it obligingly fails with a nice stack trace in the Android Studio test runner.

So far so good.

Cutting Myself Some Slack

Now I should write a failing test before adding any functionality. I should, but I don’t actually know anything about Android development, let alone how to write sensible tests. Luckily XP has a get out - we can write a Spike to investigate how something might be done.

But what is that I am trying to to do?

The Requirement

I know right? How is there not already an app in the Play Store for this?

Spike

I noodle around in the visual designer replacing the “Hello World” label with a “Press to Test” button, then look for how to implement the audiovisual extravaganza. Googling for “android show popup message” yields links for Toast and SnackBars and I arbitrarily choose the later as being more modern, probably. More Googling, this time for “android button click event” and I cobble together the following.

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        button.setOnClickListener { view ->
            Snackbar.make(view, "BOOM!", Snackbar.LENGTH_SHORT)
                .setAction("Action", null).show()
        }
    }
}

This is what my friend Richard Care would describe as “monkey-see, monkey-do” code. In particular I have no real idea where R.layout.activity_main and button come from, except for some theory that code is being generated to reflect the contents of resource XML files. But no matter, because it works.

Now For That Test

I suspect that there might be ways to test our UI in unit tests, with fake bits of Android, but there’s nothing like the reassurance of a test that actually runs on a device, albeit an emulated one. So I Google for “Android UI testing” and find Espresso is the recommended approach. Except that when I look at the latest examples, they seem to using different packages than the example code generated by Android Studio. In the end I plump for using androidx.test by updating a whole bunch of referenced libraries and Googling for the errors caused until they went away.

Armed with Espresso, I can launch my activity (although I’m not clear why that is different from running the app) and press the button in code.

import androidx.test.InstrumentationRegistry
import androidx.test.espresso.Espresso
import androidx.test.espresso.action.ViewActions.click
import androidx.test.espresso.matcher.ViewMatchers.withId
import androidx.test.ext.junit.runners.AndroidJUnit4
import androidx.test.rule.ActivityTestRule

import org.junit.Test
import org.junit.runner.RunWith
import org.junit.Assert.*
import org.junit.Rule

@RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {

    @get:Rule
    val activityRule = ActivityTestRule(MainActivity::class.java)

    @Test
    fun click_button() {
        Espresso.onView(withId(R.id.button)).perform(click())
    }
}

When I run this in Android Studio, the emulator helpfully reacts and shows the snackbar, so that gives lots of confidence that we’re on the right track.

Now more Googling for how to find and check the snackbar, leading after about half an hour of experiments to the following

@RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {

    @get:Rule
    val activityRule = ActivityTestRule(MainActivity::class.java)

    @Test
    fun click_button() {
        val snackBarMatcher = allOf(
            withId(android.support.design.R.id.snackbar_text),
            withText("BOOM!")
        )

        onView(snackBarMatcher).check(doesNotExist())

        onView(withId(R.id.button)).perform(click())
        onView(snackBarMatcher).check(matches(isDisplayed()))

        Thread.sleep(3000)
        onView(snackBarMatcher).check(doesNotExist())
    }
}

Hey that’s not at all bad. Espresso seems well thought out, and although the Thread.sleep is a bit icky I can live with it for now. The test name is bad, but it will get sorted out when we have more context too. I guess we should call the end of our spike and settle down into TDD.

The Button Text

The only thing that our app doesn’t do (assuming you have a low extravaganza-threshold) is change the button text. Before implementing that, I work out how to check the text in a test, and refactor the existing test a little.

@RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {

    @get:Rule
    val activityRule = ActivityTestRule(MainActivity::class.java)

    @Test
    fun button_message_changes_on_pressing() {
        onView(buttonMatcher).check(matches(
            allOf(
                isDisplayed(),
                withText("PRESS TO TEST")
            )))
    }

    @Test
    fun clicking_button_shows_temporary_BOOM_message() {

        onView(snackBarMatcher).check(doesNotExist())

        onView(buttonMatcher).perform(click())
        onView(snackBarMatcher).check(matches(isDisplayed()))

        Thread.sleep(3000)
        onView(snackBarMatcher).check(doesNotExist())
    }
}

private val buttonMatcher = withId(R.id.button)

private val snackBarMatcher = allOf(
    withId(android.support.design.R.id.snackbar_text),
    withText("BOOM!")
)

Now we’re fully into test-first mode, I go looking for how to test touch down and touch up interactions in Espresso. Copying and pasting the code from the answer into a Finger object let’s me write

    @Test
    fun button_message_changes_on_pressing() {
        onView(buttonMatcher).check(matches(
            allOf(
                isDisplayed(),
                withText("PRESS TO TEST")
            )))

        onView(buttonMatcher).perform(Finger.pressAndHold())
        onView(buttonMatcher).check(matches(
            allOf(
                isDisplayed(),
                withText("RELEASE TO DETONATE")
            )))

        onView(buttonMatcher).perform(Finger.release())
        onView(buttonMatcher).check(matches(
            allOf(
                isDisplayed(),
                withText("PRESS TO TEST")
            )))
    }

which isn’t pretty but does fail, so I can write some production code. More Googling reveals setOnTouchListener, allowing me to cobble together the following

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        button.setOnTouchListener { view, event ->
            when (event.actionMasked) {
                MotionEvent.ACTION_DOWN -> button.text = "RELEASE TO DETONATE"
                MotionEvent.ACTION_UP -> button.text = "PRESS TO TEST"
            }
            true
        }

        button.setOnClickListener { view ->
            Snackbar.make(view, "BOOM!", Snackbar.LENGTH_SHORT)
                .setAction("Action", null).show()
        }
    }
}

Build it, run the tests and get home in time for tea and medals. Except that the button_message_changes_on_pressing fails. I’m used to this though - knowing UI tests it’s probably some timing issue. So I run the app in the emulator, and find that, in fact, our earth-shattering kaboom is missing.

Running the test again shows that Espresso was trying to tell me that

androidx.test.espresso.NoMatchingViewException: No views in hierarchy found matching: (with id: com.oneeyedmen.presstotest:id/snackbar_text and with text: is "BOOM!")

along with a really handy dump of the whole view hierarchy. Well, it isn’t handy now, but I can see it being very useful when debugging problems where a view is present but invisible, or sitting under the wrong parent.

Luckily it doesn’t take long to realise that the problem is that returning true from the touch listener is telling Android that we have handled the event, so it isn’t processing the up to make a click. Return false, and the tests both pass. Not only that, but the app actually works!

Tidy Up

All that’s required is a little test tidy up

@RunWith(AndroidJUnit4::class)
class ExampleInstrumentedTest {

    @get:Rule
    val activityRule = ActivityTestRule(MainActivity::class.java)

    @Test
    fun button_message_changes_on_pressing() {
        val button = onView(buttonMatcher)

        button.check(isDisplayed(withText("PRESS TO TEST")))

        button.perform(Finger.pressAndHold())
        button.check(isDisplayed(withText("RELEASE TO DETONATE")))

        button.perform(Finger.release())
        button.check(isDisplayed(withText("PRESS TO TEST")))
    }

    @Test
    fun clicking_button_shows_temporary_BOOM_message() {

        onView(snackBarMatcher).check(doesNotExist())

        onView(buttonMatcher).perform(click())
        onView(snackBarMatcher).check(isDisplayed())

        Thread.sleep(3000)
        onView(snackBarMatcher).check(doesNotExist())
    }
}

private val buttonMatcher = withId(R.id.button)

private val snackBarMatcher = allOf(
    withId(android.support.design.R.id.snackbar_text),
    withText("BOOM!")
)

private fun isDisplayed(matcher: Matcher<View> = Matchers.any(View::class.java)) = matches(
    allOf(
        ViewMatchers.isDisplayed(),
        matcher
    )
)

and we’re done for the day.

Wrap Up

Running tests on the emulator is remarkably painless with Android Studio. Each one does take a couple of seconds though, so it would be nice to be able to test our logic as part of unit tests run in a nice fast Gradle test VM instance. In the next installment I’ll take a look at that.

My New Test Model

2019-03-29T00:00:00-05:00

Abstract

I examine the strengths and weaknesses of the The xUnit and Spec testing models in terms of expressiveness, extensibility and their ability to reuse test code. I then propose a revised model which, by treating fixtures as a first-class abstraction, significantly improves on the existing frameworks.

Introduction

The xUnit family of testing frameworks was started by Kent Beck with SUnit in 1989. It defines tests as methods of a class, and the test fixture as fields of that class. This model is simple and well suited to Test Driven Development, but in my experience is neither very expressive nor easily extensible.

Spec (for specification) testing introduced a more expressive language in order be readable by less technical people as part of Behaviour Driven Development. Its model is also more extensible than xUnit’s, but at the expense of significant complication in the management of test fixtures.

As a long time JUnit (and occasional RSpec, unittest and Jasmine) user, neither model fully meets all my needs. I write tests to drive designs, document code, demonstrate system behaviour, prevent regressions and find existing bugs; but existing test tools don’t to allow me to flow between these roles, or to reuse test code in different modes.

This paper explores the difficulties that I have with the xUnit and Spec models and proposes a revised model that works significantly better for me.

Fixtures

A lot of the discussion that follows will revolve around test fixtures. What are they?

Wikipedia says:

A test fixture is something used to consistently test some item, device, or piece of software. Test fixtures can be found when testing electronics, software and physical devices.

The JUnit 4 Wiki says:

A test fixture is a fixed state of a set of objects used as a baseline for running tests. The purpose of a test fixture is to ensure that there is a well known and fixed environment in which tests are run so that results are repeatable.

So a test fixture is something to give us consistency / repeatability. In software we try where possible to create a fixture that encapsulates all the state that can affect the result of running the test, or is affected by the running of the test. That way, by creating a fresh fixture for each test, we can prevent one test run from affecting a later one.

There is a notable difference between a physical test fixture and a software test fixture. When testing a physical part it is mounted in the fixture - they are separate. When testing a software object, then the fixture will generally create that object - the subject under test will be a property of the fixture. This isn’t a hard and fast rule though; in particular when testing stateless objects, stand-alone functions, or external services.

xUnit

The xUnit lineage of test frameworks began with SUnit. SUnit’s model was to compose a fixture from the instance variables of a class, so that independent fixtures can be created by creating a new instance of the class. Tests are then written as methods on that class, and can reference the fixture as their own instance variables.

This model was carried forward to JUnit, NUnit and all the xUnits. Here is a (pedagogically stilted) JUnit test, written in Kotlin, for a function to move the contexts of one mutable list into another.

class MoveIntoTests {

    // the subjects under test
    lateinit var source: MutableList<String>
    lateinit var destination: MutableList<String>

    @Test
    fun `empty list to empty list`() {
        source = emptyMutableList()
        destination = emptyMutableList()
        moveInto(source, destination)
        assertEquals(emptyMutableList(), destination)
    }

    @Test
    fun `empty list to non-empty list`() {
        source = emptyMutableList()
        destination = aNonEmptyList.toMutableList()
        moveInto(source, destination)
        assertEquals(aNonEmptyList, destination)
    }

    @Test
    fun `non-empty list to empty list`() {
        source = aNonEmptyList.toMutableList()
        destination = emptyMutableList()
        moveInto(source, destination)
        assertEquals(aNonEmptyList, destination)
    }

    @Test
    fun `non-empty list to non-empty list`() {
        source = aNonEmptyList.toMutableList()
        destination = aNonEmptyListToo.toMutableList()
        moveInto(source, destination)
        assertEquals(aNonEmptyListToo + aNonEmptyList, destination)
    }

    @AfterEach
    fun `source should be empty`() {
        assertEquals(emptyMutableList(), source)
    }
}

private var aNonEmptyList = listOf("apple")
private var aNonEmptyListToo = listOf("banana", "cherry")

Here the two subjects under test (source and destination) are properties of the test class and hence form the fixture. To run the tests, JUnit collects all the methods annotated with @Test and, for each, it creates an instance of the test class and then invokes the particular test method. In this way our tests can mutate the components of the fixture safe in the knowledge that the next test will have a fresh copy.

nonEmptyList1 and nonEmptyList2 might ordinarily be properties and so also part of the fixture. In this case I have moved them into the top level scope to emphasise that, as immutable collections of immutable values, they can be shared between tests without cross-contamination of state.

This is a nice simple model, but it has some limitations.

xUnit Tests are Defined by Methods

Having tests defined as methods means that more tests need more methods - or special treatment. Consider repeating a test a number of times - the test framework has to provide a mechanism for this, as we can’t (in most static languages) define test methods at runtime. JUnit 5 has a @RepeatedTest annotation to allow us to do this. If we want to check that empty list to empty list really behaves as expected we can write something like

@RepeatedTest(value = 100, name = "attempt {currentRepetition}")
fun `empty list to empty list`() {
    source = emptyMutableList()
    destination = emptyMutableList()
    moveInto(source, destination)
    assertEquals(emptyMutableList(), destination)
}

but RepeatedTestis a facility that the JUnit authors had to provide - I couldn’t implement it myself. I know how to repeat things in Kotlin, but every time I want to repeat tests in JUnit I have to look up how to cast this magic spell.

How about repeating a test with different parameters? Again there is an annotation @ParameterizedTest, and again it is very much a special case rather than something that I as a test author could write given the model of the test framework.

Even something as simple as not running a test requires framework support. JUnit 5 ends up with an @Disabled annotation, plus @DisabledOnOs, @DisabledIfSystemProperty, @DisabledIfEnvironmentVariable, @DisabledIf - which can evaluate Javascript expressions but not (out of the box) the language in which I am actually writing the tests, and finally an entire ExecutionCondition extension API.

These limitations are caused by using methods as the way of defining tests, without providing any good way to treat them as first-class units of execution. There have been attempts to address this - JUnit 4 introduced TestRules -

an alteration in how a test method, or set of test methods, is run and reported.

and Theories

The Theories runner allows to test a certain functionality against a subset of an infinite set of data points.

but these are both object-oriented knives in what should be a functional gunfight. The most promising avenue is JUnit 5’s Dynamic Tests; unfortunately this appears to be a completely different execution model to the rest of the framework and so does not integrate well with ‘normal’ tests.

Only One Fixture Setup per xUnit Class

Another limitation with the xUnit model is that every test in a class shares the same fixture setup code. In our example source and destination need to be initialised differently for each test, so each test has to do that initialisation. This leads to the ugliness of the lateinit - our way of telling Kotlin that someone will populate that value dreckly. An alternative is to create a different test class for each initial fixture state, or to nest classes to express context. Both are not the standard model and take significant effort to maintain.

This might not seem like a major problem, but tests are often trying to do the difficult job of communicating to a human how a complicated system will behave in different circumstances. When that communication is through examples it is important for the human to be able to see the commonality and difference between the examples without expending precious mental bandwidth.

The quest for tests so expressive that product owners could understand what they are telling us about the behaviour of our systems motivated the creation of another test framework family, the Specs.

Spec(ification)s Bring Structure

Spec-like frameworks, beginning I believe with RSpec, allow us to demonstrate behaviour in different contexts. In the Kotlin framework Spek our previous example might look like this.

object MoveIntoSpec : Spek({

    describe("moveInto") {
        context("empty list to empty list") {
            val source = emptyMutableList()
            val destination = emptyMutableList()

            it("should not change source or destination") {
                moveInto(source, destination)
                assertEquals(emptyMutableList(), destination)
                assertEquals(emptyMutableList(), source)
            }
        }

        context("empty list to non-empty list") {
            val source = emptyMutableList()
            val destination = aNonEmptyList.toMutableList()

            it("should not change source or destination") {
                moveInto(source, destination)
                assertEquals(aNonEmptyList, destination)
                assertEquals(emptyMutableList(), source)
            }
        }

        context("non-empty list to empty list") {
            val source = aNonEmptyList.toMutableList()
            val destination = emptyMutableList()

            it("should move source items into destination") {
                moveInto(source, destination)
                assertEquals(aNonEmptyList, destination)
                assertEquals(emptyMutableList(), source)
            }
        }

        context("non-empty list to non-empty list") {
            val source = aNonEmptyList.toMutableList()
            val destination = aNonEmptyListToo.toMutableList()

            it("should move source items into destination") {
                moveInto(source, destination)
                assertEquals(aNonEmptyListToo + aNonEmptyList, destination)
                assertEquals(emptyMutableList(), source)
            }
        }
    }
})

Look at how the variable parts of the fixture, source and destination, agree with the context description. We have named a context and expressed it in code. This structure is difficult to impose with xUnit frameworks, but is the point of specs. xUnit only allows a single fixture setup and then we’re straight into tests - here different contexts can set up the fixtures in different ways before their tests are run. Specs allow the Given of Given When Then to be expressed in the context structure.

Specs are Dynamic

A convenient side-effect of the Spec model is that those context and it blocks are evaluated at runtime, which means that in Spek, if I want to be really sure that moving an empty list into an empty list results in an empty list, I can just write plain old code to repeatedly create tests.

context("empty list to empty list") {
    val source = emptyMutableList()
    val destination = emptyMutableList()

    (1..100).forEach { run ->
        it("should not change source or destination attempt $run") {
            moveInto(source, destination)
            assertEquals(emptyMutableList(), destination)
            assertEquals(emptyMutableList(), source)
        }
    }
}
// Note that we will see later that this test is logically flawed

Parameterised tests are almost as easy to write in Spec-like frameworks. As we will see later - no need to look up what magic annotations your framework authors have conjured up - just define a source of parameter values and create contexts or tests on the fly for whatever combinations you like. Compared to xUnit our tests (and contexts) have been promoted to first-class status, so that we can bring the tools of functional programming to bear.

There are problems hiding in those contexts though.

Spec Fixture Lifecycle

Your obsessive-compulsive side might think that there should be two levels of nested contexts in this example, so that we can better express their commonality and difference. As someone who just had to look up whether to hyphenate obsessive-compulsive, I agree.

object MoveIntoNestedSpec : Spek({
    describe("moveInto") {
        context("empty source") {
            val source = emptyMutableList()

            context("empty destination") {
                val destination = emptyMutableList()

                it("should not change destination") {
                    moveInto(source, destination)
                    assertEquals(emptyMutableList(), destination)
                }
            }
            context("non-empty destination") {
                val destination = aNonEmptyList.toMutableList()

                it("should not change destination") {
                    moveInto(source, destination)
                    assertEquals(aNonEmptyList, destination)
                }
            }

            afterEach {
                // source should always be emptied
                assertEquals(emptyMutableList(), source)
            }
        }

        context("non-empty source") {
            val source = aNonEmptyList.toMutableList()

            context("empty destination") {
                val destination = emptyMutableList()

                it("should contain just the source") {
                    moveInto(source, destination)
                    assertEquals(aNonEmptyList, destination)
                }
            }

            context("non-empty destination") {
                val destination = aNonEmptyListToo.toMutableList()

                it("should contain destination plus source") {
                    moveInto(source, destination)
                    assertEquals(aNonEmptyListToo + aNonEmptyList, destination)
                }
            }

            afterEach {
                assertEquals(emptyMutableList(), source)
            }
        }
    }
})

Ah - that’s so much better.

It’s such a shame that it doesn’t work.

The problem is that those source variables are shared between the child contexts and are mutable. In the non-empty source context, val source = nonEmptyList1.toMutableList() is executed only once, when the contexts are being defined, before running the tests. So should contain destination plus source finds that source has been cleared out by the previous test and there is nothing to add.

There are two common solutions to this problem in Spec-like tests. The first is the use of before blocks to initialise mutable fixture state

context("non-empty source") {
    lateinit var source: MutableList<String>
    beforeEach {
        source = aNonEmptyList.toMutableList()
    }
    //...

    context("non-empty destination") {
        lateinit var destination: MutableList<String>
        beforeEach {
            destination = aNonEmptyListToo.toMutableList()
        }

        it("should contain destination plus source") {
            moveInto(source, destination)
            assertEquals(aNonEmptyListToo + aNonEmptyList, destination)
        }
    }
}

The second is the use of a special lazy construct - Spek calls this memoized

context("non-empty source") {
    val source by memoized { aNonEmptyList.toMutableList() }

    //...
    context("non-empty destination") {
        val destination by memoized { aNonEmptyListToo.toMutableList() }

        it("should contain destination plus source") {
            moveInto(source, destination)
            assertEquals(aNonEmptyListToo + aNonEmptyList, destination)
        }
    }
}

The beforeEach method is tedious, error-prone, and doesn’t sit well with an immutable, functional style. Initialisation blocks like memoized are nicer, but force users to internalise the test lifecycle and differentiate between the handling of mutable and immutable state. Everything can appear to be fine until you add a test, or reorder what is there, and the tests that were passing now fail.

Spec Fixture Scope

Another problem with the Spec model is that tests rely on receiving the fixture properties in their lexical scope. As a result extracting common code and reusing it between tests is hard, at least in statically typed languages. If you try to extract it("should contain destination plus source") for use in another test you’ll find it is tied into place by source and destination. In xUnit, where the fixture has a type (the type of the test class), we routinely extract a supertype with the fixture properties and methods that rely on these properties. Concrete classes can then inherit from this to share fixtures, setup, tests and utilities. We can’t do that in the Spec model, so have to fall back on framework-provided special cases.

The Best of Both Worlds

In summary: xUnit has a simple fixture model that we can understand and reuse, but limited facilities for expressing fixture variations, and poor support for manipulating tests as first class constructs; Specs address these limitations of xUnit, but at the expense of a complicated and not easily reusable fixture model.

What if we could combine the nested scopes and programmatic test generation of the Spec model with the simple and predictable fixtures of the xUnit system? We would then be able to execute business-friendly specs and security-friendly fuzz testing. We could run a test for every file in a directory of examples or skip a test every Tuesday afternoon without recourse to the framework manual. And we would be able to combine test contracts into suites of tests for concrete implementations.

Try this One Weird Trick

The key to these benefits is to promote the fixture to a first-class abstraction, separate from the tests.

There are two aspects to this promotion. The first aspect is type, the second lifecycle. In xUnit the type of the fixture is the type of the test class, and they share the same lifecycle. In Specs, the fixture lifecycle is decoupled from the test lifecycle, but there is no separate fixture type.

Let’s see what happens when we extract a fixture type in our nested context Spek test.

object MoveIntoNestedFixtureSpec : Spek({

    // Create a fixture class with all our mutable test state
    data class Fixture(
        val source: MutableList<String> = emptyMutableList(),
        val destination: MutableList<String> = emptyMutableList()
    )

    describe("moveInto") {
        //...
        context("non-empty source") {
            val fixture by memoized {
                Fixture(aNonEmptyList.toMutableList())
            }

            context("empty destination") {
                it("should contain just the source") {
                    moveInto(fixture.source, fixture.destination)
                    assertEquals(aNonEmptyList, fixture.destination)
                }
            }

            context("non-empty destination") {
                val fixture by memoized {
                    fixture.copy(destination = aNonEmptyListToo.toMutableList())
                    // ↑ this fixture is the parent fixture
                }
                it("should contain destination plus source") {
                    moveInto(fixture.source, fixture.destination)
                    assertEquals(aNonEmptyListToo + aNonEmptyList, fixture.destination)
                }
            }

            afterEach {
                assertEquals(emptyMutableList(), fixture.source)
            }
        }
    }
})

In exchange for explicitly creating and managing the fixture down the context tree, we have a simple rule - if a test references a mutable thing, either in setup or execution, initialise that thing in the fixture and initialise the fixture in a by memoized block.

Now that the fixture has its own type, you can also see that the tests and common setup code can be extracted and reused, as they now depend only on Fixture (and constants).

The downside of this approach is the irritation of not having access to the fixture properties without qualification - fixture.source.moveInto(fixture.destination) gets old pretty quickly compared to having the properties directly in scope. If we could solve that problem then this model could meet our design goal of having the flexibility and expressiveness of a Spec with the predictable model and fixture reuse of an xUnit.

Introducing Minutest

Minutest is a Kotlin library created to explore this approach. It takes advantage of a Kotlin language feature that allows the value of the implicit receiver this to be provided by framework code. Because fixtures are special, in that there will only be one instance per test, Minutest allows the test developer to specify how to create, mutate and translate the single instance in contexts, and then returns it to the tests as the implicit receiver.

Let’s look at our contexts in Minutest.

class MoveIntoMinutests : JUnit5Minutests {

    data class Fixture(
        val source: MutableList<String> = emptyMutableList(),
        val destination: MutableList<String> = emptyMutableList()
    )

    fun tests() = rootContext<Fixture> // Contexts are parameterised by the fixture type
    {
        context("empty source") {

            // The fixture block is run once per test and must return an instance of the fixture
            fixture {
                Fixture(emptyMutableList(), emptyMutableList())
            }

            context("empty destination") {
                test("should not be changed") {
                    // here 'this' is the fixture inherited from the 'empty source'
                    moveInto(source, destination)
                    assertEquals(emptyMutableList(), destination)
                }
            }

            context("non-empty destination") {

                // the deriveFixture block translates from the parent fixture
                deriveFixture {
                    parentFixture.copy(destination = aNonEmptyList.toMutableList())
                }

                test("should not be changed") {
                    moveInto(source, destination)
                    assertEquals(aNonEmptyList, destination)
                }
            }

            after {
                // after blocks also have the fixture as 'this'
                assertEquals(emptyMutableList(), source)
            }
        }

        context("non-empty source") {
            fixture {
                Fixture(aNonEmptyList.toMutableList())
            }

            context("empty destination") {
                test("should contain just the source") {
                    moveInto(source, destination)
                    assertEquals(aNonEmptyList, destination)
                }
            }

            context("non-empty destination") {
                deriveFixture {
                    parentFixture.copy(destination = aNonEmptyListToo.toMutableList())
                }

                test("should contain destination plus source") {
                    moveInto(source, destination)
                    assertEquals(aNonEmptyListToo + aNonEmptyList, destination)
                }
            }

            after {
                assertEquals(emptyMutableList(), source)
            }
        }
    }
}

Here the fixture block is called by the framework once for each test, so that stateful fixtures will not contaminate future tests. deriveFixture allows a context to change the fixture from its parent. All fixture, test and before and after blocks have access to the value of the fixture in their context as this.

This model has proved to be both expressive and flexible. As a Spec, Minutest allows easy manipulation of contexts and tests, so that we can quickly refactor the previous example to a parameterised test without having to look up how to use a special annotation.

class MoveIntoParameterisedMinutests : JUnit5Minutests {

    data class Fixture(
        val source: MutableList<String> = emptyMutableList(),
        val destination: MutableList<String> = emptyMutableList()
    ) {
        override fun toString() = "$source and $destination"
    }

    val scenarios = listOf(
        Fixture(emptyMutableList(), emptyMutableList()),
        Fixture(aNonEmptyList.toMutableList(), emptyMutableList()),
        Fixture(emptyMutableList(), aNonEmptyListToo.toMutableList()),
        Fixture(aNonEmptyList.toMutableList(), aNonEmptyListToo.toMutableList())
    )

    fun tests() = rootContext<Fixture> {
        scenarios.forEach { scenario ->
            context("Given $scenario") {
                fixture {
                    scenario
                }
                test("moves items from source to destination") {
                    val originalSource = source.toList()
                    val originalDestination = destination.toList()
                    moveInto(source, destination)
                    assertEquals(originalDestination + originalSource, destination)
                    assertEquals(emptyMutableList(), source)
                }
            }

            willRun(
                "root",
                "  Given [] and []",
                "    moves items from source to destination",
                "  Given [apple] and []",
                "    moves items from source to destination",
                "  Given [] and [banana, cherry]",
                "    moves items from source to destination",
                "  Given [apple] and [banana, cherry]",
                "    moves items from source to destination"
            )
        }
    }
}

This is of course overkill for our little example, but the ability to compose tests, contexts and fixtures allows us to reuse them all in creative ways. Let’s generalise our previous example to any mutable collection

data class Fixture(
    val source: MutableCollection<String>,
    val destination: MutableCollection<String>
)

fun ContextBuilder<Fixture>.supportsMoveTo(
    collectionOf: (List<String>) -> MutableCollection<String>
) {
    context("empty source") {
        //...
    }
    context("non-empty source") {
        fixture {
            Fixture(collectionOf(aNonEmptyList), collectionOf(emptyList()))
        }

        context("empty destination") {
            test("should contain just the source") {
                moveInto(source, destination)
                assertEquals(collectionOf(aNonEmptyList), destination)
            }
        }

        context("non-empty destination") {
            deriveFixture {
                parentFixture.copy(destination = collectionOf(aNonEmptyListToo))
            }

            test("should contain destination plus source") {
                moveInto(source, destination)
                assertEquals(collectionOf(aNonEmptyListToo + aNonEmptyList), destination)
            }
        }

        after {
            assertEquals(collectionOf(emptyList()), source)
        }
    }
}

/*-
We can use this contract to test with different `MutableCollection`s by a specifying the concrete factory `collectionOf`
-*/

class ArrayListTests : JUnit5Minutests {
    fun tests() = rootContext<Fixture> {
        supportsMoveTo { content -> ArrayList(content) }
    }
}

class LinkedListTests : JUnit5Minutests {
    fun tests() = rootContext<Fixture> {
        supportsMoveTo { content -> LinkedList(content) }
    }
}

class SetTests : JUnit5Minutests {
    fun tests() = rootContext<Fixture> {
        supportsMoveTo { content -> LinkedHashSet(content) }
    }
}

Other Features

Minutest users are still exploring ways to exploit this revised test model to make tests more expressive and comprehensive. Other features that become simple to implement include:

Property based testing
Sharing fixtures between test runs
Immutable fixtures
Mapping from one fixture type to another to expand or contract the focus of tests.
Logging test execution
Refactoring - for example flat TDD tests to a Spec or property-based tests.

Other Implementations

Minutest is a Kotlin library that integrates with JUnit 5 (and experimentally JUnit 4) to run its tests and provide assertions. While some of the expressiveness in the test specification requires Kotlin language features, I believe that a similar model could be implemented in other languages. In fact a similar, albeit dynamically typed, fixture model is available for Jasmine.

Conclusions

By giving the fixture its own type, the problems of fixture lifecycle and reuse inherent in the Spec model can be mitigated. A framework such as Minutest that embraces this approach can reap further rewards in test expression, productivity and flexibility.

Property Based Testing With Minutest

2019-02-22T00:00:00-06:00

This is the bonus episode of a mini-series on my new test library, Minutest.

In Part 1 we looked at the Test Driven Development of an extension function, partition.

In Part 2 we refactored the code into a Spec -style, with nested contexts giving a more formal specification of what the code does.

In this part I’ll look at a different style of testing with Minutest - Property Based Testing. Instead of constructing example inputs and testing against expected outcomes, Property Based Testing throws many inputs at a system and checks the outcomes against a set of properties that we can say should hold true independent of the exact input.

The Code Under Test

Let’s remind ourselves of the code we’re testing.

/**
 * Partitions the receiver into lists which contain the items that match the input corresponding predicate.
 *
 * If multiple predicates match an item, the item is attributed to the first which matches.
 */
fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> {
    val grouped: Map<((T) -> Boolean)?, List<T>> = this.groupBy { item ->
        predicates.firstMatch(item)
    }
    return predicates.map { grouped.getOrDefault(it, emptyList()) }
}

private fun <T> Iterable<(T) -> Boolean>.firstMatch(item: T): ((T) -> Boolean)? =
    this.find { predicate -> predicate(item) }

What Are Our Properties?

The key to Property Based Testing is to find properties of outcomes that are true no matter what the input. For example, the length of the output list will be the same as the length of the predicates, no matter what predicates are provided. What other properties can you think of?

Yes, it isn’t easy.

Let’s park that for a while while we implement a bit of mechanism.

Unrolled Tests

What we want to do is run the same tests for a set of input values. We’ll represent the inputs with a Fixture data class and write the following.

class PartitionPropertyTests : JUnit5Minutests {
    data class Fixture(
        val items: List<Int>,
        val predicates: List<(Int) -> Boolean>
    )

    fun tests() = rootContext<Fixture> {
        context("empty predicates") {
            fixture { Fixture(emptyList(), emptyList()) }
            test("result size equals predicates size") {
                assertEquals(predicates.size, items.partition(predicates).size)
            }
            test("something else that should be true for any fixure") {}
        }
        context("some predicates") {
            fixture { Fixture(emptyList(), listOf(::isNegative, ::isZero, ::isPositive)) }
            test("result size equals predicates size") {
                assertEquals(predicates.size, items.partition(predicates).size)
            }
            test("something else that should be true for any fixture") {}
        }
        // we could go on
    }
}

Building Contexts In Code

Now obviously this doesn’t scale. Luckily Minutest is All Just Kotlin™ so we can build those contexts in code.

val fixtures = listOf(
    Fixture(emptyList(), emptyList()),
    Fixture(emptyList(), listOf(::isNegative, ::isZero, ::isPositive))
)

fun tests() = rootContext<Fixture> {
    fixtures.forEach { fixture ->
        context("Inputs $fixture") {
            fixture { fixture }
            test("result size equals predicates size") {
                assertEquals(predicates.size, items.partition(predicates).size)
            }
            test("something else that should be true for any fixture") {}
        }
    }
}

Scale Out and Down

You can probably see where this is going - all we have to do is flesh-out the set of fixtures and tests.

data class Fixture(
    val items: List<Int>,
    val predicates: List<(Int) -> Boolean>
) {
    fun result() = items.partition(predicates)
}

fun fixtures() = combinationsOf(
    listOf(
        emptyList(),
        listOf(1),
        listOf(-1, 0, 1, 2, 3),
        listOf(3, 2, 1, 0, -1),
        listOf(-1, -1, 0, 0, 1, 2, 3)
    ),
    listOf(
        emptyList(),
        listOf(::isZero),
        listOf(::isNegative, ::isZero, ::isPositive),
        //...
        listOf(::isPositive, ::isBiggerThan10)
    )
).map { (items, predicates) -> Fixture(items, predicates) }

fun tests() = rootContext<Fixture> {
    fixtures().forEach { fixture ->
        context("Inputs $fixture") {
            fixture { fixture }
            test("result size equals predicates size") {
                assertEquals(predicates.size, result().size)
            }
            test("first output has every item in input that matches first predicate") {
                if (predicates.isNotEmpty()) {
                    assertEquals(items.filter { predicates[0](it) }, result()[0])
                }
            }
            test("no item in output doesn't match the corresponding predicate") {
                predicates.zip(result()).forEach { (predicate, resultForPredicate) ->
                    assertTrue(resultForPredicate.filterNot(predicate).isEmpty())
                }
            }
            test("every output item is in the input") {
                val outputs = result().flatten().toSet()
                assertTrue((outputs - items).isEmpty())
            }
            // ...
        }
    }
}

I’m not an experienced Property Based Tester - I’m still finding my feet when it comes to finding good inputs and properties that are minimal and expressive - but I do know that every time I thought of a new property, or came up with an interesting new list of items or predicates, my confidence in the implementation increased. The testing was exploratory, but with the advantages of automation and combinatorial growth. Not only that, but each new case increased my confidence that any changes to the behaviour caused by refactoring would be caught by one of the hundreds of tests generated this way.

Generating Inputs

There are other ways of generating the inputs for Property Based Testing. One would be to generate random lists of items and predicates rather than combinations of given ones. Another would be to start with some givens and mutate them, either consistently or randomly. Both would give more confidence that we hadn’t rigged the input, at the expense of having to manage the randomness.

It’s also worth noting that if we generate many more tests we may come across limitations in our test reporting tools. I think that you could use Minutest to generate only one outer context and one inner test in these circumstances, but collect any failures from all fixtures into a single result per test. I’ll write another post if I can.

Putting It All Together

You are not alone if you couldn’t read these tests and predict what the partition method actually did. Maybe I’m just not good enough at specifying the properties - or maybe we do need human-centred examples in that role. Luckily there is nothing to stop us having the examples for humans and the property based tests for regression. We can merge the specification tests from last time with these, and run them both with Minutest - they can even share the same Fixture.

class PartitionTests : JUnit5Minutests {

    data class Fixture(
        val items: List<Int> = emptyList(),
        val predicates: List<(Int) -> Boolean> = emptyList()
    ) {
        fun result() = items.partition(predicates)

        fun assertResultIs(vararg expected: List<Int>) {
            assertEquals(expected.asList(), result())
        }
    }

    fun specs() = rootContext<Fixture> {
        fixture { Fixture() }

        context("some items") {
            withItems(-1, 0, 1, 2, 3)
            context("no predicates") {
                test("returns empty list") {
                    assertResultIs()
                }
            }
            context("everything matches something") {
                withPredicates(::isNegative, ::isZero, ::isPositive)
                test("returns a list for each predicate") {
                    assertResultIs(listOf(-1), listOf(0), listOf(1, 2, 3))
                }
            }
            context("a predicate doesn't match any item") {
                withPredicates(::isBiggerThan10, ::isNegative, ::isZero, ::isPositive)
                test("returns an empty list for that predicate") {
                    assertResultIs(emptyList(), listOf(-1), listOf(0), listOf(1, 2, 3))
                }
            }
            context("an item doesn't match any predicate") {
                withPredicates(::isNegative, ::isPositive)
                test("item is not in the returned lists") {
                    assertResultIs(listOf(-1), listOf(1, 2, 3))
                }
            }
            context("an item matches more than one predicate") {
                withPredicates(::isBiggerThan10, ::isNegative, ::isZero, ::isPositive, ::isBiggerThan1)
                test("item is assigned to the first matching predicate") {
                    assertResultIs(emptyList(), listOf(-1), listOf(0), listOf(1, 2, 3), emptyList())
                }
            }
            context("repeated predicates") {
                withPredicates(::isNegative, ::isNegative, ::isZero, ::isPositive)
                test("item is assigned to each predicate") {
                    assertResultIs(listOf(-1), listOf(-1), listOf(0), listOf(1, 2, 3))
                }
            }
            context("lambda predicates") {
                withPredicates({ x -> isNegative(x) }, { x -> isPositive(x) })
                test("item is assigned to each predicate") {
                    assertResultIs(listOf(-1), listOf(1, 2, 3))
                }
            }
            context("items in a different order") {
                withItems(3, 2, 1, 0, -1)
                withPredicates(::isNegative, ::isZero, ::isPositive)
                test("input order is preserved") {
                    assertResultIs(listOf(-1), listOf(0), listOf(3, 2, 1))
                }
            }
        }
        context("no items") {
            context("no predicates") {
                test("returns empty list") {
                    assertResultIs()
                }
            }
            context("some predicates") {
                withPredicates(::isNegative, ::isZero, ::isPositive)
                test("returns an empty list for each predicate") {
                    assertResultIs(emptyList(), emptyList(), emptyList())
                }
            }
        }
    }

    private fun fixtures() = combinationsOf(
        listOf(
            emptyList(),
            listOf(1),
            listOf(-1, 0, 1, 2, 3),
            listOf(3, 2, 1, 0, -1),
            listOf(-1, -1, 0, 0, 1, 2, 3)
        ),
        listOf(
            emptyList(),
            listOf(::isZero),
            listOf(::isNegative, ::isZero, ::isPositive),
            listOf(::isPositive, ::isZero, ::isNegative),
            listOf(::isZero, ::isZero, ::isPositive),
            listOf(::isZero, ::isZero, ::isPositive, ::isPositive),
            listOf(::isPositive, ::isBiggerThan1),
            listOf(::isPositive, ::isBiggerThan10)
        )
    ).map { (items, predicates) -> Fixture(items, predicates) }

    fun properties() = rootContext<Fixture> {
        fixtures().forEach { fixture ->
            context("Inputs $fixture") {
                fixture { fixture }
                test("result size equals predicates size") {
                    assertEquals(predicates.size, result().size)
                }
                test("first output has every item in input that matches first predicate") {
                    if (predicates.isNotEmpty()) {
                        assertEquals(items.filter { predicates[0](it) }, result()[0])
                    }
                }
                test("no item in output doesn't match the corresponding predicate") {
                    predicates.zip(result()).forEach { (predicate, resultForPredicate) ->
                        assertTrue(resultForPredicate.filterNot(predicate).isEmpty())
                    }
                }
                test("every output item is in the input") {
                    assertTrue((result().flatten().toSet() - items).isEmpty())
                }
                test("the order of items in each output is the same as the order in the input") {
                    result().forEach { listInResult ->
                        val correspondingInputItems = items.filter { listInResult.contains(it) }
                        assertEquals(correspondingInputItems, listInResult)
                    }
                }
                test("no item is in more than one output") {
                    // Outputs for the same predicate are considered the same output
                    result().forEach { listInResult ->
                        val otherItems = result().filterNot { it == listInResult }.flatten()
                        assertEquals(listInResult - otherItems, listInResult)
                    }
                }
                test("identical predicates result in the same output") {
                    val resultsBySamePredicate: Map<(Int) -> Boolean, List<List<Int>>> =
                        predicates.zip(result()).groupBy(
                            { (predicate, _) -> predicate }, { (_, resultForPredicate) -> resultForPredicate }
                        ).filterValues { it.size > 1 }
                    resultsBySamePredicate.values.forEach { resultForPredicate ->
                        assertTrue(resultForPredicate.size > 1)
                        assertTrue(resultForPredicate.itemsAreTheSame())
                    }
                }
            }
        }
    }
    private fun <T> Iterable<T>.itemsAreTheSame() = this.toSet().size == 1

    private fun TestContextBuilder<Fixture, Fixture>.withItems(vararg items: Int) =
        before_ { copy(items = items.asList()) }

    private fun TestContextBuilder<Fixture, Fixture>.withPredicates(vararg predicates: (Int) -> Boolean) =
        before_ { copy(predicates = predicates.asList()) }
}

Conclusion

Property Based Tests allow us to specify the behaviour of code in a very different way to the usual examples or specifications. They can allow easy working of edge-cases, increase our rigour and give increased robustness against regressions, but may not communicate well with the users of our code. Combining specifications and property based tests looks promising, especially where the tests can share infrastructure and setup.

Test Driven to Specification with Minutest - Part 2

2019-02-17T00:00:00-06:00

This is the second part of a mini-series on Minutest.

In Part 1 we looked at the Test Driven Development of an extension function, partition.

In this installment we’ll look at refactoring the (in many ways pretty arbitrary) tests that resulted into a more formal specification of what our code does.

The Code Under Test

To remind ourselves - here is the code:

fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> {
    val grouped: Map<((T) -> Boolean)?, List<T>> = this.groupBy { item ->
        predicates.firstMatch(item)
    }
    return predicates.map { grouped.getOrDefault(it, emptyList()) }
}

private fun <T> Iterable<(T) -> Boolean>.firstMatch(item: T): ((T) -> Boolean)? =
    this.find { predicate -> predicate(item) }

The TDD Tests

Here are the tests from the end of our TDD session.

class PartitionTests : JUnit5Minutests {
    fun tests() = rootContext<Unit>() {
        test("every item matches a predicate and every predicate matches an item") {
            val items = listOf(-1, 0, 1, 2, 3)
            val predicates = listOf(::isNegative, ::isZero, ::isPositive)
            val result = items.partition(predicates)
            assertEquals(listOf(listOf(-1), listOf(0), listOf(1, 2, 3)), result)
        }
        test("an item matches no predicate") {
            val items = listOf(-1, 0, 1, 2, 3)
            val predicates = listOf(::isZero, ::isPositive)
            val result = items.partition(predicates)
            assertEquals(listOf(listOf(0), listOf(1, 2, 3)), result)
        }
        test("a predicate matches no item") {
            val items = listOf(0, 1, 2, 3)
            val predicates = listOf(::isNegative, ::isZero, ::isPositive)
            val result = items.partition(predicates)
            assertEquals(listOf(listOf(), listOf(0), listOf(1, 2, 3)), result)
        }
        test("some items no predicates") {
            val items = listOf(-1, 0, 1, 2, 3)
            val predicates = emptyList<(Int) -> Boolean>()
            val result = items.partition(predicates)
            assertEquals(emptyList<List<Int>>(), result)
        }
        test("some predicates no items") {
            val items = emptyList<Int>()
            val predicates = listOf(::isNegative, ::isZero, ::isPositive)
            val result = items.partition(predicates)
            assertEquals(listOf(emptyList<Int>(), emptyList(), emptyList()), result)
        }
        test("no predicates no items") {
            val items = emptyList<Int>()
            val predicates = emptyList<(Int) -> Boolean>()
            val result = items.partition(predicates)
            assertEquals(emptyList<List<Int>>(), result)
        }
    }
}

Personally, even though we’ve tried to be formal in the names of our tests, I find that it’s hard to see the structure in this flat list. Another problem is that the test names tell us the conditions - the ‘whens’, but don’t actually tell us what the assertions mean. We can fix that with more words, but then we have a sea of text to deal with.

Finally there is all that duplication.

Removing Duplication

I guess that’s the easiest thing to address. Let’s extract a check function.

fun tests() = rootContext<Unit>() {
    test("every item matches a predicate and every predicate matches an item") {
        val items = listOf(-1, 0, 1, 2, 3)
        val predicates = listOf(::isNegative, ::isZero, ::isPositive)
        check(items, predicates, listOf(listOf(-1), listOf(0), listOf(1, 2, 3)))
    }
    test("an item matches no predicate") {
        val items = listOf(-1, 0, 1, 2, 3)
        val predicates = listOf(::isZero, ::isPositive)
        check(items, predicates, listOf(listOf(0), listOf(1, 2, 3)))
    }
    test("a predicate matches no item") {
        val items = listOf(0, 1, 2, 3)
        val predicates = listOf(::isNegative, ::isZero, ::isPositive)
        check(items, predicates, listOf(listOf(), listOf(0), listOf(1, 2, 3)))
    }
    test("some items no predicates") {
        val items = listOf(-1, 0, 1, 2, 3)
        val predicates = emptyList<(Int) -> Boolean>()
        check(items, predicates, emptyList())
    }
    test("some predicates no items") {
        val items = emptyList<Int>()
        val predicates = listOf(::isNegative, ::isZero, ::isPositive)
        check(items, predicates, listOf(emptyList(), emptyList(), emptyList()))
    }
    test("no predicates no items") {
        val items = emptyList<Int>()
        val predicates = emptyList<(Int) -> Boolean>()
        check(items, predicates, emptyList())
    }
}

private fun <T> check(items: List<T>, predicates: List<(T) -> Boolean>, expected: List<List<T>>) {
    assertEquals(expected, items.partition(predicates))
}

That’s a bit better, or at least a bit shorter. Actually our cognitive load looking at the tests is reduced, because we can be reasonably sure that each is doing the same thing.

Grouping into Contexts

We’re still left with a flat list of 6 tests though. Minutest allows the grouping of tests into contexts - let’s try just grouping what we have.

fun tests() = rootContext<Unit>() {
    context("some items") {
        test("every item matches a predicate and every predicate matches an item") {
            // ...
        }
        test("an item matches no predicate") {
            // ...
        }
        test("a predicate matches no item") {
            // ...
        }
        test("no predicates") {
            // ...
        }
    }
    context("no items") {
        test("some predicates") {
            // ...
        }
        test("no predicates") {
            // ...
        }
    }
}

This helps a bit, but there is no relationship in code between the context and its name. That would be a nice feature, and Minutest supports this with fixtures. I think of the fixture as the world of the tests - all the state we need to concern ourselves with.

Adding a Fixture

At the moment the fixture type is Unit - the one in rootContext<Unit>. Let’s change that to be a class containing the items that we want to test with. This seems overkill for now, but bear with me.

class PartitionTests : JUnit5Minutests {

    class Fixture(val items: List<Int>)

    fun tests() = rootContext<Fixture>() {
        context("some items") {
            // ...
            test("ok") {}
        }
    }
}

If you run this you get a ~~nasty ClassCastException which is a bad way of~~ IllegalStateException: Fixture has not been set in context "some items" telling you that you need to tell Minutest how to provide a fixture to the tests.

We can do that with a fixture block, which returns the fixture that should be used in tests in the context. So in some items we use Fixture(listOf(-1, 0, 1, 2, 3)), and in no items, Fixture(emptyList()).

class PartitionTests : JUnit5Minutests {

    class Fixture(val items: List<Int>)

    fun tests() = rootContext<Fixture>() {
        context("some items") {
            fixture {
                Fixture(listOf(-1, 0, 1, 2, 3))
            }
            test("every item matches a predicate and every predicate matches an item") {
                val predicates = listOf(::isNegative, ::isZero, ::isPositive)
                check(items, predicates, listOf(listOf(-1), listOf(0), listOf(1, 2, 3)))
            }
            test("an item matches no predicate") {
                val predicates = listOf(::isZero, ::isPositive)
                check(items, predicates, listOf(listOf(0), listOf(1, 2, 3)))
            }
            test("a predicate matches no item") {
                // Note that we have added a predicate to allow the same items as the other tests in this context
                val predicates = listOf(::isBiggerThan10, ::isNegative, ::isZero, ::isPositive)
                check(items, predicates, listOf(listOf(), listOf(-1), listOf(0), listOf(1, 2, 3)))
            }
            test("no predicates") {
                val predicates = emptyList<(Int) -> Boolean>()
                check(items, predicates, emptyList())
            }
        }
        context("no items") {
            fixture {
                Fixture(emptyList())
            }
            test("some predicates") {
                val predicates = listOf(::isNegative, ::isZero, ::isPositive)
                check(items, predicates, listOf(emptyList(), emptyList(), emptyList()))
            }
            test("no predicates") {
                val predicates = emptyList<(Int) -> Boolean>()
                check(items, predicates, emptyList())
            }
        }
    }
}

The reason that this compiles is that inside a test block, this is the fixture that was created, so that items now refers to Fixture.items.

Now we can do the same with predicates, introducing a context for the different values.

class PartitionTests : JUnit5Minutests {

    data class Fixture(val items: List<Int>, val predicates: List<(Int) -> Boolean>)

    fun tests() = rootContext<Fixture>() {
        context("some items") {
            fixture {
                Fixture(listOf(-1, 0, 1, 2, 3), emptyList())
            }
            context("every item matches a predicate and every predicate matches an item") {
                deriveFixture {
                    // derive fixture allows us to take the parent fixture and return a derivation
                    parentFixture.copy(predicates = listOf(::isNegative, ::isZero, ::isPositive))
                }
                test("returns a list for each predicate") {
                    check(items, predicates, listOf(listOf(-1), listOf(0), listOf(1, 2, 3)))
                }
            }
            context("an item matches no predicate") {
                deriveFixture {
                    parentFixture.copy(predicates = listOf(::isZero, ::isPositive))
                }
                test("item is not in the returned lists") {
                    check(items, predicates, listOf(listOf(0), listOf(1, 2, 3)))
                }
            }
            context("a predicate matches no item") {
                deriveFixture {
                    parentFixture.copy(
                        predicates = listOf(::isBiggerThan10, ::isNegative, ::isZero, ::isPositive)
                    )
                }
                test("returns an empty list for that predicate") {
                    check(items, predicates, listOf(emptyList(), listOf(-1), listOf(0), listOf(1, 2, 3)))
                }
            }
            context("no predicates") {
                test("returns empty list") {
                    check(items, predicates, emptyList())
                }
            }
        }
        context("no items") {
            //...
        }
    }
}

You can see now that there is a direct relationship between the (nested) name of the context and the state of the fixture. Not only that, but because the conditions are moved out into the contexts, the tests are free to describe just the result - eg item is not in the returned lists.

Removing More Duplication

A little bit of refactoring gives the following.

class PartitionTests : JUnit5Minutests {

    data class Fixture(
        val items: List<Int> = emptyList(),
        val predicates: List<(Int) -> Boolean> = emptyList()
    ) {
        fun assertResultIs(vararg expected: List<Int>) {
            assertEquals(expected.asList(), items.partition(predicates))
        }

        fun withPredicates(vararg predicates: (Int) -> Boolean) =
            copy(predicates = predicates.asList())
    }

    fun tests() = rootContext<Fixture> {
        context("some items") {
            fixture {
                Fixture(items = listOf(-1, 0, 1, 2, 3))
            }
            context("everything matches something") {
                deriveFixture {
                    withPredicates(::isNegative, ::isZero, ::isPositive)
                }
                test("returns a list for each predicate") {
                    assertResultIs(listOf(-1), listOf(0), listOf(1, 2, 3))
                }
            }
            context("a predicate doesn't match any item") {
                deriveFixture {
                    withPredicates(::isBiggerThan10, ::isNegative, ::isZero, ::isPositive)
                }
                test("returns an empty list for that predicate") {
                    assertResultIs(emptyList(), listOf(-1), listOf(0), listOf(1, 2, 3))
                }
            }
            context("an item doesn't match any predicate") {
                deriveFixture {
                    withPredicates(::isNegative, ::isPositive)
                }
                test("item is not in the returned lists") {
                    assertResultIs(listOf(-1), listOf(1, 2, 3))
                }
            }
            context("no predicates") {
                test("returns empty list") {
                    assertResultIs()
                }
            }
        }
        context("no items") {
            //...
        }
    }
}

This is longer than the original, but, for me at least, the contexts give my brain room to consider other cases, and somewhere to put the tests that doesn’t overwhelm the reader.

Hiding Minutest

Nitpicking - I’m not happy with the deriveFixture calls there - they leak Minutest minutiae into our tests. Luckily this is All Just Kotlin™ and so we can make better named operations to do the job for us. I’m going with withItems and withPredicates.

Now our spec reads in a way that we might even show to our customers (except, perhaps, for the term predicates). We even have the cognitive space to add tests that show what happens in some special cases.

class PartitionTests : JUnit5Minutests {

    data class Fixture(
        val items: List<Int> = emptyList(),
        val predicates: List<(Int) -> Boolean> = emptyList()
    ) {
        fun assertResultIs(vararg expected: List<Int>) {
            assertEquals(expected.asList(), items.partition(predicates))
        }
    }

    fun tests() = rootContext<Fixture> {
        fixture { Fixture() }

        context("some items") {
            withItems(-1, 0, 1, 2, 3)
            context("no predicates") {
                test("returns empty list") {
                    assertResultIs()
                }
            }
            context("everything matches something") {
                withPredicates(::isNegative, ::isZero, ::isPositive)
                test("returns a list for each predicate") {
                    assertResultIs(listOf(-1), listOf(0), listOf(1, 2, 3))
                }
            }
            context("a predicate doesn't match any item") {
                withPredicates(::isBiggerThan10, ::isNegative, ::isZero, ::isPositive)
                test("returns an empty list for that predicate") {
                    assertResultIs(emptyList(), listOf(-1), listOf(0), listOf(1, 2, 3))
                }
            }
            context("an item doesn't match any predicate") {
                withPredicates(::isNegative, ::isPositive)
                test("item is not in the returned lists") {
                    assertResultIs(listOf(-1), listOf(1, 2, 3))
                }
            }
            context("an item matches more than one predicate") {
                withPredicates(::isBiggerThan10, ::isNegative, ::isZero, ::isPositive, ::isBiggerThan1)
                test("item is assigned to the first matching predicate") {
                    assertResultIs(emptyList(), listOf(-1), listOf(0), listOf(1, 2, 3), emptyList())
                }
            }
            context("repeated predicates") {
                withPredicates(::isNegative, ::isNegative, ::isZero, ::isPositive)
                test("item is assigned to each predicate") {
                    assertResultIs(listOf(-1), listOf(-1), listOf(0), listOf(1, 2, 3))
                }
            }
            context("lambda predicates") {
                withPredicates({ x -> isNegative(x) }, { x -> isPositive(x) })
                test("item is assigned to each predicate") {
                    assertResultIs(listOf(-1), listOf(1, 2, 3))
                }
            }
            context("items in a different order") {
                withItems(3, 2, 1, 0, -1)
                withPredicates(::isNegative, ::isZero, ::isPositive)
                test("input order is preserved") {
                    assertResultIs(listOf(-1), listOf(0), listOf(3, 2, 1))
                }
            }
        }
        context("no items") {
            context("no predicates") {
                test("returns empty list") {
                    assertResultIs()
                }
            }
            context("some predicates") {
                withPredicates(::isNegative, ::isZero, ::isPositive)
                test("returns an empty list for each predicate") {
                    assertResultIs(emptyList(), emptyList(), emptyList())
                }
            }
        }
    }

    private fun TestContextBuilder<Fixture, Fixture>.withItems(vararg items: Int) =
        // before_ allows us to replace an existing fixture, and to have multiple calls
        before_ { copy(items = items.asList()) }

    private fun TestContextBuilder<Fixture, Fixture>.withPredicates(vararg predicates: (Int) -> Boolean) =
        // so that we can combine withs in the same context
        before_ { copy(predicates = predicates.asList()) }
}

Conclusions

I won’t claim that this is a particularly good specification for our partition - I’m just finding my way with this style and find it hard to strike a balance between formality and comprehension. But I do know that had I tried to write a spec before writing any code I would have floundered - this way we TDD’d our way to working code quickly, then produced a more formal and comprehensive set of tests that capture more of the behaviour, with a good degree of rigour between the contexts and the fixtures.

If you liked this, please try the bonus episode where I look at Property Based Testing the same code using Minutest.

[Post updated 2019-02-22]

Test Driven to Specification with Minutest - Part 1

2019-02-16T00:00:00-06:00

This is the first in a mini-series on my new test library, Minutest. I’ll post the other installments over the next few days.

Minutest allows the writing of tests in nested contexts, in a Spec-style. But I find that, unless the problem is very well, erm, specified, then specs are hard to write. In Test Driven Development we’re often exploring the shape of a solution in a way that doesn’t fit well with specs, at least until we’re done.

At the same time the tests that have helped us test drive functionality may not be ideal to communicate to future developers what our code does, or to support safe refactoring of that code.

The ideal situation then would be for our test framework to support us as we TDD our way to some code, and then refactor the tests to a more formal specification. That is the subject of this series.

The Problem

The functionality that I’m going to implement was inspired by a question on the Kotlin Slack. It boils down to partitioning a list into other lists, rather like the standard fun <T> Iterable<T>.partition(predicate: (T) -> Boolean): Pair<List<T>, List<T>> but for more than one predicate and more than two output lists.

First Test

This is TDD, so we had better start with a test. In Minutest it looks like this.

class PartitionTests : JUnit5Minutests {
    fun tests() = rootContext<Unit>() {
        test("check I can run a test") {}
    }
}

We can point to this in IntelliJ and run it - if all is well we are ready to get on with the job proper.

Driving an Interface

Now let’s sketch out the form of the function that we want to implement by creating an example.

fun tests() = rootContext<Unit>() {
    test("explore the interface") {
        val items = listOf(-1, 0, 1, 2, 3)
        val predicates = listOf(
            Context2::isNegative,
            Context2::isZero,
            Context2::isPositive
        )
        val result = items.partition(predicates)
        assertEquals(listOf(listOf(-1), listOf(0), listOf(1, 2, 3)), result)
    }
}

This won't compile until we add our predicates and a stub implementation.

fun isNegative(x: Int) = x < 0

fun isZero(x: Int) = x == 0
fun isPositive(x: Int) = x > 0

fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> = TODO()

Now it compiles but fails with the kotlin.NotImplementedError from our TODO

Driving an Implementation

I’m not clever enough to get this working in one step, so I’m going to step back and try just a degenerate case. We can keep the first test but SKIP it for now, and then add a test that I can get to pass.

fun tests() = rootContext<Unit>() {
    SKIP - test("explore the interface") {
        val items = listOf(-1, 0, 1, 2, 3)
        val predicates = listOf(::isNegative, ::isZero, ::isPositive)
        val result = items.partition(predicates)
        assertEquals(listOf(listOf(-1), listOf(0), listOf(1, 2, 3)), result)
    }
    test("no predicates") {
        val items = listOf(-1, 0, 1, 2, 3)
        val predicates = emptyList<(Int) -> Boolean>()
        val result = items.partition(predicates)
        assertEquals(emptyList<List<Int>>(), result)
    }
}

This has the twin advantages of being a case that we are going to have to test for, and being easy to implement.

fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> = emptyList()

Woo-hoo, a passing test.

OK, now lets add another easy test.

test("predicates but no items") {
    val items = emptyList<Int>()
    val predicates = listOf(::isNegative, ::isZero, ::isPositive)
    val result = items.partition(predicates)
    assertEquals(listOf(emptyList<Int>(), emptyList(), emptyList()), result)
}

This fails because we need to return a list for each predicate - I can do that.

fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> =
    predicates.map { emptyList<T>() }

We now have two passing tests, and one skipped until I’m ready to step up. I suppose that had better be now. Reinstate the failing test and just go for it.

fun tests() = rootContext<Unit>() {
    test("items and predicates") {
        val items = listOf(-1, 0, 1, 2, 3)
        val predicates = listOf(::isNegative, ::isZero, ::isPositive)
        val result = items.partition(predicates)
        assertEquals(listOf(listOf(-1), listOf(0), listOf(1, 2, 3)), result)
    }
    test("no predicates") {
        val items = listOf(-1, 0, 1, 2, 3)
        val predicates = emptyList<(Int) -> Boolean>()
        val result = items.partition(predicates)
        assertEquals(emptyList<List<Int>>(), result)
    }
    test("predicates but no items") {
        val items = emptyList<Int>()
        val predicates = listOf(::isNegative, ::isZero, ::isPositive)
        val result = items.partition(predicates)
        assertEquals(listOf(emptyList<Int>(), emptyList(), emptyList()), result)
    }
}

My approach is to group the items by whether they match the predicate.

Firstly I’ll add a function body buy some time,

fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> {
    return predicates.map { emptyList<T>() }
}

then sketch out the intermediate.

fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> {
    val grouped: Map<(T) -> Boolean, List<T>> = this.groupBy { item ->
        predicates.firstMatch(item)
    }
    return predicates.map { emptyList<T>() }
}

private fun <T> Iterable<(T) -> Boolean>.firstMatch(item: T): (T) -> Boolean = TODO()

Now firstMatch is just find, except that returns a nullable. Hmmm, I suppose that is the case where there isn’t a predicate to match an item. We can relax the type of the keys in our map to get it to compile

fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> {
    val grouped: Map<((T) -> Boolean)?, List<T>> = this.groupBy { item ->
        predicates.firstMatch(item)
    }
    return predicates.map { emptyList<T>() }
}

private fun <T> Iterable<(T) -> Boolean>.firstMatch(item: T): ((T) -> Boolean)? =
    this.find { predicate -> predicate(item) }

There should be one value of grouped for each of our predicates - so that this type-checks

fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> {
    val grouped: Map<((T) -> Boolean)?, List<T>> = this.groupBy { item ->
        predicates.firstMatch(item)
    }
    return grouped.values.toList()
}

Almost There?

Running the tests, items and predicates passes, so we did something right, but the other two fail, so they’ve proved worthwhile too. no predicates reports:

Expected :[]
Actual   :[[-1, 0, 1, 2, 3]]

while predicates but no items says:

Expected :[[], [], []]
Actual   :[]

It’s time to actually think…

…

and come to the conclusion that in both these cases we’re not returning a list for each predicate. Previously we mapped over the predicates to create a list for each in the output.

Let’s do that, fetching the predicate’s own entry from the groups, and noting that the null-safety prevents us from just indexing and instead makes us take the right decision to return an empty list if there are no items matching the predicate.

fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> {
    val grouped: Map<((T) -> Boolean)?, List<T>> = this.groupBy { item ->
        predicates.firstMatch(item)
    }
    return predicates.map { grouped.getOrDefault(it, emptyList()) }
}

Triumph - our tests pass - let’s review.

Working Code

class PartitionTests : JUnit5Minutests {
    fun tests() = rootContext<Unit>() {
        test("items and predicates") {
            val items = listOf(-1, 0, 1, 2, 3)
            val predicates = listOf(::isNegative, ::isZero, ::isPositive)
            val result = items.partition(predicates)
            assertEquals(listOf(listOf(-1), listOf(0), listOf(1, 2, 3)), result)
        }
        test("no predicates") {
            val items = listOf(-1, 0, 1, 2, 3)
            val predicates = emptyList<(Int) -> Boolean>()
            val result = items.partition(predicates)
            assertEquals(emptyList<List<Int>>(), result)
        }
        test("predicates but no items") {
            val items = emptyList<Int>()
            val predicates = listOf(::isNegative, ::isZero, ::isPositive)
            val result = items.partition(predicates)
            assertEquals(listOf(emptyList<Int>(), emptyList(), emptyList()), result)
        }
    }
}

fun <T> Iterable<T>.partition(predicates: List<(T) -> Boolean>): List<List<T>> {
    val grouped: Map<((T) -> Boolean)?, List<T>> = this.groupBy { item ->
        predicates.firstMatch(item)
    }
    return predicates.map { grouped.getOrDefault(it, emptyList()) }
}

private fun <T> Iterable<(T) -> Boolean>.firstMatch(item: T): ((T) -> Boolean)? =
    this.find { predicate -> predicate(item) }

Now if this code was just part of a test, or maybe some throwaway script, I’d probably stop there. But otherwise there are too many loose ends for me to feel comfortable that I know the way that the code would behave in different situations. I like the implementation - it’s declarative and the types feel good, but you’d still have to puzzle out the edge cases; and if that’s the case, then any refactoring might change the behaviour. Which in something like partition could have ripple-effects through any dependent code.

Adding More Tests

Looking through our test names, we can see a lack of precision. If we fix that we can see that we are missing the case of no items and no predicates. This is just the sort of edge case that could happen and return an unexpected value - lets check it.

fun tests() = rootContext<Unit>() {
    test("items and predicates") {
        val items = listOf(-1, 0, 1, 2, 3)
        val predicates = listOf(::isNegative, ::isZero, ::isPositive)
        val result = items.partition(predicates)
        assertEquals(listOf(listOf(-1), listOf(0), listOf(1, 2, 3)), result)
    }
    test("some items no predicates") {
        val items = listOf(-1, 0, 1, 2, 3)
        val predicates = emptyList<(Int) -> Boolean>()
        val result = items.partition(predicates)
        assertEquals(emptyList<List<Int>>(), result)
    }
    test("some predicates no items") {
        val items = emptyList<Int>()
        val predicates = listOf(::isNegative, ::isZero, ::isPositive)
        val result = items.partition(predicates)
        assertEquals(listOf(emptyList<Int>(), emptyList(), emptyList()), result)
    }
    test("no predicates no items") {
        val items = emptyList<Int>()
        val predicates = emptyList<(Int) -> Boolean>()
        val result = items.partition(predicates)
        assertEquals(emptyList<List<Int>>(), result)
    }
}

Thankfully that passes, giving us some confidence that our algorithm is sound.

Now what about items that slip through the predicate net? We can predict that they won’t be returned in any of the lists. Is this the behaviour that we want?

The alternative would be to return these items in another list at the end of the result, so that the result is one list larger than the size of the predicates list. I dislike this because of the asymmetry, and because (I predict but haven’t proved) if we want a catch-all we can simply pass in a last predicate that always returns true.

Finally we already know that predicates that match no item are represented by an empty list in the output - but let’s write a test to prove it so that our future selves don’t have to think it through.

Our TDD Solution

Our final tests for today’s installment are:

class PartitionTests : JUnit5Minutests {
    fun tests() = rootContext<Unit>() {
        test("every item matches a predicate and every predicate matches an item") {
            val items = listOf(-1, 0, 1, 2, 3)
            val predicates = listOf(::isNegative, ::isZero, ::isPositive)
            val result = items.partition(predicates)
            assertEquals(listOf(listOf(-1), listOf(0), listOf(1, 2, 3)), result)
        }
        test("an item matches no predicate") {
            val items = listOf(-1, 0, 1, 2, 3)
            val predicates = listOf(::isZero, ::isPositive)
            val result = items.partition(predicates)
            assertEquals(listOf(listOf(0), listOf(1, 2, 3)), result)
        }
        test("a predicate matches no item") {
            val items = listOf(0, 1, 2, 3)
            val predicates = listOf(::isNegative, ::isZero, ::isPositive)
            val result = items.partition(predicates)
            assertEquals(listOf(listOf(), listOf(0), listOf(1, 2, 3)), result)
        }
        test("some items no predicates") {
            val items = listOf(-1, 0, 1, 2, 3)
            val predicates = emptyList<(Int) -> Boolean>()
            val result = items.partition(predicates)
            assertEquals(emptyList<List<Int>>(), result)
        }
        test("some predicates no items") {
            val items = emptyList<Int>()
            val predicates = listOf(::isNegative, ::isZero, ::isPositive)
            val result = items.partition(predicates)
            assertEquals(listOf(emptyList<Int>(), emptyList(), emptyList()), result)
        }
        test("no predicates no items") {
            val items = emptyList<Int>()
            val predicates = emptyList<(Int) -> Boolean>()
            val result = items.partition(predicates)
            assertEquals(emptyList<List<Int>>(), result)
        }
    }
}

If I didn’t have a Part 2 I think that I’d probably remove some of duplication from this code before checking it in. But it’s much like many of my TDD tests before Minutest - OK, probabably good enough, but lacking some rigour and expressiveness.

In Part 2 we will look at refactoring this code to be more spec-like. With luck it will be more expressive and less duplicated while covering more of the behaviour of our implementation.

Failure is not an Option - Functional Error Handling in Kotlin. Part 5 - Embracing Either

2018-06-04T00:00:00-05:00

This is Part 5 in a series looking at functional error handling in Kotlin. The parts are

In this episode we’ll look at an case where the functional style has some definite benefits over using exceptions, and then go on to try to solve a problem that you will come across that is less easily finessed.

Encapsulating failure

Let’s start this episode with the case where using the functional Either type has some real advantages over exceptions.

Let’s say that I’m processing a potentially huge file, translating every line to a number that I want to sum.

fun sumLines(reader: BufferedReader): Long =
    reader.lineSequence().map(Integer::parseInt).fold(0L, Long::plus)

(Ordinarily we could call .sum() on the sequence, but that returns an Int which may overflow too quickly, so here I accumulate to a Long)

We run this for 5 minutes before discovering that some lines are corrupt, so that parseInt throws, aborting the whole process. Sigh. An expedient approach is just to catch the exception, log in place, and substitute a 0.

fun sumLines(reader: BufferedReader): Long =
    reader.lineSequence()
        .map {
            try {
                Integer.parseInt(it)
            } catch (e: Exception) {
                System.err.println(e.message)
                0
            }
        }
        .fold(0L, Long::plus)

I might be happy with that for a one-off solution, but this hard-codes our error handling and, as is sometimes the case with exceptions, hides the happy path which is the point of the algorithm.

Remembering our functional definition of parseInt

fun parseInt(s: String): Either<Exception, Int> = resultOf { Integer.parseInt(s) }

we could delay our gratification by creating a sequence of Either<Exception, Int>, and then fold over that.

fun sumLines(reader: BufferedReader): Long {
    val ints: Sequence<Either<Exception, Int>> = reader.lineSequence().map(::parseInt)
    return ints.fold(0L) { acc, intResult ->
        intResult.fold(
            { exception ->
                System.err.println(exception.message)
                acc
            },
            { result ->
                acc + result
            }
        )
    }
}

Hmmm, I’m not sure that is much better, especially with those two nested folds that really don’t mean the same thing unless you close your eyes and imagine very hard. The least we can do is to de-nest them

fun sumLines(reader: BufferedReader): Long {
    val ints: Sequence<Int> = reader.lineSequence()
        .map(::parseInt)
        .map { intResult ->
            intResult.fold(
                { exception ->
                    System.err.println(exception.message);
                    0
                },
                { result -> result }
            )
        }
    return ints.fold(0L, Long::plus)
}

We can make things better by writing a little extension to allow us to not use Either.fold when we just want to use the value

inline fun <L, R> Either<L, R>.orElse(whenLeft: (L) -> R): R = when(this) {
    is Left -> whenLeft(this.l)
    is Right -> this.r
}

so now we have

fun sumLines(reader: BufferedReader): Long {
    val ints: Sequence<Int> = reader.lineSequence()
        .map(::parseInt)
        .map {
            it.orElse { exception ->
                System.err.println(exception.message)
                0
            }
        }
    return ints.fold(0L, Long::plus)
}

and we can pull that idea up a level

fun <L, R> Sequence<Either<L, R>>.eachOrElse(whenLeft: (L) -> R): Sequence<R> = this.map {
    it.orElse(whenLeft)
}

leaving us with the the still-a-bit-ugly

fun sumLines(reader: BufferedReader): Long {
    val ints: Sequence<Either<Exception, Int>> = reader.lineSequence().map(::parseInt)
    return ints
        .eachOrElse { exception ->
            System.err.println(exception)
            0
        }
        .fold(0L, Long::plus)
}

Now though we have at least successfully separated what to do in the case of an error from our fundamental flow, allowing us to move the responsibility to the caller

fun sumLines(reader: BufferedReader, onError: (Exception) -> Int): Long =
    reader.lineSequence()
        .map(::parseInt)
        .eachOrElse(onError)
        .fold(0L, Long::plus)

fun sumLines(reader: BufferedReader): Long =
    sumLines(reader) { exception ->
        System.err.println(exception)
        0
    }

This is typical of the functional programming approach - push all the nastiness to the outside of the system, leaving nice pure referentially-transparent core algorithms. Now that we’ve had that inspiration, we could of course do the same thing with exception-based code

fun sumLines(reader: BufferedReader, onError: (Exception) -> Int): Long =
    reader.lineSequence()
        .map {
            try {
                Integer.parseInt(it)
            } catch (e: Exception) {
                onError(e)
            }
        }
        .fold(0L, Long::plus)

Which of these implementations is better is open to debate. Pretty much every programmer will be able to read the try version and work out what it does after a while, and I think that it will be more efficient. When you’re used to the declarative style though the eachOrElse version reveals the intention without having to run your brain as a virtual machine. These days I’d probably go with eachOrElse until I needed the performance.

Before we go on, eagle-eyed readers might spot a source of failure that I had missed. Did you?

I’ll give you a couple of minutes.

What I’d failed to spot is that, if this were Java, calls to any methods on a Reader would declare an IOException, and we’d be forced to consider that reading every line might fail. Here the flow of control is inverted, so that the line sequence pokes strings at our code, but even so, every time it does the file could have been deleted or the WiFi could have gone down. So map can still throw an IOException that we are not revealing by returning Either<Exception, Long>. Our function should be declared as

fun sumLines(reader: BufferedReader, onError: (Exception) -> Int): Either<Exception, Long> = resultOf {
    reader.lineSequence().map(::parseInt).eachOrElse(onError).fold(0L, Long::plus)
}

Frankly this has depressed me a bit. Even when I’m trying to demonstrate how an Either type can help with error handling in Kotlin, I mess it up. I suppose that we’re actually no worse off than Python or C# or almost any other language developers though - we do the best we can, sometimes we make mistakes, and when we find them we do our best to fix them.

Dealing with Control Flow

If you try this functional style of error handling, sooner or later you’ll run into places where, well, things get icky. Let’s interpret three strings as ints and add them, the old-fashioned way, with exceptions

fun addAsInts(s1: String, s2: String, s3: String): Either<Exception, Int> = resultOf {
    Integer.parseInt(s1) + Integer.parseInt(s2) + Integer.parseInt(s3)
}

Now how about with our functional parseInt and map / flatMap?

fun addAsInts(s1: String, s2: String, s3: String): Either<Exception, Int> =
    parseInt(s1).flatMap { i1 ->
        parseInt(s2).flatMap { i2 ->
            parseInt(s3).map { i3 ->
                i1 + i2 + i3
            }
        }
    }

“Really Duncan - that’s better than exceptions?” I hear you cry.

No, I can’t say that it is; not with a straight face anyway. We can make things a bit better if we’re prepared to use our orElse and early returns

fun addAsInts(s1: String, s2: String, s3: String): Either<Exception, Int> = resultOf {
    parseInt(s1).orElse { return Left(it) } +
        parseInt(s2).orElse { return Left(it) } +
        parseInt(s3).orElse { return Left(it) }
}

which in turn can be sweetened a bit with

inline fun <L, R> Either<L, R>.onLeft(abortWith: (Left<L>) -> Nothing): R = when(this) {
    is Left -> abortWith(this)
    is Right -> this.r
}

to give us

fun addAsInts(s1: String, s2: String, s3: String): Either<Exception, Int> = resultOf {
    parseInt(s1).onLeft { return it } +
        parseInt(s2).onLeft { return it } +
        parseInt(s3).onLeft { return it }
}

but frankly my functional programmer friends hate early returns (which also complicate referential transparency) as much as they hate exceptions, so this is frowned upon.

What is required here is what Haskell calls do-notation, which is a way to sequence expressions only evaluating the next if the previous didn’t ‘fail’. This is of course the role of exceptions and/or early returns in other languages.

Functional programmers value referential transparency so much that Scala programmers are resigned to using their for-comprehensions to solve this problem, and Arrow bends Kotlin’s coroutines to the same end. With Arrow I could write

fun parseInt(s: String): Try<Int> = Try {
    Integer.parseInt(s)
}

fun addAsInts(s1: String, s2: String, s3: String): Try<Int>  =
    Try.monad().binding {
        val i1: Int = parseInt(s1).bind()
        val i2: Int = parseInt(s2).bind()
        val i3: Int = parseInt(s3).bind()
        i1 + i2 + i3
    }.ev()

(or at least I could until they changed the API again :-(

Personally I’d rather keep things on a level that I can understand, and maybe even implement myself. So I’d be inclined to define

fun <R> Either<Exception, R>.orThrow(): R = when(this) {
    is Left -> throw this.l
    is Right -> this.r
}

and use it like this

fun addAsInts(s1: String, s2: String, s3: String): Either<Exception, Int> = resultOf {
    parseInt(s1).orThrow() +
        parseInt(s2).orThrow() +
        parseInt(s3).orThrow()
}

Now I’m really not enough of a functional programmer to argue that this is as functionally-pure as do-notation, but from the trenches where I stand I can’t see the difference. Combining exceptions with an Either type in this way at least plays to the strengths of the JVM in exception handling, but you may prefer Arrow, even if examining the bytecode it produces is enough to crash IntelliJ.

I think that’s enough for today. In the next installment I hope to make some recommendations for how an API might document its failure modes in a manner that combines the best of checked exceptions and Either types.

Failure is not an Option - Functional Error Handling in Kotlin. Part 4 - Either v Exception

2018-04-21T00:00:00-05:00

This is Part 4 in a series looking at functional error handling in Kotlin. The parts are

In this fourth episode I’ll compare the pros and cons of using an Either type compared to Exceptions. This is hard-won information - there are plenty of functional programmers who will tell you the advantages of Either - I haven’t yet found one who will tell you the disadvantages.

That said, from here on in you should know that I’m less sure of myself. There may be solutions to problems that I haven’t been exposed to, or better ways of achieving the same ends. Part of my motivation for writing this series is to get things straight in my head and to start a discussion, so please let me know when I’m wrong. The last post started an interesting discussion on Reddit. Discuss this post

So, I may be wrong, but I believe that even in Haskell, our goto source for FP patterns, Either does not replace exceptions. They can still be raised by those conditions that Java would signal with Errors or unchecked Exceptions - out of memory, integer divide by 0, out of bounds array access etc. So, referring back to Part 1, we should consider using Either only for those cases where checked exceptions would be appropriate, and returning null to signal a failure isn’t good enough.

One easy-to-overlook feature of checked exceptions is that they obey laws of type-substitutability. So, as FileNotFoundException extends IOException, given

class FileInputStream {
    public FileInputStream(String name) throws FileNotFoundException {...}
    public int read() throws IOException {...}
}

I can invoke methods and pass on the common exception superclass

public int readFirstByte(String filename) throw IOException {
    return FileInputStream(name).read(); // TODO - close stream
}

We can do the same thing with Either.

fun openFile(name: String): Either<FileNotFoundException, InputStream> = TODO()

fun InputStream.eitherRead(): Either<IOException, Int> = TODO()

fun readFirstByte(filename: String): Either<IOException, Int> =
    openFile(filename).flatMap { it.use { it.eitherRead() } }

So the types work, now we just have to implement the functions.

@Suppress("UNCHECKED_CAST")
fun openFile(name: String): Either<FileNotFoundException, InputStream> =
    resultOf {
        FileInputStream(name)
    } as Either<FileNotFoundException, InputStream>

@Suppress("UNCHECKED_CAST")
fun InputStream.eitherRead(): Either<IOException, Int> =
    resultOf {
        this.read()
    } as Either<IOException, Int>

Hmmm, those casts look mighty suspicious. If, say, this.read() ended up throwing an IndexOutOfBoundsException, then the runtime has no way of knowing that the type of the result is wrong, which will potentially probably lead to really nasty debugging sessions. And while the signature of this.read() is unlikely to change, if we were calling a more volatile API, it could add a type of thrown exception and our code would continue to compile but be subtly broken.

I suppose I could change resultOf to check the type of exception seen is as expected

inline fun <reified L, R> fastidiousResultOf(f: () -> R): Either<L, R> =
    try {
        Right(f())
    } catch (e: Exception) {
        if (e is L)
            Left(e)
        else
            throw e // or throw IllegalStateException(e)?
    }

but that seems asking for trouble unless everyone really internalises its behaviour. It all seems really complicated compared to those nice checked exceptions that everyone complained about, but maybe I was just used to making those work for me.

Even if we dig ourselves deeper into our Exception type hole, Either can’t express another common checked exceptions idiom - a list of exceptions.

public int doSomething(String aParam) throws MyApiException, IOException { ... }

In practice therefore, when using Either the lowest actual subtype of the exception that may be returned does not seem to be specified in function signatures; Exception seems the norm -

fun openFile(name: String): Either<Exception, InputStream> =
    resultOf {
        FileInputStream(name)
    }

which is a shame, because despite all this functional cleverness we seem to be back to the de-facto situation of - here is a function that can either succeed, or fail with pretty much any Throwable.

Actually it isn’t quite that bleak. We’ve divided our error handling into

Errors, which we probably can’t do anything about at any level, and are allowed to propagate as Throwables, and
Exceptions, that are returned in the Either, albeit with their actual type usually lost.

Furthermore, reviewing my current code-base’s use of Either for error handling, I find that we only return Either for functions that we reasonably expect to fail for one of a few reasons

The function is only defined for a subset of its inputs (partial functions like parseInt).
External systems don’t behave as we expect (usually expressed as IOException).
Some other condition not related to our inputs is is not met (eg timeouts).
The function calls another function that we expect to fail.

Our convention is that if a public function can reasonably expect to fail, it should return an Either, usually with Exception as the Left type. In practice then we have the equivalent of a single checked exception type (we don’t necessarily know what can go wrong, but we know that something can) that plays nicely with higher-order functions such as List.map. If calling code needs to do different things with different failures then it is going to have to check the runtime type of Left, and if you want to know what types you need to check for you’re going to have to examine the whole expression tree, but this is what usually happens in practice even with checked exceptions - we handle a few predictable special cases and then see what else occurs in production.

One situation that this convention doesn’t help with is functions that don’t return any result - things like Writer.writeln(s: String). In Java this throws IOException, which is checked, so you are forced to consider the failure. We could return Either<IOException, Unit> in these cases, but as the caller is not processing a return value, it is easy to ignore the error as well. Real functional programmers don’t have to worry about these cases, as functions called just for their side-effects will only be invoked in the context of special error handling (the IO Monad). This hasn’t actually come up in our real-world codebase so far so I don’t know what to advise - maybe throwing the exception is the least worst thing here.

I think it may be time to the recap where we are with the whole exceptions v error results.

Early APIs returned error codes or set global error flags, but it was easy to forget to check these.
Exceptions were introduced to force explicit error handling, but it was still hard to know if a function could fail in practice.
Checked exceptions were introduced to make expected failure conditions more explicit.
Checked exceptions prove hard to reconcile with higher-order functions, go back 2 spaces.
Either types give many of the advantages of unchecked exceptions and knowing that a function can fail in practice, but only for functions where the caller is relying on using the result.

So, you’re now asking, should I be using exceptions or Either in my code base?

Well you won’t be fired for using exceptions. Just claim that you haven’t read these posts, point out that there is not a standard result type in the standard library, and continue business as usual. You can document, log and debug your way out of the problems that will occur in the same way as you always have. If you’d like to raise your game though, then I’d definitely experiment with the functional style. In my last two Kotlin projects we have used a specialised Result variant of Either, similar to the one presented here, as our default strategy, and I’d say that it has been easier to work with than the alternative exception free-for-all.

If you’re a library writer and want to return Either from your functions, the lack of a functional error type in the standard library is going to cause you a problem. My advice would be to define a minimal Result type in your own package and return that - your clients can always write their own extension functions to convert to other representations.

I think that I’m done for this episode. I’m aware that this series is a little disjointed and that the parts are individually too short. It’s a consequence I’m afraid of my working out what I think by writing, and needing to publish to get some feedback to inform future posts (Discuss this post). Maybe I’ll be able to pull it all together into a single coherent article in the future. In the meantime, I think that there are two more posts to come - a review of existing Kotlin result types, and a post on making functional error handling work for you.

Failure is not an Option - Functional Error Handling in Kotlin. Part 3 - Result and Fold

2018-04-18T00:00:00-05:00

This is Part 3 in a series looking at functional error handling in Kotlin. The parts are

In this third episode I’ll look at some of the pros and cons of using an Either type for functional error handling.

If you were ahead of the game in the last installment you’ll have noticed that we mostly caught exceptions and translated them to a Either.Left to signify failure.

fun parseInt(s: String): Either<String, Int> = try {
    Right(Integer.parseInt(s))
} catch (exception: Exception) {
    Left(exception.message ?: "No message")
}

You may further have said to yourself that it seemed a shame to loose all that nice stack-tracey information that Java went out of it’s way to gather and use to create the exception. You may even have thought that we should have said

fun parseInt(s: String): Either<Exception, Int> = try {
    Right(Integer.parseInt(s))
} catch (exception: Exception) {
    Left(exception)
}

If so, I think you’re right. Let’s define a helper function to remove some of the boilerplate and refactor.

inline fun <R> resultOf(f: () -> R): Either<Exception, R> = try {
    Right(f())
} catch (e: Exception) {
    Left(e)
}

fun parseInt(s: String): Either<Exception, Int> = resultOf { Integer.parseInt(s) }

fun BufferedReader.eitherReadLine(): Either<Exception, String> = resultOf {
    this.readLine()
}

fun doubleString(s: String): Either<Exception, Int> = parseInt(s).map { 2 * it }

fun doubleNextLine(reader: BufferedReader): Either<Exception, Int> =
    reader.eitherReadLine().flatMap { doubleString(it) }

Either<Exception, R> is such a common pattern that you may be tempted give it a special name. Arrow calls it Try. There are various other libraries that provide the abstraction as a Result type, including a proposal for the standard library.

There are a couple of big problems though with making a concrete type that locks in Left to be Exception.

The first problem is, should that be Exception, or Throwable? Arrow says Throwable, but I disagree. If we catch Throwable we are catching Errors, which indicate serious problems that a reasonable application should not try to catch. In fact the Scala code on which the Arrow Try is based does not catch Error. The JetBrains Stdlib proposal catches all Throwables, but it is designed to represent the way that an expression terminated, rather than being used for functional error handling.

The second big problem is that functions do not always fail with an exception. Especially if we’re just validating parameters in local functions it seems overkill to create an exception, quite an expensive operation, just to in order to represent a failure. So we’re probably going to want an Either<String, R> for these cases.

A different way of formalising the use of Either as Result is offered by Result4K. This defines a sealed Result type with with Ok or Err subtypes, making the Right and Left cases clear. It’s my favourite of the solutions I’ve seen so far, but then it is from the home team.

For now then, let’s stick with our generic Either type and see what more we can do with it.

If there’s one thing that categorises functional programming with monads (no I don’t, but Either is) it’s delayed gratification. By mapping and flatmapping we don’t actually have to deal with errors at low levels - just pass their occurrence back to our callers. As with exceptions though, in the end someone will have to report failure.

Back in Part 2 we used when to switch on the type of Either

val result: Either<Exception, Int> = parseInt(readLine() ?: "")
when (result) {
    is Right -> println("Your number was ${result.r}")
    is Left -> println("I couldn't read your number because ${result.l}")
}

The formalisation of this on Either is called fold. It takes two functions and returns the result of calling the first if the Either is Left, the second if it is Right.

inline fun <L, R, T> Either<L, R>.fold(fl: (L) -> T, fr: (R) -> T): T =
    when (this) {
        is Right -> fr(this.r)
        is Left -> fl(this.l)
    }

This lets us actually do something on the outside of our system

parseInt(readLine() ?: "").fold(
    fr = { int -> println("Your number was $int") },
    fl = { exception -> println("I couldn't read your number because $exception") }
)

Again you have to squint quite hard to see the similarity between Either.fold and List.fold, but it turns out that functional programmers are so good at squinting that they have given it a special name - catamorphism.

Personally I hate fold. Sure it unwraps the values for you, but putting lambdas inside a function invocation is ugly, and you pretty much always need to name the arguments because you want the success case first, but that doesn’t happen naturally when success is Right. I’ll often fall back on the raw when formulation because it has the braces in the right place and reads naturally. I suppose the nastiness of fold at least encourages the user to only do it as a last resort, which is as it should be.

Before we finish this episode, let’s just review what we have.

sealed class Either<out L, out R>

data class Left<out L>(val l: L) : Either<L, Nothing>()

data class Right<out R>(val r: R) : Either<Nothing, R>()

inline fun <L, R1, R2> Either<L, R1>.map(f: (R1) -> R2): Either<L, R2> =
    when (this) {
        is Right -> Right(f(this.r))
        is Left -> this
    }

inline fun <L, R1, R2> Either<L, R1>.flatMap(f: (R1) -> Either<L, R2>): Either<L, R2> =
    when (this) {
        is Right -> f(this.r)
        is Left -> this
    }

inline fun <L, R, T> Either<L, R>.fold(fl: (L) -> T, fr: (R) -> T): T =
    when (this) {
        is Right -> fr(this.r)
        is Left -> fl(this.l)
    }

inline fun <R> resultOf(f: () -> R): Either<Exception, R> =
    try {
        Right(f())
    } catch (e: Exception) {
        Left(e)
    }

There’s an awful lot of usefulness packed into a few lines of code there - a testament to Kotlin. The nice thing about using data classes and extension methods is that I don’t have to provide everything that you could want to do with Either as a method - you can add your own.

If you’re beginning to experiment with functional error handling, I’d start with the definitions up above. If you find yourself doing something the same way more than twice, look to see if you add an extension method that expresses that operation. For example, I have

inline fun <L, R, T> Either<L, R>.assertSuccess(f: (R) -> T) =
    when (this) {
        is Right -> f(this.r)
        is Left -> error("Value was not a Right, but a Left of ${this.l}")
    }

which I use in test code in one of two ways

val result: Either<Exception, String> = someOperation()

result.assertSuccess {
    assertEquals("banana", it)
}

val rightValue: String = result.assertSuccess { it }

Once you’ve internalised some of the idioms, I’d then introduce a specific Result type, it can be it’s own sealed class or a typealias for Either. Or you can import someone else’s library and experiment with it.

This series isn’t done yet. As you’ll find out if you do go down this path, there are plenty of pain points still to be addressed!

Failure is not an Option - Functional Error Handling in Kotlin. Part 2 - Either

2018-04-18T00:00:00-05:00

This is Part 2 in a series looking at functional error handling in Kotlin. The parts are

In this second episode I’ll look at why functional programming tends to avoid exceptions, and what it uses instead.

A distinguishing feature of functional programming is Referential Transparency. When this applies an expression may be transparently replaced by the result of its evaluation. So if I write

val secondsIn24hours = 60 * 60 * 24

then I can replace 60 * 60 with 3600 or 60 * 24 with 1440 without affecting the results, in fact the compiler may decide to replace the whole expression with 86400 for us. In contrast

val dayLengthInHours = secondsIn(today()) / 60.0 / 60 / 24

is not referentially transparent, because today() will yield a different result than it did yesterday, and any day may have have a leap second applied.

Some say that exceptions break referential transparency. I’m not convinced that it isn’t possible to unify the two, but I would accept that exceptions muddy the referential transparency waters. Why should we care? Because referential transparency makes it a lot easier to reason about the behaviour of a program, which in turn leads to fewer errors and more opportunities to refactor and optimise.

If we want these things, then how should we signal and recover from errors? A time-honoured solution is to return a special value, often null. Let’s say that we want to parse a string into an integer. We could define

fun parseInt(s: String): Int? = TODO()

In Kotlin our clients would know that the result was Int?, and would explicitly be forced to deal with the case where the string did not represent an int, so this would be a fine solution. In Java it would be less fine, as there is no way to make the client consider null. In this case we could (for Java 8+) define

public static Optional<Integer> parseInt(String s)

where Optional<T> has been provided for just this purpose of signalling nullability.

Note that the standard Java function is

public static int parseInt(String s) throws NumberFormatException

The problem here is that NumberFormatException is an unchecked exception, so that clients are not forced deal with it. Was that a good decision? I’d say in retrospect not in this case, but we’ll return to the problem of partial functions later.

If we only care to know that something was amiss, and not the details, then a nullable or optional result is a good solution. But what if we want to convey where our parsing failed - what was the first character that wasn’t a digit? The NumberFormatException thrown by Java is able to carry that information (it doesn’t, but ho-hum). How can we replicate that behaviour in a functional way? In other words, how do we return either the error, or an integer?

The answer, as they say, is in the question. We define a type Either, which can hold one of two types, but only one at a time.

sealed class Either<out L, out R>

data class Left<out L>(val l: L) : Either<L, Nothing>()

data class Right<out R>(val r: R) : Either<Nothing, R>()

For no good reason that I can see, the convention is that Right is used for a result, Left for an error. If we stick to this convention we could define

fun parseInt(s: String): Either<String, Int> = try {
    Right(Integer.parseInt(s))
} catch (exception: Exception) {
    Left(exception.message ?: "No message")
}

How would we use this? As it is a sealed class, when expressions and smart casting work really nicely to let us write things like

val result: Either<String, Int> = parseInt(readLine() ?: "")
when (result) {
    is Right -> println("Your number was ${result.r}")
    is Left -> println("I couldn't read your number because ${result.l}")
}

which admittedly is pathologically not functional, but gives the general idea. By returning an Either we force our clients to deal with the fact that we may have failed - in effect we have reproduced some of checked exceptions in a functional form. To embrace this style you make all your functions return Either and when they in turn invoke something that could fail, pass on any failure or unwrap the success and act on it.

fun doubleString(s: String): Either<String, Int> {
    val result: Either<String, Int> = parseInt(s)
    return when (result) {
        is Right -> Right(2 * result.r)
        is Left -> result
    }
}

Whilst using when to unwrap an Either is cute, it quickly gets old, so we write

inline fun <L, R1, R2> Either<L, R1>.map(f: (R1) -> R2): Either<L, R2> =
    when (this) {
        is Right -> Right(f(this.r))
        is Left -> this
    }

which allows us to write the previous function as

fun doubleString(s: String): Either<String, Int> = parseInt(s).map { 2 * it }

Why is that function called map and not invokeUnlessLeft? Well if you squint you may be able to see that it is kind of the same thing as List.map. Practice that squinting, because we are now going to define

inline fun <L, R1, R2> Either<L, R1>.flatMap(f: (R1) -> Either<L, R2>): Either<L, R2> =
    when (this) {
        is Right -> f(this.r)
        is Left -> this
    }

This unpacks our value and uses it to invoke a function that in turn might fail (as it returns Either). What can we do with that? Well lets say we want to read from a Reader and print double the result.

fun BufferedReader.eitherReadLine(): Either<String, String> =
    try {
        val line = this.readLine()
        if (line == null)
            Left("No more lines")
        else
            Right(line)
    } catch (x: IOException) {
        Left(x.message ?: "No message")
    }


fun doubleNextLine(reader: BufferedReader): Either<String, Int> =
    reader.eitherReadLine().flatMap { doubleString(it) }

This code will return a Left with the failure if eitherReadLine fails, otherwise it will return the result of doubleString, which may itself be either a Left for failure or a Right with the final int result. In this way a chain of map and or flatMap calls acts like a series of expressions which might throw an exception - the first failure aborts the rest of the computation.

Frankly, if you come from an OO background this style does take some getting used to. In my experience no amount of reading helps - you just have to knuckle down and start writing code this way until it becomes less painful. There are other helper functions that we can write to make it less verbose and perhaps easier to read and reason with, but the approach is still not without cost compared to just throwing and managing exceptions.

The next episode will look at more details of functional error handling and how to unify it with exceptions.

Failure is not an Option - Functional Error Handling in Kotlin. Part 1 - Exceptions

2018-04-17T00:00:00-05:00

This is Part 1 in a series looking at functional error handling in Kotlin. The parts are

In this first post I’ll look at why Kotlin did not follow Java’s example and distinguish between checked and unchecked exceptions.

Java has a simple and effective way of signalling and recovering from errors - exceptions. When the language was designed these were in fashion, for good reason - they are a simple and effective way of signalling and recovering from errors.

Where Java went out on a limb was to introduce checked exceptions. Most languages treat exception types equally - code either succeeds and returns or fails and throws. Java distinguished between those conditions that you had to handle (checked exceptions) and those that you didn’t (all other exceptions, errors and throwables). Personally I like the scheme; but many didn’t, and for good reasons.

Checked exceptions lead to a situation where the exceptions from lower level code would routinely be caught in order to rethrow them as another type to conform to the caller’s interface. Most other languages never bothered with this malarkey - unless intervening code was going to release resources, log or retry, exceptions generally propagated directly from the thrower to some top level generic handler that said, in essence, “Something went wrong somewhere.” In practice that was what most Java exception handling boiled down to as well, except that the report could now say that the you had a RuntimeException caused by a MyAPIException caused by a TheirAPIException caused by an ICustomerWcfServiceGetCustomerDetailBusinessFaultDetailFaultFaultMessage caused by an IOException. Sigh. Maybe the detractors had a point.

One thing that isn’t in doubt is that the need to declare checked exceptions in method signatures made Java more verbose. More functional languages than Java have notation to describe a function type, so it is easy to see why the Kotlin language designers balked at (URI, Int) -> Status throws IOException, YourAPIException and just decided to do away with the distinction between checked and unchecked exceptions. In practice when it came to its own functional abstractions - lambdas and streams - Java did the same thing. The functional interfaces invoked by stream methods don’t declare a checked exception; so you have to deal with any such issues inside your lambdas, even if all you do is convert to a runtime exception. This is for a good reason - if a lambda throws a checked exception, then anything that invokes the lambda can throw that exception, but now Stream.map needs to be able to declare the throwing of the lambda’s exception somehow. (I think that this would have been possible in Java, at least for one exception per lambda, but it would lead to more complication than the already rococo generics could bear.)

The de-facto position in Kotlin and (some of) Java 8 is therefore that errors are signified by raising an exception, and that code does not declare what exceptions it may raise. This leaves any code liable to failure in any way at any time, which in practice means that we revert to the situation in other languages with exceptions.

So were checked exceptions just a bad idea? Well no. We’ve lost important information when we discard the concept - the way we expect code might fail, and how defensive we need to be about that. In practice it is usually far more likely that IO will fail than that our process will run out of memory - hence IOException is checked and OutOfMemoryError is not. When a Java method says throws FileNotFoundException we can reason that if the user picked that file from a list it is probably still there and the exception is unlikely; if they typed-in a file name then the chances of it being right are about 10% and we’d better sit in a loop until it is. Either way we’re forced to think about the possibility and consider what to do locally, rather than abdicating responsibility to a generic handler after aborting the whole operation.

In the next part of this series I’ll look at why functional languages tend to avoid exceptions, and what they use instead.

Nothing? can save us

2018-04-02T00:00:00-05:00

Imagine, if you can, that you need to capture data about your Customers

data class Customer(
    val id: String,
    val firstName: String,
    val lastName: String,
    val email: String,
    val addressLine1: String?,
    val addressLine2: String?,
    val addressLine3: String?,
    val postcode: String?,
    val iso2CountryCode: String,
    val phone: String?
)

Due to the power of Kotlin null safety, we can see that we are required to capture a firstName, lastName, email and iso2CountryCode, but that everything else is optional.

We dutifully write an HTML form with validation rules to ensure that we have what we need, post it off to some HTTP handler, and set to to write the code to persist our data class to some datastore -

interface Customers {
    fun save(customer: Customer)
    fun findById(id: String): Customer?
    fun gdpr() = TODO("2018-05-25")
}

If you’re ahead of me here, or just a run of the mill pedant, you’ll be saying, “But Duncan, what about that id?”

Ah yes. Identifiers are usually dished out by the datastore, so we don’t have one at the point that we create a Customer to save. In Java null would be a valid value for the id (as well as every other reference), but if we specify the id as nullable in Kotlin; every time we reference a Customer id we will have to explicitly allow for the null - it’s going to get tedious pretty quickly.

An alternative is to have a separate class to model the unsaved customer.

data class UnsavedCustomer(
    val firstName: String,
    val lastName: String,
    val email: String,
    val addressLine1: String?,
    val addressLine2: String?,
    val addressLine3: String?,
    val postcode: String?,
    val iso2CountryCode: String,
    val phone: String?
)

interface Customers {
    fun save(customer: UnsavedCustomer): Customer
    fun findById(id: String): Customer
}

That’s quite nice, because it shows that act of saving a customer is to turn an UnsavedCustomer into a (Saved)Customer. But the duplication between Customer and UnsavedCustomer is going to be a source of eternal shame.

Nothing? can save us

Let’s make the type of the id generic - constrained to String?

data class CustomerData<out T : String?>(
    val id: T,
    val firstName: String,
    val lastName: String,
    val email: String,
    val addressLine1: String? = null,
    val addressLine2: String? = null,
    val addressLine3: String? = null,
    val postcode: String? = null,
    val iso2CountryCode: String,
    val phone: String? = null
)

So our id must be compatible with nullable String.

A saved Customer has an id, so we can tighten String? to String in this case.

typealias Customer = CustomerData<String>

val existingCustomer = Customer("123", "Fred", "Flintstone", "fred@bedrock.org", iso2CountryCode = "US")

Now a customer’s id must be non-null, which makes our life simple

val thisIsFine: String = existingCustomer.id

val doesntCompile = Customer(null, "Fred", "Flintstone", "fred@bedrock.org", iso2CountryCode = "US")

We want our unsaved customers have a null id. The type of null is Nothing?, which is helpfully substitutable for String?

typealias UnsavedCustomer = CustomerData<Nothing?>

val newCustomer = UnsavedCustomer(null, "Fred", "Flintstone", "fred@bedrock.org", iso2CountryCode = "US")

You can’t use a String where you expect a Nothing? though, so null is the only acceptable value of an UnsavedCustomer id

val typeMismatch = UnsavedCustomer("123", "Fred", "Flintstone", "fred@bedrock.org", iso2CountryCode = "US")
    // Required: Nothing? Found: String

val typeMismatch2: String = newCustomer.id
    // Required: String Found: Nothing?

Because of this you can’t use an UnsavedCustomer where you expected a saved one, or vice versa

val typeMismatch3: Customer = newCustomer
val typeMismatch4: UnsavedCustomer = existingCustomer

although if you want to treat them alike you can define AnyCustomer

typealias AnyCustomer = CustomerData<*>

val ok: AnyCustomer = newCustomer
val okToo: AnyCustomer = existingCustomer

val AnyCustomer.isSaved get() = id != null

To convert an Unsaved to a saved customer, supply the missing id

@Suppress("UNCHECKED_CAST")
fun UnsavedCustomer.saved(id: String) = (this as Customer).copy(id)

Full-disclosure - I thought this trick up a while ago, but we’ve only just put it into some production code, so I don’t know how it will turn out in practice. I’ll report back if there turn out to be any wrinkles that I haven’t spotted.

KotlinConf

2017-11-04T00:00:00-05:00

My talk on the Cost of Kotlin Language Features at KotlinConf went pretty well I think.

If you missed it, and don’t want to wait for it on Netflix, the slides (with speaker notes) are available on Google Slides.

The thing that impressed me most about the conference? I knew that JetBrains was a company of very clever people; I discovered that they are a company of very nice people. This was I believe their first big event, and they went about it with heart.

Kudos JetBrains - you deserve success for Kotlin.

The Cost of Kotlin Language Features - Preliminary Results Part 5 - Properties

2017-10-28T00:00:00-05:00

This is Part 5 in a series examining The Cost of Kotlin Language Features in preparation for my presentation at KotlinConf in November. The series consists of

I’m publishing these results ahead of KotlinConf to give an opportunity for peer-review, so please do give me your feedback about the content, experimental method, code and conclusions. If you’re reading this before November 2017 it isn’t too late to save me from making a fool of myself in person, rather than just on the Internet. As ever the current state of the code to run the benchmarks is available for inspection and comment on GitHub.

This post looks at the cost of Kotlin properties. Since my last post I’ve set up a Raspberry Pi booting to Bash to run the benchmarks - the result is a lot less noisy measurements and so better fidelity. John Nolan continues to be the statistical brains behind the Hamkrest matchers used to show the relationships between results.

For the first time these benchmarks use different State objects between Java and Kotlin. In Java we have

@State(Scope.Benchmark)
public class JavaState {

    public String field = "hello";

    public String getField() {
        return field;
    }

    public String getConstant() {
        return "hello";
    }
}

so that we can benchmark field access, getter access to that field, and calling a method that returns a constant.

@Benchmark
public String field_access(JavaState state) {
   return state.field;
}

@Benchmark
public String getter(JavaState state) {
   return state.getField();
}

@Benchmark
public String method_access(JavaState state) {
   return state.getConstant();
}

Kotlin doesn’t allow direct access to a field, so the closest State is

@State(Scope.Benchmark)
open class KotlinState {

    val fieldProperty = "hello"

    val methodProperty get() = "hello"

    fun getConstant() = "hello"
}

which we benchmark thus

@Benchmark
fun field_property(state: KotlinState): String {
    return state.fieldProperty
}

@Benchmark
fun method_property(state: KotlinState): String {
    return state.methodProperty
}

@Benchmark
fun constant_method(state: KotlinState): String {
    return state.getConstant()
}

Here are the results for a typical test batch

It looks like everything is the same speed except for field_property. Let’s ask the stats on all the test results that I’ve collected so far.

assertThat(JavaProperties::field_access, ! probablyDifferentTo(JavaProperties::getter))

This first assertion says that we can’t see any statistical difference between accessing the Java field directly, and calling the getter. That’s a surprise to me, as the former is a field access and the latter a method call and the same field access. I suppose that HotSpot has worked its magic (you may recall that we run these benchmarks many times and only start measuring after HotSpot has had time to do so) inlining the getter so that the running code is effectively the same.

The second assertion

assertThat(JavaProperties::method_access,
    probablyFasterThan(JavaProperties::field_access, byMoreThan = 0.0002, butNotMoreThan = 0.0005))

shows that calling a method that returns a constant is faster than accessing a field directly. Again that’s a little surprising, but the size of the effect is very small, between 0.02 - 0.05%.

Accessing a Kotlin method property (where we define a get() operation for the property) is unsurprisingly indistinguishable from calling a method, as that is what it is.

assertThat(KotlinProperties::method_property, ! probablyDifferentTo(KotlinProperties::constant_method))
assertThat(KotlinProperties::method_property, ! probablyDifferentTo(JavaProperties::method_access))

That just leaves the outlier - the standard Kotlin property with a backing field. This turns out to be statistically significantly faster than all the other access methods by between 1.5 and 2 %.

assertThat(KotlinProperties::field_property, probablyFasterThan(JavaProperties::field_access,      byMoreThan = 0.015, butNotMoreThan = 0.02))
assertThat(KotlinProperties::field_property, probablyFasterThan(JavaProperties::getter,            byMoreThan = 0.015, butNotMoreThan = 0.02))
assertThat(KotlinProperties::field_property, probablyFasterThan(JavaProperties::method_access,     byMoreThan = 0.015, butNotMoreThan = 0.02))
assertThat(KotlinProperties::field_property, probablyFasterThan(KotlinProperties::method_property, byMoreThan = 0.015, butNotMoreThan = 0.02))
assertThat(KotlinProperties::field_property, probablyFasterThan(KotlinProperties::constant_method, byMoreThan = 0.015, butNotMoreThan = 0.02))

Now I really don’t understand this result. I’ve seen it visually in all the Raspberry Pi test runs and those assertions pass on the amalgam of over 5000 benchmark samples, so it isn’t a statistical aberration, but I can’t see how it can be.

Looking at the bytecode, this:

public final field_property(LcostOfKotlin/properties/KotlinState;)Ljava/lang/String;
@Lorg/openjdk/jmh/annotations/Benchmark;()
@Lorg/jetbrains/annotations/NotNull;() // invisible
  @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
 L0
  ALOAD 1
  LDC "state"
  INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
 L1
  LINENUMBER 9 L1
  ALOAD 1
  INVOKEVIRTUAL costOfKotlin/properties/KotlinState.getFieldProperty ()Ljava/lang/String;
  ARETURN

calling this

public final getFieldProperty()Ljava/lang/String;
@Lorg/jetbrains/annotations/NotNull;() // invisible
 L0
  LINENUMBER 9 L0
  ALOAD 0
  GETFIELD costOfKotlin/properties/KotlinState.fieldProperty : Ljava/lang/String;
  ARETURN

appears to be reliably executing faster than

public field_access(LcostOfKotlin/properties/JavaState;)Ljava/lang/String;
@Lorg/openjdk/jmh/annotations/Benchmark;()
 L0
  LINENUMBER 9 L0
  ALOAD 1
  GETFIELD costOfKotlin/properties/JavaState.field : Ljava/lang/String;
  ARETURN

I can believe that HotSpot magic and speculative execution could boost the Kotlin property to the same speed as direct field access, but I don’t see how it can be reliably faster.

Addendum

Luckily the same John Nolan wondered if the final modifiers could be significant. That led me to looking at the field definition, which in Kotlin is

private final Ljava/lang/String; fieldProperty = "hello"

That final probably allows HotSpot to more aggressively inline the value of the variable, safe in the knowledge that no-one can (should) change it. So it’s time to fire up the Raspi and run some more benchmarks to test this hypothesis.

Add-addendum

Well this is even more puzzling! I’ve added final field access Java benchmarks

@Benchmark
public String final_field_access(JavaState state) {
    return state.finalField;
}

@Benchmark
public String getter_of_final(JavaState state) {
    return state.getFinalField();
}

and a mutable Kotlin property benchmark

@Benchmark
fun mutable_property(state: KotlinState): String {
    return state.mutableProperty
}

Against expectations, the Kotlin mutable property seems no slower than the immutable property, and the Java final field access is significantly slower than non-final field access.

We get a clue why the Java final field is slower when we examine the bytecode.

public final_field_access(LcostOfKotlin/properties/JavaState;)Ljava/lang/String;
  @Lorg/openjdk/jmh/annotations/Benchmark;()
   L0
    LINENUMBER 19 L0
    ALOAD 1
    INVOKEVIRTUAL java/lang/Object.getClass ()Ljava/lang/Class;
    POP
    LDC "hello"
    ARETURN

The value has been inlined, but only after a nugatory(?) call to get the class of the object that it came from. I really don’t understand this at all, but if you do, you could answer this Stack Overflow question.

Add-add-addendum

Well it turns out that the inlining of non-static final primitive and string fields is required by the Java Language Specification, and that getClass call is a cheap null check to make sure that it can’t resolve if the referenced object is null. Given the confusion, and that, for me at least, this is the slower option, I’m not sure that was a good idea.

And I still don’t know why the Kotlin property access is significantly faster than all other options. Let’s hope someone at KotlinConf can put me out of my misery.

The Cost of Kotlin Language Features - Preliminary Results Part 4 - Nullable Primitives

2017-10-15T00:00:00-05:00

This is Part 4 in a series examining The Cost of Kotlin Language Features in preparation for my presentation at KotlinConf in November. The series consists of

This post looks at the cost of nullability. My aim with this batch wasn’t to measure individual aspects of nullability so much as to simply model typical usage, so I picked adding to a nullable Int as an example.

As usual we start with a baseline, consuming a constant expression passed in to the benchmark

open class KotlinPrimitives {

    @Benchmark
    fun _1_baseline(state: IntState, blackhole: Blackhole) {
        blackhole.consume(state._41)
    }
}

And now a take that and add one to it

    @Benchmark
    fun _2_sum(state: IntState, blackhole: Blackhole) {
        blackhole.consume(state._41 + 1)
    }

Irritatingly, for the data that I have, that addition isn’t statistically detectable.

    @Test
    fun `kotlin sum is not detectable`() {
        assertThat(this::_1_baseline, ! probablyDifferentTo(this::_2_sum))
    }

Looking at the bytecode we see that the problem is that the code that we want to measure is swamped by code that we don’t want to measure, but has to be there to make the measurements possible.

  public final _2_sum(LcostOfKotlin/primitives/IntState;Lorg/openjdk/jmh/infra/Blackhole;)V
  @Lorg/openjdk/jmh/annotations/Benchmark;()
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 1
   L0
    ALOAD 1
    LDC "state"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
    ALOAD 2
    LDC "blackhole"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
   L1
    LINENUMBER 22 L1
    ALOAD 2
    ALOAD 1
    INVOKEVIRTUAL costOfKotlin/primitives/IntState.get_41 ()I
    ICONST_1
    IADD
    INVOKEVIRTUAL org/openjdk/jmh/infra/Blackhole.consume (I)V
   L2
    LINENUMBER 23 L2
    RETURN

That ICONST_1 IADD hidden in the middle is what we want to measure! Ah well. I could do better by not using the Blackhole, and by arranging for field access to the IntState._41, and switching off null-checks (ideally we’d have a way to do that for just this method with an annotation), but, well, I have a more detectable fish to fry.

    @Benchmark
    fun _3_sum_nullable(state: IntState, blackhole: Blackhole) {
        blackhole.consume(state.nullable_41!! + 1)
    }
    
    @Test
    fun `sum nullable is slower`() {
        assertThat(this::_2_sum, probablyFasterThan(this::_3_sum_nullable))
        assertThat(this::_2_sum, ! probablyFasterThan(this::_3_sum_nullable, byAFactorOf = 0.001))
    }

Adding a not-null !! assertion is statistically slower (when testing a value that is never null), but by less than 0.1%. In the bytecode we see the cost of the nullability

    DUP
    IFNONNULL L2
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.throwNpe ()V
   L2
    INVOKEVIRTUAL java/lang/Integer.intValue ()I

is a check for Integer != null and fetching the Int out of its box.

What if we don’t know that Int can’t be null? We can use the Elvis operator to substitute 0 for null.

    @Benchmark
    fun _4_sum_always_null(state: IntState, blackhole: Blackhole) {
        blackhole.consume(state.nullInt ?: 0 + 1)
    }

    @Test
    fun `sum always null is slower`() {
        assertThat(this::_2_sum, probablyFasterThan(this::_4_sum_always_null))
        assertThat(this::_2_sum, ! probablyFasterThan(this::_4_sum_always_null, byAFactorOf = 0.001))
    }

Uh huh - again, slower, but by less than 0.1% when our value is always null. Here’s the bytecode

    DUP
    IFNULL L2
    INVOKEVIRTUAL java/lang/Integer.intValue ()I
    GOTO L3
   L2
    POP
    ICONST_1
   L3
    INVOKEVIRTUAL org/openjdk/jmh/infra/Blackhole.consume (I)V

Argghh!

Can you see the problem?

It took me a while.

We aren’t doing the addition at all any more. The Elvis operator has a lower precedence than addition. So the expression we’re measuring turns out to be

    state.nullInt ?: (0 + 1)

and the compiler is smart enough to know that 0 + 1 = 1 for all values of 0 and 1, so it just substitutes ICONST_1.

Sigh.

I’ll have to fix that an re-run the benchmarks (which takes several hours), but in the meantime, luckily, I already have some more results.

For a long time I couldn’t detect any statistically significant different between the versions with and without null checks. I formed a hypothesis that, when the value being checked was always null or not null, branch prediction in either HotSpot or the processor was eating the cost of the check. So I measured with Int?s that were randomly null or not

    @Benchmark
    fun _5_sum_50_50_nullable(state: IntState, blackhole: Blackhole) {
        blackhole.consume(state.`50 50 NullableInt` ?: 0 + 1)
    }

    @Benchmark
    fun _6_sum_90_10_nullable(state: IntState, blackhole: Blackhole) {
        blackhole.consume(state.`90 10 NullableInt` ?: 0 + 1)
    }

It turns out that branch prediction isn’t detectable

    @Test
    fun `branch_prediction is undetectable_50_50`() {
        assertThat(this::_4_sum_always_null, ! probablyDifferentTo(this::_5_sum_50_50_nullable))
    }

    @Test
    fun `branch_prediction is undetectable_90_10`() {
        assertThat(this::_4_sum_always_null, ! probablyDifferentTo(this::_6_sum_90_10_nullable))
    }

but I suppose that Speculative Execution may play a part in the throughput of null checks.

The Cost of Kotlin Language Features - Preliminary Results Part 3 - Invocation

2017-10-14T00:00:00-05:00

This is Part 3 in a series examining The Cost of Kotlin Language Features in preparation for my presentation at KotlinConf in November. The series consists of

In this post I look at the basic building block of functional programming - the first-class function. This allows us to treat a unit of computation as just another reference, to be passed around and then invoked when needed. I haven’t (yet?) compared the Kotlin performance to the equivalent Java - here I just compare different Kotlin code.

The function we’re going to invoke is simple

fun aFunction(i: Int) = 2 * i

and our baseline is invoking it directly.

open class Invoking {

    @Benchmark
    fun baseline(intState: IntState) : Int {
        return aFunction(intState.randomInt)
    }
}

[In this case, I’m not consuming the result in a JMH blackhole as previously, but instead returning it to the framework, which promises to make sure that the result is referenced so that the benchmark computation has to be performed. Also note that, nervous of other processor optimisations, I use a random variable rather than a fixed value for each benchmark execution when dealing with primitives.]

The simplest thing we can do is to create a function reference and invoke that, rather than invoking the function directly.

    @Benchmark
    fun invoke_via_reference(intState: IntState) : Int {
        return (::aFunction)(intState.randomInt)
    }

This turns out to be between 1% and 2% slower than the baseline.

    @Test
    fun `invoke via reference is slower than plain invocation`() {
        assertThat(this::baseline, probablyFasterThan(this::invoke_via_reference, byAFactorOf = 0.01))
        assertThat(this::baseline, ! probablyFasterThan(this::invoke_via_reference, byAFactorOf = 0.02))
    }

This result is surprising in two ways.

As far as I can see, the compiler could compile invoke_via_reference to the same code as the baseline.
Given that it hasn’t; < 2% is really not much slower.

So let’s fall back to the bytecode

public final baseline(LcostOfKotlin/primitives/IntState;)I
  @Lorg/openjdk/jmh/annotations/Benchmark;()
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
   L0
    ALOAD 1
    LDC "intState"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
   L1
    LINENUMBER 16 L1
    ALOAD 1
    INVOKEVIRTUAL costOfKotlin/primitives/IntState.getRandomInt ()I
    INVOKESTATIC costOfKotlin/invoking/InvokingKt.aFunction (I)I
    IRETURN
    
public final invoke_via_reference(LcostOfKotlin/primitives/IntState;)I
  @Lorg/openjdk/jmh/annotations/Benchmark;()
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
   L0
    ALOAD 1
    LDC "intState"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
   L1
    LINENUMBER 22 L1
    GETSTATIC costOfKotlin/invoking/Invoking$invoke_via_reference$1.INSTANCE : LcostOfKotlin/invoking/Invoking$invoke_via_reference$1;
    CHECKCAST kotlin/jvm/functions/Function1
    ALOAD 1
    INVOKEVIRTUAL costOfKotlin/primitives/IntState.getRandomInt ()I
    INVOKESTATIC java/lang/Integer.valueOf (I)Ljava/lang/Integer;
    INVOKEINTERFACE kotlin/jvm/functions/Function1.invoke (Ljava/lang/Object;)Ljava/lang/Object;
    CHECKCAST java/lang/Number
    INVOKEVIRTUAL java/lang/Number.intValue ()I
    IRETURN

Instead of just INVOKESTATIC costOfKotlin/invoking/InvokingKt.aFunction, invoke_via_reference has additional calls to get the instance of the function as a function type, check that it is the right type, box the parameter because there are no primitive specialisations of the function types, invoke the function through its interface with the boxed parameter, and finally unbox the result. All this costs just ~1.5% throughput.

[Writing this I realise that if the benchmark tax (calls to checkParamaterIsNotNull and IntState.getRandomInt) is high, it reduces the apparent cost of the invocations we actually want to measure. In this case I don’t believe that they are - in particular note that the random int is calculated outside the benchmark not on demand.]

Anyhoo; the point of first class functions is to be passed around. Let’s simulate that by passing them to something whose job it is to invoke them.

inline fun <T> applier(t: T, f: (T) -> T) = f(t)

Now we can benchmark lambdas and function references.

    @Benchmark
    fun passed_as_lambda(intState: IntState) : Int {
        return applier(intState.randomInt) { aFunction(it ) }
    }

    @Benchmark
    fun passed_as_function_reference(intState: IntState) : Int {
        return applier(intState.randomInt, ::aFunction)
    }

For the data that I’ve gathered, these are statistically indistinguishable from the baseline!

Again, there are two reasons for that exclamation mark. Let’s look at the bytecode.

public final passed_as_lambda(LcostOfKotlin/primitives/IntState;)I
  @Lorg/openjdk/jmh/annotations/Benchmark;()
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
   L0
    ALOAD 1
    LDC "intState"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
   L1
    LINENUMBER 27 L1
    ALOAD 1
    INVOKEVIRTUAL costOfKotlin/primitives/IntState.getRandomInt ()I
    ISTORE 2
   L2
    LINENUMBER 66 L2
    ILOAD 2
    ISTORE 3
   L3
    LINENUMBER 27 L3
    ILOAD 3
    INVOKESTATIC costOfKotlin/invoking/InvokingKt.aFunction (I)I
   L4
   L5
    NOP
   L6
    IRETURN

[passed_as_function_reference is identical mutatis mutandis]

Pretentious! Moi? Anyhoo; because of the wonders of inline functions, we haven’t had to pay the price of boxing and unboxing our int, nor of invoking a function type through its interface. If you ignore the line numbers and loading and storing - this is the same code as the baseline. Which is remarkable.

And yet it isn’t the same code as the baseline, because of the loading and storing, which are setting things up on the JVM stack to be used in the function. In the baseline the result of INVOKEVIRTUAL of getRandomInt puts the argument to INVOKESTATIC aFunction straight onto the operand stack. In this code the result of getRandomInt is pulled from the stack into a local variable 2, which is pushed onto the stack in order to be stored in local variable 3, which is pushed onto the stack in order to be used by aFunction. This faffing is a bit surprising, as is the fact that I can’t measure its cost in terms of benchmark speed.

I think that the local variable hokey-cokey is required because we expect to be able to set a breakpoint in the applier and see the values of its parameters, although that may only explain one of the in-outs. My inability to detect any time taken in the dance may be because these are very fast instructions to execute, or it may be that HotSpot detects that they are not necessary and optimises them away. Your virtual machine mileage may vary.

My brain hurts now, so just one more benchmark before lunch.

val aBlock: (Int) -> Int = { 2 * it }

    @Benchmark
    fun passed_as_value_of_function_type(intState: IntState) : Int {
        return applier(intState.randomInt, aBlock)
    }

Here we invoke via a variable of function type, rather than directly with a lambda or function reference. In this case the speed is slower than the baseline and the other applier applications by 1-2%. Why?

  public final passed_as_value_of_function_type(LcostOfKotlin/primitives/IntState;)I
  @Lorg/openjdk/jmh/annotations/Benchmark;()
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
   L0
    ALOAD 1
    LDC "intState"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
   L1
    LINENUMBER 37 L1
    ALOAD 1
    INVOKEVIRTUAL costOfKotlin/primitives/IntState.getRandomInt ()I
    INVOKESTATIC java/lang/Integer.valueOf (I)Ljava/lang/Integer;
    ASTORE 2
    INVOKESTATIC costOfKotlin/invoking/InvokingKt.getABlock ()Lkotlin/jvm/functions/Function1;
    ASTORE 3
   L2
    LINENUMBER 68 L2
    ALOAD 3
    ALOAD 2
    INVOKEINTERFACE kotlin/jvm/functions/Function1.invoke (Ljava/lang/Object;)Ljava/lang/Object;
   L3
    CHECKCAST java/lang/Number
    INVOKEVIRTUAL java/lang/Number.intValue ()I
    IRETURN

Here we pay the price of getting the value of aBlock, and, because it is a function type and we are using a primitive, boxing and unboxing in order to call it, via its interface. This is this interesting because it shows that, for inline functions, the cost of the function depends not just on its definition, but also the code into which it is inlined.

The Cost of Kotlin Language Features - Preliminary Results Part 2 - Strings

2017-10-08T00:00:00-05:00

This is Part 2 in a series examining The Cost of Kotlin Language Features in preparation for my presentation at KotlinConf in November. The series consists of

In this post I look at String interpolation, the language feature that allows you to embed Kotlin expressions inside strings.

Let’s build the string Hello World from its greeting and subject, in Java first.

public class JavaStrings {

    @Benchmark
    public void baseline(StringState state, Blackhole blackhole) {
        blackhole.consume(state.getHello());
        blackhole.consume(state.getWorld());
    }

    @Benchmark
    public void concat(StringState state, Blackhole blackhole) {
        blackhole.consume(state.getHello() + " " + state.getWorld());
    }

}

Here we just concatenate two strings passed into the benchmark, separated by a space. See Part 1 for an explanation of the state and blackhole parameters.

In Kotlin we could write

open class KotlinStrings {

    @Benchmark
    fun baseline(state: StringState, blackhole: Blackhole) {
        blackhole.consume(state.hello)
        blackhole.consume(state.world)
    }

    @Benchmark
    fun concat(state: StringState, blackhole: Blackhole) {
        blackhole.consume("${state.hello} ${state.world}")
    }
}

Graphically the throughput for a particular test run looks like this

A visual inspection suggests that the Kotlin version may be as quick as the Java one, and indeed, for the data I have, I cannot tell them apart.

    @Test
    fun `cannot detect the difference between Java and Kotlin`() {
        assertThat(JavaStrings::concat, ! probablyDifferentTo(KotlinStrings::concat))
    }

In other words, “We cannot say that (the mean throughput of the Java version is different than the mean throughput of the Kotlin version for a 95% confidence)”.

Whilst this is true for the data that I have, I suspect that it is not true to say that there is no cost to Kotlin’s string interpolation. Examining the bytecode for the Java (v 1.8.0_131) version we see.

  public concat(LcostOfKotlin/strings/StringState;Lorg/openjdk/jmh/infra/Blackhole;)V
  @Lorg/openjdk/jmh/annotations/Benchmark;()
   L0
    LINENUMBER 18 L0
    ALOAD 2
    NEW java/lang/StringBuilder
    DUP
    INVOKESPECIAL java/lang/StringBuilder.<init> ()V
    ALOAD 1
    INVOKEVIRTUAL costOfKotlin/strings/StringState.getHello ()Ljava/lang/String;
    INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringBuilder;
    LDC " "
    INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringBuilder;
    ALOAD 1
    INVOKEVIRTUAL costOfKotlin/strings/StringState.getWorld ()Ljava/lang/String;
    INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringBuilder;
    INVOKEVIRTUAL java/lang/StringBuilder.toString ()Ljava/lang/String;
    INVOKEVIRTUAL org/openjdk/jmh/infra/Blackhole.consume (Ljava/lang/Object;)V
   L1
    LINENUMBER 19 L1
    RETURN

while the Kotlin (v 1.1.4) has

  public final concat(LcostOfKotlin/strings/StringState;Lorg/openjdk/jmh/infra/Blackhole;)V
  @Lorg/openjdk/jmh/annotations/Benchmark;()
     [null checks, see Part 1]
   L1
    LINENUMBER 21 L1
    ALOAD 2
    NEW java/lang/StringBuilder
    DUP
    INVOKESPECIAL java/lang/StringBuilder.<init> ()V
    LDC ""
    INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringBuilder;
    ALOAD 1
    INVOKEVIRTUAL costOfKotlin/strings/StringState.getHello ()Ljava/lang/String;
    INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringBuilder;
    BIPUSH 32
    INVOKEVIRTUAL java/lang/StringBuilder.append (C)Ljava/lang/StringBuilder;
    ALOAD 1
    INVOKEVIRTUAL costOfKotlin/strings/StringState.getWorld ()Ljava/lang/String;
    INVOKEVIRTUAL java/lang/StringBuilder.append (Ljava/lang/String;)Ljava/lang/StringBuilder;
    INVOKEVIRTUAL java/lang/StringBuilder.toString ()Ljava/lang/String;
    INVOKEVIRTUAL org/openjdk/jmh/infra/Blackhole.consume (Ljava/lang/Object;)V
   L2
    LINENUMBER 22 L2
    RETURN

Both create a StringBuilder and then append the parts of the string, but the Kotlin compiler has appended a vestigial empty string representing the parts of the template string before the first substitution. As it is hard to believe that appending an empty string to a StringBuffer is free, the Kotlin code must be slower, just not, in this case, detectably so.

As a aside, both the Kotlin and the Java compilers miss the trick of initialising the StringBuilder with the first part of the concatenation - both cases should really generate code equivalent to

new StringBuilder(state.getHello()).append(state.getWorld())

Apart from the un-needed empty string, I found no places where Kotlin string interpolation had any cost over vanilla Java. Indeed where strings are known constant at compile time, they are composed then.

    fun `the compiler optimizes this to a constant`() = "${"hello"} ${"world"}"

    fun `and even this`() = "${"${"hello" + " " + "world"}"}"

both compile to

    LDC "hello world"
    ARETURN

although

private val hello = "hello"
private val world = "world"

fun `this is not a constant expression`() = "$hello $world"

The Cost of Kotlin Language Features - Preliminary Results Part 1 - Baselines

2017-10-07T00:00:00-05:00

Following on from my last post Lessons Learned Writing Java and Kotlin Microbenchmarks, this post presents some preliminary results examining The Cost of Kotlin Language Features in preparation for my presentation at KotlinConf in November.

The code behind the investigation is all available on GitHub. There you will find a framework that uses JMH to run benchmarks, and then runs JUnit tests on the results to check assertions about their relative speed. The actual benchmarks and tests that I’m running are also there, along with benchmark results and the sets of those results that are used in the tests. The separation allows me to reject results that are suspect for any of the reasons discussed in my previous post.

Note that, since that post, John Nolan has helped greatly with the statistics of comparing benchmarks and contributed the statistical comparators that the framework uses.

With this in mind, let’s dive straight in and look at the baseline tests.

First Java

public class JavaBaseline {
    @Benchmark
    public void baseline(StringState state, Blackhole blackhole) {
        blackhole.consume(state);
    }
}

and then Kotlin

open class KotlinBaseline {
    @Benchmark
    fun baseline(state: StringState, blackhole: Blackhole) {
        blackhole.consume(state)
    }

    @Test
    fun `java is quicker but not by much`() {
        assertThat(JavaBaseline::baseline, probablyFasterThan(this::baseline, byAFactorOf = 0.005))
        assertThat(JavaBaseline::baseline, ! probablyFasterThan(this::baseline, byAFactorOf = 0.01))
    }
}

With JMH benchmarks it is important that data used in the benchmark is passed in from the outside world (via State objects), and consumed by the benchmark framework (using the Blackhole)- these limit optimisations that would allow some or all of the benchmark code to be skipped altogether. As pretty much any JMH test requires the use of these devices, I’ve included them in the baseline benchmarks.

In the @Test we see assertions about the relative performance of the @Benchmark methods. The test shows that the Java code is quicker by between 0.5 % and 1%, with the default Confidence Interval of 95%.

Here is a sample benchmark run shown graphically (the error bars are the mean error with 99.9% confidence).

Given that the Kotlin and Java look functionally identical, you might expect that we shouldn’t be able to detect any difference in the runtime of the code. There is a subtle difference though, revealed when we examine the bytecode emitted by the Java compiler.

public baseline(LcostOfKotlin/strings/StringState;Lorg/openjdk/jmh/infra/Blackhole;)V
  @Lorg/openjdk/jmh/annotations/Benchmark;()
   L0
    LINENUMBER 12 L0
    ALOAD 2
    ALOAD 1
    INVOKEVIRTUAL org/openjdk/jmh/infra/Blackhole.consume (Ljava/lang/Object;)V
   L1
    LINENUMBER 13 L1
    RETURN

compared to that from the Kotlin compiler

  public final baseline(LcostOfKotlin/strings/StringState;Lorg/openjdk/jmh/infra/Blackhole;)V
  @Lorg/openjdk/jmh/annotations/Benchmark;()
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 1
   L0
    ALOAD 1
    LDC "state"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
    ALOAD 2
    LDC "blackhole"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
   L1
    LINENUMBER 14 L1
    ALOAD 2
    ALOAD 1
    INVOKEVIRTUAL org/openjdk/jmh/infra/Blackhole.consume (Ljava/lang/Object;)V
   L2
    LINENUMBER 15 L2
    RETURN

The Kotlin compiler always checks that arguments passed to non-nullable parameters are, in fact, not null (at least for public functions). I suppose that in a pure Kotlin world these could be dispensed with, and there is a compiler flag to disable them, but the reasoning goes that any old Java code could be invoking your method with nulls, so it’s better to find this out as early as possible.

This price is paid for invocation of every public Kotlin method, so it’s as well that the check is relatively cheap. In this case you have to run the benchmarks for many iterations before you can see a statistically significant effect.

In the rest of this series I’ll rarely compare Java results directly to Kotlin, but when I do, it’s worth remembering that at least some of any slowdown detected will be due to parameter null checks rather than the rest of the benchmark method.

I hope to have the next installment of this series, covering string interpolation, published in the next few days.

Edit - here are the parts as I write them

Lessons Learned Writing Java and Kotlin Microbenchmarks

2017-09-16T00:00:00-05:00

I’m presenting a session, The Cost of Kotlin Language Features, at KotlinConf in November. In the best scientific tradition I proposed the session because I was interested in the topic, knew little about it, and wanted to be forced to learn. That’ll teach me!

Coincidentally, the day that my proposal was accepted, Christophe Beyls published a series of 3 articles on Medium titled Exploring Kotlin’s hidden costs. Well that was pleasing - Jetbrains were paying for me to go to San Francisco and Christophe was doing all the work! I filed the links for later ~~plagarism~~ research and got on with booking flights.

Later that month Renato Athaydes went even further. Christophe had inferred costs by examining the generated bytecode, but Renato pointed out that it is notoriously difficult to predict Java’s performance from examining the code because of the HotSpot JIT compiler. So he wrote some JMH benchmarks to measure the actual performance and published the results in Kotlin’s hidden costs - Benchmarks. As my conference proposal had promised both bytecode analysis and benchmarks, this concluded my work and I got on with my day job.

Except that there was a nagging doubt remaining - both from my own reading of his results and other peoples comments. Oh dear, I am actually going to have to understand this stuff if I’m to present at a conference.

So I set to writing some benchmarks of my own to understand how they work. (At the time of writing these are work in progress, so I’d welcome feedback, but please be gentle.) I quickly learned how hard it is to force a computer to do any calculation that you then ignore, and not to effectively inline any value that you don’t change. A good guide to working through these issues is JMH Java Microbenchmark Harness.

Even once I had introduced @State, used a Blackhole, and run enough warmup iterations that HotSpot would do its thing, I still got some perplexing results. To understand why they were odd we need some more background.

How JMH runs benchmarks

JMH works (by default) by running benchmark methods in a tight loop until 1 second has elapsed, counting the number of loops in that time. These 1 s iterations are repeated a configurable number of times, proceeded by a configurable number of 1 s warmup iterations whose times are reported, but ignored. All iterations for a single benchmark are performed in a separate VM than that orchestrating the measurements, with new VMs created to an outer batch.

The default settings are 10 (serial) JVM forks, in each of which 20 warmup iterations are performed followed by 20 measurements, for a total of 200 1 s iterations per benchmark.

You can select multiple benchmarks for a run by regex, these will be run one after the other by the same orchestrator. So if there are 3 benchmarks there will be 30 forked VMs created, and 600 total measurement iterations.

Here is an example of JMH output at the start of a run

# JMH version: 1.19
# VM version: JDK 1.8.0_131, VM 25.131-b11
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 20 iterations, 1 s each
# Measurement: 20 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: com.oneeyedmen.kostings.baselines.JavaBaseline.baseline

# Run progress: 0.00% complete, ETA 00:13:20
# Fork: 1 of 10
# Warmup Iteration   1: 293010554.087 ops/s
# Warmup Iteration   2: 296380175.612 ops/s
...
# Warmup Iteration  20: 367022161.575 ops/s
Iteration   1: 369132107.601 ops/s
Iteration   2: 368540396.014 ops/s
...
Iteration  20: 355492595.855 ops/s

# Run progress: 5.00% complete, ETA 00:12:56
# Fork: 2 of 10
# Warmup Iteration   1: 267261211.186 ops/s
...

How JMH reports benchmarks

Once all the measurement iterations in all the forks for a benchmark are complete, JMH reports the average throughput for that benchmark, together with the 99.9% confidence interval, assuming a Gaussian distribution. So that run finished with this report

# Run complete. Total time: 00:13:31

Benchmark                 Mode  Cnt          Score         Error  Units
JavaBaseline.baseline    thrpt  200  359400874.797 ± 3190513.351  ops/s
KotlinBaseline.baseline  thrpt  200  363343435.012 ± 5184390.204  ops/s

There’s a lot of spurious precision going on there, making it hard to see the wood for the trees. Luckily JMH can also output to a CSV file,

"Benchmark","Mode","Threads","Samples","Score","Score Error (99.9%)","Unit"
"com.oneeyedmen.kostings.baselines.JavaBaseline.baseline","thrpt",1,200,359400874.797,3190513.351,"ops/s"
"com.oneeyedmen.kostings.baselines.KotlinBaseline.baseline","thrpt",1,200,363343435.012,5184390.204,"ops/s"

and we can use that to generate graphs that humans can read. Here we show the 99.9% confidence interval with error bars.

Interpreting relative results

Here is the code that was being benchmarked.

public class JavaBaseline {
    @Benchmark public void baseline(StringState state, Blackhole blackhole) {
        blackhole.consume(state);
    }
}

open class KotlinBaseline {
    @Benchmark fun baseline(state: StringState, blackhole: Blackhole) {
        blackhole.consume(state)
    }
}

Looking at the histogram, this leaves us with a problem - the Kotlin code seems to be quicker, despite doing the same thing. The results say that there is a 99.9% chance that the Java is not faster than 362,591,388 ops/s, and a 99.9% chance that the Kotlin as not slower than 358,159,044 ops/s. This leaves an overlap between their confidence levels that allows Java to be faster, but with not much confidence (is someone able to do the maths?).

Is it possible that Java is likely slower than Kotlin for this benchmark, as the results suggest?

Let’s look at the bytecode. First Java

  // access flags 0x1
  public baseline(Lcom/oneeyedmen/kostings/strings/StringState;Lorg/openjdk/jmh/infra/Blackhole;)V
  @Lorg/openjdk/jmh/annotations/Benchmark;()
   L0
    LINENUMBER 12 L0
    ALOAD 2
    ALOAD 1
    INVOKEVIRTUAL org/openjdk/jmh/infra/Blackhole.consume (Ljava/lang/Object;)V
   L1
    LINENUMBER 13 L1
    RETURN
   L2
    LOCALVARIABLE this Lcom/oneeyedmen/kostings/baselines/JavaBaseline; L0 L2 0
    LOCALVARIABLE state Lcom/oneeyedmen/kostings/strings/StringState; L0 L2 1
    LOCALVARIABLE blackhole Lorg/openjdk/jmh/infra/Blackhole; L0 L2 2
    MAXSTACK = 2
    MAXLOCALS = 3

and then Kotlin

  // access flags 0x11
  public final baseline(Lcom/oneeyedmen/kostings/strings/StringState;Lorg/openjdk/jmh/infra/Blackhole;)V
  @Lorg/openjdk/jmh/annotations/Benchmark;()
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 1
   L0
    ALOAD 1
    LDC "state"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
    ALOAD 2
    LDC "blackhole"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkParameterIsNotNull (Ljava/lang/Object;Ljava/lang/String;)V
   L1
    LINENUMBER 15 L1
    ALOAD 2
    ALOAD 1
    INVOKEVIRTUAL org/openjdk/jmh/infra/Blackhole.consume (Ljava/lang/Object;)V
   L2
    LINENUMBER 16 L2
    RETURN
   L3
    LOCALVARIABLE this Lcom/oneeyedmen/kostings/baselines/KotlinBaseline; L0 L3 0
    LOCALVARIABLE state Lcom/oneeyedmen/kostings/strings/StringState; L0 L3 1
    LOCALVARIABLE blackhole Lorg/openjdk/jmh/infra/Blackhole; L0 L3 2
    MAXSTACK = 2
    MAXLOCALS = 3

So Kotlin is checking the parameters aren’t null, and running the exact same code as the Java. Unless HotSpot is able to deduce that some later code doesn’t have to be run because Kotlin has made this check, I can’t see any way that the Java can be slower, but our benchmarks show that it is, probably.

Running a number of different benchmarks, with different fork counts, and different numbers of iterations, produced many of these perverse results. The numbers can’t lie, but are we asking them the right question?

Systematic Errors

The confidence intervals given by JMH are based only on the measurements that it has taken - they aren’t a reflection of the actual truth. In order to get an idea of what causes the measurement variation that we interpret as errors, we have to drill further into the data.

In addition to the CSV output, JMH has a JSON output.

{
    "lots-of-stuff-deleted": "..."
    "warmupIterations" : 20,
    "warmupTime" : "1 s",
    "warmupBatchSize" : 1,
    "measurementIterations" : 20,
    "measurementTime" : "1 s",
    "measurementBatchSize" : 1,
    "primaryMetric" : {
        "score" : 3.656186120562128E8,
        "scoreError" : 1833316.6776747273,
        "scoreConfidence" : [
            3.637852953785381E8,
            3.674519287338875E8
        ],
        "scorePercentiles" : {
            "0.0" : 3.168042199792653E8,
            "50.0" : 3.654796412185938E8,
            "90.0" : 3.7526561404113406E8,
            "95.0" : 3.781350703702088E8,
            "99.0" : 3.821109699100871E8,
            "99.9" : 3.8282245766638625E8,
            "99.99" : 3.8282245766638625E8,
            "99.999" : 3.8282245766638625E8,
            "99.9999" : 3.8282245766638625E8,
            "100.0" : 3.8282245766638625E8
        },
        "scoreUnit" : "ops/s",
        "rawData" : [
            [
                3.599636727894533E8,
                3.670631984977747E8,
                3.678476209011342E8,
                3.750361228656685E8

Here we have access to each measurement. I don’t have the data for the run we’ve just seen, but here are the histograms for another set of benchmarks.

There is a lot of uncertainty in that KotlinStrings.baseline, I wonder what the measurements look like?

Whoa! I don’t know what went on at the end of that run, but it wasn’t normal. JavaStrings.Concat seems to have taken a wobble around iteration 60 as well. Clearly these results should not be trusted.

Another run gives the following results.

Note the much smaller error bars reflecting the lack of variation in the KotlinStrings.baseline measurements.

Causes of Systematic Errors

What causes dropouts like these? I suppose anything that causes CPU or memory bandwidth to be switched away from the benchmark VM, which is many of the little conveniences that we expect computers to do for us without being asked. To get stable benchmarks on my MacBook Pro I have had to:

Disable Time Machine, which otherwise backs up after a period of user inactivity.
Disable CrashPlan, ditto.
Close IntelliJ, so that it doesn’t refresh it’s indices after noticing the results files being written.
Make sure Finder isn’t viewing the results folder.
Disable various agents (Dropbox, Google Earth) which check for updates periodically.
Disable the screensaver.
Disable switching off the screen after inactivity.
Disable sleep after inactivity.
Not use the computer while the benchmarks are running.

Even then I must have missed something, because looking at the results I still get occasional dropouts, so I now make it a rule to view the graph of throughput for each benchmark batch, and to re-run those which show obviously non-random variation.

Further Investigation

Plotting a histogram of the distribution of measurements might give insight into whether they are random or systematic errors. No-doubt some statistics could be brought to bear to automatically reject results that were not sufficiently Gaussian.

I suspect that Intel Turbo-Boost, where a processor ramps up its clock in response to demand, but then slows down again as it heats up, may be another cause of systematic errors.

Conclusion

Despite the best efforts of JMH to allow the writing of reliable micro-benchmarks, the modern computing environment is complex and it is hard to remove all systematic errors. It is not enough to look at the summarised results - you have to examine the distribution of benchmark measurements to have confidence in the numbers.

Postscript - Speeding up benchmarks

Needless to say, benchmarking can take time. The default settings take at least 10 * (20 + 20) = 400 s per benchmark. A faster machine doesn’t help much, because each measurement takes 1 s - you just get better fidelity in that measurement. You can speed up a run by scrimping on the warmups - looking at the warmup iteration results for my benchmarks they don’t get noticeably faster after the 3rd iteration, so spending 20 s warming up is just wasting good loops, as this graph, of throughput against iteration with no warmup, shows.

Ideally it would be nice to give JMH a target confidence and have it continue to run a benchmark until that level was reached.

I have also come to the conclusion that the default of 10 forks per benchmark needlessly slows the test run. As long as each benchmark is run in its own VM, it is significantly quicker to run 200 iterations in that than 20 in each of 10. There may be some situations in which the forking is important, but for the micro-benchmarks I am running I have not yet detected any unexpected behaviour using a single fork.

Chapter 1 - Spike

2017-06-16T00:00:00-05:00

{aside} This is a draft of Chapter 1 of a book, tentatively titled “Programming Episodes”. I’m publishing it here for feedback - essentially I’d like to know whether to carry on writing or get a real job this summer. If you make it to the end, I’d be really grateful if you’d let me know what you think. {/aside}

Chapter 1 - Spike

I’m writing this in Lisbon airport, an hour before my flight back to London is due to leave. On this holiday I’ve decided I’m pretty committed to writing this book, and now I’m wondering what technologies I should use for its production.

What criteria should I use to make the decision? Well the working title of the book is “Programming Episodes - learning modern development by example” so that might help. Defining modern development is something that I hope will become clear by the end, but as I think it through, I realise that deciding how to write the book is fundamentally an engineering decision. Engineering is about trade-offs, choice, optimisation. It’s about making the best use of your time or your client’s money. And very often in programming that comes down to trading the cost to make something now against the cost of fixing it later.

I know that this book will take a long time to write, and I also know that it will be full of code samples that we’d both like to actually work by the time you come to read them. So I’m looking for a way that I can mix code and prose in a way that is easy to write and maintain.

And that’s why these words are being written into the IntelliJ IDE rather than Word. They are plain text now, although I’m already thinking of them as Markdown - I just haven’t needed to emphasise anything yet. I suspect that this prose will end up as comments in Kotlin source files, and that some sort of build system will put the book together for me. In fact, if you’re reading this on paper you can’t see it yet, but this file is already surrounded by a Gradle build and is in the context of a Git repository. This is the way that modern development projects in Kotlin will start, and so it feels rather fitting that the book does too.

The First Kotlin File

Risk reduction is high on the list of modern programming concerns. I want to know that the idea of embedding the content of this book in its own source code is viable. So I’m going to start with a spike - a prototype that goes deep into the heart of the problem but doesn’t attempt breadth. This way we hope to find major problems that might scupper an idea with the least effort.

I’m typing this text as comments into a Kotlin source file called 2-spike.kt. Kotlin uses the same /* block comment markers */ as Java and C. So far I’ve found that the markers don’t get in the way of my text, but that I also don’t get a nice Markdown preview of the formatting. That’s alright though, as the nature of Markdown is that I can *see* the formatting even though it’s only characters.

So text is OK. What about code?

import org.junit.Test
import kotlin.test.assertEquals

class SpikeTest {

    @Test fun `2 plus 2 is 4`() {
        assertEquals(4, add(2, 2))
    }
}

fun add(a: Int, b: Int) = a + b

{aside} That’s a JUnit 4 test written in Kotlin. I’m going assume that you can read and understand JUnit tests, and at least get the gist of Kotlin when it is written as a direct translation of Java. {/aside}

Well I can run that directly in the IDE and from the Gradle build tool, so I guess that’s a start. Thinking ahead it occurs to me that I’m going to want to be able to write comments in the code. These could get confused with the comments that are the book text , so I’d better start using a different marker for the latter. Taking a lead from Javadoc, which extends standard block comments, but doesn’t change their parsing by the compiler, I’m going to go with

⁠/*-
This is book text which the compiler will ignore.
⁠-*/

/* This is a standard comment block
 ⁠*/
fun the_compiler_will_process_this() {

}

for now and see how it goes.

It occurs to me that this is the opposite of the normal Markdown approach, where we can embed code inside blocks, with the text as the top level. Here the code is the top level, and the Markdown is embedded in it. The advantage is that we have to do nothing to our text to compile it, the disadvantage is that something is going to have to process the files to do something with the text before we can render the Markdown. That seems reasonable to me though, as I write code for a living, and an awful lot of that code does just this sort of thing to text. In fact, now that I have a programming task to do, I can see whether or not I can describe that task in a way that makes sense to both of us.

First Attempt at Publishing

Let’s write some code to take a mixed prose and source file and write a Markdown version. I’m not entirely sure what that output should look like yet, but I’ll know it when I see it. This is the sweet spot for Approval Tests, which will allow us to make rapid progress but at the same time know when something that used to work doesn’t any more.

When you run an Approval test it stores the output from the test, and compares it to a approved version of that output. If they differ, or if there was no approved version, the test fails. You could write this logic yourself, indeed I have, so I’ll use the OkeyDoke Approval Tests library, which integrates with JUnit through a Rule.

OK, time to write some code.

class CodeExtractorTests {

    @Rule @JvmField val approver = approvalsRule()

    @Test fun writes_a_markdown_file_from_Kotlin_file() {
        val source = """
        |/*-
        |Title
        |=====
        |This is Markdown paragraph
        |-*/
        |
        |/* This is a code comment
        |*/
        |fun aFunction() {
        |   return 42
        |}
        |/*-
        |More book text.
        |-*/
        """.trimMargin()
        approver.assertApproved(translate(source))
    }
}

fun translate(source: String) = source

Here I’ve written an example file content as a Kotlin here document, and then, for now, an identity translate function just to get us running. Running the test creates a file CodeExtractorTests.writes_a_markdown_file_from_Kotlin_file.actual with the contents

/*-
Title
=====
This is Markdown paragraph
-*/

/* This is a code comment
*/
fun aFunction() {
   return 42
}
/*-
More book text.
-*/

which is what we assertApproved. The test fails, as there was no approved content to compare with the actual. We can make it pass by approving the content with

cp 'CodeExtractorTests.writes_a_markdown_file_from_Kotlin_file.actual' 'CodeExtractorTests.writes_a_markdown_file_from_Kotlin_file.approved'

and running it again.

Of course we have just approved something that we know to be incorrect, but we’re taking baby steps here. Now we need to improve the translate function. I was about to start by stripping out the lines beginning with /*- and -*/, but if we do that first we’ll loose information about where the code starts. In fact thinking it through I realise that this page has code that we don’t want to view (the package and import statements at the top), and I’m sure that in general there will be other code that is required to compile but doesn’t contribute to the narrative. Maybe we need to explicitly mark code to be included.

@Test fun writes_a_markdown_file_from_Kotlin_file() {
    val source = """
    |package should.not.be.shown
    |/*-
    |Title
    |=====
    |This is Markdown paragraph
    |-*/
    |import should.not.be.shown
    |//`
    |/* This is a code comment
    |*/
    |fun aFunction() {
    |   return 42
    |}
    |//`
    |/*-
    |More book text.
    |-*/
    """.trimMargin()
    approver.assertApproved(translate(source))
}

Here I’ve used a line comment with a backtick //`` to mark the beginning and end of the code we want to see.

Now we can first implement the code to strip out the block comments that hide our prose from the Kotlin compiler

fun translate(source: String) = source.split("\n")
    .filterNot { it.startsWith("/*-") || it.startsWith("-*/") }
    .joinToString("\n")

The first run of the test fails as the actual file is different from the approved - inspecting it I can see that the differences are indeed the stripped out block comment markers - the file is now

package should.not.be.shown
Title
=====
This is Markdown paragraph
import should.not.be.shown
//`
/* This is a code comment
*/
fun aFunction() {
   return 42
}
//`
More book text.

Let’s get to work putting the code between our special markers into a Markdown code block.

fun translate(source: String): String {
    var inCodeBlock = false
    var inTextBlock = false
    return source.split("\n")
        .map {
            when {
                !inCodeBlock && it.startsWith("//`") -> {
                    inCodeBlock = true
                    "```kotlin"
                }
                inCodeBlock && it.startsWith("//`") -> {
                    inCodeBlock = false
                    "```"
                }
                !inTextBlock && it.startsWith("/*-") -> {
                    inTextBlock = true
                    ""
                }
                inTextBlock && it.startsWith("-*/") -> {
                    inTextBlock = false
                    ""
                }
                inTextBlock -> it
                inCodeBlock -> it
                else -> ""
            }
        }
        .joinToString("\n")
}

Now I won’t pretend that was easy to write, or that I’m proud of it, but it does work, yielding



Title
=====
This is Markdown paragraph


```kotlin
/* This is a code comment
*/
fun aFunction() {
   return 42
}
```

More book text.

Note that I’ve chosen to leave blank lines where markers and ignored text are for now, as they make making sense of the output easier.

Now of course, I have to try the code on the file that I’m typing into right now, as that is the real point.

fun main(args: Array<String>) {
    val markdown = translate(File("src/test/java/com/oneeyedmen/book/Chapter_01_Spike/01_c_Spike-First-Attempt-at-Publishing.kt").readText())
    File("build/delme").apply {
        mkdirs()
        resolve("out.md").writeText(markdown)
    }
}

It doesn’t quite work as I expected - it doesn’t find publish code markers ('’’) when they are indented with spaces. I suppose we should add that case to our test suite.

@Test fun writes_a_markdown_file_from_Kotlin_file() {
    val source = """
    |package should.not.be.shown
    |/*-
    |Title
    |=====
    |This is Markdown paragraph
    |-*/
    |object HiddenContext {
    |  //`
    |  /* This is a code comment
    |  */
    |  fun aFunction() {
    |     return 42
    |  }
    |  //`
    |}
    |/*-
    |More book text.
    |-*/
    """.trimMargin()
    approver.assertApproved(translate(source))
}

and implement quickly and dirtyly to see if it’s good.

fun translate(source: String): String {
    var inCodeBlock = false
    var inTextBlock = false
    return source.split("\n")
        .map {
            when {
                !inCodeBlock && it.trim().startsWith("//`") -> {
                    inCodeBlock = true
                    "```kotlin"
                }
                inCodeBlock && it.trim().startsWith("//`") -> {
                    inCodeBlock = false
                    "```"
                }
                !inTextBlock && it.trim().startsWith("/*-") -> {
                    inTextBlock = true
                    ""
                }
                inTextBlock && it.trim().startsWith("-*/") -> {
                    inTextBlock = false
                    ""
                }
                inTextBlock -> it
                inCodeBlock -> it
                else -> ""
            }
        }
        .joinToString("\n")
}

This works, and, after fixing some places in this file that I had messed up the formatting, it works here too.

It feels like there is something general trying to get out of that inBlock.. code, but we’ll come back to it when I’m less tired. I’ll just make a small change to make it look less bad.

fun translate(source: String): String {
    var inCodeBlock = false
    var inTextBlock = false
    return source.split("\n")
        .map { line ->
            when {
                !inCodeBlock && line.firstNonSpaceCharsAre("//`") -> {
                    inCodeBlock = true
                    "```kotlin"
                }
                inCodeBlock && line.firstNonSpaceCharsAre("//`") -> {
                    inCodeBlock = false
                    "```"
                }
                !inTextBlock && line.firstNonSpaceCharsAre("/*-") -> {
                    inTextBlock = true
                    ""
                }
                inTextBlock && line.firstNonSpaceCharsAre("-*/") -> {
                    inTextBlock = false
                    ""
                }
                inTextBlock -> line
                inCodeBlock -> line
                else -> ""
            }
        }
        .joinToString("\n")
}

fun String.firstNonSpaceCharsAre(s: String) = this.trimStart().startsWith(s)

Combining Files

We’re almost done in this first stint. The next thing I need to do to increase my confidence that this is working is to combine all the sections of this chapter into a whole, so that I and others can read it.

This text is being written into 01_d_Spike-Combining-Files.kt, in a directory (actually Java / Kotlin package) com.oneeyedmen.book.Chapter_01_Spike. Listing all the source in the directory we see

01_a_Spike-Introduction.kt
01_b_Spike-First-Kotlin-File.kt
01_c_Spike-First-Attempt-at-Publishing.kt
01_d_Spike-Combining-Files.kt

In the time it’s taken to write the contents, these filenames have changed several times, with the aim to allow me to see what is in them and at the same time process them automatically. Let’s see if I succeeded in that second goal by processing all the files in sequence.

fun main(args: Array<String>) {
    val dir = File("src/main/java/com/oneeyedmen/book/Chapter_01_Spike")
    val translatedLines: Sequence<String> = sourceFilesIn(dir)
        .flatMap(this::translate)

    val outDir = File("build/book").apply {
        mkdirs()
    }

    outDir.resolve(dir.name + ".md").bufferedWriter(Charsets.UTF_8).use { writer ->
        translatedLines.forEach {
            writer.appendln(it)
        }
    }
}

fun sourceFilesIn(dir: File) = dir
    .listFiles { file -> file.isSourceFile() }
    .toList()
    .sortedBy(File::getName)
    .asSequence()

private fun File.isSourceFile() = isFile && !isHidden && name.endsWith(".kt")

fun translate(source: File): Sequence<String> = translate(source.readText(Charsets.UTF_8))

fun translate(sourceLines: String): Sequence<String> {
    var inCodeBlock = false
    var inTextBlock = false
    return sourceLines.splitToSequence("\n")
        .map { line ->
            when {
                !inCodeBlock && line.firstNonSpaceCharsAre("//`") -> {
                    inCodeBlock = true
                    "```kotlin"
                }
                inCodeBlock && line.firstNonSpaceCharsAre("//`") -> {
                    inCodeBlock = false
                    "```"
                }
                !inTextBlock && line.firstNonSpaceCharsAre("/*-") -> {
                    inTextBlock = true
                    null
                }
                inTextBlock && line.firstNonSpaceCharsAre("-*/") -> {
                    inTextBlock = false
                    null
                }
                inTextBlock -> line
                inCodeBlock -> line
                else -> null
            }
        }
        .filterNotNull()
}

fun String.firstNonSpaceCharsAre(s: String) = this.trimStart().startsWith(s)

Looking back at the last version of the translate function you’ll see that I have changed returning a Sequence - this will allow me to avoid having all the text of all files in memory. I’ve also put nulls into that Sequence where we are skipping a line, and then filtered out nulls from the Sequence with filterNotNull, so that we don’t have a blank line for each source line we aren’t outputting.

The rest of the code is pretty standard Kotlin - not very pretty or abstracted yet but good enough. It would probably read very much the same if I wrote it in Python - which I would if I didn’t have Kotlin here as it would be just too verbose in Java.

I had to up update the test as well as the code to account for the change to the signature of translate. The test is now

class CodeExtractorTests {

    @Rule @JvmField val approver = approvalsRule()

    @Test fun writes_a_markdown_file_from_Kotlin_file() {
        val source = """
        |package should.not.be.shown
        |/*-
        |Title
        |=====
        |This is Markdown paragraph
        |-*/
        |object HiddenContext {
        |  //`
        |  /* This is a code comment
        |  */
        |  fun aFunction() {
        |     return 42
        |  }
        |  //`
        |}
        |/*-
        |More book text.
        |-*/
        """.trimMargin()
        approver.assertApproved(translate(source).joinToString("\n"))
    }
}

and the approved file shows no blank lines where we the skip a line in the source

Title
=====
This is Markdown paragraph
```kotlin
  /* This is a code comment
  */
  fun aFunction() {
     return 42
  }
```
More book text.

Running the code and looking at the rendered Markdown with IntelliJ’s Markdown plugin I see one glaring problem. Where one file ends and another starts we need to separate them with a blank line if the combined Markdown isn’t to be interpreted as a contiguous paragraph. Let’s fix that by adding a blank line to the end of each file’s lines.

val translatedLines: Sequence<String> = sourceFilesIn(dir)
    .flatMap { translate(it).plus("\n") }

There is one other issue that I can see, which has to do with trying to show in the text how I mark the prose regions. Where I wrote

I'm going to go with

⁠/*-
this is book text
⁠-*/

the markers were interpreted as markers and messed up the output. I add a pipe character | to the beginning of those marker lines to get things running. I don’t have a solution to this at the moment, bar the pipe, but suspect that we’ll need some way of escaping our own codes. I’m trusting that as I gain fluency with Markdown something clever will come up. If you can see the markers without |s above I guess I succeeded in the end.

Conclusions

What can we learn from this episode?

Well I learned that this model works for me when actually explaining concepts. Unless you’re viewing the source version of this text, you won’t see that there is a subtlety around the different versions of the code existing in the same source file. When explaining code on my blog I would have to move forward and back between different source versions, cutting and pasting them into the code. Here, I can keep the different versions in the same file, which is a big productivity win.

My learning that is a demonstration of the big topic - risk reduction. Before I started work on this book I identified the following main risks:

No-one will want to read what I write on this subject
It will be too much effort to maintain a code-heavy text

These are typical of the 2 major risks facing a new software project - failure in the market and failure to be able to deliver economically (if at all). Unless we are certain that we can build something that our customers will want we had better address both risks early on. The way to address them is by gathering information as quickly and as cheaply as possible.

If you are reading this text in anything other than the finished book, then you’ll see that I will have addressed the market risk by releasing the product before it was finished in order to gauge if there is any enthusiasm. The chapter itself has been me addressing the technical risk that it’s too hard to write about code by

Hypothesising that I could write the book text within the source code rather than vice versa, and that that would be an efficient process, if I had a tool to extract it.
Experimenting to see if the hypothesis is true by trying it out.

I’ve done just enough here to show that the book-in-comments approach has legs. No doubt I’ll spend days on tooling to produce the book before I’m finished - the code is rough and the inability to write about my own notation without invoking it is a shame - but right now I have confidence without too much cost.

Tooling is another take-away. Modern developers will go to great lengths to increase their own productivity by building tools to leverage their time. Sometimes these are simple scripts, sometimes full applications. I’m hoping to leverage the power of an IDE that I already know with some text processing to produce something greater than the sum of its parts.

Leverage is my final thought. Can we find ways of solving several problems at once? In this case I’ve managed to test the writing model, written some tooling to support it, and perhaps demonstrated some aspects of modern programming, all at the same time. We should obviously strive for humility, but I’m secretly a bit pleased with myself.

Request for Feedback

Well you’ve either got this far, or skipped here having decided that the content wasn’t from you. Could you spare a couple of minutes to help me gauge whether I am wasting my time?

If so …

… please create an email to me, copy the following into an email, start answering at the top, keep going until you don’t think you owe me any more or your precious time, and send. I’m sorry that I don’t have an embedded form or anything - yet. Maybe if there is enough encouragement the processing of the form will become a chapter!

Should I continue writing this book? [y | n]

Are you in the book’s target market (3 - 10+ years of programming experience)? [y | n]

What do you think of the recursive nature of the material, writing about the development of the software to assist the writing? [0 (disaster) - 10 (triumph)]

How likely are you to recommend the book to a friend or colleague? [0 (not at all) - 10 (extremely)]

Would you pay to read the completed book? [y | n]

Do you care about paper copies of books? [y | n]

Is there anything else you’d like to say? [ … ]

If you do like what I’ve written, then there is more available on Leanpub where you can read the other chapters for free. And please invite a friend.

Thank you

Duncan

Objects and Functions in Kotlin - Part 2

2017-05-17T00:00:00-05:00

In Part 1 we started looking at the issue of encapsulating configuration information in object and functional styles. In this second part I’ll continue the exploration before looking at expressing differences in behaviour.

Let’s first refactor the two previous solutions to look even more similar by adding a static createEmailSender function to invoke the OO constructor.

fun send(
    email: Email,
    serverAddress: InetAddress,
    username: String,
    password: String) {
    TODO()
}

object OOContext {

    interface ISendEmail {
        fun send(email: Email): Unit
    }

    class EmailSender(
        val serverAddress: InetAddress,
        val username: String,
        val password: String
    ) : ISendEmail {
        override fun send(email: Email) =
            send(email, serverAddress, username, password)
    }

    fun createEmailSender(
        serverAddress: InetAddress,
        username: String,
        password: String): ISendEmail = EmailSender(serverAddress, username, password)
}

object FPContext {
    fun createEmailSender(
        serverAddress: InetAddress,
        username: String,
        password: String): (Email) -> Unit =
        { message ->
            send(message, serverAddress, username, password)
        }
}

The OO solution is quite a bit more wordy than the functional one, but that is because it has defined two new types, whereas the functional version has just reused a function type. Both are implemented as manual partial application of the send(email, serverAddress, username, password) function. In use the two methods are virtually identical:

object OOCaller {

    val sender: ISendEmail = OOContext.createEmailSender(inetAddress("smtp.rebelalliance.org"), "username", "password")

    fun sendDistress(sender: ISendEmail) {
        sender.send(Email("kenobi@rebelalliance.org", "Help me", "..."))
    }

    fun onUnderAttack() {
        sendDistress(sender)
    }
}

object FPCaller {

    val sender: (Email) -> Unit = FPContext.createEmailSender(inetAddress("smtp.rebelalliance.org"), "username", "password")

    fun sendDistress(sender: (Email) -> Unit) {
        sender(Email("kenobi@rebelalliance.org", "Help me", "..."))
    }

    fun onUnderAttack() {
        sendDistress(sender)
    }
}

The only visible differences are the types of the senders and the fact that we have to call ooSender.send(...) where we just invoke fpSender(...). The latter is in fact syntactic sugar for fpSender.invoke(...), so this latter difference really just comes down to the name of the method called.

This points to a mapping between a function and an object that has a single method with the same signature. Or, put another way, between the function type and a single method interface. This similarity is at the heart of Java 8’s functional programming facilities. Java doesn’t have a syntax for function signatures, so it substitutes interfaces with a single method of the signature that it would like to express.

The same is true for Kotlin, which although it does have a function signature syntax, also defines interfaces that are the equivalent. So in Kotlin, where Function1<P, R> is magicked into existence by the compiler, we could write

    fun sendDistress(sender: Function1<Email, Unit>) {
        sender(Email("kenobi@rebelalliance.org", "Help me", "..."))
    }

I won’t mention this again, as the (Email) -> Unit syntax prevails, but every now and again IntelliJ will introduce the Function types as part of a refactoring, so it’s best to be prepared.

Can we bring our OO and FP solutions even closer together? Why yes we can. Because the function type is just an interface, EmailSender can implement it. How? By defining a method invoke with the signature of the function type.

object OOContext2 {

    class EmailSender(
        val serverAddress: InetAddress,
        val username: String,
        val password: String
    ) : ISendEmail, (Email) -> Unit {

        override fun send(email: Email) =
            send(email, serverAddress, username, password)

        override fun invoke(email: Email) = send(email)

    }
}

If we do this then we can use our class-based sender in place of our functional one.

object MixedCalling {

    val sender: (Email) -> Unit = OOContext2.EmailSender(inetAddress("smtp.rebelalliance.org"), "username", "password")

    fun onUnderAttack() {
        FPCaller.sendDistress(sender)
    }
}

Now our OO solution has become quite a bit more complicated in order to fit in with the FP approach. This calls into question the usefulness of our ISendEmail interface. We can see that it is equivalent to the function type (Email) -> Unit - all it does is give the name send to what happens when you invoke it. Maybe we could just use the type (Email) -> Unit everywhere in place of ISendEmail?

If you think that might not be expressive enough, then maybe you aren’t a functional programmer. Luckily there is a middle ground - we can use a typealias to give a name to the functional type, thus communicating our intent.

typealias ISendEmailToo = (Email) -> Unit

object OOContext3 {

    class EmailSender(
        val serverAddress: InetAddress,
        val username: String,
        val password: String
    ) : ISendEmailToo {

        override fun invoke(email: Email) =
            send(email, serverAddress, username, password)
    }

    val sender: ISendEmailToo = EmailSender(inetAddress("smtp.rebelalliance.org"), "username", "password")

    fun onUnderAttack() {
        FPCaller.sendDistress(sender)
    }
}

Note that as the typealias is just an alias - it doesn’t define a new type - you can write ISendEmailToo and (Email) -> Unit interchangeably, so that FPCalling.sendDistress doesn’t have to be retrofitted with the typealias in order to be called.

There is another way of bridging the OO FP gap that doesn’t involve making your classes implement function types - create a function at the point of invocation. Given our old class-based solution:

object OOContext4 {

    class EmailSender(
        val serverAddress: InetAddress,
        val username: String,
        val password: String
    ) {
        fun send(email: Email) =
            send(email, serverAddress, username, password)
    }
}

we could create a lambda to pass to our functional code

    val sender = OOContext4.EmailSender(inetAddress("smtp.rebelalliance.org"), "username", "password")

    fun onUnderAttack() {
        FPCaller.sendDistress { sender.send(it) }
    }

or just pass a method reference

    fun onUnderAttack() {
        FPCaller.sendDistress(sender::send)
    }

Can we do the opposite - pass our functional sender into something that expects an ISendEmail? Unless I’ve missed a trick; that requires more ceremony, at least in Kotlin, as we have to create an anonymous object implementing ISendEmail to perform the thunk.

    val sender: (Email) -> Unit = FPContext.createEmailSender(inetAddress("smtp.rebelalliance.org"), "username", "password")

    fun onUnderAttack() {
        OOCaller.sendDistress(object: ISendEmail {
            override fun send(email: Email) = sender(email)
        })
    }

Seeing an anonymous implementation of an interface reminds me that we could have implemented an OO solution without introducing a named class.

    fun createEmailSender(
        serverAddress: InetAddress,
        username: String,
        password: String): ISendEmail =
        object : ISendEmail {
            override fun send(email: Email) = send(email, serverAddress, username, password)
        }

In the unlikely event that you’ve made it this far, you may be wondering when to use which method. The fundamental question is, I suppose, should your primary interface for expressing email sending be the ISendEmail interface, which helpfully names its single method, or the ISendEmailToo typealias or its equivalent (Email) -> Unit, which is itself an alias for the generic interface Function1<Email, Unit>?

That last statement gives a clue that at the generated bytecode level things are going to be much the same - the compiler is going to be generating implementations of either interface if we don’t.

If you’re a Kotlin programmer and have to interoperate with Java, then I’d tend to define ISendEmail, as otherwise you’ll be exposing Kotlin function types to your callers’ Java code.

Otherwise my recent practice has been to define the type of parameters or properties as the function type where I can, providing a typealias where required to fully communicate intent. This allows me to provide implementations as either:

Classes that implement the function type
Functional partial application
Ad-hoc lambdas
References to methods with the same signature

I’ve come to consider the functional types as a kind of duck-typing - because they’re built-in to the language you can substitute objects without having to rely on a specific shared type - passing a method reference where a Ruby programmer might pass an object with a known method name.

Of course this only applies where we are talking about interfaces that can be expressed by only one function. Where we want to delegate several operations we could pass multiple functions, but by this stage an actual interface with named methods is probably on the cards. It is surprising though just how often you can express a dependency with a function type, and how expressive and flexible they can make your code.

Stay tuned for Part 3, where I’ll look at another way that objects and functions solve the same problem in different ways.

Objects and Functions in Kotlin - Part 1

2017-05-17T00:00:00-05:00

Imagine, if you can, that you need to send email from some code that you are writing. Just that - not receive mail, or list sent messages - just fire and forget.

As a user of this facility, I’d like to call a function, passing it the information in the email. So given a representation of an email

data class Email(
    val to: String,
    val subject: String,
    val body: String
)

then I’d like to invoke a function like

fun send(email: Email) {
    TODO()
}

Of course when I come to implement this function, I discover that all sorts of other information is required to actually send email. This isn’t information about the email itself, but rather configuration about how to send it. Things like the sending email server’s IP address, login details, other security credentials, timeouts, senders email address - all the things that most people don’t know but you need to set up their new computer. Given the multitude of different email systems, in the end API’s like JavaMail basically punt and pass a whole dictionary, but for now, let’s let 3 parameters stand in for the lot.

fun send(
    email: Email,
    serverAddress: InetAddress,
    username: String,
    password: String) {
    TODO()
}

As a client things have just gotten a lot less convenient. Everywhere I want to send email has to know about this petty configuration - I’ll be passing it around from the top to the bottom of my codebase. I solve that problem by putting it into global variables which works fine until I discover that every run of my unit test suite now ends up sending 5 emails! There must be a better way of hiding these petty details.

You’re probably an object-oriented programmer, so you have a ready-made solution to this problem.

    class EmailSender(
        val serverAddress: InetAddress,
        val username: String,
        val password: String
    ) {
        fun send(email: Email) {
            TODO()
        }
    }

Now when I want to send email I need access to an EmailSender (which I also have to pass around). Instead of calling a function, we invoke a method. We tell it to send, and we don’t have to tell it the petty details it needs because it already knows.

    val sender = EmailSender(inetAddress("smtp.rebelalliance.org"), "username", "password")

    fun sendDistress(sender: EmailSender) {
        sender.send(Email("kenobi@rebelalliance.org", "Help me", "..."))
    }

    fun onUnderAttack() {
        sendDistress(sender)
    }

As a bonus, if I extract an interface, I can configure my tests to use a dummy and not actually send emails.

    interface ISendEmail {
        fun send(email: Email): Unit
    }

    class EmailSender(
        val serverAddress: InetAddress,
        val username: String,
        val password: String
    ) : ISendEmail {
        override fun send(email: Email) {
            TODO()
        }
    }

Note that this is not implementation hiding. The static function hides the details of how to connect to a server and send email. Here we are just hiding configuration. Granted we can see that the OO approach could lead to SmtpEmailSender and X400EmailSender implementations, but we aren’t at that stage yet.

How would a functional programmer solve the same problem?

Remember that we’re trying to get to a function with this signature

    fun sendMessage(email: Email) {
        TODO()
    }

given an implementation that looks like this

    fun sendMessage(
        email: Email,
        serverAddress: InetAddress,
        username: String,
        password: String) {
        TODO()
    }

This is partial application - fixing a number of arguments to a function to give a function with fewer arguments. While some languages provide built-in support for this sort of thing, in Kotlin the easiest approach is to write a function to partially apply our configuration.

What we want is a function that takes the configuration, and returns a function that knows how to send a message.

    fun createEmailSender(
        serverAddress: InetAddress,
        username: String,
        password: String): (Email) -> Unit {
        TODO()
    }

This is the functional analog of the OO case, where the EmailSender constructor is a function that ‘returns’ an object that knows how to send a message.

In longhand, our function can return an inner function that captures the arguments it requires from the parent

    fun createEmailSender(
        serverAddress: InetAddress,
        username: String,
        password: String): (Email) -> Unit {

        fun result(email: Email) {
            send(email, serverAddress, username, password)
        }

        return ::result
    }

We can then make the return a lamba expression

    fun createEmailSender(
        serverAddress: InetAddress,
        username: String,
        password: String): (Email) -> Unit {

        val result: (Email) -> Unit = { message -> send(message, serverAddress, username, password) }

        return result
    }

before inlining all the stuff to leave this functional definition

    fun createEmailSender(
        serverAddress: InetAddress,
        username: String,
        password: String): (Email) -> Unit =
        { message ->
            send(message, serverAddress, username, password)
        }

ie createMessageSender is a function that returns a lamba that calls sendMessage combining the lambda’s single message argument with the configuration from createMessageSender. This lambda is also known as a closure, as it closes-over the values that it requires from it’s enclosing context, capturing them for use later.

To use this function, we can create it in one place and invoke it in another, very much as we did with the object solution.

    val sender: (Email) -> Unit = createEmailSender(inetAddress("smtp.rebelalliance.org"), "username", "password")

    fun sendDistress(sender: (Email) -> Unit) {
        sender(Email("kenobi@rebelalliance.org", "Help me", "..."))
    }

    fun onUnderAttack() {
        sendDistress(sender)
    }

That concludes Part I. If you’re an OO programmer who hasn’t been exposed to functional techniques, I suspect that it has been hard to follow. If you can though, stick with it, as you do get used to the style, and it does allow us to solve problems and express our intent in new and interesting ways.

In Part 2 I’ll look at ways in which we can interoperate between the object and functional worlds.

Test Driven Development v Testing Part 3 - Approval Tests

2017-01-21T00:00:00-06:00

This isn’t the post I thought I was going to write for Part 3 last week, but it occurred to me that I could reasonably TDD FizzBuzz with Approval Tests too, so here goes.

Approval Testing is basically the application of Golden Master Testing to in-development rather than legacy code. You really don’t need any test infrastructure to do this, but as I wrote some, we’re going to use it!

Okey-doke is a library that integrates with JUnit via a Rule.

public class FizzBuzzApprovalsTests {    
    @Rule public ApprovalsRule approver = ApprovalsRule.usualRule(); 
}

The nature of Approval Testing is that we just write code and approve its output if it meets our expectations. So let’s add a test using the approver and see what happens.

    @Test
    public void test() {
        approver.assertApproved(fizzBuzz(1));
    }

Nothing can happen until the code compiles, and approvals testing is all about moving fast, so let’s just add an implementation that covers most cases.

    public String fizzBuzz(int i) {
        return String.valueOf(i);
    }

Running this gives a test failure

java.lang.AssertionError: 
Expected :null
Actual   :1

which is telling us that nothing was expected (there is not yet any approved output for this test). A file FizzBuzzApprovalsTests.test.actual is created with the actual output, viz

Okeydoke also tells you what to do if you want to do to approve the output

To approve...
cp '/Users/duncan/Documents/Work/website-jekyll/site/FizzBuzz/src/com/oneeyedmen/play/FizzBuzzApprovalsTests.test.actual' '/Users/duncan/Documents/Work/website-jekyll/site/FizzBuzz/src/com/oneeyedmen/play/FizzBuzzApprovalsTests.test.approved'

Do we want to approve? Well, the output of 1 is right for the case we’ve considered, so let’s copy the actual to the approved file as suggested and run the tests again. This time they pass as they are the same. We could add the FizzBuzzApprovalsTests.test.approved to version control as it is effectively part of our test sources.

Now we could write another test to cover other numbers, but with Approval Tests it’s easy to check a lot of cases at once, so let’s check 1 … 31 by building a string with the results and checking that

    @Test
    public void test() {
        approver.assertApproved(
            IntStream.range(1, 32).mapToObj(this::fizzBuzz).collect(joining(","))
        );
    }

which fails

org.junit.ComparisonFailure: 
Expected :1
Actual   :1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31

but it is what we want so we approve it, and run the tests again just to make sure they pass now.

Now instead of writing another test we can just add to our implementation.

    public String fizzBuzz(int i) {
        if (i % 3 == 0) return "Fizz";
        return String.valueOf(i);
    }

and run the test

org.junit.ComparisonFailure: 
Expected :1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31
Actual   :1,2,Fizz,4,5,Fizz,7,8,Fizz,10,11,Fizz,13,14,Fizz,16,17,Fizz,19,20,Fizz,22,23,Fizz,25,26,Fizz,28,29,Fizz,31

Hmmm, it’s getting difficult to see what’s going on there, as the expected and actual are out of sync on the line. As this is deliberate practice, I’m going to stash the production change and fix the output format first.

IntStream.range(1, 32).mapToObj(this::fizzBuzz).collect(joining("\n"))

Running the test this time we can’t see a diff in the console, but crucially IntelliJ says

org.junit.ComparisonFailure:  <Click to see difference>

Clicking to see the difference gives us a diff view showing something like

Expected                    Actual
1,2,3,4,5,6,7,8,9,10,11..   1 
                            2 
                            3 
                            4 
...

which was the point of the formatting change, so we approve, then unstash the production change to give a failing test with the diff

Expected                    Actual       
1                           1
2                           2
3                           Fizz
4                           4
5                           5
6                           Fizz
7                           7
...

which is going to let us make a lot more sense of the results. So we approve, and then write more production code.

    public String fizzBuzz(int i) {
        if (i % 3 == 0) return "Fizz";
        if (i % 5 == 0) return "Buzz";
        return String.valueOf(i);
    }

This causes another approvals failure

Expected                    Actual
1                           1
2                           2
Fizz                        Fizz
4                           4
5                           Buzz
Fizz                        Fizz
7                           7
8                           8
Fizz                        Fizz
10                          Buzz
11                          11
Fizz                        Fizz
13                          13
14                          14
Fizz                        Fizz
16                          16
...

which is closer, so we approve that too. One last step

    public String fizzBuzz(int i) {
        if (i % 3 == 0) return "Fizz";
        if (i % 5 == 0) return "Buzz";
        if (i % 15 == 0) return "FizzBuzz";
        return String.valueOf(i);
    }

and we run the tests, which pass. A quick check-in, and we’re home in time for tea and medals.

Luckily my pair, maybe it was you, points out that they shouldn’t have passed, as the approved file still shows Fizz at 15 rather than FizzBuzz. Complacency is the main problem I have with Approval tests, but luckily we hadn’t pushed, so our embarrassment is local. A quick fix

    public String fizzBuzz(int i) {
        if (i % 15 == 0) return "FizzBuzz";
        if (i % 3 == 0) return "Fizz";
        if (i % 5 == 0) return "Buzz";
        return String.valueOf(i);
    }

gives the desired change

Expected                    Actual
1                           1
2                           2
Fizz                        Fizz
4                           4
5                           Buzz
Fizz                        Fizz
7                           7
8                           8
Fizz                        Fizz
10                          Buzz
11                          11
Fizz                        Fizz
13                          13
14                          14
Fizz                        FizzBuzz
16                          16
17                          17
Fizz                        Fizz
19                          19
20                          Buzz
Fizz                        Fizz
22                          22
23                          23
Fizz                        Fizz
25                          Buzz
26                          26
Fizz                        Fizz
28                          28
29                          29
Fizz                        FizzBuzz
31                          31

which we approve and then amend commit. I think we got away with it.

Compared to the example-based tests in Part 1, or the theories of Part 2, our final test is very simple

    @Test
    public void test() {
        approver.assertApproved(
            IntStream.range(1, 32).mapToObj(this::fizzBuzz).collect(joining("\n"))
        );
    }

In fact it makes no sense until you look into FizzBuzzApprovalsTests.test.approved and see

1
2
Fizz
4
Buzz
Fizz
7
8
Fizz
Buzz
11
Fizz
13
14
FizzBuzz
16
17
Fizz
19
Buzz
Fizz
22
23
Fizz
Buzz
26
Fizz
28
29
FizzBuzz
31

Does this make enough sense, given that we have to read it separately from the test? Maybe not, especially given that my previous encouragement to make tests more communicative after the TDD. Let’s finish by making that approved file more explicit

IntStream.range(1, 32).mapToObj((i) ->  i + "\t = \t" + fizzBuzz(i)).collect(joining("\n"))

1	 = 	1
2	 = 	2
3	 = 	Fizz
4	 = 	4
5	 = 	Buzz
6	 = 	Fizz
7	 = 	7
8	 = 	8
9	 = 	Fizz
10	 = 	Buzz
11	 = 	11
12	 = 	Fizz
13	 = 	13
14	 = 	14
15	 = 	FizzBuzz
16	 = 	16
17	 = 	17
18	 = 	Fizz
19	 = 	19
20	 = 	Buzz
21	 = 	Fizz
22	 = 	22
23	 = 	23
24	 = 	Fizz
25	 = 	Buzz
26	 = 	26
27	 = 	Fizz
28	 = 	28
29	 = 	29
30	 = 	FizzBuzz
31	 = 	31

Once you’re in the flow with Approval tests they can be very productive, especially when you don’t need the help of writing a test to work out what the next step is. I suppose we’re skipping the test driving and going straight to the proof-against-regression bit of testing. They are also splendid for getting existing code without tests under test quickly and cheaply, although I suppose that is just Golden Master Testing.

The main downside is that we can’t look directly in the test to see examples of what the code does - we have to look in the approved file. Those examples may also fail to be good documentation to allow us to understand what the code does - in this case they’d do for another programmer, but not for an 8 year-old. Or maybe vice-versa, I’m often wrong.

One application where they really do shine is where we are incrementally improving an algorithm and stopping when it is good enough. This time last year I was parsing a list of strings into first name, given name, title etc. Given a corpus of 2000 names I could approve each improvement and know when a change had led to better or worse results very quickly. Example-based tests would have become unwieldy very quickly in comparison. They can be also be a cheap warning when other things have changed too - I once wrote an Approval Test for the contents of our deployed lib directory so that we didn’t accidentally upgrade dependencies when making changes to the unfathomable Maven build.

I still think that there’s another post to come in this series. I’ll let you know on Twitter when it arrives.

Test Driven Development v Testing Part 2 - Theory Tests

2017-01-15T00:00:00-06:00

As promised at the end of Part 1, I’ll now go through the steps of TDDing FizzBuzz with JUnit theories, inspired by Dominic Fox’s excellent post titled How to TDD FizzBuzz with JUnit Theories. My reasons for doing this are

To compare theories with the ‘traditional’ example-based process shown in Part 1.
Because informative though Dominic’s post was, it didn’t actually show any TDD, just some tests that may or may not have been the end result of some test driving.
To better understand JUnit theories myself.

So, let’s go.

There is more scaffolding and less IntelliJ handholding when writing a JUnit theory test, but I guess we’ll get used to it. Actually I’m just going to copy Dominic’s setup, but using 31 as the upper bound of the values that we’re going to check, as anything more is just a waste.

@RunWith(Theories.class)
public class FizzBuzzTheoryTests {
    @DataPoints
    public static final int[] numbers = IntStream.range(1, 31).toArray();
}

Now what is our first test? In Part 1 this was easy, we just had to assert something about the result for number 1. With theories it’s harder, because we have to assert something that is true for all numbers, will drive our implementation, and allows us to make a very small change. The best I can come up with is

    @Theory
    public void is_not_null(int i) {
        assertNotNull(fizzBuzz(i));
    }

which is at least uncontentious and, through its failure to compile, makes us add our first line of implementation.

    public String fizzBuzz(int i) {
        return "";
    }

This passes straight away, so we can add another theory to drive more code.

    @Theory
    public void is_string_of_number(int i) {
        assertEquals(Integer.valueOf(i), fizzBuzz(i));
    }

I must say that this step left me feeling a bit icky. Firstly, we have done exactly what Dominic complained about in his article - defined the code to be written in the test. I suppose that we could find another expression of the same thing, "" + i maybe. Or taken Dominic’s approach and just specified that the result should have only digits, at the expense of under-specifying. Let’s just say that this is a toy example, for real code there may be a way of specifying the result that is less complicated than the production code.

Also, we’ve written something that we know is not going to be true later on - this test is only going to be valid for a subset of the ints. We could fix that now, but my inner pair is eager to see a green bar, so we go ahead and write the code

    public String fizzBuzz(int i) {
        return String.valueOf(i);
    }

which passes, and then go back and retrofit the assumption.

    @Theory
    public void is_string_of_number_for_other_numbers(int i) {
        assumeTrue(i % 3 != 0 && i % 5 != 0);
        assertEquals(String.valueOf(i), fizzBuzz(i));
    }

which continues to pass. Now for 3’s

    @Theory
    public void is_fizz_when_divisible_by_3(int i) {
        assumeTrue(i % 3 == 0);
        assertEquals("Fizz", fizzBuzz(i));
    }

leading to

    public String fizzBuzz(int i) {
        if (i % 3 == 0) return "Fizz";
        return String.valueOf(i);
    }

and 5’s

    @Theory
    public void is_buzz_when_divisible_by_5(int i) {
        assumeTrue(i % 5 == 0);
        assertEquals("Buzz", fizzBuzz(i));
    }

leading to

    public String fizzBuzz(int i) {
        if (i % 3 == 0) return "Fizz";
        if (i % 5 == 0) return "Buzz";
        return String.valueOf(i);
    }

which still fails!

Now this is the interesting part of the theories TDD. At this stage in Part 2 we had to add a test ourselves to drive out the case of numbers divisible by 15. This time, our naive implementation fails the is_buzz_when_divisible_by_5 test in this case. So a point to the more formal specification, but now we have a few things to do to get to a passing case.

We can try just fixing the code

    public String fizzBuzz(int i) {
        if (i % 15 == 0) return "FizzBuzz";
        if (i % 3 == 0) return "Fizz";
        if (i % 5 == 0) return "Buzz";
        return String.valueOf(i);
    }

but that leads to 2 broken tests! As this is deliberate practice, we should probably back out the production change and fix the tests first, Mikado-style.

    @Theory
    public void starts_with_fizz_when_divisible_by_3(int i) {
        assumeTrue(i % 3 == 0);
        assertTrue(fizzBuzz(i).matches("Fizz.*"));
    }

    @Theory
    public void ends_with_buzz_when_divisible_by_5(int i) {
        assumeTrue(i % 5 == 0);
        assertTrue(fizzBuzz(i).matches(".*Buzz"));
    }

My internal pair and I had a long discussion over this. Should we test for the simple contains, or starts/ends with? In the end we plumped for the start and end, as I felt that they expressed the problem better. The tests are still broken of course, and for longer than he was comfortable with, but at least we know the fix is easy.

    public String fizzBuzz(int i) {
        if (i % 15 == 0) return "FizzBuzz";
        if (i % 3 == 0) return "Fizz";
        if (i % 5 == 0) return "Buzz";
        return String.valueOf(i);
    }

Phew. Are we done? I’m pretty sure that is_not_null is now superfluous, so deleting that and reordering to tell a story we have

    @Theory
    public void starts_with_fizz_when_divisible_by_3(int i) {
        assumeTrue(i % 3 == 0);
        assertTrue(fizzBuzz(i).matches("Fizz.*"));
    }

    @Theory
    public void ends_with_buzz_when_divisible_by_5(int i) {
        assumeTrue(i % 5 == 0);
        assertTrue(fizzBuzz(i).matches(".*Buzz"));
    }

    @Theory
    public void is_string_of_number_for_other_numbers(int i) {
        assumeTrue(i % 3 != 0 && i % 5 != 0);
        assertEquals(String.valueOf(i), fizzBuzz(i));
    }

By the standards of Part 1, then I think that yes, we are done. Our tests will guard against regression, they make good documentation; they might be broken by a determined editor. Personally, they feel a little unfinished though. Perhaps because of their formality, you realise that they don’t fully specify the behaviour - you couldn’t re-implement the code given the test because nothing actually specifies what fizzBuzz(15) should return. So I’d end up adding

    @Theory
    public void is_fizzbuzz_when_divisible_by_15(int i) {
        assumeTrue(i % 15 == 0);
        assertEquals("FizzBuzz", fizzBuzz(i));
    }

which seems a bit arbitrary, but every other formulation ended up worse. Dominic had a different set of theories, but his also suffer from not actually allowing you to re-implement the code from the spec.

Let’s remind ourselves what the tests from Part 1 looked like.

    @Test public void fizz_for_multiples_of_three() {
        assertEquals("Fizz", fizzBuzz(3));
        assertEquals("Fizz", fizzBuzz(6));
    }

    @Test public void buzz_for_multiples_of_five() {
        assertEquals("Buzz", fizzBuzz(5));
        assertEquals("Buzz", fizzBuzz(10));
    }

    @Test public void fizzbuzz_for_multiples_of_three_and_five() {
        assertEquals("FizzBuzz", fizzBuzz(15));
        assertEquals("FizzBuzz", fizzBuzz(30));
    }

   @Test public void test_other_numbers() {
        assertEquals("1", fizzBuzz(1));
        assertEquals("2", fizzBuzz(2));
    }

What have we learnt? For me, TDDing with theories was a lot less comfortable than TDDing with examples, but then I have a lot more experience with the former. In this case I’m not convinced that either my theories or Dominic’s are better tests than the example tests, but they do appeal to the mathematical side of me. Comparing them you can see that the previous tests were a bit sloppy, in particular we should have had names like fizz_for_multiples_of_three_that_arent_also_multiples_of_five.

Which tests better express the rules of FizzBuzz? I think that humans probably value examples over formality, and mathematicians vice versa, so perhaps it depends which way you lean. To me the theories feel like I need to read between the lines, to infer behaviour that should be evident, but maybe I’m just bad at writing them.

I can better imagine writing theories after the TDD cycle, to better specify and catch regressions. Some algorithms and data structures would benefit from theories to verify that pre and post conditions apply during their TDD. I suppose I’d like to think that any concurrent data structures I use were specified and tested in this mathematical style, but perhaps documented with example tests.

If you like this post, or even if you don’t, then you should read Nat Pryce’s article about using property tests, closely related to JUnit Theories, to TDD another old favourite, the Diamond Kata.

And re-reading Dominic’s post in the light of my trying the theories, I think that there is a Part 3 of this series coming, so follow me on Twitter if you want to know when it is published.

Test Driven Development v Testing Part 1 - Example Tests

2017-01-14T00:00:00-06:00

Dominic Fox wrote a very good post recently titled How to TDD FizzBuzz with JUnit Theories. In it he wrote “For a long time I was of the opinion that a) FizzBuzz couldn’t be meaningfully TDD’d, and b) this illustrated a common pitfall with TDD.”

As it happens I attended an excellent workshop led by Jon Jagger at XP2016. We test-drove FizzBuzz several times as different pairs, coming up with different approaches to the testing and the solution.

A typical session went something like this.

First write a test.

    @Test public void test() {
        assertEquals("1", fizzBuzz(1));
    }

That doesn’t compile, so let’s add the function.

    public String fizzBuzz(int i) {
        return null;
    }

That fails, so let’s add the simplest implementation.

    public String fizzBuzz(int i) {
        return "1";
    }

This succeeds, but seems like cheating. Let’s write a test to prove it.

    @Test public void test() {
        assertEquals("1", fizzBuzz(1));
        assertEquals("2", fizzBuzz(2));
    }

That fails, so we can write some production code,

    public String fizzBuzz(int i) {
        return String.valueOf(i);
    }

and maybe rename the test.

    @Test public void test_other_numbers() {
        assertEquals("1", fizzBuzz(1));
        assertEquals("2", fizzBuzz(2));
    }

OK, now for Fizz. Baby steps - this is deliberate practice.

    @Test public void fizz() {
        assertEquals("Fizz", fizzBuzz(3));
    }

Fix the failing test with the simplest fix.

    public String fizzBuzz(int i) {
        if (i == 3) return "Fizz";
        return String.valueOf(i);
    }

Call the cheating by adding a breaking assertion,

    @Test public void fizz_for_multiples_of_three() {
        assertEquals("Fizz", fizzBuzz(3));
        assertEquals("Fizz", fizzBuzz(6));
    }

which we can then fix.

    public String fizzBuzz(int i) {
        if (i % 3 == 0) return "Fizz";
        return String.valueOf(i);
    }

Buzz now. Let’s take 2 steps forward this time -

    @Test public void buzz_for_multiples_of_fice() {
        assertEquals("Buzz", fizzBuzz(5));
        assertEquals("Buzz", fizzBuzz(10));
    }

which fails, and we’ll go straight to the complicated implementation.

    public String fizzBuzz(int i) {
        if (i % 3 == 0) return "Fizz";
        if (i % 5 == 0) return "Buzz";
        return String.valueOf(i);
    }

Now there’s just one lingering doubt in the back of our minds that we can explore with a test, viz

    @Test public void fizzbuzz_for_multiples_of_three_and_five() {
        assertEquals("FizzBuzz", fizzBuzz(15));
        assertEquals("FizzBuzz", fizzBuzz(30));
    }

The simplest fix is probably this

    public String fizzBuzz(int i) {
        if (i % 15 == 0) return "FizzBuzz";
        if (i % 3 == 0) return "Fizz";
        if (i % 5 == 0) return "Buzz";
        return String.valueOf(i);
    }

but personally I think that this is a better, if less efficient, expression of the intent

    public String fizzBuzz(int i) {
        String result = "";
        if (i % 3 == 0) result = result + "Fizz";
        if (i % 5 == 0) result = result + "Buzz";
        return result == "" ? String.valueOf(i) : result;
    }

and yes, I am being deliberately provocative with the == ;-)

Let’s have a look at those tests all together.

   @Test public void test_other_numbers() {
        assertEquals("1", fizzBuzz(1));
        assertEquals("2", fizzBuzz(2));
    }

    @Test public void fizz_for_multiples_of_three() {
        assertEquals("Fizz", fizzBuzz(3));
        assertEquals("Fizz", fizzBuzz(6));
    }

    @Test public void buzz_for_multiples_of_five() {
        assertEquals("Buzz", fizzBuzz(5));
        assertEquals("Buzz", fizzBuzz(10));
    }

    @Test public void fizzbuzz_for_multiples_of_three_and_five() {
        assertEquals("FizzBuzz", fizzBuzz(15));
        assertEquals("FizzBuzz", fizzBuzz(30));
    }

and compare them to those that Dominic said “contain the logic of the implementation, only in an obscure and intractable form.”

    @Test
    public void multiplesOfThreeButNotFiveAreFizz() {
        for (int i = 1; i <= 100; i++) {
            if ((i % 3 == 0) && !(i % 5 == 0)) {
                assertEquals("Fizz", unit.apply(i));
            }
        }
    }
     
    @Test
    public void multiplesOfFiveButNotThreeAreBuzz() {
        for (int i = 1; i <= 100; i++) {
            if (!(i % 3 == 0) && (i % 5 == 0)) {
                assertEquals("Buzz", unit.apply(i));
            }
        }
    }
     
    public void multiplesOfThreeAndFiveAreFizzBuzz() {
        for (int i = 1; i <= 100; i++) {
            if ((i % 3 == 0) && (i % 5 == 0)) {
                assertEquals("FizzBuzz", unit.apply(i));
            }
        }
    } 
// ...and so on

or this, which he said “feels wildly unsatisfactory … It’s hard to see this as really rigorously testing anything.”

    @Test
    public void testSomeJudiciouslyChosenValues() {
        assertEquals("Fizz", unit.apply(3));
        assertEquals("4", unit.apply(4));
        assertEquals("Buzz", unit.apply(5));
        assertEquals("FizzBuzz", unit.apply(15));
        // just in case
        assertEquals("FizzBuzz", unit.apply(30));
    }

Are any of these tests good enough? Which ones provide best coverage and communication? I’ll leave you to mull that over while I ride my bike.

OK, I’m back. What was your verdict? Would you be happy with any of those tests for FizzBuzz?

For me, we can discount testSomeJudiciouslyChosenValues easily, but only because it doesn’t go out of its way to describe the system under test. Using judiciously chosen values to demonstrate and verify the behaviour of a system isn’t of itself a bad thing. If it was, pulling on this thread would unravel pretty much all of Behaviour Driven Design.

Looking at the set beginning with multiplesOfThreeButNotFiveAreFizz, I agree with Dominic - they are pretty obscure and intractable. But then, they were probably designed to show theories in a good light, so perhaps they are a straw man. The fact is that testing the first 100 numbers is arbitrary and misses the opportunity to communicate the fact that FizzBuzz has period of 15. A sane implementation can be verified in far fewer examples, and an insane implementation, one that looks at the test and goes out of it’s way to break it, could throw UnsupportedOperationException at 101, or return “Banana” at random with a probability of 1:10000.

The traditional goal of unit tests is to prevent accidental breakage during modifications of this or other code - regression. Well-written tests can also serve to demonstrate what the code does - communication. In TDD they have another job, to guide the implementation. I would argue that the tests we wrote actually TDDing FizzBuzz here fulfil the three roles of regression, communication and guiding the implementation pretty well. They helped us write the code, the names tell us what the code does, and the examples exemplify the names.

There have been times when the tests that I have written to guide the implementation of a system have not communicated well, or have holes that might allow regressions to slip through (what is fizzBuzz(0)?). Before we consider the job done, we should look at the tests that helped us drive the design and judge them against the regression and communication criteria. If they fall short, then add examples, refactor the tests, maybe even recast them as theories. Don’t expect the test artifact of TDD to be perfect first time, because TDD Is About Design, Not Testing.

Tune in later for Part 2, where I’ll repeat TDD FizzBuzz using JUnit theories from the outset.

More Kotlin Extension Fun

2016-10-06T00:00:00-05:00

This is a follow-up to my last post, Extension Functions Can Be Utility Functions. Here I’ll look at more pros and cons of extension functions.

The article brought out 3 extension function use-cases.

Adding a little local expressiveness with private methods like JsonNode.toAddress()
Adding methods you wish were available on types that you don’t own - JsonNode.getNonBlankText(name: String)
Keeping your own classes small by defining operations as extension functions rather than actual methods - AccessType.toUIString()

On the Kotlin Slack Channel, Daniel Wellman asked

“How do you decide when methods should be extension functions and when they should be members (functions) on classes/objects _that you own_?”

Nathan Armstrong replied

“I think it’s usually when it encapsulates operations on a type in some context tangential to the one the type was made for. For example, when writing code that intersects two domains or concerns, using extension methods can help you use your existing types more naturally alongside one another without leaking that local utility into other parts of the codebase”

He’s right. So right that I’m going to say that again with many more words and claim the credit.

Most of my Java projects have ended up with a package of domain objects - Person, Address etc. Kotlin makes these nice and simple with data classes, and when 1.1 allows inheritance these will become even more useful.

Over time these classes accrete operations that make sense to our application. Things like Person.asJson() and Address.isCloseMyHouse(). These operations dilute the essence of the class for our happy convenience, but after a while we can’t see the platonic wood for expedient trees.

Extension functions let us express the PersonNess in the Person class, and the JSONness elsewhere. In this case maybe type-classes would be better, but for now, we can move our JSON writing and useful-to-us-but-not-really-essential operations off of the class and into extension functions.

A nice side-effect of this is that we can build our domain classes in a module that doesn’t have dependencies on Jackson and some SQL library and the whole of the rest of our app. Maybe we’ll finally get reuseable abstractions for those fundamental real-world things that every app has to implement from scratch because of all the special cases that define our Customer rather than yours.

A word of warning though. Extension functions are not polymorphic. If I have

class Person

class Customer: Person

fun Person.toJson() = ...

fun Customer.toJson() = ...

the bindings are as follows:

Person().toJson() // calls Person.toJson()

Customer().toJson() // calls Customer.toJson()

val person: Person = readCustomer(42)
person.toJson() // calls Person.toJson() no matter what the actual type of person

If you like this, Nat Pryce and I are going to be talking about Expressive Kotlin at the Kotlin Night London next Wednesday, 12 October 2016.

Thanks to Springer Nature for allowing me to publish examples from their code. If you’re looking for a Kotlin job in London, they are hiring - please contact me using one of the links in the sidebar.

Extension Functions Can Be Utility Functions

2016-10-05T00:00:00-05:00

I enjoyed Extension functions are not utility functions

I thought that I disagreed, but on more careful reading maybe not. Instead I’ll pull out some extension functions from my current codebase.

We’re finding that extension functions are a simple way to make code more expressive.

Without an extension function:

private fun addressFrom(node: JsonNode): Address = ...

val addresses = nodes.map { addressFrom(it) }

With an extension function:

private fun JsonNode.toAddress(): Address = ...

val addresses = nodes.map { it.toAddress() }

You might the find first example more readable like this:

private fun toAddress(node: JsonNode): Address = ...

val addresses = nodes.map(::toAddress)

in which case fill yer boots, but my mind doesn’t work that way.

Extension functions work well here, but the place where they really shine is method chaining.

Without them:

private fun addressFrom(node: JsonNode): Address = ...

private fun isInUK(address: Address): Boolean = ...

val isInUK = isInUK(addressFrom(node))

Extension functions don’t make you turn your brain inside out:

private fun JsonNode.toAddress(): Address = ...

private fun Address.isInUK(): Boolean = ...

val isInUK = node.toAddress().isInUK()

You may have noticed the private in the examples above - most of the extension functions that we write this way are used locally - just to make a few lines of code more intuitive. But that’s by no means universally true - this is from some library code to extend Jackson in more general ways:

fun JsonNode.getExpected(name: String): JsonNode = get(name) ?: throw MissingPropertyException("property '$name' is missing")

fun JsonNode.getNonBlankText(name: String) = getExpected(name).asText().apply {
    if (isBlank()) throw JsonInterpretationException("property $name is blank")
}

I had wondered whether this style was best suited to operations on types that we don’t own. But I find we have code that acts on our own types:

fun suggestionsTemplateDataFor(suggestions: List<JournalJson>) = mapOf(
    "results" to suggestions.withOAText()
)

private fun List<JournalJson>.withOAText(): List<Map<String, Any?>> = this.map { it.withOAText() }

private fun JournalJson.withOAText() = TreeMap(this.asPropertyMap().plus( "openAccessText" to this.accessType.toUIString()))

private fun AccessType.toUIString() = when (this) {
    AccessType.OPEN_ACCESS -> "Yes"
    AccessType.SUBSCRIPTION -> "No"
    AccessType.OPEN_CHOICE -> "Optional"
    else -> "Unknown"
}

Here JournalJson and AccessType are simple data classes, unencumbered by all the operations that the rest of the system would like them to have; because they can define operations like withOAText() as extension functions when they need them. And if defining an extension on List<JournalJson> isn’t a utility function I don’t know what is, but here it really helps us to at least get the gist of what is going on at the top level.

Hidden in there is a call to

fun Any.asPropertyMap(): Map<String, Any?> = SpaceBlanket(this)

which is the archetypal extension function definition - high-level and fundamental.

There are more thoughts on this subject in a follow up article - More Kotlin Extension Fun

If you like this, Nat Pryce and I are going to be talking about Expressive Kotlin at the Kotlin Night London next Wednesday, 12 October 2016.

On Epics

2016-07-28T00:00:00-05:00

The phrase epic has connotations. Epics are embarked on until they are completed. They require heroics. People don’t suspend one epic in order to start another. An epic is only successful at its end.

Our agile planning should be more fluid and less valiant. We should be prepared to change course. We should pick our battles carefully. At every stage we should pick the story that delivers the best return on investment. We should seek to deliver value every day, not just at the end of a piece of work.

By all means break a story into tasks. Please do categorise stories by theme if it helps. But be wary of embarking on an epic.

On Minimal

2016-06-13T00:00:00-05:00

Despite not having seen him for perhaps 20 years, I continue to learn from Paul Earwicker.

One thing that still resonates with me was his advice on design. Say you were tasked with producing the lightest possible bike frame that wouldn’t fail under normal use, but had no access to modelling tools. How should you proceed?

You could design a frame that was obviously strong enough, but how then would you know how much material you could remove to make it light enough? And where would you remove it from? No, faced with this challenge the rational approach (provided you don’t value your front teeth) is to make a frame that is not strong enough, and ride it. When it breaks, reinforce the places where it broke. Repeat until it doesn’t break, and you will have a minimal frame.

When it comes to software, I’m working to deliver a Minimal Viable Product. But my customers and I are constantly disagreeing on whether a feature or enhancement is in or out of the minimal set. They want to add things, I want to deliver what we have. Why? Because as with the bike frame, it’s easy to know minimal when you see it, but hard to predict what makes a thing viable. If you actually want to ship an MVP, you have to approach it from below, delivering a product that turns out not to be viable and adding to it until it is. You have to look back and say, ah, that minimal release was the first one that was viable.

Why are my customers fighting this? Well they’re rational too, and know that a consequence of this process is that your first few releases will be failures. If you can’t survive the embarrassment of walking home with a broken bike frame, then maybe it’s best to make sure that your MVP process is a bit less public.

How I Learned to Write Legacy Code

2016-05-23T00:00:00-05:00

The following is the text of a talk I gave at Imperial College in February. The images are those used in the talk, and are used here without explicit permission.

Hello. As Antonio says I’m Duncan McGregor. I graduated with a Physics degree from Imperial in 1989, when Windows looked like this.

From Imperial I joined the Ministry of Defence Management Training Fast Stream, where I worked in various mainly R&D roles. I left the MoD in 1994, when Windows looked like this.

and had a few permanent jobs in software companies before taking the plunge and becoming a contractor in 2000, when Windows looked like this.

These days (when I don’t even know what Windows looks like any more) I work mainly in London, largely on the JVM, and in various different application areas, most recently in academic publishing.

When Robert asked me to talk about legacy code and participate in your discussion, I thought I’d dredge up a few war stories and see if we, or at least I, can learn something from my career.

My first real exposure to commercial code was at MTA, a small consultancy I joined after I was ejected from the MoD. The consultancy had just been commissioned to extend a Lisp expert system that planned Royal Marines amphibious landings – essentially which troops and equipment should leave which ships on which boats and helicopters at what time to hit the beach at H-hour, with the most scary marines at the front.

I bring this project up because (with a bit of a bash) it fits the original definition of legacy – essentially “something of value left by old people to young people.” This wasn’t an albatross - it was an asset. Just because I hadn’t written it didn’t mean that it was bad - it did the job, was capable of being extended, and made the company more money.

This contrasts with my next job, at ESL, a company that wrote X.400 email systems. No one remembers X.400 any more – it was email designed by international standards bodies rather than Unix hackers, and so died the death that most over-engineered solutions do when disintermediated by free software. The codebase, and the people who knew how to tend it, were the company’s only asset – and when I joined they had just been bought out by an American firm who had won a contract with the US Department of Defence and needed code to fulfil it.

Unfortunately this codebase was much more like our usual software definition of legacy – old and hard to work with. It was, I think, originally quite well engineered – there were sensible conventions, and reasonable encapsulation for a C codebase. But this was lost under the accretion of years of new features and misunderstandings. I remember that, years after leaving the job, I finally realised that the memory management scheme – where heaps were allocated by upstream code to allow downstream functions to return pointers – was a clever way of taking advantage of the pipeline nature of message decoding to make up for lack of garbage collection. But if anyone at the company understood that by the time I arrived, they certainly didn’t explain it. Instead we had heuristics for when heaps had to be allocated and destroyed that seemed arbitrary because the original model had been forgotten.

The result of this and other code issues was painfully slow progress and eventual bankruptcy, so I took a job at Roke Manor Research, then a part of Siemens. Work there was on nice new R&D projects with rubbish logos,

so I learned little about legacy code - until I discovered Extreme Programming.

This was 1999, and it’s easy to forget just how revolutionary Ward Cunningham and Kent Beck‘s ideas were. Up to this point we considered that finding all the requirements for a software system before starting work was a worthy goal. That way the system could be designed to meet those requirements – and the team could implement the design with least waste.

At least, that’s true for the first version, but after that, you have the system as it is and have to work out how to hack Version 2’s features into the existing system. I can’t find the quote, but Kent said something like – “I read that 80% of all effort on software is maintenance – and wondered how we could get that up to 95%.”

I won’t claim that I really understood this at the time, but it is crucial. Your first release is not only building the software – it’s building the team that builds the software. If when it delivers Version 1, that team has only learnt how to implement features given a clean slate, things are going to go very wrong when faced with a list of features and an existing codebase.

This was a lesson that was evidently not learned by Sony - my first client when I started contracting. It gave the prototype Set Top Box application my team had developed to another team to ‘productionise.’ But it was a validation of XP when it comes to code quality. The team took the code and, having studied it, came back and asked, “How did you guys develop this – because we want to be doing that!” That’s an existence proof for a way of writing legacy code that people want to work in – Test Driven Development and You Ain’t Gonna Need It.

Around this time I took a commission to write an interface between the Tradacoms EDI system and a soft-furnishing manufacturer’s accounting package. I didn’t know it at the time, but over 10 years this system would process hundreds of millions of pounds worth of orders and invoices. (If I had have known, I would have charged considerably more.) I wrote the software in Python, because I wanted to learn the language. This is not the best reason for choosing a technology for a system that you’re going to end up supporting for over a decade.

The software was upgraded perhaps 8 times over its lifetime. Every time I returned to the code it was after enough of a break to have to work out how it functioned just from the code and documentation – it was legacy code every time. Python isn’t ideal for this – the dynamic typing makes reasoning with the code harder than in a statically typed language. Nevertheless, in the dozen years that it was running data was never data lost because of a coding error, in fact I don’t think I ever had a bug report that wasn’t actually a feature request. How? Well I knew that this was critical code and wrote tests accordingly, both at the unit and acceptance level, and more regression tests to make sure that upgraded code gave the same output as its predecessor on old messages.

Returning to a piece of code after a break is also an ideal time to add the comments and tests that would have helped you understand the code quicker this time had they been there. I really do think that this codebase got better with age – it came to express the problem domain so that new features were often just a matter of configuration or wiring up existing code in a new way. This is of course legacy code nirvana – and I’ve not been lucky enough to work on another system that aged so well.

That may be because it’s so much easier to work on your own code – even when the old you wasn’t half as good as the current you is. In late 2002 I started another XP project in Sony – a little video editing product that a pair of us released in 9 months and went on to extend. I then left the project – at that point the code wasn’t perfect, but it was really fit for purpose, with a good architecture and test coverage.

I would return to work on the same codebase, now being extended by a team of 6 plus QA, a little over 2 years later. Actually it was in remarkably good shape – with much of the original architecture intact, proving that an evolutionary design can support change. Crucial was that my original partner on the project was still in charge, so that our discovery of solutions and ways of thinking about the domain had not been lost. He also kept the culture of tests, if not always TDD, so that new code was forced by this constraint into being at least more than usually habitable.

This theme of tests driving code quality resurfaced in 2012 when I went to work on a major satellite broadcaster’s set top box software. When I arrived this had recently had its first release, albeit 4 years late. Some of that time had been spent retrofitting tests to the codebase.

Now I suppose if you agree with Michael Feathers - that legacy code is code without tests - it should logically follow that you can turn your decaying stately home into a plush modern mansion just by adding the tests that weren’t there.

It won’t surprise you to know that this is not true. Test Driven Development works because it makes writing software with sensible seams, logical interfaces and defined dependencies the path of least resistance. But if you’re tasked with adding tests to existing code - it’s probably easier not to refactor - just throw mocks at the problem, and if they aren’t powerful enough, then surely powermocks will be. The resulting tests will then prevent you from changing the behaviour of your code accidentally, or, unfortunately, deliberately.

Moving onto another contract at a major fashion retailer reinforced the lesson that tests done badly can prevent any change. Here things were better factored but hobbled by a test suite that tied the tests to the Spring dependency injection framework. The result was individual tests that could take seconds to start, tests that required chasing through the filesystem and class hierarchy to find the data that they relied on, and tests that depended on other tests having run first. And don’t get me started on the tests that started failing on 1 Jan 2015 – my second week, because that was as far into the future as far as anyone could imagine this codebase lasting.

What went wrong here? I suppose that in the end there were just too many people working on the code for too long, in a culture of ‘good enough.’ But the truth is that most developers are not good enough to prevent poorly architected systems from eventually collapsing under their own weight, most architects are not good enough to design systems that can cope with average developers, and most clients won’t pay for the overhead in any case.

But I don’t want to end on a negative note! I’ve just come from a meeting planning work on a multi-man-decade academic peer-review system for a client that does understand the importance of these things (and will be hiring in the summer!) In the end, what we do is engineering – it requires knowledge, experience, creativity and taste – and these are the very things that can keep our legacy code a habitable place to work.

Konsent

2016-05-12T00:00:00-05:00

I’ve just released Konsent, an acceptance testing library for Kotlin.

The basic idea is that you can write your tests in a nice IDE-friendly and refactorable Kotlin DSL, but they generate Gherkin that your customer can read and approve as the spec.

So you write this

@RunWith(Konsent::class)
@Preamble(
    "As a developer named Duncan",
    "I want to know that example.com is up and running")
class KonsentExampleTests : ChromeAcceptanceTest() {

    val duncan = actorNamed("Duncan")

    @Scenario(1) fun `Example_dot_com loads`() {
        Given(duncan).loadsThePageAt("http://example.com")
        Then(duncan) {
            shouldSee(::`the page location`, pathContains("example.com"))
            shouldSee(::`the page title`, equalTo("Example Domain"))
            shouldSee(::`the page content`, containsALink("More information...", "http://www.iana.org/domains/example"))
        }
    }

    @Scenario(2, "Following a link from example.com") fun cant_have_dots_in_quoted_method_names() {
        Given(duncan).loadsThePageAt("http://example.com")
        When(duncan).followsTheLink("More information...", "http://www.iana.org/domains/example")
        Then(duncan).shouldSee(::`the page location`, equalTo(URI("http://www.iana.org/domains/reserved")))
    }

    @Scenario(3) fun `Dispensing with the given when then`() {
        duncan.he.loadsThePageAt("http://example.com")
        duncan.he.followsTheLink("More information...", "http://www.iana.org/domains/example")
        duncan.shouldSee(::`the page location`, equalTo(URI("http://www.iana.org/domains/reserved")))
    }
}

and they read this

Feature: Konsent Example Tests
    As a developer named Duncan
    I want to know that example.com is up and running

    Scenario: Example_dot_com loads
        Given Duncan loads the page at "http://example.com"
        Then Duncan sees the page location "location contains "example.com"
        and Duncan sees the page title is equal to "Example Domain"
        and Duncan sees the page content contains a link [More information...](http://www.iana.org/domains/example)

    Scenario: Following a link from example.com
        Given Duncan loads the page at "http://example.com"
        When Duncan follows the link [More information...](http://www.iana.org/domains/example)
        Then Duncan sees the page location is equal to http://www.iana.org/domains/reserved

    Scenario: Dispensing with the given when then
        Duncan loads the page at "http://example.com"
        Duncan follows the link [More information...](http://www.iana.org/domains/example)
        Duncan sees the page location is equal to http://www.iana.org/domains/reserved

Thanks to my patrons at Springer Nature, on whose dime this approach has been developed. We’re hiring, so get in touch if you’re in London and this stuff interests you.

Stay tuned - I’m planning a post soon on how to implement a DSL like this in Kotlin.

Interface v Implementation

2016-05-11T00:00:00-05:00

Context

Here at Springer Nature (still hiring :-) we have a single-page webapp that recommends academic journals via a small web service which in turn queries an Elastic Search index. The Elastic Search index needs to be refreshed periodically - we’re not sure how often, probably once a month or so. The data that is indexed comes from an XML web service, merged and transmogrified in XPath’y ways.

If you’ve been reading along, yes, this is the same refresher I looked at in the first 2 episodes.

Now to Run It

We’d finished the code to update the index, and just needed a way to run it periodically. My colleague imported Quartz, and created a dummy Job implementation that printed “Hello World” on the 28th day of every month, courtesy of Quartz’s CronTrigger.

Our IndexUpdater class has a constructor that takes the source and destination details, and it has a run() method that we’d like Quartz to call on the configured schedule - simples.

I started to tackle this task on the train on the way into work. The Internet connection was patchy, masking the fact that the Quartz website was down. But I had the source, and it seemed to be saying that I had to create a JobDetail object (via the inevitable JobDetailBuilder). The JobDetail references an implementation of a Job interface, that has an execute method. So all I have to do is make my IndexUpdater implement Job, and rename run to execute!

Except that isn’t what Quartz does. It doesn’t take an instance of a class that implements Job. It takes a reference to a class that implements Job, instantiates it using a no-arg constructor, then calls newlyCreatedInstanceOfJob.execute(). It requires reflection to do this, its one, erm, job.

How on earth did this get to be the case? How is the simple case of just calling a method on an object not the default? Somewhat tongue-in-cheek I tweeted:

“Is there any way to persuade Quartz to - how to describe this crazy use case - invoke a function on a schedule?”

and then asked the same question on Stack Overflow.

Of course you can write code to thunk between the simple case and the reflecto-magic case …, but surely it should be that the reflecto-magic is implemented in terms of the simple method call, not vice versa? Quartz developers, how did it get like that?

What Quartz, Really?

Arriving at work, I was asked, “Why are you using Quartz if its interface is so bad?”

Now while there is a bit of me that distrusts the work of programmers who would have made the JobDetail their first choice, I do think that parsing Cron expresssions, running a job on schedule, making sure that a job can only be running one instance, recovering from errors etc, is a complex task. I’m pretty sure that if I tried it there would be at least one bug remaining after a day or two or work. I first used Quartz at least 10 years ago and there are thousands of projects depending on it - the chances are it’s reliable and stable where it counts, even if enterprisey and naff in its interface.

Even after banging my head against that interface, we had scheduled tasks running with a configurable Cron pattern and invokable on demand via a POST in under 2 hours of work. Standing on the shoulders of giants should be our default choice, even if we sometimes have to wear orthotics to be comfortable there.

JMock and Kotlin

2016-05-01T00:00:00-05:00

JMock may have lost the mocking war, but for the London crowd it’s still the go-to mocking tool. It turns out that a little Kotlin fairy-dust can make it even more expressive.

In my last post we saw a pretty vanilla use of JMock. In fact there was one nice Kotlin’ism, defining an extension method on Mockery. So where in Java we would write

mockery.checking(new Expectations() { {
    oneOf(progress).reset(2);
    oneOf(indexer).createIndex();
} })

in Kotlin, by defining

fun Mockery.expecting(block: MyExpectations.() -> Unit) {
   this.checking(MyExpectations().apply(block))
}

we can write

mockery.expecting {
    oneOf(progress).reset(2)
    oneOf(indexer).createIndex()
}

In addition, in the name of more readable tests, I had added a subclass of Expectations to allow me to write, for example

mockery.expecting {
   givenActiveJournalIds(
       "1" to throwException(RuntimeException("oops")),
       "2" to returnValue(journal2))
   }
   ...

This was achieved with

class MyExpectations : Expectations() {
   fun givenActiveJournalIds(vararg idResultPairs: Pair<String, Action>) {
       allowing(journals).loadActiveIds()
       will(returnValue(idResultPairs.map { it.first }))
       idResultPairs.forEach { pair ->
           allowing(journals).loadJournalWithArticles(pair.first, 99)
           will(pair.second)
       }
   }
}

which is tighter in Kotlin, but nothing that you couldn’t do in Java.

Looking at a one of the tests, this plugs together into something like

@Test fun reports_exceptions_and_continues() {
   mockery.expecting {
       val x = RuntimeException("oops")

       givenActiveJournalIds(
           "1" to throwException(x),
           "2" to returnValue(journal2))

       oneOf(progress).reset(2)
       oneOf(indexer).createIndex()

       never(indexer).index(JournalJson(journal1))
       oneOf(progress).exception("1", x)

       oneOf(indexer).index(JournalJson(journal2))
       oneOf(progress).indexed(journal2)
   }
   refresher.refresh(journals, indexer, emptySet())
}

This isn’t bad, but one of JMock’s problems is that it doesn’t differentiate between interactions that are queries, and those that are operations. In this case the refresher queries the journals to find what needs to be indexed, then operates on the indexer to add them. Introducing givenActiveJournalIds gives a clue, but it would be nice to see the split more formally.

What I’d like to see is

@Test fun reports_exceptions_and_continues() {
    val x = RuntimeException("oops")
    mockery.given {
        activeJournalIds(
            "1" to throwException(x),
            "2" to returnValue(journal2))
    }.whenRunning {
        refresher.refresh(journals, indexer, emptySet())
    }.thenExpect {
        oneOf(progress).reset(2)
        oneOf(indexer).createIndex()

        never(indexer).index(JournalJson(journal1))
        oneOf(progress).exception("1", x)

        oneOf(indexer).index(JournalJson(journal2))
        oneOf(progress).indexed(journal2)
    }
}

This can be achieved through 2 extension methods and a little class -

fun Mockery.expecting(expectations: Expektations.() -> Unit): Mockery {
    this.checking(Expektations().apply(expectations))
    return this
}

fun Mockery.whenRunning(block: () -> Unit) = ThenClause(this, block)

class ThenClause(private val mockery: Mockery, private val block: () -> Unit) {
    fun thenExpect(expectations: Expektations.() -> Unit) {
        mockery.expecting(expectations)
        block()
        mockery.assertIsSatisfied()
    }
}

Actually that trick can be played in Java 8 too.

If you’re really eagle-eyed, you may have noticed Expektations pop up in that example. That’s to support my final trick, an extension method on Nothing. Why?

Well what if we wanted to simulate an exception in the indexing?

@Test fun `reports exceptions in indexing and continues`() {
    val x = RuntimeException("oops")
    mockery.given {
        activeJournalIds(
            "1" to returnValue(journal1),
            "2" to returnValue(journal2))
    }.whenRunning {
        refresher.refresh(journals, indexer, emptySet())
        executor.runUntilIdle()
    }.thenExpect {
        oneOf(progress).reset(2)
        oneOf(indexer).createIndex()

        oneOf(indexer).index(JournalJson(journal1))
        will(throwException(x))
        oneOf(progress).exception("1", x)

        oneOf(indexer).index(JournalJson(journal2))
        oneOf(progress).indexed(journal2)
    }
}

That will() as a separate statement has always bothered me about JMock - it should bind to the previous statement, but can’t because indexer.index(...) returns Unit (void in Java). In Kotlin we can define

class Expektations: Expectations() {
    fun Any?.will(action: Action) = super.will(action)
    // fun Nothing.will(action: Action) = super.will(action) Update 2016-05-17 - looks like this isn't needed, as Unit extends Any
}

and now within our expectation blocks we can write

{
    oneOf(indexer).index(JournalJson(journal1)).will(throwException(x))
    // or
    allowing(journals).loadJournalWithArticles("1", 99).will(returnValue(journal1))
}

With a bit of work I think that would allow a typed JMock Action so that you could only returnValue with the correct type, but I haven’t pulled on that thread yet.

The final bonus marks in this post go for spotting

@Test fun `reports exceptions in indexing and continues`()

Ignoring the fact that it breaks Prism’s Kotlin highlighter, this hack was pointed out by Nat Pryce - backticks allow spaces in method names, giving beautifully readable specs in code and test runners.

Mocks v Approvals Tests Part 2

2016-04-23T00:00:00-05:00

Aka: Test Driven Design v Test Driven Development

Context

In Part 1 we looked at testing an algorithm using Approvals Tests.

In this Part 2, we’ll start again using JMock, and see what happens.

Quick Recap

The components pieces are Journals

class Journals(...) {
   fun loadActiveIds(): List<String> {...}
   fun loadJournalWithArticles(id: String, articleCount: Int): Journal? {...}
}

and the JournalIndexer

class JournalIndexer(...) {
   fun createIndex() {...}
   fun index(journal: JournalJson) {...}
}

The final step is to:

create the index
load the active ids
remove some that are excluded
pass the remainder to loadJournalWithArticles
then pass the returned Journals to the JournalIndexer.index().

And we should survive exceptions, and loadJournalWithArticles may return null (no such journal), but the indexer can’t index null. And it would be nice to keep track of how far we’ve got, and what worked and what didn’t.

We’re writing a class called IndexRefresher.

Mocking

Pulling in JMock wasn’t too painful, but my pair and I did argue quite a bit more about the form of the test. This is what it ended up looking like, after maybe half a day, plus a little more playing with how to express expectations in Kotlin.

class IndexRefresherTest {

   @Rule @JvmField val mockery = JUnitRuleMockery()

   val progress = mockery.mock(IndexRefresher.Progress::class.java)
   val refresher = IndexRefresher(99, progress)

   val indexer = mockery.mock(ICanIndexJournals::class.java)
   val journals = mockery.mock(ITellYouAboutJournals::class.java)

   val journal1 = Journal("1", "title", null, null, emptyList(), 0.0, AccessType.OPEN_ACCESS, true, null)
   val journal2 = Journal("2", "title", null, null, emptyList(), 0.0, AccessType.OPEN_ACCESS, true, null)

   @Test fun indexes() {
       mockery.expecting {
           givenActiveJournalIds(
               "1" to returnValue(journal1),
               "2" to returnValue(journal2))

           oneOf(progress).reset(2)
           oneOf(indexer).createIndex()

           oneOf(indexer).index(JournalJson(journal1))
           oneOf(progress).indexed(journal1)

           oneOf(indexer).index(JournalJson(journal2))
           oneOf(progress).indexed(journal2)
       }
       refresher.refresh(journals, indexer, emptySet())
   }

   @Test fun skips_excluded_journals() {
       mockery.expecting {
           givenActiveJournalIds(
               "1" to returnValue(journal1),
               "2" to returnValue(journal2))

           oneOf(progress).reset(1)
           oneOf(indexer).createIndex()

           never(indexer).index(JournalJson(journal1))
           never(progress).indexed(journal1)

           oneOf(indexer).index(JournalJson(journal2))
           oneOf(progress).indexed(journal2)
       }
       refresher.refresh(journals, indexer, setOf("1"))
   }

   @Test fun skips_not_found_journals() {
       mockery.expecting {
           givenActiveJournalIds(
               "1" to returnValue(null),
               "2" to returnValue(journal2))

           oneOf(progress).reset(2)
           oneOf(indexer).createIndex()

           never(indexer).index(JournalJson(journal1))
           oneOf(progress).noJournal("1")

           oneOf(indexer).index(JournalJson(journal2))
           oneOf(progress).indexed(journal2)
       }
       refresher.refresh(journals, indexer, emptySet())
   }

   @Test fun reports_exceptions_and_continues() {
       mockery.expecting {
           val x = RuntimeException("oops")

           givenActiveJournalIds(
               "1" to throwException(x),
               "2" to returnValue(journal2))

           oneOf(progress).reset(2)
           oneOf(indexer).createIndex()

           never(indexer).index(JournalJson(journal1))
           oneOf(progress).exception("1", x)

           oneOf(indexer).index(JournalJson(journal2))
           oneOf(progress).indexed(journal2)
       }
       refresher.refresh(journals, indexer, emptySet())
   }

   private fun JUnitRuleMockery.expecting(block: MyExpectations.() -> Unit) {
       this.checking(MyExpectations().apply(block))
   }

   inner class MyExpectations : Expectations() {
       fun givenActiveJournalIds(vararg idResultPairs: Pair<String, Action>) {
           allowing(journals).loadActiveIds()
           will(returnValue(idResultPairs.map { it.first }))
           idResultPairs.forEach { pair ->
               allowing(journals).loadJournalWithArticles(pair.first, 99)
               will(pair.second)
           }
       }
   }
}

Here’s the implementation.

class IndexRefresher(private val articleCount: Int, private val progress: Progress) {

   interface Progress {
       fun reset(total: Int)
       fun indexed(journal: Journal)
       fun noJournal(id: String)
       fun exception(id: String, x: Exception)
   }

   fun refresh(journals: ITellYouAboutJournals, indexer: ICanIndexJournals, excludedIds: Set<String>) {
       val idsToProcess = journals.loadActiveIds().filterNot { excludedIds.contains(it) }
       progress.reset(idsToProcess.size)

       indexer.createIndex()

       idsToProcess.forEach { journalId ->
           try {
               val journal = journals.loadJournalWithArticles(journalId, articleCount)
               if (journal != null) {
                   indexer.index(JournalJson(journal))
                   progress.indexed(journal)
               } else {
                   progress.noJournal(journalId)
               }
           } catch(x: Exception) {
               progress.exception(journalId, x)
           }
       }
   }
}

Comparing the Code

Whilst the algorithm is identical between both implementations, it’s instructive to look at the detail differences.

Most obviously the mock-generated version has an abstraction for Progress, while in the approved version, the progress seam is introduced by overriding the println method. This forces the implementation to be an open class - in Java you wouldn’t notice this, but in Kotlin classes are final by default (probably my biggest beef with the language). The Progress interface is undoubtedly nicer than the plain-old-println in the approved version, and a consequence of being made to think about the relationship between the refresher and reporting progress while we were writing the spec in the test. We could have introduced the interface in the approved version, but didn’t have to in order to test it, so we came down on the side of pragmatism. This pragmatism also explains the nasty hiding of the println global function by a method - obviously we were originally just printing to the console, and the override gave us a dirty way to capture that output into the approved file.

More subtly, while both versions have an interface, ICanIndexJournals, to allow us to introduce selectable indexing behaviour, the mocked version introduces ITellYouAboutJournals, whereas the approved version is able to use the production Journals implementation. This introduction of interfaces just to facilitate testing is common, if controversial. Mockista’s argue that it introduces seams that better express the structure of the problem, and can aid extensibility and reuse. Countering that is the introduction of complexity that is not actually solving todays’ problem, and the difficulty finding a different name for the interface to a thing from its default implementation - should it be IIndexer / Indexer or Indexer / IndexerImpl. Since discovering the IAmAnInterfaceNameThatTellsYouWhatIDo convention I’m more delighted by the opportunity for whimsy than bugged by the two names, but you may disagree.

By the way, if you’re wondering why both versions pass the journals and indexer into the refresh method rather having them as fields, it’s because in the actual code we want to rebuild them from scratch when we reindex (once a month or so) to avoid having any state lying around between runs.

Comparing the Tests

If the implementation is largely the same, the tests differ wildly:

The mock test runs very much faster
The approvals test combines all the things that it is verifying into one test, the mock test shows what happens
The details of the behaviour of the refresher are hidden inside in the approved file, and even then require interpretation. The mock makes the behaviour clear in the test.
The mock test is able to show the behaviour of the code in the face of exceptions, the approvals test doesn’t. I suppose it could, but it would complicate things quite a bit.
The approvals test exercises the Journals code against the real API, at least on its first run.
The approvals test relies on HTTP caching magic, the mock test on JMock magic.
The mock test took and implementation took half a day, perhaps longer than expected because I was working with a new pair. The approvals test and implementation took maybe an hour, but would have been longer had the HTTP caching not already been in place.

You’ll have probably made up your mind which you prefer by now, but I’m still on the fence. The JMock test seems a lot more valuable in showing the behaviour of the code, but it doesn’t tell you anything you couldn’t get by looking at the actual implementation for 2 minutes. It will tell you if a change to that implementation changes its behaviour, but because it’s a strict unit test, won’t tell you if changes to any of its collaborators will change its behaviour. The approvals test is pretty useless at telling you what the code does, but that doesn’t matter if you can work it out in 2 minutes, and it will fail if changes to Journals or some of its dependencies change what it tries to index.

Mocks v Approvals Tests Part 1

2016-04-23T00:00:00-05:00

Aka: Test Driven Design v Test Driven Development

Context

In my current gig at Springer Nature (we’re hiring :-) I’ve been TDDing some Kotlin code to index data from a web service into Elastic Search. All the pieces are in place, they just need to be wired together and invoked.

The component pieces are Journals

class Journals(...) {
   fun loadActiveIds(): List<String> {...}
   fun loadJournalWithArticles(id: String, articleCount: Int): Journal? {...}
}

and the JournalIndexer

class JournalIndexer(...) {
   fun createIndex() {...}
   fun index(journal: JournalJson) {...}
}

The final step is to:

create the index
load the active ids
remove some that are excluded
pass the remainder to loadJournalWithArticles
then pass the returned Journals to the JournalIndexer.index().

There are complications of course, we should survive exceptions, and loadJournalWithArticles may return null (no such journal), but the indexer can’t index null. And it would be nice to keep track of how far we’ve got, and what worked and what didn’t. Let’s write a class called IndexRefresher.

But First, A Test

Whether this is T.D.Development or T.D.Design, I should write a test first. We probably don’t want to hit the web service API or Elastic Search with a unit test, so my first instinct is to reach for JMock. But as it happens this little service hasn’t needed JMock yet, so let’s see how we get on without it.

Approvals Testing

What we do have is an HttpFetcher abstraction that allows us to replay previous responses and an Approvals Testing library. We can use the former to use our actual implementation of Journals, and the latter to lock down the progress reporting and indexed data. The test takes quite a bit of setup, but the first time we run it we will hit the real data and can approve the results from processing it.

class IndexRefresherApprovalsTest {

   @Rule @JvmField val approver = ApprovalsRule.fileSystemRule("src/test/java")

   val transcript: Transcript by lazy { approver.transcript() }
   val cacheDir = File("src/test/resources/refresher-cache")
   val articles = Articles(ContentAPI(CachingHttpFetcher(File(cacheDir, "articles-http-cache"), httpFetcher)))
   val journals = Journals(ContentAPI(CachingHttpFetcher(File(cacheDir, "journals-http-cache"), httpFetcher)), articles)

   val refresher = object : ScratchIndexRefresher(5) {
       override fun println(message: String?) {
           transcript.appendLine(message)
       }
   }

   @Test fun indexes() {
       val indexer = object : ICanIndexJournals {
           override fun createIndex() {
               transcript.append("Index created")
           }
           override fun index(journal: JournalJson): Id<JournalJson> {
               transcript.appendFormatted(journal, jsonFormatter).endl()
               return Id(journal.id)
           }
       }

       val exclusions = readExcludedJournalIdsFrom(journals.javaClass, "/excluded-journals.csv")
       refresher.refresh(journals, indexer, exclusions)
   }
}

The implementation

open class ScratchIndexRefresher(private val articleCount: Int) {

   fun refresh(journals: Journals, indexer: ICanIndexJournals, excludedIds: Set<String>) {
       val idsToProcess = journals.loadActiveIds().filterNot { excludedIds.contains(it) }
       println("Starting - total journals = ${idsToProcess.size}")
       idsToProcess.forEach { journalId ->
           try {
               val journal = journals.loadJournalWithArticles(journalId, articleCount)
               if (journal != null) {
                   indexer.index(JournalJson(journal))
                   println("Indexed $journalId (${journal.title})")
               } else {
                   println("No data for journalId $journalId")
               }
           } catch(x: Exception) {
               kotlin.io.println("Exception processing journal ${journalId}, ${x.message}")
           }
       }
   }

   open fun println(message: String?) = kotlin.io.println(message)

}

The file we approve to pass the test next time looks like this

Starting - total journals = 2976
No data for journal Id 41406
{
 "id" : "299",
 "name" : "Plant Cell Reports",
 ...
}
Indexed 299 (Plant Cell Reports)
No data for journal Id 13646
{
 "id" : "12914",
 "name" : "BMC International Health and Human Rights",
...

Let’s suspend judgement of this code and test until we see what a mocking approach comes up with - it’s after the break.