Continuous Integration of Scala projects and arbitrary statistics over time
continuous-integration, graph, sbt, scala, statistics
Solution
You could write a Jenkins plugin to gather this stdout and do all you are asking for. As an alternative to grabbing standard out, redirect that output to a file, and have the plugin read the file.
At any rate, there are many plugins doing that sort of thing. Just look up Jenkins tutorials, and maybe grab a plugin that does something similar as an example.
As a final note, I know that Travis CI also supports Scala, though I have no experience with it, so I have no clue if they support your requirements or not.
Problem
I am looking for a Continuous Integration tool for various Scala projects. So far I found Jenkins, which apparently supports Scala project via a plugin for Sbt, and Pulse, which does not directly support Scala projects but claims that it makes it easy to integrate any command line build tool, which matches Sbt. I don't have any practical experience with CI, but read a couple of articles about it, usually related to Java. Besides the usual features, e.g. support for various code versioning systems and publishing nightly builds I'd like to be able to record a bunch of statistics, plot them over time and trigger notifications if certain numbers change unexpectedly, i.e., by more than a defined factor. In essence, each test from a test suite prints a number of statistics at the end, which could be extracted by a regular expression and then persisted (associated with the test case they came from). Ideally, there also is an easy way to create queries over the recorded statistics, e.g., to compare the last run of the test suite with an earlier one. However, this is probably not something that can be done directly in a CI tool, but it should at least be possible to export the recorded data in a queryable format, e.g., XML or JSON. Any hints, links or experience reports are welcome! EDIT 1: The statistics are not JVM-statistics in some sense, e.g., code coverage or memory consumption. They are related to the actual work performed by the projects, e.g., the number of comparisons performed by a sorting algorithm, or the number of conflicting assignments in a SAT solver. The numbers are printed to stdio at the end of each run, or written to a file. The CI tool should "only" allow me to render them as graphs over time, or to perform simple analyses (sum, meridian), or to send a mail when a number diverts from the average.