Agent based modelling was first developed in the 1970s, but its adoption as a technique has been constrained by significant challenges:

  1. 1

    Harnessing multiple data sets

  2. 2

    Validating computationally intensive models

  3. 3

    Exploring their large parameter spaces efficiently

Our journey started ten years ago with simple conceptual models of a few hundred agents using very limited agent rule sets and no data.

We are now running multi-million agent simulations driven by multiple, diverse data sets and validated against behavioural data, on cloud-based clusters of hundreds of cores. This is made possible by our Cloud-based technology platform, Sandpipr, which supports our agile model development and deployment process.


Sandpipr consists of the following integrated components:

Data Store

Data ingestion, verification, storage, version management

Model development environment

Development, run-at-scale, verification and validation, evaluation and comparison, cataloguing and documentation

Deployment Platform

Model exploration, reporting & visualisation

Whilst we created Sandpipr to support our own particular data science needs it has become apparent that those needs are generic to the developing data science field.

We are now working towards making Sandpipr available to third parties as a general purpose platform for agile model development by data science teams. You can read more about our vision of agile data science in our white paper.

Download white paper