Parables is a system for testing smart contracts. Smart contracts are hard to get right and the cost of making mistakes is high. The way we avoid mistakes is by being exceptional at testing.

Parables was built at PrimaBlock to support thorough testing of contracts. We wanted to make use of property testing, but found that conventional testing frameworks like Truffle were too slow to support that.

Property testing typically requires that the thing under test is executed hundreds of times with different valued, randomized parameters. For this reason, individual test cases must be fast. Parables is able to execute complex contract interactions in microseconds since we do it directly on top of the parity virtual machine. We also intend to make testing a first-class citizen of parity by extending the necessary primitives to get it done the right way.

This book is a user guide, suitable for people who want to learn how to do testing on top of parables.

It requires an understanding of Rust, and that you have the cargo toolchain installed. If you don't have it, you can get it through rustup.rs.

We will guide you all the way from setting up a new project, to performing full-scale property testing.

So sit down, buckle your seat belt, and enjoy the trip!

You can find generated documentation for the parables framework at ./doc/parables_testing/, in particular you should check out the prelude which links to everything that is imported by default.

Parables runs as a regular application using a custom testing framework. To run this you should set up a regular rust application. This can easiest be done using cargo init.

cargo init --bin my-contract-tests

Modify the Cargo.toml to add a dependency to parables:

[dependencies]
parables-testing = {git = "https://github.com/primablock/parables"}

[build-dependencies]
parables-build = {git = "https://github.com/primablock/parables"}

If you want parables to rebuild your project when you change a contract, add the following build.rs.

fn main() {
    println!("cargo:rerun-if-changed=contracts/SimpleContract.sol");
    println!("cargo:rerun-if-changed=contracts/SimpleLib.sol");
    println!("cargo:rerun-if-changed=contracts/SimpleLedger.sol");
}

Finally you should set up a main method that uses TestRunner to schedule tests in src/main.rs.

#[macro_use]
extern crate parables_testing;

use parables_testing::prelude::*;

fn main() -> Result<()> {
    let mut tests = TestRunner::new();

    tests.test("something cool", || {
        assert_eq!(1, 2);
    });

    let reporter = StdoutReporter::new();
    tests.run(&reporter)?;

    Ok(())
}

At this stage you can test that everything works with cargo run.

cargo run

something cool in 0s: failed at src/main.rs:9:9
assertion failed: `(left == right)`
  left: `1`,
 right: `2`

Now it's time to add a smart contract.

Create the contracts directory, and write the SimpleContract code below into contracts/SimpleContract.sol.

mkdir contracts

/// contracts/SimpleContract.sol

pragma solidity 0.4.24;

contract SimpleContract {
    uint value;
    address owner;

    event ValueUpdated(uint);

    constructor(uint initial) public {
        value = initial;
        owner = msg.sender;
    }

    modifier ownerOnly() {
        require(msg.sender == owner);
        _;
    }

    function getValue() public view returns(uint) {
        return value;
    }

    function setValue(uint update) public ownerOnly() {
        value = update;
        emit ValueUpdated(update);
    }
}

Compile the contract using solcjs.

We then load it by adding the contracts! macro to the top of our file.

#[macro_use]
extern crate parables_testing;

use parables_testing::prelude::*;

contracts! {
    simple_contract => "SimpleContract.sol:SimpleContract",
};

fn main() -> Result<()> {
    let mut tests = TestRunner::new();

    tests.test("something cool", || {
        assert_eq!(1, 2);
    });

    let reporter = StdoutReporter::new();
    tests.run(&reporter)?;

    Ok(())
}

In the next section we will walk you through how to write your first contract test.

To test a smart contract, we run it on top of the Ethereum Virtual Machine (EVM) built by parity.

Parables provides a wrapper for this using the Evm type.

Don't worry, we will walk you through line-by-line what it is below.

#[macro_use]
extern crate parables_testing;

use parables_testing::prelude::*;

contracts! {
    simple_contract => "SimpleContract.sol:SimpleContract",
};

fn main() -> Result<()> {
    // Set up a template call with a default amount of gas.
    let owner = Address::random();
    let call = Call::new(owner).gas(1_000_000);

    // Set up a new virtual machine with a default (null) foundation.
    let foundation = Spec::new_null();
    let evm = Evm::new(&foundation, new_context())?;

    // Deploy the SimpleContract.
    let simple = evm.deploy(simple_contract::constructor(0), call)?.address;

    // Wrap the virtual machine in a Snapshot type so that it can be shared as a snapshot across
    // threads.
    let evm = Snapshot::new(evm);

    let mut tests = TestRunner::new();

    tests.test("get and increment value a couple of times", || {
        let evm = evm.get()?;
        let contract = simple_contract::contract(&evm, simple, call);

        let mut expected = U256::from(0);

        let out = contract.get_value()?.output;
        assert_eq!(expected, out);

        // change value
        expected = 1.into();

        contract.set_value(expected)?;
        let out = contract.get_value()?.output;
        assert_eq!(expected, out);

        Ok(())
    });

    let reporter = StdoutReporter::new();
    tests.run(&reporter)?;

    Ok(())
}

We will now walk through this line-by-line and explain what it is.

# #![allow(unused_variables)]
#fn main() {
use parables_testing::prelude::*;
#}

This imports everything necessary to write parables test into the current scope.

Check out the prelude documentation as a reference for what is imported.

# #![allow(unused_variables)]
#fn main() {
contracts!();
#}

This makes use of ethabi's derive module to build a type-safe model for the contract that we can use through the simple_contract module.

Through this we can import functions, events, and the contract's constructor.

# #![allow(unused_variables)]
#fn main() {
let owner = Address::random();
let call = Call::new(owner).gas(1_000_000);
#}

In main, we start by creating a random owner, and set up the template model we will be using for our calls.

# #![allow(unused_variables)]
#fn main() {
let foundation = Spec::new_null();
let context = new_context();
let evm = Evm::new(&foundation, context)?;
#}

Time to set up our foundation and our contract context. A foundation determines the parameters of the blockchain. The null foundation is the default foundation, which makes it operate like your modern Ethereum blockchain. But we also have access to older foundations like [morden].

The currently available foundations are:

• Spec::new_null - The most default foundation which doesn't have a consensus engine.
• Spec::new_instant - A default foundation which has an InstantSeal consensus engine.
• Spec::new_test - Morden without a consensus engine.

For more details, you'll currently have to reference the Spec source code.

# #![allow(unused_variables)]
#fn main() {
let simple = evm.deploy(simple_contract::constructor(0), call)?.address;
#}

For the next line we link our contract, and deploy it to our virtual machine by calling its constructor.

Note that the first argument of the constructor is the code to deploy.

# #![allow(unused_variables)]
#fn main() {
let evm = Snapshot::new(evm);
#}

Finally we want to wrap our virtual machine in the Snapshot container. The virtual machine has some state that needs to be synchronized when shared across threads, but it is clonable. The Snapshot class provides us with a convenient get() function that handles the cloning for us.

Next we enter the code for the test case.

# #![allow(unused_variables)]
#fn main() {
let evm = evm.get()?;
#}

This line takes a snapshot of the virtual machine. The snapshot is guaranteed to be isolated from all other snapshots, letting us run many tests in isolation without worrying about trampling on each others feets.

# #![allow(unused_variables)]
#fn main() {
let contract = simple_contract::contract(&evm, simple, call);
#}

This line sets up the simple contract abstraction as contract. Through this you can easily call methods on the contract using the specified evm, address, and call as you'll see later.

# #![allow(unused_variables)]
#fn main() {
let mut expected = 0;

let out = contract.get_value()?.output;
assert_eq!(expected, out);

// change value
expected = 1;

contract.set_value(expected)?;
let out = contract.get_value()?.output;
assert_eq!(expected, out);
#}

This final snippet is the complete test case. We call the getValue() solidity function and compare its output, set it using setValue(uint), and make sure that it has been set as expected by getting it again.

So it's finally time to run your test! You do this by calling cargo run.

cargo run

Parables provides the necessary speed to perform property testing of smart contracts.

We make use of the excellent proptest framework to accomplish this.

Let's rewrite our example from the last chapter to instead of testing that we can get and set some well-defined numeric values, we test a wide range of values.

# #![allow(unused_variables)]
#fn main() {
tests.test("get and increment value randomly", pt!{
    |(x in any::<u64>())| {
        let x = U256::from(x);

        let evm = evm.get()?;
        let contract = simple_contract::contract(&evm, simple, call);

        let out = contract.get_value()?.output;
        assert_eq!(U256::from(0), out);

        contract.set_value(x)?;
        let out = contract.get_value()?.output;
        assert_eq!(x, out);
    }
});
#}

For the heck of it, let's introduce a require that will prevent us from setting the field to a value larger or equal to 1000000.

function setValue(uint update) public ownerOnly() {
    require(value < 1000000);
    value = update;
    emit ValueUpdated(value);
}

What does our test case say?

cargo run

get and increment value randomly in 0.144s: failed at src/main.rs:36:9
Test failed: call was reverted; minimal failing input: x = 1000000
        successes: 0
        local rejects: 0
        global rejects: 0

What's happening here is actually quite remarkable. When proptest notices a failing, random, input, it tries to reduce the value to minimal failing test. The exact strategy is determined by the type being mutated, but for numeric values it performs a binary search through all the inputs.

For more information on property testing, please read the proptest README.

In the next section we will discuss how to expect that a transaction is reverted.

Now we will use the contract from the last section, but instead of simply failing, we will change the assert to expect the revert to happen for some specific input.

Given that getValue looks like this (from the last section).

function setValue(uint update) public ownerOnly() {
    require(value < 1000000);
    value = update;
}

We do that by changing the test case to this:

# #![allow(unused_variables)]
#fn main() {
tests.test("get and increment value randomly within constraints", pt!{
    |(x in any::<u64>())| {
        let x = U256::from(x);

        let evm = evm.get()?;
        let contract = simple_contract::contract(&evm, simple, call);

        let out = contract.get_value()?.output;
        assert_eq!(U256::from(0), out);

        let result = contract.set_value(x);

        // expect that the transaction is reverted if we try to update the value to a value larger
        // or equal to 1 million.
        let expected = if x >= U256::from(1000000) {
            assert!(result.is_reverted());
            U256::from(0)
        } else {
            assert!(result.is_ok());
            x
        };

        let out = contract.get_value()?.output;
        assert_eq!(expected, out);
    }
});
#}

Instead of an error, our test should now pass.

cargo run

get and increment value randomly within constraints in 0.686s: ok

Every address has an implicit account associated with it. The account acts like a ledger, keeping track of of the balance in ether that any given address has.

Accounts are always in use, any transaction takes into account the amount of ether being attached to it and any gas being used.

To make use of balances, we first need to provide an address with a balance.

# #![allow(unused_variables)]
#fn main() {
let foundation = Spec::new_null();
let evm = Evm::new(&foundation, new_context());

let a = Address::random();
let b = Address::random();

evm.add_balance(a, wei::from_ether(100));
#}

The first way we can change the balance of an account is to transfer ether from one account to another using a default call.

# #![allow(unused_variables)]
#fn main() {
let call = Call::new(a).gas(21000).gas_price(10);
let res = evm.call_default(b, call)?;
#}

We can now check the balance for each account to make sure it's been modified.

Note that account a doesn't have 90 ether, we have to take the gas subtracted into account!

# #![allow(unused_variables)]
#fn main() {
assert_ne!(evm.balance(a), wei::from_ether(90));
assert_eq!(evm.balance(a), wei::from_ether(90) - res.gas_total());
assert_eq!(evm.balance(b), wei::from_ether(10));
#}

If you want a more streamlined way of testing that a various set of account balances are as you expect, you can use a Ledger.

The above would then be written as:

# #![allow(unused_variables)]
#fn main() {
let foundation = Spec::new_null();
let evm = Evm::new(&foundation, new_context());
let mut ledger = Ledger::account_balance(&evm);

let a = Address::random();
let b = Address::random();

// sync the initial state of the accounts we are interested in.
ledger.sync(a)?;
ledger.sync(b)?;

evm.add_balance(a, wei::from_ether(100));
ledger.add(a, wei::from_ether(100));

let call = Call::new(a).gas(21000).gas_price(10);
let res = evm.call_default(b, call)?;
// we expect the bas price to be deducted.
ledger.sub(a, res.gas_total());

// consume the ledger and verify all expected stated.
ledger.verify()?;
#}

Suppose we have the following contract:

pragma solidity 0.4.24;

contract SimpleLedger {
    mapping(address => uint) ledger;

    function add(address account) payable {
        ledger[account] += msg.value;
    }

    // used for testing
    function get(address account) returns(uint) {
        return ledger[account];
    }
}

The contract has a state which is stored per address.

A Ledger can be taught how to use this using a custom LedgerState.

# #![allow(unused_variables)]
#fn main() {
let a = Address::random();
let b = Address::random();

let call = call.sender(a);

let evm = evm.get()?;

let simple = evm.deploy(simple_ledger::constructor(), call)?.address;
let simple = simple_ledger::contract(&evm, simple, call.gas_price(10));

let mut balances = Ledger::account_balance(&evm);
let mut states = Ledger::new(State(&evm, simple.address));

evm.add_balance(a, wei!(100 eth))?;

// sync all addresses to initial states.
balances.sync_all(vec![a, b, simple.address])?;
states.sync_all(vec![a, b, simple.address])?;

// add to a
let res = simple.value(wei!(42 eth)).add(a)?;
balances.sub(a, res.gas_total() + wei!(42 eth));
balances.add(simple.address, wei!(42 eth));
states.add(a, wei!(42 eth));

// add to b
let res = simple.value(wei!(12 eth)).add(b)?;
balances.sub(a, res.gas_total() + wei!(12 eth));
balances.add(simple.address, wei!(12 eth));
states.add(b, wei!(12 eth));

balances.verify()?;
states.verify()?;

return Ok(());

pub struct State<'a>(&'a Evm, Address);

impl<'a> State<'a> {
    /// Helper to get the current value stored on the blockchain.
    fn get_value(&self, address: Address) -> Result<U256> {
        use simple_ledger::simple_ledger::functions as f;
        let call = Call::new(Address::random()).gas(10_000_000).gas_price(0);
        Ok(self.0.call(self.1, f::get(address), call)?.output)
    }
}

impl<'a> LedgerState for State<'a> {
    type Entry = U256;

    fn new_instance(&self) -> Self::Entry {
        U256::default()
    }

    fn sync(&self, address: Address, instance: &mut Self::Entry) -> Result<()> {
        *instance = self.get_value(address)?;
        Ok(())
    }

    fn verify(&self, address: Address, expected: Self::Entry) -> Result<()> {
        let value = self.get_value(address)?;

        if value != expected {
            return Err(format!("value: expected {} but got {}", expected, value).into());
        }

        Ok(())
    }
}
#}

Parables provides a helper to set up an account.

Accounts have an address, a private, and a public key.

Through the Account structure we can sign payloads according to the ECRecovery scheme.

# #![allow(unused_variables)]
#fn main() {
let mut crypto = Crypto::new();
let account = Account::new(&mut crypto)?;

let mut sig = account.signer(&mut crypto);

// add things to the signature
sig.input(pool);
sig.input(scenario.owner);
sig.input(code);
sig.input(expiration);

let sig = sig.finish()?;
#}

Parables capatures all events emitted by your contract, and allows you to easily assert that a specific set of events have been emitted.

A typical test would do something like the following.

# #![allow(unused_variables)]
#fn main() {
let evm = evm.get();
let contract = simple_contract::contract(&evm, simple, call);

contract.set_value(100)?;
contract.set_value(200)?;

for e in evm.logs(ev::value_updated()).filter(|e| e.filter(Some(100.into()))).iter()? {
    assert_eq!(U256::from(100), e.value);
}

assert_eq!(1, evm.logs(ev::value_updated()).iter()?.count());
assert!(!evm.has_logs(), "there were unprocessed logs");
#}

Note that converting the drainer into an iterator through the iter() method is a fallible operation since it needs to decode all events.