What's being tested?

Details of the Test Levels

ERCx runs property tests. The properties and the corresponding tests are described in the tables below. Each test has its own row, indicating the conformance level of that test (ABI, Minimal, Recommended, and Desirable) the name of that test (as displayed in the report) and an informal description of the property being tested. Test levels are understood as follows.

  • Tests of level ABI (Signature) check the signatures (i.e., name, inputs, and outputs) of the token functions/events.
  • Tests of level Standard check obligations and recommendations from the standard, i.e., the properties that contain the keywords MUST and SHOULD respectively, but also any property stated in the respective EIP specification.
  • Tests of level Security check properties we deem important to the sane functioning of a token and complement the standard specification. Some of these properties were violated in ERC-20 hacks. Some othesr consider additional functions not in the standard. Examples of security properties include (i) self-transferring tokens is allowed but must not modify the balance (ii) increase of allowance should rely on an increaseAllowance function.
  • Tests of level Features check feature properties of the contract; these are neither desirable nor undesirable properties but indicate implementation choices of the contract. Examples of fingerprint properties include the infinite approval property; which holds if once the approval is set to type(uint256).max, it is not decreased by transferFrom operations.

Possible reasons for failed tests

For most of the tests at non-ABI level, we will initialize tokens to dummy users for each test before we test the intended property. In the event where many of non-ABI tests fails, you can click and refer to the reason of each failing tests to find out the possible reason why it failed:

  • stdStorage find(StdStorage): Slot(s) not found. - There are some inaccessibility issues with the storage slots of the token and as a result, we are unable to initialize tokens to the intended dummy users. Thus, the results from the test evaluation are inconclusive.
  • EvmError: Revert - Some functions reverted when called during the execution of the test suite. Please check if the subjected function in the contract revert unexpectedly.
  • The 'vm.assume' cheatcode rejected too many inputs - The test suite cannot generate appropriate inputs required for the functions called in the test. Please check if there is any inappropriate limit set for the inputs of the subjected function.
  • Arithmetic over/underflow - Some inputs caused the subjected function to have arithmetic over/underflow. This usually occurs when there is some issue with the arithmetic operation within the function, e.g., the deposit, mint,withdraw or redeem function of ERC-4626 token. Please check that the limits as specified by the respective EIP standard is followed. Note that our test suites follow limits (if any) from Openzeppelin implementations. If your contract is implementing other forms of limits, the results from the test evaluation are inconclusive.
  • Any other reasons, e.g., Error: Assertion Failed or NULL - The token indeed does not satisfy all these properties OR there are some issues with the token's transfer or minting function which result in insufficient tokens to test the intended property.

If there are too many unexpected failed tests, please contact us so that we can resolve the issue ASAP.


Details of Property Tests

In the below description of properties, we use the following terminology:

  • tokenSender: address that sends tokens (usually in a transaction);
  • tokenReceiver: address that receives tokens (usually in a transaction);
  • tokenApprover: address that approves tokens (usually in an approval);
  • tokenApprovee: address that tokenApprover approves (usually in an approval).

Tests of level ABI (Signature) check the signatures (i.e., name, inputs, and outputs) of the token functions/events.

Filter by:
PropertyTest name
For ERC-4626 test suite, there are several tests with the phrase "up todelta-approximation" in their test descriptions. These are the tests where calling of functions such as deposit,withdraw, etc, are being carried out and conversation of shares to assets, and vice-versa, will take place. As math operations in Solidity is done entirely using fixed-point (i.e., no decimal value), rounding errors may occur if the contract does not follow the required rounding rules stated in the EIP-4626 standard. However, in the event where the contract does not follow the required rounding rules, there is a global uint256 variable, delta, for the test suite where the user can set to provide some leeway for such errors. This delta value represents the maximum approximation error size (an absolute value given in the smallest unit such as Wei) whenever equality assertion check is carried out. For example, x - y <= delta is being checked whenever there is a check for x == y. It is important to note that delta should only be set to a reasonable small value so that the adversarial profit of exploiting such rounding errors stays relatively small compared to the gas cost. The default value of delta is set to 0 as all tests are supposed to pass at this value if the contract follows the required rounding rules.