FM Tools Data Schema

Type: object

The schema for fm-tools/data

Type: string

The identifier of the tool (equal to the file name without file-name extension).

Type: string

The name of the tool.

Type: string

A description of the tool.

Type: array of string

The supported input languages of the tool.

No Additional Items

Each item of this array must be:


B

Type: const
Specific value: "B"

C

Type: const

C is a general purpose programming language.

Specific value: "C"

CUDA

Type: const
Specific value: "CUDA"

DVE

Type: const
Specific value: "DVE"

ETF

Type: const
Specific value: "ETF"

Event-B

Type: const
Specific value: "Event-B"

Java

Type: const

Java is a general purpose programming language.

Specific value: "Java"

LLVM-IR

Type: const
Specific value: "LLVM-IR"

MCRL

Type: const
Specific value: "MCRL"

MCRL2

Type: const
Specific value: "MCRL2"

OpenCL

Type: const
Specific value: "OpenCL"

PNML

Type: const
Specific value: "PNML"

Promela

Type: const
Specific value: "Promela"

SYCL

Type: const
Specific value: "SYCL"

TLA+

Type: const
Specific value: "TLA+"

Wasm

Type: const
Specific value: "Wasm"

SV-LIB

Type: const

SV-LIB is an SMT-LIB-based exchange format for imperative programs.

Specific value: "SV-LIB"

Python

Type: const

Python is a general purpose programming language.

Specific value: "Python"

MoXI

Type: const

MoXI (Model eXchange Interlingua) is an SMT-LIB-based intermediate language for symbolic model checking.

Specific value: "MoXI"

Btor2

Type: const

Btor2 is a word-level format for expressing model-checking problems in sequential circuits over bit-vectors and arrays.

Specific value: "Btor2"

SMV

Type: const

SMV is a formal modeling language for symbolic transition systems and temporal properties.

Specific value: "SMV"

VMT-LIB

Type: const

VMT-LIB is an SMT-LIB-based language for describing symbolic transition systems and properties.

Specific value: "VMT-LIB"

Z

Type: const
Specific value: "Z"

DIMACS CNF

Type: const

DIMACS CNF is the standard exchange format for propositional satisfiability (SAT) problems.

Specific value: "DIMACS CNF"

GRAT

Type: const

GRAT is an unsat certificate format produced by SAT solvers and verified by proof checkers such as GRATchk.

Specific value: "GRAT"

Pseudo-Boolean

Type: const

Pseudo-Boolean is an input format for pseudo-boolean optimization and satisfiability problems.

Specific value: "Pseudo-Boolean"

Type: stringFormat: uri

The URL of the project associated with the tool.

Type: stringFormat: uri

The URL of the tool's repository.

Type: string

The SPDX license identifier of the tool.

Type: string

Tool-info module name of benchexec in the format benchexec.tools.<tool>.

Type: enum (of string)

The version of the fm-tools/data format.

Must be one of:

  • "2.0"

Type: array of string

Maintainers of the tool.

No Additional Items

Each item of this array must be:

Type: array of object

Information about the maintainers of the tool.

No Additional Items

Each item of this array must be:

Type: object

Type: string
Must match regular expression: ^[0-9]{4}-[0-9]{4}-[0-9]{4}-[0-9]{3}[0-9X]$

Type: string or nullFormat: uri

Type: array of object

Information about different versions of the tool.

No Additional Items

Each item of this array must be:


Type: object

The following properties are required:

  • version
  • benchexec_toolinfo_options
  • required_ubuntu_packages
  • doi
Type: object

The following properties are required:

  • version
  • benchexec_toolinfo_options
  • required_ubuntu_packages
  • url

Type: string

Type: string or null
Must match regular expression: 10\.5281/zenodo\.[0-9]+$

Type: string or nullFormat: uri

Type: array of string

Options that should be given to the tool. ${witness} is a placeholder for the path to the witness.

No Additional Items

Each item of this array must be:

Type: array of string
No Additional Items

Each item of this array must be:

Type: array of object

Information about the tool's participation in competitions.

No Additional Items

Each item of this array must be:

Type: object

Type: string
Must match regular expression: (SV-COMP|Test-Comp|SAT Competition) 20[1-9][0-9]$

Type: enum (of string)

Must be one of:

  • "Validation of Correctness Witnesses v1"
  • "Validation of Correctness Witnesses v2"
  • "Validation of Violation Witnesses v1"
  • "Validation of Violation Witnesses v2"
  • "Validation of SV-LIB Witnesses v1"
  • "Verification"
  • "Test Generation"
  • "Validation of Test Suites"
  • "Validation of Test Suites Clang Formatted"
  • "Validation of Test Suites Clang Unformatted"
  • "Validation of Test Suites GCC Formatted"
  • "Validation of Test Suites GCC Unformatted"
  • "Main"
  • "Agile"
  • "NoLimits"
  • "Parallel"
  • "Anniversary"

Type: array of enum (of string)
No Additional Items

Each item of this array must be:

Type: enum (of string)

Characterizes the competition participation.
inactive means that the tool was not submitted by the developers.
meta_tool means that the tool is composed of other tools.
ai means that the tool uses an LLM or other sophisticated kind of artificial intelligence.
unqualified means that the tool was not able to successfully participate.
auxiliary means that the tool is just an auxiliary tool, not a real contestant.

Must be one of:

  • "inactive"
  • "meta_tool"
  • "ai"
  • "unqualified"
  • "auxiliary"

Type: array of object

Information about the competition participants of the tool.

No Additional Items

Each item of this array must be:

Type: array of string

The categories the tool actually runs and is scored in for this track.
A meta category may be listed only if the tool runs every base
category below it (excluding validation-only categories).
If a tool is supposed to run in individual base categories, these must be listed instead.
For validators, categories are witness-type-agnostic (e.g. C.Overall).

No Additional Items

Each item of this array must be:

Type: string
Must match regular expression: ^(C|Java|Python|SV-LIB|MoXI)\.[a-zA-Z].*$

Type: array of string

Categories in which the tool participates (via inclusion of a covering meta- or base-category
in 'categories'), but for which no individual results should be displayed
in the results report.
The tool will still run those tasks and their results still count toward the
score of any meta category the tool participates in. This never removes tasks
from the set the tool is executed on, but it is purely a display concern.
This applies uniformly to base and meta categories.
For example, it is possible to hide a base category like C.no-data-race.Concurrency,
or to hide a meta-category like C.Overall.
In case a meta-category is hidden, ONLY the meta-category score is hidden. Any non-hidden included subcategories are still displayed.

No Additional Items

Each item of this array must be:

Type: string
Must match regular expression: ^(C|Java|Python|SV-LIB|MoXI)\.[a-zA-Z].*$

Type: array of string

Techniques used by the tool.

No Additional Items

Each item of this array must be:


CEGAR

Type: const

CounterExample-Guided Abstraction Refinement is a model-checking technique
that iteratively refines the abstract model of the transition system by
analyzing spurious counterexamples and learning a more precise abstraction.

Literature:

Specific value: "CEGAR"

CEGIS

Type: const

Counter-Example Guided Inductive Synthesis

Literature:

Specific value: "CEGIS"

Predicate Abstraction

Type: const

Predicate Abstraction is an abstract domain
in which abstract states represent a set of concrete states using predicated from a given set of predicates (called the precision).
The set of predicates is fixed for one abstraction computation, but can chance from location to location.
There are two kinds of predicate abstraction: cartesian predicate abstraction and boolean predicate abstraction.

Literature:

Specific value: "Predicate Abstraction"

Symbolic Execution

Type: const

Symbolic Execution is model-checking technique
in which a symbolic store assigns values to variables,
where the symbolic values can be constrained by a set of path constraints.

Literature:

Specific value: "Symbolic Execution"

Bounded Model Checking

Type: const

Bounded Model Checking (BMC) is a model-checking technique
that unrolls the transition system a finite number of times.

Literature:

Specific value: "Bounded Model Checking"

k-Induction

Type: const

k-Induction generalizes 1-induction.
1-Induction uses as premise that the property holds for the (one) previous object, while
k-induction generalizes it to using as premise that the property holds for the k previous objects.

Literature:

Specific value: "k-Induction"

Property-Directed Reachability

Type: const

Property-Directed Reachability is a model-checking technique
that constructs a sequence of invariants (also called frames)
that describe the reachable states after a number of transitions,
by continuously refining the invariants directed by paths that
reach a target state.

Literature:

Specific value: "Property-Directed Reachability"

Explicit-Value Analysis

Type: const

Explicit-Value Analysis is a model-checking technique
that stores explicit values for program variables,
with similarities to constant-propagation data-flow analyses
and explicit-state model checking.

Literature:

Specific value: "Explicit-Value Analysis"

Numeric Interval Analysis

Type: const

Numeric Interval Analysis is a data-flow analysis
that is based on the abstract domain of intervals,
in which each program variable is mapped to an interval of values.

Literature:

Specific value: "Numeric Interval Analysis"

Shape Analysis

Type: const

Shape Analysis

Specific value: "Shape Analysis"

Separation Logic

Type: const

Separation Logic provides reasoning about the heap,
which makes is particularly useful for verifying memory safety
and other properties that require the analysis of the program heap.

Literature:

Specific value: "Separation Logic"

Bit-Precise Analysis

Type: const

Bit-Precise Analysis is a technique in which the
abstract domain for tracking values treats variables of type int
as bit vectors, instead of approximating them as (unbounded) integers.

Specific value: "Bit-Precise Analysis"

ARG-Based Analysis

Type: const

ARG-Based Analysis is a model-checking technique
that stores successor relations in the set of reached abstract states,
which is particularly useful for constructing error paths
once a violation of the specification is found.

Literature:

Specific value: "ARG-Based Analysis"

Lazy Abstraction

Type: const

Lazy Abstraction is a technique from software model checking
that lazily refines the abstract model only when and where needed,
resulting in abstract models (ARGs) that can have different precisions
on different paths and on different locations on a path.

Literature:

Specific value: "Lazy Abstraction"

Interpolation

Type: const

Interpolation is a technique from model checking
that computes for a two formulas A and C, for which A → C holds,
a new formula B such that A → B → C
and B contains only symbols that occur in A and C.

Literature:

Specific value: "Interpolation"

Automata-Based Analysis

Type: const

Automata-Based Analysis is a technique from software model checking
in which automata are used in the analysis algorithm,
either to represent the abstract state space or the specification.

Literature:

Specific value: "Automata-Based Analysis"

Concurrency Support

Type: const

Concurrency Support comprises techniques
that make it possible to verify concurrent programs,
which use more than one execution thread and shared memory.

Specific value: "Concurrency Support"

Ranking Functions

Type: const

Ranking Functions are used to prove the termination of loops.
A ranking function maps the program's state to a well-founded set,
often the natural numbers, such that every loop iteration of the program
strictly decreases the value of the function.
If a ranking function for a loop can be found,
it guarantees that the loop terminates.

Literature:

Specific value: "Ranking Functions"

Evolutionary Algorithms

Type: const

Evolutionary Algorithms are heuristic optimization algorithms
that are inspired by biological evolution.
They use mechanisms such as mutation, crossover, and selection
to evolve a population of candidate solutions
towards an optimal or near-optimal solution.

Literature:

Specific value: "Evolutionary Algorithms"

Algorithm Selection

Type: const

Algorithm Selection is a technique that
inspects the input and based on information from the input
(for example, by extracting a feature vector) selects one
algorithm (that seems suited for the input)
from a set of given algorithms.

Literature:

Specific value: "Algorithm Selection"

Portfolio

Type: const

Portfolio is a technique from economics
in which the risk is spread over several targets.
Algorithmic portfolios run several algorithms
in parallel or in sequence (while sharing resources),
in the hope that one of the algorithms finds the solution.

Literature:

Specific value: "Portfolio"

Interpolation-Based Model Checking

Type: const

Interpolation-Based Model Checking is a model-checking algorithm from 2003
that uses interpolants to over-approximate sets of reachable states.

Literature:

Specific value: "Interpolation-Based Model Checking"

Logic Synthesis

Type: const

Logic Synthesis

Literature:

Specific value: "Logic Synthesis"

Task Translation

Type: const

Task Translation is a technique
that translates a verification task t into another verification task t'
such that an existing verification technique for t' can be used to derive
a verification result for t.

Literature:

Specific value: "Task Translation"

Floating-Point Arithmetics

Type: const

Floating-Point Arithmetics provides
precise reasoning about floating-point numbers in programs.
For example, expressions in the program can be translated
to SMT formulas using the theory of floating-point arithmetics.

Literature:

Specific value: "Floating-Point Arithmetics"

Guidance by Coverage Measures

Type: const

Guidance by Coverage Measures is a technique
that uses coverage measures to guide the process of verification or test generation.

Specific value: "Guidance by Coverage Measures"

Random Execution

Type: const

Random Execution is a technique for black-box testing
in which input values are randomly generated to construct
a test suite for the system under test.

Specific value: "Random Execution"

Targeted Input Generation

Type: const

Targeted Input Generation is a technique for software testing
that focuses on generating input values
that are likely to trigger specific behaviors or execution paths in the system.

Specific value: "Targeted Input Generation"

Template-Based Predicate Inference

Type: const

Template-Based Predicate Inference is a technique for software verification
that synthesizes predicates to be added to the precision of predicate abstraction
by trying to instantiate predicates (e.g., invariants) from a given set of templates.

Specific value: "Template-Based Predicate Inference"

Deductive Verification

Type: const

Deductive Verification

Specific value: "Deductive Verification"

Abstract Interpretation

Type: const

Abstract Interpretation is a mathematical framework used to define and prove the soundness of abstractions.
In static analysis, it allows one to soundly approximate program semantics using abstract domains,
thereby automatically inferring invariants to verify program properties without executing the program.

Literature:
- 'Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints', Proc. POPL, 1977
- 'Systematic design of program analysis frameworks', Proc. POPL, 1978

Specific value: "Abstract Interpretation"

Fuzzing

Type: const

Fuzzing is a dynamic software testing technique
that automatically generates a large number of random or semi-random inputs
to test programs for bugs, vulnerabilities, and crashes.
It is particularly effective at finding security vulnerabilities and robustness issues.

Literature:
- 'The Fuzzing Book', Online, 2023

Specific value: "Fuzzing"

Constrained Horn Clauses

Type: const

Constrained Horn Clauses (CHC) is a technique to transform transition systems
into Horn clauses with constraints from a background theory and
delegate the actual verification to CHC solvers.

Literature:
- 'Horn Clause Solvers for Program Verification', Proc. Festschrift 75th Birthday Yuri Gurevich

Specific value: "Constrained Horn Clauses"

Type: array of string

Frameworks or solvers used by the tool.

No Additional Items

Each item of this array must be:


CPAchecker

Type: const

CPAchecker

Specific value: "CPAchecker"

CProver

Type: const

CProver

Specific value: "CProver"

ESBMC

Type: const

ESBMC

Specific value: "ESBMC"

JPF

Type: const

JPF

Specific value: "JPF"

Ultimate

Type: const

Ultimate

Specific value: "Ultimate"

Eldarica

Type: const

Eldarica

Specific value: "Eldarica"

Golem

Type: const

Golem

Specific value: "Golem"

JavaSMT

Type: const

JavaSMT

Specific value: "JavaSMT"

MathSAT

Type: const

MathSAT

Specific value: "MathSAT"

CVC

Type: const

CVC

Specific value: "CVC"

CSIsat

Type: const

CSIsat

Specific value: "CSIsat"

SMTinterpol

Type: const

SMTinterpol

Specific value: "SMTinterpol"

Z3

Type: const

Z3

Specific value: "Z3"

PCSat

Type: const

PCSat

Specific value: "PCSat"

PySMT

Type: const

PySMT

Specific value: "PySMT"

MiniSAT

Type: const

MiniSAT

Specific value: "MiniSAT"

Glucose

Type: const

Glucose

Specific value: "Glucose"

Apron

Type: const

Apron

Specific value: "Apron"

Viper

Type: const

Viper

Specific value: "Viper"

LiSA

Type: const

LiSA

Specific value: "LiSA"

AFL++

Type: const

AFL++ (American Fuzzy Lop Plus Plus)

Specific value: "AFL++"

Bitwuzla

Type: const

Bitwuzla

Specific value: "Bitwuzla"

Boolector

Type: const

Boolector

Specific value: "Boolector"

Smt-Switch

Type: const

Smt-Switch

Specific value: "Smt-Switch"

Yices2

Type: const

Yices2

Specific value: "Yices2"

Klee

Type: const

Klee is a symbolic executor of LLVM IR code.

Specific value: "Klee"

Symbiotic

Type: const

Symbiotic is a program analysis framework integrating scripts for compilation and various
3rd party libraries and tools for program slicing, point-to analysis, and verification.

Specific value: "Symbiotic"

Frama-C

Type: const

Frama-C is an open-source, extensible platform dedicated to the source-code analysis of
C software. It integrates several analysis techniques, including abstract interpretation,
deductive verification, and runtime verification.

Specific value: "Frama-C"

Astral

Type: const

Astral is a solver for various fragments of separation logic with inductive definitions.
Its approach is based on a translation to SMT, allowing it to leverage multiple existing
SMT solvers as backends.

Specific value: "Astral"

Goblint

Type: const

Goblint is an abstract interpretation framework that
is dedicated to the sound thread-modular analysis of multi-threaded
programs, especially for data races.

Specific value: "Goblint"

Type: array of object

Actors/components that are used by the tool.

No Additional Items

Each item of this array must be:

Type: object

Type: array of object

Relevant literature references for the tool.

No Additional Items

Each item of this array must be:

Type: object

Type: string
Must match regular expression: 10\.[0-9]{4,9}\/.*

Type: string

Type: integer

Type: array of object

Supplementary material such as competition proceedings, tutorials, and other resources that lack the publication guarantees of peer-reviewed literature.

No Additional Items

Each item of this array must be:


Type: object

The following properties are required:

  • doi
Type: object

The following properties are required:

  • url

Type: string
Must match regular expression: 10\.[0-9]{4,9}\/.*

Type: stringFormat: uri

Type: array of object

Custom licenses used by the tool (applicable if no matching SPDX identifier exists).

No Additional Items

Each item of this array must be:

Type: object

Type: string

Type: string