On 8 December 2010 12:57, Dave Martin dave.martin@linaro.org wrote:
On Wed, Dec 8, 2010 at 12:04 PM, Peter Maydell peter.maydell@linaro.org wrote:
Hi. As part of my work on qemu I've written a simplistic random instruction sequence generator and test harness.
I'd thought about this sort of thing before, but there were some interesting challenges...
Is it random random, or does it try to mould to the architectural envelope?
For example, randomly choosing opcode and shift operation bits in ALU instructions will exercise the interpreter much more interestingly than randomly choosing the offset bits in a branch instruction; and some potentially highly interesting and error-prone instructions like barriers, WFE/SEV, and all that kind of thing will get basically no testing from a random-number generator, since they tend to map to a single, fixed bit pattern.
As I say, it's pretty simplistic: I was aiming for a "good enough" tool that would let me find and test qemu bugs. The generator works by: * pick a pattern at random from all the ones it's using * generate a random 32 (or 16) bits * mask in the should-be-zero/one bits * apply constraints (eg regs never r13 or r15, any user-specified constraint); if the constraint is broken then just restart with a new random number * force the cond field to "always" a configurable proportion of the time
So the effect is that the testing will be distributed evenly over the given instructions even if some take a much smaller share of encoding space.
However for other reasons the tool isn't really suited for testing weird things like WFE, SEV or barriers -- there's a disclaimer about this in the README. I think for that sort of thing you'd do much better with a rather more directed kind of testing.
-- PMM