For my lexer (tokenizer) all of the ASCII 7-bit characters (0x00 to 0x7F) have a specific token. As SWI-Prolog supports Unicode, the character codes go from 0x0000 to 0xFFFF.
In my lexer, since there are many characters that will not map to a specific token there is an unknown token (tokUnknown).
To ensure that all of the characters with code between 0 and 127 (0x00 to 0x7F) do not have tokUnknown, test cases are needed.
The test case needs a simple lexer to convert the character to a token.
tokenizer_unknown(Token) -->
(
white_space_char(W), !, white_space(W, S),
{ Token = tokWhitespace(S) }
;
[S],
{ special_character(S,Token) }
;
digit(D), !, number(D, N),
{ Token = tokNumber(N) }
;
letter(L), !, word(L, W),
{ Token = tokWord(W) }
;
[_],
{ Token = tokUnknown }
), !.
Here is the test case for the character with code 0.
:- begin_tests(unknown).
test(001) :-
Code = 0,
char_code(Char,Code),
Chars = [Char],
phrase(tokenizer_unknown(Token),Chars,Rest),
assertion(Rest == []),
assertion(Token \== tokUnknown).
:- end_tests(unknown).
Writing test in this manner requires 128 different test to check for tokUnknown.
SWI-Prolog unit testing library plunit has an option forall to generate data.
Based on the documentation the test should look like this:
test(002, [forall(???)]) :-
char_code(Char,Code),
Chars = [Char],
phrase(tokenizer_unknown(Token),Chars,Rest),
assertion(Rest == []),
assertion(Token \== tokUnknown).
Can the forall option be used to write just one test case instead of 128 individual test cases for this test series?
Can you give a working version of the test case using forall?
Follow-up
The template for forall is forall(:Generator).
When I first saw this I was totally confused and almost walked away going back to writing large numbers of test, but stuck with it knowing how valuable and easily this should be for making parameterized tests, e.g. JUnit 5 or NUnit 3. Parameterized test can then be used for fuzzing and fuzzing can be enhanced to generate counter examples, e.g. QuickCheck, FsCheck
Example 1
In the hard coded test
test(001) :-
Code = 0,
char_code(Char,Code),
Chars = [Char],
phrase(tokenizer_unknown(Token),Chars,Rest),
assertion(Rest == []),
assertion(Token \== tokUnknown).
I wanted to make Code a variable that changed for each test. I also knew the constraints for Code, i.e. 0 to 127.
So for this simple generator all that was needed was a predicate that generated values from 0 to 127 when called and returned them as a variable, e.g. Code.
between/3 fills the requirement, e.g.
?- between(0,3,Code).
Code = 0 ;
Code = 1 ;
Code = 2 ;
Code = 3.
As can be seen by looking at the answer, just give the predicate to forall, e.g.
forall(between(0, 127, Code))
Example 2
This test is to check that all individual whitespace characters or sequence of whitespace characters for ASCII 7-bit characters return as tokWhitespace and that the whitespace character(s) are a string value of the token.
The custom with whitespace tokens is not to include the characters in the token, but here they are included because it is easier to remove them if needed then wonder why the OP didn't do it. Since this is for learning they are included.
Hard coded tests
:- begin_tests(white_space).
test(001) :-
String = "\t",
string_codes(String,Codes),
phrase(whitespace(Tokens),Codes,Rest),
assertion(Tokens == tokWhitespace("\t")),
assertion(Rest == []).
test(011) :-
String = "\t\r",
string_codes(String,Codes),
phrase(whitespace(Tokens),Codes,Rest),
assertion(Tokens == tokWhitespace("\t\r")),
assertion(Rest == []).
test(043) :-
String = "\s\s\s",
string_codes(String,Codes),
phrase(whitespace(Tokens),Codes,Rest),
assertion(Tokens == tokWhitespace("\s\s\s")),
assertion(Rest == []).
:- end_tests(white_space).
In this example the variables are String, e.g. "\t", and the value in the token tokWhitespace, e.g. "\t".
The single whitespace characters are:
?- code_type(Char,space).
Char = 9 ; % tokHorizontalTab \t
Char = 10 ; % tokLineFeed \n
Char = 11 ; % tokVerticalTab \v
Char = 12 ; % tokFormFeed \f
Char = 13 ; % tokCarriageReturn \r
Char = 32 ; % tokSpace \s
Char = 160 ; % Yes, there are space characters defined beyond 7-bit ASCII. See: https://en.wikipedia.org/wiki/Whitespace_character#Unicode
Char = 5760 ;
...
One lesson learned from decades of writing lexer/tokenization tests is that each individual character needs to be tested. Also the test should not generate the values in the same manner as the check in lexer/tokenizer. In this case the test should not rely on code_type/2 because that is used in the lexer/tokenizer and if code_type/2 where some how to get a bug the tests would not detect it. So the test cases will get the characters via a different means, in this example they will be from a list.
A second lesson learned from decades of testing recursive code is that the test needs to test at least three levels deep. In this example the test of whitespace characters will test sequences up to three characters.
A third lesson is that using functional composition with functions such as combinations, permutations, type constructors, and type destructors, etc. reduce the combinatorial explosion of writing test data generators; conversely they contribute to combinatorial explosion of test cases. To do this in Prolog requires translating functional concepts to Prolog predicates.
Based on these lessons some helper predicates are needed.
comb(0,_,[]).
comb(N,[X|T],[X|Comb]) :-
N>0,
N1 is N-1,
comb(N1,T,Comb).
comb(N,[_|T],Comb) :-
N>0,
comb(N,T,Comb).
variation_string(N,L,String) :-
between(1,N,N0),
comb(N0,L,L1),
permutation(L1,L2),
string_chars(String,L2).
variation_number(N,L,String,Number) :-
between(1,N,N0),
comb(N0,L,L1),
permutation(L1,L2),
string_chars(String,L2),
number_chars(Number,L2).
Example usage:
?- variation_string(3,['\t','\r','\n'],String).
String = "\t" ;
String = "\r" ;
String = "\n" ;
String = "\t\r" ;
String = "\r\t" ;
String = "\t\n" ;
String = "\n\t" ;
String = "\r\n" ;
String = "\n\r" ;
String = "\t\r\n" ;
String = "\t\n\r" ;
String = "\r\t\n" ;
String = "\r\n\t" ;
String = "\n\t\r" ;
String = "\n\r\t" ;
false.
To keep the reading of forall simpler a helper predicate is created.
generator_ascii_7bit_char_type_white(R) :-
variation_string(3,['\t','\n','\v','\f','\r','\s'],R).
Now to simply use the generator with forall in the test.
:- begin_tests(white_space).
test(000, [forall(generator_ascii_7bit_char_type_white(String))]) :-
string_codes(String,Codes),
phrase(whitespace(Tokens),Codes,Rest),
assertion(Tokens == tokWhitespace(String)),
assertion(Rest == []).
:- end_tests(white_space).
With such little code all these test were created and run (each dot represents a separate test case).
% PL-Unit: white_space ............................................................................................................................................................ done
Example 3
This example test a non-deterministic predicate so needs to use findall. This also has two input parameters and two output parameters for the predicate.
The signature of findall/3 is
findall(+Template, :Goal, -Bag)
To use two values with finall/3 the Template is not a tuple, e.g. (A,B), but a list, e.g. [A,B], and the Bag is a list of list, e.g. [["1",1],["2",2]] where each item in the list is a result and items in the inner list are the values for corresponding Template parameters.
This example test variation_number/4
:- begin_tests(variation_number_4).
variation_number_4(0,[],[]).
variation_number_4(1,[],[]).
variation_number_4(2,[],[]).
variation_number_4(3,[],[]).
variation_number_4(0,['1'],[]).
variation_number_4(1,['1'],[["1",1]]).
variation_number_4(2,['1'],[["1",1]]).
variation_number_4(3,['1'],[["1",1]]).
variation_number_4(0,['1','2'],[]).
variation_number_4(1,['1','2'],[["1",1],["2",2]]).
variation_number_4(2,['1','2'],[["1",1],["2",2],["12",12],["21",21]]).
variation_number_4(3,['1','2'],[["1",1],["2",2],["12",12],["21",21]]).
variation_number_4(0,['1','2','3'],[]).
variation_number_4(1,['1','2','3'],[["1",1],["2",2],["3",3]]).
variation_number_4(2,['1','2','3'],[["1",1],["2",2],["3",3],["12",12],["21",21],["13",13],["31",31],["23",23],["32",32]]).
variation_number_4(3,['1','2','3'],[["1",1],["2",2],["3",3],["12",12],["21",21],["13",13],["31",31],["23",23],["32",32],["123",123],["132",132],["213",213],["231",231],["312",312],["321",321]]).
variation_number_4(0,['1','2','3','4'],[]).
variation_number_4(1,['1','2','3','4'],[["1",1],["2",2],["3",3],["4",4]]).
variation_number_4(2,['1','2','3','4'],[["1",1],["2",2],["3",3],["4",4],["12",12],["21",21],["13",13],["31",31],["14",14],["41",41],["23",23],["32",32],["24",24],["42",42],["34",34],["43",43]]).
variation_number_4(3,['1','2','3','4'],[["1",1],["2",2],["3",3],["4",4],["12",12],["21",21],["13",13],["31",31],["14",14],["41",41],["23",23],["32",32],["24",24],["42",42],["34",34],["43",43],["123",123],["132",132],["213",213],["231",231],["312",312],["321",321],["124",124],["142",142],["214",214],["241",241],["412",412],["421",421],["134",134],["143",143],["314",314],["341",341],["413",413],["431",431],["234",234],["243",243],["324",324],["342",342],["423",423],["432",432]]).
test(000, forall(variation_number_4(Len,L,R0s))) :-
findall([R,N],variation_number(Len,L,R,N),Rs),
assertion(Rs == R0s).
:- end_tests(variation_number_4).
