Scryer-prolog: Benchmarks

[rusterize]

I get the following error just doing cargo build.

   Compiling rusty-wam v0.7.6 (file:///tmp/rusty-wam)
error[E0583]: file not found for module `parser`
  --> src/prolog/mod.rs:25:9
   |
25 | pub mod parser;
   |         ^^^^^^
   |
   = help: name the file either parser.rs or parser/mod.rs inside the directory "src/p
rolog"

error: aborting due to previous error

error: Could not compile `rusty-wam`.

[mthom]

Did you download the parser submodule when you cloned the REPL? It should be cloned with the --recursive flag, so that the parser submodule is downloaded, like this:

$> git clone http://github.com/mthom/rusty-wam --recursive

[rusterize]

ah, I didn't notice that. Thanks.
BTW, why did you make the parser a submodule? (just interested to learn the benefit you saw)

[mthom]

Good question, haha. No reason, really. I felt some guilt over copying the BinProlog parser source wholesome to get the parser I have, although it's also open source, and copying it is within the terms of its license. I felt the need to call attention to that by publishing it as a separate git module, I guess?

[rusterize]

Ahh I see. I would like to run some benchmarks on rusty-wam, can you suggest any common swi prolog/sicstus benchmarks?

[mthom]

@triska may have some suggestions for benchmark programs, he's the resident Prolog expert. I'm still a dilettante.

[rusterize]

I see, I'll look around a little while I wait for Triska's answer. BTW, I have been reading his http://metalevel.at tutorials and they are the best I have found on using prolog properly.

[mthom]

Yeah, he definitely knows his stuff. I've had a lot of fun reading it.

[rusterize]

Just found the sicstus benchmarks on their website, I'll try to use them with rusty-wam. https://sicstus.sics.se/performance.html

[mthom]

A few predicates used by the harness (like cputime) aren't supported yet, just so you're aware. There are ways around that in the shell, of course.

[rusterize]

hmm. It can't read files yet right? I'll try piping them through stdin

[mthom]

It can't read files directly yet, that's right.

[rusterize]

hmm..I get a run of UnexpectedEOF when I pipe through stdin.....

[mthom]

I'll have a look at that tomorrow. It's getting late.

[rusterize]

ok, thanks!

[rusterize]

Just for a test, I tried this:

data([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23, 24,25,26,27,28,29,30]).
nreverse([], []).
nreverse([X|L0], L) :- nreverse(L0, L1), concatenate(L1, [X], L).
concatenate([], L, L).
concatenate([X|L1], L2, [X|L3]) :-  concatenate(L1, L2, L3).
?- data(D), nreverse(D,Out).

It fails with false...probably [H|T] is not implemented? I'll play around some more.

Thanks for your work!

[rusterize]

Was able to narrow down the problem:
The following succeds as it should:

prolog> c([], L, L).
prolog> ?- c([],[1,2,3],L).
true.
L = [1, 2, 3].

But the following fails:

prolog> c([], L, L).
prolog> c([X|L1], L2, [X|L3]) :-        c(L1, L2, L3).
prolog> ?- c([],[1,2,3],L).
false.

[triska]

Thank you for the kind words!

The "van Roy" set of benchmarks indeed have long been an important reference point for comparing the performance of Prolog systems. They are also available for SWI-Prolog:

https://github.com/SWI-Prolog/bench

Neng-Fa Zhou, the author of B-Prolog, kindly maintains a comparison of different systems at:

http://www.picat-lang.org/bprolog/performance.htm

These benchmarks are definitely very useful to detect weak spots in your VM. Still, I recommend to not get carried away with optimizing for this particular set of benchmarks: There are much more important improvements such as:

• JIT indexing can turn an O(N) lookup in traditional systems into an O(1) lookup
• integer constraints can turn permutation sort from an O(N!) algorithm into a polynomial one
• tabling can turn a program that does not terminate in traditional systems into a terminating one
• attributed variables allow you to turn impure I/O into pure and efficient I/O (phrase_from_file/2)
• etc.

None of these aspects is covered in any way by the benchmarks, and if you optimize your VM for this set, you may make it hard or error-prone to implement some of these features.

[mthom]

In order for multi-line inputs to work, they must be contained in braces: :{ / :} for inputs of a single predicate, :{{ \ :}} for multiple predicates, like so:

prolog> :{
c([], L, L).
c([X|L1], L2, [X|L3]) :- c(L1, L2, L3).
}:

With the previous attempt at inputting this predicate, you are in fact giving two distinct definitions of c/3, where the second overwrites the first. This is also what prevents piping a file in from the shell from working.

This convention is temporary until I can begin using rusty-wam from emacs.

[rusterize]

@triska Thank you so much for your answer, very useful! I encourage you to please write more at metalevel.at, I don't think there is anyone writing anything similar to your tutorials, which are so important to use prolog properly.

@mthom thanks for pointing out the ':{ :}' issue, I'll try with that