Benchmarkdotnet: Diff view for disassembler output

[adamsitnik]

From private conversation with @JosephTremoulet

trying to replicate (in some fashion) the various rewrites we apply under COMPlus_JitDiffableDisasm, so that a straightforward textual diff identifies significant code changes and not simply things getting loaded at different addresses, etc.
some sort of diff view integrated into the html like code reviews on GitHub etc.

Imho we should print the disasm in some smarter way and use some js library for having nice visualization of the diff

[rolshevsky]

@adamsitnik @AndreyAkinshin Hello,
Can you guys please assign me on this issue? I'm gonna try to implement it.

[adamsitnik]

@rolshevsky awesome!

@AndreyAkinshin could you assign @rolshevsky ?

[AndreyAkinshin]

@rolshevsky, I added you to the collaborators list. Please, accept the invitation, then I will be able to assign.

[AndreyAkinshin]

@rolshevsky, done.

[rolshevsky]

@adamsitnik @AndreyAkinshin, thanks.

[rolshevsky]

Hi @adamsitnik,

Currently our Asm class contains text representation of asm instruction with instruction address:

0517180e 40              inc     eax
0517180f b964000000      mov     ecx,64h 

Is it possible to expand this class with additional property which will be contain instruction without address?

inc     eax
mov     ecx,64h

Using this we can implement 2 modes for showing diffs in our DiffView (with asm instruction address and without).

What do you think about it? Does it make sense?

[adamsitnik]

@rolshevsky it's a very good idea!

I believe that we should implement #546 first. So the view would be even more user friendly. Example:

HasFlagBench.Bench.HasFlag():
LO:
    xor     eax,eax
    mov     edx,dword ptr [rcx+8]
L1:
    mov     r8d,edx
    and     r8d,0Ch
    cmp     r8d,0Ch
    sete    r8b
    mov     byte ptr [rcx+0Ch],r8b
    inc     eax
    cmp     eax,3E8h
    jl      L1
L2:
    ret

[adamsitnik]

@rolshevsky I implemented #546. Please take a look at DisassemblyPrettifier.Prettify. It looks much better now and you should be able to compare same asm even if addresses change.

[adamsitnik]

No updates from @rolshevsky for almost a year, I make the issue up-for-grabs again.

[adamsitnik]

@AndreyAkinshin could you please assign @wojtpl2 to this task?

[AndreyAkinshin]

@wojtpl2, I added you to the collaborator list, you should get an invitation letter.

[WojciechNagorski]

Hi,
Can I use this code https://github.com/google/diff-match-patch/blob/master/csharp/DiffMatchPatch.cs

/*

• Diff Match and Patch
• Copyright 2018 The diff-match-patch Authors.
• https://github.com/google/diff-match-patch
  *
• Licensed under the Apache License, Version 2.0 (the "License");
• you may not use this file except in compliance with the License.
• You may obtain a copy of the License at
  *
• http://www.apache.org/licenses/LICENSE-2.0
  *
• Unless required by applicable law or agreed to in writing, software
• distributed under the License is distributed on an "AS IS" BASIS,
• WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
• See the License for the specific language governing permissions and
• limitations under the License.
  */

I did research and did not find anything better for .Net with the MIT license.

Of course I can implement the Diff algorithm myself. What do you think? What I should do?

[adamsitnik]

@wojtpl2 maybe we could youse git diff?

```log
C:Usersadsitnik>git diff -?
Not a git repository
To compare two paths outside a working tree:
usage: git diff [--no-index]
````

[WojciechNagorski]

Ok, I tried and looks promising;)

[WojciechNagorski]

Probably I found a bug.

When do you save `-asm.pretty.md' file, you generate the name in this way: https://github.com/dotnet/BenchmarkDotNet/blob/1e62355f209a25c7a33f9ab7e7e03b0afe7d851f/src/BenchmarkDotNet/Exporters/PrettyGithubMarkdownDisassemblyExporter.cs#L29

e.g.:
BenchmarkDotNet.Samples.IntroDisassemblyAllJits_CallVirtualMethod_ShortRun-asm.pretty.md

But IntroDisassemblyAllJits.CallVirtualMethod test is running for Multiple JIT. This file is overwritten for each JIT.

https://github.com/dotnet/BenchmarkDotNet/blob/e8bf99272bc410601863bda338b57d0c46baad70/samples/BenchmarkDotNet.Samples/IntroDisassemblyAllJits.cs#L11

Maybe BDN should generate file names like this:

BenchmarkDotNet.Samples.IntroDisassemblyAllJits_CallVirtualMethod_ShortRun_Platform-X64_Runtime-MonoRuntime_IterationCount-3_LaunchCount-1_WarmupCount-3-asm.pretty.md
BenchmarkDotNet.Samples.IntroDisassemblyAllJits_CallVirtualMethod_ShortRun_Platform-X86_Runtime-MonoRuntime_IterationCount-3_LaunchCount-1_WarmupCount-3-asm.pretty.md
BenchmarkDotNet.Samples.IntroDisassemblyAllJits_CallVirtualMethod_ShortRun_Jit-LegacyJit_Platform-X64_Runtime-ClrRuntime_IterationCount-3_LaunchCount-1_WarmupCount-3-asm.pretty.md
BenchmarkDotNet.Samples.IntroDisassemblyAllJits_CallVirtualMethod_ShortRun_Jit-LegacyJit_Platform-X86_Runtime-ClrRuntime_IterationCount-3_LaunchCount-1_WarmupCount-3-asm.pretty.md
BenchmarkDotNet.Samples.IntroDisassemblyAllJits_CallVirtualMethod_ShortRun_Jit-RyuJit_Platform-X64_Runtime-ClrRuntime_IterationCount-3_LaunchCount-1_WarmupCount-3-asm.pretty.md
BenchmarkDotNet.Samples.IntroDisassemblyAllJits_CallVirtualMethod_ShortRun_Jit-RyuJit_Platform-X64_Runtime-CoreRuntime_Toolchain-CsProjCoreToolchain_IterationCount-3_LaunchCount-1_WarmupCount-3-asm.pretty.md
BenchmarkDotNet.Samples.IntroDisassemblyAllJits_CallVirtualMethod_ShortRun_Jit-RyuJit_Platform-X64_Runtime-CoreRuntime_Toolchain-CsProjCoreToolchain_IterationCount-3_LaunchCount-1_WarmupCount-3-asm.pretty.md

But I know that the name is too long. What do you think?

[WojciechNagorski]

And I have question about the names of files with diff.
As I wrote above IntroDisassemblyAllJits.CallVirtualMethod test generate 7 different disassemblies. It gives 21 combinations. :dizzy_face:

Maybe we should create a separate directory for each test?

[WojciechNagorski]

I send #927 PR. I came up with my solution to the problem.