Nim: Bypass untyped template arguments to C macro

[derekdai]

I'd like to add usdt to my program, but no way to bind <sys/sdt.h> due to untyped C macro arguments.

Description

In the C code below, both test and p1 are untyped to C compiler, which create a probe point at compile time.

/* test.c */
#include <sys/sdt.h>

int main()
{
    DTRACE_PROBE(test, p1);

    return 0;
}

To run it, Install dependencies and build test.c

$ sudo apt install systemtap-sdt-dev
$ gcc -o test test.c

Use bpftrace to detect hitting of probe point

$ sudo apt install bpftrace
$ sudo bpftrace -e 'usdt:/home/derekdai/test:test:p1 { printf("Hello\n"); }'

Every time test runs, bpftrace print Hello.

If nim can accept template in below form, maybe more C macro can be bound.

template DTRACE_PROBE(namespace, name: untyped) {.importc, header: "<sys/sdt.h>".}

DTRACE_PROBE(job1, start)
job1()
DTRACE_PROBE(job1, end)

Alternatives

• <sys/sdt.h> is static way to add user probe points, if solution above is too dirty to implement, the only way to support usdt will be to bind libstapsdt which generate usdt at runtime, but since nim is static and system language, static way is more appropriate in my opinion.
• generate usdt by nim compiler

### Additional Information
-

[timotheecour]

Use emit ?

[alaviss]

emit can be used for quick-n-dirty wrapping, but you can also just translate that macro content into Nim.

[alaviss]

This should work:

{.emit: """
/*INCLUDESECTION*/
#include <sys/sdt.h>
""".}

template DTRACE_PROBE(namespace, name: untyped) =
  {.emit: ["DTRACE_PROBE(", astToStr(namespace), ", ", astToStr(name), ");"].}

proc main() =
  DTRACE_PROBE(a, b)

main()

Quirks:

• The template only works when used inside a proc. This is due to how Nim defaults top-level {.emit.} to put code in a proc declaration section instead of the module initializer. We can probably fix this though.

[timotheecour]

I believe this is one the things that https://github.com/nim-lang/Nim/pull/13953 helps with

[alaviss]

I think the best way for now would be to translate the macro you're wrapping into Nim. I do understand that some macros can be ridiculously complex though, so maybe we can provide a specialized flag which can be used for proc to simplify the process (ie. {.untyped.} which allows proc to take untyped args)?.

[alaviss]

I believe this is one the things that #13953 helps with

I don't think so, more emit hacks are never wise.

[timotheecour]

that comment seems like a copy paste from https://github.com/nim-lang/Nim/pull/13953#issuecomment-613594967 ; I don't see why that's a hack, it allows you to control which C section it goes to, including HERE, which the current emit doesn't allow (see https://github.com/nim-lang/Nim/issues/13943)

[alaviss]

The /*HERE*/ flag can be useful for cases like this, though I'd argue that we should never move them unless a flag is specified, but that'd be breaking change.

The name of the flag makes no sense though. In the perspective of top-level statements, HERE can perfectly be a function declaration since they also happen there as well.

[alaviss]

I'd prefer that {.emit.} slowly disappear to be fair, but forcing everyone to translate the all kinds of C macros into Nim might be a bit far-fetched.

[timotheecour]

The name of the flag makes no sense though

alternative name suggestions welcome, but it conveys that the code is emitted at where the emit appears (or at least at beginning of proc) instead of moved to a top level section, so I think the name reflects this well.

See example usages here https://github.com/nim-lang/Nim/pull/13953/files#diff-b549242434b5227b402f18f17c4752ec

I'd prefer that {.emit.} slowly disappear to be fair

emit is a necessary evil; you should only use it when other approaches don't work. But for things like C function macros or anything that's not a function call, it remains awefully useful and practical, and duplicating macro content is often undesirable, unportable, or impossible (eg, the stringfification, c_sizeof to verify sizeof, {.emit: "throw;".} etc). Just use it with caution that's it.

Shifting this discussion to dtrace, and regardless whether this issue can be closed, I think we could start offering dtrace-specific apis to make it easier to use in nim; it's an incredibly powerful tool for debugging (performance, counters, logic etc). Allows tracing running applications not written with dtrace in mind, or custom tracing of applications with dtrace hooks that can be changed at runtime.

All with zero overhead if you're not tracing, so it's easy to switch on/off. As an example, nim binary itself could be instrumented to have counters for various things (number of allocations, number of calls to key functions, vm calls etc)

[alaviss]

Shifting this discussion to dtrace, and regardless whether this issue can be closed, I think we could start offering dtrace-specific apis to make it easier to use in nim; it's an incredibly powerful tool for debugging (performance, counters, logic etc). Allows tracing running applications not written with dtrace in mind, or custom tracing of applications with dtrace hooks that can be changed at runtime.

Sounds like it could be a library

[timotheecour]

yes; the D language (not the dlang one ;-)) could potentially be generated from nim or some nim DSL too.

[derekdai]

Hope nim compiler can support usdt. I will close this issue because @alaviss solve my problem, thank you!