Kotlin-native: [Feature Request] Get rid of alloc and memscoped

[RUSshy]

Hello

In SWIFT working with C stuff is totally transparent, there is no need to do stuff like that:

memscoped {} or native.alloc

Would be cool to not have to do that

Thanks

[Techcable]

How are you suggesting native memory should be managed and freed?
Swift uses reference counting, but kotlin isn't guaranteed to, and may use a garbage collector instead.
In order to free memory deterministically, kotlin must either guarantee reference counting, (breaking compatibility with java and javascript), or use a specialized syntax like this.
Leaving native memory completely up to the control of the garbage collector isn't acceptable since it won't necessarily free it fast enough, like with ByteBuffer direct memory.

[balthild]

I'm interested in what compatibility referenced to and how reference counting break compatibility.

[Techcable]

@balthild
Code that uses reference counting frees all resources and memory deterministically, without having to wait for the garbage collector to clean up and finalize resources.
Whenever a new object is allocated, it's freed as soon a possible without having to wait for a GC cycle to start.
Consequently, if you have an open resource like a FileInputStream, it'll be closed automatically once the method finishes, without having to invoke close manually.
Although this may seem like a good thing, once code starts relying on this behavior it's impossible to switch to traditional garbage collection later, without breaking the assumption that resources will be closed automatically.

One historical example is the python interpreter, whose original implementation (cpython) uses reference counting, so code like open('example.txt').write("hello world") will automatically close the resource.
For years, code was written upon this assumption, and worked perfectly fine.
Eventually, the pypy JIT compiler was released, which used a tracing garbage collector, since the high overhead of reference counting wasn't considered acceptable in the faster JIT compiled code.
Code that previously worked fine, and assumed automatic closing of resources would leak file descriptors when run on pypy.
Eventually, the python team introduced a new 'with' syntax that always guaranteed deterministic closing of resources regardless of the implementation, similar to Java's try with resources.
Unfortunately, python's native API was so fundamentally bound to the reference counting implementation, that pypy couldn't provide compatibility with it.
Users had to choose between's pypy's fast JIT compiler and tracing garbage collector, or the large volume of existing code bound to cpython's reference counting system.

I think that kotlin native should keep reference counting as an implementation detail, both to allow it to switch to a tracing garbage collector in the future, and so that code written for kotlin native will also work on the JVM and in javascript, which don't use reference counting.

[olonho]

Could you please provide an example of working non-leaking Swift code interoping with C API that needs memory allocations showing scope of your proposal?

[RUSshy]

for example:

public class Window
{
    public var ptr: OpaquePointer

    init(mode: VideoMode, title: String, style: VideoStyle)
    {
        ptr = sfWindow_create(mode.ptr, title, style.rawValue, nil)
    }

    deinit
    {
        sfWindow_destroy(ptr)
    }
}

I thought for Arc it doesn't need manual memory management ?

[balthild]

@Techcable Thanks for the explanation!

[RUSshy]

oh, well i am not expert, i didn't know the detail sorry, so i guess i should close this ? ^^

[napperley]

I thought for Arc it doesn't need manual memory management ?

Had the same understanding for Arc that it is a form of automatic memory mangement according to this answer. Is that not the case? Biggest downside to ARC from what I have learned is that you need to carefully avoid cyclic references through management of weak references.

Are lambdas in Kotlin weak referenced provided a reference to them isn't stored in a constant or variable?

[olonho]

For memory management whenever automated reference counting is feasible we do that. Also we do collect cyclic garbage. So you may think of Kotlin/Native the same way as Kotlin/JS and Kotlin/JVM.

The only place where semi-automatic memory management provided with memScoped {} is relevant is cases, where we need to allocate objects in C heap, pass to the C API and release where by semantics of C API object is no longer used. Not sure if Swift, or any other language could do anything better than that, as C doesn't allow fully automated memory management.

[msink]

This really should be "try with resources" - ma

[msink]

... malloc can return null.
Когда память кончилась.