Crystal: Add file-private `require`

[asoffa]

Hi,

Suppose I have a file that requires a large number of resources that are used for its internal behavior, e.g.

# in file sophisticated.cr:
require "a"
require "b"
...
require "z"

module Sophisticated
...
end

Now, any file that requires "sophisticated" will inherit the contents of all of the files a.cr, ..., z.cr that sophisticated.cr requires for its implementation and any requires within a.cr, ..., z.cr recursively. This can lead to code pollution, especially when there are modules, classes, etc. with similar names (and even redefinitions in the worst case if there are identical names). Therefore, it would be nice to have a new keyword (e.g. use) or some sort of marker for require (e.g. private require) so that the implementation details within sophisticated.cr are not leaked into files that merely use sophisticated.cr, e.g.

# in file sophisticated.cr:

# contents of a.cr, ..., z.cr now exposed to sophisticated.cr, but
# not to files that require/use it
use "a"
use "b"
...
use "z"

module Sophisticated
...
end

On the other hand, the leaking behavior is sometimes desirable for e.g. splitting up a module across multiple files, so it would be nice to have both the current behavior of require and this proposed addition.

This would be along the same lines as #3280, but for requiring files instead of defining modules, classes, constants, methods, etc.

Thoughts?

[RX14]

Duplicate of https://github.com/crystal-lang/crystal/issues/140. I don't think the core team's opinions have changed much.

[asoffa]

Hmm, it seems that the main issue mentioned in #140 was (from @asterite's comment):
"What @Nami-Doc says is the main reason Ruby doesn't implement this: monkey-patching. In any file you can reopen any other class. What would a file that reopen a class return?"

The use system I am proposing here would be identical to require, except that it wouldn't be recursive. For the case of a file reopening a class, it would behave as if the file were required for any file that uses it and not required for any file that doesn't use it (and no change from the current behavior if the file is required instead of used).

The same monkey-patching "issue" (really a feature IMHO) also arises with require in a project such as the following:

# in file first.cr:
module A
  def self.first
    "first"
  end
end

# in file second.cr:
module A
  def self.second
    "second"
  end
end

# in file test.cr:
require "first"
A.first   #=> "first"
A.second  # error: doesn't work w/o `require "second"`

So the full A module isn't available in all cases anyway.

Now suppose a file required by another file required by yet another file that first.cr requires is updated so that it happens to require second.cr. Now the code in test.cr "mysteriously" works, and there's no way I can prevent it from working, even if I don't want A.second to be visible without the user explicitly asking for it with require/use

[asoffa]

Another issue: the order of requires. Suppose there's a library with

# in file first.cr
module A
  def self.first
    "first"
  end
end

# in file another_first.cr
module A
  def self.first
    "another_first"
  end
end

and the library user uses two files dependency1.cr and dependency2.cr that require files that require files that require first.cr and another_first.cr, respectively:

# in test.cr
require "dependency1"
require "dependency2"

Dependency1.do_your_thing_that_uses_first_very_indirectly  # "another_first" used

Now the user merely switches the order of the require statements:

# in test.cr
require "dependency2"
require "dependency1"

Dependency1.do_your_thing_that_uses_first_very_indirectly  # "first" used

This then leaves the user scratching his/her head, wondering why merely switching the order of the require statements caused completely different behavior that had seemingly nothing to do with A.first at all. (Perhaps the user didn't even know that module A existed behind the scenes.)

The above examples are perhaps a bit contrived, but hopefully illustrate why it would be useful to have require in its current form not be the only option for using code from another file.

And what if dependency1 and dependency2 above need to use different versions of the same Crystal library? Crystal does not allow this if restricted to the current require system

[ozra]

tl;dr - but I think this is about a feature I'd die for to have in Crystal, I'll just add my wording, and then you shoot me if it was straying from the subject.

• I'd want require for requiring files (duh! we've got it).
• One can currently include a module in the current module / "namespace". That's good _sometimes_ - but:
• What I'd really really would like to have is using similar to in C++, that should be looked-up in _only within the scope, the using-statement is in_.

It would simply add the module namespace to "naked name lookups"-list within whatever narrow scope I choose (_for instance maybe within one branch of an if). That/those module(s) should then be removed from lookup-list when that scope ends. Needless to say then: _without including_ it in mine, and _polluting the namespace_ of my module. (If the user of my module also wants the features of a module mine is also using - they'll just include it for using, or even module-inclusion, themselves.

I don't want to expose 3rd party modules just to be able to use them without namespace qualification.

If this is already possible, and I totally missed something here, I've been out of the loop to long, and I'm then also _very happy_. Please say it's so! :)

[asoffa]

@ozra What you propose above (a scope-private extend/include) complements extend/include, whereas what I propose above (a file-private require) complements require. I agree that both would be useful

[asoffa]

@ozra By the way, what you propose above is (sort of) available in the following form:

module SomeReallyLongModuleName
  def self.say_hi
    puts "hi"
  end
end

module UsingSomeReallyLongModuleName
  # almost like the `using SomeReallyLongModuleName` that you propose:
  private alias S = SomeReallyLongModuleName

  def self.call_say_hi
    S.say_hi  # still need to prefix with `S.`, but this isn't nearly as cumbersome as
              # prefixing with `SomeReallyLongModuleName.`, especially if you have a
              # lot of calls to methods in `SomeReallyLongModuleName`
              # ...and it keeps clear where the `say_hi` method is coming from
  end
end

Now, if only something analogous were available with requiring files so that the global namespace didn't get polluted... :-)

[ozra]

@asoffa - thanks for the sharing of tricks - I feel it's "all or nothing" though :)

[asoffa]

4439 (adding import hooks that allow a file to return a value such as a private class instance) proposes a solution to encapsulating implementation details by allowing the use of private values across files.

Discussion of the using proposal mentioned above is in #3319

[ysbaddaden]

Closing as duplicate of #140. You may voice yourself in closed issues if you really have to, but plase don't duplicate issues.