Tiddlywiki5: Range operator

[kookma]

The range operator seems have issues. I reproduced the bugs in attached tid file which can be tested on https://tiddlywiki.com/prerelease/
It works wrongly for negative steps and when begin is smaller than stop.
Test Range Operator.zip

Some examples are given below:

Example 4
Seems to work as expected.
The <step> is not given! <begin> is bigger than <end>, so it seems the -1 is assumed as <step>. Other language like python return empty list!!

<$list filter="[range[10,0]]" variable="x">
<<x>>
</$list>

That renders as:

10 9 8 7 6 5 4 3 2 1 0

Example 5
Seems to work Wrongly.
The <step> is negative! <begin> is smaller than , so it should return empty list.

<$list filter="[range[1,10,-2]]" variable="x">
<<x>>
</$list>

That renders as:

1 3 5 7 9

Example 6
Seems to work Wrongly.
The <step> is positive! <begin> is bigger than <end>, so it should return empty list.

<$list filter="[range[10,0,2]]" variable="x">
<<x>>
</$list>

That renders as:

10 8 6 4 2 0

[Jermolene]

Hi @kookma thanks!

I think that this behaviour is by design. Is that right @EvanBalster? (See #3346)

[EvanBalster]

Hello, @kookma, @Jermolene

The range operator was written for counting tasks, so step size is automated and the range is inclusive. I actually feel the current behavior of the one-argument range[N] operator is a mistake, for this reason; in most documentation-oriented tasks, it should start at 1 rather than zero.

This "layman's range" counts up or down automatically rather than returning an empty range, because range[10,1] is the most intuitive way to write a down-count. Extending that, range[1000,0,10] is meant to mean "count down by tens". I don't disallow a negative increment (perhaps I should) and these work just like positive increments. I may add support for separators to layman's range.

I have been thinking about making a separate "programmer's range" operator (perhaps rangex or iterate or something) that would function more in line with your expectations. This type of range would be zero-indexed, exclusive at the end of the range and would respect the sign of the increment.

My main source of hesitation regarding these changes has been that some TiddlyWiki users have already incorporated my original range operator, and the change from starting at 0 by default to starting at 1 by default might be a little disruptive. Better to do that before it goes into a release, though, I suppose!

@Jermolene, seeking your guidance on next steps.

[Jermolene]

Many thanks @EvanBalster. I'm inclined to leave the range operator as it is because it's been in the prerelease for quite a while.

@kookma can you work with the current behaviour?

[EvanBalster]

Hey, @Jermolene —

To be clear, the pull requests I'm considering are:

• range[] with one parameter starts from 1 instead of 0
  -- (I think this is what wiki users will want 95% of the time)
• Extend range to handle decimal separators (preferably in a locale-independent fashion)
• Add "programmer's range" as a new operator

Of the three, the first is the only one with a possibility of being disruptive to wikis built on prerelease. But I speculate that the impact would be low.

[Sorry if this is getting a bit tangental to the issue.]

[BurningTreeC]

Hi @EvanBalster , @Jermolene , for me a change

[BurningTreeC]

... would be no big issue

[Jermolene]

Hi @EvanBalster @BurningTreeC OK let's switch it to 1-based counting -- I'm hoping to release v5.1.18 tomorrow or Monday, so we need to be quick :smile:

[EvanBalster]

@Jermolene Here it is: https://github.com/Jermolene/TiddlyWiki5/pull/3609

[Jermolene]

Thanks @EvanBalster

[Jermolene]

Fixed in b9df224f99c425923c1fd8c25826e4d00a5fb39e

[kookma]

@Jermolene
Well I can use the current range operator keeping in mind that: It is not a real range function like Python!

@EvanBalster
Evan, if you insisted in having range as a counting function (counting down or up) you should prevent range from having a negative step, and in TW documentation add a part to clearly state this. I think it is error prone now, as it is different from what a programmer has seen in other similar language.

By the way, I have attached a revised version of range.js as it detects all cases but needs to be approved for syntax and performance, as I know little JS. If you use this, it not only handles counting up and down but also it works like a range programmer function.

Mohammad

$__plugins_ebalster_formula_filters_range.js_Rev02.zip

[Jermolene]

This issue is now blocking the release of v5.1.19. I don't have time to implement and test @kookma's desired modifications. @EvanBalster what do you think? If you want to make those changes and can do so in the next few hours I'd be happy to merge them.

[kookma]

@Jermolene
No problem. I appreciate your great work for 5.1.18.
We may remain for next release and the comments from @EvanBalster .
I will discuss this in more details with @EvanBalster

Thank you Jeremy! Thank you all

Best wishes for TW 5.1.8

Mohammad

[kookma]

Should I keep this open or close and make a new issue for 5.1.19 pre?

[EvanBalster]

Hey, Jeremy, Mohammad —

Mohammad's modifications here are geared toward creating a "programmer's range", which I think would need to be a new operator. (I have considered names like rangex or for; this needs some consideration.)

In any case a new range operator would probably constitute a new feature, so I suggest we punt this issue to 5.1.19 and I can work with Mohammad on an implementation. I'll probably include that implementation in a new version of my formula plugin.

[EvanBalster]

Actually, wait — my mistake. Mohammad's code here is geared toward making the current range more sensitive to potentially invalid input.

My position on this is that fault-tolerance (accepting "bad" input and producing the most sensible result) is a safer policy for a TiddlyWiki module like this. I'd rather just roll with it if something generates a [range[1,2.0001]] or if someone writes [range[10,0,1]], for example. A programmer's range would almost certainly be more fastidious.

[kookma]

Hi Evan,
Great to know your approach in developing range operator! Fault tolerance, a good terminology for this case!

So we will discuss this case inTW 5.1.19.

Cheers
Mohammad

[kookma]

Evan,
just curious to know: Do you still recommend to state in documentation that , the rangeoperator with current approach accepts float type STEP?

[EvanBalster]

Hey, Mohammad —

[range[2.5]] is definitely another weird case, and the output (1 2) may be unintuitive. In this case, we have no <begin> and no <step>, only <end>.

I think about this being used with an HTML slider, which might slide a float value from 0.0 to 10.0. Range will add a number to the list every time the integer part of the slider goes up by one. In that use-case, it seems reasonable to "tolerate" fractional numbers, and to count up to the integer part of the number given as <end>.

Errors aren't so helpful in TiddlyWiki because they make a mess of the page, and they can happen deep inside wiki rendering where they cause problems that are hard to find. I prefer to be "fault-tolerant", only throwing errors when absolutely necessary, to avoid harmless mistakes (like range[2.001]) breaking somebody's wiki.

EDIT: Rereading your question, I'm not sure it's important to state that we accept float step. You can do it, but it's not very useful. It also won't break your wiki if you do.

[kookma]

Hello Evan,
Good to know your philosophy behind this implementation!
By the way I believe in simple code and easy to understand logic! I prefer to use integers here! I see you use it for more complex cases.

[AnthonyMuscio]

FYI - the same method I use to create zero filled numbers with the range operator

<$list filter="[range[.000,.1,0.001]]" variable=number>
  <$list filter="[<number>removeprefix[0.]]">

  </$list>
</$list>

Can be used to generate 0 to N "Programmers range" and will most likely still work when such a "feature" is included in the future. ie there is or can be a work around that will remain valid despite future mods.

<$list filter="[range[.000,.1,0.001]]" variable=number>
  <$list filter="[<number>removeprefix[0.0]]">

  </$list>
</$list>

can be used to generate a range starting at zero.

<$list filter="[range[.000,.1,0.001]]" variable=number>
  <$list filter="[<number>removeprefix[0.00]]">

  </$list>
</$list>

Above tested.

Regards
Tony

[Jermolene]

Thanks @kookma @EvanBalster. We'll leave things as they are, and revisit an alternate formulation of the range operator for 5.1.19