I've encountered enough situations where I need a non-local break or return in closures, that I've convinced myself that we really need this feature. It seems to crop up fairly often that while iterating a collection you want to break from the closure or return from the enclosing method. The only work around right now is to use a for loop which is pretty ugly.

Suggestions for syntax?

You can be descriptive like notOnlyLeaveThisFunctionButTheSouroundingOneAsWell but perhaps you want somethign shorter like überReturn.

Anyway, what happens here:

Int foo(|->Void| bar)
{
    bar()   // returns Void, but über returns Str, see below
    return 42
}

...
foo(|,| { überReturn "a string" })

How do you integreat that into the type system? Has each function a return type and a über return type?

Only to make sure nobody get's this wrong, something like this is valid:

Int someFunction()
{
    someLocalFunction := |->Str| {
       if (someCondition)
           return "text"
       else
           überReturn 42
    }

    someString := someLocalFunction()
}

But I need to make sure a function can only über return values of a type compatible with the return type of the calling function.

Something like

...
someLocalFunction := |->Str-->Int| { ... }

I'm partial to Smalltalk's syntax - so how would the ^ work?

In which context does über return work? Can I do this?

Void return42()
{
    über return 42    
}


Void foo()
{
    if (shouldReturn42)
       return42();

    ...
}

I'm partial to Smalltalk's syntax - so how would the ^ work?

I don't have any strong preferences, I could live with ^. Although I think a new keyword such as throwreturn or methodreturn might be a little more readable and easy to learn.

Helium has brought up some good points which are really asking how this gets implemented. The implementation I was thinking about would work like this:

Obj foo(Obj[] list)
{
  list.each |Obj x| { throwreturn x }
}

// would compile into...

Obj foo(Obj[] list)
{
  try
  {
    list.each |Obj x| { throw NonLocalReturnErr("acme::Bar.foo", x) }
  }
  catch (NonLocalReturnErr e)
  {
    if (e.qname === "acme::Bar.foo") return e.val
    throw e
  }
}

So restrictions would be:

• you can't use a throwreturn unless inside a closure
• it only works inside in the declared method
• otherwise the NonLocalReturnErr will propagate up the stack until caught

Now we just need a keyword or other syntax to force a closure to use throwreturn instead of return.

Void foo(nonlocal |Str -> Void| f) 
{
}

f := |Str -> Void| { return; }

foo(f) // <= ERROR! Types do not match, since f uses 'return'

Then we're all set for a clean and mutually agreeable return to unified with blocks. :-)

In general, I don't trust non-local returns unless the block can't move (ie. you can't assign the block to a variable and call it later). Of course that pretty much means it ceases to be a closure (but perhaps thats a good thing?).

Before going down this road, you'll need to define how a non-local return actually works. Can the closure be assigned to a variable? If so then it can be called incorrectly from a later method once the original has completed. If that happens, how does the interaction with a try..finally block work?

And it needs to be defined in such a way that accidently calling a non-local return doesn't usually happen. I'd suggest throw return as the most descriptive keyword, as its closest to what is happening underneath.

In general, I don't trust non-local returns unless the block can't move (ie. you can't assign the block to a variable and call it later).

Since 90% of closures are arguments to methods you can't really make that restriction. I think the common case is pretty safe. And if you did try to use it outside of its declaring method you will get a NonLocalReturnErr exception (which is what should happen IMO). Although I do think it is very important that using a nonlocal return only works in its containing method, not any method (which is why I plan to check the method qname).

So right now I would say my leaning is towards a methodreturn keyword.

Now we just need a keyword or other syntax to force a closure to use throwreturn instead of return.

This is a pretty interesting idea John. I need to digest it a bit, but it could turn out to a missing piece of the unification puzzle.

I am more partial to something like "return super" or something similar since it conveys the meaning better of what you are trying to do (rather than the implementation). You also avoid introducing a new keyword.

I can't see immediately how to make a closure with non-local returns apparent to be read, i.e. knowing |Int->Int| will return me a String when being given one, e.g. as a method parameter.

IMHO catching specifically thrown Errs is probably the best and most readable way to handle non-local breaks and returns. It would be nice to see use cases when the absence of the proposed additions makes the language non-(re)usable.

I agree that we need specific use cases for why non-local returns in closures are needed. From the Java closures debate, I don't believe the case is proven. In fact, I'd go further and say that had BGGA not included non-local returns, it would have got a lot less objections, and might even have made Java 7.

BTW, FCM closures does allow for a form of non-local return in the JCA statement. JCA proposes that ordinary developers can write language features, but this is done by a special kind of method declaration (JCA doesn't specify an implementation in bytecode). Essentially, the aim was to provide a mechanism that avoids the possibility for a developer to ever experience a non-local return (things just work). If it looks like a control statement, then return works like a control statement. If it looks like a code block with arguments (a closure), then return works like a method and returns to the invoker.

More broadly, I would be in favour of the advanced feature of allowing anyone to write control statements in Fan if it can be achieved separately from closures. I'm generally opposed to using closures with non-locals. (Alternate strategies include AST manipulation or hygenic macros).

BTW, my overall thesis accepts that many languages merge the concepts of closures with local and non-local returns. However, on closer examination, these language all tend to be solely expression based (Scala), whereas Fan and Java are statement and expression based. All languages that I've found with statements use local returns for closures, and avoid non-locals AFAICT - Nice, C#, Javascript, C++ proposal (I'm happy to be proved wrong on this BTW). Hence, I don't believe BGGA suits Java, and I don't believe non-local returns in closures suits Fan.

As I said though, I am willing to support a separate language feature that allows non-locals). This feature would prevent the application developer from holding a reference to the block of code that might contain the non-local return. In fact, I'd prefer it to not be a portable block of code at all - its simpler and easier all round to just embed the relevant code into the client bytecode.

Note that the implication of two language features is two different syntaxes for the caller - one for writing a closure, one for writing a control statement block. I'd also suggest that this approach should, if at all possible, allow all the existing language constructs to be written using the control statement feature.

One final point - it doesn't have to be easy to write the code to process a control statement. In fact there are benefits to making it hard - specifically, avoiding dialects of Fan.

I agree that we need specific use cases for why non-local returns in closures are needed.

My main requirement is for short circuiting iteration. For example the code I was writing last night which prompted this:

// I would like to write this
override Line closes(Block open)
{
  closeBlocks.each |Block b| 
  {
    newLine := closeBlocks[i].closes(this, open)
    if (newLine != null) methodreturn newLine      
  }
  return null
}

// instead I write this
override Line closes(Block open)
{
  for (i:=0; i<closeBlocks.size; ++i)
  {
    newLine := closeBlocks[i].closes(this, open)
    if (newLine != null) return newLine
  }
  return null
}

Often code like this indicates that maybe List or Map should have extra methods to just do it for me. But often the break condition and return value aren't necessarily the same. There might be a couple more methods we could add which would solve most cases.

To date I've just been solving this with an old school C styled for loop. But non-local returns aren't necessary the only (or best) solution. I'm not quite sure that this problem actually warrants the non-local complexity. But I do keep stumbling over it.

If you do a throw surely you can catch the result and terminated the loop?

Or how about a List.search(|V->Obj|)? It breaks and return the value if the closure returns anything not null.

More generally, I am starting to wonder whether closures should include throws SomeTypeOfErr in its definition. This would be similar to how methods works in Java, but since closures are being passed around as parameters this will provide richer type information.

I agree non-local returns would clean up code and better express developer intent (no, throwing exceptions is not the way to "clean up" code constructs like Brian's example!).

I also share Stephen's concern for the dangers of holding references to closures that do non-local returns.

And finally, I'm still a strong advocate of unified with blocks (using bindscope), because they work with existing code perfectly (except for the absurd case of x = x, which should be changed anyway, IMO), and enable a level of expressiveness not seen in most languages.

So here's my proposal: {} is the only closure that permits non-local returns. Such closures cannot be stored in any variable, they recycle the return keyword, and they accept at most a single parameter. If you need to refer to that parameter, you use the keyword it.

override Line closes(Block open)
{
  closeBlocks.each {
    newLine := closeBlocks[i].closes(this, open)

    if (newLine != null) return newLine      
  }
  return null
}

Obj getFirst(Obj[] list)
{
  list.each { return it }
}

foo := { } // ERROR! Closure cannot be stored in a variable, only passed to a method

Now with blocks become a non-local return closure of type bindscope, so that while it may be referred to explicitly, it is not usually necessary since slots bind against it second (after first binding to local variables).

This is starting to look like something which could work. I like the simple rule that a non-local return closure cannot be stored in a variable, but can only be passed to a method directly. However, the downside is that a simple refactor changes the meaning of the code:

Void process() {
  list.each { return }
}
Void process() {
  block := |,| { return }
  list.each(block )
}

Perhaps that is OK. I'm not completely sure about the limitation of one argument it however. For example, looping around a map with key and value becomes something that could not be done with a non-local return.

So the theoretical use case is that a map loop should be able to be used with local or non-local returns. So the basic pattern we are dealing with is:

Void process() {
  map.each |Str key, Int val| { ... }
}

What if we separate the syntax for local return closures from non-local return closures? Non-local return ones would have to be able to use just { ... } as they should look like control structures. That means the current syntax for local return closures would need to change. How about this?

Void process() {
  map.each #(Str key, Int val) { ... }
}

Void process() {
  map.each |Str key, Int val| { ... }
}

The former has local return. The latter has non-local return.

Note that I have deliberately changed the arguments from || to () in the local return version. This is so that a return always returns to the closest set of round bracket arguments. The # symbol (or maybe a fn keyword) is simply a wildcard for a method name, as it is a kind of anonymous function/method.

Note that when you have no arguments, these simplify well:

Void process() {
  something.handle # { ... }
}

Void process() {
  something.handle { ... }
}

No more |,|.

This could then be merged with the "first argument is it" concept quite well.

However, the downside is that a simple refactor changes the meaning of the code:

That's not really a simple refactor -- that's changing one kind of closure to another. If you did it in an IDE, then it would know to propagate the return value.

Perhaps that is OK. I'm not completely sure about the limitation of one argument it however. For example, looping around a map with key and value becomes something that could not be done with a non-local return.

I don't think this is a good use-case. If this feature were already in Fan, the standard libraries would have been designed with it in mind. So you'd do something like:

map.each {
   echo("$it.key => $it.val");
}

Or, if the single parameter of each was declared bindscope, even simpler:

map.each {
   echo("$key => $val");
}

So I don't think the additional complexity introduced by a whole separate syntax for arbitrary argument non-local return closures is justified by the meager benefits -- which would cease to be benefits with slight modifications to the core.

So to summarize my proposal:

1. {} is always a closure with non-local return.
2. {} may only be passed as a parameter to a method; no references may be stored to {} closures.
3. {} may only be used when a closure with zero or one parameters is expected.
4. If {} is used for a closure accepting one parameter, the name of the parameter is it, which becomes a new keyword analogous to this.
5. System libraries are modified to be friendlier to {} closures.

John - I like where you are going, but the problem is that any solution with {} closures has to take into account unified with-blocks. That proposal kind of fizzled mainly because of the non-local return issue (so this is the flip side of the conversation).

My use case for non-local returns is pretty specific to short circuiting iteration. So maybe a full blown language feature is serious overkill (especially with the complexity it introduces). So let's go back to alexlamsl suggestion of just adding some new methods to List and Map:

Obj List.foo(|V,Int->Obj| c)
Obj Map.foo(|V,K->Obj| c)

These methods would iterate the collection and return the first non-null value returned by the closure. I'm not sure what the right name for foo is because it is kind of cross between each, find and map. Sometimes you are just iterating with a break, other times you care about the mapped return value. Some suggestions:

• eachBreak (doesn't convey when used as find )
• findFirst (doesn't convey iteration each with break)
• findMap (doesn't convey iteration each with break)
• search (confused with find)
• until

Any other ideas?

I think with these methods, that most of my non-local return use cases could be solved strictly with the API.

John - I like where you are going, but the problem is that any solution with {} closures has to take into account unified with-blocks.

In fact, that's exactly what I was doing. The proposal is tailor made for the reintroduction of unified with blocks.

How so? In this proposal, a unified with block becomes a bindscope closure with non-local return.

Meaning unified with blocks work perfectly with the vast majority of existing code:

Label {
   text = "foo"
}

The only case that doesn't work is when the LHS = RHS (x = x).

The expressive power of with blocks increases, because you can have arbitrary code nested in a with-block:

Label {
   if (htmlMode) text = "<span>foo</span>"
   else text = "foo"
}

And all returns in unified with blocks are non-local (since they are a {} closure):

Void readFile(Str file) {
    file.open(file) {
        v = read()
        if (v == 1) return; // no more data
        ...
    }
}

As you point out, the primary reason UWBs died is because non-local returns were not on the table. Now that they are, I really think people will be more open to this idea. So it really solves much more than the short-circuiting problem.

John is solving most of my objections at the moment. I'd need to revisit why I'd moved against UWBs, but this is actually looking like a really solid simple concept right now.

If we revisit the unified with-block, then we need to solve:

• bindscope feature
• still need to figure out how to do collection implicit add
• return means different things in |,| {} versus {}
• new rules/meta-data for closures which can and cannot be saved to local variables
• lots of magic rules dictating implicit variables, bindscope, etc

It just seems like a tremendous amount of complexity compared to what we have today which are fairly simple, general purpose closures and with-blocks. I don't love the current design, and really would like to unify the two. But it seems like we would be adding a lot of complexity for marginal gains.

At this point I think a breaking iterator on List and Map will be good enough for my non-local returns, so I retract my requirement. Regardless of unified-with-blocks, I'd still want eachBreak API for list and map. So suggestions are welcome.

I think the problem with the API approach is that the API is trying to do two things. As a general rule I've found any API that does two things tends to have issues down the line. The fact that they can't be easily named is another clue.

The basic options are:

1. One form of closure, passed to method (local return) with APIs instead of non-local returns
2. Two forms of closures, inline (non-local return) and passed to method (local return) with two different syntaxes where the parameters received by the closure are specified
3. Two forms of closures, inline (non-local return) and passed to method (local return) with two different syntaxes where the parameters received by the closure are NOT specified and it is used
4. Two forms of closures, inline (non-local return) and passed to method (local return) with two different syntaxes where the parameters received by the closure are NOT specified and it is used together with bindscope

Having looked at this again, my preferred option is #2. This avoids the hacky workarounds of APIs, without the magic quality of it and bindscope (it doesn't work well when you nest these blocks). This was my syntax suggestion, but there are many other possibilities (including the opposite way around, which works better with function types like |Str,Int->Void|):

Void process() {
  map.each #(Str key, Int val) { ... }  // local
}
Void process() {
  map.each |Str key, Int val| { ... }  // non-local
}

In the list of five points, this approach has no bindscope, return means different things (which is a Good Thing), simple rules for what cannot be assigned to local variables, and no magic rules on implicit variables (which is a Good Thing).

The hard one is collection add, which is necessary if this also integrates with-blocks (that could be done in a second phase). I do wonder if the collection add issue is an issue because there is no general collection add operator in Fan. I'm sure I've seen something like this elsewhere...

coll << itemToAdd << anotherItemToAdd

An operator like this would easily enable add in with-blocks, just drop the "coll".

I've added an eachBreak to both List and Map. Whatever we decide about non-local returns, these are very handy methods to have.