There has been lots of different discussions related to various problems we'd like to solve:

• how to avoid use of |,| with closures and just use {}
• how to enable validation of with-blocks which run after a constructor
• how to pass with-blocks as references (potential solution to above)
• how to utilize with-blocks to create immutable derivatives (changing one or more fields in a deep immutable tree)

I think the new direction spurred by JohnDG's topic is towards the unification of closures and with-blocks. If we could unify these two features into a single more powerful feature, then many problems get solved:

• no need for using |,| {} anymore, closures can use just {}
• with-blocks become passable references, enabling more powerful libraries
• passing a with-block into a constructor gives the class designer control over validation

I really believe unifying with-blocks and closures into one concept is the right direction if we can swing it. Unlike some of the previous discussions, my pursuit is true unification - without any special distinctions between with-blocks and closures. If we are successful, we are just left with only closures (although more powerful closures) and the concept of with-blocks will cease to exist.

Scoping

The biggest difference b/w with-blocks and closures is how identifiers are bound to their scope:

x |,| { y = z }  =>  y is bound like any other identifier in outer scope
x { y = z }      =>  y is bound as x.y regardless of outer scope

This is the most fundamental problem to solve. There are two solutions:

• require some explicit prefix such as with.y = z
• use a different operator than = for declarative expressions

It is still my requirement that serialization remain a true subset of the Fan language. So I'm going to discount the first solution, because that doesn't give us a clean serialization. The obvious choice for a new operator is the colon since it keeps serialization quite readable. In fact it is consistent with map literal syntax:

Person { name: "Jack"; age: 30 }

Declarative Closures

So how could we apply that syntax within a closure? It requires enhancing closures with a couple new features. First we want to write code just like the expression above. How do we specify what name: and age: actually bind to? We need them to bind to an explicit type so that we get static type checking. My proposal is a new statement type called colon-assign with a syntax of field: expr which binds to the first parameter of a closure. Colon-assigns are only valid inside a closure (that name sucks - please provide alternate suggestions).

// this expression
|Person p|  { name: "Brian" }

// translates into
|Person p|  { p.name = "Brian" }

Looks really clean, and might be a nice general purpose feature applicable to any closure:

children.each |Widget w| { enabled: false }

Implicit Closure Parameter

So the next problem is that we don't want to explicitly specify a parameter to the closure:

// we want this longhand notation
Person.make |Person p|  { name: "Brian" }

// to be this shorthand
Person.make { name: "Brian" }

My proposal is that a closure declared with no parameters as a suffix to an expression takes one implicit parameter which is the type of the expression. So expr { ... } is implied to be expr |T it| { ... } where T is the type of expr. I don't really love that, but this moves us towards the groovy model of an implied argument called "it".

So if we go back to our original example:

// this shorthand
Person { name: "Brian" }

// compiles into this longhand:
Person.make |Person it| { it.name = "Brian" }

Setting Const Fields

We still have to deal with the const field issue, because now we have closures that need to be granted access to set const fields. But we don't want the closure called after construction time (which of course could happen with a general purpose closure). My proposal is that the compiler detects these cases and generates an invalidation hook. If we assume that Person.name is const:

// this code
Person { name: "Brian" }

// would generate this
closure := |Person it|
{
  if (invalid) throw Err("Closure has been invalidated")
  it.name = "Brian"
}
Person.make(closure)
closure.invalid = true

Implicit Calls

Another problem to solve: we don't want to manually add closure parameters to all our methods and remember to call them. The current rule is that if a method declares a Func parameter as the last argument, then you can declare a closure as a suffix to the call:

list.each |Foo f| { ...}   =>  list.each(|Foo f| { ...})

This rule would continue to apply. If you want to explicitly handle a closure in your constructor or factory method, then declare it as a parameter and off you go. The new rule I propose is that if a method (constructor or otherwise) doesn't declare a Func as its last parameter, then you can still declare a suffix closure which is implicitly called once:

Person.make { name: "Brian"}

// compiles into
p := Person.make
(|Person it| { it.name = "Brian" }).call1(p)

This rule would work with any expression, which would continue to allow you do stuff like this:

Str[,] { capacity: 5 }

Potential use as Anonymous Classes

It sure seems like it would be cool to use this feature to create declarative structures which don't operate on existing types. For example if a no argument closure is defined using colon-assigns, that those declarations become named fields on the closure itself:

x := { name: "Jack"; age: 30 }   
x()
x.type         =>   |->sys::Void|
x.type.fields  =>   Str name, Int age
x->name        =>   "Jack"
x->age         =>   30

This would often be a nicer way to declare things than map literals. Although it isn't ideal because someone has to explicitly call the function to compute the fields. Maybe the trap method does that automatically. I'm not sure about this feature, but something to think about.

Unresolved Issues

My major unresolved issue is how to deal with collections which currently are implicit calls to add:

GridPane
{
  Label { text="alpha" } 
  Label { text="beta" }
}

// compiles into
GridPane.make
.add(Label { text="alpha" } )
.add(Label { text="beta} )

That feature has proved itself invaluable coding with the FWT. So I'm loathe to get rid of it. The best I've come up with is an unlabeled colon:

GridPane
{
  : Label { text="alpha" } 
  : Label { text="beta" }
}

We need a solution for collections though.

I also haven't given much though to how these feature could work with creating cloned immutable structures.

Summary

The whole proposal is still pretty rough, but I think it really moves us in the right direction. Summary of my proposed changes:

• change serialization syntax to use colon
• add support for colon-assigns statements
• add support for implicit closure parameter
• add implicit closure calls
• add const setter invalidation
• with-blocks become a historical footnote

Most excellent! I feel this is definitely the right direction and succeeds in greatly increasing the expressive power of the language.

I'll mull this over for a day or two. In the meantime, I have some immediate thoughts. I'll post each one in a separate reply.

First, I don't see the need for a colon. Instead, I suggest the following: adding a keyword called bindscope that binds a variable to a scope (though doubtless there is a better name for such a keyword). This keyword would only be valid on parameters passed to methods. For example:

|bindscope Person p|  { name = "Brian" }

Because the parameter p has modifier bindscope, the compiler statically consults the slots of Person when resolving symbols such as name. This would be a handy feature in general (not just in the context of with blocks).

Then you can say that when someone uses a no-arg closure, {}, it is compiled to a closure having a single parameter with modifier bindscope.

One can imagine specifying multiple parameters with bindscope, to bring them all into the current scope, to eliminate lots of needless prefixing with the object names. A basic precedence rule (left-to-right) could apply in such cases. e.g.:

|bindscope Person p, bindscope Employee e|  { name = "Brian" }

name refers to p.name, unless it does not exist, in which case it refers to e.name, unless that does not exist, in which case the scope is traversed upward until a match is found.

It's necessary to handle this case, but I don't imagine that people would use this feature with two objects with identical slot names. It would be used in other contexts: for example, you have a File object and you have a Record object and you want to bring both in scope so you can do things like write(field1).

Second, on adding items to a collection or other object exposing a suitable add method. I propose any expression followed by a comma character evaluates to an implicit call to an add method in the same scope.

For example:

GridPane
{
  Label { text="alpha" },
  Label { text="beta" },
}

is equivalent to:

GridPane
{
  add(Label { text="alpha" })
  add(Label { text="beta" })
}

Purely for aesthetic/convention reasons, you can drop the requirement to add a comma after the last item, as long as there is another comma on the same line:

GridPane
{
  Label { text="alpha" }, Label { text="beta" }
}

Now lists don't need any magic.

Note that you're actually very close to being able to do the same thing even without the comma. As in Java, expressions are not ordinarily legal in Fan -- e.g. (True | False); would give a compiler error. The one case where this is allowed is new XXX(), which can have side-effects in Java and which is therefore allowed. If you disallow this, then you can always interpret standalone expressions as an implicit call to an add method. In which case you wouldn't need a comma. But I think I prefer the comma as it makes it more explicit that something is going on behind the scenes.

On construction, I like the idea, though I might prefer a different boolean field name, such as constructed or postConstruction, indicating whether or not the object is in the post-construction phase of its existence. Need to think about this more.

Hmmm.

You did have a completely different direction in mind.

It sort of reminds me of the June syntax that I described earlier where the caller rather than the callee determines how that first parameter is used. This syntax here is prettier though less explicit.

I sort of like it, but I also have some concerns and comments:

1. The colon makes it look perhaps too declarative rather than looking like a code block (if all you have are colon assigns, at least).
2. Any closure without a leading |,| should do non-local return and should be expected (though perhaps not enforced) to execute synchronously.
3. There are namespace collision risks with the current lovely add feature. This actually helps to solve that if we can find good syntax.
4. Sadly, this leaves no convenient way to call arbitrary methods on the "with" object as the current syntax has. But then the This return type convention is more helpful again.
5. The implicit parameter should use the given type rather than inferring the object type if the closure itself is explicit, so children.each { enabled: false } instead of requiring children.each |Widget w| { enabled: false }.
6. I'm not so sure that I like the closures as anonymous classes thing above. At least, it's not automatically a result of the other rules, so I'd defer that for now, even if the rest of this flies.
7. Except for the constructor issue, I would tie unexpected closures to objects (like current "with" blocks) rather than as secret last parameters to methods.
8. There's opportunity for confusion here, too:

Widget {
  text: "Hello"
  if (blah) { // 'size' belongs to the new widget.
    size: [400, 300]
  }
  doSomething { // 'border' looks perhaps connected to the outer widget
    border: Color.blue
  }
}

I was getting around the inconsistency between built-in constructs and closures in June by having no built-in constructs. Everything played by the same rules. Maybe you'll need to do the same here. That is, the size inside the if would be illegal. No object is associated with those curly braces. Alternatively, you could uglify the syntax again to be more like my June plans:

Widget {: // Leading colon says we are binding to the first param.
  text = "Hello"
  if (blah) { // No ':' here so 'size' belongs to the new widget.
    size = [400, 300]
  }
  doSomething {: // Leading colon again for new scope.
    border = Color.blue
  }
}

But I don't necessarily recommend that. Pretty is nice.

If either the closure type signature at the callee (as I discussed in the "Wish List" thread) or the block for the caller (in the example just above) has to declare the "with" object explicitly, then a lot of the ambiguity goes away.

Going back to your new proposal, for thoughts on how to do add, I think : by itself is ambiguous, too:

Widget {
  title
  : "Cool"
}

Maybe some custom operator instead like +: or <: would work:

Widget {
  title // definitely a method call now
  <: "Cool" // Calls 'firstParam.add("Cool")'
}

It would be ultra cool if the bullet list char for Fandoc were the same as the add operator, but I'm not sure how to do that very well.

Another view would be to make a << shortcut for add and then use the mental image for <<: as a "with" mode operator, but that starts to get really long, and people might think other operators would work. But I'm not sure other operators make sense for "with" mode.

Lots to think about :-)

I'm in two minds over the name: value syntax. Its not a bad solution the problem, and looks quite neat, but it does take them away from being "normal" code. If this is a unification proposal, I'd like to see them becoming more like normal code, not less. (ie. I should be able to write normal code like if/else in the block, and I should be able to use the : operator anywhere)

I think thats why John's bindscope concept feels quite neat. Its just a way of forcing something into the scope. My concern with it is whether managing scope is something thats too complex for the target developers - maybe, maybe not.

On the add topic, I think what jars with me is that there can only be one add method. What if a class has two child lists? (A profile having a list of addresses and a list of telephone numbers). The one add' feels magical and restrictive. Random thought - add another layer:

Profile {
  name = "Stephen";
  address = [
    Address {
      street = "My street"
    },
    Address {
      street = "My other street"
    }
  ];
  telephone = [
    Telephone {
      number = "123456789"
    },
    Telephone {
      number = "987654321"
    }
  ]
}

Definitely more verbose, but also more powerful.

With Brian's new proposal, you can write arbitrary code in the closure block. The : assignment is just a new kind of statement that assigns to members of the first param. That's it.

I think what jars with me is that there can only be one add method. What if a class has two child lists?

In that case the object itself isn't a collection, but rather has named fields which are themselves collections. That case is already handled today just like your example. The case implicit add handles is when a class itself is collection (not when it contains collections).

On construction, I like the idea, though I might prefer a different boolean field name, such as constructed or postConstruction,

This would be a pure compiler generated thing, not visible to user code (other than the exception we throw).

First, I don't see the need for a colon. Instead, I suggest the following: adding a keyword called bindscope that binds a variable to a scope

This is a really interesting idea - I especially like it because it doesn't break any of the existing code using with-blocks. I'm almost wondering if turns a closure into a more restricted with-block. Today with-blocks don't allow arbitrary statements - just assignments or methods on the target. If we had something general purpose like this I'm not sure how those two things interplay. Would we still allow general purpose statements?

I propose any expression followed by a comma character evaluates to an implicit call to an add

I like this because it does solve the problem with an intuitive syntax. I'm just wondering if it wouldn't be a little tricky to read/write correctly especially if mixed into general purpose closure statements. But I'd have to say this is the best proposal so far.

Any closure without a leading |,| should do non-local return and should be expected (though perhaps not enforced) to execute synchronously.

This was one of the reasons for requiring |,| - but is it really a reason to give up the {} closure syntax? I agree it could be confusing because {} blends in more like the blocks of if/for/while statements. But I do hate the |,| syntax and would love to see it go away. So which is worse?

The implicit parameter should use the given type rather than inferring the object type if the closure itself is explicit

This was a rule I considered, and in fact was the first example I wrote. I just wasn't sure about biting off yet another rule.

Except for the constructor issue, I would tie unexpected closures to objects (like current "with" blocks) rather than as secret last parameters to methods

Not sure I understand this. The rule is that expr {} binds the closure as last parameter if expr is call that takes a Func, otherwise the closure is invoked once after the expression (just like now).

That is, the size inside the if would be illegal.

I think that makes sense - colon-assigns would work within the scope of closure (never inside further nested scopes such as if/for/while statements).

{: // Leading colon says we are binding to the first param.

I might have missed this in earlier proposals, but the {: operator seems like an interesting idea - that might be something to chew on.

and I should be able to use the : operator anywhere

I agree with this - I'm just not sure what it means. If the colon operator was truly orthogonal to closures than that would remove some of the smell in this design. For example I toyed with colon-assign working against method parameters too. I could definitely see it somehow associated with constructor initialization (haven't thought that thru yet).

I haven't really sold myself on this design - I just wanted to get my ideas in concrete form to get some brainstorming going. After John's reply, I'm thinking I like the bindscope design more than the colon-assign design. I'm not sure how restrictive it makes the closure versus playing nice with other statements.

This is a really interesting idea - I especially like it because it doesn't break any of the existing code using with-blocks. I'm almost wondering if turns a closure into a more restricted with-block. Today with-blocks don't allow arbitrary statements - just assignments or methods on the target. If we had something general purpose like this I'm not sure how those two things interplay. Would we still allow general purpose statements?

Yes, without question. Even in with blocks I see the need for temporary structures, expressions, conditionals, and the like -- not with blocks as they are used today, to initialize fields, but the more powerful concept of with blocks as a closure.

So not only does a keyword give you compatibility with all source code written today, but it expands the power of both with blocks, closures, and functions themselves.

Here's an ordinary function that uses bindscope to reduce clutter:

Void accept(bindscope Visitor visitor)
{
   visitHead(head)
   visitTOC(toc)
   ...
}

The method invocations are on visitor (if they exist), or higher up if they don't.

It's a simple rule that is compatible with strong tool support (IDEs already highlight variables differently if they refer to this and not to a parameter or a local variable, and one can imagine them doing the same for slots bound to the scope of a parameter).

The this parameter itself can be thought of as an implicit parameter with modifer bindlocal. Less magic for this -- a nice side-benefit.

This was one of the reasons for requiring |,| - but is it really a reason to give up the {} closure syntax? I agree it could be confusing because {} blends in more like the blocks of if/for/while statements. But I do hate the |,| syntax and would love to see it go away. So which is worse?

The more integrated closures/with-blocks are, the more developers will be confused by return issues. I see two possible ways to simplify this.

First: introduce a syntax for a closure that doesn't return anything (||?). This closure doesn't even return Void. It simply doesn't return. If you declare a closure parameter with this type, then it's return value is thrown upward to the callee, where it is used as the return value for that method. For example:

closure.call()
// No statement after this is executed, because call() never returns

This gives the method accepting the closure power to decide whether non-local return is a good idea. The implicitly generated closures for with blocks would of course use non-local returns.

Second (completely different approach): if the return value of a closure is not caught (by assignment or evaluation in an expression), then it's automatically a non-local return. This approach has the benefit that it's always intuitive with what the code looks like it's trying to do. However, it has a serious drawback, in that removing an assignment to a closure evaluation can radically alter the meaning of the code, possibly even introduce a syntax error (if the types differ).

I like this because it does solve the problem with an intuitive syntax. I'm just wondering if it wouldn't be a little tricky to read/write correctly especially if mixed into general purpose closure statements. But I'd have to say this is the best proposal so far.

Sure, it would be a little odd to see lists of items out in the open; e.g.:

class DefaultSecurityManager : AbstractSecurityManager {
   new make() {
      HiddenFilesFilter(), SymbolicLinkFilter(), // <--- add'ed

      ...
   }
}

Would developers use this outside of with blocks? Only time would tell. I can definitely see them using it for custom list-like structures; e.g.:

tree := RedBlackTree { a, b, c, d; capacity = 100 }

In any case, it's a very clear rule that's easy to apply. If an expression is followed by a comma, it's treated as a list element and therefore added using whatever add evaluates to in the current scope (keeping in mind that bindscope modifies the binding scope).

Explicit bindscope is clearer than implicit.

Implicit, along with closures on object (return values) rather than explicit parameters, has versioning problems. For example, the original proposal on this thread could have these seemingly safe changes cause behavior problems without compilers errors (similar to why you might want the explicit override keyword):

Version 1:

Widget doSomething() {
  return this
}

Version 2:

Widget doSomething(|Widget| actionForEachKid := null) {
  // Seems backwards compatible.
  if (actionForEachKid != null) {
    kids.each |kid| {actionForEachKid(kid)}
  }
  return this
}

Calling code:

// Does something different for each version of the above code.
doSomething {
  title: "My Title"
}

Note that the .{ would avoid this particular problem, since for v1 you'd have to say doSomething.{title: "My Title"}, and it would mean the same thing for v2.

Anyway, bindscope (or whatever it's called) might still suffer from similar problems without the .{. I'd have to think through it.

By the way, my {: example above is a lot like bindscope, just with an implicit parameter. I don't really like it all the way, but I wish there was something like it that would work. Maybe there is.

Also for bindscope, there are two ways you could make it work, as I mentioned before, at the callee or at the caller. The callee is less flexible but allows for prettier (and less obvious code):

// Callee defines bindscope, somewhat like constructors in Fan today.
Void each(|bindscope Widget| action) {...}
// ...
parent.each {title = "Something"}

// Caller defines bindscope.
Void each(|Widget| action) {...}
// ... note no absolute need for a name, and even the type can be inferred.
// ... without a name, could use an 'it' keyword for reference, perhaps.
parent.each |bindscope| {title = "Something"}

Caller-defined bindscope is obviously more flexible. Use it as you see fit. It's just uglier when used, even with shorthand like {: ... }.

John, cool idea on bindscope being available on normal parameters, by the way.

As for tree := RedBlackTree { a, b, c, d; capacity = 100 }, I don't like it. Lists (and blocks and whatnot) should always come last. In this case after the attribute-like things. I'd have to see it as tree := RedBlackTree {capacity = 100; a, b, c, d}, but then it feels like it needs something to start it (especially if Fan has destructuring assignment as in a, b := c, d), which is why I started looking at obscure operators.

But maybe requiring at least one comma really is enough. Maybe destructuring assignment (if supported) should require explicit lists like they are planning for ES4 (as in [a, b] := [c, d]).

Summary

The basic idea is that {} would be a valid closure, if it appears after an instance or after a method invocation. In all cases, the closure would accept an implicit, compiler-specified parameter of the form:

bindscope XXX it

where XXX denotes the class in question (e.g. Person).

For methods that don't explicitly accept a closure as their last argument, invocations of those methods may still be followed by a closure. In this case, the closure is known as a suffix closure, and it is automatically called once by the compiler, immediately and in the same thread:

Person.make { name = "Brian"}

Meanwhile, the comma operator , is a new postfix operator that is compiled down to a method called add, in the same way that + is an infix operator that is compiled down to a method called plus. Thus any expression, A, is compiled down to add(A). Highly consistent with current Fan behavior.

Returning to a previous example of some of the things this new proposal could empower:

doc := 
html 
{
   body 
   {
      h1 { "Summary", }

      p(["class": "lead"]) 
      {
         "This is the only paragraph in this document.", 
      }
   }
}.toStr

This generates an HTML document of the form:

<html>
   <head></head>
   <body>
      <h1>Summary</h1>
      <p class="lead">This is the only paragraph in this document.</p>
   </body>
</html>

Exactly how is all this accomplished? Well, html is a constructor method for the class html, which accepts a closure as its last argument. The closure uses bindlocal, so that inside the html suffix closure, the reference to body evaluates to html.body. body is a function that accepts a closure as its last argument, which again uses bindlocal, and so on.

Generally, these functions all accept a map that defaults to null (in this example the map is only used for p), but which otherwise specifies the attributes of the HTML node. Every time they evaluate a closure, they grab all the text that has been added thus far, and use it for the contents of the node (clearing the list after they are done). An exception is thrown if you try to generate an invalid document.

Next I'll try an SQL example (after thinking on it for a bit).

obj {} and {} on return types is ambiguous across versions, as I mentioned above. If we feel override is necessary, we should care about this, too. I think new methods can be special as always taking an implicit last closure param.

I think , as a postfix operator just might work. It would be cool. We'd have to watch for a couple of edge conditions, though.

OK, first pass at an SQL-like DSL:

people := selectFrom(Person.type)
{
   record->id, record->firstName, record->lastName,

   where 
   {
      record->firstName = 'John'
   }
}.toList

echo("Hello, $people[0]->firstName!")

selectFrom is a two-arg function that returns a result set (which can be converted to list, map, etc.). Last arg is a no-arg bindscope closure, such that record and where are actually slots of a parameter implicitly passed to the closure (record is a dynamic class, where is a function accepting another no-arg bindscope closure). Prior to the where invocation, the dynamic class record records which fields are accessed, so it knows what to retrieve during the SQL query. During the where closure invocation, the assignments are used to limit the scope of the query. Type checking is done to assure that data is compatible with Person.type.

Likely there are even better ways. This is just the first thing that occurred to me after a bit of thinking.

I think what's possible already in Fan today looks perhaps more readable:

persons := querySession
	.from(Person.type)
	.selectOnly(["id", "firstName", "lastName"])
	.where |Person p| {return p.firstName == "John"} // Requires AST access.
	.execute

I think it also gets back into that "Is Fan for DSLs?" question.

I guess long detailed discussions wear me out, so I'm feeling less ambitious at the moment. Here's my current recommendation:

1. Require , (or some other syntax) for add to avoid current namespace ambiguity problems. I think this is important.
2. Use . { for post construction "with" blocks. Less vital, but it allows for future features without ambiguity.
3. Otherwise leave Fan as is. We can figure out nonlocal return, LHS and/or RHS bindscope, arbitrary code support, and shortcuts like {: ... } at a future date. Well, leaving Fan as is implies some support for LHS (AKA callee-defined) bindscope.

More details on , edge cases:

SomeContainer {
  doSomething
  somethingToAdd, // Comma required or else ambiguous.
  anotherThing,
  doSomethingOrNot // <- What does this mean?
  obviouslyDoSomething
}

SomeContainer {
  doSomething
  somethingToAdd, secondThing,
  thirdThing,;
  doSomethingOrNot // Definitely does something because of ';'.
}

Or even just one item still needs the comma:

SomeContainer {
  doSomething
  somethingToAdd,; // Comma required.
  doSomethingOrNot // Definitely does something because of ';'.
}

Inline version examples: SomeContainer {doSomething; somethingToAdd,; obviouslyDoSomething} or SomeContainer {doSomething; somethingToAdd, secondThing}.

But I haven't thought of anything obviously better yet. One option looks nice, but it goes down the path of TIMTOWTDI, and it doesn't look declarative for serialized objects:

SomeContainer {
  doSomething
  add:
    somethingToAdd, secondThing,
    thirdThing
  doSomethingOrNot // Lack of ',' terminates the list now.
}

That add: is a very different proposal from the colon-assigns at the top of this thread. In this case, it takes every item in the list and calls the method (here add) one at a time for each.

I'm not proposing it. Just brainstorming.

But anyway, I think less ambitious while clearing up ambiguity and allowing space for the future might be better than figuring everything out. If we do want to figure everything out, I have more to say, but I'll avoid it for now.

Concerning ambiguity, this kind of feature also isn't safe across versions:

The method invocations are on visitor (if they exist), or higher up if they don't.

Current Fan "with" blocks are sweet because (except for the almost lovely add feature) the bindings are always unambiguous.

I think it also gets back into that "Is Fan for DSLs?" question.

Whether or not it is, Fan will be used for DSLs -- as shown in the very snippet of Fan code you quoted (which, by the way, is impossible to implement efficiently, because you have to load every Person in the database in order to perform the filtering).

Likely, that snippet looks more readable to you because you're used to DSLs in Java, which are invariably based on the dot method syntax (the only real way to do DSLs in Java).

Fan brings more to the table with closures. With unified with blocks, it brings a lot more to the table, surpassing the expressive power of Ruby.

Here's an example that would not be so easy to replicate in dot method syntax:

transaction {
   insert(newRecord)
   
   ...
}

The whole transaction block is atomically committed to the database. Keep in mind arbitrary statements can occur inside the block, so you can build temporary structures, loop, and so forth -- constructs impossible to express with dot method.

Now here's another example that demonstrates asynchronous IO:

file.read(1024) {
   echo("Read: $data.toStr")
}

In fact, the code block that follows is executed in a different thread (assuming Fan allows this, since there is no data sharing), when the data is available. Lots of interesting possibilities here.

Will post more examples if I have more time.

Ok, just to provide some direction to the discussion:

1. is there strong support for unifying with-blocks and closures?
2. are we comfortable allowing {} as a closure even with the nonlocal return issues? There is little point in further discussion about unification unless we all really want {} closure syntax.
3. is there a preference for John's bindscope proposal over the colon-assign proposal? Let's eliminate one of these and focus on one.
4. if so, is the proposal that bindscope works like with-blocks today (binds only base expr of assignment or call); or that bindscope works more like the implicit this (where all expressions bind against this). This could be a really confusing issue.

There are lots of other issues including the collection adds, but I think the key issues above are the ones to nail down first.

1. is there strong support for unifying with-blocks and closures?

Yes. :)

2. are we comfortable allowing {} as a closure even with the nonlocal return issues?

Yes. It's not ideal -- ideally, non-local returns would be addressed at the same time. But given the choice, I'd rather see {} included than not. I don't think returns will be used often for with block syntax.

4. if so, is the proposal that bindscope works like with-blocks today (binds only base expr of assignment or call); or that bindscope works more like the implicit this (where all expressions bind against this). This could be a really confusing issue.

bindscope should work like implicit this. Programmer's know how that works and it would be fantastic to have that power in the language itself.

1. is there strong support for unifying with-blocks and closures?

Not yet.

2. are we comfortable allowing {} as a closure even with the nonlocal return issues? There is little point in further discussion about unification unless we all really want {} closure syntax.

Getting rid of |,| is good, but the implications I've seen so far are less good. Basically, anything involving non-local returns exposed to developers is something I'd probably have to argue against based on my experience with BGGA vs FCM closures in Java. Non-local returns are just plain confusing.

Now thats not to say that the DSLs that result from syntax similar to that displayed above using non-local returns aren't good (actually they tend to be sweet). I just believe that the solution doesn't have to actually involve the non-locals.

3. is there a preference for John's bindscope proposal over the colon-assign proposal? Let's eliminate one of these and focus on one. 4. if so, is the proposal that bindscope works like with-blocks today (binds only base expr of assignment or call); or that bindscope works more like the implicit this (where all expressions bind against this). This could be a really confusing issue.

I'm struggling with the direction here I think. The colon assign looks fine in a with-block, but extending it to be valid in any closure feels weird. The problem is that the LHS of the colon (or equals) is not in the same scope as the RHS.

The bindscope appears strong at first glance, but suffers from the problem that just adding to the scope list isn't really what you want for with-blocks.

I also think that this debate is probably about more than just the construction/closure/with concepts, its also about where Fan will sit on the language complexity/power line. Some of these discussions lead towards full expressive power closures, which I believe are best for a language which is all expressions and no statements. My gut feel is that Fan shouldn't go there - Scala already does that. I also have a snaking suspicion that typical Java/C# developers are happier with a number of discrete language features rather than fewer, more powerful ones.

I haven't particularly got my own answer though. I do keep on seeing a construction-with-block and seeing a set of data being passed to the constructor - either a map or named parameters.

Saying this isn't confusing (and neither are nonlocal returns in similar syntactic context):

if (something) {
  return blah
}

That said, here are my opinions:

1. I don't think unification is required.
2. If we go unification, I prefer bindscope (or whatever syntax is chosen) to colon assigns, because colon assigns are ambiguous.
3. Non-local return is needed when |,| is missing for closures.

Also, Ruby does have "with" block support. When calling a closure, you can control the self object in a dynamic fashion (as you can in ECMAScript, too). But Ruby doesn't really have full lexical scoping, so dynamic scoping makes more sense there (and ECMAScript only has explicit this references in its current form).

Tom, a return statement containing a return isn't confusing because its a return statement. Closures are something different. A closure is a block of code that is "picked up" and executed elsewhere. In that scenario, return naturally has a different meaning.

With FCM+JCA closures, we have return mean "return to caller" when its a closure, and "return to enclosing method" when its like a statement. This is like Fan today wrt closures.

I see the return aspect as key to the learnability of these features, including the transition from Java/C#. Note that C++, C#, Javascript, Nice and FCM/CICE closures all use return to return to the closure AFAIK - BGGA closures is the odd one out, and it doesn't work because Java has statements as well as expressions.

Thus, I'm moving to seeing three separate features - closures (blocks of code executed elsewhere where return is liked to the block), with-blocks (solely declarative, no returns), control-statements (the ability for developers to write their own control statements - a block of code that can only be executed "in place").

BTW, I don't dispute that modelling them as one feature creates a more expressively powerful language. My view is that unifying them changes Fan from the realtively simple language it is today.

A closure is just a function plus it's declaring environment. It encloses the variables from it's environment.

void foo()
{
    int x = 42;
    void bar()
    {
        printf("%i", x);
    }
    bar();  // prints 42
}

bar here is a closure, because it encloses the variable x from it's declaring environment. It the language does not allow this it does not have closures but in this case just nested functions.

An orthogonal concept is that of an anonymous function (labmda expression, sometimes called lambda function). An anonymous function might be a closure (depending on the language).

I know "anonymous function" (or related) is the right term. Just that two concepts usually go together these days, and it's easier to say "closure".

Anyway, I think if(blah) {doSomething} is exactly support for anonymous functions (and closures -- since they're friends in most modern/sane languages) that we've lived with for years on end without noticing it. And I think things that look the same should act the same.

So if we don't want it to act the same, we shouldn't make it look the same.

Well this is interesting. It doesn't sound like there is strong support for unifying with-blocks and closures. If we can't get consensus on that, then further discussion on unification doesn't make sense.

I personally want unification only for the {} closure syntax. Although I can see the other side of the coin how nonlocal returns make that confusing. So maybe we ought to focus on that issue (or else just decide to keep closures and with-blocks as two different constructs).

Regarding bindscope working like an implicit this, I don't really think that is what I want. Take this example:

class Foo
{
  Void open() 
  {
    Window { content = buildIt }.open
  }

  Widget buildIt() { ... }
}

Today this works just fine, because only the assignment to content is bound to the with-block target. The buildIt call binds to the implicit this. So I'm not really sold on the bindscope proposal yet, because although it is more general purpose, it seems like it makes my common case harder.

Between these two issues, I'm now kind leaning against the unification proposal.

I would propose not allowing unified with blocks to return anything -- i.e., their function return type is Void. That way non-local return can be added later (if necessary).

Regarding bindscope working like an implicit this, I don't really think that is what I want.

Yes, the buildIt refers to Window.buildIt and not Foo.buildIt. But remember, if Window.buildIt does not exist, then it DOES refer to Foo.buildIt, just like this works.

If Window really does have a method called buildIt, I would expect the most intuitive thing is that content = buildIt inside the with block actually refers to the version in Window, not the one in Foo (but maybe that's just me).

I would propose not allowing unified with blocks to return anything -- i.e., their function return type is Void. That way non-local return can be added later (if necessary)

The problem is that return can be used inside a void method/closure too. So return would be effectively a break, not a return if you were trying to use {} as a control block.

Yes, the buildIt refers to Window.buildIt and not Foo.buildIt. But remember, if Window.buildIt does not exist, then it DOES refer to Foo.buildIt, just like this works.

So I think what you are proposing is that bindscope adds a level to the scoping rules, but doesn't fully replace the implicit this. If I setup a bindscope variable, then scoping would be:

1. first bound to the bindscope variable
2. bound to local variables
3. then bound to the implicit this and enclosing class

Not sure what I think about that - seems like scoping could get kind of confusing. Then again with-blocks aren't perfect. Today you can do this:

Point foo(Int x, Int y) 
{
  return Point { x = x; y = y }
}

That works because only the lhs of the assignments bind to the with-block target. I've been doing that in my own code, but not sure I really like it.

I think that the bindscope rules are probably just that little bit too complex - powerful, but complex.

Brian, you say you find the point example a little odd, but I don't think it needs to be if you're willing to change with-block syntax:

// named parameter style (actually setting slots)
Point foo(Int x, Int y) {
  return Point (x = x, y = y)
}

// use @ for storage like in setters
Point foo(Int x, Int y) {
  return Point (@x = x, @y = y)
}

// like a map
Point foo(Int x, Int y) {
  return Point [x: x, y: y]
}

Each syntax has merits, it just happens to be different to that used today. If I'm not mistaken, any of these three would allow closures without |,| too.

I think I might need to outline each of the use-cases we have in a separate thread.

The problem is that return can be used inside a void method/closure too. So return would be effectively a break, not a return if you were trying to use {} as a control block.

Right, I forgot about that. You'd almost need a way to encode a function that can't return prematurely.

So I think what you are proposing is that bindscope adds a level to the scoping rules, but doesn't fully replace the implicit this. If I setup a bindscope variable, then scoping would be:

Yes, pretty much. Here's a complicated example method (the likes of which you would probably never see in real code):

Void foo([bindscope This this,] bindscope A a, bindscope B b) {
}

The order of binding is as follows:

1. Search local variables to see if any match (local variables must always have precedence, since the natural order of scoping is inner to outer).
2. Search b to see if any slots match.
3. Search a to see if any slots match.
4. Search this to see if any slots match.
5. Search higher scopes, like normal.

I include the implicit this parameter to emphasize the behavior for a and b is identical to this, except for the layering, which says b has higher priority than a, and a has higher priority than this. Also to emphasize the feature is just a generalization of what the compiler auto-magically does with this.

Note that with unified with blocks implemented using bindscope, you could always refer to the context explicitly, with Groovy-style it, because the closure is simply passed a bindscope variable called it.

Point foo(Int x, Int y) 
{
  return Point { x = x; y = y }
}

You should hide such code as it will turn up on blogs as an example of Why Not Fan. :-)

The left-side binding should always tie to the bindscope object in the same fashion that binding works on "with" blocks today, if we go bindscope. It avoids versioning ambiguity.

So its been a week. Does anybody have new thoughts?

My current thinking is that the best strategy is to do nothing and leave things as is. I would love to unify closures and with-blocks, but the potential problems don't seem worth the risk.

No new thoughts on this proposal. I think that Tom's first code example (post 5 in this thread) concerns me in that I can't separate the colon-assign in a closure from that in a with-block.

I do believe that we still have problems to solve with construction and with-blocks however - notably, validation post with-block (especially const fields), deserialization (replacing and validating the input against evil), changing a field on an immutable.