A New Type CalledExpression

Instances of this type are actually expressions and correspond to the Expression production in the Fan grammar.

The class has a method called eval which evaluates the expression, returning the result of the evaluation as an Obj (which of course would be auto-casted to the type the user wanted).

Example with boolean expressions:

Void callExpr(Expression expr) 
{
   if (expr.eval()) echo("yes")

   else echo("no")   
}

callExpr(4 > 2) // yes
callExpr(4 < 2) // no

Note that the expression is not evaluated until the eval method is invoked. In other words, this feature is essentially equivalent to lazy evaluation. In the programming language D, it would be expressed as follows for the boolean case:

void callExpr(lazy boolean expr) {
   if (expr) writefln("yes");

   else writefln("no");
}

However, the Expression concept is more powerful as it allows arbitrary types of expressions to be processed by a single handler.

Note that every time eval is invoked, the expression is re-evaluated. This behavior is essential for code whose job is to wait until some condition is satisfied.

Internally, Expression is simply implemented as a closure that returns an object of the type required by the expression. eval simply calls the closure.

Other applications:

debug("This string is not actually constructed here, saving time: $foo, $bar")

Remove Magic fromwith Blocks

Imagine that functions can be called with specific contexts (a context corresponds to a scoped location). A function is always called with the context in which it is defined, which is why it has access to variables and classes in that context. Well, suppose you can supply an additional context when invoking the function, whose variables eclipse those in the definition context? In this case, a with block can be thought of as a closure invoked with an additional context. As such, we can now specify the behavior of with blocks as a method of the class:

class MyClass 
{
   Void withBlock(|,| block)
   {
      block.callContext(this.context)
   }
}

(Actual syntax is irrelevant and could likely be improved. In any case it would be automatically generated by the compiler.)

Now with blocks have no more magic. They are ordinary closures evaluated in two contexts, where the class context has precedence over the definition context.

This affords numerous benefits. There is no need for new handlers, since if someone wants this behavior, they can override the compiler-provided withBlock implementation and do their own validation. If they only want to validate on startup, then they can just declare withBlock as a once method -- no special syntax is needed. For example:

class MyClass 
{
   once Void withBlock(|,| block)
   {
      block.callContext(this.context)

      if (myField == null) throw ArgErr.make
   }
}

Similarly, it's possible to do things like performing a commit in a database transaction after the with block is evaluated:

db.doTransaction {
   insert(record)
   ///
}

In this example, the doTransaction method returns a Transaction instance, which has methods such as insert and so forth, and which overrides the withBlock so that after evaluation, the database commands are committed to the database.

Most importantly, the combination of features (1) and (2) allow the construction of extremely powerful Domain Specific Languages. For example:

Whenever(a > b) 
{
   // Stuff to do
}

This is a DSL for a rule engine. Whenever is a method accepting an Expression and returning an instance of WheneverBlock, which implicitly stores a reference to the Expression and which overrides the withBlock method to store a reference to the closure. Perhaps a separate thread then polls the expression and executes the with block closure when the expression evaluates to true.

Similarly, it's possible to implement custom conditionals:

If(a > b)
{
   // ...
}

Not even Ruby has this expressive power.

Others To Follow Later

I really dig this train of thought. What you are suggesting is almost exactly what closures are today with one big tweak. I think the primary idea in your suggestion is that a parameter of a method can implicitly turn an expression into a closure:

// today
Void func(|->Bool| f) { if (f.call) echo("yes") }
func(|->Bool| { return a == b })

// proposal
func(lazy Bool f) { if (f) echo("yes") }
func(a == b)

I would say that the proposal is nothing but syntax sugar for the first example. But man that could be some seriously powerful syntax sugar! Quite Smalltalk'ish.

I'm not sure I follow the context replacement. And I'm still not sure how I tackle ctor with-blocks without a lot of boiler plate code.

Scala also has the lazy param evaluation feature (which I think they call "call by name"). I think it would be great, but I doubted recommending it. That said, I still very much like the feature.

In June, I was going to automatically convert expressions to anonymous functions. As in, if it expected |->Bool| and you gave it a Bool, it would automatically wrap it in a function. I think this is safe, but I haven't necessarily thought through all cases.

Oh, and I think anything called Expr (or similar) should be AST/LINQ-related.

As for the "with" block recommendation here, I don't agree with it. I think "with" blocks should be syntactically obvious to the caller. See my discussion here of my opinions on some pros and cons of related topics.

lazy would imply to me "call by need" rather than "call by name". That would be realy confusing.

It's strange seeing lazy semantics in a strict, macroless language. Seeing func(a == b) makes it seem like standard applicative order evaluation is used, which would cause widespread heartattacks in early Java adopters of Fan.

Lazy evaluation is really cool and useful, though. Instead of allowing a method alone to decide whether or not a parameter is lazily evaluated, it might be nicer to have a lazy macro or keyword that creates an Expression object (I don't like the term Expression here either... too many connotations). The advantage is the caller explicitly states the expression as lazy. Seeing the keyword in blue in the IDE will alert new users that something weird's going on.

Maybe a syntax like:

func(lazy Bool f) { if (f) echo("yes") }
func(lazy(a == b))

Of course, laziness is weird. If you introduce something like this, you need to be very clear on

• When do expressions get evaluated?
• If multiple promises are used, what order are they evaluated?
• How many times does each promise get evaluated?

You gotta be careful with lazy semantics. We don't want to our launch missiles twice!

Still don't quite see what are the differences between |->Bool| and lazy Bool.

But the discussions here prompted me to one question - can we have once closures? Would they be meaningful at all?

Even though lazy may be convenient, I do not prefer the D-style syntax, for the following reasons:

1. The function using the lazy argument does not have control over when or how often the lazy argument is evaluated.
2. One function may not process a generic expression, but must instead require specific types.

Although perhaps (2) can be mitigated in Fan (e.g. lazy Obj object), the issues with (1) still remain.

Now returning to with blocks. The concept comes from Ruby, or more generally, from any dynamic language with an eval construct capable of executing an arbitrary string as code. In such contexts, the question naturally arises: Where do symbols bind to? For example, if you have the string, eval("echo(y)"), where does the value of the symbol y come from? Generally, the context of the evaluate is the context of the eval statement. But Ruby and perhaps some other languages allow you to use different contexts. For example, sometimes it is more convenient to eval some code in the context of a calling function.

A similar idea applies to with blocks. If, as I have argued, a with block is a no-arg closure (|,|), then just like other closures, it is evaluated in the context in which it is defined -- that is, symbols are bound to the context in which it is defined. But for with blocks, there is one important exception: symbols in the class have precedence over symbols in the definition context.

So if we have some notion for the context of a class (which could be, for example, this.context), and if we have an alternate version of Func.call that can specify an additional context whose symbols take precedence over the definition context, then we can dispense with with magic altogether, and implement it in the fashion previously defined:

class MyClass 
{
   once Void withBlock(|,| block)
   {
      block.callContext(this.context)
   }
}

This method would always be provided by a compiler. However, a user could override it:

class MyClass 
{
   override once Void withBlock(|,| block)
   {
      super.withBlock(block)
   }
}

Now keep in mind that this.context and block.callContext needn't have any semantic meaning. That is, it's not necessary to flesh out what a context is in order to be able to create expressions that refer to one (similar to Ruby). It's only important that the withBlock method be able to have a non-magical implementation.

With this, you get the many benefits previously described:

1. No need for new validator, since it can be encoded in withBlock method.
2. Astounding possibilities for domain specific languages.

I've only touched on (2) briefly with custom conditional and looping constructs and database applications. But I can think of others without trying:

file.open {
   seek(102)
   while ((int c = read())
   {
      // Do something
   }
}

Here seek and read and other methods are methods of the object returned by file.open. Once the with block has finished, the file is closed automatically.

tag("html") {
   tag("head") {
   }
   tag("body") {
      tag("h1") {
         text = "This is the heading!"
      }
      tag("p") {
          text = "This is the body!"
      }
   }
}

Here an HTML document is constructed with a very clean syntax.

And so on. The possibilities are endless!

---

I highly recommend that if we want "call by name" that we consider the auto enclose Type in |->Type| strategy:

Void until(|->Bool| check, |,| action) {
  while (!check()) {
    action()
  }
}

// elsewhere ...

i := 1
until(i < 10) |,| {
  i++
}

I think it's clear, but to make it all really work in expected fashions, we need to reconsider nonlocal return and closure syntax. It could still be useful on its own, but people will march right towards custom block structures and hit the other walls.

Or in other words, it might be best deferring this feature for now unless we want to reconsider all the other issues.

Or in other words, it might be best deferring this feature for now unless we want to reconsider all the other issues.

Agreed - I'm not sure I want to tackle something this complex for 1.0. I'm willing to tackle something like this if it provides a cleaner solution to the with-block/constructor problem. But I'm not convinced something like is really better compared to the extra language complexity.

Not to shutdown the discussion - good discussions like these can lead to all sorts of inspiration.

I see Fan as being friendlier say than Java, but it's not oriented toward domain specific languages (such as what you get from Scala). I think there are pros and cons to it. I had intended June to be more DSL-oriented than Scala in some ways (just not so operator crazy). But I think Fan might be best to stay as is for now.

However, maybe non-constructor "with" blocks should change syntax to obj.{/*...*/} for these two reasons:

1. It think it's more obvious to newcomers while requiring about the same typing effort.
2. It reserves "obj {/*...*/}" for a future world where people might be more demanding on the DSL subject. Just don't implement this feature for Fan 1.0.

Again, I think "Type {/*...*/}" should remain as is.

I see Fan as being friendlier say than Java, but it's not oriented toward domain specific languages

One of the things I like about Java is that there isn't "dialects" - in general a Java programmer can read/write anybody else's code. Maybe not as powerful as creating your "own mini-language", but I think this is a good thing. That said I think most DSLs I've seen are really just trying to do declarative programming which I personally think Fan really nails.

However, maybe non-constructor "with" blocks should change syntax to obj.{/*...*/} for these two reasons

Those are good reasons. I'm not sure I'm convinced on that yet, but the future proofing is a really good argument.

One more reason: Type {/*...*/} (or Type.makeWithBlah(blah) {/*...*/}) vs. obj.{/*...*/} gives Stephen the distinction he wants between constructor and later blocks. In the former, you can set const fields, but not in the latter. Note that the named constructor case makes the reservation of syntax for future use not quite as complete (at least not psychologically), but I still think it might be okay to make the reservation.

Interesting relationship to the dup {} proposal, too. In this case, obj.dup {name = "Blah"} and obj.dup.{name = "Blah"} have different meanings. Still not convinced on the magical dup thing, but it's interesting to see the ramifications.

Agreed - I'm not sure I want to tackle something this complex for 1.0.

Much of this functionality can be implemented using a much simpler proposal to provide pre/post with-block hooks. All the functionality could be provided if the pre hook has veto power and the post hook has loop power.

Here's an improved version of the HTML example:

doc := html {
   head {
   }
   body {
      h1 {
         "Heading"
      }
      p {
         "This is the first paragraph."
      }
   }
}.toStr

Quite nice.

Back to my list of why obj.{}. Initially, if the called method can control the context (as suggested by John), then it makes things like dup less magical. So the future proofing argument comes back stronger.

But more interesting for the present are these alternatives: obj?.{}, obj->{}, and obj?->{}. The ? form just dodges the whole block if null, and the -> form makes all the calls inside to be duck calls. For example:

someObj->{
  quack(10)
  waddle
}

// same as

someObj->quack(10)
someObj->waddle

I like John's idea of with-block just being a special closure.

I like where you are coming from Tom regarding applying ?. -> to a {} block. Another good argument for your style. Although:

• that notation doesn't work well with how I like to do my braces
• I don't really like making a syntactic distinction b/w ctor versus non-ctor case (to me this is a negative)

I think there is some grand unifying concept hanging just out of reach, waiting to be discovered. But I'm struggling to make it tangible...

I've thought more, and we've already got the unification principle here with John's recommendation (only I'd modify it some).

For constructors and dups (or Stephen's new), the common thread is that the "with" block directly relates to the method call. For "obj.{}", there is no method. Constructors (and perhaps dup) are only special today in that the compiler enforces the implied closure parameter and context-specific callback rather than requiring them to be given manually. So it's not about const support or not. It's about whether the "with" block is tied to a method call or not.

Concerning brace alignment, . and { should still be separate tokens for this syntax. So you could say this:

obj. // Or space it for 'obj .' to make it look less like a period?
{
  // Code here
}

I really think that's good enough.

I've become 100% convinced on the obj.{} subject, by the way. Of course, I can't make the decisions. I also don't think Fan will come to a grinding crash over its current syntax, even though I'm convinced it's not the best choice.

And going a bit deeper, the method name actually is blank on obj.{}.

For example, tied to a method we have obj.something {…} vs. obj.(nothing) {…}. Also, obj?.something {…} vs. obj?.(nothing) {…}. In the "nothing" case, no method is called. Rather, it just operates on the current object. Alternatively, you could view it as an implicit call to a noop method:

This noop(|,| withBlock) {
  withBlock.callContext(this.context)
  return this
}

As for, obj->{}, that's a slightly trickier case which would require more compiler fanciness. Imagine obj->method {}. Any code inside the braces would only have an Obj to operate on, since it wouldn't know the method being called. But compiler rules could say that the obj->(noop) {} just cheats and makes duck calls to whatever context.

Also, I think callContext is not the right thing. The method signature itself should state that the callback will be used as a with block and on what type. For example:

This noop(|(This)| withBlock) {
  withBlock(this)
  return this
}

I put (This) in parens to imply that it's a "with" argument rather than a normal one. I think this level of explicitness is good.

I'm not sure I fully understand the proposal(s). But some of the things which don't seem quite right:

• having two closely related, but subtlety different "code blocks" (closures and with-blocks) doesn't seem right - the idea of passing code around needs to be unified
• I also don't like the subtle differences b/w passable with-blocks and normal with-blocks
• I'm not sure how the easy case is easy - I have to manually remember to specify an extra parameter and remember to call out to the block?

I'm not sure how we are moving towards a simpler more general feature, instead it sounds like we're creating more shades of gray. I haven't gotten that ah-ha feeling yet.

My recommendation is the same as before (as my third comment on this discussion). I was just describing the theory justifying it (which I worked out after the fact). I feel it's extremely solid. Sorry I don't have time to try to do it better justice for describing it right now.

Summary is that I think the intuitive feel for using obj.{} actually has some serious meat behind it, too.

Maybe really quick summary:

1. As suggested by John, every "with" block is a closure callback. It just has special context (special handling of the first parameter).
2. The with block immediate follows the method call it's associated with (which is also the obj.method {} recommendation for closures since they're the same thing now). That's why the {} follows immediately in constructors for the with blocks (as in Type {} or Type.makeReallyCool {}).
3. For later with blocks on objects directly, there is no method call. Or rather, it's a phantom "do nothing but call the closure as a with block" method, so you say obj.{}. The phantom method call is between the . and the {. Tutorials don't need to teach it that way, but it's the formality behind the syntax.

And concerning the easy question, all constructors take an implicit final closure param which is implicitly called at the end of the constructor. Other cases (other than constructors and obj.{}) require explicit effort, but that's okay. You might want bind the "with" to something other than this, such as in John's examples.

Adding to the Wish List:

I'd like to not have to use & to curry a function. e.g.:

Num multiply(Num x, Num y) // |Num, Num -> Num|
{
   return x * y;
}

double := multiply(2) // |Num -> Num|

echo(double(5).toStr) // 10

Because there is no overloading in Fan, and because y has no default value, the above invocation of multiply in the definition of double is unambiguously a partially evaluated function. So it would be nice if it were auto-curried (seems like as long as we don't allow overloading, may as well derive the full value of it).

Although perhaps there are reasons why this is not allowed that I haven't thought of.

One more thing while I'm still thinking about it: Auto-Conversion.

If a compiler expects a type X, but is given a type Y that doesn't fit it, then currently an error will be generated. Very good. However, I think a small modification to this scheme would be extremely handy.

If a class defines a method X toX, then it's taken as a conversion function, which converts the class to the type X.

The obvious benefit is that you can use all classes in expressions expecting a Str, and toStr will automatically be invoked. However, there are other benefits as well: in numerical equations (no need to perform manual conversions), in types that have representations not in the type hierarchy (for example, widgets -- a label could be converted to a string, a text box could be converted to a number, etc.).

The corollary would be a static method named fromX, in the style of fromStr, which would enable you to convert from type X to the class. This would enable lots of really cool stuff, like generating XML representations of objects, as well as generalizing the notion of serialization. e.g.:

XMLNode xml := "<rect x1=\"1\" y1=\"2\" x2=\"3\" y2=\"4\"/>" // XMLNode.fromStr automatically invoked

Rect r := xml // Rect.fromXMLNode automatically invoked

I'd like to not have to use & to curry a function. e.g.

I think it should be explicit. Otherwise if you forget parameters, you'll get a partial application instead of a compiler error. Considering that forgetting parameters is such a common mistake, it seems like a small tradeoff to force use of the & operator.

One more thing while I'm still thinking about it: Auto-Conversion.

I think this has been brought up before. I have mixed emotions, but I'd have to say I lean against it. Seems like something easily exploited to make code harder to read and understand. I'm also worried a feature like this might interact badly with the implicit casting feature. So if anything I'd suggest a wait and see approach, something like could be added in the future without breaking any code.

I think it should be explicit. Otherwise if you forget parameters, you'll get a partial application instead of a compiler error.

You still get a compiler error, because it's quite unlikely the return type of the function has the exact same type as a curried function.

For example, in the above case, if you forget the last parameter, then what's being returned from multiply is not in fact a number (as will be assumed in subsequent code), but a function. This itself is sufficient to cause a compiler error, except in the sole case that the return value of the function is itself a function whose signature is identical to a curried version of itself (which is very improbable).

You still get a compiler error, because it's quite unlikely the return type of the function has the exact same type as a curried function.

I agree, although it is unlikely to be as useful an error message. More importantly there are a lot of APIs which just take Obj, which would happily accept the curried Func instead of whatever the real return was supposed to be.

Well, these cases of auto-curry are pathological:

// Type inference might delay where we see the error.
value := multiply(2)

// And collections even more so.
Obj[] values := [,]
values.add(multiply(2))

As for automatic type conversion, I really don't like it. For example:

Something a := getSomething
OtherThing b := a
a = b

You'd think that a was the same object it started out as, but with automatic type conversion, identity changes unexpectedly. Also, data loss and corruption happens way too easily because people abuse it in clever ways that are hard to track down.

Well, these cases of auto-curry are pathological:

I understand where you're coming from, but honestly, the issue is exactly like auto-casting, which Fan already supports. If you get a parameter wrong, you get a runtime exception (or, if you really know what you're doing, a failed testcase). With nice IDE support, curried methods would be color coded (mmmm, maybe tumeric, often used in curries), so you'd probably never make the mistake to begin with.

I prefer less syntax to more, especially when that less syntax is more powerful. I don't see any good reason to require & to curry a function when Fan's syntax does not permit the ambiguity that would require currying to be different than invocation.

Functional languages such as Haskell have no special syntax for currying, and they've done just fine.

You'd think that a was the same object it started out as, but with automatic type conversion, identity changes unexpectedly. Also, data loss and corruption happens way too easily because people abuse it in clever ways that are hard to track down.

In this case, it would be up to the API designer whether the benefits of auto-converting justified the risks involved (and I don't agree that your pathological example is a realistic one because no one would ever do this). Auto-converting, at least for primitives, already exists in JavaScript, and the feature is quite handy. C++ has its own version of this feature (overloading a conversion operator), which has proven similarly useful.

With a well-designed API, it allows you to safely eliminate lots of boilerplate when dealing with families of related types.

One more feature I'd like to add to the wish list: shared memory for all objects declared with a shared modifier. Albeit with one caveat: you can only access shared memory objects inside an atomic block.

atomic {
   // Can access shared memory in here
}

In other words, native support for Software Transactional Memory in Fan (ideally with pluggable backends, so some implementations can be distributed).

The semantics of strong STM are quite safe, and the performance, very high.

Actually, if Fan fully exposed the AST for all code, then it should be possible to implement STM using unified with blocks and dynamic compilation. Though I'm not sure what else might be necessary for a high-performance implementation.

With support for an Expression-like construct, you could also do conditional atomic regions:

atomic(in.bytesAvailable > 0) {
   // Can access shared memory in here
}

STM is the likely candidate for replacing message passing-forms of concurrency. But it's far too bulky and hideously difficult to use in most languages that have not been designed with STM in mind (in my opinion).

Functional languages such as Haskell have no special syntax for currying, and they've done just fine.

What???

foo (a, b) = a + b

-- let's try what you suggest:
bar = foo 1        

GHC responds with:

No instance for (Num (t, t))
  arising from the literal `1' at ...
Possible fix: add an instance declaration for (Num (t, t))
In the first argument of `foo', namely `1'
In the expression: foo 1
In the definition of `bar': bar = foo 1

Doesn't look too good. Let's see if we can fix it with explicit currying:

bar = curry foo $ 1

works :)

(In Haskell you normaly define functions in their curried form.)

I remember pain from automatic conversions in C++, but I have to admit I didn't live too long in that world. And I think auto-curry and auto-convert are very different from auto-cast. Auto-cast provides errors in the same place in the same way as manual cast and no real difference in behavior. I think auto-curry and auto-convert change things much more than auto-cast.

Helium, that's because you're passing a tuple. Functions are not usually defined in this way unless there is some reason for atomicity.

Instead, functions are defined as follows:

foo a b = a + b

All functions defined this way (and hence, most functions in Haskell) are implicitly curried. Haskell functions do not accept more than one parameter -- at most, they accept a single parameter. So the above is really shorthand for:

foo a = \b -> a + b

That is, foo a returns a lambda function that accepts b. So you can do things like:

foo 2 4
=> 6

But also:

bar = foo 2
bar 4
=> 6

In other words, no need for special notation when currying a function (actually it's called partial evaluation in the Haskell world; when imperative languages rip off features from functional languages, terminology often changes).

Tom, I wrote C++ for more than 10 years (close to 15, on and off), even writing several books on C++ programming. Overloading the conversion operator has numerous benefits in reducing the clutter of source code.

Clearly, I'm in the minority, but I do think it's significant that the feature exists in two languages I have used, as well as others I haven't used (C#, F#, Visual Basic, etc), indicating it's not the death trap you're making it out to be. It's a feature that's not used often, but when it's used well, it can greatly increase readability.

I don't necessarily think John's proposal are bad, but I'm not just ready to implement them. At this point Fan is quite concise, so I don't feel we need to make any trade-offs for additional conciseness at the expense of readability. I want time and experience with the current features before adding any more features into the mix. Like I said, both of these features can be added in the future without breaking backward compatibility.

JohnDG, that was my point, well at least part of it. In most functional languages you write functions in their curried form in most imperative functions you write them in their tupled form.

Haskell does not implicitly convert tupled functions into curried functions. & in Fan is like curry in Haskell only that it works for n-tuples rather than only for pairs.