#290 Obj wildcards

brian Wed 16 Jul 2008

Today I allow Obj to be used as a wildcard.

But I'm wondering if that feature shouldn't be expanded.

We could expand it a lot and say that any upcast is implicit:

Num num := 4
Int int := num // ok

That seems a bit too extreme though.

But I'm thinking that we should expand this feature a little and allow Obj[] to be assigned to any list type. This really jives with my emptyList proposal which would require a cast today. But I've run into a couple times now.

tactics Wed 16 Jul 2008

I assume that this is how dynamic trap calls are implemented? (I had a suspicion it was an automatic upcast).

brian Wed 16 Jul 2008

I assume that this is how dynamic trap calls are implemented?

I wouldn't quite say it is how dynamic trap calls are implemented, they are implemented via Obj.trap. But I get what you are asking - yes, it is how they plug back into the static type system, since Obj.trap always returns Obj, we insert a implicit cast when needed. For example:

// this code
if (foo->bar) {...}

// compiles to
if ((Bool)foo.trap("bar")) {...}

It wasn't originally that way, so it made dynamic calls a pain in the butt to use. It seemed quite logical to wildcard Obj, kind of like C wildcards void*.

tactics Wed 16 Jul 2008

Yeah. I meant how the typing was implemented.

tompalmer Wed 16 Jul 2008

I would really, really, really love to have automatic up and downcasting everywhere. I know it carries risks of CastErr, but I'm extremely willing to accept that. (And I think it especially fitting in a language flexible enough to allows duck calls.)

brian Wed 16 Jul 2008

I would really, really, really love to have automatic up and downcasting

I'm not sure I know what automatic downcasting means - that is already ok right?

Int i := 4
Num n := i

Automatic upcasting everywhere is tempting, but I do think that might be going too far in the dynamic direction. It would be a bit better than Ruby/Python because you would get a runtime error at the cast location versus a the duck typed method call. And you could check incompatible types at compile time.

I could see myself getting convinced with the right arguments, but right now I'd have to say I'm against it.

tompalmer Wed 16 Jul 2008

I think maybe you use "up" and "down" the opposite from me, so I was just talking generally.

I think automatic downcasting (deeper into the class hierarchy) would be good because (1) you don't have generics to take care of it for you and (2) people usually know what they are doing anyway.

In duck typed languages (and in old-timey Java), there is rarely an issue with getting the right type into the right place. Documenting those types and knowing what's going on after the fact is sometimes tough in duck typing, but there aren't usually problems with getting things where they need to go.

If you can look at the type that's expected (easy in code anyway in Fan, and especially easy with an IDE) then I don't see a need to tell the compiler what it and I both already know anyway.

JohnDG Wed 16 Jul 2008

Automatic upcasting everywhere is tempting, but I do think that might be going too far in the dynamic direction.

If you don't autocast, then the developer's just going to do it manually. You won't be introducing more errors, just making the developer's life easier.

Static type checking prevents you from doing stupid things like trying to cast incompatible classes. It doesn't prevent you from casting. So it makes sense to do autocasting everywhere, I think.

jodastephen Wed 16 Jul 2008

I support the original idea of Obj[] being wild. That seems to match Obj.

On more general auto-casting, I'm still skeptical. One case where it can be justified is when it is guaranteed:

if (num is Int) {
  // should be able to treat num as an Int here without a cast
}

tompalmer Thu 17 Jul 2008

I don't see the evidence in practice for any concern. I think of explicit casting as being like checked exceptions but not quite as bad. They sound meaningful on paper but don't matter so much in the wild.

Again, people survive Ruby, Python, and JavaScript just fine in text editors. I think Fan (especially in an IDE) has nothing to worry about.

tompalmer Thu 17 Jul 2008

Or maybe a different way to look at it. How is automatic casting from Obj to Int safer than automatic casting from Num to Int? Why should giving the compiler more information make it less reliable?

(I'm assuming we're all against allowing casts directly, say, from Int to Str here. Just to make sure we're clear on that.)

brian Thu 17 Jul 2008

OK, I will propose something more ambitious. Let's say this:

  1. Anytime where Type A is used and Type B is expected:
    1. If A.fits(B) the call is ok as is (current behavior)
    2. Else if B.fits(A) then we insert an implicit cast (to keep JVM verify happy)
    3. Else it is a compile time error
  2. We remove the cast operator from Fan since it is no longer needed

Or another way to say it, if it is possible that a given type could be the required type then we implicitly cast. If we know it is impossible it is a compile time error.

Example:

Void f(Int x) { ... }

// implicit cast inserted: Int.fits(Num)
Void a(Num x) { f(x) }  =>  f((Int)x)

// compile time error: !Int.fits(Str)
Void a(Str x) { f(x) }  =>  f((Int)x)

This is a pretty easy change to make, but a pretty big change to the language's type system.

Comments?

JohnDG Thu 17 Jul 2008

Most excellent. And should lead to no more errors, since, as I mentioned before, a developer will simply manually cast if you don't do it for her.

Makes the language much cleaner to use and removes the need for generics.

tompalmer Thu 17 Jul 2008

Very glad to hear the autocasting rules.

Still, getting rid of casting entirely is brave. There are cases where ((Something)blah.getSomeObj).doSomething is handy. Now, I've always hated that syntax in Java, and maybe forcing an intermediate assignment makes for cleaner code, but this use case ought to be considered before getting rid of casts.

I like the syntax (blah.getSomeObj as Something).doSomething better (fewer parens), and I doubt you want to get rid of as since you like it for certain styles of coding, but in this case it is misleading as we get a NullErr instead of the meaningful CastErr. So, we either need to (1) change the meaning of as or (2) retain (Type) casts or (3) be willing to swallow the need for intermediate assignment.

Maybe doable. Just want to make sure the issue is considered.

brian Thu 17 Jul 2008

There are cases where ((Something)blah.getSomeObj).doSomething is handy.

You're right about that. I don't really like it, but that is a good reason to keep the cast operator.

I agree as looks better but reports a less useful exception.

JohnDG Thu 17 Jul 2008

I don't think the proposal is to eliminate all casting, just casting when the type of an expression is known. There are lots of cases where it isn't. For example, if you do Str n := object, then the compiler knows the target type is Str and can insert the required cast. But if you do, Str n := object.foo(), then the compiler doesn't know the type of object (being of type Object), and therefore cannot automatically cast into whatever interface or class provides the method foo.

In general, there might be many kinds of interfaces/classes that a method with signature foo, so the problem is not decidable. Which forces you to do dynamic typing (in the above case, use -> instead of .).

This holds even for the example ((Something)blah.getSomeObj).doSomething. The compiler won't know what to do if you don't insert a cast.

brian Thu 17 Jul 2008

I don't think the proposal is to eliminate all casting, just casting when the type of an expression is known.

I agree with you and Tom. Assume my proposal amended.

So is there widespread agreement to implement implicit casting this way? I'm surprised to not hear counter arguments.

jodastephen Thu 17 Jul 2008

I remain a little skeptical, but don't have strong arguments on the subject. That is to say, my gut suspects that this won't be ideal but I have no proof. The exception is within if (obj is Point) type statements, when everything is clear anyway.

andy Fri 18 Jul 2008

Obj[] wildcards make perfect sense (and would definitely make things like List.map easier).

I'm not sold on the "auto-casting" though. That seems way to easy to neglect and end up with unexpected runtime exceptions.

JohnDG Fri 18 Jul 2008

That seems way to easy to neglect and end up with unexpected runtime exceptions.

I don't agree. Think of it from the perspective of a coder. They type:

StyledFrame foo = |

where | denotes the caret. Now they already know in their mind what they are going to do on the RHS -- in this case, let's assume they're doing widget.getContainer().

If the language doesn't support auto-casting, then the very next tokens they're going to type are as follows:

StyledFrame foo = (StyledFrame)|

This is boilerplate because the compiler already knows the type of the variable (and while it's true in this case, there's no substantial savings because of type inference, as part of an expression or parameter there would be).

Now the developer can type what he really wants to:

StyledFrame foo = (StyledFrame)widget.getContainer()

In other words, he's going to insert the cast anyway. If there's going to be a runtime cast exception, then it will be there in any case -- with or without the automatic cast. Why not just let the developer write:

StyledFrame foo = widget.getContainer()

brian Fri 18 Jul 2008

I totally agree with you John, although even today you could write that as:

foo := (StyledFrame)widget.getContainer()

Not quite as nice as this though:

StyledFrame foo := widget.getContainer()

Where this really comes in helpful is in assigning to existing local variables or passing expressions to methods. But at the same time, those cases aren't quite so obvious:

Void something(StyledFrame f) {...}

something((StyledFrame)widget.getContainer)  // today
something(widget.getContainer)               // with this feature

Those cases are a little more non-obvious than declarations. But I've definitely convinced myself this feature is a good thing - it strikes a nice balance.

andy Fri 18 Jul 2008

That is the simple case I agree with - my argument was when you're multiple calls removed (assume these methods are defined all over the place):

Void a(Num x)
{
  ...
  b(x)
}

Void b(Num x)
{
  ... 
  c(x)
}

Void c(Int x)
{
  ...
  c.isAlpha
}

a(1.3) -> NullErr

Assume you want or need to fail gracefully, and accidently never check the type, since you don't have to cast. The exception may only manifest itself at runtime with certain input.

Though in thinking through my example, I think that is a developer issue, and shouldn't prevent the more common useful case we're enabling.

tompalmer Fri 18 Jul 2008

And you'd actually get a CastErr, not a NullErr. In any case, I agree that even given examples like this (to be aware of issues), the feature would be okay.

jodastephen Fri 18 Jul 2008

I think I am opposed to this. The cast operator is a signal for coders (especialy maintenance coders) as to where a CastErr has occurred.

I am especially opposed for passing arguments to methods. Its none obvious, and I believe that it will make writing and using a smart editor harder, as the number of possible parameters for the method has been increased.

Random idea - how about a cast keyword:

Void something(StyledFrame f) {...}

something((StyledFrame)widget.getContainer)  // today
something(cast widget.getContainer)          // with the keyword

This survives refactoring better, and is just as clear (if not clearer).

JohnDG Fri 18 Jul 2008

I think I am opposed to this. The cast operator is a signal for coders (especialy maintenance coders) as to where a CastErr has occurred.

That's reflected in the stack trace -- and if line number isn't sufficient, then future versions of Fan might incorporate column number as well.

Dynamic languages have shown that runtime exceptions just aren't an issue, even for very large code bases. And this proposal is extremely modest, falling far short of dynamic languages, merely saving developers tiresome typing in cases where they would already be inserting a cast.

tompalmer Fri 18 Jul 2008

And as I've mentioned before, IDEs would make it easy to mouse over and remember expected types. (Also, for those concerned enough, there could possibly be a feature to highlight automated casts like this. I doubt I'd use the feature myself, though.)

jodastephen Fri 18 Jul 2008

I hear these arguments. But I've also heard them before with auto-boxing in Java. And that introduced a plague of unexpected NPEs in code that doesn't look like it could generate them.

merely saving developers tiresome typing in cases where they would already be inserting a cast

This is all about the reading of code, not about the writing of code.

BTW, I agree with the general statement that runtime cast errors are rare.

brian Fri 18 Jul 2008

One thing I really liked from Eiffel was the ?= operator which was basically an unsafe/cast assignment:

Foo f ?= notFoo()

That doesn't quite work as cleanly in Fan which has both := and = assignment. Plus Eiffel didn't really solve the problem for method calling. You might be able to do something like:

func(?notFoo)

But then again just having the compiler generate the implicit case seems a lot cleaner and more elegant.

I agree that a refactoring could introduce an error which isn't caught. But in order for that to happen you would have to change a parameter to be a more specific type, but be calling the method with a another sub type. That seems really rare and ok enough to leave to a runtime exception (as a tradeoff for the niceties this feature would bring). I would say a very high percentage of method signatures involve things like Str, Int, Bool, Enums, etc - these would almost always be caught at compile time.

And I agree we haven't lost any static type IDE support (which could be configured to show those locations).

It is really things like this we are trying to make easier (to write and read):

// today
funcWithSql((SqlService)Thread.findService(SqlService.type))

// with this feature
funcWithSql(Thread.findService(SqlService.type))

Right now I'm tempted to make Thread.findService return Obj instead of Thread to avoid the casts which will occur 100% of the time in an API like that.

I can easily see both sides of this argument. But I would say right now I'm one step across the fence in favor of this feature. I think it makes code a lot easier to read and write by trading off only a small loss in compile time checking versus runtime checking.

helium Fri 18 Jul 2008

// today
funcWithSql((SqlService)Thread.findService(SqlService.type))

// with this feature
funcWithSql(Thread.findService(SqlService.type))

// with generics it would be even shorter and verifyable :p
funcWithSql(Thread.findService<SqlService>())

SCNR

I didn't have many cast exceptions in my career so I don't think this will cause many problems, but perhaps that's because I don't have many casts in my code, but perhaps that's because I don't program much Java.

jodastephen Fri 18 Jul 2008

Thread.findService looks like a classic case where generics are used in Java. So, this is kind of an argument about whether we define the result of that method more closely, or allow more loose cast-free conversions in general.

Option 1: Link the parameter to the return type. I tried some code samples, and didn't like them - generics aren't Fan's style

Option 2: Just allow the casts:

Service findService(Type type) { ... }

Service s := findService(SqlService.type)
SqlService s := findService(SqlService.type)

Option 3: The method must declare that auto-casts are valid:

(Service) findService(Type type) { ... }

Service s := findService(SqlService.type)
SqlService s := findService(SqlService.type)

ie. the brackets around the return type tell the compiler that it can auto-cast to subtypes in the calling code.

I think I quite like the control that Option 3 gives, together with a cast operator. But is the extra complexity worth it? Maybe it isn't, I'm still on the fence, but just on the cautious side.

alexlamsl Sat 19 Jul 2008

I guess I'm being thick, but I don't seem to understand the need for upcasting beyond existing Obj wildcards. Almost all of the time when I have a need for casting is on return value from another method, and Fan's duck-typing would already provide the equivalence of upcasting.

Moreover, Obj wildcards would already cover cases like:

A a := session.getAttribute("a");

brian Sat 19 Jul 2008

Moreover, Obj wildcards would already cover cases like:

Only if session returns an Obj and nothing more specific with current design.

I guess I'm being thick, but I don't seem to understand the need for upcasting beyond existing Obj wildcards.

The need is that often you still have APIs which return a non-specific type which isn't Obj:

Thread Thread.findService
Obj[] List.map
Obj[] Type.emptyList // proposed

The same thing occurs for method parameters. The point is that APIs like these return something similar to an Obj, but also know it is a more specific class. So it seems to make sense to expand Obj wildcard to include any implicit casting anything which isn't known to be an incompatible type.

I think I quite like the control that Option 3 gives, together with a cast operator. But is the extra complexity worth it? Maybe it isn't, I'm still on the fence, but just on the cautious side.

I love the ideas, but in this case I don't think the extra complexity is worth it. I'm still in favor of the feature according to my original proposal (but keeping cast operator).

tompalmer Sat 19 Jul 2008

Right now I'm tempted to make Thread.findService return Obj instead of Thread to avoid the casts which will occur 100% of the time in an API like that.

That's a really good reason to automate casting everywhere. If only the extreme case is allowed (i.e., for Obj) then it could encourage less clear APIs. I call it the extreme case because if we're concerned about safety, I think it's obviously the least safe (though perhaps most common need) of all.

That we don't stress out over Obj wildcards (or maybe I've missed that in the conversation above) also shows that we're really not that concerned.

brian Mon 21 Jul 2008

This feature is implemented for next build according to my proposal:

OK, I will propose something more ambitious. Let's say this:

Anytime where Type A is used and Type B is expected:

  1. If A.fits(B) the call is ok as is (current behavior)
  2. Else if B.fits(A) then we insert an implicit cast (to keep JVM verify happy)
  3. Else it is a compile time error

Or another way to say it, if it is possible that a given type could be the required type then we implicitly cast. If we know it is impossible it is a compile time error.

Example:

Void f(Int x) { ... }

// implicit cast inserted: Int.fits(Num)
Void a(Num x) { f(x) }  =>  f((Int)x)

// compile time error: !Int.fits(Str)
Void a(Str x) { f(x) }  =>  f((Int)x)

tompalmer Mon 21 Jul 2008

Sweet.

alexlamsl Tue 22 Jul 2008

Sounds pretty handy!

Login or Signup to reply.