I have a few questions after reading this page: http://www.fandev.org/doc/docLang/JavaFFI.html

I'm considering a few things in Fan that will need to cooperate with some java code. Also I'm considering rewriting the Netbeans plugin i'm making in Fan, cause that would just be cooler :) and a good learning experience.

Anyway here are the questions:

1. What the minimum JVM version required to run compiled fan code.
2. It says java classes can be subclassed only 1 level, is that still the case ? will it change ? and why ? Is that just for performance reasons or am I missing a larger issue. That seems quite limiting.
3. It also says you can't override methods that are overloaded in the Java class. That one might be the most troublesome to me, because it's pretty common for Java classes to use overloading. Why is it ?

Thanks

I'm considering rewriting the Netbeans plugin i'm making in Fan

I think that is a great idea. Plus it will much better for sharing code.

What the minimum JVM version required to run compiled fan code.

Today I use version 49 of classfile which corresponds to Java 1.5. Although I've designed Fan to be easily backported to J2ME which is 1.2.

It says java classes can be subclassed only 1 level, is that still the case ? will it change ? and why ? Is that just for performance reasons or am I missing a larger issue. That seems quite limiting.

It is because Java and Fan have different constructor mechanisms. Java has overloaded constructors with same name as class and Fan has named "factory" style constructors. I allow you to call Java constructors via Fan shorthand syntax because I know it is a Java class. But once you subclass 1 level deep you have a funny 1/2 Java/Fan constructor setup which won't work.

It also says you can't override methods that are overloaded in the Java class. That one might be the most troublesome to me, because it's pretty common for Java classes to use overloading. Why is it ?

It is common to use and you can call them. What I don't allow is subclassing where you have to override them. Why is because Fan doesn't allow overloading. Do you have a specific use case where you need that?

None of these problems is insurmountable, but they will probably require some inelegant solutions. So I'd rather not tackle them until we have clear use case for them.

I ran into it a few weeks back trying to override ArrayList.add(). It's not hard to imagine a variety of validation checks, etc. that could happen there. It's a little contrived in this case (I was just playing around) as I don't ever recall really needing to do that even in java code. But in, say, swing code, overriding and subclassing are pretty common techniques. While there's FWT that might mitigate the need to write swing code, with the netbeans plugin, I can imagine it won't be long before this is a problem. That's not to say it's something that needs to be fixed in fan, though. One could always write an intermediate subclass in java that calls to a new abstract method that eventually gets implemented in fan. Not exaclty elegant but it would probably work. Seems like I had to do something like that in tradewinds now that I think about it.

My main concern was if i try to use it with sap.

They use struts(meh), and the base method to implement(execute()) is itself overloaded. http://struts.apache.org/1.x/struts-core/apidocs/org/apache/struts/action/Action.html

Struts is pretty common and there are others frameworks & API's (incl swing as cheeser said) that do this kind of stuff as well.

I was thinking of doing a Java "wrapper" as cheeser mentioned, but that would get old very quick if i have to make one for each action i want to extend.

I had not noticed that fan doesn't support overloading until now, so i guess it's a big part of the problem.

Anyhow, i don't need this yet, so it can wait for now.

Maybe a facet (such as @overload="execute" here) or facet + name-prefix-matching (such as @overload ActionForward executeBlah(...)) for interoperability with other systems could be an FFI workaround. Personally, I love that Fan doesn't support overloading, but I can see the interop concern. So this might be a compromise.

Or even just allow overriding the most specific parent method (without any facet) could happen to cover here, since you almost always want to use the HTTP version in Struts here.

Most specific would make sense. On the other hand your facet solution might be easier to implement and would probably be more flexible (maybe it could allow to deal with extending abstract methods too). I could live with that.

My intention is never for Fan to be a complete replacement for Java - I think Java will always be the "assembly language of the JVM". However I think all the limitations addressed by tcolar need to solved at some point.

For me this is a question of priorities for getting 1.0 into shape. Are these features required for a successful 1.0?

If you need features like these to use Fan in real world production code, then I'm pretty motivated to help you. Production experience with Fan is really important right now as we prepare for a beta period.

The fact that it's not a replacement only means the integration is more important.

I think i can live without those features for now however.

As far as whether you need it for Fan 1.0 i don't know. You might for "marketing" :) reasons.

From what I've seen most ppl looking at Fan/Scala/Clojure are probably coming from Java/.NET, and I've read many of them where driven to those languages because it means they can reuse existing code/libraries.

From my experience as a long time Java guy(10+ years), I prefer Fan over Scala & Clojure because of it's clean and familiar syntax.

Scala is a bit too complex for my taste and they push functional programming to much in my opinion (not that it's bad, but that's a big scary step from Java).

I feel Fan is an easier, cleaner transition from Java (it looks like Java but you can at your leisure start using closures then functions and so on at your own pace).

Anyway what I'm trying to say is that if the main target actually are people migrating from Java/.Net, then the FFI might be a very important point to those people when comparing with say Scala or Groovy.

Look at me, I sound like a salesman :)

Promoted to ticket #670 and assigned to brian

Well no matter what we should probably track those two features as something we need to do:

1. It says java classes can be subclassed only 1 level, is that still the case ? will it change ? and why ? Is that just for performance reasons or am I missing a larger issue. That seems quite limiting.
2. It also says you can't override methods that are overloaded in the Java class. That one might be the most troublesome to me, because it's pretty common for Java classes to use overloading. Why is it ?

The overload problem is probably more important of the two and is quite similiar to 604 - potentially use the same solution.

Renamed from Java FFI questions to Java FFI enhancements

The overload problem is probably more important of the two and is quite similiar to 604 - potentially use the same solution.

I was thinking the same thing, which is why I mentioned the prefix option. I think the facet might be more important here, though. For operators, there's just a fixed set of names to consider, and they'll be well known to experienced Fan programmers, but for overriding foreign code, any old name could pop up. But maybe the override keyword is really enough to guess what the programmer meant. Maybe no @overload is needed. I'm not sure.

I agree that for adoption reasons, not being to overload like that is going to bite people more often than not. It would definitely make for better marketing/adoption to have it in 1.0. But perhaps if it were on the timeline for a 1.1, that might be sufficient, too.

Right now I'm trying to write a Fan "wrapper" to a jar file (Sap Java connector) so i can then use easily within Fan projects without having to put FFI's all over the place.

Anyway, what i'm wondering is where to put the jar, the FFI docs says it needs to be in the JVM path or system classpath. Neither is practical, I would want the jar to be be distributed with the pod.

Is there a way to "reference" the jar ? Maybe something like:

@podLibDirs=[`java/lib`] or @podDepends = [`java/lib/my.jar`]

Is there a way to do this now ?

If not I suppose I could manually mess with the classloader, something like: http://weblogs.java.net/blog/malenkov/archive/2008/07/how_to_load_cla.html

As of now I think the Fan runtime adds its /lib/java/ext/{system} dir to the load.library.path, so you have to put jars (and dlls) there -- that's what I'm doing with Jogl.

I guess if you're providing a launcher for your app, you could set up the classpath to be whatever.

I think this is still being worked out, far as I know that's what it is now.

I see Fan java as it's own classLoader FanClassLoader, i assume all classes are loaded through it.

If so it would be nice to have a loadJar(File) method in it to be able to load jar at runtime. that's fairly easy.

Would it make more sense for a jar dependency to go directly inside the pod, or "next" to it ?

Am i missing something obvious ?

I'd think, wherever possible pack up your resources inside the pod.

I doubt if there's now a classloader hook -- could be an interesting experiment.

I think if you just put your classfiles into the pod zip file that the class loader will try to load them (although it might filter out classes which don't begin with "fan/" - although if you remove that and it works well, we can add that patch).

But the intention right now is put those jars into "lib/java/ext". There are many cases where you want to install files into specific directories during deployment (such as etc files now). So my intention is that we solve the generic problem when we get to cloud based pod repositories.

Yesterday i was experimenting with this and hacked together some code that works though i needed to add a 3 lines definition in FanClassLoader (to give access to definClass() which is protected).

I don't know if that's the best way to do it, but i was thinking this coupled with something like javaRuntimeLibs := [`path/to/myjar`] in pod.jar would do the job (boostrap would call the following registerJar() method with javaRuntimeLibs items).

Anyway it's not high priority but i think it's kinda useful, and probably better than trying to mess with the classpath in each OS boot script.

using [java] fan.sys::FanClassLoader
using [java] fanx.interop::ByteArray


**
** LoaderTest
**
class LoaderTest
{
    
   static Void registerJar(File jar)
   {
       echo("Will register in classloader: $jar")
       Zip zip := Zip.open(jar)
       [Uri:File]? contents := zip.contents
       contents.each |File file, Uri uri|
       {
           path := uri.toStr
           if(path.endsWith(".class"))
           {
                Buf bytes := file.readAllBuf
                className := toClassName(file.toStr)
		echo("name: $className")
		ByteArray classBytes := toByteArray(bytes)
/*
That's what I can't do without adding a method to FanClassLoader, basically it's just a public pass-through to defineClass() which is protected.
*/
                //FanClassLoader.registerClass(className, classBytes);
           }
       }
   }

   // not very efficient but couldn't find anything better
   static ByteArray toByteArray(Buf bytes)
   {
       javaArray := ByteArray(bytes.size)
       for(i:=0 ; i!= bytes.size; i++)
          javaArray.set(i, bytes.read)
       return javaArray
   }

   static Str toClassName(Str classPath)
   {
       Str name := classPath[0..-7] // clip .class
       name = name.replace("\\","/").replace("/",".");
       if(name.startsWith("."))
           name = name[1..-1]
       return name
   }

   static Void main()
   {
       // test
       jar := File(`/tmp/ant.jar`)
       registerJar(jar)

       // now we can load the classes
       theClass := FanClassLoader.loadClass("org.apache.tools.ant.taskdefs.AntStructure ")
       echo(theClass)
   }


}
pre<

I think the proper way to do it if we wanted to let the Fan class loader load arbitrary Java classes would be have the pod declare the packages it supports so that the classloader knows where to resolve things from. But it would be rather expensive to setup. The reason it is so efficient now is that we can look at the class name and figure out the pod "fan.{pod}.{type}". I'll have to think about it.

But if you can just make calls to defineclass() you can just define(using defineclass not loadClass which needs to resolve) a whole jar(or multiple) in the classloader (basically regsiter all the dependenices), and then it should be able to find them without any modification.

So what that does is that I separately register class/jars into the classloader at runtime(or boot time ideally), so the FanClassloader does not even have to look for them itself since i just told it where it is and gave it the class code it needs.

Am i missing something ?

But if you can just make calls to defineclass() you can just define(using defineclass not loadClass which needs to resolve) a whole jar(or multiple) in the classloader

not sure I follow that, you mean you want to explictly load every class from a jar manually?

usually class loading is a callback model where class loading is done lazily

Forget what I said before, that wasn't quite right.

If fanclassloader extended urlclassloader(subclass of secureclassloader) then we would have the addUrl (jar) method available

http://www.colar.net/jotdoc/jdk1.6/index.html?page=java/net/URLClassLoader-source.html

Anyway it's not too important.

If fanclassloader extended urlclassloader

I do think I need to change this, my understanding is the UrlClassLoader would be a better base class (although the reasoning escapes me)

Yeah, I'm sorry my loadClass vs defineClass stuff was confusing and mostly wrong.

But assuming you extends URLClassloader instead of SecureClassLoader and just give a public passthrough to addURL(), that would be quite flexible.

URLClassloader is nice because it has all the smarts to find the classes given some url's so you don't need to manually implement that part.

I think this makes sense -- even with the working repo, if we were to make libraries loadable from the {working}/lib/java|dotnet|whatever/(system} -- it'll quickly get unwieldy, things stepping on each other or at least not being obvious what they belong to.

Making the repo be the unit of distribution might make sense -- a repo would be a few related pods, maybe some java libs, and you'd start your app with fan_home pointing at the fan distro, and fan_repo pointing to the app. Then there's no worries about applications colliding, they live in different directories, run in different VMs.

That ought to continue to be possible -- but there probably needs to be the focus on the pod as the unit that packages up everything it needs, to run. That seems like the right way to let everything interoperate, except in the case of, say, large shrinkwrap applications, or special customizations of the system.

So anyway, I'm thinking TColar's classloader suggestion sounds useful for short-term at least, and like Brian says we need to solve the larger issues with the whole cloud thing.