#341 Uri Naming

brian Sun 24 Aug 2008

Part of the Fan vision is a unified architecture for naming based on Uris. Naming is one of the critical aspects of an architecture which often gets neglected. Naming systems which I consider inspirational:

  • file systems: a hierarchy of named directories and files; easily understood and served us well for decades
  • web: the URI encapsulates the location of resources in a simple string; URLs give us enough information to dereference the resource
  • rdbms: "identifying" a record is strictly based on SQL queries - a deliberate downplaying of "canonical identifiers"

A solid naming subsystem smooths the way for interoperability at the higher levels of the architecture. Couple use cases to consider:

The fwt::Command class should easily be initialized from a localized property file. But how do I specify the icon file? The icon might be bundled as a resource in a pod (like icons.pod), it might be on the file system, or it might be plugged in using some application specific mechanism. How do I capture that in a simple string?

A tool like Flux is based upon the notion of browsing Uris, resolving those Uris into Fan objects, then searching the type database for tools available to view/edit the resources. The naming APIs serve as the interface between the Flux tool and the infinite possibilities of plugging in resources.

The same model applies to the webapp subsystem. The naming APIs define the standard interface between web infrastructure code and how we plug in application specific resources to expose on a website.

Fan has standardized on Uris as the representation for names. They aren't perfect, but they cleanly map to both file systems and the web which is a critical requirement. One of the great principles of Uris is "any new space of identifiers or address space can be represented as a subset of URI space" - or put another way we can plug other naming systems into URIs (see URIs Axioms for WebArch for a great read on the subject).

One of the fundamental designs decisions I made at a early stage was normalizing the File API to use Uris. Also note that File is the standard type for representing any file like object including files in a pod, files inside a zipfile, or files on the web. This simplifies the upper layers of the stack - for example the FileWeblet can handle web plumbing like etags and last modified for any file - it doesn't care whether the file lives on the file system or inside a pod.

So I think we have a pretty good foundation, but there is still more work to do. The following is a list of tasks and my proposed solutions:

Scheme Handling

We need a mechanism to plug in scheme handlers for "file", "http", "https", "ftp", etc. I propose a new class UriHandler which is responsible for resolving a Uri into a Fan object. UriHandlers will be registered into the type database using the "uriHandler" facet:

@uriHandler="http"
class HttpUriHandler : UriHandler {...}

The Uri.get method will be enhanced to use UriHandlers. It is an open issue decide what happens if a Uri isn't absolute (doesn't have a scheme). Most likely I will assume "fan:" scheme (see below).

File Uris

In order to play nice with other Uris, file uris need to be implied to have a "file:" scheme. There will probably be some subtle semantic changes to the File APIs. For example File.normalize will add the implicit "file:" scheme for local files.

Namespace in a Namespace

There is a need to solve the general purpose problem of namespace in a namespace. The quintessential example is how do you identify a file inside a zip file? There is a well defined namespace inside a zip file, but it is orthogonal to how we identify the zip file itself. Unfortunately Uris don't handle this case well, which is a shame because it forces us to create some Fan specific solution (if you know of general purpose techniques please share).

To solve this problem in a general purpose manner we need to identify some special syntax which can be used to embed a nested scheme in a standard Uri which Uri.get can trap on. Ideally we'd use one ore more of the chars sub-delims production of RFC 3986 for this purpose:

sub-delims  = "!" / "$" / "&" / "'" / "(" / ")"
                  / "*" / "+" / "," / ";" / "="

I don't know exactly what the syntax should look like, but here some proposals:

// syntax
file:/dir/foo.zip!zip/dir/file.txt
file:/dir/foo.zip/(zip)/dir/file.txt
file:/dir/foo.zip&zip/dir/file.txt

// translates into
`file:/dir/foo.zip`.get + `zip:/dir/file.txt`

I like the first one the best because ! looks closest to the pipe symbol - I think about this problem as piping name spaces together. I think Java uses something similar for identifying files inside a jar file.

Fan Scheme

Anything not already in a standard scheme such as "file" or "http" would fall under the umbrella of the "fan" scheme. The "fan" scheme would map to the Namespace API. The Fan runtime will predefine a bunch of resources under "fan:/sys/". Applications can plug in their own namespaces or Uri/resource mappings.

These are my proposed predefined Uri mappings:

fan:/sys/pod/foo            => Pod.find("foo")
fan:/sys/pod/foo/file.text  => Pod.find("foo").files[`file.text`]
fan:/sys/type/qname         => Type.find("qname")
fan:/sys/service/qname      => Thread.findServce("qname"), already implemented
fan:/sys/homeDir            => Sys.homeDir
fan:/sys/homeDir/a/b/c      => Sys.homeDir + `a/b/c`

An example of mapping UI commands to icons:

cut.text=Cut
cut.icon=fan:/sys/pod/icon/x16/text-cut.png
custom.text=Custom
custom.icon=fan:/sys/homeDir/customIcons/foo.png

So that is my game plan - my first priority is the use case above (how to define icons with a Uri). That requires at least UriHandler, "fan:" scheme, and predefined mappings for "/sys/pod/...".

Comments?

katox Sun 24 Aug 2008

I'll dare to add a link to java URIs for jars.

In Sun JDK implementation inner classes are referenced as this

jar:http://www.foo.com/bar/baz.jar!/COM/foo/Quux.class$InnerClass

My further comments would be that URI schemes are nice but it is very easy to underspecify their meaning. Look at JNDI - it is basically one big ugly untyped map.

Resolving the reference object is one part, finding a correct handler is the other. Fortunately URIs are mostly opaque. Using an appropriate fragmentation you can delegate (and chain) its parts to various handlers seamlessly. There may be some differences besides locating a resource - like that a file on a filesystem is seekable but a streamed, ciphered or compressed file might be not - but that shouldn't matter. The harder part may be how to dereference and handle the resource having context and the relevant part of its URI. This strongly reminds the Java classloaders. A hierarchical delegation might be a neccessity - I'm not sure how would it go with the proposed simple facet.

There is also concern with versioning. If you refer a pod by some URI - which version of a pod you mean? Is it up to the particular component to get the correct one (knowning its preferences or dependencies)? Shouldn't the URIs as durable and absolute identifiers constist of such metadata?

brian Mon 25 Aug 2008

Resolving the reference object is one part, finding a correct handler is the other.

In the terminology of my post, UriHandler (which actually I'm going to rename UriScheme) is responsible for resolving a Uri to a Fan object. This tends to be a protocol issue, where most schemes like "file", "http", and "ftp" would by definition resolve to a sys::File.

How you work with that file is the next step and might be application specific. This is where Fan's type database comes into play since it easily lets us creates type relationships. In Flux you typically bind a viewer/editor based on mime type:

@fluxViewMimeType="x-directory"
internal class DirView : View

@fluxViewMimeType="image"
internal class ImageView : View

@fluxViewMimeType="text"
class TextEditor : View

If you refer a pod by some URI - which version of a pod you mean?

In a given Fan installation there is exactly one active version of a pod loaded into memory at a time so this isn't an issue. I just went looking for the docs on the subject and discovered that I've been remiss in documenting dependencies. I'll fix that, until then the best doc is sys::Depend.

katox Mon 25 Aug 2008

In a given Fan installation there is exactly one active version of a pod loaded into memory at a time so this isn't an issue.

This is certainly not an issue in a small installation. In bigger systems (such as in Java or OS) such an approach poses a problem.

Let's say we have pods parser-1.0 and net-1.0 which our Application A depends on. Let's say we develop some functionality above them.

net-1.0  <-|
parser-1.0  <-- A-0.1

Then we happily develop a new version of our Application where we upgraded to net-2.0 becouse it has more powerful API. The graph should look like this:

net-2.0  <-|
parser-1.0  <-- A-0.2

But what if net-2.0 started to use the parser pod internally? They might have chosen an incompatible version parser-1.1.

parser-1.1 - net-2.0.1  <-|
          parser-1.0    <-- A-0.2

Nothing changed from perspective of A-0.2. It should still be able to use it's old parser (maybe parser-1.1 is buggy but that doesn't have to be a problem for net-2.0 which uses it lightly).As of A-0.2 we have two choices - rewrite A to support parser-1.1 (might be a problem) or to stick with net-1.0 (also might be a problem).

One version of a pod creates a problem becouse it breaks encapsulation. Application A should have no knowledge about net internals. Moreover, forcing a global change is unnecessary and may lead to errors.

I think it is safe to allow multiple versions of one pod loaded at a time if dependencies are managed properly. To avoid duplication we can let our application depend on an interval of version (or we can say "any or grater"). Unused pods (not referenced by a service a or by an other living pod or loaded directly) could be garbage collected.

"foo 1.2-1.4" Any version between 1.2 and 1.4 inclusive

I'd suggest to change it to exclusive. Let's say there is a stable version 1.3.x.x (sustaining) and there is 1.4 (experimental). We may want our application depend on any proper 1.3 bugfixing version. Since we don't know what will be the last 1.3 release it is safer to specify 1.4 exclusively. If this is not a an option we would be forced to upgrade our dependencies with every release or to specify some version "large enough" to take care of this.

brian Mon 25 Aug 2008

Katox, let's keep this thread on Uri naming. I've created another thread to discuss pod versioning.

andy Mon 25 Aug 2008

I vote for the ! for the delimiter. And I think your proposal for the fan: scheme all looks good.

Login or Signup to reply.