Bob Balaban's Blog

     
    alt

    Bob Balaban

     

    In Theory...

    Wiring JavaScript libraries Into the Domino Server

    Bob Balaban  December 6 2007 12:52:35 PM
    Greetings, Geeks!

    I have mentioned Dojo in this blog a couple of times. Nononono, not that Dojo, this Dojo. For example, here, and here.

    The many, many responses (a LOT of responses) were all interesting, and a few patterns emerged, such as (I am paraphrasing here):
    • Dojo is great
    • Dojo sucks
    • My favorite JavaScript framework is [insert name here]
    • Watch out for browser dependencies
    • Handhelds don't support Javascript, so what are you going to do?

    And so on. One principle I came away with was the following: no matter what JavaScript framework/libraries we ship with some as-yet-undisclosed version of Domino (what I've been referring to as NMFR - Next Major Feature Release), we better be really sure that we don't force our beloved Developer community to use it -- we MUST leave the door open for people (Developers are people too, you know) to be able to plug their own favorite JavaScript stuff in, even if it requires more typing for them than if they used our selection. This seems eminently fair and sensible to me.

    We've had a number of design discussions within the WebApps team back at Domino HQ, during which a couple of major points relating to this dual-headed objective became apparent:
    • We should probably create a new sub-directory in the ...\data\domino tree for JavaScript code, such as Dojo (as one example)
    • We should create a short-cut URL that the Web Server will understand to that directory (it would also be nice if that short-cut location were admin-configurable)
    • Since all of the JS code simply sits on the server's disk in a designated location, it would therefore be easy for people to add their own fave raves. They have to figure out how to do that on all of their servers, of course.
    • We should make it as easy as possible for a Developer to add a reference to the main "entry point" in their preferred JavaScript lib on each page of their Web app. The "main" entry point is generally the one that bootstraps the loading of the JS lib into the browser. This would normally be done in Domino Designer.
    • Once that reference is inserted, the Developer needs to be able to easily enter whatever other references (to widgets, for example) are needed, also done in the Designer environment
    • If Developers just so happen to be using the JS stuff that we select as our "default", Designer would do a bit more of the work for them. Otherwise, Developers will still be able to use "pass-through HTML" to do what they need to do.

    This seems reasonable to me. Well, of COURSE it does, I was in on all the discussions! But YOU weren't. So, fellow Geeks, please tell me what you think of this idea.

    Thanks! (in advance)

    (Need expert application development architecture/coding help? Contact me at: bbalaban, gmail.com)
    This article ©Copyright 2009 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.
    Comments

    1Andy Burnett  12/08/2007 7:22:18 PM  That sounds like a splendid idea

    Reducing the overhead of wiring in a library would be a really useful addition. Go for it!

    2Nathan T. Freeman  12/08/2007 10:06:20 PM  Wiring JavaScript libraries Into the Domino Server

    Can you define "pass-thru HTML" in this case?

    If you're talking about the case that it is today, then you aren't giving developers much. If you're talking about some different ways to modify the output HTML of the Domino server (such as some post-HTML rendering model, and computed HTML attributes on every Designer object that generates an HTML tag, whether in read or edit mode) then you're empowering developers, even if they aren't using your built-in toolkit.

    3Matt White  12/09/2007 6:22:04 AM  Wiring JavaScript libraries Into the Domino Server

    That sounds like exactly the way to go. It gives the right combination of easy integration of the default framework plus the flexibility of other frameworks if the developer chooses to use them.

    4Wild Bill  12/09/2007 6:40:39 AM  Wiring JavaScript libraries Into the Domino Server

    It *does* sound exceptionally fair and reasonable.

    Based on some months work with Frameworks (jQuery in our case), a few more for the 'to-do' list.

    1. Allow HTTP cache header construction for static items - such as design elements, and of course files on the hard drive. Currently Domino does NOT do this, and has to be coerced.

    2. Get the domino server to automatically gzip compress static elements (such as file attachments and static files on the hard drive).

    Combine these two, and all of a sudden, the clients DONT have to download 150-450k of JS code on each page refresh.

    (Yes it can be done manually, but thats hardly 'kick-ass' is it?)

    Both of these things have been built into 'standard' web services for a number of years now, and have in fact been built into DWA since 7-ish.. Opening up support for GZIP compressed elements to us developers would gain huge kudos (beers, etc) for (Mr Leeland? ) the http server team..

    Especially if this could be retrofitted to nd7. Cos lets face it, lots of customers have a policy of not messing with servers for a couple of years at a time..

    ;-)

    ---* Bill

    5Scott Cochrane  12/09/2007 8:52:20 AM  Wiring JavaScript libraries Into the Domino Server

    This all sounds good .. I'd totally agree with all 6 points. I've got to get a 'but' in though..

    The devil as is often said is in the detail - or with software, the implementation.

    Covering the bullet points in order..

    On point 1: I can currently put JS libraries on the server in a directory - a new directory is nice but hardly ground breaking.

    On point 2: A short cut will be good, but is this simply replacing the path that I currently have to use? If so, again its nice but hardly saving me much.

    On points 4 + 5: this I think is the 'make or break'. If what you mean is that in Designer I could pick 'dojo' or 'jQuery' or whatever is installed on the server as my main framework and it takes care of the 'entry point' then fantastic. If I could then browse the framework directory and simply reference what I need via a check box then even better.

    On point 6: I take the point, I wouldn't expect IBM to support every framework under the sun. However as Nathan points out in 2 (and I'd agree), something needs to be done with the Domino http rendering engine to make it play better with js frameworks. If your doing this as part of providing your default framework, then great, hopefully that should go a long way to sorting out the html rendering.

    It all sounds way better than where we right now .. I hope it all happens.

    6  12/09/2007 10:18:55 AM  Wiring JavaScript libraries Into the Domino Server

    I completely agree with the theory, and I'm glad (but not surprised) you're talking about flexibility for developers by planning ahead for other frameworks. The devil, though, is indeed in the detail. I'm with Nathan, I'd like more explanation of what we would have to do if we chose not to use Dojo.

    Do we simply do exactly what we're doing now? Because it sort of sounds that way. Pass-through HTML suggests that any references to that alternate framework will be entirely hand-written. If we can't point matching functions and attributes of built-in Domino elements at the alternate framework with a simple override function, what have we gained? In other words, if Domino's new JavaScript isn't *fully* exposed for us to manipulate, what's the point? And I don't mean by hacking the Dojo libraries to point to other functions, I mean by simply overriding the entry points. You mentioned the "main" entry point, which is well and good, but I'm not clear on what that really means.

    Here's what I would like to see:

    At database (via a new Database script element) and design element level (via JSHeader?), allow us to define Domino.grid = Mylib.MyCoolGrid; Domino.richTextEditor = Mylib.TinyMCE; Domino.dojo.calendarWidget = jQuery.calendar - all examples completely made up and non-existent, but you get the point. To truly be flexible, we need to be able to override a given Domino widget so that we can extend it. We don't want to have to override 100% of the built-in functionality every time we want to override one element. Maybe this is exactly what you were talking about, but it didn't sound that way.

    To do that effectively, we will also need documentation on how Domino uses these Dojo libraries. I don't mean the standard Dojo documentation, I mean the methods that Domino is calling and what is being passed and returned. If I have that detail, I can override your library with one of my own, without breaking ANY of Domino's expected behavior (unless I want to), and extend it with additional functionality that only my application uses.

    The best way you could support alternate frameworks, by the way, is to ship with an example demonstrating how to use an alternate framework. That would put the proverbial money where your mouth is, and would make it much more likely that the methodology required to implement an alternate framework actually makes sense to a developer. Eat your own dog food, in other words.

    7Rob McDonagh  12/09/2007 10:19:42 AM  Wiring JavaScript libraries Into the Domino Server

    Oops, sorry, the above entry is mine. Left the name field blank by mistake.

    8Jim Knight  12/09/2007 2:39:37 PM  Wiring JavaScript libraries Into the Domino Server

    My favorite is personally prototype. But it seems to me easy to point to the latest js library. And if you integrate it you will always have to worry about versioning whereas if you just make it easy to add in, you don't have to decide on the favorites.

    I'm selling and constantly building a Domino based CMS and there are a couple things interesting here. #1, I would like to be able to insert the latest Tiny MCE updates without having to use webdav (basically the method described here: { Link } In a nutshell to be able to represent a folder structure within a single db (as opposed to on a server drive). The advantage of the webdav method is to be able to treat those structures as if they were folder containers of files. If this could be done inside the designer itself it would be nice - a drag/drop interface for file structures of images/javascript etc. It would also open the designer up to a drag/drop of html and images and javascript generically. Even if not drag drop, if you could import from a file structure into the designer, it would be nice. Just some way of doing it in bulk. Before finding about webdav, we were coding the full path into the names in the files/image resources area.

    I've mentioned before but if Domino could simply spit out decent HTML (close input tags, etc), it would go a long way - won't harp on it because that's not what you're asking here.

    I agree a LOT about gzip from Wild Bill. That would be really nice.

    9Tim Tripcony  12/09/2007 2:44:06 PM  Wiring JavaScript libraries Into the Domino Server

    Okay, after sleeping on this, here are my thoughts:

    As others have pointed out, this sounds good, but essentially still leaves us where we are now, albeit with an incremental improvement. The existing state of things is that we either let Domino do all the work for us, or take things almost entirely into our own hands. Going forward, it'd be the same thing, except that "settling for" the default Domino rendering would presumably produce a better interface than it does now. But if I decided that I preferred to use an alternate framework - say, for example (surprise, surprise), Ext - I'm back to "rolling my own": scrapping everything that Domino is offering to do automatically for me and switching to manual mode ("I'm too close for missiles, switching to guns"). Of course, if the fallback rendering is still CSS-friendly and XHTML compliant, I'd at least be more likely to use Form elements instead of what I'm doing now: Page elements with the Content-Type set to HTML and coding everything by hand (with the occasional Computed Text thrown in to at least be able to leverage some Formula) to ensure that my CSS isn't getting hijacked by a myriad of font tags.

    Since we're using the term "wiring", I'd like to expand on one of Rob's points: one of Ext's strongpoints is its use of adapters. Depending on the underlying framework you want to use (Prototype, jQuery, YUI, or just Ext as a standalone), you include the corresponding adapter .js file (ranging from 13 KB to 35 KB). Whichever one you choose just maps various functions already defined in that framework to code in Ext that calls those functions. For example, Ext.Ajax = Ajax.Request, etc. So I'm assuming you'll have something like a static domino.js file that sits somewhere on the server, with a bunch of calls to whatever framework you end up choosing; for the sake of discussion, let's just assume for now that it's Dojo. So maybe you'll have functions that render a view as a Dojo grid, etc. What if you created that middle layer, which genericizes the rendering within domino.js and maps Dojo-specific calls to those generic functions within, say, adapter-dojo.js? Then... release a *readable* version of both files (I'm assuming that the versions actually being loaded would be compressed in some way to the point of being entirely incomprehensible for the sake of minimizing bandwidth footprint, in addition to gzipping them, which - as Bill points out - should be a given), and let customers and partners come up with their own adapter files for their favorite framework (and, if time and resources permit, give us a headstart on that for some of the more popular ones). Then... somewhere in the configuration (probably Internet Site rules), give us a way to define an adapter: just a label and the location of the file. Finally... on any given element, we choose between the default framework and any that we've defined. That way, if the vanilla rendering is sufficient, it's all done for us; if we want something else, we do the adaptation ONCE and can then utlize it everywhere.

    10Scott Skaife  12/09/2007 3:33:54 PM  Wiring JavaScript libraries Into the Domino Server

    Two things that I see as very important.

    1) Do not make the pointer to the framework server wide. The pointer needs to be database specific. If not, what happens when someone wants to transition to a new framework? We have hundreds of databases across dozens of applications with only 2 full time developers. With these limited resources, it would be impossible to do a server wide switch to a new framework. When we make best practices changes, we update applications at the time that we are making other enhancements.

    2) Put the framework files in a database. Domino applications are moved between servers via replication. Domino serves files to browsers. I cannot fathom any reason that one would throw away one of Domino's strengths for no apparent gain. Obviously having a database template with a directory import would be necessary, but that would be trivial. If the files HAD to be text files on the server, the directory should be populated by the files in the 'framework database', although I can't figure out why the framework files can't be served directly from a database.

    11Thomas Bahn  12/09/2007 5:41:05 PM  Wiring JavaScript libraries Into the Domino Server

    Whatever library IBM chooses as "standard" (and Dojo is certainly the preferred one; remember OpenAjax?), it will suffer from the same problem as Java in Notes/Domino and even more so:

    JavaScript libraries are still advancing at a very high pace. New versions come out, new features are added etc.

    The JVM of a new Notes/Domino version is normally one version behind the current one from Sun. Until customers get to this ND version, usually another version of Java has been published.

    The same would probably be true for any JavaScript library. But would any Web developer be satisfied by a one, two or even more years old version of any JavaScript library?

    12John Smart  12/09/2007 7:11:24 PM  Wiring JavaScript libraries Into the Domino Server

    Hell, yeah!

    And I expect it would be a great thing and implemented pretty well, but even worst case, if IBM did it so badly that certain people blog-ranted about it for a month, it would still be better than nothing.

    13Charlie Phillips  12/10/2007 11:02:46 AM  Wiring JavaScript libraries Into the Domino Server

    I know that since this it is not a JavaScript library my question is somewhat off-topic, but I have heard that there was going to be some effort at integrating or at least opening up to OpenLaszlo. Is there any truth to that?

    Thanks,

    Charlie

    14mdm-adph  12/10/2007 11:28:08 AM  Wiring JavaScript libraries Into the Domino Server

    Forget about choosing a particular framework to implement -- I'd be happy with just a standard set of pre-defined set of Domino server JavaScript object variables that I could use in the framework of my choice.

    Oh yeah, and Gzip compression, too.

    15Rich Waters  12/10/2007 11:28:16 AM  Wiring JavaScript libraries Into the Domino Server

    I'm glad that your bringing some of these issues to the community rather than just making a decision and forcing it out to all of us. I agree with several others above in that if all you are providing is a folder structure and a way to include those libraries you're not really adding anything that we couldn't go and build right now. The "adapter" mentioned is really the best solution I could think of to really open things up and allow developers to plug in whatever library they want. I think some inspiration from projects such as jMaki could be taken. They have mapped a bunch of JavaScript frameworks down to a single format, by changing a single parameter you can render components with different frameworks. Providing adapters for several popular frameworks would be ideal, but at the very least (like Tim said), provide access to all of the middle layer of code and full documentation about how things need to be mapped to function. As you said above, some people love one framework, others hate it but love another... this will always be the case. The opportunity is there to open up the process and get the community involved to help prove that Domino is a valid server for "Web 2.0" applications.

    Also, Wild Bill mentioned back-porting these changes to nd7. I most certainly agree that no mater which implementation changes are introduced they should be included with an update for nd7. With a lot of companies taking several years to upgrade to new releases it would mean that most of these changes wont have an effect on most developers for quite some time. Open source projects and products built would then have to maintain two versions because not all of their users or customers would have the newest technology? Or would that become a way for IBM to force its customers to purchase upgrades...

    gzip - Duh, the server should make this completely transparent to the developer.

    16Nathan T. Freeman  12/10/2007 12:10:06 PM  Wiring JavaScript libraries Into the Domino Server

    "Or would that become a way for IBM to force its customers to purchase upgrades..."

    I'm not sure that's a fair characterization of what it means to advance versions, Rich. Providing new capabilities with new versions is not forcing an upgrade. Having something be intentionally broken in a prior release, or refusing to support a prior release -- *that* is forcing an upgrade. But simply making version 2 better than version 1? Isn't that what versions are all about?

    17Kerr  12/10/2007 12:47:54 PM  Wiring JavaScript libraries Into the Domino Server

    I wonder what percentage of domino licences are under a passport subscription anyway? And don't you need a subscription to get point releases anyway, making the difference of putting the code in a 7.0.x vs 8.0.x release moot from a licensing point of view.

    The main advantage to having any update available in the 7.0.x code stream is that it's generally not a strategic decision to upgrade point releases.

    18Patrick Kwinten  12/10/2007 1:59:06 PM  Wiring JavaScript libraries Into the Domino Server

    i use currently prototype because it is easy to understand and already beneficial from the beginning. Also the documentation is friendly and lots of examples/articles to learn from. those where the criteria i have set while choosing a framework...

    19Rich Waters  12/10/2007 3:05:28 PM  Wiring JavaScript libraries Into the Domino Server

    Nathan, Perhaps I was a bit harsh. I certainly agree that upgrades in software should mean new features. I was just commenting that there are a lot of products and other projects that are released as notes databases. If those projects want to continue support for old versions it will require maintaining two versions or adding some means of server detection to their code then acting differently depending on which it is. I would peg this as a reason why not a lot are using json output format, the output is different depending on the server version, or non-existent on 6x.

    20Erik Brooks  12/11/2007 8:37:50 AM  Wiring JavaScript libraries Into the Domino Server

    @9 has the right idea. An extra "adapter" layer will add great choice of framework. I would like emphasize something, though:

    *FAR* more important than IBM/Lotus spending time developing additional "adapters" for extra frameworks is ensuring that *as much as possible* is exposed to the adapter API in the first place.

    Don't bother giving us 3 different adapters for working with views if all we can do is "Next Page" / "Previous Page". Instead, give us 1 adapter with hooks into all of the low-level stuff -- refresh, expand/collapse, "restrict to single category", "starts with", etc. etc.

    The adapters themselves (besides the out-of-the-box one named "Cujo") can then be OpenNTF projects. *We* can support adapter plugins after the fact, but only IBM can decide what stuff is going to be exposed to the adapters by NMFR.

    21Erik Brooks  12/11/2007 8:44:14 AM  Wiring JavaScript libraries Into the Domino Server

    I'd also like to observe that I think that the chance of this work getting backported to 7.0 is exactly 0.0%. Just think of all of the pieces involved... Web server rendering enhancements/changes (XHTML? CSS? JSON?), Designer implentation of new features (already underway for Eclipse -- forget backporting that to the "old-school" UI for ND7)... the list goes on.

    22Chris  12/12/2007 1:30:50 PM  Wiring JavaScript libraries Into the Domino Server

    This sounds like the coolest new feature ever (with the exception of JSP, which eventually got pulled). I can't wait for something like this to be implemented in Domino!