Bob Balaban's Blog

     
    alt

    Bob Balaban

     

    What if.....DXL? (#1)

    Bob Balaban  April 17 2007 05:00:00 AM
    Greetings, Geeks!  

    This is the first of what I hope to be a series of "What if" postings, where I throw some ideas out there and you carefully consider them and provide me with the fruits of your thinking. You are helping me do my job! Yes!!

    Naturally, the IBM lawyers make me say this: everything I present here is my own personal opinion, and does not represent a promise, a commitment, or even a particularly firm intention on the part of IBM to actually DO anything, necessarily (but, you never know). We're just talking!

    We have this thing called DXL (Domino XML). It's an XML representation of Notes data (or design, which is also data of a kind). Using various mechanisms, you can take a Notes object (document, form, view, an entire database), and convert the proprietary, binary data format that Notes/NSF use in the product to an XML markup. You can also take some DXL and "import" it, converting back to the NSF format.

    This is pretty cool, and useful, too, for a number of reasons. For one thing, the fact that you can do this with ANY Notes object gives you (in theory!) access to all sorts of stuff (especially design stuff) that you can't get at through the normal LotusScript or Java APIs. Our "scripting" interfaces (LS, Java) have tended (deliberately) to focus on data manipulation, not design manipulation. Of course, someday we'd like to do it all, and have everything available via every API, but that hasn't happened yet, and we've always felt that programmable manipulation of data is the more important choice, given limited developer resources at IBM. Besides, there is already a tool that gives you (the Notes/Domino application developer) access to lots of design settings and choices: Domino Designer.

    Of course, having said all that, we've all on one occasion or another had the need (or at least the desire) to modify a view column, or tweak a database property, or create a table with fields in the cells, or any one of a thousand other things that require design manipulation/creation.

    Enter DXL. You can export it. It's XML, so it's mostly just text, so you can edit it. You can import it again. The Import options include things like "update" or "replace". So, instead of having to learn the C API to locate a form in an NSF, and tweak some setting on some field in it, you could just export the form's DXL, edit the stream (or the file) and re-import, updating the form's design. In theory!

    In reality, for most things "round-tripping" DXL does work. But it's not 100% lossless, which makes it inappropriate for many types of applications. There are 3 categories of data that tend not to survive the DXL "round-trip" experience:

    1) Binary stuff, especially crypto data (digital signatures, keys, etc.)
    2) Data that might be new in a new release of the product (a new setting or data type), for which the DXL supporting code has not yet been written.
    3) "Unclean" (my word for it) rich text CD-record data.

    Category 1 is something we could handle with a bit of work. Category 2 will always be a problem (we have the same issue with NotesPeek and other things too), but maybe there's something we could do around our internal development procedures that would help. Category 3 is a potential never-ending black hole of effort, concluding only in the far, far future when the universe experiences total entropy heat-death. Having said that, most of the time rich-text CD records are actually ok, it's just that it's sort of like HTML: some browsers are more forgiving of weird tag layout and ordering than others. People (and software) can put strange rich text into a Notes document, and a lot of times the Notes editor (and other services) will handle it just fine. Some of the time, DXL does not handle it just fine (some of the time, neither does the editor....)

    But, perhaps there is a DXL solution-with-a-restriction: namely, "clean" rich text works, "unclean" rich text might not (but we'd tell you about it, in theory).

    So, Geek Faction, here is the question: What if.....we fixed this to work better? Would you use it? What would you use it for? Archiving? Web app data transfer? Email? On-the-fly design? Tell!

    More to come on this topic....for example we haven't even begun to plumb the depths of the Rich Text Conundrum (no, that is NOT a film with Matt Damon....)


    References (Redbooks):

    "XML Powered by Domino How to use XML with Lotus Domino" -- http://www.redbooks.ibm.com/abstracts/sg246207.html?Open
    "Domino Designer 6: A Developer's Handbook" -- http://www.redbooks.ibm.com/abstracts/sg246854.html?Open
    Comments

    1Jesper Kiaer  04/17/2007 6:31:59 AM  What if.....DXL? (#1)

    yeah fixing DXL would be great! :-)

    We have tried using DXL on different projects but everytime, we had to move away from using again, because of problems with the imported DXL.

    DXL is such a great with lots of potential uses, but the implementation of the "idea" has just not been good enough.

    brgds

    Jesper Kiaer

    { Link }

    2Jeff Gilfelt  04/17/2007 6:43:23 AM  What if.....DXL? (#1)

    If what you are saying is that 100% fidelity will never be achieved on a DXL "round-trip", perhaps the export could identify and include a flag attribute in each node where the content is potentially "unclean". At least then developers could use XSLT to identify problem areas and handle them, rather than having a nasty surprise at runtime after the re-import.

    3Christopher Semturs  04/17/2007 6:44:39 AM  What if.....DXL? (#1)

    We are using DXL for our complete application development. Basically, we implement our applications (the new name for database :-)) in a high-level language and generate the NTF via DXL - NTF is just the compiler output. :-)

    Based on the feedback I got by some IBM-internals, we are one of the most advanced power-users regarding DXL-import into design elements.

    Having said that, the DXL is absolutely powerful and useful for us, it is our key-technology for our NTF-compiler. Some of our customers are using it also for specific tasks (e.g. generating views).

    Performance could be improved (as always :-)), and it could be more feature-complete in regards to certain attributes for columns, views, ... - but that's it. Keep it alive!

    If you need more information on this compiler-approach, let me know. I think it could be an interesting exchange of ideas and approaches.

    4Ian Connor  04/17/2007 6:48:37 AM  Depends on how you define fixed

    It would certainly open up the ability to do better version control (aka source control diffs for notes databases) as well as easier database management.

    Also, if it performed really well, it would remove a lot of problems when using Notes data in web applications. I have always been a little cautious serving documents via DIIOP or even LSXBE because of performance concerns (the http engine seems faster). Having said that, the http HTML is not clean. So getting XML might give you better control (depends how fixed it would be).

    I sympathize that taking a very mature (old) rich text standard into XML is going to be hard work but it really has been holding back many 3rd party developers from understanding the Notes structure and thus we don't see the same number of addins or export/imports that we might otherwise see.

    5Fabio Pignatti  04/17/2007 6:50:25 AM  What if.....DXL? (#1)

    We are very very interesting in using DXL for revision control. We are not focused on data import/export through DXL, but the "design side" must work great.

    Some of mine guys told me some "hide when" doesn't work properly after a export/import process.

    6Andrew Price  04/17/2007 6:53:44 AM  What if.....DXL? (#1)

    1/ We would love to use XSLT + DXL to apply consistent new styles to thousands of design elements in hundrededs of dbs at once. Reliable DXL would be a huge plus for us -- though to be honest we haven;t messed with it yet ;)

    2/ Just in the last couple of weeks we have come across an acute need to transform DXL -> PDF so a reliable way of doing that would be nice.

    3/ I have felt for many years that IBM should hire Ben Langhinrichs and/or buy Geniisoft's portfolio.

    Great to see you back at IBM Bob! If only you could entice Ned Batchelder too ....

    Cheers

    Andy

    7Ricardo Lucio  04/17/2007 7:04:37 AM  What if.....DXL? (#1)

    Reliable DXL import/export would allow us to have things like a UML drawer to generate forms/java/lotusscript classes based on class diagrams, and vice-versa.

    I'd really be interested in developing such kind of stuff, which would be very useful mainly for application documentation.

    Thanks a lot Bob for openning this door for us, developers, to give our opinions.

    Ricardo Lucio

    (PS.: I saw you here at IBM São Paulo last week but didn't have a chance to give you a handshake. Nice to see you!)

    8Nathan T. Freeman  04/17/2007 7:48:00 AM  What if.....DXL? (#1)

    Bob, I think you're pretty clear on what I think the potential of DXL is. Other places where I've used it include...

    Transforms for pure data import into Notes documents.

    Language translation of NTFs.

    Manipulation of embedded object target NSFs.

    9Dragon Cotterill  04/17/2007 7:50:14 AM  What if.....DXL? (#1)

    Personally speaking, no it's not something I would use.

    Damien Katz wrote a set of LotusScript classes many moons ago which gives you access to all the design elements. I've used that piece of code many times to go in and change things. Need to change the colour of every ActionBar on every Form and View? - three lines of code and it's done. Same goes for adding an additional shared action. I've even used it for creating new design elements (views) based on users requirements (this was before we had the ability to move columns about in views).

    So long as you're careful about the various flags and stuff you don't normally break things.

    What I really find lacking is the complete lack of information as to what fields and flags mean what in which design element. But then, like so many developers (me included) it would involve writing documentation. And you know how we hate that.

    10Tobias Mueller  04/17/2007 8:14:45 AM  What if.....DXL? (#1)

    I would use it for:

    (clean) rich text DXL => XSLT => ODF

    ODF => XSLT => (clean) rich text DXL

    Or instead ODF OpenXML

    11Ben Langhinrichs  04/17/2007 8:22:20 AM  What if.....DXL? (#1)

    I think it would be an excellent idea, especially if the flag could be returned that said "Unclean!". The hardest part about using DXL now is not that it doesn't always work, it is that you don't know whether it worked or not.

    That said, this will still only work if the DXL is pretty robust and pretty forgiving, as it will drive people nuts if they find that everything is unclean except the simplest rich text.

    (btw, I like the "plumb the depths of the Rich Text Conundrum" - be afraid, be very afraid)

    12Philip Storry  04/17/2007 8:59:08 AM  What if.....DXL? (#1)

    Data transfer.

    I'm not a developer, I'm an Admin. But I'm a brave Admin, a foolhardy Admin even - the kind of Admin that thinks the solution is never complete until it's as automated as humanly possible.

    So I do a teensy bit of development^Whacking about in Designer.

    The other day, I was mulling a problem when suddenly I remembered DXL. I was fired up. I figured that this might be the solution...

    I just wanted to get some data from an application we have, and produce some stats/look for anomalies. We already have hefty stats tools here - so getting the data out of Domino was a preferred option, as re-writing existing software wasn't going to be fun.

    Researching DXL showed me that this was not the solution I'd hoped for. As all the work will be done elsewhere, my hope was that I could grab the data I needed via http. I can get the views, but not the documents.

    It's not a huge problem. I could write a DLX "wrapper" database, which returns documents as DXL for me.

    Traverse the views via http as normal, then call the wrapper application and get the documents. If I do it right, it will even be nicely re-usable...

    But it's a little more work than I'd wanted. DXL is good, but just not quite good enough. So that project is now on the back-burner, as there are other things I can do that are going to be faster to complete and thus give returns on investment faster.

    A pity, really. DXL would have been ideal for my needs. I suspect that there are plenty of times when DXL would have been similarly useful for others, had it been able to pass documents as well as views via http.

    (Apologies if this is way off base for your request, and if it is then consider it an ill-informed rant. *grins*)

    13Tim Tripcony  04/17/2007 9:22:12 AM  What if.....DXL? (#1)

    Bob, I'm actually writing an entire web-based IDE for Domino based on DXL. So far it supports agents, script libraries, shared actions, database script, and shared resources (files, images and stylesheets), and in a few days, views and folders as well. I don't imagine it could ever be a complete replacement for Domino Designer, but it's already proven a convenient alternative when I don't have Designer available. I'd love to have your feedback on it (it's available for download on my site)... I call it "XIDED": XML IDE for Domino. So, to answer your question, yes, I'd definitely like to see any improvements that can be made to DXL. At the moment, my biggest pet peeve is that XML nodes that can contain X/HTML (i.e. code elements in agents and script libraries) are not CDATA, which can make parsing very difficult.

    14Thomas Bahn  04/17/2007 10:22:59 AM  What if.....DXL? (#1)

    If it is not possible to create an XML tree of some RichText items, what if this data is exported in a binary format (Base64 coded), as it is done already with some kind of data? Perhaps there could be a flag in the DXLExporter's arguments to switch this behavior on or off.

    This way we could get an reliable round-trip, which enables some of the use cases the other commenters wrote about (like source version control), we could at least modify all other kinds of data (string, numbers, dates).

    It cannot be too complicated to do a binary output (like the object (de)serialization in Java) and encode/decode it, could it?

    15Frank Dröge  04/17/2007 11:10:38 AM  What if.....DXL? (#1)

    I'd like to see more of the magic that Rocky weaved when he did the dynamic table stuff in Notes access for SAP solutions. As far as I know, it could have been easier if DXL had behaved as it should.

    16Julian Buss  04/17/2007 11:12:50 AM  What if.....DXL? (#1)

    we would use it for design stuff like developing an interface for notes design documents to a SVN repository (that means: having version control stuff).

    If you think about making DXL reliable and complete for design elements, please think about giving us events like QueryOpen, QuerySave and QueryClose for design elements, too. We would need them for a version control system.

    Honestly I don't understand why there is no way to have some kind of version control for notes design elements in the first place. I love developing in Notes, but not having any version control is ridiculous.

    17Bob Obringer  04/17/2007 11:41:38 AM  Downstream

    Following the link to my site will give a pretty good explanation of MY needs in this area. The important thing for me is the ability to get at the design over HTTP. As with the new JSON support in views, it would be great for the design too.

    Reading views is a wonderful thing, both design and data. If we could do the same thing with framesets, outlines, forms, etc, via ajax, it would open up a world of possibilities for web app design.

    I understand this isn't what you are referring to with DXL, but getting at both design and data over HTTP and XML/JSON could be really powerful.

    18Ron Kats  04/17/2007 12:02:50 PM  What if.....DXL? (#1)

    At this moment we are very very very eager ti get a repository for design elements, but (really big but) our company desided to move to Exchange (for mail only). That's why our budget is extremely low (even internal), so making one my own is aproject I started today (in my iown spare time.

    Using DXL seems the best approach I think. So the better it works the hapier it makes me ;-)

    By the way: does anyone have some suggestions?

    19Richard Schwartz  04/17/2007 1:02:55 PM  What if.....DXL? (#1)

    We need two things: 1. The ability to query whether a particular piece of DXL is round-trip clean or not. 2. The ability to request that the generated DXL include uninterpreted base64 encoded binary representations of every item, or of just the items that would be "unclean" -- with the ability to import that DXL-cum-binary cleanly. And if that means packaging up externally referenced things with the binary, such that an import of the DXL will lose any efficiencies gained through sharing of externally referenced things, so be it.

    20Nathan T. Freeman  04/17/2007 3:37:35 PM  What if.....DXL? (#1)

    @13 - Bob knows who you are, Tim. I made sure that I passed your name along a few weeks ago. :)

    21Tim Tripcony  04/17/2007 5:41:10 PM  What if.....DXL? (#1)

    @20 - Thanks, Nathan. I appreciate the referral. :)

    @13 - Ron, there's a free version control tool for Domino that uses DXL available on OpenNTF, called DesignCatalog:

    { Link }

    It has its limitations, but might give you a head start on implementing your own.

    22Mac Guidera  04/17/2007 6:19:17 PM  What if.....DXL? (#1)

    If Domino XML Language were to become fully vetted and filled out, then the possibilities to work with the platform becomes limitless. Having the ability to round-trip would be super in Notes applications! However, by exposing any Note as XML, the potential to work with NSF based applications within the enterprise arena goes way up. The confidence that comes from having an open standards based window into an application provides for easy integration and interoperability from who knows what.

    23Mac Guidera  04/17/2007 6:20:01 PM  What if.....DXL? (#1)

    "Build it and they will come."

    24Dwight Wilbanks  04/17/2007 9:49:22 PM  What if.....DXL? (#1)

    Someone created a form with 200 fields and now I need to change the hide formulas on half of them.

    Lets add computed images to the left of each required field

    Make sure that fonts are consistant, for the client, get rid of all that default san serif, for the web change it all back to default san serif (we use css for fonts).

    These are tasks that are incredibly time consuming to do manually. And can be done fairly quicky in DXL, but, its not easy enough for the masses yet.

    I think the biggest problem with DXL is the rich text support, its CD records can get really messy and I can imagine the black hole that it must be to support it at IBM.

    Along the lines of "what if", what if a new alternative was provided? Add to the types of field, a new type that could be maintained (based on odf). When you could certify the apps you are developing would no longer be accessed by legacy notes version, switch your forms over to an ODF design store. Possibly allow a read only rich text mirror element for older notes clients.

    Also, it would be a big help to be able to export out the CD records directly when something does get corrupt, I've been able to correct stuff be eliminating junk CD records, I personally would not mind if they were named exacly the way they are in the .H file. I wrote code for it quite a while ago using api, and I had moderate success (pre-dxl days).

    25Ralf M Petter  04/18/2007 2:08:09 AM  What if.....DXL? (#1)

    I had a project two years ago, where i will create Reports for our CRM system in Rich Text. The LS classes are to restricted to do this kind of work, so i tried DXL, but i was very disappointed, because there are so many rich text features not supported in DXL (6.0.4) So please extend the DXL features and this will be the most powerfull tool i can imagine.

    Ralf M Petter

    26Bernd Hort  04/18/2007 3:40:45 AM  What if.....DXL? (#1)

    I use DXL for quite a lot aspects.

    One great thing is generating PDF documents with FOP. (DXL->XSL-FO->PDF) But the rich text was far to complex for my projects so far. So I only implemented a subset. I know that there are some developers in different companies working on it. So instead of reinventing the wheel a dozen times it would be realy great if IBM could provide use with a XSLT style sheet for XSL-FO. I know that this is unlikely, but if nobody asks for it, it will never happen. ;-)

    27Brian Miller  04/18/2007 11:01:47 AM  What if.....DXL? (#1)

    First up, here's an idea from "out of the box". Why on Earth *do* we still use CD? Isn't it time that ND switched over to a more modern internal representation? Granted, I know that backwards-compatibility with CD is going to be needed for a long time, but it's a really worthwhile thing to do to plan for Notes 9 to use a better and more robust way of dealing with RT to begin with. If the internal representation is more like an organized tree structure that directly corresponds to an XML representation, then the import/export gets faster, and it gets more intuitive for the programmers at IBM. We (eventually) will no longer have to worry about Rich Text on the back end being unimplemented, since it'll be a much easier thing to tackle. And, it can build in safeguards to prevent corruption - something like XML validation.

    Aside from the time commitment for building and testing, which is not small by any means, there's no downside - just happy customers at the end.

    For the more realistic here-and-now stuff... DXL is actually really close to being "very useful" already.

    My big issue is that I want *some* representation of every kind of design element. Right now, there isn't necessarily a definition in the DTD for "older" kinds of elements. (Navigators come to mind here.) And yes, it needs to be able to round-trip with 100% fidelity, including such small details as hide-when formulas.

    I like the idea (@14) of having the option (at export time) to export RT as a big encoded blob of CDATA, rather than breaking it down into its component parts. This may not be as important for design elements (my concern) as it is for data documents. But, as long as you can use this technique to guarantee 100% fidelity, you've gone a long way towards making a lot of people very happy.

    As for actually dealing with exporting complex/bad CD, I think that the exporter should give it the old college try whenever possible, and put an attribute of "notclean=true" on anything that it doesn't grok, or can't be sure about. You're ultimately catering to two different types of customers - people who want to transform Domino data into something else (who would prefer exhausting all attempts to make wacky CD records useful somehow), and people who want round-trip fidelity (who would use the "binary blob" flag, and wouldn't need to worry about whether the CD is wacky or not).

    One other thing that I would like to see is a way to use DXL to extract the data from images into actual image files. Preferably in memory, so I can point an image src to the URL of an agent which extracts/sends the binary data in one of the common image formats we know and love.

    28Paul T. Calhoun  04/18/2007 12:37:54 PM  What if.....DXL? (#1)

    I have uses DXL extensively since it was available as a separate toolkit. Primarily Design element manipulation. I have a tool that pulls all of the Java Agents out of a database and generated .java source files for importing into any Java IDE (Eclipse, RAD7, etc). Any and all improvements in this area that further the designers capabilities are welcome.

    29Scott Skaife  04/18/2007 6:46:08 PM  What if.....DXL? (#1)

    Full fidelity round trips would be a huge deal. Being able to store our code in the same repository as all of the other platforms do, would really help with the acceptance. Of course if this could lead to the ability to change the design on the fly, (e.g. add fields to a form while the user is editing) that would be extra special.

    I do like idea in comment #24. Have a 'dirty' form for legacy, and a 'clean' form for new development. It seems to me that this has worked very well for the shift from Navigators to Outlines. New development is done with outlines, and if we need to update a Navigator, we convert it. Other than a couple of small apps that will probably never be updated again, I don't think we have a single navigator at my site.

    30Remo Linter  04/19/2007 7:17:16 AM  What if.....DXL? (#1)

    Yes would be great when it would work good. I was trying with, but destroyed Views and can not handle Layers in Forms. :-( Perhaps other Problems too. But this Problems are let me give up.

    31Erik Brooks  04/19/2007 8:01:06 AM  What if.....DXL? (#1)

    If you recall my first theme from your original "Got a new job!" posting:

    (1) Stuff you can't do in Notes at all.

    An Example for This Discussion: There are many design element attributes that you can't manipulate programatically.

    If IBM's answer is "Just export your existing element to DXL, manipulate it, and re-import!" then we MUST have full round-trip fidelity.

    If there are attributes that might get corrupted on a DXL round-trip then they must be preserved. The biggest holdup here is rich-text.

    I think we all understand that rich text is a mess. But if all I'm concerned about is exporting a form to DXL, changing its "Automatically Open In Edit Mode" property and re-importing, then the rich text needs to be preserved or else I have NO way to accomplish my goal through DXL.

    With that caveat DXL still has some value (you can try to create a NEW form from scratch with DXL using 'clean' rich text and it can have the flags set that you need, and you can delete the existing form).

    I vote that the two main goals of DXL work should be:

    (1) Complete support for all of the 'simple' stuff that isn't yet supported. If there's a checkbox somewhere that doesn't have a representation in the DTD, it needs one.

    (2) A 'Preservation mode' for all complex stuff that IBM decides they can't easily accomplish with #1 (e.g. rich text). This would be the first step to true round-trip fidelity, after which IBM could focus on supporting round-trip fidelity *with modification* of those complex elements.

    32Jerry Gassie  04/19/2007 2:26:08 PM  What if.....DXL? (#1)

    I like DXL. I used a lot of DXL initially, but I soon became hampered by memory limitation issues and some bugs (CRLF breaking up tag lines in some cases, had an SPR on that one). Biggest usage I have for it now is to dump messages to extract embedded images and convert them to files. Either way, my biggest complaint has been the memory constraints when creating/updating documents. I'd love to see those issues resolved instead of having to use workarounds like multiple updates to the record.

    33Ladonkin Dmitry  04/20/2007 7:21:12 AM  Bad idea

    Lotus developers use DXL for changing design elements and content of Rich-text fields like heavy injured people use crutch for moving. They just have no other choice then use DXL (or C API). IBM must heal developers, not just providing crutch to them.

    Everyone who needs to change content of rich-text items develop his own tools. I think IBM must provide tools for front-end changing Rich-Text items.

    Lotus is good for data storaging and managing but bad for data presentation. I think IBM must provide tools for very flexible user interface.

    34Wayne Sobers  04/20/2007 9:52:22 AM  Why.....DXL?

    Why the focus on DXL at all?

    If its "data" its XML. The richtext issue aside, there are already several ways to extract data from Domino.

    If we are talking about design, then what is your intent?

    DXL is certainly not RAD. It's useful for documenting systems, and would be one way of enforcing corporate standards.

    Focus on giving the client flexibility at run time. Many of the design elements have huge potential for better data presentation. Let me size and move layers at run time. Give me access to keystroke events. Get a decent HTML/Java script renderer for frames - that alone would give client developers access to a huge set of resources which could be leveraged for use in Web development.

    Plus which developers are you targeting - corporate teams, business partners, lone down in the basement SMB do it all admins?

    It is a useful idea - Foxpro programmers have been doing stuff like that since to the DOS days, but you need to have all the pieces working. So I now have to learn XSLT along with XML nuances and DXL trip ups and I'm build tools instead of building applications.

    You should be incorporating the best aspects from the SmartSuite apps since the company is now behind open office.

    35Markus Koestner  04/20/2007 10:31:29 AM  Version control

    We´re using DXL internal for changemanagement and version control in an Eclipse plug-in.

    We´re exporting all design elements to a folder and are then able to sync it with CVS. So far it works fine. Only with some design elements (like JAVA agents or script libraries with hidden source code), the export fails or hangs completly. I really like DXL, because it opens the Notes world even more to web standards. But it is as always: I only want to use it, if it is stable and the errors are limited or fixed quickly.

    36Rich Collette  04/20/2007 11:31:38 AM  DXL, Embedded Views and Getting the Basics Right

    Bob, are you saying that IBM is finally taking the Juila Child approach in its own environment? Will other buggy or incomplete features in Notes be corrected?

    I'll start with some specifics with DXL but will get more general.

    1. DXL is currently an attribute centric XML schema and because of that, it makes transformations more difficult. In fact many mapping/transformation tools that currently exist do not provide that nice connect the lines type of schema mapping that you would get would normally be able to do with an element centric schema (ie. the element name is the field name).

    2. I would like to see appropriate use of schema datatypes for values. For example, datetime fields should have an xsd:dateTime type rather than <itemdata type="400"> (in 6.5.4)

    3. The XML parser is an essential part of processing DXL and it doesn't seem to have been updated in quite some time. Having a nodeset function is really a requirement once you get beyond the most basic transformations. I've used the apache parser but then I am limited to backend agents. I would really like to see the parser upgraded. Maybe it is in 8. I haven't had time to play with it yet.

    Oddly, we do not have the ability to specify the database server and path using formulas for embedded views like we can with outline entries. We also cannot change what database an embedded view points to. You have to delete and recreate the embedded view to point it to another database. So if you have production, test, and development instances as we do, you wind up creating three computed subforms for each embedded view so that you can display the correct view in the correct environment. Adding a new view to production? Well you can't even create the embedded view in your development instance until the new view has been added to the production database. Trying to be smart one day, I thought I could change the db id for an embedded view using dxl (we had recreated a test instance due to ongoing replication issues). The db id is there in the exported dxl but you cannot import the embedded view on a subform using DXL (in 6.5.4). So yes, having DXL work with greater fidelity would be helpful in certain instances.

    But in many cases we're resorting to DXL due to short comings in the product, like the example I have given. It seems like new features are constantly added, a percentage of those only work marginally and then IBM never goes back and completes or fixes those features. Totally not the Julia Child mentality you previously mentioned. For example, try a uidoc.refresh in a single category view ({ Link } ) This has been crashing in Notes 6, 7 and now 8 (I believe a support engineer confirmed the 8 crash at lotussphere for me). To see a crashing issue continue through 3 major release cycles hardly shows a commitment to code quality. As new features are added, it seems we have to deal with a compounding list of issues rather than a manageable set.

    So in closing, while fixing DXL is great, IBM should be directing more of its resources to fixing existing functionality and delivering what developers have been requesting and needed for the last many years rather than trying to add more buzzword functionality that sells the CIO at a new account. Please see the following list of issues that have been bothering me for some time and remain to be addressed.

    { Link }

    Thanks,

    Rich

    37Nathan T. Freeman  04/20/2007 2:29:16 PM  What if.....DXL? (#1)

    I'm missing why people are saying that DXL is for dealing with *shortcomings in the product.* It's a programming vector for design elements in this case. Is it that you think Lotuscript should be made more robust to address these issues? @Formulas? Java support? What programming vector would need to be enhanced for it not to be a "shortcoming?"

    I'll agree that lack of programmability on embedded object targets is ridiculous. Ironically, we could easily solve this if we had an embedded frameset capability (like the IFRAME, really.) Then it's just a one-frame frameset with a programmatic pointer to our target element.

    Actually, that's an extremely cool approach, and it's consistent with one of the most powerful browser solutions.

    38Stephan H. Wissel  04/23/2007 9:13:56 AM  What if.....DXL? (#1)

    Excellent idea to fix DXL. I think we have 2 distinct tasks at hand:

    DXL for design

    DXL for data

    In design we won't encounter "funny RichText", so that could be completed first. Little steps here:

    - add "Import ..." to the DXL tools (later File-Import for data?)

    - Allow in a future Eclipse based IDE to switch between the visual representation of a form or view to the DXL view (= view the source code)

    Of course while musing about XML in Domino. have an option to create a form from

    a) a schema

    b) a web service

    c) map existing form fields to a schema

    (borrow all that code from Component Designer)

    also allow DXL data to be read written by HTTP

    ?ReadDocumentXML ?SaveDocumentXML

    ... with XSLT Filters to avoid exposure of stuff we don't want to expose.

    39Bob Balaban  04/23/2007 9:25:48 AM  What if.....DXL? (#1)

    Greetings, Geeks!

    I am back from a long business trip, ready to pick up the conversation again. Thanks (once again) to everyone for the through-provoking and interesting comments.

    Some specific replies:

    @2 - yes, i like the idea of multiple options for dealing with "unclean" content -- dump to binary only, flag it, etc. Of course that means writing a validator, which could be less then perfect...

    @3 - Chris, would love to learn more (via email) about what you're doing, sounds very innovative. I'd be especially interested in your list of missing features.

    @4 - Stay tuned! You are clearly thinking some of the same things I am thinking....

    @5 - Source control (including diff-ability) is one of my top use cases.

    @6 - Would you rather transform stored elements via DXL (as I infer from your comment), or would you rather store DXL natively, along with a style sheet to be applied at rendering time?

    @7 - Nice to meet you too, thanks for posting!

    @10 - Interesting! I'm curious, why would you prefer DXL over ODF native formats?

    @11 - Good point, Ben. Since the current mechanism is export/import, it's possible that an import operation could successfully load DXL data, but then render in unintended ways. I can't think of a robust way to fix this problem -- it seems as though we might need some kind of validator, though of course such a tool will never be perfect. And would people be willing to pay the price in performance?

    @13 - Thanks Tim, will check out your site asap

    @14 - Hi Thomas. This is an idea we've been discussing for a while. To answer your question, no it's not "hard", but it can be a bit slow. My current preference is to offer this as an option.

    @15 - Hi Frank! Yes, Rocky had to apply some workarounds when things did not go as expected, but he did get it to work just great. He did publish some details of the technique on his blog { Link } and you can probably find more material in his Lotusphere presentation.

    @19 - Rich, I do believe I understand your use case fairly well... :-)

    @24 - Thanks for the suggestions, Dwight. Stay tuned for a future "What if..." posting (probably more than one by the time all's been said) on rich text issues.

    @26 - Keep asking! You just never know....

    @34 - Fair question, Wayne. But if you read through the other comments here, you'll see a number of examples of how people are putting DXL to use, even with the current limitations. If these aren't useful to you, that's fine too. Several of your other suggestions are things which are under consideration, stay tuned for future posts on those topics.

    @36 - No promises at this stage of the game, but of course we want to improve our stuff. Regarding your specific suggesions: 1 & 2 are very relevant, will take under advisement. 3 - we are hoping to update the parsers in a future release.

    Using @formulas for specifying the source of an embedded view is an oft-requested feature. We hear ya.

    40Tobias Mueller  04/27/2007 4:03:44 PM  What if.....DXL? (#1)

    Ok. Let me explain.

    I use OpenOffice as a reportengine for Notes through XSLT. Extract XML data from Notes(DXL) transform it with a stylesheet to valid OpenOffice XML files, zip all back to a odt file. This works fine but not for unclean richtext.

    41Colin  05/01/2007 5:13:07 PM  What if.....DXL? (#1)

    You know the big loser in this is going to be the third party vendors; right now, we either have to buy their tools or keep CAPI programmers on staff.

    Human readable (open source) design? This is going to be sweet!

    No more posts on the message boards: "Well, you could write an api solution or you could buy RichTextMagic by WiccaSoft..."

    42Mark Demicoli  05/01/2007 11:52:37 PM  What if.....DXL? (#1)

    What DXL allows you to do (in theory!) is breathtaking.

    My QAF workflow engine (link below) uses DXL to create subforms containing fields that performs specific functions in the workflow process. With DXL I've been able to produce this sort of functionality, however more recently it has let me down dramatically with a new feature, which implements a 'codeless data dictionary keyword field' feature. With a few mouseclicks, a keyword field can be configured and QAF dumps the field in your form - or updates the existing field. And here's the catch! It normally destroys complex forms on reimport. IF DXL was complete we could do such fancy things!

    Qenos Application Framework:

    { Link }

    43Mark Demicoli  05/02/2007 12:09:24 AM  What if.....DXL? (#1)

    oh and just for the record (and because I like to stir people up).. my theory for DXL being conceived was for Notes apps to have a potential migration / conversion path to the Workplace Designer platform (yes i remember seeing an Import Notes Database button in some flash preso!)... Then the momentum behind 'scrap notes' waned and here we are, we've come full circle (with this cool DXL thing that is almost working) :) Long live Notes.

    44Andrew Price  05/03/2007 8:08:03 PM  What if.....DXL? (#1)

    @39:

    "@6 - Would you rather transform stored elements via DXL (as I infer from your comment), or would you rather store DXL natively, along with a style sheet to be applied at rendering time?"

    We need to use Designer to work directly on the dbs, and sometimes even live ("generated") dbs so the elements would be transformed via DXL rather than stored only in it. :)

    45Bill McCuistion  05/04/2007 3:30:48 PM  What if.....DXL? (#1)

    I recently had the occasion to use DXL to import data into a NSF application. In summary, an Excel workbook had been used to track certain service requests. Problem was that the simple import 1-2-3 truncated text in cells at about the 240 char and some of the cells contained several K of text data.

    So, I pulled-out my PHP5 Excel-Reader class, read the worksheet in native format, and used the DOM to generate the DXL - one document per data row. Then I grabbed the Java DXL import sample, changed two lines of code and the import was complete, save for some tweaking to improve the format of the new RTF data.

    I have in mind to use DXL to integrate ND7 to a Quickbooks SDK (QBXML) application, and hopefully replace *some* Java-agents concerned with the document CRUD-cycle.

    The RT-validator (RTidy?) would be useful, even on a "best-effort" basis, to start with. My biggest head-ache with RT is with embedded objects or file-attachments in a RT field (not the body). Perhaps an option in the RTidy app could "move" the embedded object/attachment into the body with a link (or note) that it had been done.

    I'm hoping that Maureen includes a SVN/CVS linkage into the DDE. I don't need it to be DXL, just a high-fidelity "source code" listing. If DXL is the chosen "language" for represting a NTF/NSF design, then OK.

    Lastly, w/r/t rendering into the browser-client, I'd opt for an option to use the Apache2 HTTP server to replace the Domino HTTP stack, so that I could run PHP to script the transformation of DXL into HTML. I find that XSLT only goes so far, and I end-up parsing XML (DXL/QBXML) to generate the appropriate HTML.

    46Bill McCuistion  05/06/2007 12:09:19 AM  What if.....DXL? (#1)

    One more comment on the RTtidy / RLint validator / cleaner-uper, the PHP routine included several filters to fix encoding schemes and bad chars in the Excel source data.

    I'm sure that I could have implemented all this in pure Java code, but found the (pre-existing and debugged) PHP routines very handy and easy to debug and apply to pulling the data from the XLS files.

    ---------

    As an aside, the DXL-importer allowed the following (invalid) construct, but raised an error message, but none-the-less imported the docs.

    <documents>

    <document> ... </document>

    <document> ... </document>

    <document> ... </document>

    ...

    </documents>

    It would have been much more complex to create one DXL file for each <document> and more complex even to import one DXL per document.

    In the QBXML schema, I'm accustomed to a toplevel <MessageSet> element that can contain 1..n "messages", such as <InvoiceAddRq>, InvoiceModRq, or InvoiceDelRq. In the DXL schema, it would be nice to have "DocumentAddRq", DocumentModRq, or DocumentDelRq, with corresponding responses (Rs) messages.

    47Samuel deHuszar Allen  05/09/2007 5:01:02 PM  What if.....DXL? (#1)

    Just mirroring my comments from the data-only Web-Access thread.

    Build an ODF object to replace Rich-Text, it's XML is not the cleanest in the whole world, but it's fresh, ready, and well-documented. I can only imagine that it'd be easier to craft new bindings and object paramaters than to rebuild the Rich-Text object, forcing it to work in ways that it was not originally envisioned 10 years ago.

    Not a super gifted programmer, but since ODF is becoming part of the Lotus offerings, it just seems to make sense.

    48Pejman  05/28/2007 6:18:57 AM  What if.....DXL? (#1)

    If DXL was 100% functional it would be easy to use Rational Application Developer instead of the designer (or any other tool)