Bob Balaban's Blog

     
    alt

    Bob Balaban

     

    Market Opportunity: Someone should create a directory service based on RDBMS

    Bob Balaban  September 10 2010 06:00:00 AM
    Greetings, Geeks!

    I had an idea a few years ago that it would be a great thing for IBM (or someone) to create a directory service that could be used BOTH for server and application security AND for application services. Like the directory in Domino does for Notes and Domino app builders today, but based on a platform (RDBMS) that everyone already has.

    When I (re-) joined IBM in 2005, I wrote it up on an internal blog site. I think I got 3 comments. But I still think it's a good idea, and, so far as I can tell, nobody's implemented it yet. See what you think.

    Here's a series of propositions that lead me to think that directory services should be based on relational database technology, especially to support collaboration technologies:

    1. Collaboration technology rests, as one of its major foundations, on directory services. So do many other kinds of applications.
    2. Most companies treat directories as security mechanisms only
    3. Developing collaboration-based applications depends in part on access to good directory services, and that means a lot more than mere security
    4. LDAP is good, but not good enough for collaboration apps
    5. Directory services would improve dramatically if they were based on a transacted, relational data store
    6. Nobody (that I know of, anyway) is combining these 2 technologies
    7.Somebody should

    I don't want this to turn into a book, so I will be brief in addressing these points.

    1. Collaboration technology rests, as one of its major foundations, on directory services
    Collaboration online, as well as a lot of other things that happen online, depend on the application (or the server) authenticating the user before allowing access. You have to know who the user is before deciding a) what they're going to be allowed to do (authorization), and b) to track what they did when they've done it. Without reliable authentication, you don't have either security OR collaboration (you can't collaborate when you don't know who is contributing to your content/workflow). Authentication and authorization both depend on access to a trusted directory service. Furthermore, directories are the right place to store the organizational information (e.g., "reports-to" hierarchies) that are required for most kinds of workflow apps.

    2. Most companies treat directories as security mechanisms only
    Again, I think this is obvious by inspection. Certainly Microsoft has this attitude -- just look at any of their so-called Single Sign-on programming examples for .net. They all depend on looking up a password or something in a SQL database, nobody (in her right mind) would try to store that stuff in Active Directory.

    3. Developing collaboration-based applications depends in part on access to good directory services, and that means a lot more than mere security
    Anyone who has developed a workflow app knows this. Beyond authentication you need authorization. And it has to accommodate per-application role-based stuff, not just coarse-grained access control. And you need it exposed in a way that's easy for AppDev types to use (LotusScript, @functions, whatever), in addition to the usual declarative mechanisms (e.g., ACLs). The directory "schema" needs to be extensible to accommodate random applicatoin needs.

    4. LDAP is good, but not good enough for collaboration apps
    The beauty of using a standard protocol like LDAP to access directory services is that you don't have to care how the service is implemented, right? Well, only if "you" are an application developer. Then you don't, but if you're an admin, you sure do have to worry about it, because you have to build, populate and maintain it. These things are expensive, especially the maintenance part. Extensibility is difficult, because the underlying schemas are usually too rigid (this is an area where Domino really shines, the directory is "just another database", and it's not hard to extend the schema). I had the misfortune recently to have to use Active Directory APIs in a .NET app I built to test whether a given string represented a valid user name. Sheesh. All I can say is, it might be inflexible, but at least it's really hard to use.

    J2ee says nothing about directories -- "just use LDAP". So they punt, which is reasonable for an app server, it gives you the ability to plug in the best available product. If you were going to implement a directory server with large data storage requirements, and with the ability to do a number of the things I mentioned above, what would you pick? I, for one, would NOT pick ISAM files. I've already said I like the flexibility provided by Domino's NSF solution, but of course that's not practical for all enterprises (especially if they don't have or want to install Domino).

    5. Directory services would improve dramatically if they were based on a transacted, relational data store
    What are the characteristics a "good" directory service MUST have? Accessibility (no problem). Standard protocols (you can layer LDAP on top of nearly anything). Easy to administer/populate/query/maintain volatile data (that's what relational is all about). Secure (duh). Must be EASY to extend to handle arbitrary, per-application, per-user and per-group data (this is almost trivial in the relational model, just create a new table and join....).

    6. Nobody is combining these 2 technologies
    I know Microsoft isn't (yet), nor is IBM. I discussed this topic with a Microsoft exec a few years ago (again, before I was an IBMer). I asked why they weren't doing this already (they have a good dbms too). "Too hard" he said. Personally, I don't think much of that as an excuse. If it's important enough, you find a way. But it's good news for everyone else -- it's a wide open space, with no other serious players. Google has yet to release an enterprise directory service of any kind, let alone the kind I'd like to see...

    9. The opportunity is there!
    Someone could take a neglected area of online computing infrastructure and elevate it into a first-class member of a collaboration suite. They could own THE directory technology foundation of the next 25 years. Think of the patents! The market share! Think of all those happy, happy application developers (not to mention end-users)!

    Of course, there are problems. You need more than just fast lookups and the ability to handle a lot of data. You need to add application artifacts (which, by definition, are not knowable in advance) seamlessly. And eventually that means that you have to modify the directory schema in some way or other. In the worst case, you might have to modify it for each class of applications that your enterprise needs to deploy (and that can be a lot). Sure, you can create generic schemas for application artifacts, up to a point, but the more you do that, the more you lose the ability to index in meaningful ways, and the more you slow down access. Sooner or later you're going to want to modify the schema.


    (Need expert application development architecture/coding help?  Want me to help you invent directory services based on RDBMS?? Contact me at: bbalaban, gmail.com)
    Follow me on Twitter @LooseleafLLC
    This article ┬ęCopyright 2010 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

    Comments

    1Simon Mottram  9/10/2010 6:26:38 PM  Market Opportunity: Someone should create a directory service based on RDBMS

    Wrap up and RDMBS via OpenLDAP. I know at least one company that has done this for an internal project. I think it was a DB2 backend

    Too hard? Well it wasn't easy but the guy that did it was new to OpenLDAP and he managed ok.

    2jake  9/12/2010 3:08:22 PM  Market Opportunity: Someone should create a directory service based on RDBMS

    Intriguing idea, but I'm still not convinced of the business case. All that for basically LDAP on ACID? (heh) If you are or you are passionate about it, Bob, you should definitely start coding. Damien started CouchDB with essentially this mission statement:

    "Couch is Lotus Notes built from the ground up for the Web." { Link }

    Which strikes me as less of a business case than you've put forth.

    You could do worse for your relational directory. Also, too hard is not necessarily true. MS SQL Server has, since the 2008 version, a hierarchyid datatype and T-SQL dedicated to data hierarchies/parent-child relationships. I bet this stuff was baked in specifically for Sharepoint (and we know where that design inspiration for that came from) but it would be a good start for storing tree structures in an ACID-compliant datastore. There is also an XML datatype that can be queried using XPath syntax in your T-SQL that could potentially be used custom attributes. If SQL Server has it, how far behind (or ahead) could the other major RDBMS vendors be?

    3Bob Balaban  9/12/2010 3:34:46 PM  Market Opportunity: Someone should create a directory service based on RDBMS

    Hi Jake - thanks for the comments. The whole point is that it's not JUST for LDAP. ACID gives you everything ELSE you need to do appdev (workflow, authentication, authorization, routing, etc etc) PLUS the basic LDAP services.

    I did think about Damien and the CouchDb model, but it took him a couple of years on his own, then another year-plus at IBM...

    4Dwight Wilbanks  10/4/2010 7:54:42 PM  Market Opportunity: Someone should create a directory service based on RDBMS

    I'm feeling a bit lame because I don't get it. What would be a use case?