Bob Balaban's Blog

     
    alt

    Bob Balaban

     

    Does Anyone Use NSFDB2 for real?

    Bob Balaban  October 21 2007 05:30:00 AM
    Greetings, Geeks!

    I'm curious about this. I think Db2 (ok, officially it's "DB2", but I prefer to type it my way) is cool. Actually, I have thought that  relational databases in general are cool ever since I learned about them and started using them in the late 70s.

    When I first heard about NSFDB2 I was President (and Janitor) of my own company, and to be honest, I had mixed feelings about the wisdom of the project. My greatest reservation was a technical one: how the heck could they ever stuff the round-peg, loosey-goosey, "semi-structured" data model of Notes NSF into the square-hole, highly-structured, not exactly rigid, but, well, semi-rigid data model of an RDBMS?

    Well, they figured it out. It works. Doesn't have the greatest performance (yet) perhaps, but the power is there: SQL queries on Notes data; @function accerss to relational data; joins across Notes databases! Very cool.

    So, this should satisfy all those developers who for years have been asking for relational capabilities in NSF. For years our answer always was: "Notes is not a relational database, and was never meant to be. If you need a relational database, go get a relational database. We give you lots of tools for moving Notes data into and out of RDBMS..." All true, but tiresome.

    So, now we've got it, and I'm wondering if anybody is actually using it. And if so, for what?

    Personally, I think the "killer app" for NSFDB2 is reporting. Why? Reporting has always been an area where Notes has been relatively weak. Lotus/IBM just never invested much in it, and really, reporting was just never considered a high priority for a collaborative application platform. Crystal Reports and other companies provided some add-on tools, but people always complained that Notes didn't have a real native reporting capability.

    Compare with the widespread availability of SQL-based reporting tools for relational database systems. Lots and lots of choices, from the simple to the very sophisticated. Now that your NSF data can be exposed via NSFDB2 and SQL, why aren't people jumping all over this to do reporting? Or, maybe they are and I just haven't been hearing about it?

    My friend and former colleague, Peter O'Kelly, once told me: "If you want to know something, ask! People know." So, I'm asking!
    Comments

    1Andrew Price  10/21/2007 7:16:23 AM  Does Anyone Use NSFDB2 for real?

    We don't, but might like to since we currently port all our data to SQL Server for reporting. We have our own tool for doing this, written in LS. It would be cool to have it all in DB2 natively, but it means learning all about DB2 and we're currently pretty happy with MS Reporting Services, so no switch yet.

    :)

    2Theo Heselmans  10/21/2007 7:48:00 AM  Does Anyone Use NSFDB2 for real?

    I don't either. The reason: none of my (current) customers use DB2. They are using Oracle and SQL-server instead. I would love to use it (particularly the sql-views), as it would mean up to date 'grouped' views. Please add the same functionality for other RDBMSs.

    3Tom Roberts  10/21/2007 9:19:24 AM  Does Anyone Use NSFDB2 for real?

    I am another that is "interested" in the possibilities presented by this, but I haven't done any testing or even pursuing further than the knowledge that it exists. The reporting capabilities are the driver for considering it, but it hasn't gone any further due to time restrictions to pursue. If someone was able to show me a solution, a path, and a realistic time estimate to implement, I would have better information to consider it. Guess I'll head over to redbooks and see what's been written there...

    4Ian Scott  10/21/2007 9:34:47 AM  Does Anyone Use NSFDB2 for real?

    We're an Oracle shop and it's simply inconceivable that we'd ever deploy DB2. I think that's a shame but it's understandable. I also think there are probably a large number of businesses in the same situation.

    I realise there's no money in it for IBM but if, as per @2, the same functionility could be introduced for other RDBMS's (ie Oracle) we would inevitably use it.

    5Jeff Gilfelt  10/21/2007 9:49:29 AM  Does Anyone Use NSFDB2 for real?

    I would love to use it for real, unfortunately IBM do not allow it. From the 7.x release notes:

    "Domino 7 with DB2 is released with Limited Availability; as such, it is not available for general usage, but is provided for evaluation purposes only."

    6Erik Brooks  10/21/2007 10:06:51 AM  Does Anyone Use NSFDB2 for real?

    I would *love*, LOVE to get NSF data in DB2 for reporting purposes. But I'm with @5 - the first version of Domino where NSFDB2 was officially supported "for general usage" is barely two months old.

    Once we test Domino 8 and migrate they we might play around with it.

    7Erik Brooks  10/21/2007 10:14:19 AM  Does Anyone Use NSFDB2 for real?

    Also, anybody running i5OS has likely felt somewhat shafted with recent NSFDB2 plans. We're not using it, but it might interest you:

    { Link }

    There's been a lot of discussion about this on the Domino Server Team blog.

    { Link }

    8Nelson Morris  10/21/2007 10:31:24 AM  Does Anyone Use NSFDB2 for real?

    Is the the answer to "better reporting" in Notes putting data into DB2? I understand the choice of DB2 in that one probably had to "hack" the RDBMS to understand Domino constructs like readers fields, etc. But are there other choices to allow more "hooks" for these other reporting tools or to provide something beyond out of the box view capabilities?

    "Collaborative application platform" does not mean there is no need for improved reporting. Maybe better Notes Java views in ND8 might help.

    Our only choice currently is to pump data out into SQL or Oracle.

    9Daniele Vistalli  10/21/2007 10:45:33 AM  Does Anyone Use NSFDB2 for real?

    I'm using it in three different projects at a customer.

    Two are related to access NSF content in a new way (using SQL) to integrate content created by partners (product catalog) into product management system.

    The last one is a CRM app we're building around Domino. In order to keep information in sync with other systems (erp and more) we use NSFDB2 and federation to achieve the result.

    Development went from weeks (with many limits due to manual SQL operations and RDBMS sync) to days with transactional guarantees.

    So far so good !!! Looking for more with DB2 9.5 coming with XML manipulation even if roadmap for NSFDB2 is not yet clear.

    In Italy many were disappointed by IBM dropping NSFDB2 on System i (the place were it really made o a lot of sense)

    10Warren Elsmore  10/21/2007 1:15:01 PM  Does Anyone Use NSFDB2 for real?

    We've got a customer that tested their app on NSFDB2 for exactly that reason. They ran it under 7.0.2 with a 1000% increase in speed (yes, one THOUSAND percent).

    I've not seen them for a few weeks, but the plan was absolutely to move to Domino 8 asap, purely because NSFDB2 would be supported. I will be back there soon...

    11Bob Balaban  10/21/2007 3:13:28 PM  Does Anyone Use NSFDB2 for real?

    @5 - It is fully supported as of v8.0

    12Vitor Pereira  10/21/2007 5:38:56 PM  Does Anyone Use NSFDB2 for real?

    You're next question should be - How many of you would use NSFDB2 if we were to support it on System i?

    13Roland Reddekop  10/21/2007 8:27:47 PM  We planned to use NSFDB2 extensively!

    We planned to use NSFDB2 extensively! (note the past tense).

    Our ERP is DB2, most of our data, reporting, in house application use DB2 natively...and we have a ton o' Notes/Domino apps that also read/write from DB2 via LCLSX and to a smaller degree DECS. So the direction toward NSFDB2 for us was a marriage made in heaven and we held off of other integration options like LEI and other third party products because...well DB2 access and query views would do it all! Life was getting good.

    BUT then came the news within weeks of the release of Notes 8.0, "Sorry, there are no plans to enable NSFDB2 on System i (aka iSeries, AS/400, i5/OS...whatever). But don't worry, we are COMMITTED to your platform."

    Seems actions don't really correspond to that proposition. Historically, its typical IBM modus operandi towards the System i customer. Man we're a bunch of whiners.

    Ironically, while System i customers who are committed to DB2, confident in it and who would be THE PIONEERS to maximize the potential of NSFDB2 are out of luck. And now we have to sit back frustrated while IBM wonder, "Hey are any of you Win, nix, or aix customers out there using this?" <silence> (sorry Bob, I am not intending to be mean as I know your intentions are great in asking the question...more power to you).

    I am looking forward to the November webcast where IBM will be giving some answers to the System i/Domino faithful:

    { Link }

    14Dan Sickles  10/21/2007 11:13:55 PM  Does Anyone Use NSFDB2 for real?

    We're looking at it for the usual reasons. Mostly static and incremental data (TBs) for reporting, not live apps.

    15Ingo Erdmann  10/22/2007 2:13:26 AM  Does Anyone Use NSFDB2 for real?

    I think it's a great piece of technology. And there are wonderful things you can do with NSFDB2, especially by utilizing Query Views.

    In my personal opinion, we don't have a customer (yet) because of two major caveats:

    - There is no way for developers (yet) to create a generic QueryView that contains relational data (with all the interesting stuff like joins and unions) that behaves like a real Notes view.

    - There is no offline functionality. I am not talking about putting DB2 to local, but take some other low footprint relational store, such as the one that I believe is an optional part of Expeditor.

    16Ralf M Petter  10/22/2007 2:16:17 AM  Does Anyone Use NSFDB2 for real?

    I would really like to implement DSFDB2 in our CRM application, because it will be a real killer function if we can access CRM Data from our ERP System programmed in RPG, but IBM has killed the support of System i after a promising Beta. So i think almost everyone which uses an i/5 is using DB/2, so every Domino installation on i/5 can benefit from NSFDB2, but how much customers running Domino on Windows are running DB/2?

    17David Leedy  10/22/2007 6:50:44 AM  Does Anyone Use NSFDB2 for real?

    I would like to play with DB2... But there are two stopping points for me... 1 is almost all of our Domino servers are on System i(iSeries/AS/400).. the other is I'd love for it to replicate and I don't think it does that. I have an app that this would really help the performance I think but we have a lot of people that replicates it locally...

    bummer...

    18Bob Balaban  10/22/2007 8:10:22 AM  Does Anyone Use NSFDB2 for real?

    @17 - Replication most certainly does work with NSFDB2 databases, both server-server and client-server. In fact, everything that works with a traditional NSF format database also works with an NSFDB2 format database: replication, full-text index/search, message delivery, LEI, everything.

    19Erik Brooks  10/22/2007 8:21:20 AM  Does Anyone Use NSFDB2 for real?

    Since I haven't looked yet - can you access the resulting XML data from a query view over the web in ND8 via ?ReadViewEntries? And similarly, does ?ReadDesign work?

    If the answer is "no" then that's a show-stopper for me.

    20Charles Robinson  10/22/2007 8:35:35 AM  Does Anyone Use NSFDB2 for real?

    No, I don't use it, mostly because of a lack of time to refactor applications to incorporate it. I've done small-scale testing and it works perfectly, and it allows me to easily share information across Notes applications. So I love the concept, it works well, the limiting factor is simply time to make use of it. Being a one man show is exhausting at times. (but I'm not telling you anything you don't already know)

    21Wayne Sobers  10/22/2007 10:23:41 AM  Does Anyone Use NSFDB2 for real?

    I will use it to off load historical data from our service management system onto a less expensive Linux system.

    I would love it if IBM would take the report component out of Approach and make it native to Notes, then you would have a complete Linux desktop solution.

    The sad thing is,if Notes had better user input tools we could have used Notes & Domino instead of getting third party software.

    22Adam Fenstermaker  10/22/2007 12:05:41 PM  Does Anyone Use NSFDB2 for real?

    We are considering moving a large NSF db to NSFDB2 as we are re-writing it to be web-based. Since we don't already have DB2 in house, this means learning the DB2 technology from scratch. I haven't really seen or heard of any "show stoppers" with NSFDB2 - only documented bug I've seen with R8 is this problem with unread marks - { Link } - are there others I should be aware of before we jump in to a feasibility study?

    23Carlos Rivera  10/22/2007 2:01:07 PM  Does Anyone Use NSFDB2 for real?

    I agree with alot of people that IBM should provide the functionality for other RDBMS, since DB2 is not the most popular in the market.

    24Daniele Vistalli  10/22/2007 3:14:13 PM  Does Anyone Use NSFDB2 for real?

    Last september I gave a presentation on NSFDB2 at the lotus community event here in italy.

    One of the resources I pushed to domino admins is this

    { Link }

    This is a free tutorial that brings you from 0 to cluster with DB2.

    I suggest everybody interested to take this as starting point. You don't need more to handle NSFDB2.

    Other considerations from the event here in Italy:

    - Everybody is "interested"

    - A few are investing in it and after initial "struggle" they're seeing the good side (I'm also moderating our italian forum on NSFDB2)

    - Many are still getting what's new

    - Everybody is hungry about the bad System i story

    So... the technology is great but if you can get it back on System i everybody will use it.

    I also hope to see more and more db2 around because i love it on Linux/Win/AIX .. but it seems IBM is not fighting the RDBMS battle at this time (even with a great product).

    25Cristian D’ALOISIO  10/23/2007 8:21:26 AM  Does Anyone Use NSFDB2 for real?

    I'm using NSFDB2 since Domino R7.

    At the beginning I used NSFDB2 on Domino for System i (= AS/400), then IBM decided not to support it anymore on Sustem i!

    So I had to switch to NSFDB2 on Domino for Windows using DB2 Federation to access data from a remote System i.

    One customer, a big internation company, is really committed to use NSFDB2 because it's very handy to mix data from Domino and from ERP system, in real-time. No more LEI or custom code to transfer data from AS/400 to Notes DBs .

    Take a look to my posts about NSFDB2 on my blog:

    { Link }

    26Bob Balaban  10/23/2007 8:26:58 AM  Does Anyone Use NSFDB2 for real?

    @19 - Query views are views, plain and simple, the one big difference being that the data does not reside in an NSF. In all other respects they act the same as "regular" Notes views (i.e., from the CAPI level, you can't tell the difference).

    @21 - I love Approach too, but it's a Win16 application, I don't think your idea is really practical.

    27Erik Brooks  10/24/2007 8:54:10 AM  Does Anyone Use NSFDB2 for real?

    @26 - Great! I was hoping that was the case. For me, the initial draw is the ability to join views across multiple NSFs, so my query views would simply work with DAVs in Notes. I'd (initially) only have DB2 around to do the actually join -- no federated data would be involved.

    That should go a pretty good distance for us in making Notes pseudo-relational. You don't get some of the nice things like referential integrity, but it's a definite step in the right direction.

    28Pejman  10/25/2007 3:48:45 AM  Does Anyone Use NSFDB2 for real?

    my customer wanted a real web relational product catalog.

    I had the choice between the classic J2EE solution (struts, hibernate, spring whatever...) , more mature but heavy to integrate with the legacy Domino apps

    or

    take the risk (at November 2007, DB2NSF had no web capabilities) to start a real relational project with a young technology.

    It's was I did and I presented to my customer the advantage of its integration with the legacy applications. We signed for the limited availability program and started the adventure. In fact I took to my own all the extra effort asked to make the catalog work. I didn't want my customer to pay for that. It so long that I was waiting for this technology that I was quite proud to participate to its maturation.

    the web site is based on 7.0.2. All web views are dynamic and the SQL parameters are injected via URL.

    { Link }

    29Pejman  10/25/2007 3:54:26 AM  Does Anyone Use NSFDB2 for real?

    Forgot another application that I'm working on.

    I developed 3 years ago a little solution for tracking the user on the a web site (browser, OS, path, the time he stays on a page etc ..)

    Now with the new NSFDB2 capabilities, I can replace all aggregation agents with a few SQL queries.

    30Adam Fenstermaker  10/25/2007 8:24:31 AM  Does Anyone Use NSFDB2 for real?

    Bob - Does IBM use (or is considering using) NSFDB2 for any of its in-house Notes/Domino applications?

    31Wayne Sobers  10/25/2007 8:41:00 AM  Does Anyone Use NSFDB2 for real?

    @26, I don't think you should *use* Approach, its' performance limitations were obvious back when it was grafted to the 1-2-3 spreadsheet, but there should be some kind of basic report writer module.

    In my environment there are a lot of what IBM would class as very small businesses. These companies end up using Access/Foxpro/Filemaker type applications, primarily because the interface can be developed quickly and because they can produce reports, invoices, quotes etc directly from the application. No third party reporting tools needed, no messing with extra run times.

    To me, there are only two reasons to use ms access for development, 1- better keyboard input handling/validation and 2- the built in report writer.

    On a different track, are there going to be personas for development personnel as there are for users and admins?

    32Michael Bourak  10/25/2007 9:00:08 AM  Does Anyone Use NSFDB2 for real?

    @28 : You meant november 06, not 07 ;)

    Great exemple of NSFDB2 usage though !

    Now, you just have to make them upgrade to ND8 :-)

    33Roberto Boccadoro  10/26/2007 6:12:56 AM  Does Anyone Use NSFDB2 for real?

    Bob, I agree with you; when I started using NSFDB2 one of the first things I saw this feature could give us was indeed the capability of doing reporting easily. I'd add another one : Business Intelligence. Now that we can have data in DB2, we can easily apply BI tools to make data analysis on Domino data, which was not easy before. I demoed this at LS07 when I used AlphaBlox with a Domino application. Is not that hard to do :-)

    RoB

    34LongLiveLotus  11/04/2007 7:53:12 PM  Does Anyone Use NSFDB2 for real?

    We're a smb that run the business on Notes (apart from ERP). We have little need to integrate the two systems and no driver whatsoever to use dB2. What will it give us that we havent already got except hassle, complexity and cost?

    35Scott  11/05/2007 6:49:35 PM  Does Anyone Use NSFDB2 for real?

    Ever since the idea was presented to me I can see a use for NSFDB2 in almost every app we are involved with ... and thats before you consider reporting. I just need to convince the right people to get it setup.. :-)

    36Nick Radov  11/06/2007 5:14:07 PM  Does Anyone Use NSFDB2 for real?

    We were very excited when we first heard about NSFDB2 as we assumed that would allow us to finally break the 64GB limit for Domino databases. But alas that artificial limit still applies even though DB2 on its own can handle much larger databases. So for us there isn't much point to NSFDB2.

    37Michael Bourak  11/07/2007 4:42:17 PM  Does Anyone Use NSFDB2 for real?

    @36 : let me disagree : as with NSFDB2 you can have query views being unions/join of multiple domino db, you can perfectly imagine having a single notes app frontending 10 domino db being 50 GB each...for the end user, it's as if it was a "500GB" application ;)

    38Bill McCuistion  11/08/2007 9:37:44 PM  Does Anyone Use NSFDB2 for real?

    Bob, I am working up a use-case for aggregating a SQL-Query against as many as 1-million NSF files.

    DB2 integration on the back-end is the hope-and-dream to make this work.

    My development team thinks that DB2's ability to embed XPATH queries in a SQL query is uber-cool. We need to determine if the NSFDB2 facility will allow us to get to the real power of UDB based upon NSF-based data.

    39sam  11/14/2007 12:37:00 PM  Does Anyone Use NSFDB2 for real?

    I have a very large application with complex views, when I transfer it to Domino 8 server under db2. Some views causes error and did not open. I have to re-test again. Besides it failed to open some 15 GB dbs running smoothly under Domino 7.

    These are serious issues that might prevent us from upgrade. I will investigate it more the following 2 weeks and update you.

    40Andrew Tjecklowsky  11/16/2007 9:20:40 AM  Does Anyone Use NSFDB2 for real?

    I could have said almost identical of Roland (@13).

    IBM.... please reconsider DB2NSF natively on System i (AS/400).... and make it the "Viper" version as well.... you have the worlds best business machinery!!!

    41Scott  11/19/2007 7:48:50 AM  Does Anyone Use NSFDB2 for real?

    As I mentioned previously I'm keen to get this implemented. Of course that means building a case for it etc.

    So is anyone willing to share experiences (Warren your example sounds good - and might be relatively local!). Being able to talk to anyone would be fantastic. Drop me an email.. cheers.

    42Michael Weitzel  12/03/2007 3:14:54 AM  Does Anyone Use NSFDB2 for real?

    I think the problem about NSFDB2 in production systems is two-fold.

    One like another comment pointed out the technology is only commercial since the 8.0 release... Still early days. Also bear in mind that there are plenty of Domino shops out there that have not yet upgraded to 8.0. That causes a huge challenge for the technology to be used in production systems.

    Also as frequent other posts have pointed out, not many companies are actually DB2 shops on the RDBMS side (or rather many companies use 'competitive technologies). While I understand IBM's vested interest in promoting the DB2 platform through technology like NSFDB2 it still means that the Oracle or SQL Server (dare I mention MySQL?) shops won't use it.

    Mike.

    43Werner Herzog  12/03/2007 7:39:52 AM  Does Anyone Use NSFDB2 for real?

    Hello,

    eg. we do not have any deployments in Austria.

    Oe of my customers is looking at nsfdb2 very seriously.

    I would be glad if anyone could mail me actual deployment references.

    thx.

    pls. send mail to:

    werner_herzog@at.ibm.com

    44Nick Radov  12/04/2007 2:28:17 AM  Does Anyone Use NSFDB2 for real?

    @37: I am aware of that option, but it isn't really practical for our application. Again, unless we can put more in a single database then there isn't much point. Hopefully IBM will address that limitation soon.

    45Craig Wiseman  12/21/2007 5:51:29 PM  Does Anyone Use NSFDB2 for real?

    I looked into getting it up and running. It's complicated, very picky about exact DB2 versions and platforms.

    The word that comes to mind is Kludge.

    Typically, upgrading Domino is pretty simple, and adding this seems like it's going to make upgrades much more complicted and dramtically increase the dependency chain.

    All for not a lot that couldn't be implemented in a cleaner and simpler way.

    I'd say put a stake through it's heart and add basic SQLfunctions to the core NSF.

    46Vlada  12/27/2007 12:11:54 PM  NSFDB2 and LIMIT SQL statement

    Hello, is it possible to use LIMIT statement in Query View?

    Something like:

    SELECT * FROM mydb.DAV LIMIT 0, 5