Bob Balaban's Blog

     
    alt

    Bob Balaban

     

    Geek-o-Terica 3: Taking out the garbage (LotusScript)

    Bob Balaban  April 27 2009 04:56:24 AM
    Greetings Geeks!

    LotusScript is a cool language. One of the reasons it's cool is a feature called "automatic garbage collection". What does that mean? It means that you can litter your code landscape with snippets of memory to your heart's content, and never have to worry about keeping track or picking it up ("deallocating", or "freeing" the allocated memory chunks) later.

    It means you can code a loop like this (skipping all the DIMs, but you get the idea):

     set view = db.GetView("$all")
     set doc = view.GetFirstDocument()
     while not (doc is nothing)
          ' do something...
          set doc = view.GetNextDocument(doc)
     wend

    Every time you do "set doc = ", the Notes back-end classes infrastructure is generating a new NotesDocument object in memory. What happens to the object that used to be stored in the "doc" variable? LotusScript automatically keeps track of all references to objects, and when there are none (in this case, when you assign a new object reference to "doc", overwriting the old one), the memory used by that object is reclaimed.

    When does garbage collection happen? It happens in LotusScript after the execution of every statement (which can lead to some interesting side-effects, as we'll see in a minute). It wasn't always like that, though. In Notes V4.0 (the first appearance of LotusScript in the product), garbage collection only happened at the end of each Sub or Function. But that wasn't good enough, because, as in the above snippet, if the View had, say, 100,000 documents in it, you'd run out of memory before you got to the end of the function.

    But, as with all great boons to humanity, automatic garbage collection in LotusScript has its dark side. Consider this:

     dim s as new NotesSession
     dim entry as NotesEntry
     set entry = s.CurrentDatabase.ACL.GetFirstEntry

    Seems innocuous, right? But you'll never get an object in "entry". Why? Because there's another rule I haven't reminded you of yet (you probably already knew it, but you didn't realize it could bite you like this in some cases). The rule is that when an object is garbage-collected, all of its child objects are also garbage-collected.  That might seem needlessly destructive, but it's actually necessary. Notes does not allow free-floating objects that exist outside their container, or "owner" object. Documents, for example, must belong to a database. Items must belong to a document, and ACLEntries must belong to an ACL object.

    So, what really happens in the above snippet? The expression "s.CurrentDatabase.ACL.GetFirstEntry" is perfectly legal in an agent, where the NotesSession will always exist, as will the CurrentDatabase. And we know that every Notes database has an ACL object with at least one ACLEntry in it. So the expression will always evaluate and create an ACNEntry object, which then gets assigned to the "entry" variable.

    But then, once that all happens, garbage collection kicks in. It says, "What can I sweep up that is no longer referenced?" Session and CurrentDatabase are special, they have to stay around until the agent is done. But, AHA! The ACL object isn't used anymore after this statement is done (the object reference is not saved in any local variables). So it gets destroyed. And, invoking the rule above, all of its instantiated children also get destroyed, including the ACLEntry instance that we just saved in "entry". All before the next statement is executed. Easy come, easy go.

    Everything would have been fine if we had coded the above thusly:

     dim s as new NotesSession
     dim entry as NotesEntry
     dim acl as NotesACL
     set acl = s.CurrentDatabase.ACL
     set entry = acl.GetFirstEntry

    This way, the NotesACL object hangs around until the Sub exits, and so our ACLEntry object is also preserved.

    Now consider this bit of logic:

     set item = doc.items(0)

    Do we have a similar problem here? Will "item" always be Nothing? Actually, no, this works. It's true that the array of NotesItem objects created by the "doc.items" property is garbage-collected, but the single NotesItem object we saved in "item" is fine. Why? Because the item is "owned" by the NotesDocument object, not by the array. The array disappears, but the document doesn't.

    Of course you would be making a mistake to code a loop such as this:

     for i = 0 to ubound(doc.items)
        set item = doc.items(i)
        ' do something....
     next i

    This is horribly inefficient, because every invocation of "doc.items" above will create an array, and then populate that array with a set of NotesItem objects. Then at the end of each statement, the array will be garbage-collected, along with all but one of the NotesItem objects. That's a lot of work to do over and over. Much better to access the array one time, save it in a Variant variable, and then iterate over the array.

    Next time: How garbage collection in Java is way different.
    Geek ya later!

    (Need expert application development architecture/coding help? Contact me at: bbalaban, gmail.com)
    Follow me on Twitter @LooseleafLLC
    This article ┬ęCopyright 2009 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.
    Comments

    1David Leedy  4/27/2009 7:17:29 AM  Geek-o-Terica 3: Taking out the garbage (LotusScript)

    Great post Bob!

    I believe that another common example of this, which gets me sometimes, is if I have a script library with a function that's supposed to return a document from a foreign database to an agent in the current database. If you are simply returning the document, you'll actually get nothing since the foreign database object gets garbage collected and wipes the found document with it.

    Something like that. I don't remember all the details. I haven't seen it in a while.

    If you put the foreign database in a global variable of the script library, that corrects the problem and allows the foreign document to be returned.

    2Jan Schulz  4/27/2009 7:47:04 AM  Geek-o-Terica 3: Taking out the garbage (LotusScript)

    Garbage collection has also bitten me, but in a slightly different way in destruction of an object: { Link }

    Anyway: is there a references, which object is the parent of some object? For example: what is the parent of a ViewEntry.document? The database or the view entry or the collection or the View?

    Currently I hold a reference to the doc and then 'intern' every doc, which I get from something else before returning it to someone outside of the Database wrapper. Also I get hold of views via a function, which puts the view into a list, so it's accessible for the lifetime of the wrapper object. This fixed the problems, but I have no idea if it is necessary in all cases...

    ' intern() is a function of a wrapper class around a NotesDatabase

    Public Function intern(doc As NotesDocument) As NotesDocument

    On Error Goto stackError

    If doc Is Nothing Then Exit Function

    Set intern = Me.getDocumentByID(doc.UniversalID)

    Exit Function

    stackError:

    Call AddToStackTrace()

    End Function

    All in all: this is a really annoying "feature" of LS...

    3Bill  4/27/2009 11:55:59 AM  Geek-o-Terica 3: Taking out the garbage (LotusScript)

    Ah. We use complex objects in LS, which in turn may contain lists or arrays of other complex objects.

    For a while we got uber-OO-compliant, using destructors to delete child objects that we'd created.

    And then it went all ABEND-like, as the garbage collector tripped up over it..

    Sometimes, its best just to leave the garbage collector do its thing.

    ---* Bill

    4Bob Balaban  4/27/2009 2:54:38 PM  Geek-o-Terica 3: Taking out the garbage (LotusScript)

    @1-Thanks David, good example of another thing to watch out for.

    @2 - Good question. I think I documented the state of object ownership in my book on Notes v4.6 and Java (the ownership rules are the same for both LS and Java), but of course things have evolved a lot since then (and the book is out of print anyway).

    In general, Documents are owned by the Database, because there are multiple ways of navigating to them. Saving references to objects is a fine way to prevent unwanted garbage collecting, but be sure you aren't doing it so much that you'll run out of memory.

    @3 - Good point, Bill. I'll extend your recommendation to say: It's ALMOST ALWAYS best to just let the LotusScript garbage collector do its thing. There are exceptions, of course.

    Trying to treat LS as if it's C++ with destructors and such seems like a LOT of extra typing to me.

    5Lars Berntrop-Bos  4/27/2009 3:20:15 PM  Geek-o-Terica 3: Taking out the garbage (LotusScript)

    Brilliant stuff, Bob.

    Finally I know why stringing a lot of object refs backfires.

    Keep up this excellent work!

    The book may be out of print but has a place on my shelf anyway!

    6Daniel Lehtihet  6/15/2009 4:57:38 PM  Geek-o-Terica 3: Taking out the garbage (LotusScript)

    Hi Bob,

    Very interesting reading. One question regarding loops: what about the delete statement? Is it redundant or does it serve a purpose (in addition to later assign a doc reference to "Nothing" when dealing with very large views for example. I am dealing with out of memory problems on a large AIX cluster where everything, except delete, is implemented. Any opinions?

    Kind regard

    Daniel

    7Bob Balaban  6/15/2009 11:20:39 PM  Geek-o-Terica 3: Taking out the garbage (LotusScript)

    @6 - Thanks for reminding me, Daniel! I just created a post on that topic:

    { Link }

    I have never needed it for the purpose you describe, however.