Bob Balaban's Blog

     
    alt

    Bob Balaban

     

    Drain Google: Sample code to get all your stuff back from GoogleApps

    Bob Balaban  July 26 2010 12:45:00 PM
    Greetings, Geeks!

    Have you tried out Google Apps yet? Are you using any of their facililties for file sharing, email, collaborative web sites, or even for posting videos (YouTube)? If so, the you may have noticed that over time you've accumulated a lot of assets in Google's "cloud".

    But what happens when you want all of that stuff back? What if you gave lots of people free GMail accounts, but then you decide you need to upgrade them to a real enterprise messaging system? Or what if you decide you just don't like GMail that much? What if you decide to move from ad hoc file-sharing on GoogleDocs to an integrated Domino/Quickr "content management" system? What if hosting mission-critical business documents in the "cloud" just makes you nervous?

    Given that broad context, I've written a Java application (runs in Eclipse, or standalone) to log into a specified Google account, find all the objects in GoogleDocs belonging to that user, and download them into a Notes Document Library NSF. I'm posting the source code for that app here, as a sample, to illustrate how it can be done.

    Why? I have a couple of reasons. One, to share some basics about the Google APIs, which I think are a good example of RESTful web services technology. Two, to explain that this is a piece of what I hope to be a larger project (which I'm calling "Drain Google"), for which I'm seeking funding, to create an app that will download ALL of your assets residing anywhere in GoogleApps (docs, sites, YouTube, mail, etc.).

    It Is What It Is

    Legal stuff: This is a sample, no claims are being made as to functionality or reliability. Take, it, modify it, use it, give it to someone else. The sample code is Copyright 2010 by Looseleaf Software. If you make money on my code, you should give me some.

    The sample app relies on the Google Apps Client API library for Java, which you can get here (basic data API), and here (GoogleDocs API). One nice thing about the Google API kits is that they give you source code, so you can compile it yourself, and debug into it using Eclipse. Good stuff!

    The sample does a search on a specified GoogleDocs account, and then does a 1-way download to your computer, storing the document (along with some metadata) in the specified Notes DocLib database. It contains some logic to try to figure out what the source document format was (much harder than you'd expect, since GoogleDocs does not "remember" the original file format of things you upload), and it parses folder tags, translating them into DocLib categories.

    One could take this sample and expand it to do 2-way synchronization with small-to-moderate levels of effort, but the sample as-is does not address that functionality. My long-term goal (see comment about funding, above) is to expand this to include other Google sources (Sites, Mail, Calendar, etc.), and to allow for 2-way synchronization of objects.

    It Isn't a Notes Agent

    This little app would make a nice Notes Agent, but it isn't currently configured that way. I did try, but ran into some....issues with the Notes Java Security Manager that prevented it from working. I couldn't run it in Notes 7, because the Google libs require a newer version of the JRE. Notes 8.5x has the right JRE, but the Security Manager got in my way, and I could not develop a workaround. Maybe I'll blog about that in more detail later. For now, you're limited to running this as a standalone Java application. It takes command-line inputs specifying your Google name, password and the path to the destination Notes NSF. Of course, it uses the Notes Java APIs to store the files in the database.

    All of the code is in one Java class (Documents.java). Not terribly complicated, but not boringly obvious, either (IMHO). I hope you can use it to learn something about how the Google APIs work, and maybe even to have some geekfun. Feel free to post comments and/or questions about the sample to this blog.

    Enjoy!
    Documents.java

    (Need expert application development architecture/coding help? Contact me at: bbalaban, gmail.com)
    Follow me on Twitter @LooseleafLLC
    This article ┬ęCopyright 2010 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.
    Comments

    1Chris Miller  7/26/2010 1:00:41 PM  Drain Google: Sample code to get all your stuff back from GoogleApps

    That is very cool! Can I reference this in my Google Apps migration series?

    2Jack Dausman  7/26/2010 1:17:24 PM  Drain Google: Sample code to get all your stuff back from GoogleApps

    Wow. Impressive and makes sense. Sounds like a building block to move GoogleApps to LotusLive. If I can think of a way to sell it, then it absolutely would be licensed from LooseLeaf.

    3Jeff Gilfelt  7/26/2010 1:18:57 PM  Drain Google: Sample code to get all your stuff back from GoogleApps

    Um, so you're going to try to compete with Google's own efforts in this space? { Link }

    Good luck.

    4Bob Balaban  7/26/2010 1:19:52 PM  Drain Google: Sample code to get all your stuff back from GoogleApps

    @Chris - Please do!

    5Bob Balaban  7/26/2010 1:25:05 PM  Drain Google: Sample code to get all your stuff back from GoogleApps

    @2 - Thanks Jack!

    @3 - Jeff, I'm not competing with anyone. This is a sample, the primary purpose is pedagogical. Having said that, there are some tricky things about recovering assets from Google (I'm planning a future post on some of those details).

    Thanks for the link, interesting site. I wonder if the Google people are really going to give the same air time to getting stuff out as they are to getting stuff in. And of course the GoogleApps API team may have a somewhat different evangelism agenda from the one held by the GoogleApps marketing (/migration) teams.

    6Stephan H. Wissel  7/26/2010 8:41:13 PM  Drain Google: Sample code to get all your stuff back from GoogleApps

    Neat. Let Damien know about it. Google -> CouchDB is a nice idea.

    7Eric Wilson  8/4/2010 2:50:38 PM  Drain Google: Sample code to get all your stuff back from GoogleApps

    Gosh! What a great idea!