Using Google Sites to Manage Meetings & Events

Using Google Sites to Manage Meetings & Events describes how to use Google Sites to coordinate and communicate among Conference and Events project team members.

Ruby and Sinatra Just Works

To be filed under the pleasant surprise for a late Friday technology jaunt, I decided to see how difficult it would be to install and get the basic "Hello World" web application running using Ruby with the Sinatra framework. The pleasant surprise was that it took less than 5 minutes! I was shocked!

Here are the steps I took to get it running on a Ubuntu 9.04 Virtual Machine:

1. Execute the following commands from the command prompt:
$ sudo apt-get install ruby
$ sudo apt-get install ruby-gems
$ sudo gem install sinatra
$ vi myapp.rb (see instructions here)
$ ruby myapp.rb

2. Point your web browser at http://localhost:4567/ and it just works! Amazing. Of course that is probably not the best way to install the latest version of ruby or ruby gems, but it works. I am going to quit and go home now. Have a great weekend.

Finger Pointing Internet Style

This morning was spent diagnosing problems with a web application. It would be one thing if it were our web application, but It wasn't. It was a 3rd party web application that one of my clients links to. And it didn't work.


Background
This problem has actually been going on for a couple of months. The organization responsible for this web application would provide my client with a link to use to access a landing page specifically designed for the client. My client would then test the link, and send back a message telling them that it was broken. Days or weeks would go by, and the process would repeat itself.


Finally it was getting down to the wire. My client has their own schedule, and this 3rd party application was becoming a problem. So after a flurry of emails, the 3rd party informed my client that everything was ready to go. And initially my client thought it was. Unfortunately it was not.


Problem 1: The Obvious
The first time a person accessed the web application, there is a server error, along with all the cryptic crap that gets spewed when a web application decides to fail, and has no graceful error processing/logging system to avoid exposing its gory details to the user.


This is a fairly common problem. One cause of this type of problem is when a link to a landing page is provided, but it requires specific information to have been previously set up in the user session. Normally users follow fairly predictable paths when accessing a web application. Early in that process, often certain session attributes are initialized for the user, for example an empty shopping cart, and the rest of the site just assumes that this already exists. But when a custom landing page is developed, and that is the first page that the user hits when coming to the site, those session attributes may not exist.


Interestingly, this was not the problem in this particular case. Sessions are often tracked with cookies, and after deleting the cookies from the browser, we discovered that the error screen was not generated. It still may be session related, but seems less likely.


The point of all this is that people who provide web applications should TEST what happens when a user comes into the site fresh. This should not be left as an interesting discovery for the users. 


Problem 2: The Obscure
The second problem occurred when my client tried to access the web application, using the link provided, from different document types. The link was placed in a word document, a PDF file, a web based email system, an Outlook based email system, and in another web page. In all cases, except the PDF file and Outlook, the link did not take the user to the correct page. The interesting thing was that doing a copy and paste of the link into the browser directly worked!


After some debugging (thanks to Firefox/Firebug), we determined that the only difference between the sequence of 4 pages that were redirected in the initial landing page was in the http header "referrer" field. Our tentative conclusion is that the web application is somehow sensitive to the referrer field, and is broken if that is set. One user of the referrer field is to control third party access to server resources, i.e. verify that the referrer is authorized to access the resource. This is just the opposite, here the referrer must be non-exist ant for the application to work.


Lessons Learned
The lesson for web application developers here? Test access to published links using a variety of different clients. And not just browsers, also client applications such as Microsoft Word, PDF files, email clients etc.


I hate to leave this discussion without definitive answers to the problems, but the truth is that since it is not our web application, there is only so much we can do. We isolated the fault domain (the 3rd party application), and provided them with sufficient documentation and test scenarios that they should be able to reproduce the problem. Whether they do so, fix the problem, and test the result remains to be seen.

Creating Master Calendar from Separate Google Apps Calendars Using Zend Framework/PHP

Introduction
The Google Apps/Calendar API enables programs to access calendar and event information. It seemed like a good way to have a program take the information from several calendars and merge them into a single master calendar. At first blush this seemed like an easy thing to do, but there are some nuances in the Google API that caused some problems.

The technology I used for this educational project was Zend Framework/PHP, in particular the ZF Google Data APIs. This is the basis for the PHP documentation on the Google API developers site, and seemed as good a place as any to get started.

I am writing this as a task that will be executed as a cron job automatically, not as a web application. The key difference between the two is how authentication is performed. For a cron job, authentication uses ClientLogin, instead of AuthSub.

The Setup
The goal of this example is to access all of the calendars available in a single user's Google Apps account. These can be calendars created by that user, calendars shared with that user by other users, either public or in the Google Apps domain, or calendars that are generated from information from other calendar sources, such as iCal.

I am not going to go into detail to describe how to set up the Zend Framework environment for use by normal command line processing applications, as that is dealt with in detail elsewhere. Having said that, the following is the basic structure required to set it up on my system:


echo "\n\nStarting calendar program...\n";


// Add the Zend Framework library to the include path 
// so that we can access the ZF classes
set_include_path('/path/to/local/zend/framwork/library/' .
    PATH_SEPARATOR .
    get_include_path()
);


// Start Zend Framework autoloader
require_once 'Zend/Loader/Autoloader.php';
Zend_Loader_Autoloader::getInstance();

Authentication

The next step is to authenticate the user in the Google Apps domain:

// Substitute your own user account here

$user = "user@domain.com";

// Substitute your own password here

$pass = "mypassword";

$serviceName = Zend_Gdata_Calendar::AUTH_SERVICE_NAME;

// Create an authenticated HTTP client

try {

    $client = Zend_Gdata_ClientLogin::getHttpClient($user, $pass, $serviceName);

} catch (Zend_Gdata_App_AuthException $e) {

    echo 'Error: ' . $e->getMessage();

    if ($e->getResponse() != null) {

        echo 'Body: ' . $e->getResponse()->getBody();

    }

}

In one example I saw, there was an additional catch section before the one here in case Google required a captcha response. Presumably this would occur if Google detected suspicious behavior during login. Unfortunately if that happens, I don't know of any way to automatically respond. After all the whole point of captcha is to require require human intervention. So I left it out, and it seemed to work fine. Maybe I am just lucky.

Load Calendar List
The next step is to load the calendar list:


// Create an instance of the Calendar service
$gdataCal = new Zend_Gdata_Calendar($client);


try {
    $calFeed = $gdataCal->getCalendarListFeed();
} catch (Zend_Gdata_App_Exception $e) {
    echo "Error: " . $e->getMessage();
}

Load Events for Each Calendar

The final step is to load each calendar's events:


// Iterate through calendars in calendar feed
foreach ($calFeed as $calendar) {
    // Print calendar title
    echo $calendar->title->text . "\n";
    $links = $calendar->getLink();
    foreach ($links as $link) {
        // This is the mystery part
        if ($link->getRel() == 'alternate') {
            try {
                $eventFeed = $gdataCal->getCalendarEventFeed($link->getHref());
            } catch (Zend_Gdata_App_Exception $e) {
                echo "Error: " . $e->getMessage();
            }
            foreach ($eventFeed as $event) {
                echo "\t".$event->title . "\n";
            }
        }
    }
    echo "\n";
}

The main stumbling block here came in trying to specify which calendar I wanted to load the events for. The default is to load the default user calendar. I was able to determine that the calendar entry includes an array of links, one of which is tagged as "alternate". This link happens to coincide with the link that you can view in Google Apps under calendar settings. I am not sure this is the recommended way, but it seems to work.

The other issue I ran into is that there are two different ways to generate an event feed. In one you specify the URL, as is done here. I the other you specify a query, based on a number of query attributes. It would have been nice to use the query approach, because ideally the program would be able to select events based on some criteria, such as start time. Unfortunately because of the way that calendar links are generated, I was not able to use the information from the calendar feed to generate a query that would work. Maybe some helpful reader can shed some light on this.

At this point the program has everything it needs to put together a master calendar. I leave the rest as an exercise for the user!

Upload Any File Type to Google Docs

Until last week, my recommendation for organizations using Google Apps for collaborative document sharing was to use Google Docs for online editing, augmented by DropBox for general file storage and offline editing. With the new capability introduced by Google last week to upload and store any file type in the Google Docs storage space, (learn more) that may change.

While the Google Docs user interface continues to improve, for those used to Microsoft Office and other desktop office apps, it still leaves something to be desired. Network latency and performance issues aggravate the problem. So while documents in the cloud have a certain appeal, the reality is that for day to day work, documents have remained in local storage. For that reason, services such as DropBox have been used to provide local document editing while sharing those documents between different users and systems.

As of last week, Google now allows the Docs workspace to be used for general purpose storage of files regardless of their types. And for files of the standard Google Docs types (eg. documents, presentations and spreadsheets), the user can choose whether to upload in the desktop native format, or convert to the Google Docs online format.

The remaining difference between this capability and a service such as DropBox is that Google Docs are not automatically synchronized with  a users desktop. If a person wants to edit a document, they first must download it, edit it, and then upload it back to the Google Docs system. Here is where other 3rd party services come in. One company in particular, Memeo Inc. has introduced Memeo Connect for Google Apps, which provides a desktop folder that is automatically synchronized with Google Docs. Priced at $9/user per year, this is an affordable solution for organizations that want to have the ability to store online and offline editable documents in the Google Docs service.

Grails on Google App Engine Rationale

After test driving Liferay 3 weeks ago, and then trying out Alfresco, I finally got serious about figuring out how best to approach building rich(er) browser based clients, which seems to boil down to which Javascript framework to use. Since the introduction of Google AppEngine support for Java, it occurred to me that one approach worth re-evaluating would be Google Web Toolkit (GWT), which allows you to write client/server apps in Java, and have the client side compiled to Javascript to run on the browser.

When I first heard about this, must be over a year ago, I thought it was a dumb idea. Having struggled since then to attempt to build browser based apps, it is starting to look pretty nice. So nice in fact that I am willing to go back into the Java side of the industry for my consulting practice, after moving away from it several years ago. My complaints at that time were:
1. There are just too many libraries and frameworks that are needed to make a system work
2. Picking the right versions was a dependency nightmare
3. All those libraries required extensive XML configuration
4. Hosting was complex and expensive, making its suitable for high end applications only.

Google's support for Java on App Engine eliminates the last objection for those applications suitable for the Google environment, which includes a number of my clients.

Having made the decision to move back into Java because of GWT, it made sense to try out Google AppEngine/Java as well. So I dutifully went through the excellent tutorials, and found it quite easy to deal with. Then I tried to actually use it to build my own web application. Having worked with several major MVC web application frameworks (Rails, Zend Framework and Catalyst), it was a major disappointment that out of the box, AppEngine doesn't offer an MVC framework for Java. So I started to look around, and explore the Java framework universe. Spring seems to be the most popular (based on shelf space at Barnes & Noble) , but the complexity of attempting to use the Spring stack (including Spring, Spring MCV, Spring Flow..., each with its own book) on AppEngine was starting to make me rethink my decision to move back into Java. That is until I discovered Grails.

Over a year ago, a collegue of mine suggested I take a look at the language Groovy. At the time it didn't seem that robust, and certainly didn't address the web application framework issue. And while Grails did attempt to solve this issue, I thought that it didn't make sense to learn both a new language and a new framework just to write web applications that would only run on high end servers.

So this week I decided it was time to give Groovy/Grails another look since several postings indicated that this now works on AppEngine. I am not going to go into the gory details here, but I can speak from personal experience that it is now possible to build a Grails app, (using JDO instead of Hibernate and GORM), and have it run on Google AppEngine. The benefit is that it takes care of the Java configuration issues mentioned earlier by using Convention over Configuration, and it provides a very capable MVC style web application framework.

The online instructions (http://www.grails.org/plugin/app-engine) and video (http://grails.org/dist/screencasts/grails-appengine-screencast.mov) are excellent , although there are several issues and bugs that I ran into. None of these seem to be show stoppers, but a couple can be major annoyances. Unfortunately that discussion will have to wait that for another day, as well as exploring how to use GWT from Grails. The Google I/O conference (http://code.google.com/events/io) is next week, there should be more information and discussions around these topics there.

Liferay Portal Demo with MySQL

In my previous post I discussed the reasons that Liferay Portal is interesting from an enterprise solutions perspective, followed by an initial perspective on the out of the box demo. Basically I was able to get the canned demo working, but then attempting to change from the embedded hsql database to mysql was not initially successful. After initially attempting it on Windows, I switched to Linux because MySQL was already installed on that server.

In searching for a solution, I ran across the following the post Windows Quick Installation Guide For Liferay 5.2.1 With Tomcat 6.0 Bundle and MySQL Database which described the procedure for running against MySQL in a Windows environment. The following procedure is an adaptation of that which worked for me in Linux (Ubuntu). Note: I am assuming that the downloads are installed in my home directory. You can install them anywhere, but need to adjust the instructions accordingly.

1. Download Java
While this is not necessary, I decided to bypass the normal Java installation using Ubuntu packages, and instead downloaded and installed the Java SE bundle from SUN. From the menu bar at the top of the page, click Download >> Java SE. While a Java runtime should be sufficient, I downloaded the Java SE Development Kit (JDK) part way down the page.

2. Install Java SDK
Once downloaded, it is necessary to make the downloaded file executable and then execute it to unpack the files. Note: The following assumes you downloaded the JDK into your home directory. From the command prompt:
chmod +x ~/jdk-6u13-linux-i586.bin
~/jdk-6u13-linux-i586.bin

3. Download the Liferay Portal bundle.
Go to the Liferay Portal site http://www.liferay.com/. Part way down the page, under the heading Liferay Portal Standard Edition click the download link. Note: There are a lot of other bundles to choose from further down on the page, but this will suffice for now. Unzip the downloaded bundle in a directory of your choice. You need to make the scripts in the bundle executable as follows:
chmod +x ~/liferay-portal-5.2.2/tomcat-5.5.27/bin/*.sh
Note: Your versions may be different, so substitute the version numbers in the bundle that you actually downloaded.

4. Monitor Tomcat server
You can optionally open a new window to monitor the Tomcat server:
tail -f ~/liferay-portal-5.2.2/tomcat-5.5.27/catalina.out

5. Run the Basic Demo
At this point you should be able to run the demo bundle using the default HSQL database and demo configuration as follows. Open a command prompt, and change to the root directory of the liferay bundle. Then set the JAVA_HOME environment variable, and run the system. Note: substitute the actual path of your installation. After the system starts up in approximately 1-2 minutes, it should automatically open a window in your browser and display the demo site.
cd ~/liferay-portal-5.2.2
JAVA_HOME=~/jdk1.6.0_13
export JAVA_HOME
tomcat-5.5.27/bin/startup.sh

6. Stop the Basic Demo
When you are done, shutdown Liferay:
tomcat-5.5.27/bin/shutdown.sh

The following steps move to the basic test system, running under mysql. It assumes that mysql is already installed on your system.

7 Remove the Basic App from the Bundle
I am not sure this is actually required for this step, but it worked when I did it, so why not.
rm -r ~/liferay-portal-5.2.2/tomcat-5.5.27/webapps/sevencogs-theme
rm -r ~/liferay-portal-5.2.2/tomcat-5.5.27/webapps/sevencogs-hook

8. Create MySQL database
From your mysql command prompt:
create database lportal character set utf8;

9. Create Configuration Extension File
In order to tell Liferay to use a MySQL database instead of the preconfigured HSQL database, it is necessary to create the an extension file with the MySQL properties. Note: substitute your own MySQL userid/password in the following
Open the following file using your favorite editor (create if necessary):
~/liferay-portal-5.2.2/tomcat-5.5.27/webapps/ROOT/WEB-INF/classes/portal-ext.properties

Enter the following lines:
jdbc.default.driverClassName=com.mysql.jdbc.Driver
jdbc.default.url=jdbc:mysql://localhost/lportal?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false
jdbc.default.username=YOURUSERNAME
jdbc.default.password=YOURPASSWORD


10. Run the MySQL Test Site
At this point you should be able to run the test site using the MySQL database. Using the same procedure as step #5, open a command prompt, and change to the root directory of the liferay bundle. Then set the JAVA_HOME environment variable, and run the system.
cd ~/liferay-portal-5.2.2
JAVA_HOME=~/jdk1.6.0_13
export JAVA_HOME
tomcat-5.5.27/bin/startup.sh

You should be able to see a line in the output window where the portal-ext.properties file is loaded, as well as where the MySQL database tables are initially loaded.

Now that I got to this point, it isn't so bad. And in all fairness, it wasn't the XML configuration swamp that kept me from success in the previous post. :-)