What’s the difference between reality and theory? In theory, there is no difference. But reality often imposes unanticipated constraints on developers. These may come in the form of bandwidth restrictions, memory limits, timeouts, or other requirements of the systems that interact with your application.
My team recently built an application that helps us analyze the scheduling and usage of conference rooms at Google. We use the new Calendar API v3 on Google App Engine to read the rooms’ schedules, which we combine with actual occupancy data to calculate utilization and other metrics.
As you might imagine, Google has a lot of conference rooms (I believe the last official count was “more than twelve.”) And many of the rooms seem to be booked fairly solid. That means we need to read a lot of data from Calendar. So much, in fact, that our queries time out if we try to read an entire calendar at once. But the API team anticipated “Google scale” use and designed a mechanism that allows us to retrieve data in batches.
The idea is simple. When you create a request, you specify the page size: the maximum number of results you’d like Calendar to return in one batch. Calendar returns the data you requested, along with an opaque page token, which you can think of as a bookmark. To retrieve the next batch of data, you ask the API for the next page token and include the new token in your next request. The page token keeps track of the results you’ve already seen, so Calendar can send the next batch each time. You repeat this process until you’ve exhausted all the results.
Here’s how we did this in Java:
public void getRoomEvents(String roomEmail) throws IOException { // Create a request to list this room’s events (see code, below) Calendar.Events.List listRequest = getListRequest(roomEmail); do { // Retrieve one page of events Events events = executeListRequest(listRequest); List eventList = events.getItems(); // Process each event for (Event event : eventList) { processEvent(event); } // Update the page token listRequest.setPageToken(events.getNextPageToken()); // Stop when all results have been retrieved } while (listRequest.getPageToken() != null); } // Create a request to list the events for a room private Calendar.Events.List getListRequest(String roomEmail) throws IOException { return calendarClient.events().list(roomEmail) .setMaxResults(1000) // Limit each response to 1000 events .setPageToken(null) // Start with the first page of results // Return an individual event for each instance occurrence of a // recurring event .setSingleEvents(true); }
We call getRoomEvents() for each room, using the room’s email address to identify it to Calendar. (You can retrieve events from your own calendar by substituting your own email address.) Then getListRequest() creates a request that we will send to Calendar. The request asks for a list of up to 1000 events from the room’s calendar.
getRoomEvents()
getListRequest()
The remainder of getRoomEvents() is a loop that executes the request, processes the results, and updates the page token in preparation for the next request. The loop continues, retrieving and processing each subsequent page of results, until the entire list has been returned. The call to getNextPageToken() indicates the end of the results by returning a null value.
getNextPageToken()
By paginating our requests we avoid timeouts and reduce memory requirements. As an added benefit, each request completes fairly quickly, which means it’s also quick to retry if an error should occur. And finally, a multithreaded application may be able to process one or more pages of results while it retrieves the next, speeding execution. These advantages have led developers at Google to adopt pagination as a best practice. Look for it in our APIs when you need to exchange large amounts of data, and consider adding it to your own services.
If you have questions about our services or APIs, or if you want to see what other developers are doing with Google Calendar, check the discussions and documentation in the Google Apps Calendar API forum.
Our newest set of APIs - Tasks, Calendar v3, Google+ to name a few - are supported by the Google APIs Discovery Service. The Google APIs Discovery service offers an interface that allows developers to programmatically get API metadata such as:
The APIs Discovery Service is especially useful when building developer tools, as you can use it to automatically generate certain features. For instance we are using the APIs Discovery Service in our client libraries and in our APIs Explorer but also to generate some of our online API reference.
Because the APIs Discovery Service is itself an API, you can use features such as partial response which is a way to get only the information you need. Let’s look at some of the useful information that is available using the APIs Discovery Service and the partial response feature.
You can get the list of all the APIs that are supported by the discovery service by sending a GET request to the following endpoint:
GET
https://www.googleapis.com/discovery/v1/apis?fields=items(title,discoveryLink)
Which will return a JSON feed that looks like this:
{ "items": [ … { "title": "Google+ API", "discoveryLink": "./apis/plus/v1/rest" }, { "title": "Tasks API", "discoveryLink": "./apis/tasks/v1/rest" }, { "title": "Calendar API", "discoveryLink": "./apis/calendar/v3/rest" }, … ] }
Using the discoveryLink attribute in the resources part of the feed above you can access the discovery document of each API. This is where a lot of useful information about the API can be accessed.
discoveryLink
Using the API-specific endpoint you can easily get the OAuth 2.0 scopes available for that API. For example, here is how to get the scopes of the Google Tasks API:
https://www.googleapis.com/discovery/v1/apis/tasks/v1/rest?fields=auth(oauth2(scopes))
This method returns the JSON output shown below, which indicates that https://www.googleapis.com/auth/tasks and https://www.googleapis.com/auth/tasks.readonly are the two scopes associated with the Tasks API.
https://www.googleapis.com/auth/tasks
https://www.googleapis.com/auth/tasks.readonly
{ "auth": { "oauth2": { "scopes": { "https://www.googleapis.com/auth/tasks": { "description": "Manage your tasks" }, "https://www.googleapis.com/auth/tasks.readonly": { "description": "View your tasks" } } } } }
Using requests of this type you could detect which APIs do not support OAuth 2.0. For example, the Translate API does not support OAuth 2.0, as it does not provide access to OAuth protected resources such as user data. Because of this, a GET request to the following endpoint:
https://www.googleapis.com/discovery/v1/apis/translate/v2/rest?fields=auth(oauth2(scopes))
Returns:
{}
Using the API-specific endpoints again, you can get the lists of operations and API endpoints, along with the scopes required to perform those operations. Here is an example querying that information for the Google Tasks API:
https://www.googleapis.com/discovery/v1/apis/tasks/v1/rest?fields=resources/*/methods(*(path,scopes,httpMethod))
Which returns:
{ "resources": { "tasklists": { "methods": { "get": { "path": "users/@me/lists/{tasklist}", "httpMethod": "GET", "scopes": [ "https://www.googleapis.com/auth/tasks", "https://www.googleapis.com/auth/tasks.readonly" ] }, "insert": { "path": "users/@me/lists", "httpMethod": "POST", "scopes": [ "https://www.googleapis.com/auth/tasks" ] }, … } }, "tasks": { … } } }
This tells you that to perform a POST request to the users/@me/lists endpoint (to insert a new task) you need to have been authorized with the scope https://www.googleapis.com/auth/tasks and that to be able to do a GET request to the users/@me/lists/{tasklist} endpoint you need to have been authorized with either of the two Google Tasks scopes.
POST
users/@me/lists
users/@me/lists/{tasklist}
You could use this to do some automatic discovery of the scopes you need to authorize to perform all the operations that your applications does.
You could also use this information to detect which operations and which endpoints you can access given a specific authorization token ( OAuth 2.0, OAuth 1.0 or Authsub token). First, use either the Authsub Token Info service or the OAuth 2.0 Token Info Service to determine which scopes your token has access to (see below); and then deduct from the feed above which endpoints and operations requires access to these scopes.
[Access Token] -----(Token Info)----> [Scopes] -----(APIs Discovery)----> [Operations/API Endpoints]
Example of using the OAuth 2.0 Token Info service:
Request:
GET /oauth2/v1/tokeninfo?access_token= HTTP/1.1 Host: www.googleapis.com
Response:
HTTP/1.1 200 OK Content-Type: application/json; charset=UTF-8 … { "issued_to": "1234567890.apps.googleusercontent.com", "audience": "1234567890.apps.googleusercontent.com", "scope": "https://www.google.com/m8/feeds/ https://www.google.com/calendar/feeds/", "expires_in": 1038 }
There is a lot more you can do with the APIs Discovery Service so I invite you to have a deeper look at the documentation to find out more.
Two months ago we announced that a few of us from the Google Apps Developer Relations team would be going around EMEA to meet with developers and talk about Google Apps technologies. We have met great developers from Germany, France, Russia, Czech Republic, Egypt, Switzerland, Israel, and Spain during Google Developer Days, hackathons, developer conferences and GTUG meetings.
This year we are continuing the tour with a series of Google Apps Script hackathons taking place in Vienna, Milan, Madrid, Munich and Dublin over the next few months. These hackathons provide a fun and hands-on way to learn about Google Apps Script and a good opportunity to give us your feedback on this technology.
For more information about the tour and to register for these events, please visit the Google Apps EMEA Developer Tour website.
We plan to organize many other Google Apps events close to you in the near future. Look for updates on the Google Apps EMEA Developer Tour website or keep an eye out for further announcements on this blog.
We recently posted some best practices for working with recurring events in Google Calendar API v3. In this blog post we’ll highlight another improved area in the v3 API: event reminders.
Google Calendar API v3 offers developers flexible control over event reminders, including per-calendar default settings and custom overrides for individual events.
The user’s default reminders for events on a given calendar can be found in the corresponding entry in the Calendar List collection. The Calendar List collection acts a bit like a list of bookmarks, containing entries for the calendars that the user owns or has looked at in the past (it corresponds to the content of the "My Calendars" and "Other Calendars" list on the bottom left in the Web version of Google Calendar). Each entry is annotated with user-specific settings for the individual calendar, such as the preferred color in the UI and the default reminders.
Google Calendar currently supports three ways of reminding its users of events: "popup", prompting a message directly in the browser, mobile phone or desktop client, as well as "email" and "sms" for messages sent through the respective channels. To change the defaults, update the Calendar List entry and include the reminder method and how many minutes in advance the user should be alerted. In the following example, we set an email reminder to be sent 60 minutes before an event, and a popup reminder 10 minutes before.
{ "summary": "Work Calendar", ... "defaultReminders": [ { "method": "email", "minutes": 60 }, { "method": "popup", "minutes": 10 } ] }
The default reminders will be applied to all existing and future events on this calendar, provided they don’t have custom reminders set already. In contrast to earlier versions of the API, newly created events will also have reminders set by default.
Sometimes, there are events that we want a special reminder for, or none at all. To override the defaults for a specific event, switch the useDefault flag in the reminders section to false, and include a set of custom reminders, or leave the list empty. When you define a set of override reminders for a recurring series, they are automatically applied to each of its occurrences, unless they have been overridden explicitly. Like the default reminders on the calendar, these are personal reminders for the user that is logged in, and will not influence the settings others might have for the same calendar or event. Here is an example that overrides the default reminders with a 15 minute SMS reminder for that specific event.
useDefault
reminders
false
{ "summary": "API Office Hours", ... "reminders": { "useDefault": false, "overrides": [ { "method": "sms", "minutes": 15 } ] } }
The defaults for the given calendar are included at the top of any event listing result. This way, reminder settings for all events in the result can be determined by the client without having to make the additional API call to the corresponding entry in the Calendar List collection.
In this post and an earlier post about best practices with recurring events, we have covered some improved areas of the latest version of the Google Calendar API. Have a look at the migration guide for a more complete view of other changes we made in the new version, and let us know what you think.
If you have any questions about handling reminders or other features of the new Calendar API, post them on the Calendar API forum.
Editor’s Note:This is a guest post from John Watkinson. John is a developer and the co-founder of Docracy, a start-up company that hosts crowd-sourced legal documents and provides free e-signing services. He recently attended, and won first place, at the Google Apps Hackathon in NYC. Here's John's story about the event.
Our company is focused on documents in the cloud, so we attended the Google Apps Developer Hackathon on December 1st in New York City to learn more about the various Google Apps APIs, and how we can best integrate with them. During the initial presentations from Ryan Boyd and Saurabh Gupta of the Google Apps team, we were delighted to learn about Apps Script. It provides a powerful JavaScript interface into Google Apps, including Docs, Calendar, Gmail, Maps and many others.
After the tech presentations, we formed teams and started hacking with the technology. I joined a team with Matt Hall (also from Docracy), Nick Siderakis, Jeff Hsin and Scott Thompson. We struck upon the idea of creating a function that would make calls to Wolfram Alpha via their developer API. It took all five of us hacking away for a few hours to parse the Alpha results properly, but eventually we had an Apps Script custom function for Spreadsheet called “wolf”. It had a kind of magic feeling about it, as you could pass simple English-language queries into the function and get back exact numeric results in your Google Spreadsheet. Some example working queries:
=wolf("distance from the earth to the moon in km")
=wolf("population of canada in 1960")
=wolf("average january temperature in buenos aires")
=wolf("9th digit of pi")
=wolf("count of olympic medals won by japan in 2008")
Any function in Apps Script can be used as a spreadsheet function directly, so our main wolf function is the entry point to our script. We use Apps Script’s UrlFetch service to make our API call to Wolfram Alpha (note, the full URL is truncated for brevity):
wolf
UrlFetch
var response = UrlFetchApp.fetch("http://api.wolframalpha.com/..." + encodeURIComponent(input));
The response from Alpha is an XML file that includes a lot of metadata we don’t need. So, we used a recursive function and the XML manipulation tools in Apps Script to track down the element of interest to us (below is a simplified version):
function findText(e) { if (e.getName().getLocalName() == 'plaintext') { return e.getText(); } else { var children = e.getElements(); for (var i = 0; i < children.length; i++) { var result = findText(children[i]); if (result) return result; } return null; } }
A little bit more work is required to parse the result, as sometimes Alpha will report numbers in words (i.e. “40.7 million”), but otherwise that’s it!
For a small script, it definitely feels pretty powerful! Our little invention earned us a first place finish at the hackathon, and the prizes were little indoor RC-controlled helicopters. I managed to irreparably damage mine in a gruesome crash into my refrigerator already, but I hear that many of the others are still serviceable and flying missions daily.
Once we were comfortable with the Apps Script Services (and aware of its capabilities with spreadsheets in particular), a side project emerged that used some of the other integration features available. Apps Script functions can listen for events that are triggered by the editing of a spreadsheet cell (if the user grants the appropriate permissions). Using these onEdit Event Handlers, we wrote an implementation of the classic game Mine Sweeper that is playable right in the spreadsheet!
You can play the game by making a copy of this spreadsheet and then clicking the menu item SheetSweeper > New Game. When you play the game, the script responds to onEdit events that are dispatched from the spreadsheet.
function onOpen() { var ss = SpreadsheetApp.getActiveSpreadsheet(); var menuEntries = [{name: "New Game", functionName: "startGame"}]; ss.addMenu("SheetSweeper", menuEntries); }
Then when the user edits a cell, the script responds and implements the mine sweeper logic:
function onEdit(event) { var ss = event.source.getActiveSheet(); var cell = ss.getActiveCell(); if (cell.getValue() == 'x') { // Flagging a mine flagCell(cell); } else { // Clearing a cell clearCell(cell); } }
We had a great time at the hackathon, and are happy to have been introduced to Apps Script. We believe it has an exciting future ahead of it.
Are you ever at an airport with no internet access, and realize that you forgot to set your "Out of Office" (OOO) message?" I forget regularly, but always remember to block off my calendar. Why not have the calendar automatically update my OOO message? We can! With the Calendar API, we can retrieve calendar events and set a vacation responder for the event duration for any user in the domain using the Email Settings API.
The first step is to authorize the Calendar and Email Settings client to make any call to the APIs respectively. The new Calendar API requires google-api-client library while the Email settings API still works with the gdata-client library in Python. These libraries use different underlying protocols and handle authorization differently.
Fortunately we can use the same code to get an OAuth 2.0 token for both the Calendar API and the Email Settings API.
from oauth2client.file import Storage from oauth2client.client import AccessTokenRefreshError from oauth2client.client import OAuth2WebServerFlow from oauth2client.tools import run SCOPES = ('https://www.googleapis.com/auth/calendar ' 'https://apps-apis.google.com/a/feeds/emailsettings/2.0/') FLOW = flow_from_clientsecrets('client_secrets.json', scope=SCOPES) storage = Storage('vacation.dat') credentials = storage.get() if credentials is None or credentials.invalid: credentials = run(FLOW, storage)
Now that we have obtained the OAuth 2.0 token from OAuth2WebServerFlow, we can use it in either library. To authorize the Calendar service:
# Create an httplib2.Http object to handle the HTTP # requests and authorize it with our good Credentials. http = httplib2.Http() http = credentials.authorize(http) # Build authorized service for Calendar API service = build('calendar', 'v3', http=http)
Authorizing the Email Settings client requires adapting the credentials:
auth2token = gdata.gauth.OAuth2Token(client_id=client_id, client_secret=client_secret, scope=SCOPE, access_token=credentials.access_token, refresh_token=credentials.refresh_token, user_agent='vacation-responder-sample/1.0') email_client = auth2token.authorize( gdata.apps.emailsettings.client.EmailSettingsClient( domain=domain))
Now its time to update the vacation responder based on calendar event using the authorized client. Lets query for events containing the string ‘vacation’ from the Calendar API. The following code snippet shows the retrieval of events.
# Convert the date to the RFC 3339 timestamp format # for Calendar API. cur_datetime = datetime.datetime.now() cur_date = cur_datetime.strftime('%Y-%m-%dT%H:%M:%SZ') events = service.events().list(calendarId=username, q='vacation', timeMin=cur_date, singleEvents=True, orderBy='startTime').execute()
By specifying a user’s email address as the calendarID, we can query against any primary calendars in our domain that are shared and readable by the administrator. If the user opts to hide their calendar, we won’t find any of their events. For additional details about the query on calendar events, refer to the calendar query parameters from the reference guide.
Next we can obtain the start and end time of the event and update the vacation responder using the Email Settings client if the event’s duration is day long or more.
if 'items' in events: for event in events['items']: startDate = event['start'] endDate = event['end'] # If event is set for the day not just for a time slot. if 'date' in startDate: email_client.UpdateVacation(username=username, enable=True, subject='Out of office', message='If urgent call me.', domain_only=True, start_date=startDate['date'], end_date=endDate['date']) print '\nVacation responder set for the days between %s to %s' % (startDate['date'], endDate['date']) break
You can download the sample and build your own application on top of it. We hope that this sample makes it easier to get started, particularly for apps that need to combine APIs from the two API clients. Please feel free to reach us in the Google Domain Info and Management Forum with any questions you have or write us feedback on this post.
