Posted by Mike Procopio, Engineering Manager, Google Drive and Wesley Chun, Developer Advocate, Google Apps
WhatsApp is one of the most popular mobile apps in the world. Over a billion users send and receive over 42 billion messages, photos, and videos every day. It's fast, easy to use, and reliable.
But what happens when people lose their phone or otherwise upgrade to a new one? All those messages and memories would be gone. So we worked with WhatsApp to give their users the ability to back up their data to Google Drive and restore it when they setup WhatsApp on a new phone. With messages and media safely stored in your Drive, there’s no more worry about losing any of those memories.
One of the biggest challenges for an integration of this scope is scaling. How do you back up data for a billion users? Many things were done to ensure the feature works as intended and is unnoticeable by users. Our approach? First, we relied on a proven infrastructure that can handle this kind of volume—Google Drive. Next, we optimized what to back up and when to do the backups—the key was to upload only incremental changes rather than transmit identical files.
On the server side (backend), we focused on optimizing byte storage as well as the number of network calls between WhatsApp and Google. As far as deployment goes, we rolled out slowly over several months to minimize the size and impact of deployment.
If you have ever used WhatsApp, you know how it gets out of your way, and lets you get started quickly: no account creation, no passwords to manage, and no user IDs to remember or exchange. This sets a high bar for any integration with WhatsApp: for it to feel like a natural part of WhatsApp, it has to be as seamless, fast, and reliable as WhatsApp itself.
By using the Google Drive API, we were able to achieve this: no need to type in any usernames or passwords, just a few taps in the app, and WhatsApp starts backing up. The best part is that all the tools used in the integration are available to all developers. With the Google Drive API, seamless and scalable integrations are as easy to use for the user as they are for developers.
To learn more about how we did it and get all the details, check out the complete talk we gave together with WhatsApp at Google I/O 2016.
Are you ready to integrate your web and mobile apps with Google Drive? Get started today by checking out our intro video as well as the video demoing the newest API, then dig in with the developer docs found at developers.google.com/drive. We're excited to see what you build next with the Drive API—and we're ready to scale with you!
Posted by Dan McGrath, Product Manager, Google Drive
Today, version 3 (v3) of the Google Drive API is available, providing developers with a leaner, faster by default, and more consistent interface to Google Drive. This latest update is designed to be easier for developers who are integrating with Drive for the first time.
In the time since we launched the original version of the Drive SDK, we’ve learned a lot about how developers are using our API and the pitfalls they’ve encountered. This version simplifies the API in several ways, including:
For example, our files.list call now only includes the most commonly used fields that we serve efficiently by default, with additional fields able to be added via the fields parameter. This makes the default call nearly 6X faster in v3 than in v2:
While future versions of the API will be more feature-focused, this release provides improved performance and ease of use over the previous version of the API (v2). We will continue to support both versions so that existing apps using v2 can run without changes (developers don’t have to upgrade). For developers with a v2 app who want to take advantage of the new improvements, we’ve got a handy cheat sheet to help with migration.
To get started using the new version of the API, check out the developer docs; and for any questions, find us with the google-drive-sdk tag on StackOverflow. We look forward to seeing what you’ll build with the new API!
string CreateFile(Stream content, string fileName, string mimeType, string folderId, string accessToken){ var bytes = Encoding.UTF8.GetBytes( "\r\n--boundary\r\nContent-Type: application/json; charset=UTF-8\r\n\r\n{\"title\":\"" + fileName + "\",\"parents\":[{\"id\":\"" + folderId + "\"}]}" + "\r\n--boundary\r\nContent-Type: " + mimeType + "\r\n\r\n"); var tmpStream = new MemoryStream(); tmpStream.Write(bytes, 0, bytes.Length); content.CopyTo(tmpStream); bytes = Encoding.UTF8.GetBytes("\r\n--boundary--\r\n"); tmpStream.Write(bytes, 0, bytes.Length); var request = WebRequest.Create("https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart"); request.Method = "POST"; request.Headers.Add("Authorization", "Bearer " + accessToken); request.ContentType = "multipart/related; boundary=boundary"; request.ContentLength = tmpStream.Length; var buffer = new byte[2048]; int readed; tmpStream.Seek(0, SeekOrigin.Begin); while ((readed = tmpStream.Read(buffer, 0, 2048)) > 0) { request.GetRequestStream().Write(buffer, 0, readed); } return request.GetResponse().GetResponseStream(); }
final RateLimiter driveApiRateLimiter = RateLimiter.create(QUOTA_RATE_PER_SECOND);
public Result performDriveApiCall(driveApiRateLimiter, otherParams){ driveApiRateLimiter.acquire(); // blocks according to rate // make API call...
1, 2, 4, 8, 16, 32, 60, 60, 60
ExponentialBackOff backoff = new ExponentialBackOff.Builder() .setInitialIntervalMillis(ONE_SECOND) .setMultiplier(2.0) .setMaxIntervalMillis(ONE_MINUTE) .setMaxElapsedTimeMillis(FIVE_MINUTES) .build();
1.04, 1.9, 4.23, 7.8, etc.
.98, 2.04, 4.1, 8.15, etc.
builder.setRandomizationFactor(RANDOMIZATION_FACTOR);
private HttpBackOffUnsuccessfulResponseHandler handler = new HttpBackOffUnsuccessfulResponseHandler(backoff); public void initialize(HttpRequest request){ request.setUnsuccessfulResponseHandler(handler); }
public void changeOwner(String user, String fileId, String newOwner) { // Find what is the current permission of the new owner on the file Permission newOwnerPermission = null; PermissionList permissionList = RetriableTask.execute(new DrivePermissionListTask(drive.permissions().list(fileId))); newOwnerPermission = findPermission(permissionList, newOwner); if (newOwnerPermission == null) { // New owner is not in the list, we need to insert it newOwnerPermission = new Permission(); newOwnerPermission.setValue(newOwner); newOwnerPermission.setType("user"); newOwnerPermission.setRole("owner"); Drive.Permissions.Insert insert = drive.permissions().insert(fileId, newOwnerPermission); RetriableTask.execute(new DrivePermissionInsertTask(insert)); } else { // New owner is already in the list, update the existing permission newOwnerPermission.setRole("owner"); Drive.Permissions.Update update = drive.permissions().update(fileId, newOwnerPermission.getId(), newOwnerPermission); update.setTransferOwnership(true); RetriableTask.execute(new DrivePermissionUpdateTask(update)); } }
public class RetriableTask implements Callable { [...] private final Callable task; [...] @Override public T call() { T result = null; try { startTime = System.currentTimeMillis(); result = task.call(); } catch (NonFatalErrorException e) { if (numberOfTriesLeft > 0) { // Wait some time, using exponential back-off in case of multiple attempts Thread.sleep(getWaitTime()); // Try again result = call(); } else { // Too many failed attempts: now this is a fatal error throw new RetryException(); } } catch (FatalErrorException e) { // This one should not be retried Throwables.propagate(e); } return result; }
// Launch user interface and allow user to select file IntentSender i = Drive.DriveApi .newOpenFileActivityBuilder() .setMimeType(new String[] { “text/plain” }) .build(mGoogleApiClient); startIntentSenderForResult(i, REQ_CODE_OPEN, null, 0, 0, 0);
Drive is a great drop zone for incoming files -- no matter if they’re coming from cameras, scanners, faxes, or countless other devices or apps. But throwing files into the root folder makes it difficult for users to find and organize their content.
I’ve seen developers create a folder when the user first connects the app with Drive to keep files organized by the app that created it. It’s a simple technique that is easy to implement and good default behavior for most applications.
With the Picker API, we can go a step further and offer users the choice of destinations too. For example, I’d like my scanner to put work files in one folder and personal files in another, or even configure multiple destinations depending on the type of document I’m scanning.
For this particular use case, the picker needs to be configured to show folders and only show folders. That requires customizing the DocsView just a little.
var docsView = new google.picker.DocsView() .setIncludeFolders(true) .setMimeTypes('application/vnd.google-apps.folder') .setSelectFolderEnabled(true);
By enabling folders & the ability to select them while simultaneously filtering out everything else, users can quickly and easily select one of their existing folders as a destination. The rest of the code to show the picker is par for the course.
// Handle user actions with the picker. var callback = function(data) { if (data.action == google.picker.Action.PICKED) { var doc = data.docs[0]; alert("You picked " + doc.id); } }; var picker = new google.picker.PickerBuilder() .addView(docsView) .setCallback(callback) .build(); picker.setVisible(true);
Not only is this easy to implement, it’s safer for users. By offloading this functionality to the Picker API, an app only needs the drive.file scope to write files into the user’s preferred location.
You can learn more about the Picker API at developers.google.com or ask questions at StackOverflow with the google-drive-sdk tag.
If your app needs to keep up with changes in Drive, whether to sync files, initiate workflows, or just keep users up to date with the latest info, you’re likely familiar with Drive’s changes feed. But periodic polling for changes has always required a delicate balance between resources and timeliness.
Now there’s a better way. With push notifications for the Drive API, periodic polling is no longer necessary. Your app can subscribe for changes to a user’s drive and get notified whenever changes occur.
Suppose your app is hosted on a server with my-host.com domain and push notifications should be delivered to an HTTPS web-hook https://my-host.com/notification:
my-host.com
https://my-host.com/notification
String subscriptionId = UUID.randomUUID().toString(); Channel request = new Channel() .setId(subscriptionId) .setType("web_hook") .setAddress("https://my-host.com/notification"); drive.changes().watch(request).execute();
As long as the subscription is active, Google Drive will trigger a web-hook callback at https://my-host.com/notification. The app can then query the change feed to catch up from the last synchronization point:
changes = service.changes().list() .setStartChangeId(lastChangeId).execute();
If your app only needs to be notified about changes to a particular file or folder your app can watch just those files rather than the entire change feed.
If you are interested in using this new feature, please refer to the documentation at developers.google.com. You can see push notifications in action with the Push Notifications Playground and view the source at Github.
Ever look at the data returned when using the Drive API? A files.list call, even if just returning a single file, can yield upwards of 4kb of data. Drive has a rich set of metadata about files, but chances are your application only needs a small fraction of what’s available.
files.list
One of the simplest but most effective optimizations you can make when building apps with the Drive API is limiting the amount of data returned to only those fields needed for your particular use case. The fields query parameter gives you that control, and the results can be dramatic.
A simple example of this is using the files.list call to display a list of files to a user. The naive query, https://www.googleapis.com/drive/v2/files?maxResults=100, generated more than 380kb of data when I ran it against my own corpus. But to render this list nicely, an app only needs a few bits of information -- the document title, icon & thumbnail URLs, the mime type, and of course the file ID.
https://www.googleapis.com/drive/v2/files?maxResults=100
Using the fields query parameter, the results can be trimmed to just the necessary fields and those needed for fetching subsequent pages of data. The optimized query is https://www.googleapis.com/drive/v2/files?maxResults=100&fields=items(iconLink%2Cid%2Ckind%2CmimeType%2CthumbnailLink%2Ctitle)%2CnextPageToken.
https://www.googleapis.com/drive/v2/files?maxResults=100&fields=items(iconLink%2Cid%2Ckind%2CmimeType%2CthumbnailLink%2Ctitle)%2CnextPageToken
After modifying the query the resulting data was only 30k. That’s more than a 90% reduction in data size! Besides reducing the amount of data on the wire, these hints also enable us to further optimize how queries are processed. Not only is there less data to send, but also less time spent getting it in the first place.
What does the new Google+ Sign-In mean for your Drive app, and why should you use it?
All APIs can be authorized using the “Sign in with Google” button, including the Drive API. To authorize additional scopes, just pass them in the markup for the “Sign in with Google” button like we’ve done in this example.
<span class="g-signin" data-scope="https://www.googleapis.com/auth/drive.file">
The “Sign in with Google” button can cater to whatever kind of application you create: web, client, or mobile. Now you can choose the authorization flow you like and get a token using the OAuth 2.0 client-side flow or server flow. There are loads of features, and the button is highly customizable.
I’ve saved my favorite feature until the end: when the user authorizes an application on the web, the mobile version of the app can be installed over the air onto their mobile device. Just add your Android package name when you create the button like in this second example, and your app will be automagically installed.
<span class="g-signin" data-apppackagename="org.aliafshar.android.driveapp">
I know many of your Drive apps have mobile and web components, so this should be really useful for you. This helps you provide your users with a beautiful and seamless experience on all of their devices.
All-in-all, we think you’ll find these features useful and recommend that you use the Google+ Sign-In as the preferred way to authorize a user with the Google Drive API from inside a user interface. Check out how to get started with Google+ Sign-In in the language of your choice.
We're looking for a small set of developers that are committed to building apps for Google Drive to join our Google Drive SDK early access program. Participants will get early access to upcoming features and the opportunity to shape the direction of the SDK. This is an ongoing program covering multiple aspects of the API, and there are two new features that we're ready to share with developers.
Some of you might have already heard of the upcoming Google Drive Realtime API at Google IO 2012. The Google Drive Realtime API will allow you to use the technology that powers the realtime collaboration features of Google products such as Google Docs and Google Sheets in your own application. It will handle all aspects of data transmission, storage, and conflict resolution when multiple users are editing.
We are looking for trusted testers for what will be a short and intense pre-release phase of the Drive Realtime API. Good candidates will be:
We also told developers about an upcoming Push Notifications system at Google IO 2012. Push Notifications will allow you to get near-instant notifications when files are modified in Google Drive. In the past you would typically have had to frequently poll the Drive API to check if files have been modified to obtain similar results, Push notifications makes this super efficient.
Please fill out our signup form to tell us more about your use case and we’ll contact you shortly.
When we launched version 1.0 of Google Play services to all Android 2.2+ devices worldwide in September, one of our main goals was to provide developers with better tools for working with OAuth 2.0 tokens in their Android apps.
Thanks to the new components, Android apps can get access to Google APIs with an easy-to-use authentication flow and can provide a consistent experience to both their users and developers. We recently decided to test that statement by writing a small camera app that automatically uploads photos you take to your Google Drive account.
We documented all the steps required to go from zero to hero in a quickstart guide. By following the step-by-step instructions in the guide, you’ll have a working Android app that uses Google Play services to perform authorization and the Google Drive API to upload files to Drive.
Do you want to learn how to build this app but prefer to watch a video tutorial instead of reading the documentation? We’ve got you covered! Check out the recording of the Google Developers Live session that covers the setup and running of the quickstart app.
If you’re building an Android app that integrates with Drive and have questions for us, please don’t hesitate to let us know on Stack Overflow.
Would you like to programmatically publish some web content? Or let your users do so in the context of your Drive app?
This is possible now with Google Drive. Your app can now insert static web assets in a publicly shared folder, and then serve those files directly to users via file names in a relative path. Google Drive site publishing supports JavaScript, so it's even possible to run a JavaScript Drive app directly from Drive.
Publishing from Drive is a simple, two-step dance: create a public folder and use a link to the root folder — the webViewLink — to publish its contents. You can refer to the Drive SDK documentation for full detail on how to work with public folders and content links, but basically the requirements are:
webViewLink
It’s important to emphasize the added simplicity provided by a webViewLink: using this link as a starting point, you can extend the path to any web asset in any subfolder without worrying about retrieving the exact file ID. What used to look like 0B2Gk2F2ImIBiaUkwY3JNX1JMaTg is now a recognizably cute path such as images/kittens.jpg. For the root path to the folder, we’ll display a list of files in the folder, or, if you have an index.html file in your folder we’ll load that as expected.
0B2Gk2F2ImIBiaUkwY3JNX1JMaTg
images/kittens.jpg
index.html
The ability to publish files this way opens lots of possibilities for Drive app developers. Writing a blogging tool, creating a process to publish updates to shared docs, outputting images in a folder in a gallery page — any Drive use case that involves presenting a file in a browser can benefit from site publishing. We look forward to seeing what you create, and we’re happy to answer your questions on Stack Overflow.
Since the public unveiling of the Google Drive SDK in April, companies like Lucidchart or HelloFax have built powerful, slick, useful Google Drive apps, and many more companies are launching compelling integrations every day. During this time, our developer community — especially on Stack Overflow — has grown substantially.
To help support our growing developer community and all the interest in integrating with Google Drive, we’re starting a series of Google Drive developer workshops. For the inaugural event, we are hosting several companies — including Shutterfly, Fedex, Autodesk, Twisted Wave, 1DollarScan and Manilla — to participate in a two-day workshop this week at the Googleplex in Mountain View, California.
During this workshop, Google engineers will be on hand to assist attendees with various parts of their Google Drive integration: things like design and implementation of features, authorization flow, and Android integration. Companies have shown that the Google Drive SDK allows for deep integration in just a couple days and we really hope that attendees of this workshop will enjoy a similar experience. Tune back in later this week to find out more about what we learned and accomplished in our workshop.
If you are interested in attending similar Google Drive workshops near you or if you want to contact the Google Drive team about a potential integration with your product, let us know.
