Compute Engine and App Engine – a Comparison


I want to show you a little demo of how Compute Engine and App Engine work. Both techs have their strengths and weaknesses, and I wanted to make something to showcase them. 

Compute Engine allows you to spin up Virtual Machines (henceforth to be referred to as “VMs” due to the fact that I can’t be bothered to write “irtual” and “achine”.) VMs give you a lot of control over your system. You can run a number of OSes, with variable processor, memory, and disk configurations. You interact with it by configuring a VM through the Developer Console or on the command line. You then SSH into your VM.

App Engine on the other hand just takes code.  You upload it and we run it. No SSH, no machine, just an upload site and a URL. App Engine by default gives you no control over the hardware running the code. The trade off is that we can immediately scale from zero load to any load you muster.

So how do these compare? App Engine scales in milliseconds? What does that look like? Compute Engine starts up in 10s of seconds? What does that mean? This demo shows off how you can build Compute Engine machines vs how fast you can spin up App Engine instances. This isn’t a one-is-better-than-the-other comparison; there are reasons to use both of these techs, and they aren’t mutually exclusive. Let me know what you think.


PHP on App Engine General Availability

appenginephpEarlier this week, Google Cloud Platform announced General Availability of PHP on App Engine.  Developers are now free to use App Engine to power their developer experience using…

Oh wait, you were already using PHP on App Engine. And have been doing so for a few months, or years.  What does this announcement mean for you?

The big bullet point here is that Google is taking the “Beta” label off the PHP on App Engine. It’s is now governed by the Service Level Agreement, and Deprecation Policy.

Now I’m not a lawyer, so all the rest of this is subject to, you know, me not being a lawyer, and therefore any interpretation herein, yada yada. You know, check with your lawyery people before taking my word for it. I’m mostly going to just describe these things, and point you to the actual documents.

Service Level Agreement

The SLA sets expectations for how much uptime Google Cloud Platform delivers, and what happens if they let you down.  It puts forth a number of uptime stats they need to hit, and what Google Cloud Platform will do if they do not meet them. It also outlines what you need to do to get compensation.

Read the SLA for more information.

Deprecation Policy

The Deprecation Policy states how long Google Cloud Platform will try and run services covered by the Deprecation Policy after a deprecation announcement, unless there is a very serious reason not to.

Read the Deprecation Policy, contained in section 7, for more information.

Please, read these with your lawyerly people. Provide them with Scotch, the promise of billable hours, and whatever else you need to give your lawyerly people to make them happy (Orphan tears?  I kid, I kid.  Please don’t sue me.)

This is a signal that PHP is joining the list of technologies that you can feel secure to choose Google Cloud Platform to host.

Atlassian Connect add-on Starter Kit for App Engine

atlassian_gcpAtlassian Connect add-ons are extensions written by third-party developers to augment Atlassian’s hosted software. We’d love for the Atlassian Community to host their add-ons with Google Cloud Platform.  To help out, I’m releasing Atlassian Connect Add On Starter Kit for App Engine on github.

Google App Engine is a great environment to get started hosting this sort of solution.

  • You just upload code.
  • We handle scaling for you.
  • App Engine serves up webservices with little to no config.
  • App Engine integrates well with other Google Developer Services

On the Atlassian side, as long as you write your services to respond the way Connect expects,  you can write your add-ons in whatever language or technology stack you want.

I wrote the Atlassian Connect Add On Starter Kit for App Engine to give a working example for getting started on App Engine using each of the supported languages.  Each language folder (Go, Java, Php, Python) contains code to create two applications:

Hello World – basically the Hello World example written by Atlassian with added App Engine configs to show the delta between a vanilla add-on and one configured to run on App Engine.

The Hello World. Nothing special.

Language Check – this application scans a JIRA instance for issue titles in languages that differ from the default language, and then labels which language they are.  The idea is that you can then easily triage issues that should be routed to staff who can support that language. It takes advantage of the Google Translate API to accomplish this.

Add-on translating with the default language set to English.
Add-on translating with the default language set to Czech.

Now, all of this isn’t to say you cannot use other technologies in Google Cloud Platform to host your add-on.  You can also host on either Compute Engine or Container Engine. After you configure one of these solutions you can just follow Atlassian’s existing tutorials to create an add-on.

Anyway you want to do it, Google Cloud Platform can help the Atlassian developer community to host their applications.  If you want to write some code and have server configuration and scaling handled by us, App Engine is a good choice.  If you would rather have more control over or flexibility with the systems you run, either Compute Engine or Container Engine can be your answer.

Atlassian Connect Add On Starter Kit for App Engine

Google Cloud Platform at Google I/O

Cloudplatform-at-IOGreat news everyone: we’re going on the road! Google Cloud Platform will be running a series of events this summer called Google Cloud Platform: Next to bring a world tour level of attention to everything we are doing in the Cloud.

This week is Google I/O. The Cloud Platform team will be at Google I/O in a few sessions and events, but you may find fewer of us on the floor and speaking at Moscone Center. However my co-workers, the Developer Advocacy team, will be overflowing to space at Galvanize SF, which is just a few blocks away. The rest of our extended team is hard at work preparing for Google Cloud Platform: Next.  We can’t wait to see you all on tour.

In the meantime, you will be able to find members of the Cloud Platform team at these sessions at Google I/O:


1:00PM – 2:00PM

Room 1 (L2)

Google Cloud Messaging 3.0


1:00PM – 1:30PM

Alcove 3 (L2)

Containers to back your mobile app


1:30PM – 2:00PM

Develop Talk 3 (L2)

Mobile app quality leaps to the cloud


3:00PM – 3:30PM

Develop Talk 3 (L2)

Building a real-time app in 5 minutes with Firebase


9:00AM – 9:30AM

Earn & Engage Talk (L2)

Real-time analytics for mobile and IoT


11:30AM – 12:00PM

Alcove 3 (L2)

Containers to back your mobile app


2:00PM – 2:30PM

Earn & Engage Talk (L2)

Building a real-time app in 5 minutes with Firebase


2:30PM – 3:00PM

Design Talk (L2)

Real-time analytics for mobile and IoT


3:30PM – 4:00PM

Develop Talk 3 (L2)

Mobile app quality leaps to the cloud

As mentioned, we will also be holding an event at Galvanize, from 1-9PM on day 1 (Thursday) of Google I/O.  Members of the Cloud Platform Developer Advocate team will be there for the entire event, and speaking in the evening.

1:00PM – 5:00PM

Tech Stop
Consult with Google Cloud Platform Developer Advocates

5:30PM – 6:00PM

Choosing between App Engine, Compute and Managed VMs
Understand the differences between Google Cloud Platform’s computing options, to make the best choice for your app.
Terry Ryan, Developer Advocate, Google

6:00PM – 6:30PM

Making the real world talk to us in real-time with mobile video
Carter Maslan, CEO, Camio

6:30PM – 7:00PM

Google for Entrepreneurs
Mary Grove, Director of Google for Entrepreneurs

7:00PM – 9:00PM

Drinks, food, chat

Finally, we’ll be hanging out at the various satellite events and in hotel bars in the evening. Track us down on Twitter – you can find us using @googlecloud’s list on Twitter, and when you find one of us, bring your questions, share your ideas, and pester us for swag! Enjoy I/O and we can’t wait to see you at Google Cloud Platform: Next.

Managed VM Not Connecting to on OS X

Ran into this issue today fooling around with Docker and Managed VMs on Google Cloud Platform. I was running on Mac OS X, which meant boot2docker was also in the mix. Figured this could help someone else because it baffled me for a bit.

I was trying to start up a standard runtime for python on Managed VM. I was using the following app.yaml:

I ran the gcloud command:

In the midst of the output, this error was buried:

What was really frustrating is that this had worked yesterday and I had changed nothing in between.  I tried a whole bunch of things. I fired a whole lot of searches on Google. In my searches I found this thread on github that suggests it’s an issue with some sort of caching in Docker. So I did the following:

  • Launched Virtual Box
  • Right clicked boot2docker-vm
  • Chose Close ->Power Off
  • Launched boot2docker again via Applications->boot2docker
  • Tried again

And now it works.

Hope this helps someone else.

When to Pick Google Bigtable vs Other Cloud Platform Databases

dbsRecently, Google Cloud Platform announced the availability of an additional database option for our customers: Google Cloud Bigtable.  Now Cloud Platform Developers have another Google managed solution for storing their data. So, when do you use which tool?

The contenders:

Cloud SQL

Are you using some sort of relational database with tables, views, and  indices? Does your application rely on stored procedures and custom table views or write joins? Do you need the certainty of transactions and ACID compliance?  Do you generally both read and write to the data, not in equal amounts, but not lopsidedly one or the other?  If you answered “yes” to a lot of these you probably want to investigate or stick with Cloud SQL, which as its name suggests, is an implementation of MySQL hosted by Google.

Cloud SQL does have all of the limitations of MySQL. Certain types of applications don’t require the complexity of normalized data with no duplication. Other cases require scaling in a manner that SQL cannot handle without additional complexity, reduced performance, or higher cost. Going into the full pros and cons of SQL vs NoSQL is beyond the scope of this post, but rest assured there are reasons that both of them exist, and both are valid choices depending on circumstances.

NoSQL: Datastore or Bigtable

If you are leaning towards a NoSQL solution you now have two Google managed NoSQL choices on our platform. What is the differentiator between Cloud Datastore vs Cloud Bigtable? They are both NoSQL solutions. Both are described as massively scalable.  Both leave you with little to no management (or No Ops for those of you playing buzzword bingo).  The answer lies in four areas:

  • Size
  • Structure
  • Analysis
  • Interface


Bigtable is optimized for mind boggling huge sets of data. Seriously, it is most cost effective when dealing with datasets that start at 1 Terabyte. Datastore can handle large data sets too, but Datastore is performance and cost optimized to handle smaller sets of data too.  Have a few GBs of data? Datastore would be the better call. Have data that might start out small, but grow to a Terabyte in time, still Datastore. Have data that starts at a Terabyte and will keep expanding? Then you’ve started down a path that might make Bigtable interesting. But size alone isn’t the only factor.


Bigtable stores data in a big honkin’ table. Yeah, the name is a little on the nose, but it’s true. There are rows and columns somewhat like relational database systems but not exactly. But it has a schema, and predefined structure.

Cloud Datastore, on the other hand, is more optimal for ad hoc storage of structured data representing objects.  Basically you define an object and then push it into Datastore. You don’t define a schema, create tables, or set up any other sort of structure before storing a record.


Do you need to analyze the data in massive aggregate scale while the database is still online and taking requests? Do you want to run MapReduce on your production data without copying it somewhere for study? Do you want to hook it up to various Big Data analysis toolkits?  If this sounds like what you want to do, Bigtable makes more sense.


If you are coming to Google Cloud Platform from other technologies, and are working with HBase, Bigtable is for you.  Bigtable is accessible through extensions to the HBase 1.0 API and is therefore compatible with a lot of the Hadoop ecosystem as well as other Big Data tools.

On the other hand,  there are also a few limitations. You cannot join. There is no SQL interface. The API gives you Put/Get/Delete individual records, or you can run Scan operations.

Datastore does not have SQL either, but has an API called GQL that while not exactly the same does abstract querying objects in a way that most SQL developers should be able to quickly understand.


Finally the product page has a great explanation of Bigtable’s relation to other Google Cloud Platform offerings:

Cloud Bigtable and other storage options

Cloud Bigtable is not a relational database; it does not support SQL queries or joins, nor does it support multi-row transactions. Also, it is not a good solution for small amounts of data (< 1 TB).

  • If you need full SQL support for an online transaction processing (OLTP) system, consider Google Cloud SQL.

  • If you need interactive querying in an online analytical processing (OLAP) system, consider Google BigQuery.

  • If you need to store immutable blobs larger than 10 MB, such as large images or movies, consider Google Cloud Storage.

  • If you need to store highly structured objects, or if you require support for ACID transactions and SQL-like queries, consider Cloud Datastore.

In short, there is a lot of awesome stuff about Cloud Bigtable, but it doesn’t mean that it is right in all cases.  It’s a NoOps, NoSQL, Big Data analysis tool, meant to be used at massive scale in conjunction with other Big Data tools.  I recommend that you check out the documentation for Bigtable as there is much more to be found there. And let me know if you need more clarification on anything.

PHP on App Engine Updating to PHP 5.5

A few weeks ago Google Cloud Platform released an update for PHP on App Engine that enabled PHP 5.5 on App Engine. It was all very exciting, and there was a forum post about it any everything.  At the bottom though there is a little note:

After 16th April, 2015 we will begin automatically migrating all applications to the php55 runtime.

You may have also seen emails from  “Google App Engine” reminding you of this.  In those notices it has been changed to:

in approximately 2 weeks we will begin automatically migrating your application over to use the PHP 5.5

If you need an extension on PHP 5.4 you can fill out a form to request one.

So this change is coming, and this post is yet another heads up to remind people.

Lumen on App Engine


Laravel announced today that they are launching a new PHP framework named Lumen for building API’s and microservices.  It has an emphasis on speed and it is compatible with a subset of Laravel, making for easy migrations to the larger framework.

I wanted to see how easy it would be to run a Lumen app on App Engine, and I fooled around with it a bit. You just do a simple setting in app.yaml to route all of the apps traffic to the default handler for the Lumen app:

With that I got a basic instance running. I did run into one problem though.  The logger tried to write to disk, which is a no-no for App Engine. If this happens you will get an error like this:

Fatal error: Uncaught exception 'UnexpectedValueException' with message 'The stream or file "[Path to your lumen app]/storage/logs/lumen.log" could not be opened

One little configuration tweak got it working though. In [path to your lumen app ]vendor/laravel/lumen-framework/src/Application.php, you have to tweak the logger a bit.

First you add a reference to the right library:

And then replace the function getMonologHandler() with this:

This advice was taken from the guide for Laravel on PHP for App Engine.

All in all, it is a relatively quick little framework, and having using App Engine to scale out the services it provides seems like a no-brainer.

All code shown here is licensed under Apache 2. For more details find the original source on Github.

Sharing Memcache Between Languages in App Engine

memcache diagramIn the process of performance testing the ability to swap out languages in App Engine detailed in this post, I stumbled on to something.  I was testing performance, and realized that the tests weren’t being accurate because of differences in caching. Ideally, to get the tests to be apples to apples, I would just have to get my PHP code and Go code to use the same Memcache instance and keys.  (I should have written my testing better, but then if I had I would never have stumbled into this.)

To start, follow the steps to get multiple languages working in a production instance or a development instance.

Assuming you are writing from PHP:

And then to read from Go:

It really is that easy. Now the hard part comes when you want to transfer complex data between the two.  Use JSON to encode the objects.  Both languages can handle it pretty effortlessly and Go on App Engine has JSON object handling built in as a codec to its memcache implementation. You could save it in another format like XML then read and write data like a string, while manually encoding and decoding.  You could also staple your had to your desk. Let’s not be a masochist and just do it in JSON – but I suppose it’s your choice.

Once you do that, it’s as simple as encoding to JSON in PHP:

Then decoding in Go.

Note a couple of things:

  • I omitted graceful memcache miss handling. I did so for brevity. Make sure you wrap your memcache code that handle cache misses.
  • If you are not familiar with Go, those ‘json:’ comments aren’t just comments, they’re instructions on how to encode/decode data between Go and JSON.  So you need them, or it won’t work correctly.

  • I ran into an issue with the original version of this code because latitude and longitudes were coming out of the database into PHP as strings and not floats. When you went to get them out of memcache in Go, it would through a type mismatch error.  There are 2 solutions to this:

    • cast them correctly to floats before you write to memcache

    • Use JSON_NUMERIC_CHECK in json_encode to get them to write as proper numerics when you write. This seems like the better solution

Why do this?  For starters I was doing it so  both versions of my API could take advantage of caching done by the other language.  But I am sure there are other uses:

  • Communication between these modules

  • Offloading an expensive data retrieval and processing step to Go then reading memcache from PHP.

  • I’m curious if anyone reading has any thoughts.

Note: This will work on either type of memcache solution on App Engine: shared or dedicated. Just make sure you handle cache misses gracefully.


All code show here is licensed under Apache 2. For more details find the original source on Github.


Two Languages in App Engine Development

In my last post I outlined getting Go and PHP to act as modules in the same App Engine instance.  However I only really tested it on a “production” App Engine instance, I didn’t test it in development, because I typically use the Google App Engine SDK for each respective language separately.

When I tried the combined dispatch.yaml on the Google App Engine SDK for PHP I got the following error on a Mac running OS 10.10.2 (Yosemite):

OSError: [Errno 2] No such file or directory: '/Applications/Development/’

When I tried the combined dispatch.yaml on the Google App Engine SDK for Go I got the following error:

The development server must be started with the --php_executable_path flag set to the path of the php-cgi binary.

And when  used the -php_executable_path option with any of the copies of PHP on my system – including the ones that are buried in the PHP SDK – I got:

_PHPEnvironmentError: No input file specified.

After struggling a bit with this here is the easiest solution I found:

  • Find the location of goroot in the go_appengine folder

  • Find the location of the SDK for php by running

  • Create a symbolic link to goroot in go_appengine in the PHP SDK folder that contains

After that you can test your dispatch file in development by running:

Where api_php is the folder your PHP module is in, and api_go is… well you know what I’m saying.

Now, I went out of my way there to say this was the easiest way of doing it.  Not that it wasn’t a hack, or that it was a supported way of doing it.  But it does work.

All code show here is licensed under Apache 2. For more details find the original source on Github.