Big Query With Ruby On Rails

Today, I would like to introduce how to connect your Ruby on Rails app with Big Query of GCP.

Before starting this tutorial, please make sure that you already have a GCP account.

0. First at all, where is Big Query in data life cycle ?

As image above, GCB can be used in Store, Process and Analyze or Explore and visualize.

Let's imagine as follow, your GCP application has 4 phases:

• Ingest: Here, just need to run your application (app engine with static site, cloud function, etc), in this phase you should create a mechanism to transfer your application data to BigQuery.
• Store: Stocking data while running your application.
• Process and analyze: After running for a long time, your application has a large amount of data to be processed and analyzed.
• Explore and visualize: Then you should have a place to watch your processed data, right ?

1. Create / select project

Go to cloud console, then select (or create a new project)

2. Enable BigQuery service

Go to this URL to enable big query API for your project

3. Create an account key

Next, back to cloud console again, then click on Service Accounts > Create service account to create a new service account

After your service account has been created, please download account key (JSON file) as following steps

Create a service account key:

1. In the Cloud Console, click the email address for the service account that you created.
2. Click Keys.
3. Click Add key, then click Create new key.
4. Click Create. A JSON key file is downloaded to your computer.
5. Click Close.
6. Copy this key to your project

4. Install Google Cloud BigQuery client library

gem install google-cloud-bigquery

5. Create your service

# frozen_string_literal: true

module BigQueries
  class BigQueryService
    require 'google/cloud/bigquery'

    def initialize
      # You can move this one to setting instead
      project_id = 'sample-project-id'
      key_file = Rails.root.join('path/to/your/account/key.json')
      @big_q = Google::Cloud::Bigquery.new project: project_id, keyfile: key_file
      super
    end

    def query(raw_query)
      @big_q.query raw_query
    end

    def datasets
      @big_q.datasets
    end

  end
end

6. Use this service

Create a controller and handle all requests that you need to send to BigQuery, and update config/routes as other controllers of RoR, e.g:

# frozen_string_literal: true

module Api
  module V1
    class BigQController < ::Api::ApplicationController
      def query
        results = BigQueries::BigQueryService.new.query(params[:query])

        render status: :ok, json: { data: results.as_json }
      end
    end
  end
end

7. Enjoy it

Now, let's try our new controller on Postman

Yay, your application can communicate with GCP Big Query now. I hope this article can help you know how to work with GCP APIs (Other API will follow same steps, with other client library) with your Ruby On Rails application.

Thanks for reading. Happy coding!