Tutorial
Connect to Github for Live Data

Configure Github Webhook

Now let’s get some real data into this data infrastructure. We’ll do this by configuring Github’s webhook service to send event data to the /ingest/RawStarEvent API.

Warning:

Since your Moose development server is running locally, you’ll need to use Pinggy or NGROK to expose the port its running on to the web. This allows for external webhook notifications from GitHub to be proxied to your local dev server.

Get a Port Forwarding URL to localhost:4000

To connect you local Moose instance over the internet, you need a port forwarding service that connects your local port to a public URL.

Run this command in your terminal:

ssh -p 443 -R0:localhost:4000 -L4300:localhost:4300 qr@a.pinggy.io

You should see a few messages printed to your terminal, followed by this prompt:

The authenticity of host '[a.pinggy.io]:443...
...
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Type yes to add Pinggy to your known SSH hosts.

You should see the below message printed to your terminal output:

                           You are not authenticated.
Your tunnel will expire in 60 minutes. Upgrade to Pinggy Pro to get unrestricted
                      tunnels. https://dashboard.pinggy.io

http://<YOUR_PINGGY_GENERATED_LINK>.a.free.pinggy.link
https://<YOUR_PINGGY_GENERATED_LINK>.a.free.pinggy.link

Copy the https URL

This is your forwarding URL. You’ll need this to set up the Github webhook in the next step.

Save Your Forwarding URL

It will come in handy later in the tutorial

Configure Github Webhook

Now you’ll configure GitHub to send webhook notifications to your Pinggy forwarding URL when someone stars your repository.

Login to your Github.com (opens in a new tab) account.

Select a Repository to Track

Navigate to this repository to set up webhook notifications.

Warning:

You must own this repository!

Open Settings > Webhooks in your Repository

  • Locate the 'Webhooks' settings in the sidebar navigation
  • On the Webhooks page, click the Add Webhook button.

Configure Webhook Settings

  • In the Payload URL field, paste in your forwarding URL you previously copied to your clipboard. Append: /ingest/RawStarEvent

  • Set Content type to application/json.

  • Secret: Leave empty.

  • Under "Which events would you like to trigger this webhook?", choose "Let me select individual events".

    • Scroll through the events to configure notifications:
      • Unselect "Pushes"
      • Select "Stars"

Save Your Settings

Click "Add Webhook" to save your configuration.

  • The new webhook will appear in the list. It won't have a green checkmark until it sends its first notification.

Trigger the Webhook: Get the Live Data Flowing!

Star & Un-star your Github Repository

Go to your Github repository and star/un-star it.

[Send us your Repository URL!](mailto:contact@fiveonefour.com)

We’ll give it a star too!

Check for POST and SUCCESS Messages in CLI

Check back on your terminal window. You should see POST and SUCCESS messages printed to the CLI:

POST ingest/RawStarEvent/0.0
SUCCESS /ingest/RawStarEvent
Congrats!

You now have live, real-time Github event data landing in your Moose app!

Re-run RawStarEvent Query in DB Explorer

Go back to your database explorer and re-run this same query on the RawStarEvent table from earlier:

Your DB Explorer
SELECT * FROM local.RawStarEvent_0_0

You should see the your own Github username and up-to-date information about your repositories!

Celebrate your Progress!

You've defined a Data Model, configured a GitHub webhook, and ingested real-time GitHub event data into your Moose application.



Your full data stack is up and running—a webserver, streaming data platform, and OLAP database—with live data flowing in. You achieved all of that with just a few lines of code!

Ingest Sample DataProcess Raw Events with Streaming Functions