Configure Github Webhook
Now let’s get some real data into this data infrastructure. We’ll do this by configuring Github’s webhook service to send event data to the /ingest/RawStarEvent
API.
Since your Moose development server is running locally, you’ll need to use Pinggy or NGROK to expose the port its running on to the web. This allows for external webhook notifications from GitHub to be proxied to your local dev server.
Get a Port Forwarding URL to localhost:4000 (opens in a new tab)
To connect you local Moose instance over the internet, you need a port forwarding service that connects your local port to a public URL.
Run this command in your terminal:
ssh -p 443 -R0:localhost:4000 -L4300:localhost:4300 qr@a.pinggy.io
You should see a few messages printed to your terminal, followed by this prompt:
The authenticity of host '[a.pinggy.io]:443...
...
Are you sure you want to continue connecting (yes/no/[fingerprint])?
Type yes
to add Pinggy to your known SSH hosts.
You should see the below message printed to your terminal output:
You are not authenticated.
Your tunnel will expire in 60 minutes. Upgrade to Pinggy Pro to get unrestricted
tunnels. https://dashboard.pinggy.io
http://<YOUR_PINGGY_GENERATED_LINK>.a.free.pinggy.link
https://<YOUR_PINGGY_GENERATED_LINK>.a.free.pinggy.link
Copy the https
URL
This is your forwarding URL. You’ll need this to set up the Github webhook in the next step.
It will come in handy later in the tutorial
Configure Github Webhook
Now you’ll configure GitHub to send webhook notifications to your Pinggy forwarding URL when someone stars your repository.
Login to your Github.com (opens in a new tab) account.
Select a Repository to Track
Navigate to this repository to set up webhook notifications.
You must own this repository!
Open Settings > Webhooks in your Repository
- Locate the 'Webhooks' settings in the sidebar navigation
- On the Webhooks page, click the Add Webhook button.
Configure Webhook Settings
-
In the Payload URL field, paste in your forwarding URL you previously copied to your clipboard. Append:
/ingest/RawStarEvent
-
Set Content type to
application/json
. -
Secret: Leave empty.
-
Under "Which events would you like to trigger this webhook?", choose "Let me select individual events".
- Scroll through the events to configure notifications:
- Unselect "Pushes"
- Select "Stars"
- Scroll through the events to configure notifications:
Save Your Settings
Click "Add Webhook" to save your configuration.
- The new webhook will appear in the list. It won't have a green checkmark until it sends its first notification.
Trigger the Webhook: Get the Live Data Flowing!
Star & Un-star your Github Repository
Go to your Github repository and star/un-star it.
We’ll give it a star too!
Check for POST
and SUCCESS
Messages in CLI
Check back on your terminal window. You should see POST
and SUCCESS
messages printed to the CLI:
POST ingest/RawStarEvent/0.0
SUCCESS /ingest/RawStarEvent
You now have live, real-time Github event data landing in your Moose app!
Re-run RawStarEvent
Query in DB Explorer
Go back to your database explorer and re-run this same query on the RawStarEvent
table from earlier:
SELECT * FROM local.RawStarEvent_0_0
You should see the your own Github username and up-to-date information about your repositories!
You've defined a Data Model, configured a GitHub webhook, and ingested real-time GitHub event data into your Moose application.
Your full data stack is up and running—a webserver, streaming data platform, and OLAP database—with live data flowing in. You achieved all of that with just a few lines of code!