Jenkins Auto-Configuration of GitHub WebHooks to Initiate Build Jobs

Wow. What a mouthful of a title.

It turns out that Jenkins has the capability to make calls to GitHub and set up WebHooks. A GitHub WebHook is a thing that can be set up to issue an HTTP(S) call back to Jenkins whenever things happen to a particular GitHub repo. In particular, things like a push to GitHub or a merge of a Pull Request will cause GitHub to notify Jenkins. What we want is for Jenkins to then pull the new code and do a build, running tests, and all that.

I am using GitHub Enterprise. That means is is an independent GitHub server (not the one on the internet at github.com) and it is running in my employer's internal network. This GitHub instance is only available to us when at work or when connected to work via VPN.

In Jenkins, I am using Cloudbees Enterprise Jenkins (the version with an Operations Center and one or more controlled Client Master instances). But this should work with the open source version of Jenkins where there is only a Master. The reason its the same is that in Cloudbees Enterprise Jenkins the Masters work with GitHub and the Operations Center does not. It means you have to configure all your masters separately.

Configure to Auto-Manage GitHub Hooks

For each master, go to the main dashboard page.

• Manage Jenkins > Configure System > GitHub > Add GitHub Server > GitHub Server (Note: I have two places to enter a GitHub server on this page. We are using the one that
• GitHub > GitHub Server > API URL:  something like https://ghub.mycompany.com/api/v3
• Click Advanced (The one after the “Add GitHub Server” button)
• Additional Actions > Manage additional GitHub actions > Convert login and password to token
• GitHub API URL (again) : https://aaghub.qcorpaa.aa.com/api/v3
• Select radio button to enter it from login and password
• Manage Hooks: Click to select
• Enter a GitHub account id and password. The account should have permission to Admin any repos that you will set up to be automatically built when the content changes. (Perhaps less permissions than that are needed. Dunno.)
• Click Create token credentials to generate
• Make note of the generated credential’s name (it’s a sequence of numbers)
• Save
• Manage Jenkins > Configure System > GitHub Servers: Comes back to same screen
• GitHub Server > Credentials: Select the newly created credential (with the long name)
• Save

Now any jobs that are set to build on a change from GitHub (and that have a GitHub repo URL available on the config page) will have Jenkins configure GitHub to notify whenever that repo changes. (Note: This will not apply for GitHub Organization job types or Multi-Branch Build job types as they extend from a different subclass that does not have that code in it yet.)

Now we need to figure out if its working:

Go to the main dashboard page on the master OR click down from there to a folder where you can add a build job.

• New Item
• Name: TestJob1
• Click Freestyle Job
• Click OK

It will go the page where you add a freestyle job.

• Source code management > select GitHub
• Repository URL: enter one
• Credentials: Select the right one (I'm assuming you know how to access a GitHub repo code)
• Build Triggers > Select Build when a changed to pushed to GitHub
• For now select Build > Execute Shell (or Windows Batch Command if appropriate)
• In the shell script box, I put "ls -al" (or "dir", respectively).
• Save

Now to the the GetHub Web UI, log in using an account that has admin permissions on the repo you are using.

• Move through the web pages to the main page for the repo
• Click "Settings" at the right side of the top line of options
• Click "Hooks and Services" in the left, vertical menu
• Verify that there is a webhook pointing to your Jenkins server. It will use the URL that you configures as the URL of the server in "manage jenkins > configure system".

If so, you are golden. If not, you have a bunch of debugging to do. I don't know all the ways it can go wrong.

Maybe the Jenkins server cannot connect to the GitHub server over the internet. Go to a command line on the Jenkins server and ping or telnet to the GitHub server and see if it connects. 

Maybe the GitHub server cannot connect to the Jenkins server. Go to a command line on the GitHub server and see if you can ping or telnet to the Jenkins server. This may not be permitted. 

You can also test this by entering the web hook by hand in the GitHub repo and testing it. Note that if you leave the one you entered by hand, you will never know if the automatic method is going to work the next time you create a build job needing to auto-build from GitHub.

Maybe you go to the Manage Jenkins page and a red box is at the top telling you that some GitHub WebHooks didn't get created. Click to view this and you might get a clue as to what is wrong. If the credentials you selected when configuring the system under Manage Jenkins are invalid or do not have admin permissions in GitHub, this sort of error can happen.

That page you just viewed may have information about how to see the log messages relating to GitHub web hooks. Set up a logging context with these three items:

1. com.cloudbees.jenkins.GitHubPushTrigger
2. com.cloudbees.jenkins.GitHubWebHook
3. org.jenkinsci.plugins.github

Set the log level to "ALL" for these three.

Then go configure the test job again and save it again. This seems to retry setting of hooks sometimes. (But if you have messages about the hooks not being created, you will need to "ignore" each and then "disignore" each again.) I'm not really sure what it is that makes Jenkins try to create the webhook again if it doesn't work the first time. Sometimes I would just go build the test job and see if the hook got set up. Someone should document this.