bob-pr-reviewer v0.0.5

An MCP (Model Context Protocol) server that gives AI assistants in VS Code the ability to read GitHub Enterprise pull requests, fetch Jira ticket requirements, and post structured review comments — all via natural language.

No clone required. No build step. No manual server startup.
VS Code launches this automatically via npx — just configure once and it works everywhere.

Table of Contents

• How It Works
• Quick Start (5 minutes)
• How to Invoke in IBM Bob
• Tools Available
• Usage Examples
• What to Expect in VS Code
• What to Expect on GitHub
• Project Structure
• For Contributors / Local Development
• Publishing a New Version
• Troubleshooting
• Security Notes

How It Works

You:    "Use the bob-pr-reviewer MCP server to review PR #1758 in sec-ci/notification"
           │
           ▼
     VS Code AI Assistant (IBM Bob / GitHub Copilot Chat / Cline)
           │  sees MCP tools available, decides which to call
           ▼
     bob-pr-reviewer (auto-launched by VS Code via npx — no clone needed)
           │  calls GitHub Enterprise API + Jira API in parallel
           ▼
     GitHub Enterprise (github.ibm.com)  +  Jira (jsw.ibm.com)
           │  returns PR diff + Jira requirements
           ▼
     AI analyzes code vs requirements → posts structured comment to PR

VS Code spawns this MCP server as a subprocess automatically using npx. The package is downloaded and cached on first use — you never need to clone this repo or run a build command.

Quick Start (5 minutes)

Full step-by-step instructions: see SETUP.md

Step 1 — Get your tokens

| Token | Where to get it | |-------|----------------| | GitHub Enterprise PAT | https://github.ibm.com/settings/tokens → scope: repo | | Jira Personal Access Token | https://jsw.ibm.com/secure/ViewProfile.jspa → Personal Access Tokens |

Step 2 — Set your tokens

Add to your ~/.zshrc (or ~/.bashrc):

export GITHUB_TOKEN=ghp_yourGitHubTokenHere
export JIRA_ISV_PERSONAL_ACCESS_TOKEN=yourJiraTokenHere

Then reload: source ~/.zshrc and restart VS Code completely.

Step 3 — Add to VS Code User Settings

Open VS Code global settings JSON (Cmd+Shift+P → "Open User Settings (JSON)") and add:

{
  "mcp": {
    "servers": {
      "bob-pr-reviewer": {
        "type": "stdio",
        "command": "npx",
        "args": ["-y", "bob-pr-reviewer"],
        "env": {
          "GITHUB_TOKEN": "${env:GITHUB_TOKEN}",
          "GITHUB_BASE_URL": "https://github.ibm.com/api/v3",
          "JIRA_ISV_PERSONAL_ACCESS_TOKEN": "${env:JIRA_ISV_PERSONAL_ACCESS_TOKEN}",
          "JIRA_BASE_URL": "https://jsw.ibm.com"
        }
      }
    }
  }
}

⚠️ This goes in User Settings (JSON) — not in .vscode/settings.json or .vscode/mcp.json in your project. User Settings apply globally across all projects.

Step 4 — Reload VS Code

Cmd+Shift+P → "Developer: Reload Window"

That's it. The server starts automatically. Open any project and the tools are available.

How to Invoke in IBM Bob

The reliable way — tell Bob to use the MCP server by name

IBM Bob will automatically call the right tools when you explicitly mention the MCP server name:

Use the bob-pr-reviewer MCP server to review PR #1758 in sec-ci/notification

Use the bob-pr-reviewer MCP server to check setup

Bob will ask you for any missing details (repo, PR number, Jira ticket, etc.) interactively — you don't need to provide everything upfront.

Why you need to mention the server name

Without the server name, Bob may try to answer from general knowledge instead of calling the actual GitHub/Jira APIs. Mentioning bob-pr-reviewer MCP server tells Bob to use the tools from this specific server.

IBM Bob limitations (current)

ℹ️ These are IBM Bob UI gaps — the MCP server code is fully correct and complete.

| Feature | IBM Bob | VS Code Copilot Chat | Cline / Roo | |---|---|---|---| | MCP tools (core functionality) | ✅ Works | ✅ Works | ✅ Works | | /server-name:command slash commands | ❌ Not yet supported | ✅ Works | ✅ Works | | @server-name mention autocomplete | ❌ Not yet supported | ✅ Works | ✅ Works |

The MCP prompts (slash commands) are registered in the server and will work automatically once IBM Bob adds support for them. No code changes will be needed.

Slash commands (work in VS Code Copilot Chat and Cline today)

MCP prompts are registered as slash commands. In clients that support them, type the command name to invoke — the AI will ask you for any required details interactively. Each command takes no arguments; the AI collects what it needs through conversation.

| Slash command | Description | |---|---| | /bob-pr-reviewer:check-setup | Run prerequisites check — verifies GitHub token, Jira token, and API connectivity | | /bob-pr-reviewer:review-pr | Review a PR — asks whether to also check against its linked Jira ticket, then posts findings | | /bob-pr-reviewer:review-pr-vs-jira | Review a PR against its linked Jira ticket requirements and post the analysis | | /bob-pr-reviewer:get-pr-info | Fetch PR metadata (title, author, branches, commit SHA, description) | | /bob-pr-reviewer:get-jira-ticket | Fetch a Jira ticket's summary, description, and acceptance criteria | | /bob-pr-reviewer:post-comment | Post a general comment on a PR |

💡 The bob-pr-reviewer: prefix ensures these commands never clash with slash commands from other MCP servers you may have loaded.

How /bob-pr-reviewer:review-pr works

When invoked, the AI will:

1. Ask you for the repository and PR number (if not already provided)
2. Ask: "Do you have any additional context documents (design docs, meeting transcripts, RFCs, ADRs, or discussion notes) that would help understand the intent of this PR? If yes, please provide URLs or local file paths."
3. If context documents are provided → calls fetch_context_document for each one
4. Ask: "Would you like me to also check this PR against its linked Jira ticket requirements?"
5. If yes → calls generate_review_findings (passing any context documents; auto-detects the Jira key from the PR), analyzes the results, presents them to you, and asks: "Would you like me to post these findings to the PR?"
6. If you approve → calls post_jira_analysis to post the comment to GitHub
7. If no to Jira check → fetches the PR diff, incorporates any context documents, performs a thorough code review, and posts findings as inline comments using post_review

Tools Available

| Tool | Description | |------|-------------| | check_setup | Re-run all prerequisites checks and show status | | validate_token | Verify your GitHub token is working | | get_pr_info | Fetch PR title, author, branches, commit SHA, description | | fetch_pr_diff | Fetch all changed files + unified diffs | | post_review | Post a review with inline comments at diff positions | | post_comment | Post a general (non-inline) PR comment | | get_jira_ticket | Fetch a Jira ticket's summary, description, acceptance criteria | | fetch_context_document | Fetch a context document from a URL or local file path (design docs, transcripts, RFCs, ADRs) | | generate_review_findings | Generate PR vs Jira analysis findings for user review (does NOT post to GitHub) | | post_jira_analysis | Post approved findings to GitHub as a PR comment (called after user reviews findings from generate_review_findings) |

Tool call chain for Jira-aware review (interactive workflow)

fetch_context_document (optional, called once per document)
    ├── accepts URL or local file path
    └── returns document text content with label

generate_review_findings
    ├── fetches PR info + diff from GitHub
    ├── auto-detects Jira ticket key from PR title/branch/description
    ├── fetches Jira requirements
    ├── appends any context documents passed via context_documents param
    └── returns structured analysis context + findings data to AI
         │
         ▼ (AI analyzes and generates findings)
         │
         ▼ (AI presents findings to USER for review)
         │
         ▼ (USER approves or rejects)
         │
         ▼ (If approved:)
post_jira_analysis
    ├── receives verdict, coverage list, gaps list, summary from AI
    └── posts formatted Markdown comment to the PR on GitHub

Usage Examples

Full PR review (with optional Jira check and context documents)

In IBM Bob:

Use the bob-pr-reviewer MCP server to review PR #1758 in sec-ci/notification

Bob will ask for the repo and PR number if not provided, then ask whether you have any context documents (design docs, transcripts, etc.), then ask whether you want to include a Jira requirements check. Answer yes to get a Jira analysis comment posted, or no to get inline code review comments.

In VS Code Copilot Chat / Cline (slash command):

/bob-pr-reviewer:review-pr

The AI asks for the repo and PR number, asks for any context documents, then asks whether to include a Jira check.

Review with a context document

In IBM Bob:

Use the bob-pr-reviewer MCP server to review PR #1758 in sec-ci/notification. I have a design doc at https://confluence.ibm.com/pages/viewpage.action?pageId=12345 that explains the intent.

Bob will call fetch_context_document to load the design doc, then incorporate it into the review alongside the PR diff and Jira requirements.

You can also provide a local file:

Use the bob-pr-reviewer MCP server to review PR #1758 in sec-ci/notification. Here is a meeting transcript: /Users/me/notes/design-meeting-2025-02-28.md

Jira-aware review only (interactive approval workflow)

In IBM Bob:

Use the bob-pr-reviewer MCP server to check PR #1758 in sec-ci/notification against its Jira ticket

The AI will:

1. Auto-detect the Jira ticket key from the PR (or ask if not found)
2. Generate findings and present them to you
3. Ask: "Would you like me to post these findings to the PR?"
4. Only post to GitHub if you approve

In VS Code Copilot Chat / Cline (slash command):

/bob-pr-reviewer:review-pr-vs-jira

The AI will ask you for the repo and PR number interactively, then follow the same approval workflow.

Fetch PR info only

In IBM Bob:

Use the bob-pr-reviewer MCP server to get the details for PR #1758 in sec-ci/notification

In VS Code Copilot Chat / Cline (slash command):

/bob-pr-reviewer:get-pr-info

Fetch a Jira ticket

In IBM Bob:

Use the bob-pr-reviewer MCP server to fetch the requirements for Jira ticket VDEV-47721

In VS Code Copilot Chat / Cline (slash command):

/bob-pr-reviewer:get-jira-ticket

Post a quick comment

In IBM Bob:

Use the bob-pr-reviewer MCP server to post a comment on PR #1758 in sec-ci/notification: "LGTM! Nice work."

In VS Code Copilot Chat / Cline (slash command):

/bob-pr-reviewer:post-comment

Verify setup

In IBM Bob:

Use the bob-pr-reviewer MCP server to check setup

In VS Code Copilot Chat / Cline (slash command):

/bob-pr-reviewer:check-setup

What to Expect in VS Code

On first use

When VS Code first launches the server, npx downloads and caches the package automatically. You'll see a brief pause — subsequent launches are instant.

To verify the server is running:

1. Open the Output panel: View → Output
2. Select MCP from the dropdown
3. You should see output like this (shown as "errors" in VS Code's panel — this is normal, all MCP server logs go to stderr):

[bob-pr-reviewer] Starting up... GitHub: https://github.ibm.com/api/v3 | Jira: https://jsw.ibm.com
[bob-pr-reviewer] Prerequisites check:
  ✓ Node.js version: Node.js 18.19.0 (requires >= 18.x)
  ✓ GITHUB_TOKEN environment variable: GITHUB_TOKEN is set (length: 40)
  ✓ GitHub token validity: Token is valid — authenticated as: your-username
  ✓ GitHub API reachability: https://github.ibm.com/api/v3 is reachable
  ✓ JIRA_ISV_PERSONAL_ACCESS_TOKEN environment variable: set (length: 38)
  ✓ Jira token validity: Token is valid — authenticated as: Your Name
[bob-pr-reviewer] ✅ All checks passed — registering all tools
[bob-pr-reviewer] MCP server ready (stdio transport)

Note: VS Code's MCP output panel labels all server log messages as "errors" — this is cosmetic. If you see the ✅ All checks passed line, the server is healthy and working correctly.

If any check fails, the server runs in degraded mode — only the check_setup tool is available, and it will tell you exactly what to fix.

Checking setup status

At any time:

In IBM Bob:

Use the bob-pr-reviewer MCP server to check setup

In VS Code Copilot Chat / Cline:

/bob-pr-reviewer:check-setup

The AI will call check_setup and show you the current status of all prerequisites.

What to Expect on GitHub

After generate_review_findings + post_jira_analysis

A structured comment is automatically posted to the PR on github.ibm.com. It looks like this:

🤖 Requirements Check: VDEV-47721

Jira: Domain auth APIs rate limit
Verdict: ⚠️ Partial Match

Requirements Coverage

• ✅ Rate limiting logic implemented in NotificationConfigResource.java
• ✅ New RateLimitConfig class added with configurable thresholds
• ⚠️ Error response format differs from acceptance criteria (returns 429 but missing Retry-After header)

Gaps / Concerns

• Missing Retry-After header in 429 responses per RFC 6585
• No unit tests for the rate limit boundary conditions
• Config reload behavior under concurrent requests not tested

Summary The PR implements the core rate limiting feature described in VDEV-47721 and correctly gates domain auth API calls per tenant. However, the HTTP 429 response is missing the Retry-After header required by the acceptance criteria, and test coverage for edge cases is absent.

Automated check by bob-pr-reviewer • VDEV-47721

After post_review (inline comments)

If you ask for a full code review with inline comments, the AI calls post_review which posts a GitHub review with:

• An overall summary
• Inline comments at specific diff positions (shown in the Files changed tab)
• A review state: COMMENT, APPROVE, or REQUEST_CHANGES

Project Structure

bob-pr-reviewer/
├── src/
│   ├── index.ts            # MCP server entry point — tool registration, startup flow
│   ├── github-client.ts    # GitHub Enterprise REST API client
│   ├── jira-client.ts      # Jira REST API v2 client (PAT auth, ADF parsing)
│   ├── prerequisites.ts    # Startup validator (Node, tokens, APIs)
│   └── tools.ts            # MCP tool schema definitions (reference)
├── dist/                   # Compiled JavaScript — included in npm package
├── .vscode/
│   └── mcp.json            # VS Code built-in MCP config for local dev (uses local dist/)
├── .bob/
│   └── mcp.json            # IBM Bob extension MCP config for local dev (uses local dist/)
├── .env.example            # Template showing required variables
├── package.json            # includes "bin" field for npx support
├── tsconfig.json
├── SETUP.md                # Detailed installation guide for end users
├── PROMPT_TEMPLATE.md      # Reusable prompt template for building new MCP servers
└── README.md

Key files explained

src/index.ts — The main server. Loads .env, runs prerequisites, registers tools and prompts. Two modes:

• Healthy: all 10 tools + all 6 prompts registered (including generate_review_findings for interactive review)
• Degraded: only check_setup tool + check-setup prompt registered (with fix instructions)

src/github-client.ts — Wraps GitHub Enterprise REST API v3. Methods: getPrInfo(), getPrFiles(), postReview(), postComment(), validateToken().

src/jira-client.ts — Wraps Jira REST API v2 with Bearer token auth. Handles Atlassian Document Format (ADF) → plain text conversion. Tries 6 custom field IDs to find Acceptance Criteria. Static extractTicketKey() regex for auto-detection.

src/prerequisites.ts — Runs checks at startup: Node.js version, GITHUB_TOKEN set, GitHub token valid (live API call), GitHub API reachable, JIRA_ISV_PERSONAL_ACCESS_TOKEN set, Jira token valid (live API call). Each failed check includes step-by-step fix instructions.

.vscode/mcp.json — Used by VS Code's built-in MCP support when this repo is open. Points to dist/index.js (local build) so changes are picked up immediately without publishing.

.bob/mcp.json — Used by the IBM Bob extension when this repo is open. Same purpose as .vscode/mcp.json but in Bob's format (mcpServers key). Both files override the global User Settings entry only while this project is open.

For Contributors / Local Development

If you want to modify the server or contribute:

git clone [email protected]:himanshu-sao-ibm/bob-pr-reviewer.git
cd bob-pr-reviewer
npm install
npm run build

Both .vscode/mcp.json and .bob/mcp.json in this repo point directly to the local build output (dist/index.js) so your changes are picked up immediately — no publishing needed. They override the global User Settings entry only when this project is open in VS Code. All other projects continue to use the published npm version.

How precedence works: Project-level MCP configs (.vscode/mcp.json for VS Code built-in, .bob/mcp.json for IBM Bob) take precedence over User Settings when the project is open. Both use the same server name (bob-pr-reviewer) — no conflicts, no duplicate servers.

.vscode/mcp.json (VS Code built-in MCP):

{
  "servers": {
    "bob-pr-reviewer": {
      "type": "stdio",
      "command": "node",
      "args": ["dist/index.js"],
      "env": {
        "GITHUB_TOKEN": "${env:GITHUB_TOKEN}",
        "GITHUB_BASE_URL": "https://github.ibm.com/api/v3",
        "JIRA_ISV_PERSONAL_ACCESS_TOKEN": "${env:JIRA_ISV_PERSONAL_ACCESS_TOKEN}",
        "JIRA_BASE_URL": "https://jsw.ibm.com"
      }
    }
  }
}

.bob/mcp.json (IBM Bob extension):

{
  "mcpServers": {
    "bob-pr-reviewer": {
      "type": "stdio",
      "command": "node",
      "args": ["dist/index.js"],
      "env": {
        "GITHUB_TOKEN": "${env:GITHUB_TOKEN}",
        "GITHUB_BASE_URL": "https://github.ibm.com/api/v3",
        "JIRA_ISV_PERSONAL_ACCESS_TOKEN": "${env:JIRA_ISV_PERSONAL_ACCESS_TOKEN}",
        "JIRA_BASE_URL": "https://jsw.ibm.com"
      }
    }
  }
}

After making code changes, rebuild and reload:

npm run build
# Then in VS Code: Cmd+Shift+P → "Developer: Reload Window"

Publishing a New Version

Who does this: Only the maintainer (himanshu-sao-ibm) needs to do this when releasing updates. Teammates never need to run these steps.

First-time setup (one-time only)

1. Create an account at npmjs.com if you don't have one

2. Log in from your terminal:

   npm login

   Enter your npm username, password, and email when prompted.

   2FA / Granular Access Token: If your npm account has 2FA enabled, you have two options:

   • Pass your OTP at publish time: ./publish.sh --otp=123456
   • Or create a Granular Access Token on npmjs.com with "Read and write" + bypass 2FA, then set it:

     npm set //registry.npmjs.org/:_authToken YOUR_TOKEN

Publishing an update (using publish.sh)

A publish.sh script is included that handles build + version bump + publish in one step:

# Make executable (one-time)
chmod +x publish.sh

# Patch release — bug fixes (0.0.5 → 0.0.6)
./publish.sh

# Minor release — new features, backwards compatible (0.0.5 → 0.1.0)
./publish.sh minor

# Major release — breaking changes (0.0.5 → 1.0.0)
./publish.sh major

# Publish current version as-is (no version bump)
./publish.sh --no-bump

# With 2FA OTP (if your npm account requires it)
./publish.sh minor --otp=123456

The script will:

1. Verify you are logged in to npm
2. Run npm run build (TypeScript → dist/)
3. Bump the version in package.json
4. Run npm publish
5. Print the package URL and next steps for users

Manual publish (without the script)

# 1. Make your code changes in src/

# 2. Build the compiled output
npm run build

# 3. Bump the version (choose one):
npm version patch   # 0.0.5 → 0.0.6  (bug fixes)
npm version minor   # 0.0.5 → 0.1.0  (new features, backwards compatible)
npm version major   # 0.0.5 → 1.0.0  (breaking changes)

# 4. Publish to npm
npm publish
# With 2FA OTP:
npm publish --otp=123456

npm version automatically updates package.json. npm publish uploads only the dist/ folder (as defined in "files" in package.json).

Verify the publish succeeded

npm view bob-pr-reviewer version
# Should show the new version number

Or check: https://www.npmjs.com/package/bob-pr-reviewer

After publishing — users get the update automatically

Users running npx -y bob-pr-reviewer will get the new version automatically on next VS Code launch. If they want to force a refresh of a cached version:

npx clear-npx-cache
# Then: Cmd+Shift+P → "Developer: Reload Window"

Troubleshooting

Server doesn't appear in VS Code

1. Check the MCP output panel: View → Output → MCP
2. Make sure you added the config to User Settings (JSON) (global), not workspace settings
3. Reload VS Code: Cmd+Shift+P → "Developer: Reload Window"
4. Verify npx bob-pr-reviewer works in your terminal

All MCP logs show as "errors" in VS Code

This is normal and expected. VS Code's MCP output panel labels all stderr output as errors regardless of content. MCP servers must use stderr for logging (stdout is reserved for the JSON-RPC protocol). If you see ✅ All checks passed, the server is working correctly.

IBM Bob doesn't show slash commands or @bob-pr-reviewer autocomplete

This is a known IBM Bob limitation — Bob does not yet surface MCP prompts as / slash commands or @server-name mentions in its chat UI. The server-side implementation is complete and correct. Use the explicit invocation pattern instead:

Use the bob-pr-reviewer MCP server to review PR #1758 in sec-ci/notification

"GITHUB_TOKEN is not set"

The token isn't reaching the server. Make sure you:

1. Added export GITHUB_TOKEN=... to ~/.zshrc
2. Ran source ~/.zshrc
3. Fully restarted VS Code (quit and reopen — not just reload window)

"GitHub API error 401: Bad credentials"

Your token is invalid or expired. Generate a new one at https://github.ibm.com/settings/tokens with repo scope.

"Cannot reach github.ibm.com"

You're not on IBM VPN. Connect to VPN and reload VS Code.

"No Jira ticket key found in PR"

The PR title/branch/description doesn't contain a recognizable Jira key pattern (e.g. ABC-1234). Provide it explicitly:

Use the bob-pr-reviewer MCP server to check PR #1758 in sec-ci/notification against Jira ticket VDEV-47721

Inline comments fail with position errors

The position field in post_review refers to the line position in the unified diff (1-based), not the actual file line number. The AI calculates these from the diff returned by fetch_pr_diff. If positions are wrong, ask the AI to re-fetch the diff and recount.

Degraded mode — only check_setup available

Ask the AI:

Use the bob-pr-reviewer MCP server to check setup

It will show exactly which check failed and how to fix it.

Security Notes

• Never commit your .env file — it's in .gitignore
• Tokens are passed via environment variables and never sent to the AI model
• The MCP server only has access to repos your GitHub token can access
• The server runs locally — no data leaves your machine except to github.ibm.com and jsw.ibm.com

License

ISC