@codacy/tools-opengrep-1

Table of Contents

• Overview
• Rule bundling
• Updating patterns
• Updating rules
• Updating the Opengrep version
• Development
• Notes for maintainers

Overview

Multi-language SAST (Static Application Security Testing) scanner using the Opengrep binary. Supports 30+ languages via YAML rule files. Uses the CLI execution strategy.

| Property | Value | | ---------------- | ------------------------------------------------------------------ | | Tool ID | Semgrep (legacy, for Codacy platform compatibility) | | Display name | Opengrep | | Codacy UUID | 6792c561-236d-41b7-ba5e-9d6bee0d548b | | Strategy | CLI | | Languages | 30+ (Python, JavaScript, TypeScript, Java, Go, Ruby, C, C++, etc.) | | Binary | opengrep | | File patterns | * (auto-detects by language) | | Pattern count | ~2500 | | Extra dependency | yaml (for YAML rule config parsing/generation) |

Rule bundling

Opengrep uses YAML rule files for detection. The adapter bundles ~2200 rules into src/rules.yaml, sourced from:

1. semgrep/semgrep-rules — community rules (pinned at commit 4ccd3b9)
2. GitLab SAST rules — MIT-licensed rules from gitlab.com/gitlab-org/security-products/sast-rules
3. Codacy custom rules — from codacy/codacy-opengrep wrapper repo

The bundled src/rules.yaml is committed to the repository for consistency and reproducibility.

Updating patterns

# Re-fetch pattern metadata from the Codacy API
pnpm prefetch

# Commit the result
git add src/patterns.json

Pattern IDs follow the format Semgrep_python.lang.security.audit.exec-detected.exec-detected (dot-separated rule paths).

Updating rules

# Re-fetch and bundle rules from all three sources
pnpm fetch-rules

# This clones the rule repos, extracts matching rules, and writes src/rules.yaml
# Only rules whose IDs match src/patterns.json are included

# Commit the result
git add src/rules.yaml

The fetch-rules script (scripts/fetch-rules.mjs):

• Clones semgrep/semgrep-rules at the pinned commit
• Clones GitLab SAST rules (latest)
• Clones codacy/codacy-opengrep for custom rules
• Extracts rules, assigns path-based IDs
• Filters to only rules matching src/patterns.json
• Writes concatenated YAML to src/rules.yaml

To update the pinned semgrep-rules commit, edit SEMGREP_RULES_COMMIT in scripts/fetch-rules.mjs.

Updating the Opengrep version

1. Update preferredVersion in src/adapter.ts
2. Run pnpm test to verify compatibility
3. Consider re-running pnpm fetch-rules if the new version supports new rule syntax

Development

pnpm build        # Build with tsup
pnpm test         # Run tests (requires opengrep in PATH)
pnpm fetch-rules  # Re-bundle rules from upstream sources
pnpm prefetch     # Re-fetch pattern metadata from Codacy API

Installing the binary:

The adapter auto-installs opengrep into ~/.codacy/tools/ when run with --install-dependencies (no manual step needed):

codacy-analysis analyze . --tool opengrep --install-dependencies

install() downloads the matching statically-linked binary from GitHub releases (no archive to unpack) and chmod +xes it. The platform/arch/libc → asset mapping mirrors opengrep's official install.sh:

| Platform | Asset | | ------------------------- | ---------------------------- | | macOS x64 | opengrep_osx_x86 | | macOS arm64 | opengrep_osx_arm64 | | Linux x64 (glibc) | opengrep_manylinux_x86 | | Linux arm64 (glibc) | opengrep_manylinux_aarch64 | | Linux x64 (musl/Alpine) | opengrep_musllinux_x86 | | Linux arm64 (musl/Alpine) | opengrep_musllinux_aarch64 |

To install manually instead:

curl -fsSL https://raw.githubusercontent.com/opengrep/opengrep/main/install.sh | bash

Notes for maintainers

• Preset mappings (src/pattern-mappings.json): prefixes are the upstream rule paths (Semgrep_<lang>.<framework>.…), which already name the framework, so this mapping is prefix-only — Opengrep's tags are topical (security, audit) and would over-match. 22 keys were added, taking ~91 default-on patterns out of repos that lack the framework: Pyramid (16), PyCryptodome (12), Kubernetes/Helm (14, via yaml.kubernetes. + codacy.k8s.; the kubernetes signal matches loose manifests under k8s//kubernetes//manifests/, not just kustomization.yaml, or the most common layout would have lost all 14), GitHub Actions (5), AWS CDK (5), Jinja2 (2), and the smaller Boto3/Twilio/Android/Sequelize/Doctrine families. The YAML-language families matter most: before this they fired in any repo containing a single .yaml file.

  The *.aws-lambda. packs (34 default-on across Python, JavaScript, Ruby and Go) are claimed by seven keys — serverless-framework, cloudformation, aws-cdk, boto3, aws-sdk, aws-sdk-go and lang:Terraform. Handler code carries no manifest of its own; the only evidence is how it is deployed or which SDK it calls, and any one of the seven is enough. Terraform earns a key because deploying a Lambda with aws_lambda_function leaves no other trace, and the SDK telltales had to learn npm's v2 aws-sdk, Go's v1 aws-sdk-go and PyPI's botocore/aws-lambda-powertools — Lambda runtimes preinstall boto3, so it is often absent from requirements.txt entirely.

  Semgrep_python.jinja2. is claimed by jinja2 and flask: Flask hard-depends on Jinja2, so gating on a direct jinja2 declaration alone dropped both Error-severity autoescape rules from most real Flask repos.

  Deliberately still universal, guarded by a test so it stays that way: *.lang.* (the generic per-language packs), csharp.dotnet. (the platform, not a library), terraform.* (already gated by the Terraform language), problem-based-packs.* (cross-language), and apex. / generic.visualforce. — Apex and VisualForce are Salesforce-only languages, so the language gate already scopes them and keying them on salesforce would only add a way to miss them in a Salesforce repo with no sfdx-project.json.

• Tool ID is Semgrep for legacy compatibility with the Codacy platform. The displayName is set to Opengrep for CLI display. Never change the tool ID without coordinating with the Codacy platform team.

• Config materialization: When Codacy patterns are specified, the adapter loads src/rules.yaml, filters to enabled rules, and writes a filtered config to .codacy/generated/opengrep-rules.yaml.

• check_id prefix stripping: Opengrep prepends a path-based namespace to check_ids derived from the config file path relative to cwd. The adapter computes and strips this prefix (e.g. codacy.generated.) plus any tmp. legacy prefix.

• Config file search: .semgrep.yaml, .opengrep.yaml, .semgrep.yml, .opengrep.yml (in that order).

• Opengrep errors: Parse errors (syntax errors, partial parsing) from Opengrep's JSON errors array are converted to structured AnalysisError objects with filePath. Stderr (scan status table) is logged at debug level only.

• Rule ID format: Semgrep-rules use {dir}.{filename}.{ruleId} (dots), GitLab rules use {dir}_{filename} (underscores), Codacy custom rules keep their original IDs.

• Install / binary download: install() downloads the bare per-platform binary from https://github.com/opengrep/opengrep/releases/download/v{version}/{asset} and chmod +xes it into ~/.codacy/tools/. The release tag carries a v prefix (v1.16.4) even though preferredVersion does not. On Linux the libc flavour is detected with an ldd /bin/sh probe (musl → musllinux, else manylinux); detection failures fall back to glibc/manylinux. Unlike install.sh, the Codacy-managed install does not verify the cosign .cert/.sig signatures — if signature verification becomes a requirement, fetch those sibling assets and validate before chmod.

• Argv-size chunking (no directory walk): The adapter always passes Opengrep an explicit file list — the runner's already git-ignored, exclude-filtered target set — so Opengrep never walks the repository tree itself. chunkFilesByArgvBudget() splits that list into the fewest, largest chunks that each fit ARGV_BYTE_BUDGET (200 KB, well under ARG_MAX), and each chunk is scanned in its own opengrep scan invocation; findings and errors are merged across chunks. Chunk by size, not count: every invocation re-compiles rules.yaml, so fewer/larger chunks amortize that fixed cost. The earlier directory/“root” fallback was removed because on huge repos (a single tracked file at the repo root collapsed the minimal-cover dirs to .) it made Opengrep walk the entire tree — re-scanning excluded/generated/vendored files and timing out at the runner's 10-minute limit.

• File-size cap: invocations pass --max-target-bytes 1000000 (~1 MB). This restores Opengrep's default size cap — the adapter previously passed 0, which disables the cap, so giant generated/minified bundles were scanned and dominated runtime. The cap applies to explicitly-listed files (verified on v1.16.4). Note: --exclude-minified-files is not used — that flag is rejected by the bundled binary unless --experimental (a different engine) is also passed, so we rely on the size cap instead. (The runner also enforces a tighter 150 KB per-file limit centrally before files reach the adapter, so this --max-target-bytes is now a secondary defense.)

• Partial results on timeout: the per-chunk loop checks ctx.signal (the runner's timeout abort) between chunks and passes it to spawnTool, so an in-flight scan is killed promptly. On abort the adapter keeps the findings from completed chunks and adds a single Timeout error (→ status: "partial") rather than losing everything. This is why chunking matters beyond ARG_MAX: it's the granularity at which a long scan can be stopped with results preserved.