----------------
🛠️ Tool: garak - LLM Vulnerability Scanner
===================
garak (Generative AI Red-teaming & Assessment Kit) is an open-source tool developed under NVIDIA's GitHub organization that systematically probes LLMs for security weaknesses. If you know nmap or Metasploit Framework, garak operates on a similar concept but targets language models instead of network services or software vulnerabilities.
What garak does
The tool probes LLMs and dialog systems for failure modes that security teams care about: hallucination, data leakage, prompt injection, misinformation generation, toxicity output, and jailbreaks. It combines three probing strategies.
Static probes use fixed test cases. Dynamic probes generate test cases based on model responses. Adaptive probes adjust their strategy based on intermediate results, which is potentially more effective at uncovering weaknesses that fixed test suites miss because the probing strategy evolves as it learns about the model's behavior.
Supported backends
• Hugging Face Hub generative models
• Replicate text models
• OpenAI API (chat and continuation models)
• AWS Bedrock foundation models
• LiteLLM
• REST-accessible endpoints
• GGUF models via llama.cpp (version >= 1046)
This range means you can run the same probe suite across different providers and compare results directly. That comparative angle is useful for organizations evaluating which model to deploy.
Installation
Standard install via pip: python -m pip install -U garak
Development version from GitHub: python -m pip install -U git+https://github.com/NVIDIA/garak.git@main
Recommended Conda environment setup with Python >=3.10, <=3.12. The tool runs as a command-line utility with the general syntax garak .
Technical context
The project has active CI pipelines for Linux, Windows, and macOS. Code formatting follows Black. An arXiv paper (2406.11036) documents the methodology. DEF CON presentation slides are available. The Discord community is active for discussion.
Practical considerations
The tool is free under Apache 2.0. It focuses on making LLMs fail in ways we don't want, which is a different posture than typical benchmarking. The adaptive probe mechanism is conceptually interesting. I'm not sure how it performs in practice against commercially deployed models with layered safety filters.
Haven't tested personally, so can't speak to performance at scale or coverage completeness against specific model families. Documentation at docs.garak.ai.
🔹 garak #LLM #red_teaming #NVIDIA #tool
🔗 Source: https://github.com/NVIDIA/garak