Audit Report
drhus/ai-auditor
8d681f7e9bee · ran in 35.5s · bundle fb69bd0f
Overall score
2.4 /4
Partial
Risk class
HIGH
1
Code passed
22 / 45
Attestation Yes
0
Outstanding ext.
1
CODE-CHECKED CLAUSES
- Strong15
- Adequate7
- Partial10
- Inadequate5
- Absent8
ATTESTATION QUESTIONS
- Yes0
- No0
- Not Applicable0
- Outstanding0
Harm
Don't hurt people
2.2/4
Partial
Truth
Don't deceive people
2.1/4
Partial
Responsibility
Don't abuse power
2.5/4
Partial
Order
Don't destabilize society
3.0/4
Adequate
⚠ 1 outstanding external confirmations — required for a complete Annex IV dossier. Complete now →
Safety
Don't harm people2.7/4Adequate6 clauses▸
Safety
Don't harm peoplePARTIALRisk management system established, implemented, documented EU AI Act, Art 9 LLM2/3 rules2/4
▸
2/4
Why we flagged it
Composite raw score 0.55 (2/3 rules matched). Supporting docs may exist outside the repo. LLM judge (confidence 0.62): Risk identification and analysis foundations are evident (risk register and threat model), but absence of documented continuous integration evaluation gates suggests incomplete implementation of the 'continuous iterative process' and 'regular systematic review and updating' requirements across the system lifecycle.
▸Evidence · 2 hits— click to view code
Suggested fix · we looked for these and found none
- ci_eval_gates
INADEQUATEResilience to errors, faults, inconsistencies EU AI Act, Art 15(4)0/3 rules1/4
▸
1/4
Why we flagged it
Composite raw score 0.16 (0/3 rules matched).
▸Evidence · 2 hits— click to view code
nippet": "import os from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler from langchain_openai import ChatOpenAI"
dOutCallbackHandler from langchain_openai import ChatOpenAI", "rule": "langchain_import" }, { "fSuggested fix · we looked for these and found none
- error_handling_at_tool_boundaries
- retry_logic
- fallback_behaviour
ADEQUATERisks and benefits to people identified NIST AI RMF, Art MAP 3.42/2 rules3/4
▸
3/4
Why we flagged it
Composite raw score 0.80 (2/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 2 hits— click to view code
STRONGData protection impact assessment (DPIA) GDPR, Art 351/1 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (1/1 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 1 hit— click to view code
STRONGAI risk assessment process ISO/IEC 42001, Art 6.12/2 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (2/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 2 hits— click to view code
PARTIALOperational planning and control ISO/IEC 42001, Art 8.1 skip1/2 rules2/4
▸
2/4
Why we flagged it
Composite raw score 0.50 (1/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 1 hit— click to view code
Suggested fix · we looked for these and found none
- presence_of_ci_workflows
Privacy
Respect boundaries1.6/4Partial6 clauses▸
Privacy
Respect boundariesABSENTUntargeted facial image scraping for face databases EU AI Act, Art 5(1)(e)1/1 rules0/4
▸
0/4
Why we flagged it
Composite raw score 1.00 (1/1 rules matched).
▸Evidence · 6 hits— click to view code
" description: "Biometric ID, categorisation, emotion recognition" - signal: critical_infra_signals category: "2" description:
otion recognition / biometric categorisation" - signal: agent_framework # any interactive AI is in scope paragraph: "50(1)"
calls (mediapipe, face_recognition, dlib, opencv haar cascade). score_mapping: { pass_default: 4, fail_on_match: 0 } remediation_h"^1.1.0", "@playwright/test": "^1.51.1", "babel-plugin-react-compiler": "*", "react": "^18.2.0 || 19.0.0-rc-de68d2f4
}, "@playwright/test": { "optional": true }, "babel-plugin-react-compiler": { "optional":…and 1 more.
ABSENTEmotion recognition in workplace and education EU AI Act, Art 5(1)(f)1/1 rules0/4
▸
0/4
Why we flagged it
Composite raw score 1.00 (1/1 rules matched).
▸Evidence · 6 hits— click to view code
ms", pattern: /\b(?:emotion_detect|emotion_recognition|sentiment_score|affect_recognition|facial_emotion|micro_expression)\b/gi }, // ---
\b(?:emotion_detect|emotion_recognition|sentiment_score|affect_recognition|facial_emotion|micro_expression)\b/gi }, // ----- data_io ----
emotion_recognition|sentiment_score|affect_recognition|facial_emotion|micro_expression)\b/gi }, // ----- data_io ----- { signal: "data_ppet": "OMMANDS for candidate in tokens[index + 1:]: if candidate in {\"|\", \";\", \"&&\", \"||\"}: break f", "rule": "employmeens[index + 1:]: if candidate in {\"|\", \";\", \"&&\", \"||\"}: break f", "rule": "employment_terms" }, {…and 1 more.
EXTERNALReal-time remote biometric identification in public spaces EU AI Act, Art 5(1)(h)EXT▸
Why we flagged it
Deployment context (public space, real-time, law enforcement use, judicial authorisation) is operational, not knowable from code. Always external.
ABSENTData and data governance practices documented EU AI Act, Art 100/3 rules0/4
▸
0/4
Why we flagged it
Composite raw score 0.10 (0/3 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 2 hits— click to view code
"^1.1.0", "@playwright/test": "^1.51.1", "babel-plugin-react-compiler": "*", "react": "^18.2.0 || 19.0.0-rc-de68d2f4
}, "@playwright/test": { "optional": true }, "babel-plugin-react-compiler": { "optional":Suggested fix · we looked for these and found none
- presence_of_data_card
- data_loading_code_quality
- bias_evaluation_present
STRONGPrivacy risk of the AI system evaluated NIST AI RMF, Art MEASURE 2.82/2 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (2/2 rules matched).
▸Evidence · 7 hits— click to view code
description: "Code redacts or hashes PII before logging or sending to external models" - rule: privacy_documentation weig
p \"$SCRIPT_DIR/mcp/redaction.js\" \"$TARGET_ABS/mcp/\" cp \"$SCRIPT_DIR/mcp/lib/\"*.js \"$TARGET_ABS/mcp/lib/\" rm -rf \"$TARGET_ABS/mcp/li
t the proof needed, redact sensitive data, and report responsibly.</p></div> </div> </div> <div class=\"foot\"><span", "rule": "
h MCP, which writes redacted audit metadata and egress information.</p> </div> <div class=\"code-card\"> <div", "rule": "pii_red
n>Audited requests, redacted URLs, visible egress</span></div> </section> <section class=\"slide\" data-title=\"Egress\"> <", "r
…and 2 more.
STRONGData protection by design and by default GDPR, Art 252/2 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (2/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 6 hits— click to view code
crypto.createHash("sha256")crypto.createHash("sha256")redact
crypto.createHash("sha256")…and 1 more.
Transparency
Don't deceive people2.3/4Partial7 clauses▸
Transparency
Don't deceive peopleSTRONGSubliminal techniques distorting behaviour EU AI Act, Art 5(1)(a)0/1 rules4/4
▸
4/4
Why we flagged it
Composite raw score 0.00 (0/1 rules matched).
Suggested fix · we looked for these and found none
- detect_manipulative_prompt_patterns
INADEQUATETransparent operation and instructions for use EU AI Act, Art 131/3 rules1/4
▸
1/4
Why we flagged it
Composite raw score 0.25 (1/3 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 3 hits— click to view code
2 required sections present
Suggested fix · we looked for these and found none
- output_interpretation_guidance
- limitations_section_present
INADEQUATEUsers informed they are interacting with an AI EU AI Act, Art 50(1)1/2 rules1/4
▸
1/4
Why we flagged it
Composite raw score 0.20 (1/2 rules matched).
▸Evidence · 1 hit— click to view code
Suggested fix · we looked for these and found none
- ai_disclosure_in_user_facing_strings
ADEQUATEAI-generated content marked as such, machine-readable EU AI Act, Art 50(2)2/2 rules3/4
▸
3/4
Why we flagged it
Composite raw score 0.82 (2/2 rules matched).
▸Evidence · 8 hits— click to view code
ent provenance | no c2pa imports → ABSENT | "API response contains `aiGenerated:true` / C2PA header" | | GDPR Art 5(1)(c) — PII in logs | gr
aiGenerated:true` / C2PA header" | | GDPR Art 5(1)(c) — PII in logs | grep `user.email` near `logger.info` | "fake PII sent → grep all captu
` | 50(2) | **1** | C2PA / watermark library imports | | `art-50/p3-emotion-biometric-disclosure` | 50(3) | **1** | Deterministic disclosure
ovenance_hooks` | C2PA, watermarking, content labelling | Article 50(2) synthetic content disclosure
o/text) | C | C2PA / watermarking libraries; metadata writers; output post-processing. | | 50(3) | Emotion-recognition / biometri
…and 3 more.
ABSENTEmotion recognition / biometric categorisation disclosure EU AI Act, Art 50(3)0/1 rules0/4
▸
0/4
Why we flagged it
Composite raw score 0.00 (0/1 rules matched).
▸Evidence · 1 hit— click to view code
Suggested fix · we looked for these and found none
- emotion_or_biometric_disclosure_string
ADEQUATEDeepfake content labelled as artificially generated EU AI Act, Art 50(4) skip1/1 rules3/4
▸
3/4
Why we flagged it
Composite raw score 0.70 (1/1 rules matched).
▸Evidence · 2 hits— click to view code
ent provenance | no c2pa imports → ABSENT | "API response contains `aiGenerated:true` / C2PA header" | | GDPR Art 5(1)(c) — PII in logs | gr
aiGenerated:true` / C2PA header" | | GDPR Art 5(1)(c) — PII in logs | grep `user.email` near `logger.info` | "fake PII sent → grep all captu
STRONGPrinciples relating to processing of personal data GDPR, Art 52/2 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (2/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 3 hits— click to view code
Auditability
Actions must be traceable1.9/4Partial8 clauses▸
Auditability
Actions must be traceableINADEQUATETechnical documentation drawn up before placing on market EU AI Act, Art 111/3 rules1/4
▸
1/4
Why we flagged it
Composite raw score 0.15 (1/3 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 1 hit— click to view code
2 required sections present
Suggested fix · we looked for these and found none
- presence_of_model_card
- architecture_docs
ABSENTAutomatic recording of events over the lifetime EU AI Act, Art 12(1) LLM1/3 rules0/4
▸
0/4
Why we flagged it
Composite raw score 0.58 (1/3 rules matched). LLM judge (confidence 0.85): While tool-call boundary logging is detected via console.log statements, the clause requires 'technical' automatic recording with persistence over the system's lifetime. Console logs are ephemeral and lack evidence of structured logging imports or persistent sink configuration, failing to meet the durability and systematic requirements for high-risk AI system compliance.
▸Evidence · 8 hits— click to view code
x.ts"), content); console.log(` rewrote index.ts (${entries.length} seeds)`); } async function main() { const args = process.argv.slic-1.0"], }; console.log(`\n=== ${owner}/${repo} (id=${input.auditId}) ===`); const t0 = Date.now(); try { const reportevt.kind === "log") console.log(` [${evt.stage}] ${evt.text}`); else if (evt.kind === "stage") console.log(` [${evt.stage}] phase=t.kind === "stage") console.log(` [${evt.stage}] phase=${evt.phase}${evt.durationMs ? ` (${evt.durationMs}ms)` : ""}`); else if (ev= "classification") console.log(` ✦ risk=${evt.classification} annex=${evt.annexIii.join("/")} art50=${evt.art50.join("/")}`); else…and 3 more.
Suggested fix · we looked for these and found none
- structured_logging_imported
- logging_persistent_sink
PARTIALLogging ensures traceability appropriate to risk EU AI Act, Art 12(2) LLM2/3 rules2/4
▸
2/4
Why we flagged it
Composite raw score 0.68 (2/3 rules matched). Supporting docs may exist outside the repo. LLM judge (confidence 0.62): The system demonstrates logging of input/output pairs and model identity (2/3 deterministic rules matched, 0.68 score), satisfying core traceability needs. However, absence of request_id logging and lack of evidence for production-grade structured logging (only console.log statements visible) create uncertainty about whether traceability is 'appropriate to intended purpose' under operational conditions, particularly for high-risk AI use cases.
▸Evidence · 8 hits— click to view code
x.ts"), content); console.log(` rewrote index.ts (${entries.length} seeds)`); } async function main() { const args = process.argv.slic-1.0"], }; console.log(`\n=== ${owner}/${repo} (id=${input.auditId}) ===`); const t0 = Date.now(); try { const reportevt.kind === "log") console.log(` [${evt.stage}] ${evt.text}`); else if (evt.kind === "stage") console.log(` [${evt.stage}] phase=t.kind === "stage") console.log(` [${evt.stage}] phase=${evt.phase}${evt.durationMs ? ` (${evt.durationMs}ms)` : ""}`); else if (ev= "classification") console.log(` ✦ risk=${evt.classification} annex=${evt.annexIii.join("/")} art50=${evt.art50.join("/")}`); else…and 3 more.
Suggested fix · we looked for these and found none
- logs_include_request_id
PARTIALContext of use established and understood NIST AI RMF, Art MAP 1.1 LLM1/2 rules2/4
▸
2/4
Why we flagged it
Composite raw score 0.30 (1/2 rules matched). Supporting docs may exist outside the repo. LLM judge (confidence 0.65): Intended use is documented (README.md evidence supports this component), but deployment context documentation is not detected in the repository scan. The clause requires both elements to be understood and documented; partial fulfillment of the core requirement warrants further review of external documentation or system specifications.
▸Evidence · 2 hits— click to view code
2 required sections present
Suggested fix · we looked for these and found none
- deployment_context_documented
PARTIALPost-deployment monitoring, appeal and override, change management NIST AI RMF, Art MANAGE 4.1 skip2/4 rules2/4
▸
2/4
Why we flagged it
Composite raw score 0.50 (2/4 rules matched).
▸Evidence · 7 hits— click to view code
in-the-loop hooks: `human_input()`, interrupt nodes in LangGraph, approval-gate functions, manual- review flags, con
For LangGraph: use `interrupt()` nodes. For custom flows: build an approval-queue pattern. Document where humans can intervene i
ic: - rule: kill_switch_present weight: 0.7 description: | Code contains a documented kill-switch /
sm (function named `kill_switch`, `emergency_stop`, `disable_agent`, a feature flag with explicit disable, an admin
med `kill_switch`, `emergency_stop`, `disable_agent`, a feature flag with explicit disable, an admin endpoint that h
…and 2 more.
Suggested fix · we looked for these and found none
- feedback_capture_present
- structured_logging_imported
STRONGRecords of processing activities GDPR, Art 301/1 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (1/1 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 1 hit— click to view code
ADEQUATEDocumented information for the AI management system ISO/IEC 42001, Art 7.5 skip1/2 rules3/4
▸
3/4
Why we flagged it
Composite raw score 0.70 (1/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 2 hits— click to view code
Suggested fix · we looked for these and found none
- presence_of_versioned_docs
INADEQUATEMonitoring, measurement, analysis and evaluation ISO/IEC 42001, Art 9.11/2 rules1/4
▸
1/4
Why we flagged it
Composite raw score 0.25 (1/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 2 hits— click to view code
x.ts"), content); console.log(` rewrote index.ts (${entries.length} seeds)`); } async function main() { const args = process.argv.slic-1.0"], }; console.log(`\n=== ${owner}/${repo} (id=${input.auditId}) ===`); const t0 = Date.now(); try { const reportSuggested fix · we looked for these and found none
- presence_of_eval_suite
Accountability
Don't abuse power2.5/4Adequate6 clauses▸
Accountability
Don't abuse powerABSENTDeployer log-retention capability supported EU AI Act, Art 26(6) LLM1/1 rules0/4
▸
0/4
Why we flagged it
Composite raw score 0.50 (1/1 rules matched). Supporting docs may exist outside the repo. LLM judge (confidence 0.85): Evidence shows only console.log statements at tool boundaries, not systematic automatic log retention meeting the six-month minimum storage requirement. No evidence of persistent log storage infrastructure, retention policies, or access controls required by Article 26(6).
▸Evidence · 2 hits— click to view code
x.ts"), content); console.log(` rewrote index.ts (${entries.length} seeds)`); } async function main() { const args = process.argv.slic-1.0"], }; console.log(`\n=== ${owner}/${repo} (id=${input.auditId}) ===`); const t0 = Date.now(); try { const reportSTRONGRisk management process documented and accountable NIST AI RMF, Art GOVERN 1.42/2 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (2/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 2 hits— click to view code
ABSENTOngoing monitoring and periodic review of risk management NIST AI RMF, Art GOVERN 1.50/2 rules0/4
▸
0/4
Why we flagged it
Composite raw score 0.00 (0/2 rules matched). Supporting docs may exist outside the repo.
Suggested fix · we looked for these and found none
- ci_eval_gates
- drift_monitoring_present
ADEQUATELeadership and commitment for AI management ISO/IEC 42001, Art 5.1 skip1/2 rules3/4
▸
3/4
Why we flagged it
Composite raw score 0.70 (1/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 1 hit— click to view code
Suggested fix · we looked for these and found none
- leadership_signoff_evidence
STRONGRoles, responsibilities and authorities ISO/IEC 42001, Art 5.31/1 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (1/1 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 1 hit— click to view code
STRONGInternal organization controls ISO/IEC 42001, Art A.51/1 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (1/1 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 1 hit— click to view code
Human Oversight
Humans stay in control2.4/4Partial5 clauses▸
Human Oversight
Humans stay in controlPARTIALEffective human oversight designed and built-in EU AI Act, Art 14(1) LLM1/3 rules2/4
▸
2/4
Why we flagged it
Composite raw score 0.50 (1/3 rules matched). LLM judge (confidence 0.62): Human-in-loop mechanisms are present (LangGraph interrupt nodes, approval gates, manual review flags) satisfying the core requirement for intervention points. However, absence of documented oversight UI and lack of dry-run capabilities for tool calls create gaps in effective real-time oversight usability and safety verification, leaving implementation incomplete against the clause's full intent.
▸Evidence · 6 hits— click to view code
in-the-loop hooks: `human_input()`, interrupt nodes in LangGraph, approval-gate functions, manual- review flags, con
For LangGraph: use `interrupt()` nodes. For custom flows: build an approval-queue pattern. Document where humans can intervene i
import ( AIMessage, HumanMessage,", "rule": "langchain_import" }, { "file": "gpt_engineer/core/aimport ( AIMessage, HumanMessage, SystemMessage, messages_from_dict, messages_", "rule": "langchain_import" },
chain.schema import HumanMessage, SystemMessage from termcolor import colored from gpt_engineer.core.ai import", "rule": "langch
…and 1 more.
Suggested fix · we looked for these and found none
- oversight_ui_present
- tool_calls_have_dry_run
PARTIALInterrupt / stop function reachable by overseer EU AI Act, Art 14(4)(d) LLM1/2 rules2/4
▸
2/4
Why we flagged it
Composite raw score 0.70 (1/2 rules matched). LLM judge (confidence 0.72): The system demonstrates a documented kill-switch mechanism (kill_switch_active=True) satisfying the 'stop button' requirement with 0.7 weight, but lacks evidence of graceful shutdown handling (0.3 weight) needed to ensure the system reaches a 'safe state' as required by the clause. Additional documentation of safe shutdown procedures is needed for full compliance.
▸Evidence · 6 hits— click to view code
ic: - rule: kill_switch_present weight: 0.7 description: | Code contains a documented kill-switch /
sm (function named `kill_switch`, `emergency_stop`, `disable_agent`, a feature flag with explicit disable, an admin
med `kill_switch`, `emergency_stop`, `disable_agent`, a feature flag with explicit disable, an admin endpoint that h
stop`, `disable_agent`, a feature flag with explicit disable, an admin endpoint that halts processing). - ru
ic: - rule: kill_switch_present weight: 0.6 - rule: feature_flag_for_disable weight: 0.4 descr
…and 1 more.
Suggested fix · we looked for these and found none
- graceful_shutdown_handler
PARTIALAbility to override / reverse the system's output EU AI Act, Art 14(4)(e) LLM1/2 rules2/4
▸
2/4
Why we flagged it
Composite raw score 0.60 (1/2 rules matched). LLM judge (confidence 0.72): The system demonstrates override capability through documented human-in-loop mechanisms (interrupt nodes, approval gates, kill-switch functions), satisfying the override_path requirement. However, evidence does not clearly establish that all AI decisions are addressable for reversal or that override authority is formally assigned to designated natural persons, leaving the proportionality and assignment aspects of the clause unverified.
▸Evidence · 6 hits— click to view code
in-the-loop hooks: `human_input()`, interrupt nodes in LangGraph, approval-gate functions, manual- review flags, con
For LangGraph: use `interrupt()` nodes. For custom flows: build an approval-queue pattern. Document where humans can intervene i
ic: - rule: kill_switch_present weight: 0.7 description: | Code contains a documented kill-switch /
sm (function named `kill_switch`, `emergency_stop`, `disable_agent`, a feature flag with explicit disable, an admin
med `kill_switch`, `emergency_stop`, `disable_agent`, a feature flag with explicit disable, an admin endpoint that h
…and 1 more.
Suggested fix · we looked for these and found none
- decisions_are_addressable
PARTIALMechanisms to supersede or deactivate AI systems NIST AI RMF, Art MANAGE 2.3 skip1/2 rules2/4
▸
2/4
Why we flagged it
Composite raw score 0.60 (1/2 rules matched).
▸Evidence · 6 hits— click to view code
ic: - rule: kill_switch_present weight: 0.7 description: | Code contains a documented kill-switch /
sm (function named `kill_switch`, `emergency_stop`, `disable_agent`, a feature flag with explicit disable, an admin
med `kill_switch`, `emergency_stop`, `disable_agent`, a feature flag with explicit disable, an admin endpoint that h
stop`, `disable_agent`, a feature flag with explicit disable, an admin endpoint that halts processing). - ru
ic: - rule: kill_switch_present weight: 0.6 - rule: feature_flag_for_disable weight: 0.4 descr
…and 1 more.
Suggested fix · we looked for these and found none
- feature_flag_for_disable
STRONGAutomated individual decision-making, including profiling GDPR, Art 222/2 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (2/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 12 hits— click to view code
in-the-loop hooks: `human_input()`, interrupt nodes in LangGraph, approval-gate functions, manual- review flags, con
For LangGraph: use `interrupt()` nodes. For custom flows: build an approval-queue pattern. Document where humans can intervene i
import ( AIMessage, HumanMessage,", "rule": "langchain_import" }, { "file": "gpt_engineer/core/aimport ( AIMessage, HumanMessage, SystemMessage, messages_from_dict, messages_", "rule": "langchain_import" },
chain.schema import HumanMessage, SystemMessage from termcolor import colored from gpt_engineer.core.ai import", "rule": "langch
…and 7 more.
Fairness
Treat people fairly3.0/4Adequate4 clauses▸
Fairness
Treat people fairlySTRONGExploiting vulnerabilities (age, disability, socio-economic) EU AI Act, Art 5(1)(b)0/1 rules4/4
▸
4/4
Why we flagged it
Composite raw score 0.00 (0/1 rules matched).
Suggested fix · we looked for these and found none
- detect_protected_attribute_targeting
STRONGSocial scoring leading to detrimental treatment EU AI Act, Art 5(1)(c)0/1 rules4/4
▸
4/4
Why we flagged it
Composite raw score 0.00 (0/1 rules matched). Supporting docs may exist outside the repo.
Suggested fix · we looked for these and found none
- detect_scoring_with_persistent_user_state
ABSENTPredictive policing solely from profiling EU AI Act, Art 5(1)(d)1/1 rules0/4
▸
0/4
Why we flagged it
Composite raw score 0.76 (1/1 rules matched).
▸Evidence · 6 hits— click to view code
crime-likelihood / recidivism / "risk to commit X" scores when the input contains only person profile data (no obje
names like `crime_risk`, `recidivism_score`, `offender_likelihood` combined with profile inputs. score_mapping: { parecidivism_score`, `offender_likelihood` combined with profile inputs. score_mapping: { pass_default: 4, fail_on_match: 0 }ms", pattern: /\b(?:crime_risk|recidivism|offender_likelihood|police_dispatch|criminal_record|sentencing_recommend)\b/gi }, // ----- migr
n: /\b(?:crime_risk|recidivism|offender_likelihood|police_dispatch|criminal_record|sentencing_recommend)\b/gi }, // ----- migration_signa
…and 1 more.
STRONGBiometric categorisation by protected attributes EU AI Act, Art 5(1)(g)0/1 rules4/4
▸
4/4
Why we flagged it
Composite raw score 0.00 (0/1 rules matched).
Suggested fix · we looked for these and found none
- detect_biometric_categorisation_by_protected_attrs
Security & Governance
Don't destabilize society3.0/4Adequate4 clauses▸
Security & Governance
Don't destabilize societyADEQUATECybersecurity measures appropriate to circumstances EU AI Act, Art 15(5)3/4 rules3/4
▸
3/4
Why we flagged it
Composite raw score 0.80 (3/4 rules matched).
▸Evidence · 12 hits— click to view code
(hallucination, prompt injection, output bias, leakage, capability escalation), mitigation owner, status. Wire eval suite into CI
ic: - rule: prompt_injection_defences weight: 0.4 description: | Code includes prompt-injection miti
Code includes prompt-injection mitigations: output filters, input sanitisation, instruction-data segregation, system-promp
Eval suite includes prompt-injection / adversarial cases" score_mapping: ">=0.85": 4 ">=0.65": 3 ">=0.40": 2 ">=
hint: | Add a prompt-injection eval set (e.g. from `promptbench`, `garak`, or your own canonical injection prompts). Sanitise to
…and 7 more.
Suggested fix · we looked for these and found none
- adversarial_eval_present
PARTIALSecurity and resilience evaluated NIST AI RMF, Art MEASURE 2.7 LLM2/3 rules2/4
▸
2/4
Why we flagged it
Composite raw score 0.60 (2/3 rules matched). LLM judge (confidence 0.72): Prompt injection defences and rate limiting are evidenced and documented (0.4 + 0.2 = 0.6 weight coverage), meeting more than half the weighted criteria. However, adversarial_eval_present (0.4 weight) remains unmatched despite references to eval suites; the evidence shows recommendations to add adversarial test cases rather than confirmation they are implemented and integrated into CI/CD, creating a material gap in the security evaluation requirement.
▸Evidence · 12 hits— click to view code
(hallucination, prompt injection, output bias, leakage, capability escalation), mitigation owner, status. Wire eval suite into CI
ic: - rule: prompt_injection_defences weight: 0.4 description: | Code includes prompt-injection miti
Code includes prompt-injection mitigations: output filters, input sanitisation, instruction-data segregation, system-promp
Eval suite includes prompt-injection / adversarial cases" score_mapping: ">=0.85": 4 ">=0.65": 3 ">=0.40": 2 ">=
hint: | Add a prompt-injection eval set (e.g. from `promptbench`, `garak`, or your own canonical injection prompts). Sanitise to
…and 7 more.
Suggested fix · we looked for these and found none
- adversarial_eval_present
ADEQUATESecurity of processing GDPR, Art 322/2 rules3/4
▸
3/4
Why we flagged it
Composite raw score 0.80 (2/2 rules matched). Supporting docs may exist outside the repo.
▸Evidence · 6 hits— click to view code
https://github.com/${owner}/${repo}`,https://github.com/owner/repo
https://github.com/${body.source.owner}/${body.source.repo}`,https://github.com/${body.source.owner}/${body.source.repo}`,…and 1 more.
STRONGResources for AI systems ISO/IEC 42001, Art A.72/2 rules4/4
▸
4/4
Why we flagged it
Composite raw score 1.00 (2/2 rules matched). Supporting docs may exist outside the repo.