Jiayi Wang ·     

💡 Overview

Large Reasoning Models (LRMs) may include unsafe content in intermediate reasoning steps even when their final answers appear safe. ReasoningShield detects these risks in Chain-of-Thought (CoT) traces. The paper contributes:

• A taxonomy of 10 risk categories and 3 safety levels.
• A 9.2K-pair benchmark with 7K training samples and 2.2K human-annotated test samples.
• 1B and 3B moderation models trained with structured risk analysis and contrastive learning.

In the paper’s evaluation, ReasoningShield-3B reaches 91.8% average F1 on open-source LRM subsets (Table 2). Figure 4 reports 89.1% F1 in its CoT moderation comparison. These numbers summarize different evaluation scopes; see the paper for the benchmark splits and baselines.

📁 Project Structure

ReasoningShield/
├── reasoningshield/                  # Inference scripts and moderation prompt
├── reasoningshield_Dataset/
│   ├── reasoningshield-train/        # Stage 1 SFT and Stage 2 DPO data
│   └── reasoningshield-test/
│       ├── qa/test_qa.csv            # Answer moderation…