Skip to content

Commit 96238c9

Browse files
nanotaboadananotaboada
committed
feat(copilot): implement token efficiency strategy (#364)
- Optimize copilot-instructions.md to 622 tokens (600-650 target) - Update AGENTS.md with accurate token counts (~2,550 tokens) - Add scripts/count-tokens.sh with auto-install tiktoken - Achieve 80% token savings when AGENTS.md not loaded
1 parent 3b546aa commit 96238c9

3 files changed

Lines changed: 166 additions & 28 deletions

File tree

‎.github/copilot-instructions.md‎

Lines changed: 34 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,40 +1,41 @@
11
# GitHub Copilot Instructions
22

3-
> **⚡ Token Efficiency Note**: This is a minimal pointer file (~500 tokens, auto-loaded by Copilot).
4-
> For complete operational details, reference: `#file:AGENTS.md` (~2,500 tokens, loaded on-demand)
5-
> For specialized knowledge, use: `#file:SKILLS/<skill-name>/SKILL.md` (loaded on-demand when needed)
3+
> **Token Budget**: Target 600, limit 650 (auto-loaded)
4+
> Details: `#file:AGENTS.md` (~2,550 tokens, on-demand)
5+
> Skills: `#file:SKILLS/<name>/SKILL.md` (on-demand)
66
7-
## 🎯 Quick Context
7+
## Quick Context
88

9-
**Project**: ASP.NET Core 8 REST API demonstrating layered architecture patterns
10-
**Stack**: .NET 8 (LTS) • EF Core 9 • SQLite • Docker • xUnit
11-
**Pattern**: Repository + Service Layer + AutoMapper + FluentValidation
12-
**Philosophy**: Learning-focused PoC emphasizing clarity and best practices
9+
ASP.NET Core 8 REST API with layered architecture
10+
**Stack**: .NET 8 LTS, EF Core 9, SQLite, Docker, xUnit
11+
**Pattern**: Repository + Service + AutoMapper + FluentValidation
12+
**Focus**: Learning PoC emphasizing clarity and best practices
1313

14-
## 📐 Core Conventions
14+
## Core Conventions
1515

1616
- **Naming**: PascalCase (public), camelCase (private)
1717
- **DI**: Primary constructors everywhere
1818
- **Async**: All I/O operations use async/await
1919
- **Logging**: Serilog with structured logging
2020
- **Testing**: xUnit + Moq + FluentAssertions
21-
- **Formatting**: CSharpier (opinionated)
21+
- **Formatting**: CSharpier
22+
- **Commits**: Subject ≤80 chars, include issue number (#123), body lines ≤80 chars, conventional commits
2223

23-
## 🏗️ Architecture at a Glance
24+
## Architecture
2425

2526
```text
2627
Controller → Service → Repository → Database
2728
↓ ↓
2829
Validation Caching
2930
```
3031

31-
- **Controllers**: Minimal logic, delegate to services
32-
- **Services**: Business logic + caching with `IMemoryCache`
33-
- **Repositories**: Generic `Repository<T>` + specific implementations
34-
- **Models**: `Player` entity + Request/Response DTOs
35-
- **Validators**: FluentValidation for input structure (business rules in services)
32+
Controllers: Minimal logic, delegate to services
33+
Services: Business logic + `IMemoryCache` caching
34+
Repositories: Generic `Repository<T>` + specific implementations
35+
Models: `Player` entity + DTOs
36+
Validators: FluentValidation (structure only, business rules in services)
3637

37-
## ✅ Copilot Should
38+
## Copilot Should
3839

3940
- Generate idiomatic ASP.NET Core code with minimal controller logic
4041
- Use EF Core async APIs with `AsNoTracking()` for reads
@@ -44,14 +45,14 @@ Validation Caching
4445
- Use primary constructors for DI
4546
- Implement structured logging with `ILogger<T>`
4647

47-
## 🚫 Copilot Should Avoid
48+
## Copilot Should Avoid
4849

4950
- Synchronous EF Core APIs
5051
- Controller business logic (belongs in services)
5152
- Static service/repository classes
5253
- `ConfigureAwait(false)` (unnecessary in ASP.NET Core)
5354

54-
## ⚡ Quick Commands
55+
## Quick Commands
5556

5657
```bash
5758
# Run with hot reload
@@ -66,12 +67,21 @@ docker compose up
6667
# Swagger: https://localhost:9000/swagger
6768
```
6869

69-
## 📚 Need More Detail?
70+
## Load On-Demand Files
7071

71-
**For operational procedures**: Load `#file:AGENTS.md`
72-
**For Docker expertise**: *(Planned)* `#file:SKILLS/docker-containerization/SKILL.md`
73-
**For testing patterns**: *(Planned)* `#file:SKILLS/testing-patterns/SKILL.md`
72+
**Load `#file:AGENTS.md` when:**
73+
- "How do I run tests with coverage?"
74+
- "CI/CD pipeline setup or troubleshooting"
75+
- "Database migration procedures"
76+
- "Publishing/deployment workflows"
77+
- "Detailed troubleshooting guides"
78+
79+
**Load `#file:SKILLS/<skill-name>/SKILL.md` (planned):**
80+
- Docker optimization: `docker-containerization/SKILL.md`
81+
- Testing patterns: `testing-patterns/SKILL.md`
82+
83+
**Human-readable overview**: See `README.md` (not auto-loaded)
7484

7585
---
7686

77-
💡 **Why this structure?** Copilot auto-loads this file on every chat (~500 tokens). Loading `AGENTS.md` or `SKILLS/` explicitly gives you deep context only when needed, saving 80% of your token budget!
87+
**Why this structure?** Base instructions (~600 tokens) load automatically. On-demand files (~2,550 tokens) load only when needed, saving 80% of tokens per chat.

‎AGENTS.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
# AGENTS.md
22

3-
> **⚡ Token Efficiency Note**: This file contains complete operational instructions (~2,500 tokens).
4-
> **Auto-loaded**: NO (load explicitly with `#file:AGENTS.md` when you need detailed procedures)
5-
> **When to load**: Complex workflows, troubleshooting, CI/CD setup, detailed architecture questions
6-
> **Related files**: See `#file:.github/copilot-instructions.md` for quick context (auto-loaded, ~500 tokens)
3+
> **Token Efficiency**: Complete operational instructions (~2,550 tokens).
4+
> **Auto-loaded**: NO (load explicitly with `#file:AGENTS.md` when needed)
5+
> **When to load**: Complex workflows, troubleshooting, CI/CD setup, detailed architecture
6+
> **Related files**: `#file:.github/copilot-instructions.md` (auto-loaded, ~650 tokens)
77
88
---
99

‎scripts/count-tokens.sh‎

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
#!/bin/bash
2+
# 📊 Token Counter for Copilot Instruction Files
3+
# Uses tiktoken (OpenAI's tokenizer) for accurate counting
4+
# Approximation: ~0.75 words per token (English text)
5+
6+
set -e
7+
8+
echo "📊 Token Analysis for Copilot Instructions"
9+
echo "=========================================="
10+
echo ""
11+
12+
# Check if tiktoken is available
13+
if command -v python3 &> /dev/null; then
14+
# Try to use tiktoken for accurate counting
15+
if python3 -c "import tiktoken" 2>/dev/null; then
16+
echo "✅ Using tiktoken (accurate Claude/GPT tokenization)"
17+
echo ""
18+
else
19+
# tiktoken not found - offer to install
20+
echo "⚠️ tiktoken not installed"
21+
echo ""
22+
echo "tiktoken provides accurate token counting for Claude/GPT models."
23+
read -p "📦 Install tiktoken now? (y/n): " -n 1 -r
24+
echo ""
25+
if [[ $REPLY =~ ^[Yy]$ ]]; then
26+
echo "📥 Installing tiktoken..."
27+
if pip3 install tiktoken --quiet; then
28+
echo "✅ tiktoken installed successfully!"
29+
echo ""
30+
# Re-run the script after installation
31+
exec "$0" "$@"
32+
else
33+
echo "❌ Installation failed. Using word-based approximation instead."
34+
echo ""
35+
USE_APPROX=1
36+
fi
37+
else
38+
echo "📝 Using word-based approximation instead"
39+
echo " (Install manually: pip3 install tiktoken)"
40+
echo ""
41+
USE_APPROX=1
42+
fi
43+
fi
44+
45+
# Only run tiktoken if it's available and we didn't set USE_APPROX
46+
if [ -z "$USE_APPROX" ] && python3 -c "import tiktoken" 2>/dev/null; then
47+
48+
# Create temporary Python script
49+
cat > /tmp/count_tokens.py << 'PYTHON'
50+
import tiktoken
51+
import sys
52+
53+
# cl100k_base is used by GPT-4, Claude uses similar tokenization
54+
encoding = tiktoken.get_encoding("cl100k_base")
55+
56+
file_path = sys.argv[1]
57+
with open(file_path, 'r', encoding='utf-8') as f:
58+
content = f.read()
59+
60+
tokens = encoding.encode(content)
61+
print(len(tokens))
62+
PYTHON
63+
64+
# Count tokens for each file
65+
echo "📄 .github/copilot-instructions.md"
66+
COPILOT_TOKENS=$(python3 /tmp/count_tokens.py .github/copilot-instructions.md)
67+
echo " Tokens: $COPILOT_TOKENS"
68+
echo ""
69+
70+
echo "📄 AGENTS.md"
71+
AGENTS_TOKENS=$(python3 /tmp/count_tokens.py AGENTS.md)
72+
echo " Tokens: $AGENTS_TOKENS"
73+
echo ""
74+
75+
# Calculate total
76+
TOTAL=$((COPILOT_TOKENS + AGENTS_TOKENS))
77+
echo "📊 Summary"
78+
echo " Base load (auto): $COPILOT_TOKENS tokens"
79+
echo " On-demand load: $AGENTS_TOKENS tokens"
80+
echo " Total (if both): $TOTAL tokens"
81+
echo ""
82+
83+
# Check against target
84+
TARGET=600
85+
LIMIT=650
86+
if [ $COPILOT_TOKENS -le $TARGET ]; then
87+
echo "✅ copilot-instructions.md within target ($TARGET tokens)"
88+
elif [ $COPILOT_TOKENS -le $LIMIT ]; then
89+
echo "⚠️ copilot-instructions.md over target but within limit ($LIMIT tokens)"
90+
else
91+
echo "❌ copilot-instructions.md exceeds limit! Optimization required."
92+
fi
93+
94+
SAVINGS=$((AGENTS_TOKENS * 100 / TOTAL))
95+
echo "💡 Savings: ${SAVINGS}% saved when AGENTS.md not needed"
96+
97+
# Cleanup
98+
rm /tmp/count_tokens.py
99+
fi
100+
else
101+
echo "❌ Python3 not found"
102+
echo " Python 3 is required for token counting"
103+
echo " Install from: https://www.python.org/downloads/"
104+
echo ""
105+
exit 1
106+
fi
107+
108+
# Fallback: word-based approximation
109+
if [ -n "$USE_APPROX" ]; then
110+
echo "📄 .github/copilot-instructions.md"
111+
WORDS=$(wc -w < .github/copilot-instructions.md | tr -d ' ')
112+
APPROX_TOKENS=$((WORDS * 4 / 3))
113+
echo " Words: $WORDS"
114+
echo " Approx tokens: $APPROX_TOKENS"
115+
echo ""
116+
117+
echo "📄 AGENTS.md"
118+
WORDS=$(wc -w < AGENTS.md | tr -d ' ')
119+
APPROX_TOKENS=$((WORDS * 4 / 3))
120+
echo " Words: $WORDS"
121+
echo " Approx tokens: $APPROX_TOKENS"
122+
echo ""
123+
124+
echo "💡 Note: Run script again to install tiktoken for accurate counts"
125+
fi
126+
127+
echo ""
128+
echo "=========================================="

0 commit comments

Comments
 (0)