Skip to content

Commit 7fea824

Browse files
authored
Merge pull request #63 from JuliaDecisionFocusedLearning/rework-type-hierarchy
Switch to horizontal type hierarchy
2 parents 341ed4a + cf031b3 commit 7fea824

File tree

21 files changed

+782
-710
lines changed

21 files changed

+782
-710
lines changed

docs/src/custom_benchmarks.md

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,14 @@ problems to the benchmark suite or integrate their own domains.
1010

1111
```
1212
AbstractBenchmark
13-
└── AbstractStochasticBenchmark{exogenous}
14-
└── AbstractDynamicBenchmark{exogenous}
13+
├── AbstractStaticBenchmark
14+
├── AbstractStochasticBenchmark{exogenous}
15+
└── AbstractDynamicBenchmark{exogenous}
1516
```
1617

1718
| Type | Use case |
1819
|------|----------|
19-
| `AbstractBenchmark` | Static, single-stage optimization (e.g. shortest path, portfolio) |
20+
| `AbstractStaticBenchmark` | Static, single-stage optimization (e.g. shortest path, portfolio) |
2021
| `AbstractStochasticBenchmark{true}` | Single-stage with exogenous uncertainty (scenarios drawn independently of decisions) |
2122
| `AbstractStochasticBenchmark{false}` | Single-stage with endogenous uncertainty |
2223
| `AbstractDynamicBenchmark{true}` | Multi-stage sequential decisions with exogenous uncertainty |
@@ -41,7 +42,7 @@ repeatedly and applies `target_policy` to each result.
4142

4243
---
4344

44-
## `AbstractBenchmark`: required methods
45+
## `AbstractStaticBenchmark`: required methods
4546

4647
### Data generation (choose one strategy)
4748

@@ -70,7 +71,7 @@ generate_maximizer(bench::MyBenchmark)
7071

7172
```julia
7273
generate_baseline_policies(bench::MyBenchmark) -> collection of callables
73-
is_minimization_problem(bench::MyBenchmark) -> Bool # default: false (maximization)
74+
is_minimization_problem(bench::MyBenchmark) -> Bool # default: true (minimization)
7475
objective_value(bench::MyBenchmark, sample::DataSample, y) -> Real
7576
compute_gap(bench::MyBenchmark, dataset, model, maximizer) -> Float64
7677
has_visualization(bench::MyBenchmark) -> Bool # default: false; return true when plot methods are implemented/available
@@ -80,7 +81,7 @@ plot_solution(bench::MyBenchmark, sample::DataSample; kwargs...)
8081

8182
---
8283

83-
## `AbstractStochasticBenchmark{true}`: additional methods
84+
## `AbstractStochasticBenchmark{true}`
8485

8586
For stochastic benchmarks with exogenous uncertainty, implement:
8687

@@ -113,7 +114,7 @@ DataSample(; x=features, y=nothing,
113114

114115
---
115116

116-
## `AbstractDynamicBenchmark`: additional methods
117+
## `AbstractDynamicBenchmark`
117118

118119
Dynamic benchmarks extend stochastic ones with an environment-based rollout interface.
119120

@@ -148,6 +149,15 @@ generate_baseline_policies(bench::MyDynamicBenchmark)
148149
# Each callable performs a full episode rollout and returns the trajectory.
149150
```
150151

152+
### Anticipative solver (optional)
153+
154+
```julia
155+
generate_anticipative_solver(bench::MyDynamicBenchmark)
156+
# Returns a callable: (env; reset_env=true, kwargs...) -> Vector{DataSample}
157+
# reset_env=true → reset environment before solving
158+
# reset_env=false → solve from current state
159+
```
160+
151161
### Optional visualization methods
152162

153163
```julia

docs/src/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ Where:
4343

4444
The package organizes benchmarks into three main categories based on their problem structure:
4545

46-
### Static Benchmarks (`AbstractBenchmark`)
46+
### Static Benchmarks (`AbstractStaticBenchmark`)
4747
Single-stage optimization problems with no randomness involved:
4848
- [`ArgmaxBenchmark`](@ref): argmax toy problem
4949
- [`Argmax2DBenchmark`](@ref): 2D argmax toy problem

docs/src/using_benchmarks.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ This guide covers everything you need to work with existing benchmarks in Decisi
88

99
A benchmark bundles a problem family (an instance generator, a combinatorial solver, and a statistical model architecture) into a single object. It provides everything needed to run a Decision-Focused Learning experiment out of the box, without having to create each component from scratch.
1010
Three abstract types cover the main settings:
11-
- **`AbstractBenchmark`**: static problems (one instance, one decision)
11+
- **`AbstractStaticBenchmark`**: static problems (one instance, one decision)
1212
- **`AbstractStochasticBenchmark{exogenous}`**: stochastic problems (type parameter indicates whether uncertainty is exogenous)
1313
- **`AbstractDynamicBenchmark{exogenous}`**: sequential / multi-stage problems
1414

@@ -65,7 +65,7 @@ sample.scenario # looks up :scenario in context first, then in extra
6565

6666
### Static benchmarks
6767

68-
For static benchmarks (`<:AbstractBenchmark`), `generate_dataset` may compute a default ground-truth label `y` if the benchmark implements it:
68+
For static benchmarks (`<:AbstractStaticBenchmark`), `generate_dataset` may compute a default ground-truth label `y` if the benchmark implements it:
6969

7070
```julia
7171
bench = ArgmaxBenchmark()
@@ -150,7 +150,8 @@ dataset = generate_dataset(bench, 50; rng=rng)
150150
gap = compute_gap(bench, dataset, model, maximizer)
151151
```
152152

153-
# Objective value for a single decision
153+
Objective value for a single decision:
154+
154155
```julia
155156
obj = objective_value(bench, sample, y)
156157
```

src/Argmax/Argmax.jl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Basic benchmark problem with an argmax as the CO algorithm.
1515
# Fields
1616
$TYPEDFIELDS
1717
"""
18-
struct ArgmaxBenchmark{E} <: AbstractBenchmark
18+
struct ArgmaxBenchmark{E} <: AbstractStaticBenchmark
1919
"instances dimension, total number of classes"
2020
instance_dim::Int
2121
"number of features"

src/Argmax2D/Argmax2D.jl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Argmax becnhmark on a 2d polytope.
1616
# Fields
1717
$TYPEDFIELDS
1818
"""
19-
struct Argmax2DBenchmark{E,R} <: AbstractBenchmark
19+
struct Argmax2DBenchmark{E,R} <: AbstractStaticBenchmark
2020
"number of features"
2121
nb_features::Int
2222
"true mapping between features and costs"

src/DynamicAssortment/DynamicAssortment.jl

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -78,17 +78,6 @@ include("policies.jl")
7878
"""
7979
$TYPEDSIGNATURES
8080
81-
Outputs a data sample containing an [`Instance`](@ref).
82-
"""
83-
function Utils.generate_sample(
84-
b::DynamicAssortmentBenchmark, rng::AbstractRNG=MersenneTwister(0)
85-
)
86-
return DataSample(; instance=Instance(b, rng))
87-
end
88-
89-
"""
90-
$TYPEDSIGNATURES
91-
9281
Generates a statistical model for the dynamic assortment benchmark.
9382
The model is a small neural network with one hidden layer of size 5 and no activation function.
9483
"""

src/FixedSizeShortestPath/FixedSizeShortestPath.jl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ Data is generated using the process described in: <https://arxiv.org/abs/2307.13
2121
# Fields
2222
$TYPEDFIELDS
2323
"""
24-
struct FixedSizeShortestPathBenchmark <: AbstractBenchmark
24+
struct FixedSizeShortestPathBenchmark <: AbstractStaticBenchmark
2525
"grid graph instance"
2626
graph::SimpleDiGraph{Int64}
2727
"grid size of graphs"

src/PortfolioOptimization/PortfolioOptimization.jl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ Data is generated using the process described in: <https://arxiv.org/abs/2307.13
1919
# Fields
2020
$TYPEDFIELDS
2121
"""
22-
struct PortfolioOptimizationBenchmark <: AbstractBenchmark
22+
struct PortfolioOptimizationBenchmark <: AbstractStaticBenchmark
2323
"number of assets"
2424
d::Int
2525
"size of feature vectors"

src/Ranking/Ranking.jl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Basic benchmark problem with ranking as the CO algorithm.
1515
# Fields
1616
$TYPEDFIELDS
1717
"""
18-
struct RankingBenchmark{E} <: AbstractBenchmark
18+
struct RankingBenchmark{E} <: AbstractStaticBenchmark
1919
"instances dimension, total number of classes"
2020
instance_dim::Int
2121
"number of features"

src/SubsetSelection/SubsetSelection.jl

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ without knowing their values, but only observing some features.
1818
# Fields
1919
$TYPEDFIELDS
2020
"""
21-
struct SubsetSelectionBenchmark{M} <: AbstractBenchmark
21+
struct SubsetSelectionBenchmark{M} <: AbstractStaticBenchmark
2222
"total number of items"
2323
n::Int
2424
"number of items to select"

0 commit comments

Comments
 (0)