You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Maintenance problem with resource constraint is a sequential decision-making benchmark where an agent must repeatedly decide which components to maintain over time. The goal is to minimize total expected cost while accounting for independent degradation of components and limited maintenance capacity.
4
+
5
+
6
+
## Problem Description
7
+
8
+
### Overview
9
+
10
+
In this benchmark, a system consists of $N$ identical components, each of which can degrade over $n$ discrete states. State $1$ means that the component is new, state $n$ means that the component is failed. At each time step, the agent can maintain up to $K$ components.
11
+
12
+
This forms an endogenous multistage stochastic optimization problem, where the agent must plan maintenance actions over the horizon.
13
+
14
+
### Mathematical Formulation
15
+
16
+
The maintenance problem can be formulated as a finite-horizon Markov Decision Process (MDP) with the following components:
17
+
18
+
**State Space** $\mathcal{S}$: At time step $t$, the state $s_t \in [1:n]^N$ is the degradation state for each component.
19
+
20
+
**Action Space** $\mathcal{A}$: The action at time $t$ is the set of components that are maintained at time $t$:
21
+
```math
22
+
a_t \subseteq \{1, 2, \ldots, N\} \text{ such that } |a_t| \leq K
23
+
```
24
+
### Transition Dynamics
25
+
26
+
The state transitions depend on whether a component is maintained or not:
27
+
28
+
For each component \(i\) at time \(t\):
29
+
30
+
-**Maintained component** (\(i \in a_t\)):
31
+
32
+
\[
33
+
s_{t+1}^i = 1 \quad \text{(perfect maintenance)}
34
+
\]
35
+
36
+
-**Unmaintained component** (\(i \notin a_t\)):
37
+
38
+
\[
39
+
s_{t+1}^i =
40
+
\begin{cases}
41
+
\min(s_t^i + 1, n) & \text{with probability } p,\\
42
+
s_t^i & \text{with probability } 1-p.
43
+
\end{cases}
44
+
\]
45
+
46
+
Here, \(p\) is the degradation probability, \(s_t^i\) is the current state of component \(i\), and \(n\) is the maximum (failed) state.
47
+
48
+
---
49
+
50
+
### Cost Function
51
+
52
+
The immediate cost at time \(t\) is:
53
+
54
+
$$
55
+
c(s_t, a_t) = \Big( c_m \cdot |a_t| + c_f \cdot \#\{ i : s_t^i = n \} \Big)
56
+
$$
57
+
58
+
Where:
59
+
60
+
- $c_m$ is the maintenance cost per component.
61
+
- $|a_t|$ is the number of components maintained.
62
+
- $c_f$ is the failure cost per failed component.
63
+
- $\#\{ i : s_t^i = n \}$ counts the number of components in the failed state.
64
+
65
+
This formulation captures the total cost for maintaining components and penalizing failures.
66
+
67
+
**Objective**: Find a policy $\pi: \mathcal{S} \to \mathcal{A}$ that minimizes the expected cumulative cost:
0 commit comments