In my domain (IT, with On-Call), there’s a practice called “Wheel of Misfortunes” or “Game hour”. This is in fact a short TTRPG session to simulate incident. This works very well. I am a paid DM 1h per week for my colleagues :)
Oh neat, our team does this but we call it “WTF Wednesday.” Usually the most senior engineer digs back into our incident log and tries to reproduce it in our dev environment, and we live-solve with him playing the role of the customer.
It is the same thing. In our case it’s not attached to the seniority. The person ending their shifts replays its incident when there has been one, with the person who is taking the pager after them. We are deeper in the infrastructure so we don’t have customers but we roleplay stakeholders (lead/head, principals, developer). My favorite is the person who has experienced something wrong but it is only this person and bad luck :P
Yeah, I think the goal is to eventually make it irrespective of seniority, but right now he’s the only one with 15+ years of institutional knowledge on the application, so he’s trying to pass on as much as he can to reduce our bus factor.
I don’t like the concept of wargames. We don’t need war to do this, nor conflict.
I see this more as an astronaut training: it has to be a solution, at least in the mind of the person proposing the situation. It also cultivates a spirit to always search for a way out of the invident.
One rule we adopted is that when the responder doesn’t know, they have to say it. Once it said, they need to say outloud what do they search. Then the focus shifts to the audience, they have to find 3 different ways to respond to what the responder is searching (to know or to do). It is hard and so far it balances well the dynamic (it is OK to not know, it is important to recognise we don’t know, and it is funny to share how we can hack our way through the system (the 3rd way is pretty hard and is in general a hack)).
I now realize that perhaps I could write a blog post on this.
In my domain (IT, with On-Call), there’s a practice called “Wheel of Misfortunes” or “Game hour”. This is in fact a short TTRPG session to simulate incident. This works very well. I am a paid DM 1h per week for my colleagues :)
Oh neat, our team does this but we call it “WTF Wednesday.” Usually the most senior engineer digs back into our incident log and tries to reproduce it in our dev environment, and we live-solve with him playing the role of the customer.
It is the same thing. In our case it’s not attached to the seniority. The person ending their shifts replays its incident when there has been one, with the person who is taking the pager after them. We are deeper in the infrastructure so we don’t have customers but we roleplay stakeholders (lead/head, principals, developer). My favorite is the person who has experienced something wrong but it is only this person and bad luck :P
Yeah, I think the goal is to eventually make it irrespective of seniority, but right now he’s the only one with 15+ years of institutional knowledge on the application, so he’s trying to pass on as much as he can to reduce our bus factor.
I would love to hear more about this if you can go in to more details?
Please share more. Sounds like a mini war game?
I don’t like the concept of wargames. We don’t need war to do this, nor conflict.
I see this more as an astronaut training: it has to be a solution, at least in the mind of the person proposing the situation. It also cultivates a spirit to always search for a way out of the invident.
One rule we adopted is that when the responder doesn’t know, they have to say it. Once it said, they need to say outloud what do they search. Then the focus shifts to the audience, they have to find 3 different ways to respond to what the responder is searching (to know or to do). It is hard and so far it balances well the dynamic (it is OK to not know, it is important to recognise we don’t know, and it is funny to share how we can hack our way through the system (the 3rd way is pretty hard and is in general a hack)).
I now realize that perhaps I could write a blog post on this.
For links, see my response to the other comment.