Rebuilding a Conduct of Operations Course After It Failed

I inherited this course halfway through development. I finished it, piloted it twice, and it failed. Then a new regulatory order made it mandatory for a much larger audience, and I took the whole thing back to analysis and started over.

That second version is what runs now.

Where it started

A subject matter expert proposed the course because he saw a need. A training needs analysis agreed, and development began under the Systematic Approach to Training — the ADDIE process my field uses when instruction has to be defensible rather than merely good.

Another training specialist did the analysis, the initial design, and part of the development. I picked it up mid-development, got up to speed, and finished it. Somewhere in that stretch the original SME moved on, which is worth naming only because it is the ordinary texture of long projects: the person who wanted the thing is often gone before the thing exists.

Why the pilots failed

The design I inherited ran as two four-hour sessions across two days. A three-hour presentation, then an hour familiarizing trainees with a commercial human performance trainer. The next day, each trainee took a turn as procedure reader and then as doer.

I could see it was going to be a difficult sell. I ran it anyway, because you learn more from piloting a design than from arguing with it.

The presentation could not fill its own slot. Sixty slides, opening with Three Mile Island — an incident everyone in the room had heard about many times. The logic was sound and the structure was this, because this; that, because that. It was dry. I don’t read from slides; I talk and glance back as I hit each point, and I am verbose. I still finished in an hour and a half, both times. A three-hour block that runs ninety minutes is not a scheduling error. It means the material could not sustain the time, and no amount of delivery skill was going to hold attention across it.

The practical wasted most of the room. Ten trainees, one trainer, two seats. Eight people watched at any given moment. In teaching I would have called it wasted student-minutes, and there were a great many of them.

I improvised mid-pilot — collapsed two four-hour days into a single six-hour day — and still finished with time to spare. That helped the schedule and did nothing for the underlying problem.

Both pilots produced the same two conclusions. The talking had to shrink to almost nothing, and everyone had to be doing something at all times.

Then the requirement changed

A new order made conduct of operations training mandatory for a defined population. Until then the course had been a good idea. Now it was a compliance obligation with a deadline — roughly 140 people to train inside a year.

The audience changed with it. What had been aimed at operators now had to work for supervisors and managers too.

That is a different course, not a revision of the old one. So I went back to analysis and started over, working from the federal regulations that govern this kind of work rather than from the existing material.

What I built

One course in two halves.

A workshop, built as five scenarios rather than a presentation. Each covers a different area of conduct of operations, and each uses a deliberately different mode of interaction so the shape of the session keeps changing:

• Trainees work in pairs to find the places where a written procedure is less complete than it looks
• A pair executes a procedure exactly as written — one reading, one doing — to complete a system lineup for starting a generator
• A full-class discussion
• Groups of five work a plant casualty, each person holding a different role. The casualty is a smoking microwave, which is deliberate: the principles are the point, and a mundane emergency keeps the focus on how people coordinate rather than on the equipment

A practical, sixty to seventy-five minutes, two trainees running two procedures and swapping reader and doer between them.

The scheduling is part of the design, not an administrative detail. The practical runs as a five-hour block every two weeks, and each pair takes one hour of it. With 140 people to move through in a year, that was the arrangement that trained everyone without a room full of people waiting their turn. The failure mode I was designing against was the one I had watched: eight people at the back of the room while two worked.

Evidence

Two pilots of the redesigned course went substantially better. That is my own observation as the instructor, and it is corroborated by trainee reaction-level feedback — anonymous paper surveys at the end of each pilot, Likert items recorded as numerical data alongside free-text responses.

The subject matter experts were satisfied with the redesign. The training managers responsible for the affected populations were noticeably happier with it than with the original. The work was later recognized with an internal award.

In the couple of months since handoff, 110 trainees have completed it. That figure is a throughput count, not an effectiveness measure — I am not claiming demonstrated behavior change on the job, because I have not measured it. What it does show is that a course which previously could not hold a room for an afternoon now moves a required population at the rate the deadline demands.

Reaction-level feedback and stakeholder satisfaction are the weakest tier of evidence in my field, and I would rather name that than dress it up. What I can say is that the redesign cleared the bar the original could not: it runs, people complete it, and the organization is willing to keep sending them.

What I am building next

The practical still depends on a single commercial trainer that seats two people. Buying more of them is expensive, and that hardware constraint is the reason the practical has to be scheduled in pairs across months.

So I am building a replacement: a simulated plant in Blender and Unity, running from one computer to two inexpensive touchscreen monitors. One screen shows an interactive procedure; the other shows the operating panel. The task is moving water between tanks — transfer and recirculation.

Water is the right choice precisely because it is unglamorous. It is legible to anyone, which means the scenario can travel to organizations outside the nuclear facilities the course was written for, and there has already been interest in that.

It also differentiates cleanly. A new operator gets the straightforward lineup moving water from the first tank to the second. An experienced one gets the harder path back.

Version two keeps the two-screen arrangement and adds VR control of both the procedure and the panel.

What I would tell someone starting this

Pilot the design you inherited before you argue with it. I suspected the first version would not work. Running it twice told me why — and “the presentation could not fill its allotted time” is a far more useful finding than “I think this is too long.”

A schedule that collapses under you is a design signal. Finishing a three-hour block in ninety minutes was the clearest evidence I got.

Count the people who are not doing anything. Wasted student-minutes were the real defect in the original practical, and the scheduling redesign exists to eliminate them.

A hardware constraint is a design constraint. One machine seating two people shaped the entire delivery model. Removing that constraint in software is not a technical side project — it is the next iteration of the course.