Training Effectiveness Agent

According to the IW Further Education Survey 2023, German companies invested an average of 1,347 euros per employee per year in further education (46 billion euros across the economy as a whole). At the same time, only 10 to 20 percent of what is learned actually reaches the workplace. Between these two numbers sits not a quality problem. There sits a measurement problem. Anyone who does not know what happens after the training cannot steer whether the investment is working.

The evaluation gap is well documented: 95 percent of L&D organisations cannot link their learning data to business outcomes. ROI measurement happens at only 5 to 8 percent of all training programmes. Between those two sit the levels that matter - knowledge transfer, behaviour change, business impact - and that is exactly where almost no one measures.

Why satisfaction says nothing about effectiveness

After every training, participants fill in a feedback form. Was the instructor good? Was the catering acceptable? Would you recommend the training? These so-called smiley sheets are nearly universal and nearly universally worthless for the question L&D actually has to answer: did this training change behaviour at work?

The Kirkpatrick model has described four evaluation levels since 1959: reaction, learning, behaviour, results. These levels are not theory. They are the industry standard. And yet three of the four levels remain unoccupied in most organisations.

Level         What is measured                Who typically measures it
────────────  ─────────────────────────────   ────────────────────────────
Reaction      Feedback forms after training    Almost all organisations
Learning      Tests, practical assignments     Few, mostly compliance
Behaviour     Behaviour change at work         Almost no one systematically
Results       Correlation with business KPIs   5-8 percent of programmes

The reason is not lack of interest. The reason is structural. Every level requires a different method, a different timing, and a different owner. Reaction can be captured on the day of the training. Knowledge transfer needs a defined interval - two to four weeks - and a test format. Behaviour change only shows after 60 to 90 days and needs the manager’s observation. Business impact requires linking training data with performance metrics that live in an entirely different system.

No L&D team with three learning specialists for 1,500 employees can manage this manually. So the smiley sheet remains.

The transfer problem is a timing problem

Hermann Ebbinghaus documented in the 1880s what every learning professional knows instinctively: within 24 hours, humans forget on average 70 percent of new information. After a week, 90 percent. The forgetting curve is not a weakness of individual learners. It is a neurological constant.

The consequence for training effectiveness is radical. When the first knowledge check happens three months after the training - because no one thinks of it earlier or lacks the capacity - it is not measuring knowledge transfer. It is measuring the forgetting curve. And when the behaviour observation never happens because the manager has no structured feedback format, the decisive question stays open: did the employee transfer what they learned into daily work or not?

The transfer problem is not an educational problem. It is an architectural problem. Who measures what, when, and how - these responsibilities are not defined in most organisations. That is why it does not happen.

Staged measurement as architecture

The Decision Layer solves this not through a better tool but through a different decision architecture. Each evaluation level is treated as an independent process step - with a defined time, a defined method, and a defined owner.

Day 0              Week 2-4           Month 2-3          Quarter+
Training ends      Knowledge check    Behaviour FB       Correlation
                                                         with KPIs
┌──────────┐     ┌──────────┐      ┌──────────┐      ┌──────────┐
│ Feedback │     │ Test or  │      │ Manager  │      │ Training │
│ collected│────▶│ practical│─────▶│ rates    │─────▶│ vs perf- │
│ automa-  │     │ assign-  │      │ behaviour│      │ ormance  │
│ tically  │     │ ment     │      │ change   │      │ compared │
└──────────┘     └──────────┘      └──────────┘      └──────────┘
     A                A                  H                 A

A = Agent / automation     H = Human decides

Reaction capture on the day of the training runs fully automatically. The agent aggregates feedback forms, identifies outliers, and produces a report. No human has to read free-text answers to see that an instructor is systematically rated poorly.

The knowledge check two to four weeks later follows a rule set: training type determines the test format. A compliance training has a multiple-choice test. A leadership development programme has a reflection assignment. A technical training has a practical exercise. The agent triggers the right check at the right time and analyses the results.

Behaviour observation after 60 to 90 days stays with humans. This is where the manager comes in - not with an elaborate assessment, but with three to five targeted questions: is the employee applying the new methods? In which situations? What has changed? The agent asks the questions at the right time, collects the answers, and attributes them to the training programme.

ROI calculation at the programme level is correlation analysis, not a causation claim. The agent links training participation with performance metrics - error rates, customer satisfaction, productivity development - and shows relationships. Whether a relationship is causal is interpreted by a human with context knowledge. Because if the error rate in the warehouse drops after a safety training, that can come from the training - or from the new shelving system introduced at the same time.

What this changes for budget conversations

When a Head of L&D stands in front of the executive team and has to defend next year’s budget, the data foundation determines the negotiating position. With smiley sheets, the conversation goes like this: participants were satisfied. We believe the trainings were useful. Please do not cut us.

With staged evaluation, it looks different: the leadership programme shows measurable behaviour change in 72 percent of participants after 90 days. The correlation between participation and customer satisfaction in sales is 0.4. The compliance training has 85 percent knowledge retention after four weeks. The SAP fundamentals training shows no measurable effect on processing times - we need a different format here.

This is no longer budget defence. This is an investment decision based on evidence.

The difference is not in the precision of the numbers. It is in the ability to distinguish between trainings that work and trainings that occupy people. Organisations that reach this maturity shift their budget not by broad distribution but by effectiveness. And they can be transparent with worker representatives about how evaluation data is used - in aggregate, at the programme level, without inferences about individual participants. (US: similar evidence demands are emerging under SEC human-capital disclosure rules, where L&D investment is increasingly expected to show workforce outcomes.)