Can you do too much systems engineering?

Our clients recognise that systems engineering is an increasingly important part of medical device
design. These days, a medical device needs to be considered as one element of a connected
system which has been defined to implement one or more objectives: drug delivery, adherence
monitoring, temperature monitoring, or a multitude of other applications. The stages up to and
including the clinical trials batch of devices are essential to ensure a successful development and
an effective end device.

So, how much should be budgeted for systems engineering activities during the overall project?
The work of Eric Honour of the University of Western Australia sets out to answer this question.
During his research, Honour interviewed both the programme managers and technical leaders of
systems engineering programmes to better understand the optimal amount of systems engineering
that should occur to facilitate the development of projects. Although these were not necessarily
medical sector programmes, the overall summary is interesting.

His research can be summarised as: “you should plan to spend around 15% of your project budget
on systems engineering.”

Most projects do not spend that much, allocating around 7% of their budget to systems engineering
and tangibly lowering their chances of success as a result. Interestingly, the data set in Honour’s
work also showed that spending more than 15% actually correlated with less effective projects in
terms of cost and schedule adherence.

While the data shows that the optimum 15% spend leads to statistically better programmes in
terms of cost and schedules adherence, it is also important to note that this does not necessarily
always lead to better products!

The challenge for engineers is to avoid analysis paralysis by trying to reduce risks to zero and to
get on with designing the product in detail at the right time. It’s also important to let the system
engineers make sound decisions early on and focus less on developing contractual documents
that don’t influence design.

Honour’s article did not find a significant correlation between systems engineering activities and
the system’s technical quality as measured by key performance parameters. Indeed, Honour
suggests that programme leaders rarely use key performance parameters to drive a programme.
They usually use the set of requirements as the driving technical factors.

In the current landscape, technical success is often measured by the technical requirements of a
product, rather than its technical qualities which will matter more to the stakeholders. This
approach perhaps represents a failure from the managers and contracts, as it will not result in the
best end products. The key to success is to find a balance between having a product with the
technical qualities to satisfy stakeholders and the technical requirements to make a useful end
product.

Honour’s data also shows that developing better products is not just about the systems
engineering and controlled development process. There is a danger you can spend too much on
process adherence instead of developing better products, focusing on the things that matter to the
stakeholders.

So, to maximise the probability of your project being a success across the board, it’s
recommended to spend around 15% of your development budget on systems engineering, but
remain relentless in innovating the product, optimising the performance in aspects that the
stakeholders care about. Only by doing this can we maximise the chances of success in cost and
schedule, but also produce the best final products.