Our investigation of the respective chapter’s individual aspects and the summary of the results lead to the question, whether, and which factors of our maturity disciplines are interconnected. We now present an assessment of the consistency of interdisciplinary answering behavior. Afterwards, the main factors of influence on portfolio maturity will be introduced, following the identification of homogeneous groups in practiced portfolio management by means of a cluster analysis.
Consistent statements concerning governance and social aspects – false perceptions in management demands for process and system quality
In order to test the questioned behavior patterns for plausibility, consistency, and rationality, a control question was determined for each discipline, as well as interdisciplinary set of indicators consisting of a set of questions within a strongly logical framework that “should be” answered similarly. Answering behavior was largely highly consistent. While questions about governance and social aspects were answered in an objectively reasonable manner by all participants, there is a clear separation between perception and reality when considering the interaction between management demands, process characteristics, and system quality. Below are two examples:
Testing the plausibility of statements concerning the quality of IT systems
• Portfolio Management – use of entitlement process (process)
• Integrity and degree of process support due to software use (systems)
• Quality of process input data (systems)
• Consideration of capacity / resource management (resource management)
Some participants claimed to be using portfolio management very heavily for planning resources, continuous monitoring, centralized project data administration, and the project approval process. However, there is no software in use, data maintenance is mostly manual, and staffing projects with qualified employees is based on area-wide personnel knowledge. Thus, a data-intensive and consequently executed best practice portfolio process with the goal of global resource optimization does not seem to be implemented yet.
Testing the plausibility of statements concerning process maturity
• Relation to strategy in the portfolio choice process (management)
• Risk analysis during project approval (management)
• Methods of project evaluation and selection (process)
• CIP, Lessons Learned, and process review (process)
• Commitment and decision loyalty (social aspects)
At this point, several participants stated to have established clear objectives and evaluation systems, as well as integrating comprehensive risk assessments into these systems. This stands in contrast to the statement that decisions are only made based on voting, emotion, intuition, and urgency. A process for continuous improvement is less frequent, and results are not binding. It is most frequently applied to decision making only. The rationale behind, and actual application of these goal systems should be questioned.
These properties of the consistency analysis also reflect the individual control questions. The self-assessment of data quality in the systems, along with that of the clear evaluation and goal-setting management systems, with more than 10 percent deviation in the context of the respective discipline’s total statements, have the tendency to be too good.
Levers of control: identification of key and success factors in the implementation and practice of R&D portfolio management
Beyond the sober assessment of single portfolio discipline’s maturities, it is especially of interest, which single questions, and consolidated disciplines concisely affect the degree of total maturity. After analyzing a company’s weaknesses, such a procedure improvement can be highly effective if focused specifically on those drivers proven to have the highest effect on total maturity. The analysis of correlations, that is to say the efficiency rate of single questions and the final maturity values of all answer sheets, shows that especially best performers have achieved significantly high maturity levels:
• Application of portfolio management inside the company as a formalized project approval process with standardized project assessment procedures. PPM as the central project data repository, for resource planning, project controlling, and risk tool, as well as for continuous monitoring and multi-project prioritization (process)
• Very positive personal connection to the topic, due to active participation, creation of incentives and goal effectiveness, good training and positive experiences in connection with portfolio management in corporate history (social aspects)
• High quality and availability of process input data due to integration into existing systems, standardized data templates, avoidance of redundancies and manual labor, especially by providing a central project data repositories (systems)
Process maturity and employee motivation: significant influence on degree of total maturity in product development portfolio management
In summary, the disciplines process and social aspects are shown to have the greatest influence on the degree of total maturity, which is supported by the fact in the analysis of correlations between all disciplines, that they have the highest active / passive sums (“has influence on” / ”is influenced by”). Thus, they can be referred to as the model’s levers of success. Governance, systems and management follow, sharing approximately the same power ranking. Only resource management is an outlier, showing neither a high individual correlation to total maturity, nor is it in significant correlation to the other disciplines. Resource management is still seen as the project leader’s sovereign responsibility, and is taken care of only after the project has been approved as part of operative project management, which, as shown in this study, leads to difficult and easily avoidable problems.
Statistical proof of the importance of process and social aspects: multifactorial analysis as an interdisciplinary method of influence analysis
In addition to assessing the influence of single disciplines on total maturity, it is important for process optimization to know which factors especially support each other. Multifactorial analysis is the empirical method of analyzing the total effect of single disciplines on total maturity. The following diagram presents the influence of our model’s disciplines on total maturity.
(Total Maturity) as vectors, based on vector length and the angle between them (see side note). Resource management’s vector had to be removed, due to its insignificant interrelation with the total model.
The importance of, and the influence on total maturity of process and social aspects is strongly supported yet again in the model by the proximity to total maturity and the vector length. These are equally followed by governance and systems. Management seems to exert strong individual influence on total maturity. However, it functions more or less independently from the practice and professionalism of other disciplines due to its short length, and is thus not an indicator for practiced total maturity. There is a clear and distinct correlation between process and systems as well as between social aspects and governance, so these areas of portfolio management practice should be analyzed and optimized together. Management is in between, the questioned themes in this discipline consequently show methodological (process / IT) as well as organizational aspects (governance / social aspects).
The second analysis portrayed in the diagram is the identification of two distinctly separate groups of key factors, to be seen above and below total maturity, respectively. As previously explained, each group’s disciplines show strong connections to each other. They can be classified as portfolio management’s functional (methodology, process, system, stringency) and qualitative (employees, framework, consciousness, support) factors. Resource management as a discipline also shows a weak correlation to process, and as such could additionally support this second factor. In summary, this results in two simple, now empirical pieces of advice on how to increase portfolio management maturity in the long term. This leads to the following insight: the factors, our six maturity disciplines management, governance, process, system, resource, and social aspects, split up into just two harmonizing groups, which act self-supportively and drive total maturity as a group. Professionalism and maturity in process, paired with the use of appropriate IT, together with aspiring towards centralized governance and employee satisfaction in terms of social aspects, are thus the two factors of success in successful product development control.
Best practice approach: ideal application of portfolio management
Based on the results of the factor analysis we have established conceptual best practices. The following descriptions are mathematically proven ideal portfolio management processes in product development, based on available investigative results.
IT-supported portfolio process with strategic and transparent project evaluation
• Establishing a standardized, company-wide, universal, methodical portfolio process. At the same time, development of a clear catalog for project evaluation with criteria for corporate strategy, risk, business case, and product quality. This agile, data-intensive process is supported by the implementation and effective use of an integrated, centralized and user-friendly portfolio management software.
Portfolio management awareness and discipline through governance
• Creating awareness for the issues and the uses of portfolio management in product development by actively integrating, training, and pointing towards goals of relevant employees and process stakeholders. This is partially achieved by clear commitment and by promoters from company leadership setting an example. It should be accompanied by the establishing of a centralized and powerful project and portfolio management office.
Cluster analysis – profiling companies in regard to behavioral patterns and targeted optimization of portfolio management maturity
A cluster analysis, along with the identification of key factors, reveals a company’s individual weaknesses and which approach leads to an increased total maturity. The statistically complex procedure investigates which groups of interview participants show similar behavior schemes in regard to the guidance of product development projects, and who are consequently facing comparable challenges. This perspective serves the development of specific solution models for each cluster. For organizational development, the motivation lies in creating more complete developmental steps, rather than just optimizing single factors. Additionally, one company’s ‘lessons learned’ helps to deduce analogies, synergies and knowledge transfer in another, homogeneous company from the same cluster. The following depiction visualizes the cluster analysis’ results as lines, which are the cluster’s typically achieved average degrees of maturity for each portfolio management discipline. Four clusters sufficiently discernable from each other were identified. A cluster does not necessarily consist of companies with the same maturity, but indicates similar patterns in their answering behavior from question to question. Similar weaknesses in portfolio processes, and thus initial points of approach for a thorough process of improvement can be identified in each cluster. There is an apparent scatter of achieved maturity values per discipline amongst all participants. The resulting average line serves as a comparison to the total average.
Cluster analysis investigates the answering behavior of all companies for the entire questionnaire, regarding to same values for each question, discipline and interdisciplinary.
Four groups of companies were identified that are roughly equal in power and in which similar patterns of portfolio management behavior is shown. This way, individual pain points and best practices can be identified for each cluster.
The complementary dendogram visualizes participating companies sorted by similarity of traits. The lines solidify vertically upwards according to similarity. It corresponds with the diagram above and serves to identify a company, as well as its homogeneity to a neighboring company in the same cluster.
The dendogram shows the allocation of a company (anonymized as an index number) to a cluster. This also shows the degree of similarity between two companies of one cluster. The closer the horizontal connecting line between the companies is to the axis, the more similar their portfolio management behavior patterns are.
This knowledge is especially advantageous when analyzing and optimizing two similar companies in the same cluster, in regard to conclusions about analogies, synergies and knowledge transfer.
Cluster 1: Consistently good portfolio maturity
– yet strong deficits and isolated handling of resource management.
• The different discipline’s manifestations are noticeably balanced in cluster 1. Especially fact-based processes seem to be well integrated and lived, also because they are supported by appropriate and encompassing software.
• Resource management as a discipline, however, reaches the lowest point of the whole study. This topic is obviously seen to have no relation to portfolio management at all.
• The result could be the highly ineffective assignment of qualified employees, as well as considerable untapped potential in the use of tight budgets.
• Introducing IT-based centralized capacity management, with a perspective on globally available resources, could make this group into forerunners in strategic R&D controlling.
Cluster 2: Process and capacity experts – but without IT and connection to strategy, with lacking employee motivation and without an established PMO?
• The companies in cluster 2 are regarded as best performers in terms of processes
• The lowest value is achieved by system quality, which is important for optimal support of the data-intensive processes, and for assuring high process quality.
• The topics of management and governance seem to pose a serious problem, which in consequence also explains social aspects’ low score, beneath the median.
• It should be analyzed and exactly understood at a high organizational level, why there was failure to integrate corporate strategy into project approval processes, to establish a central PMO, and to provide sufficient training in regards to the topic of portfolio management. Thus, opportunities of effectively selecting the right product development projects have also been missed so far.
Cluster 3 & cluster 4: best and low performers
– pain points are processes, their IT support, as well as insufficient resource oversight
Although there are not necessarily similarities in maturity within a cluster, cluster 3 shows distinctly many members of the top 25th percentile, vice versa in cluster 4 many members from the lower 25th percentile. Both clusters show similar progress patterns, parallel to the median, however on a higher, respectively lower general level. The interpretation is evident:
• Weaknesses in process, and especially in system maturity are clearly visible. In comparison to all other companies, these are especially grave in cluster 4, as the maturity in other disciplines seems comparable.
• There is evidently much room for improvement in resource management.
• Companies in cluster 4 clearly stand out amongst the other companies in terms of governance and social aspects, as well as in management. Knowledge and a fundamentally positive attitude towards these themes are highly developed.
• Focused increases in process maturity as an identified key driver, along with the optimization of accompanying systems should moreover inevitably contribute to synergetic effects in other disciplines and positive general solution. This also counts for best performers.