In 2020, we conducted a survey among the Capella community to gain insights into the usage of the workbench and the expectations surrounding Model-Based Systems Engineering (MBSE). The results of this survey were published in a previous article.
At the end of 2022, we conducted a follow-up study with a new set of questions. We received twice as many responses compared to 2020, with a total of 104 actionable answers.
Profile of the Respondents
The survey respondents came from 28 different countries, compared to 16 in 2020, with the majority coming from five European countries: France, Germany, Netherlands, UK, and Spain.
Other European countries represented are Austria, Belgium, Denmark, Italy, Norway, Poland, Romania, Slovakia, Sweden, and Switzerland.
Respondents from other regions included North America (USA and Canada), South America (Argentina, Brazil, and Chile), Asia/Pacific (Australia, China, India, Pakistan, and Thailand), the Middle East (Iran and Turkey), and Africa (South Africa).
They have a strong level of experience in MBSE, with 67% of them personally working with MBSE for at least three years.
Like in 2020, the majority of respondents are employed in the Defense and Aerospace sectors (51%). While there have been some changes in other sectors, they are not substantial. It is worth noting that respondents could select multiple domains.
Additionally, the companies they work for are primarily large organizations with over 1000 employees.
Capella Usage in Organizations
We asked the respondents about how Capella was being used in their organizations.
The level of adoption varies among the organizations, with a good balance between evaluations (18%), usages by individuals (23%), usages by teams (28%), and the official MBSE tool (27%).
It was observed that 66% of the organizations where Capella is the official MBSE tool were large companies with more than 1000 employees.
We also used the Net Promoter Score (NPS) to gauge respondents' likelihood of recommending Capella to friends or colleagues.
The NPS system categorizes respondents into three groups based on their rating from 0 to 10: promoters (9-10), detractors (0-6), and passives (7-8). The NPS score is calculated as the difference between the percentage of promoters and detractors. A positive NPS score is generally considered good (30% or higher), medium (0% to 29%), or negative when it falls below zero.
Image from customergauge.com
The NPS score for Capella was found to be highly dependent on the level of adoption.
The survey revealed that once Capella is adopted as a common tool for all MBSE projects within an organization, systems engineers are highly convinced of its value. Conversely, during the adoption phase, opinions tended to be negative, possibly indicating that onboarding to the tool can be challenging and requires expert support.
When asked about the number of projects on which Capella had been used, nearly two-thirds of the respondents (62%) reported using the tool on several projects.
Again, those who extensively used Capella (more than six projects) are highly satisfied.
We explored the types of projects on which Capella was used, and the results indicate that Capella can be applied in diverse contexts.
Analyzing the NPS score based on project types revealed high satisfaction across almost all kinds of projects, except for "Collaboration with partners & subcontractors" which scored below 30%. This lower satisfaction might be due to interoperability concerns when partners or subcontractors utilize other tools based on different approaches.
Then we asked a series of questions about the usage of the Capella tool itself.
Similar to 2020, the most used perspectives were System Analysis, Logical Architecture, and Physical Architecture. However, the End-Product Breakdown Structure perspective saw a decrease in usage, with only 5% of respondents indicating frequent use (-7% compared to 2020).
But the proportion of usage for different perspectives varies depending on the project types, with some perspectives being used more frequently (represented in green) and others less frequently (represented in red).
Regarding representations, the Architecture diagram is used by almost all respondents (96%). When compared to 2020, there is increased usage of Capability diagram (+4%), Modes & States diagram (+6%), Class diagram (+5%), and Interface diagram (+11%).
In terms of features, respondents were asked to select three features they found most valuable. The three most notable features are Functional Chains (66%) which are natively supported in Capella, Methodological Guidance (57%) based on the Arcadia method, and Model Navigation (47%) which allows users to explore the model and manage system complexity.
In addition to these on-the-shelf features, Capella users benefit from a wide ecosystem of open-source or commercial add-ons that extend the capabilities of the workbench. There a notable changes about their usage since 2020.
Three open-source add-ons have gained popularity: PVMT (+11%) and M2Doc (+19%), and Python for Capella, a newly introduced add-on that has been rapidly adopted (24%).
Conversely, the Requirement Viewpoint, which was previously the most used, saw a decrease in usage (-10%), although it still maintains a relatively high usage ratio (48%). And the Basic Mass Viewpoint experienced an 11% decline in usage, falling to 9%.
Among the commercial add-ons, Team for Capella is used by 32% of the respondents to facilitate collaboration among users. This percentage increases to 50% in cases where Capella is used by teams across multiple projects, and reaches 58% in organizations where Capella is the official MBSE tool.
Another commonly used commercial add-on (18%) is the pure::variants connector for Capella (50% of those using Capella for Product Line Engineering).
Some add-ons are provided through customized distributions integrating pre-installed extensions and customizations. 9% use customized distributions created by software providers and 29% developed in-house. However, the majority of respondents (62%) still utilizes the standard Capella distribution and installs add-ons manually.
MBSE in general
Beyond Capella, we sought respondents' opinions on MBSE in general, and its adoption in their organization.
To the question “How likely is it that you would recommend MBSE to a friend or colleague?”, the answer is positive, with a 38% NPS score.
To learn more, we reused some of the questions asked by Kaitlin Henderson (Virginia Tech) and Alejandro Salado (The University of Arizona) in their study “The Effects of Organizational Structure on MBSE Adoption in Industry: Insights from Practitioners”.
Respondents highlighted the improved quality of systems developed and better engagement and experience of team members as benefits of adopting an MBSE approach.
To facilitate MBSE adoption across organizations, a core MBSE team dedicated to this objective was present in most cases (64%).
This proportion increases to 75% in organizations with over 1500 employees and 84% in cases where Capella is the official MBSE tool.
Facilitating MBSE adoption also relies on resources available to the Capella practitioners.
The most frequently utilized resources are human-based, including training (30%), experienced peers (29%), and coaching (26%). Surprisingly, documents were mentioned by only 6% of respondents, despite the abundance of available materials on the topic (e.g., https://mbse-capella.org/resources.html).
This is a sign that MBSE is a matter of collaboration that requires exchanges and alignment between engineers. It’s not a discipline that can be practiced alone in an ivory tower.
In about half of the cases, these resources are managed by specific MBSE personnel, ranging from designated roles assigned to individuals dispersed across the organization (23%), to a separate MBSE team (21%), and even a dedicated MBSE department (6%).
Conclusion
The survey results confirm that Capella, and MBSE in general, is a valuable solution for teams and organizations embarking on systems engineering projects.
However, leveraging the benefits of the tool and the corresponding methodology necessitates human-based support to share knowledge and best practices. This is essential for acquiring and consolidating a collective experience within the organization, which can then be applied across multiple projects.
Successful establishment of this community of systems engineers - within an organization and across projects - leads to tangible benefits in project outcomes.
