Not a good thing because these processes may sometimes generate results which are either unsuited, unanticipated, expensive to remediate or sustain, or confusing to the operators of the fleet. In short, these processes, if not carefully executed, may have serious effects on the Readiness of the DoD war-fighting abilities.
Still, this lack of attention regarding the process is many times a function of system complexity instead of indifference: The entire process, end-to-end, is disjointed and confusing and, because the key actors speak different languages and operate under different incentives, the entire process is difficult to fully understand, even to those who are closest to the action.
So, what’s the deal? A key first step is recognising that we shouldn’t mob the ball and that there are multiple pathways and tools that we should use in the right combinations for the challenge at hand. Use experience and critical thinking to select the right methods amongst the full range of options and provide the rationale to the acquisition decision makers.
Policy and practice both agree it is vital to think critically about the task at hand and then pick the right combination of tools and methods. Otherwise, we mob the ball on a certain approach even when not appropriate, resulting in an inevitable backlash from senior Defense policymakers and Congress.
For senior leadership, don’t lock in on a single method – it doesn’t lead to success. Let your staff know that you’ll open to approaches beyond the flavor of the month, and that you’ll tolerate some risk if they do their homework by bringing a logical and well-reasoned program approach that fits the challenges we are up against.
DoD has stood up a new Futures Command. to show they are serious about ending years-long, expensive modernisation programs and unclear requirements. While failing early and often can prevent some of those issues before an requirement becomes a program of record, there are still some risks that, by trying to be a more quick and innovative enterprise, the command could still run into cost increases and drawn-out timelines if it jumps too hard on emerging technologies,.
“There are a variety of ways to fail when it comes to developing these technologies,” the way to look at it is, you can make sure that you have it right, or believe that you’ll get it right.”
DoD has applied many best practices including close collaboration with senior leadership. But there are always parts of the plan that could be improved. Plans to develop weapons systems with emerging technology could come back to bite.
Proceeding into weapon systems development at earlier stages of technology maturity raises the risk that the resulting systems could experience cost increases, delivery delays, or failure to deliver desired capabilities.
It may not be the best idea to create programs around emerging technologies, before having a chance to test them in an operational environment.
“As DoD identifies the capability, there are technologies used to achieve that capability. There is an advantage to maturing those technologies before you begin what’s called a program of record.”
DoD is already required to test in a “relevant” environment but a training situation, for example, might not give a good enough gauge of whether something will be useful.
When you test something in an operational environment, you might discover the technology isn’t as mature as you thought it was.“ And if you’ve already stood up a program for it waiting for that technology to mature could delay delivery. And as DoD has learned, what they have in hand at the end might already be outdated.
To help encourage this mindset in the acquisition workforce, leaders launched a collection of digital tools that illustrate the full range of contracting methods. As programs develop their strategies the goal is to discourage acquisition mob ball and instead encourage acquisition professionals to apply the strategy, discipline, and other skills necessary to play the game well.
Military executives tend to grab onto a hot new contract approach and mob the ball as soon as the whistle blows. It’s the only thing they focus on, despite what their coaches told them about playing positions. If by chance the ball gets kicked free, the mob moves to another part of the field and again surrounds the ball.
Eventually, they learn to play proper positions and be in the right place at the right time for the needs of the game. They begin to actually play and develop strategy, discipline, and teamwork skills. They improve their game, and it’s a lot more fun for everyone. But then some of them join the acquisition workforce and start playing acquisition mob ball.. This is frustrating for everyone.
“Engineers” in the services are responsible for the acquisition of systems to equip the fleet. They are the organisations to interface directly with industry to develop and build new systems for the fleet.
Requirements, or descriptions of systems are dished out along with the funding necessary but often the specific intentions undergo unanticipated “translations” once they are delivered.
Officers responsible for the actual process of making things undertake a difficult of voyage of discovery, often ranging far beyond what may have been the original expectations. As long as the top-level requirements are met, at cost, much of what will happen in this part of the service and then in industry, is typically over DoD staff’s “radar-horizon.”
Two factors drive the command, which stand far above any other considerations; cost/schedule but it must translate into high performance in the field when they are finally in the war fighter’s hands.
Many in DoD who are involved in this process, are fully occupied by considerations related to “the budget,” acting as the primary interface between DoD and Congress who always provides changes to the funding up to the very last moment, meaning that not only is it difficult to plan the large movements in the acquisition process, but it is also impossible for the command to control execution..
To be plain, the fleet itself is not well connected to this process, and as far as any involvement in the discussions regarding the actual building of systems, it is outside of the loop. There are few if any mechanisms which allow the operational forces a voice at any point in the process.
Industry presumes that the engineering shop speaks for the Fleet. This is not the case. Further, there is no sensible inquiry into what the fleet may want or need. Nor is there a considered methodology for gathering feedback from front-line operators. Writ large, the operational chain-of-command, which might actually have a sensible, real-time opinion on these material topics is not directly connected to those making the decisions and far removed from those building the systems.
Operating forces are supposed to be represented in this process by the respective Type Commanders, but these organisations are only loosely tied to the real decision makers who reside at the Pentagon. To be fair, in some communities, the relationships, authorities, responsibilities that exist between specific Type Commanders and their respective advocates in the Pentagon are close and well defined. In others, they are far looser and less specific.
The one agency technically charged to represent the fleet is Fleet Force Command, which produces a document called the “Integrated Prioritised Capability List” of the combatant commander’s highest priority requirements, defining shortfalls in key programs which impact capability of commanders to accomplish their missions.
But the list is disconnected from the real churn in Washington; the ongoing, minute, minute-to-minute interplay which takes place between command and Congress. Ultimately, the List is too broad in addition to being brief, and it ends up, in the words of one flag officer: “little more than a top-level restatement of the obvious.”
Once the necessary funding and requirements has been provided, to a certain degree, command loses track of the engineering activities. These requirements are further developed, and then the radar is put out for bid by industry via a Request for Proposal. It is in this process that things become even more disconnected from what the fleet would want and technically speaking, only be a few layers deep.
“Cross-functional teams are a really good thing. We have a whole list of leading practices for these teams. And, really, our concern there is just that the services continue to do some of the things that they’ve been doing in those pilots.” Everything that’s gone well, and also what hasn’t, should be recorded and preserved according to the team report, but there is no agreed upon process.
“Getting those down and communicating those as this new command stands up is going to be vital, to make sure that they’re not reinventing the wheel or committing the same mistakes as they stand up this new command.
“The directive that set up the cross-functional teams said it would be a good idea to get the lessons learned,’ but there was nothing formal in place-- the leaders response was, absolutely, that’s something we need to do."
Every program has a unique set of characteristics, circumstances, and goals. Best acquisition strategy practices are very clear that program managers and contracting officers should pick the right approaches and contract types for the challenges at hand.
The contractors usually do the deep work necessary to build the weapons system and the results may be unexpected. In short, what comes out may differ from the expectations elemental to the entering concept This is the price of submitting requirements which are insufficiently detailed. Also, this is the cost of the engineers operating without the direct and deep involvement of line officers.
As for the various industrial competitors for the contract, they understand that their respective offerings will not only need to meet the stated, top-level requirements but will have to do so at a price point which will be in line with both Congress and DoD desires regarding cost and schedule.
Leaders are launching a series of pilot programs to try and reduce the time for getting companies on contract by setting up a schedule of regular events with key industry partners, and aside from those scheduled events.
We want to look at suppliers from a portfolio point of view and not just get caught up in each program.” Then a series of direct dialogues between industry executives and DoD bosses to try and move faster in researching and engineering new weapons systems.
Ultimately, someone wins the competition as judged by a selection board, not inclusive of fleet-personnel. The winner will then embark on the actual design, and eventually construction of the weapons systems with varying degrees of oversight occurring at key Project Milestone Decision Points.
The defense acquisition system uses three milestones to oversee and manage major defense acquisition programs:
Milestone A decision:
Approves program entry into the technology maturation and risk reduction phase, which involves developing technologies and reducing risks before committing the resources needed for complete system development.
Milestone B decision:
Approves entry into the engineering and manufacturing development phase, which commits acquisition managers to developing a specific system for production and fielding.
Milestone C decision:
Approves entry into the production and deployment phase, during which the contractor produces system units for fielding.
As industry works, unanticipated technical issues are discovered and costly schedule changes are introduced. For example, DoD timelines are altered, or unanticipated technical issues arise. Trade-offs are then decided upon and some of these may have significant end-product impacts. Unfortunately, these trade-offs are usually not coordinated by key DoD principals and are never known to the fleet.
Beyond that, many of the daily, critical decisions regarding production are taking place far below the level of the few DoD officers assigned to the project.. As competition has increased in recent years, cost has become increasingly critical in order for industry to win key contracts.
One issue not receiving enough attention is the experience levels of many of those working on military contracts has declined in recent years: Experience is expensive, and to win competitions for work, it is often necessary to sacrifice experience.
Finally, there is a technology factor at play in this process. Industry and DoD engineers sensibly want to keep moving forward, technologically speaking. It is more profitable to industry and more interesting in general to those who live in the material world, day-to-day.
Driving down technical risk before entering full-rate production helps bridge the gap between a promising technology and a full-rate production program by maturing technology and developing initial concepts of operation before committing to a major acquisition program.
In industry, new technology has a higher profit margin and promises the potential for additional sales. In the military, it is essential to maintain a technological air gap between our forces and those of potential adversaries.
However, it is one thing to provide a high-tech solution to the fleet, and it is entirely another to support that system through training and in-service support.
Front line units are overwhelmed with new technologies, which may or may not be a net plus, regardless of their newness. The problem here is that while there is a large constituency for new systems, there is very little incentive associated with the support of existing systems.
While all of this may seem complex, the truth has more variables than represented here. still, the point is that the entire process is driven with contradictions and conflicts. Whatever may be the solutions to these problems, they are well worth considering. To not consider them while focusing on officer appointments and rotations seems like standing in a lake full of alligators and worrying about the mosquitos.
"We've got people in there who know what they're doing. It's time to roll up our sleeves and get on with it."
“It’s all about velocity. We’re trying to experiment. We are trying to prototype. We are trying to iterate and get stuff downrange quickly.”
Myth Busters
Myth: All defense acquisition programs have large cost growth.
Reality: Cost control has improved significantly.
Not only is cost growth significantly lower than historical levels, but recent efforts have dramatically lowered cost growth further. Multiple measures show statistically lower cost growth on major programs. Historical analyses also show that cost controls are better than in decades past We do still have legacy problems on older programs. Total research, development, test and evaluation [RDT&E] cost growth is still rising due to older programs. We need to do better through continued incremental improvements, but recent improvements focused on acquisition fundamentals and an empowered DoD workforce have been more successful than hands-off acquisition reforms.
Myth: Defense programs usually cut quantity e.g., to pay for cost growth
Reality: Most major programs deliver the original baseline quantity or more.
We don’t as a rule cut program quantity. Most major defense acquisition programs actually produce the quantities we originally planned at Milestone B. This runs counter to the impression given from just focusing on certain high-visibility programs that incurred major cuts in quantity.
Myth: Swings in O&S cost estimates indicate poor program management.
Reality: The dynamics of cost estimates indicate that O&S costs appear to be heavily driven by external inflation factors.
Analysis shows that the recent dynamics of program O&S costs estimated during acquisition correlate with the dynamics of labor, fuel, and maintenance costs. While this aligns with intuition, it also indicates that O&S cost increases involve both factors that the acquisition system cannot control as well as some that can, in part be controlled [e.g., system reliability, fuel efficiency, maintenance]. Operational tempo also affects O&S costs through many of these factors [e.g., the amount of fuel consumed and maintenance costs], and changes in forecast tempo will affect O&S costs independent of both inflation and weapon system performance. Thus, while the acquisition system needs continued attention to the levers it can control with full knowledge that their effects often will not be seen for decades. Stakeholders need to recognise the strong influence of other factors on O&S costs.
Myth: Program requirements are unstable.
Reality: High-level requirements seldom change on major programs, and very few programs have many changes.
Most Major Defense Acquisition prorams showed no changes that we could trace from the original baseline to the latest Selected Acquisition Report for the program. Moreover, of the few programs with any traced changes, most had only one. This is commensurate with experts’ experience indicating that changes are largely made at the engineering level as development seeks ways to meet high-level requirements. Changes, however, are not always bad. Some changes reflect prudent requirement reductions to unforeseen high costs of options uncovered in development or new affordability pressures. Other changes address new threats that otherwise would render an unmodified system obsolete upon delivery. Thus, flexibility, measured response, and continued tradeoffs, together with smart management attention to cost implications, are more important.
Myth: The DoD cannot acquire systems quickly.
Reality: DoD acquisition can be timely and responsive.
Despite criticism that defense acquisition is too slow, the highlights below show that schedule growth is lower than cost growth in development, and cycle times for major programs have not increased much from decades past even while challenged with dramatic increases in weapon system complexity. This is not to say that internal processes cannot be improved, so efforts continue to institutionalize streamlining and smart decision making .
Myth: Defense acquisition is broken.
Reality: The acquisition system for decades has given DoD the most capable military in the world and has been improving both in the past and more recently.
While there is no absolute definition for sufficiency, annual performance report indicate system functions reasonably well compared to the past and continues improving. We cannot look at a single metric to measure the performance of the defense acquisition system, and many metrics work at odds with each other. For example, the so-called “iron triangle” of cost, schedule, and technical performance has long shown that emphasising one or two dimensions often is done at the expense of the others. While cost --followed by schedule-- metrics are the easiest to quantify, we find all three dimensions indicate stability and, in many cases, significant improvement.
Systems engineering can be defined as an iterative process of top-down synthesis, development, and operation of a real-world system that satisfies, in a near optimal manner, the full range of requirements for the system. Systems engineering teams must work together on a set of inputs to achieve the desired output where the output is a system/capability that meets the user’s needs and requirements in a near optimal manner.
Must elicit requirements from customers and potential product/service users, validating and prioritising customer/user requirements, Defining requirements in a manner that is executable and verifiable, Identifying alternative solutions to achieve requirements, Isolating balanced and robust solutions that “best” meet requirements, and Verifying implemented solutions satisfy requirements.
Systems engineering must account for the entire life cycle of the system/capability acquisition. The life cycle functions that systems engineering accounts for are development, manufacturing/production/construction, deployment/fielding, operation, support, disposal, training, and verification. Systems engineering ensures that the correct technical tasks are accomplished during the acquisition process through planning, tracking, and coordinating phases.
1. Apply systems engineering approach balances total system performance total ownership costs within the family-of-systems, systems-of-systems context
2. Develop systems engineering plan approval describe program overall technical approach, including processes, resources, metrics, and applicable performance incentives.
3. Detail timing, conduct, and success criteria of technical reviews
4. Develop total system design solution balance cost, schedule, performance, and risk,
5. Develop/Track technical information required for decision making,
6. Verify technical solutions satisfy customer requirements,
7. Develop cost-effective/supportable system throughout the life cycle,
8. Adopt open systems approach to monitor internal and external interface compatibility for systems and subsystems,
9. Establish baselines and configuration control
10. Create focus and structure of interdisciplinary teams for system and major subsystem level design.