Good Requirements Not Enough: OTE

Good Requirements Not Enough: OTE

The equipment for Son of FCS — most of it recently canceled by the Army — met the service’s requirements but did not meet the much more demanding standard of actually improving how soldiers fought, according to the Pentagon’s top operational tester.

This story appears to have much wider relevance than just the Army. Any service can draft what it thinks are sound requirements, but if they are not derived from and then embedded in how soldiers, sailors or airmen fight then the weapons which result can be irrelevant or much less useful than hoped. In these days of tightening budgets, few mistakes like that can be sustained.

“All of the E-IBCT system met or came close to meeting most of their documented requirements. The program office designed the E-ICBT systems to the specifications it was given and was largely successful in doing so,” Michael Gilmore, director of Operational Test and Requirements testified before the House Armed Services tactical air and land forces subcommittee yesterday. “Yet, in operational testing, these systems demonstrated little operational utility.”


With all the criticism of the Army’s development of requirements over the last decade, this might seem another classic example of what we might cattily call the Crusader syndrome. The Army develops a rock solid requirement for a weapon that it says it must have but can’t really manage to develop or build. But this case seems a bit different. The Army’s requirements could and were met. They just didn’t add much value to what soldiers do, which is fight.

Some of the reasons for the service’s failure, Gilmore said, is that, “the E-IBCT requirements document did not sufficiently link its largely technical specifications to desired operational outcomes. The requirements and specifications were necessary, but well short of sufficient, to assure military utility.”

And Gilmore makes it clear this is not just an Army problem: “This situation is not unique to the E-ICBT program. Program requirements must be operational in nature and clearly linked to a useful and measurable operational capability. Contract specifications must be both necessary and sufficient to assure operational effectiveness in combat.”

Gilmore gives the Army credit for trying to improve the systems after the Limited Utility Tests started in 2009. They led to “considerably improved reliability found in the operational test conducted in 2010.” But improved results could not mask the fundamental problem that the systems just weren’t worth deploying. If the Army had done “better developmental testing” before the 2009 LUT, Gilmore thinks the Army would have found out the limits of its systems sooner. And we could all have saved some money, time and institutional bandwidth.

 

Join the Conversation

In other words, specification compliance does not define satisfactory performance.

The U.S. Naval Test Pilot School has been teaching that for decades. Pity that so few outside the test community are willing to listen.

Gilmore is possibly trying to cover up for Army & DoD failure in corrupt leadership that puts corporate interests & congressional districts ahead of taxpayers, warfighters, and national security. This is another wakeup call, another damning indictment of DoD’s general officer corps being disconnected from warfighting requirements. How the heck does one become a general anyway? Is it by knowing how to accomplish missions, enforcing discipline, winning wars, and improving the national security posture of the United States, or is it more about schmoozing, politicking, and defending the defense industry & congressional leadership???

We would have greatly improved results in acquisitions if we started with what works in the field, and make incremental improvements. We could do things like incorporate better materials, develop alternatives for obsolescent parts, incorporate more reliable parts to improve system reliability, solve operational deficiencies in fielded systems, develop competetive sources for fielded equipment, and put our systems to rigorous test in realistic joint warfighting exercises. Instead, we keep on getting suckered into investing in moneypit silver bullets that promise to “lift the fog of war”. CHANGE. not the balogna Obama promise way. REAL CHANGE.

>Gilmore is possibly trying to cover up for Army & DoD failure in corrupt leadership that puts corporate interests & congressional districts ahead of taxpayers, warfighters, and national security.

Yea they are just pathetic excuses to cover-up the corruption — oh we discovered at the last moment after spending $20 billion nothing was worth a shit — complete baloney.

Just go back and read what the same people were saying about the FCS right up the cancellation — how well the operational testing is going and how much the troops like everything.

In the extreme sense, the problem with starting with what works in the field and improving it is that you end up with a ox cart with carbon fiber body and genetically modified oxen that can run all day. You have to have a balance between new technology and evolutionary upgrades. Of course, when you want to skip a generation of procurement, as happened after the Cold War, every new program shoots for the stars because there will be no evolutionary upgrades.

This isn’t a philosophical issue it’s one of deliberate project mismanagement and corruption. Not bothering to validate the design isn’t a mistake it’s corruption and should lead to a court-marshal and long prison sentences.

Not necessarily. I take it you are not an engineering genius. Say you have an effective system, say like F-16, Abrams, or Bradley, and you know what works, and what is deficient. You keep what works and you fix what doesn’t work. If you have to go all the way back to the operational requirements, then you keep what is good, and you change what needs to be changed. You develop systems smartly, develop competetive sources so we don’t get suckered in, and you don’t waste billions of dollars and decades of time on the garbage that we currently have. Shooting for the stars has its time and place in Basic and Applied R&D, however, when the warfighter is counting on a system to replace their 50’s-80’s era technology, we do not need to be wasting our time with crap like EFV.

You are correct. No engineering/acquisition strategy/policy will work if the leadership is corrupt. You are correct in that the root of the problem is Core Values. In the 90’s USAF studied this and developed Core Values Integrity First, Service Before Self, Excellence in All We Do. Then its leadership practiced the opposite of integrity first. Organizations and people can sell all their gimmicks and promises, it all boils down to integrity. If you build systems upon foundations of lies, they will collapse, and you will drag our nation into the slimepit along with your corrupt decision making. Eat it, CMIC.

yup. the position of DOT&E was created so the People (thru Congress) would have independent, objective analysis vs the lies, evasiveness, and stonewalling they usually get from uniformed general officers. DOT&E is definitely a step in the right direction, but you can see that any position under the taint of the Pentagon gets dragged down by the inertia of the Building. Thank God for PA&E, GAO, OMB, and independent watchdogs, or we would never have a hope of escape being the victims of their force feeding garbage. Terminate the chaff, keep the wheat, expose the liars, praise & reward the producers with actual integrity. We need miracles, big time. 2012. Another chance for the people to take back this country.

Like I said, sounds like this DOT&E is trying to save the Army’s face a little bit. OT&E IS involved from the get go. They inform the USD (AT&L) of the sufficiency of the TEMP. Usually their voice is disregarded or they are shouted down. The USD (AT&L) is under intense pressure to green light systems, even if he knows they are doomed to fail. However, all Services know they have to make DOT&E happy at some point, it’s foolish to even begin concepts that are at high risk to not satisfy DOT&E.

If the Army actually learned something useful from the FCS experience, something that could actually be applied to next generation force development, I’d be a bit less cynical than I am right now. Saying in effect that the requirements don’t matter is just about the worst thing you can do to an acquisition system that is, well, stressed. To put it mildly. Who gets held responsible for the requirements being wrong ? Where is the post mortem — and what are you going to do to make it right ? What I see in the 2009 changes to DoD 5000.2 makes things absolutely worse — there is LESS time committed to getting the requirements right up front, there is MORE hand-waving about risk and cost, when in fact all you are doing is trying to transfer both from the government to industry. Is this any way to run a railroad ? No it is not. So what you end up with — and DOT&E will be very happy with this — is more bent metal going nowhere. Pretty soon you’re back to ACTDs and nothing but leave-behind prototypes. Nothing goes to production by working hard and playing by the rules.

PA&E (now called the CAPE, from the merger of PA&E and the CAIG) is part of the problem. They approve every new technology program and don’t weigh honest costing. GAO have point out problems, but always after te fact, and they are toothless in doing anything because their own bosses (the Congress) is invested in programs in their districts. OMB set top level funding and they lets DoD run amok. Also toothless. What independent watchdogs? The ones who mistake buying more of what the military needs buts calls it pork?

Fixing this acquisition problem requires leaders with real values and the ability to kick some butt. And it requires firing both military and civilian program managers, and a whole bunch of CEOs.

2012? You referring to the end of the world (Aztecs) or the desire to replace an ineffective Democratic politician with an ineffective Republican one? We’ve done this before. Nothing changes after the election. What we need is new political parties that represent something more than just an emotional outburst of frustration.

Ultimately the senior leaders like SecDef, USD AT&L, and the Service generals are responsible for the requirements being wrong. As far as working hard & playing by the rules, tough. If we let people get away with that, the whole enterprise goes down. This is in fact what we have today — we have a system that rewards cheaters. Actually it’s not the system — it’s the leaders that manage the system. The culture is broke, it is corrupt. It serves the needs of congressional districts, industry profits, and senior leaders egos, reputations, and careers. Break The Cycle: Lower risk MDAP programs. Buy more COTS & NDI’s so we know what the costs are and can get systems fielded sooner. Accept the limitations, then modify in follow on development programs.

Here’s an idea. We could at least for one moment pretend like we are capitalists. The Army could announce an up coming competition for a rifle or radio. Tell the vendors in general terms what they want that thing they are planning to purchase to do. Tell them how many thousands or millions of them they want. Tell them the date, place, and minimum number of test articles to provide. They could go off and test them. Rank the competitors in order of performance. Negotiate the best product for the best price. Fire all the procurement drones and bureacrats and specification writers you hired back in the days when we forgot what capitalism is and how it works, and go on from there. It’s not f’ing rocket science.

Yeah, no system is perfect, and they managed to buy the worst option even back in the days when they procured weapons this way, but the system we are using now is total crap. Paying a profit on development is just asking to be screwed and these defense contractors are all lining up for a turn at us. Let’s go back to a system that at least worked most of the time. It would be easy for the Army to do this. Most of what they buy is cheap to develop and they buy in large quantities so that the profit percentage required to keep the supplier companies in business is fairly low. It’s really a no-brainer.

OK, so the US Army has not successfully developed a major aircraft or ground combat platform since the Carter administration. And successful is defined as producing something after significant cost and schedule over runs that is eventually produced and needs significant modification once it is fielded. Acquisition reform was defined to prevent the Bradley, Apache experience. So for the Army, Acquisition Reform has prevented new platforms from being produced and fielded.

TRADOC has lost the ability to define operationally effective requirements and trade those against achievable and cost.

RFPs are nutty take it or leave it propositions. Contractors are eliminated from the competition if they don’t propose a 7500lb, $7.5M stealthy attack helicopter or a ground combat vehicle that robotically refuels, rearms, rewaters and provisions itself under armor.

TRADOC is always surprised when its programs become bogged down with the unachievable or useless. TRADOC confuses what it thinks it needs for what is achievable or useful.

Maybe somebody should ask Dr. Gilmore why he spent $7.0 million dollars of taxpayer money on analyzing the results from LUT 10 when ATEC produced findings that indicated that all EIBCT systems exceeded their CPD requirements by a factor of 2 and improved their MTBTSA by at least a factor of 5. Military utility findings are qualitative and DOT&E can be influenced by those that control its funding…

sounds like you are the best person to ask that question. lest us know how it turns out.

right on point. Army is one jackass stubborn institution. never admit a mistake. that’s why they keep repeating them. all the Services are like to this to some degree, but the Army’s a piece of work.

The Army doesn’t learn something until it is front page news, Congress drags their embarassed faces into the Capitol, and they have to hire the most outside independent expert imaginable tog et to the truth. check out the Gansler report.

don’t agree with you on the contract fee thing, but do agree with you on the value of buying non-developmental items. nothing wrong with paying contractor profit on development. the fee/contract structure is much less the problem than the overall corruption and incompetence of a lot of people, starting at the top.
one of my theories is that the Army suffers from some kind of technological penis envy of USAF and USN. let’s face it, if Army engineers were truly talented they’d probably work for USAF/USN. The Army does not cultivate an acquisition officer corps. we got guys leading brigades one day and leading contractors the next. did you read the Gansler report on the Army contracting fiascos? One colonel said he had field grade responsiblities with L.T. qualifications.

Well, how’s that “paying profit on development” thing working out for ya’? Seems to be sucking pretty badly to me. They could go back to what worked for them, or they could continue to do the same thing that has failed miserably each time they’ve used it, each time hoping for a better result. Given that it’s my money they’re f’ing away, I’d like to see them go back to what worked. Just me thinking again.

The Army doesn’t need to buy “commercial off the shelf” crap to make an old fashioned “profit on delivered goods” type of contract work for them. They can buy to MIL-SPEC, let the potential contractors cover the development costs and buy the best weapons for the best price just like any other business that buys stuff in huge quantities does. They have no reason at all to imitate the USAF. In fact, they should be showing the USAF the way, not following their lead straight to hell.

Part of what’s missing from the development process is the ability to clearly link MBTE Vignettes (operational scenarios) to Capability Needs (Gaps) and a Program. This is due, in part, to the chasm that exists between TRADOC and the materiel developers. It is also due to the sheer complexity of these problem domains — understanding and visualizing the relationship between the operational requirements, the capability gaps, and the programs themselves is a daunting task (on a good day). Here’s a link to a program the Army is beginning to use that helps address this problem: http://​www​.commercebasix​.com/​w​o​r​d​p​r​e​s​s​/​?​p​=​407 Not a perfect solution but a start in the right direction.

there’s no such thing as a perfect solution. In DoD’s vain mission to find ‘perfect’ solutions, DoD burns money and time that could be applied to manage other problems. DoD ends up not accomplishing the ‘perfect’ acquisition solution, and ends up not accomplishing the real world missions either. But the burning of money and time — that mission is accomplished.

*required

NOTE: Comments are limited to 2500 characters and spaces.

By commenting on this topic you agree to the terms and conditions of our User Agreement

AdChoices | Become a fan on and follow us on
© 2013 Military Advantage
A Monster Company.