Data Dump For YouTube Troopers

Data Dump For YouTube Troopers

Troops in Afghanistan are about to get a big help called Valiant Angel for sorting and moving the massive amounts of still and video imagery collected by dozens of aerial drones and other aircraft orbiting the region via the intel shop at Joint Forces Command. Troops and intelligence analysts are simply overwhelmed by the vast amounts of drone supplied digital video that is served up daily in massive quantities.

Digital connections worsen the farther one travels from large, fixed headquarters. Small units in isolated combat outposts, for example, are often stuck with the equivalent of a 56k dial-up connection; they lack the big data pipes to tap into video feeds or access the massive images stitched together by wide area surveillance platforms (WAS).

WAS aircraft, such as the Army’s “Constant Hawk” or the Air Force’s forthcoming “Gorgon Stare,” carry large suites of cameras that snap photos of areas up to four-by-four kilometers in size at two frames per second. WAS aircraft orbiting Afghanistan can easily spit out 7 terabytes (7,000 gigabytes) of data every day, and that’s just from one aircraft, said Air Force Col. Skip Krakie, on a conference call with reporters Wednesday.


Before Valiant Angel, one of the only ways to get that massive amount of data back to intel analysts in the U.S was to ship on a plane.

The solution was to turn to the commercial video industry and buy largely off the shelf equipment, he said. “The commercial industry struggles with these same problems as well, how do you move large digital data around so you can stream it to somebody’s computer or television.” The Valiant Angel program doesn’t build anything new, rather it bundles commercially available hardware and software into something troops can use to work their way through massive volumes of digital data.

Drone video is fed into Valiant Angel hard drives and then made accessible to users, even those in remote locations, over the military’s Secret Internet Protocol Router Network (Siprnet). Troops can then use a JFCOM provided “Thick Client” software package to view video and images and discuss a video clip using instant messenger in a chat room. The software captures that chat and embeds it into the video stream, along with important key words, such as “IED explosions,” allowing end users to search for relevant streams, Krakie said.

JFCOM is also developing a web interface for users that might not have access to the software package. Video clips will also be tied to a “Google Earth like” interface, so they can see where the drone was scanning. Embedded in the software will be “YouTube like” instructional videos.

To collect and make useful WAS imagery, a data volume that is “almost incomprehensible,” Krakie said, JFCOM is providing a network operations center, co-located with Constant Hawk or Gorgon Stare ground control sites, that downloads the imagery and makes it usable to a much larger audience that might not have the big pipes. Today, only about a dozen analysts are able to view this data. “We’re going to expand that to everyone who is on the network,” he said. Units will be able to scan imagery of ground they might patrol for any signs of insurgents digging in IEDs, for example.

Lockheed Martin is the prime contractor on Valiant Angel and it will be shipped to Afghanistan within the next two weeks, Krakie said. A 60-day assessment will follow, then the remainder of the equipment will be shipped over in late summer.

Tags: , , ,

Join the Conversation

Sounds like a great system, but for the life of me I can not understand why we are feeding this info to our enemies. Loose lips sink ships policy must be a thing of the past. We need to stop giving out info to the enemy intelligence or the troops will pay the price on the ground.

God Bless Our Troops

I guess I’m missing where we give it to the enemy. I think you still have to have a SIPRNet account — and not everyone gets one.

But then, I may have missed relevant info?…

It is so nice, to reach out and touch someone.

Really, they are just not figuring that out? And only 56k connection for this stuff to move? Really I get better than that on my portable satellite dish. Even public networks can be made pretty private with some encryption inside the packet.

The idea that they are just now after years of collecting data at huge rates asking how everyone else moves that data is just silly. Broadcasters have been doing this exact same thing for decades, often encrypting the data so that doesn’t even need to be added.

Beyond all that, the in ability to manage the video and pictures in a way that doesn’t require the shipping around of Petabytes of data a day is just sad.

I am not suggesting I have all the answers but it does seem like someone was asleep at the switch if I can come up with a half dozen ideas in a few minutes and don’t even work in the industry.

nraddin — compressed video isn’t good video. Compression brings loss of detail and detail is what they need. Thus large file sizes. You are right about the rest of your thoughts.

Almost right. There actually are lossless compression algorithms for both video and for still photos. Problem is that their compression rate isn’t very good so there wouldn’t be much point in doing it.

Personally, I’d still use something high quality lossy compression for most of the data and allow the end-user to request lossless data for details of areas of concern. Some data is better than none and with some of the lossy compression methods you can get significant compression with virtually indiscernible loss of quality (to the naked eye). The biggest problem I see with that approach is that doing the high quality compression takes significant computer time so someone somewhere is going to have to have something more than your average PC to compress that data down to something manageable but still useful — and that will likely need to be in-theater.

FWIW

There are lossless forms of compression, they don’t as good a ratio as other compressions but there are lots of good way to compress stuff without loosing any data at all. Much like zipping up a file makes it smaller but the file at the other end still works just fine, you can compress video. if you couldn’t compress video without loss you could never view a 1080p video (~500mbit per sec) because most HDDs can’t read data off the disk fast enough. A HD 1080p video renders around 24frames a sec or ~50million pixels a second, or a picture 7054x7054 with a color depth of 10bit (~ 1 billion colors).

That’s a great point HasBeen. The ability to only pull the data you need is key, looking a google earth for example you do not load all the data at once, just a very rough outline. Only once you start to zoom on that area does detailed data of that area even load. They did say something about Google Earth like interface, we can only hope that take that load last zooming technology with them.

Just because numbers are kind of my thing. I think I should point out that a 90min HD movie has more data in it as a 1pixel per meter map of the all of Afghanistan. It’s around 25gigs in size uncompressed.

Unless you work closely with a Hollywood cinematographer, all the 1080p video you have ever seen is heavily compressed with losses that change the details, but still look decent most of the time. Changing the details is sometimes not good for surveillance.

I once worked with an image compression format called MrSID, which is great because it stores the image in successively higher resolution/quality, so you just stop reading it when you have the resolution/quality you need.

The storage capacity of the Valiant Angel system is 16pb. The feeds are live and are commented on live with chat metadata in real time. This article fails to mention the low-res conversion feature of VA to allow the feeds to be uploaded to a low bandwith device to a soldier in the field with a PDA or something of that nature.

*required

NOTE: Comments are limited to 2500 characters and spaces.

By commenting on this topic you agree to the terms and conditions of our User Agreement

AdChoices | Like us on , follow us on and join us on Google+
© 2014 Military Advantage
A Monster Company.