In a final five-minute flurry, MarmotE watched its lead slip. At the buzzer, the team placed second and won $1 million in the DARPA Spectrum Collaboration Challenge.
The team of four current and former Vanderbilt researchers have worked together since 2016 to create an AI-radio system that can manage the wireless spectrum, finding unused slices in its rigid bands to jump to for more bandwidth—collaboratively with other users—in complex scenarios created to test the teams’ systems. MarmotE won $1.5 million in preliminary rounds, taking the top prize in 2017 and second place in 2018, bringing their total winnings to $2.5 million.
The SC2 champion would be the team whose AI-managed radio system collaborated best when matched up with the diverse lineup of systems other teams had built and brought to the finale.
“It was surprising to see how few matches there were in the final round and how much depended on the very last one. To quote our team leader Peter Volgyesi, ‘We simultaneously won and lost $1M in the last 5 minutes,’” said Sandor Szilvasi, PhD’14, a radio frequency and FPGA (field-programmable gate array) engineer in Atlanta and former research assistant in Vanderbilt’s Institute for Integrated Systems.
The live-streamed championship at Mobile World Congress 2019 on Oct. 23 was the culmination of the agency’s three-year competition that brought 10 finalists to the floor of the Los Angeles Convention Center. The championship featured dozens of matches in five rounds where teams were eliminated based on scores and performance. The eventual $2 million first-place winner, University of Florida’s GatorWings faced elimination in the final round.
Building a superior radio
“Our entire team owes a lot to Miklos Maroti, the main architect of our radio and our mathematician. He has the brain and the stamina and would not settle for a ‘good enough’ solution,” said Peter Volgyesi, a research scientist at the Institute for Software Integrated Systems and a lecturer in the School of Engineering’s Department of Electrical Engineering and Computer Science.
“I was glad to see how especially well the MarmotE radio performed in the Wildfire scenario, which mimics disaster relief when communication infrastructure collapses and an ad-hoc system should be deployed quickly and efficiently,” said Peter Horvath, a former postdoctoral scholar at ISIS and now an associate professor at Budapest University of Technology.
“It also was nice to see Miklos’ excellent modem design shine when compared to LTE (4G mobile standard), still a state-of-art communications standard, in the same scenario. Overall, the MarmotE design was among the most performant, well-balanced radio designs, always finishing in the top half of the league,” Horvath said.
“Designing scoring functions well aligned with the overall intent of SC2 was definitely the challenge for the [DARPA] organizers. Properly identifying what strategy those really boil down to in terms of radio design was the one for us,” Szilvasi said.
Volgyesi said the team didn’t target the peculiarities of the complex scenarios during development. “We focused on a radio solution that performed equally well in almost any situation. I believe we showed a very steady performance throughout the matches and during all the competition years.”
The team open-sourced the modem portion of the radio design prior to the championship event, making it available to the general public. They are talking about ways to use the radio in academic and industrial settings, with a “focus on academia.”
In real time for the first time
The championship event allowed the 10 finalists’ radio systems to duke it out in real time for the first time. The head-to-head matchup was made possible by DARPA’s efforts in relocating Colosseum, the massive radio frequency emulator at the heart of the competition, to the Los Angeles Convention Center floor just before the event.
Teams earned points in five rounds of matchups based on how many radio connections their systems could make during the scenarios. The catch: If a competitor stifled one of the other teams and hogged too much of the spectrum, everyone would be penalized and no one would earn points. At the end of each round the lowest performing team was eliminated.
The rounds were dubbed Alleys of Austin (a squad of soldiers in a concentrated effort to sweep an urban neighborhood); Off the Cuff (disaster relief agencies setting up operations just after a natural disaster); Wildfire (wildfire response scenario where radio communications with the important air tankers had to take precedence over everything else); Slice of Life (each team’s radio acted as a Wi-Fi hotspot for a different shop or restaurant in a mall); and the last round for the live finals, Trash Compactor (radio systems were forced to share an increasingly narrow slice of spectrum.
Trash Compactor was challenging as the five remaining teams—MarmotE, Erebus, Zylinium, Andersons and GatorWings—struggled to share the spectrum as the available amount disappeared.
“Off the Cuff was a very good scenariano for MarmotE. We got 40% more points than the second best team,” said Maroti, an associate professor at the University of Szeged, Hungary, and a Vanderbilt research associate professor in electrical engineering and computer science. “Wildfire was clearly our best scenario. We managed to close a very difficult link between a simulated air tanker and ground crews, scoring a lot of points while other teams did not even try this.
“Trash Compactor was an interesting scenario. GatorWings almost got eliminated in the elimination round,” he said.
“The first match was the scariest. However, the moment we were certain we made to the final five, I think we turned from being scared to sort of euphoric,” Horvath said. “This sort of high-stakes event is somewhat of a moonshot: there is only one chance, there is no room for mistakes and unexpected things like last-minute surprises from competitors or organizers can happen. Thankfully, Peter [Volgyesi] insured that no last-minute bugs appeared in the design and the surprises from other teams had limited impact on the final scores.”
Szilvasi added, “MarmotE always placed in the top three during elimination rounds where scores are based on results from 10 to 40 matches. We never even got close to being eliminated. It would have been interesting to see more scenarios that we prepared for but didn’t get featured in the final event.”
What about strategy?
“We always helped the weakest team in the ensemble based on their reported scores. We placed special attention to go in with full power and use all available spectrum when other teams were over the bonus line, however we pulled back if the others were below the threshold,” Maroti said.
Volgyesi said strategy about changing behavior “kept us awake” during the last phase of the competition.
“How do we change our behavior between being altruistic versus full-on competitive. The first choice was essential for surviving the elimination rounds, whereas the second behavior was the winning strategy in the final rounds. Fortunately, we could build on a very robust and high-performance radio engine and I believe we managed to find a good balance at the end.”
The scoring rules evolved over the three-year competition.
“The last phase introduced elements that called not purely for cooperation but also for competitive behavior. Precisely interpreting the general rules and especially addressing a paradoxical requirement on behavior is something we discussed several times a year, and that led to crucial strategy changes even in the final weeks,” Szilvasi said.
Euphoria and relief
Personally, they are all happy it’s over.
“The three-plus years of the competition undoubtedly brought us closely together and it also brought the competing teams close to each other,” Szilvasi said. “Throughout the live event it felt like we were cheering not only for ourselves but also for our competitors, which was a rare, positive but slightly odd experience.”
Volgyesi is glad the intensity is over. “We could work almost around the clock during the most intense periods of the competition. We had team members in Hungary, Germany, Atlanta and Nashville, so we often said, ‘It’s always midnight somewhere.’ Although separated in space, we became very good friends during this time.
“I am ready to take up any new challenge with these guys,” said Volgyesi. “However, I’d like to enjoy the split second before our success becomes a force that pushes us forward.”
A potential challenge for some of the successful SC2 participants is the opportunity to continue their work with support from the National Science Foundation’s Platforms for Advanced Wireless Research program, Volgyesi said.
In the end, they all wanted to thank their families for their support and understanding. “I’m very happy it is over,” Maroti said. “Luckily, we are all still married.”
Contact: Brenda Ellis, (615) 343-6314
brenda.ellis@vanderbilt.edu