There were no Avengers at the time. They were simply a rag tag group of people. Directionless, without true motivation to fight back. Manipulating them is easy.

But eventually, they came together and defeated me. But not before I destroyed a city and brought an army to wreck havoc.