Swordsman

And they proceeded to bum around for a while.

Barton learned how to use bows and arrows and swords as weapons from the guy who was at the carnival or at the circus, who was actually a supervillain known as Swordsman.

But Barton was kind of in the wrong place at the wrong time when the Avengers fought swordsmen.

So Hawkeye was like considered a villain for the Avengers for a while.