I can plug in anytime to the controller and it works.

Try pluging in before you start the game and wait for the found new hardware sound.