Welcome to the Intel® Embedded Community
I think you do need a power switch. Please see this other thread discussing what looks like identical situation to yours:
And if you don't already have it, the Users Guide at link below describes the use of the power switch, which "provides slightly different behavior
from a traditional power switch." (see page 32)
I hope this helps.
J. Felix McNulty
Community Moderator (Intel contractor)