It's just you.


Seriously, you are right. Just more riding and practice will suffice. I have a partially paralyzed right leg, and although I added an ISCI hand break for safety, it was not because the brake pedal was hard to find. I had no problem finding the pedal, my problem was in trusting my bum leg to react quickly enough in an emergency. But I don't have a problem finding the pedal.
However, there are vendors (see Reggy at TricLed) who have a larger aftermarket pedal that might solve your issue. Customers like Bob Denman (and many others i am sure) have pics of the modification I believe.
Edit: AllAround, there is another thread on this same subject right now with pics. See post:
http://www.spyderlovers.com/forums/showthread.php?t=39932