I'm trying to accomplish something similar - Firefox and FireGPG installed on my PC, but running GPG from the Portable Apps folder. (This is a work PC so I don't have rights to install GPG.)
I have manually specified the path to gpg.exe in the FireGPG options, but it seems to fail at reading my private key. Using Portable Thunderbird + Enigmail works fine with my keypair, so I know my keys are OK.
Definitely looking forward to FireGPG working with Portable Apps. Keep up the good work!