I remember when I first upjetted I was like "what the hell where'd my power go??" but hey man... go by the plug and logic straight up unless you want to soft / hard seize it one of these days... A leaner jet gives you MORE power, but is also likely not giving you enough lubrication / cooling... keep in mind these are air cooled engines for the most part so the air / fuel mixture is absolutely key. I would keep the bigger jet in if you want to be safe.
Now, as far as "fixing it" goes, you could get more air via pipe / filter upgrade which will mean that you can then upjet and have more gas AND more air so you'll still have a good ratio just MORE of both thus more power (unless you get too much of both). You could also change your gearing to gain back a little extra power either low end or top end whichever you are wanting to get back.
Good luck guy