I think this is the only one of the 3 prime-time Eureeka's Castle specials that never got an official VHS release.
Here's what I remember about it: Eureeka gets some kind of magic box, and tells Magellan, Batly, Bogge, and Quagmire not to touch it, but they do anyway. As a result, Magellan gets shrunk, Batly gets turned invisible, and Bogge and Quagmire get stuck together inside of a giant peanut butter sandwich. Eventually Mr. Knack examines the box and sees that it's malfunctioning because the batteries were put in backwards, so he fixes it abd everyone gets changed back to normal.
I'm tempted to say that someone out there must have it because there are bumpers from its original airing on Youtube, but the last time I said something like that all it was make the Muppet Wiki admin who posted the Blue Bird screencap act suspicious when someone asked him about it on his talk page, but whatever, I don't want to start any trouble with this guy