Have considered making the prototype with Flash? There was a flash RPG that was advertised here for a bit
dragonfable I think. I didn't play it but I've seen my kids tool around with similar flash-type RPG-ish games. Looks like it could work for a 2D RPG prototyping tool. Haven't tried it myself though.
Once you have your game system and concepts ironed out you could move to a 3D engine afterwards.