It makes much more sense when you view Jedi/Sith as the trope of the warrior monk who has achieved enlightenment. They have gained mystical powers beyond mortal ken etc, but mostly they used it to hit people with laser swords and public knowledge could dismiss a lot of that as physical training and conditioning.