Physics is not really some esoteric study presided over by guardians who make you take exams for no apparent reason other than cruelty, although it may seem like it at times. Physics is the human study of your world. So don't think of physics as something just in books and the heads of professors, locking everybody else out.
Physics is just the result of a questioning mind facing nature. And that's something everyone can share.