you don't "pass" relic castle you go to it later and meet team plasma (you also get a fossil if you talk to the backpacker!)
Chat with our AI personalities
At one point in the plot, you will be told to go there. Only then will he move and allow you access.
if you go down to the very bottom of the relic castle go up the all the stairs and you should be out. hope this helps! =)
In relic castle
Walk down as far as the backpacker and then walk left one, and then down towards the stairs.
you have to get to the bottom