Greynull Nepris:
First find out the fort's layout and personnel schedule.
Second, figure a way in.
Third, free the prisoner with least disturbance possible. Might even be possible to have a dummy look like the prisoner.
Fourth, get out of town, perhaps by boat to a near shore.
Second first, first second. Byron has offered to cast invisibility to help so go for it.
If the prisoner is found why leave them there especially since the he's accessible and can be freed at that instance.