CastorShinjiro's Persona is a reference to mythology like most others in the series. In Greek and Roman mythology, Castor and Polydeuces are twin brothers. They are both considered to be patrons of horses, but Castor was depicted as a talented horse trainer while Polydeuces was a skilled boxer. The two brothers were always together. There are various stories about them, however most agree on one ending. As a mortal, Castor dies when he is ambushed and impaled by a spear. Polydeuces, who is immortal, does not die and is completely devastated when his brother passes away. He misses him so much that he offers to either share his immortality with Castor or join him in Hades. Zeus decides to put the twins in the heavens instead and they became part of the constellation Gemini. Even today, the two brightest stars in the constellation are named after them!
As you may have guessed by drawing similarities to Persona 3 (or already known), Polydeuces is Akihiko's original Persona. Just like the two horse patrons, Shinjiro and Akihiko are practically brothers who were always together in the past. Akihiko is a boxer just like Polydeuces. Shinjiro is not a horse trainer, but his Persona is depicted riding on one. Castor being mortal and thus causing Polydeuces despair parallels Shinjiro's ultimate fate and grief experienced by Akihiko. Their designs also are symbolic of the myth. Polydeuces is white in color (immortal), whereas Castor looks like a darker version of him with a spear through his chest (death). The spear through Castor could also be a representation of the pain and self-hatred Shinjiro feels.
In the Persona 3 Official Design Works book, the designer says Castor's model was to be Polydeuces' older brother. His riding horse and military affairs are that of an excellent brave warrior of Sparta. He also notes that Polydeuces lost Castor right in front of his eyes when he was impaled and killed in action.
Castor & Pollux @ Myths Encyclopedia