Truthfully this was Ellie's backstory episode. We know most of Joel's story - was a soldier (perfect for surviving post-apocalyptic hellscapes), lost his daughter to a frightened soldier, and has worked as a plug with his hardened (now dead) partner and he has PTSD plus doesn't want to feel anymore.
We have been told some of Ellie's history: got bitten in the mall where she was not supposed to be (the Fireflies probably came to check on Riley and found Ellie), doesn't have any kin, and grew up in the FEDRA schools.
She has lost everyone she cared for, and here is a father figure dying right in front of her, gives her an "out" (Go back to my brother with the alive horse) and we think that she just might.... flashback - "let's die and spend all the time we have left together.... we are still alive until we aren't".... so that we understand why Ellie will not leave and fight with/for Joel.
Plus it was a sweet episode. The kind of episode that Station Eleven would have made (and maybe mid-run Walking Dead).