it would make sense to bring him back out, i wouldn't do it though. Let him gain some confidence.
One of the reasons most starters wind up in the pen is lack of control, so his stuff could play up in shorter stints. If he can't locate the fastball, yank him.