Jimmy Carter’s statement is almost correct. Because he is talking about tax dollars, which are inherently government action, his statement should have been: “If you don’t want your tax dollars to help the poor, then stop saying you want a government based on Christian values, because you don’t”.
Considering state-church separation, and that congress isn’t supposed to make any law respecting the establishment of religion, this is a very supportable notion. But that would require Carter to admit that the government is not the country.
Edit: Turns out this is also factually wrong, because it is misattributed. I owe president Carter an apology for claiming he didn’t understand the different between “country” and “government”.