My parents grew up in Yiddish-speaking households. Their English is unaccented, because they went to public schools. They spoke Yiddish between themselves when they didn't want "the kids" to understand, but of course, we did.
Later in life I came to understand that speaking Yiddish was a particular flavor of reactionary toward American culture, and I no longer care for hearing it spoken at all. However, songs in Yiddish still have a nice tam.